2023-07-18T03:08:16.232 INFO:root:teuthology version: 0.0.1.dev147+gff9991b 2023-07-18T03:08:16.233 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --name rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746 --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/blogbench}} --verbose -- /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746/orig.config.yaml 2023-07-18T03:08:16.290 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T03:08:16.389 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746 branch: wip-rishabh-2023Jul13-m-quota description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/blogbench}} email: null first_in_suite: false job_id: '7342746' kernel: client: sha1: distro flavor: default kdb: true sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Jul13-m-quota ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul13-m-quota sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul13-m-quota suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi064.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi134.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b timestamp: 2023-07-18_02:11:50 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-18T03:08:16.390 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa; will attempt to use it 2023-07-18T03:08:16.391 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks 2023-07-18T03:08:16.391 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-18T03:08:16.393 INFO:teuthology.task.internal:Checking packages... 2023-07-18T03:08:16.421 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '81cf785aa7781c4e08381c540f890cb9bc987ef9' 2023-07-18T03:08:16.421 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-18T03:08:16.422 INFO:teuthology.packaging:ref: None 2023-07-18T03:08:16.422 INFO:teuthology.packaging:tag: None 2023-07-18T03:08:16.422 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul13-m-quota 2023-07-18T03:08:16.423 INFO:teuthology.packaging:sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:08:16.423 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul13-m-quota 2023-07-18T03:08:16.658 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4746.g81cf785a 2023-07-18T03:08:16.658 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-18T03:08:16.664 INFO:teuthology.task.internal:no buildpackages task found 2023-07-18T03:08:16.665 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-18T03:08:16.678 INFO:teuthology.task.internal:Saving configuration 2023-07-18T03:08:16.703 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-18T03:08:16.709 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-18T03:08:16.738 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 03:03:07.611963', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-18T03:08:16.763 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 03:03:07.613345', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-18T03:08:16.788 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 03:03:07.610380', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-18T03:08:16.789 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-18T03:08:16.795 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-07-18T03:08:16.795 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-07-18T03:08:16.795 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-07-18T03:08:16.795 INFO:teuthology.run_tasks:Running task console_log... 2023-07-18T03:08:16.905 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7bacdc9550>, signals=[15]) 2023-07-18T03:08:16.907 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-18T03:08:16.917 INFO:teuthology.task.internal:Opening connections... 2023-07-18T03:08:16.918 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2023-07-18T03:08:16.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T03:08:16.988 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2023-07-18T03:08:16.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T03:08:17.058 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2023-07-18T03:08:17.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T03:08:17.126 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-18T03:08:17.132 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2023-07-18T03:08:17.154 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2023-07-18T03:08:17.154 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2023-07-18T03:08:17.214 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="Red Hat Enterprise Linux" 2023-07-18T03:08:17.214 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="8.6 (Ootpa)" 2023-07-18T03:08:17.214 INFO:teuthology.orchestra.run.smithi040.stdout:ID="rhel" 2023-07-18T03:08:17.214 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="fedora" 2023-07-18T03:08:17.214 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="8.6" 2023-07-18T03:08:17.214 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el8" 2023-07-18T03:08:17.215 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-18T03:08:17.215 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2023-07-18T03:08:17.215 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-18T03:08:17.215 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://www.redhat.com/" 2023-07-18T03:08:17.215 INFO:teuthology.orchestra.run.smithi040.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-18T03:08:17.215 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-18T03:08:17.215 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:08:17.216 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-18T03:08:17.216 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-18T03:08:17.216 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-18T03:08:17.216 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-18T03:08:17.217 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2023-07-18T03:08:17.250 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2023-07-18T03:08:17.271 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2023-07-18T03:08:17.271 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2023-07-18T03:08:17.331 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="Red Hat Enterprise Linux" 2023-07-18T03:08:17.331 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="8.6 (Ootpa)" 2023-07-18T03:08:17.332 INFO:teuthology.orchestra.run.smithi064.stdout:ID="rhel" 2023-07-18T03:08:17.332 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="fedora" 2023-07-18T03:08:17.332 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="8.6" 2023-07-18T03:08:17.332 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el8" 2023-07-18T03:08:17.332 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-18T03:08:17.332 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2023-07-18T03:08:17.332 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-18T03:08:17.333 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://www.redhat.com/" 2023-07-18T03:08:17.333 INFO:teuthology.orchestra.run.smithi064.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-18T03:08:17.333 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-18T03:08:17.333 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-07-18T03:08:17.333 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-18T03:08:17.333 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-18T03:08:17.333 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-18T03:08:17.334 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-18T03:08:17.334 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2023-07-18T03:08:17.366 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2023-07-18T03:08:17.386 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2023-07-18T03:08:17.386 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2023-07-18T03:08:17.445 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="Red Hat Enterprise Linux" 2023-07-18T03:08:17.446 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="8.6 (Ootpa)" 2023-07-18T03:08:17.446 INFO:teuthology.orchestra.run.smithi134.stdout:ID="rhel" 2023-07-18T03:08:17.446 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="fedora" 2023-07-18T03:08:17.446 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="8.6" 2023-07-18T03:08:17.446 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el8" 2023-07-18T03:08:17.447 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-18T03:08:17.447 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2023-07-18T03:08:17.447 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-18T03:08:17.447 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://www.redhat.com/" 2023-07-18T03:08:17.447 INFO:teuthology.orchestra.run.smithi134.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-18T03:08:17.447 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-18T03:08:17.447 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-18T03:08:17.447 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-18T03:08:17.448 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-18T03:08:17.448 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-18T03:08:17.448 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-18T03:08:17.448 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2023-07-18T03:08:17.475 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-18T03:08:17.485 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-18T03:08:17.492 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-18T03:08:17.493 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2023-07-18T03:08:17.495 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2023-07-18T03:08:17.497 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2023-07-18T03:08:17.515 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-18T03:08:17.527 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-18T03:08:17.527 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2023-07-18T03:08:17.555 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2023-07-18T03:08:17.557 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2023-07-18T03:08:17.588 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-18T03:08:17.672 INFO:teuthology.run_tasks:Running task kernel... 2023-07-18T03:08:17.686 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'} 2023-07-18T03:08:17.687 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-07-18T03:08:17.687 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-07-18T03:08:17.688 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-18T03:08:17.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:08:17.708 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2023-07-18T03:08:17.767 INFO:teuthology.orchestra.run.smithi040.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-18T03:08:17.768 INFO:teuthology.task.kernel:Running kernel on smithi040: 4.18.0-372.9.1.el8.x86_64 2023-07-18T03:08:17.768 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum install -y kernel 2023-07-18T03:08:19.023 INFO:teuthology.orchestra.run.smithi040.stdout:Updating Subscription Management repositories. 2023-07-18T03:08:19.024 INFO:teuthology.orchestra.run.smithi040.stdout:Unable to read consumer identity 2023-07-18T03:08:20.696 INFO:teuthology.orchestra.run.smithi040.stdout:Copr repo for python3-asyncssh owned by ceph 9.5 kB/s | 3.5 kB 00:00 2023-07-18T03:08:29.446 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 1.8 MB/s | 16 MB 00:08 2023-07-18T03:08:33.071 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 408 kB/s | 24 kB 00:00 2023-07-18T03:08:33.790 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-07-18T03:08:33.837 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-18T03:08:33.839 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2023-07-18T03:08:33.839 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-18T03:08:33.966 DEBUG:teuthology.orchestra.run.smithi040:> echo no | sudo yum reinstall kernel || true 2023-07-18T03:08:34.316 INFO:teuthology.orchestra.run.smithi040.stdout:Updating Subscription Management repositories. 2023-07-18T03:08:34.316 INFO:teuthology.orchestra.run.smithi040.stdout:Unable to read consumer identity 2023-07-18T03:08:34.585 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:01 ago on Tue 18 Jul 2023 03:08:33 AM UTC. 2023-07-18T03:08:34.635 INFO:teuthology.orchestra.run.smithi040.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-18T03:08:34.668 INFO:teuthology.orchestra.run.smithi040.stderr:Error: No packages marked for reinstall. 2023-07-18T03:08:34.716 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum reinstall -y kernel || true 2023-07-18T03:08:35.064 INFO:teuthology.orchestra.run.smithi040.stdout:Updating Subscription Management repositories. 2023-07-18T03:08:35.065 INFO:teuthology.orchestra.run.smithi040.stdout:Unable to read consumer identity 2023-07-18T03:08:35.327 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:02 ago on Tue 18 Jul 2023 03:08:33 AM UTC. 2023-07-18T03:08:35.368 INFO:teuthology.orchestra.run.smithi040.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-18T03:08:35.395 INFO:teuthology.orchestra.run.smithi040.stderr:Error: No packages marked for reinstall. 2023-07-18T03:08:35.441 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q kernel | sort -rV | head -n 1 2023-07-18T03:08:35.523 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-07-18T03:08:35.524 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-07-18T03:08:35.524 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-07-18T03:08:35.525 DEBUG:teuthology.parallel:result is None 2023-07-18T03:08:35.525 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-18T03:08:35.532 INFO:teuthology.task.internal:Creating test directory... 2023-07-18T03:08:35.533 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-18T03:08:35.535 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-18T03:08:35.537 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-18T03:08:35.557 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-18T03:08:35.564 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-18T03:08:35.576 INFO:teuthology.task.internal:Creating archive directory... 2023-07-18T03:08:35.577 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-18T03:08:35.592 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-18T03:08:35.599 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-18T03:08:35.633 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-18T03:08:35.641 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-18T03:08:35.642 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-18T03:08:35.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:08:35.685 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-18T03:08:35.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:08:35.704 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-18T03:08:35.706 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-18T03:08:35.727 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-18T03:08:35.740 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-18T03:08:35.758 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-18T03:08:35.760 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-18T03:08:35.774 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-18T03:08:35.777 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-18T03:08:35.799 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-18T03:08:35.801 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-18T03:08:35.808 INFO:teuthology.task.internal:Configuring sudo... 2023-07-18T03:08:35.809 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-18T03:08:35.811 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-18T03:08:35.817 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-18T03:08:35.870 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-18T03:08:35.885 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-18T03:08:35.885 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-18T03:08:35.888 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-18T03:08:35.890 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-18T03:08:35.927 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-18T03:08:35.988 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-18T03:08:36.044 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-18T03:08:36.132 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-18T03:08:36.170 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:08:36.171 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-18T03:08:36.237 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-18T03:08:36.289 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-18T03:08:36.342 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-18T03:08:36.431 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-18T03:08:36.473 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-07-18T03:08:36.474 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-18T03:08:36.543 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-18T03:08:36.599 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-18T03:08:36.649 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-18T03:08:36.738 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-18T03:08:36.778 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-18T03:08:36.778 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-18T03:08:36.847 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2023-07-18T03:08:36.850 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2023-07-18T03:08:36.854 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2023-07-18T03:08:36.912 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-18T03:08:36.915 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-18T03:08:36.948 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-18T03:08:37.354 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-18T03:08:37.363 INFO:teuthology.task.internal:Starting timer... 2023-07-18T03:08:37.363 INFO:teuthology.run_tasks:Running task pcp... 2023-07-18T03:08:37.373 INFO:teuthology.run_tasks:Running task selinux... 2023-07-18T03:08:37.382 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-07-18T03:08:37.383 DEBUG:teuthology.orchestra.run.smithi040:> sudo service auditd rotate 2023-07-18T03:08:37.445 INFO:teuthology.orchestra.run.smithi040.stdout:Rotating logs: 2023-07-18T03:08:37.447 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2023-07-18T03:08:37.503 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2023-07-18T03:08:37.504 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2023-07-18T03:08:37.567 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2023-07-18T03:08:37.568 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-18T03:08:37.569 DEBUG:teuthology.orchestra.run.smithi040:> /usr/sbin/getenforce 2023-07-18T03:08:37.592 INFO:teuthology.orchestra.run.smithi040.stdout:Permissive 2023-07-18T03:08:37.592 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2023-07-18T03:08:37.611 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2023-07-18T03:08:37.612 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2023-07-18T03:08:37.634 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2023-07-18T03:08:37.635 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi040.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi064.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive'} 2023-07-18T03:08:37.635 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-18T03:08:37.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:08:37.663 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-18T03:08:37.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:08:37.691 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-18T03:08:37.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:08:37.718 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-18T03:08:37.719 DEBUG:teuthology.orchestra.run.smithi040:> sudo /usr/sbin/setenforce permissive 2023-07-18T03:08:37.748 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2023-07-18T03:08:37.777 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2023-07-18T03:08:37.804 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-18T03:08:37.813 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-07-18T03:08:37.820 INFO:teuthology.repo_utils:Fetching main from origin 2023-07-18T03:08:37.966 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-18T03:08:37.985 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-18T03:08:37.986 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi040.front.sepia.ceph.com,smithi064.front.sepia.ceph.com,smithi134.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-18T03:18:36.560 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com')] 2023-07-18T03:18:36.562 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-07-18T03:18:36.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T03:18:36.636 DEBUG:teuthology.orchestra.run.smithi040:> true 2023-07-18T03:18:36.702 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-07-18T03:18:36.703 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-07-18T03:18:36.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T03:18:36.774 DEBUG:teuthology.orchestra.run.smithi064:> true 2023-07-18T03:18:36.841 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2023-07-18T03:18:36.842 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2023-07-18T03:18:36.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T03:18:36.911 DEBUG:teuthology.orchestra.run.smithi134:> true 2023-07-18T03:18:36.976 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2023-07-18T03:18:36.977 INFO:teuthology.run_tasks:Running task clock... 2023-07-18T03:18:36.995 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-18T03:18:36.995 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-18T03:18:36.996 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-18T03:18:36.998 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-18T03:18:36.999 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-18T03:18:37.001 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-18T03:18:37.001 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-18T03:18:37.033 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-18T03:18:37.033 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-18T03:18:37.047 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-18T03:18:37.049 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-18T03:18:37.049 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-18T03:18:37.061 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-18T03:18:37.116 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2023-07-18T03:18:37.126 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: ntpd: command not found 2023-07-18T03:18:37.127 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2023-07-18T03:18:37.133 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2023-07-18T03:18:37.139 INFO:teuthology.orchestra.run.smithi040.stdout:506 Cannot talk to daemon 2023-07-18T03:18:37.143 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-18T03:18:37.145 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2023-07-18T03:18:37.155 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-18T03:18:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-18T03:18:37.159 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-18T03:18:37.170 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-18T03:18:37.173 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-18T03:18:37.229 INFO:teuthology.orchestra.run.smithi064.stderr:bash: ntpq: command not found 2023-07-18T03:18:37.233 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-18T03:18:37.233 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2023-07-18T03:18:37.234 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:18:37.234 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:18:37.234 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:18:37.234 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:18:37.238 INFO:teuthology.orchestra.run.smithi040.stderr:bash: ntpq: command not found 2023-07-18T03:18:37.241 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-18T03:18:37.241 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2023-07-18T03:18:37.241 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:18:37.241 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:18:37.242 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:18:37.242 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:18:37.243 INFO:teuthology.orchestra.run.smithi134.stderr:bash: ntpq: command not found 2023-07-18T03:18:37.246 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-18T03:18:37.246 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2023-07-18T03:18:37.246 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:18:37.246 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:18:37.246 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:18:37.247 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:18:37.248 INFO:teuthology.run_tasks:Running task pexec... 2023-07-18T03:18:37.258 INFO:teuthology.task.pexec:Executing custom commands... 2023-07-18T03:18:37.259 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi040.front.sepia.ceph.com 2023-07-18T03:18:37.259 DEBUG:teuthology.orchestra.run.smithi040:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-18T03:18:37.260 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi064.front.sepia.ceph.com 2023-07-18T03:18:37.260 DEBUG:teuthology.orchestra.run.smithi064:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-18T03:18:37.260 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi134.front.sepia.ceph.com 2023-07-18T03:18:37.261 DEBUG:teuthology.orchestra.run.smithi134:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-18T03:18:37.657 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2023-07-18T03:18:37.663 INFO:teuthology.orchestra.run.smithi064.stdout:Updating Subscription Management repositories. 2023-07-18T03:18:37.673 INFO:teuthology.orchestra.run.smithi040.stdout:Updating Subscription Management repositories. 2023-07-18T03:18:38.445 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:52 ago on Tue 18 Jul 2023 03:16:46 AM UTC. 2023-07-18T03:18:38.489 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:43 ago on Tue 18 Jul 2023 03:16:55 AM UTC. 2023-07-18T03:18:38.503 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:28 ago on Tue 18 Jul 2023 03:17:10 AM UTC. 2023-07-18T03:18:40.169 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2023-07-18T03:18:40.170 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2023-07-18T03:18:40.170 INFO:teuthology.orchestra.run.smithi134.stdout: Package Architecture Version Repository Size 2023-07-18T03:18:40.170 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2023-07-18T03:18:40.170 INFO:teuthology.orchestra.run.smithi134.stdout:Resetting modules: 2023-07-18T03:18:40.170 INFO:teuthology.orchestra.run.smithi134.stdout: container-tools 2023-07-18T03:18:40.170 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-18T03:18:40.171 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2023-07-18T03:18:40.171 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2023-07-18T03:18:40.171 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-18T03:18:40.440 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2023-07-18T03:18:40.441 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2023-07-18T03:18:40.441 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2023-07-18T03:18:40.441 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2023-07-18T03:18:40.442 INFO:teuthology.orchestra.run.smithi064.stdout:Resetting modules: 2023-07-18T03:18:40.442 INFO:teuthology.orchestra.run.smithi064.stdout: container-tools 2023-07-18T03:18:40.442 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-07-18T03:18:40.442 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2023-07-18T03:18:40.442 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2023-07-18T03:18:40.442 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-07-18T03:18:40.508 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-18T03:18:40.509 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-18T03:18:40.510 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2023-07-18T03:18:40.510 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-18T03:18:40.510 INFO:teuthology.orchestra.run.smithi040.stdout:Resetting modules: 2023-07-18T03:18:40.510 INFO:teuthology.orchestra.run.smithi040.stdout: container-tools 2023-07-18T03:18:40.511 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:18:40.511 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-18T03:18:40.511 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-07-18T03:18:40.512 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:18:41.111 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:54 ago on Tue 18 Jul 2023 03:16:46 AM UTC. 2023-07-18T03:18:41.447 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:46 ago on Tue 18 Jul 2023 03:16:55 AM UTC. 2023-07-18T03:18:41.489 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:31 ago on Tue 18 Jul 2023 03:17:10 AM UTC. 2023-07-18T03:18:43.406 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2023-07-18T03:18:43.889 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2023-07-18T03:18:44.268 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2023-07-18T03:18:44.301 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-18T03:18:44.691 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:01:58 ago on Tue 18 Jul 2023 03:16:46 AM UTC. 2023-07-18T03:18:44.759 INFO:teuthology.orchestra.run.smithi064.stdout:Updating Subscription Management repositories. 2023-07-18T03:18:44.774 INFO:teuthology.orchestra.run.smithi040.stdout:Updating Subscription Management repositories. 2023-07-18T03:18:45.606 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:35 ago on Tue 18 Jul 2023 03:17:10 AM UTC. 2023-07-18T03:18:45.609 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:50 ago on Tue 18 Jul 2023 03:16:55 AM UTC. 2023-07-18T03:18:46.428 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2023-07-18T03:18:46.521 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================================ 2023-07-18T03:18:46.522 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2023-07-18T03:18:46.522 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================================ 2023-07-18T03:18:46.522 INFO:teuthology.orchestra.run.smithi134.stdout:Installing group/module packages: 2023-07-18T03:18:46.522 INFO:teuthology.orchestra.run.smithi134.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-18T03:18:46.522 INFO:teuthology.orchestra.run.smithi134.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-18T03:18:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-18T03:18:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-18T03:18:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-18T03:18:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-18T03:18:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-18T03:18:46.523 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2023-07-18T03:18:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-18T03:18:46.523 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-18T03:18:46.524 INFO:teuthology.orchestra.run.smithi134.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-18T03:18:46.524 INFO:teuthology.orchestra.run.smithi134.stdout:Installing module profiles: 2023-07-18T03:18:46.524 INFO:teuthology.orchestra.run.smithi134.stdout: container-tools/common 2023-07-18T03:18:46.524 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-18T03:18:46.524 INFO:teuthology.orchestra.run.smithi134.stdout:Enabling module streams: 2023-07-18T03:18:46.524 INFO:teuthology.orchestra.run.smithi134.stdout: container-tools rhel8 2023-07-18T03:18:46.524 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-18T03:18:46.525 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2023-07-18T03:18:46.525 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================================================ 2023-07-18T03:18:46.525 INFO:teuthology.orchestra.run.smithi134.stdout:Install 10 Packages 2023-07-18T03:18:46.525 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-18T03:18:46.525 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 18 M 2023-07-18T03:18:46.526 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 65 M 2023-07-18T03:18:46.526 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2023-07-18T03:18:46.603 INFO:teuthology.orchestra.run.smithi134.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.2 MB/s | 94 kB 00:00 2023-07-18T03:18:46.620 INFO:teuthology.orchestra.run.smithi134.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 259 kB/s | 25 kB 00:00 2023-07-18T03:18:46.637 INFO:teuthology.orchestra.run.smithi134.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 361 kB/s | 41 kB 00:00 2023-07-18T03:18:46.662 INFO:teuthology.orchestra.run.smithi134.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 838 kB/s | 49 kB 00:00 2023-07-18T03:18:46.696 INFO:teuthology.orchestra.run.smithi134.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.8 MB/s | 524 kB 00:00 2023-07-18T03:18:46.721 INFO:teuthology.orchestra.run.smithi134.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-18T03:18:46.746 INFO:teuthology.orchestra.run.smithi134.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.2 MB/s | 213 kB 00:00 2023-07-18T03:18:46.830 INFO:teuthology.orchestra.run.smithi134.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 26 MB/s | 2.2 MB 00:00 2023-07-18T03:18:46.938 INFO:teuthology.orchestra.run.smithi134.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 32 MB/s | 7.0 MB 00:00 2023-07-18T03:18:47.030 INFO:teuthology.orchestra.run.smithi134.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 20 MB/s | 8.0 MB 00:00 2023-07-18T03:18:47.031 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2023-07-18T03:18:47.031 INFO:teuthology.orchestra.run.smithi134.stdout:Total 36 MB/s | 18 MB 00:00 2023-07-18T03:18:47.231 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2023-07-18T03:18:47.249 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2023-07-18T03:18:47.249 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2023-07-18T03:18:47.431 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2023-07-18T03:18:47.435 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2023-07-18T03:18:47.510 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-18T03:18:47.553 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2023-07-18T03:18:47.612 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================================================ 2023-07-18T03:18:47.612 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2023-07-18T03:18:47.613 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================================================ 2023-07-18T03:18:47.613 INFO:teuthology.orchestra.run.smithi040.stdout:Installing group/module packages: 2023-07-18T03:18:47.613 INFO:teuthology.orchestra.run.smithi040.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-18T03:18:47.613 INFO:teuthology.orchestra.run.smithi040.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-18T03:18:47.614 INFO:teuthology.orchestra.run.smithi040.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-18T03:18:47.614 INFO:teuthology.orchestra.run.smithi040.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-18T03:18:47.614 INFO:teuthology.orchestra.run.smithi040.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-18T03:18:47.614 INFO:teuthology.orchestra.run.smithi040.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-18T03:18:47.615 INFO:teuthology.orchestra.run.smithi040.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-18T03:18:47.615 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2023-07-18T03:18:47.615 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-18T03:18:47.615 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-18T03:18:47.616 INFO:teuthology.orchestra.run.smithi040.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-18T03:18:47.616 INFO:teuthology.orchestra.run.smithi040.stdout:Installing module profiles: 2023-07-18T03:18:47.616 INFO:teuthology.orchestra.run.smithi040.stdout: container-tools/common 2023-07-18T03:18:47.616 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:18:47.617 INFO:teuthology.orchestra.run.smithi040.stdout:Enabling module streams: 2023-07-18T03:18:47.617 INFO:teuthology.orchestra.run.smithi040.stdout: container-tools rhel8 2023-07-18T03:18:47.617 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:18:47.617 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-18T03:18:47.618 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================================================ 2023-07-18T03:18:47.618 INFO:teuthology.orchestra.run.smithi040.stdout:Install 10 Packages 2023-07-18T03:18:47.619 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:18:47.620 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 18 M 2023-07-18T03:18:47.620 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 65 M 2023-07-18T03:18:47.621 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-18T03:18:47.654 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================================ 2023-07-18T03:18:47.654 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2023-07-18T03:18:47.655 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================================ 2023-07-18T03:18:47.655 INFO:teuthology.orchestra.run.smithi064.stdout:Installing group/module packages: 2023-07-18T03:18:47.655 INFO:teuthology.orchestra.run.smithi064.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-18T03:18:47.655 INFO:teuthology.orchestra.run.smithi064.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-18T03:18:47.655 INFO:teuthology.orchestra.run.smithi064.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-18T03:18:47.656 INFO:teuthology.orchestra.run.smithi064.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-18T03:18:47.656 INFO:teuthology.orchestra.run.smithi064.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-18T03:18:47.656 INFO:teuthology.orchestra.run.smithi064.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-18T03:18:47.656 INFO:teuthology.orchestra.run.smithi064.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-18T03:18:47.657 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2023-07-18T03:18:47.657 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-18T03:18:47.657 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-18T03:18:47.657 INFO:teuthology.orchestra.run.smithi064.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-18T03:18:47.658 INFO:teuthology.orchestra.run.smithi064.stdout:Installing module profiles: 2023-07-18T03:18:47.658 INFO:teuthology.orchestra.run.smithi064.stdout: container-tools/common 2023-07-18T03:18:47.658 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-07-18T03:18:47.658 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling module streams: 2023-07-18T03:18:47.658 INFO:teuthology.orchestra.run.smithi064.stdout: container-tools rhel8 2023-07-18T03:18:47.659 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-07-18T03:18:47.659 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2023-07-18T03:18:47.659 INFO:teuthology.orchestra.run.smithi064.stdout:============================================================================================================ 2023-07-18T03:18:47.659 INFO:teuthology.orchestra.run.smithi064.stdout:Install 10 Packages 2023-07-18T03:18:47.660 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-07-18T03:18:47.661 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 18 M 2023-07-18T03:18:47.661 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 65 M 2023-07-18T03:18:47.661 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2023-07-18T03:18:47.712 INFO:teuthology.orchestra.run.smithi040.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 421 kB/s | 41 kB 00:00 2023-07-18T03:18:47.724 INFO:teuthology.orchestra.run.smithi064.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.4 MB/s | 94 kB 00:00 2023-07-18T03:18:47.736 INFO:teuthology.orchestra.run.smithi040.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 204 kB/s | 25 kB 00:00 2023-07-18T03:18:47.749 INFO:teuthology.orchestra.run.smithi064.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 268 kB/s | 25 kB 00:00 2023-07-18T03:18:47.762 INFO:teuthology.orchestra.run.smithi040.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 986 kB/s | 49 kB 00:00 2023-07-18T03:18:47.775 INFO:teuthology.orchestra.run.smithi064.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 345 kB/s | 41 kB 00:00 2023-07-18T03:18:47.787 INFO:teuthology.orchestra.run.smithi040.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 544 kB/s | 94 kB 00:00 2023-07-18T03:18:47.800 INFO:teuthology.orchestra.run.smithi064.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-07-18T03:18:47.821 INFO:teuthology.orchestra.run.smithi040.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-07-18T03:18:47.834 INFO:teuthology.orchestra.run.smithi064.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-07-18T03:18:47.846 INFO:teuthology.orchestra.run.smithi040.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-18T03:18:47.859 INFO:teuthology.orchestra.run.smithi064.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-18T03:18:47.871 INFO:teuthology.orchestra.run.smithi040.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-07-18T03:18:47.884 INFO:teuthology.orchestra.run.smithi064.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-07-18T03:18:47.947 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2023-07-18T03:18:47.993 INFO:teuthology.orchestra.run.smithi064.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 20 MB/s | 2.2 MB 00:00 2023-07-18T03:18:48.023 INFO:teuthology.orchestra.run.smithi040.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 14 MB/s | 2.2 MB 00:00 2023-07-18T03:18:48.102 INFO:teuthology.orchestra.run.smithi064.stdout:(9/10): buildah-1.26.2-1.module+el8.6.0+16771+2 24 MB/s | 8.0 MB 00:00 2023-07-18T03:18:48.130 INFO:teuthology.orchestra.run.smithi040.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 25 MB/s | 7.0 MB 00:00 2023-07-18T03:18:48.135 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-18T03:18:48.185 INFO:teuthology.orchestra.run.smithi064.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 22 MB/s | 7.0 MB 00:00 2023-07-18T03:18:48.186 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2023-07-18T03:18:48.186 INFO:teuthology.orchestra.run.smithi064.stdout:Total 34 MB/s | 18 MB 00:00 2023-07-18T03:18:48.222 INFO:teuthology.orchestra.run.smithi040.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 17 MB/s | 8.0 MB 00:00 2023-07-18T03:18:48.224 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-18T03:18:48.224 INFO:teuthology.orchestra.run.smithi040.stdout:Total 30 MB/s | 18 MB 00:00 2023-07-18T03:18:48.301 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-18T03:18:48.393 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2023-07-18T03:18:48.413 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2023-07-18T03:18:48.414 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2023-07-18T03:18:48.430 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-18T03:18:48.451 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-18T03:18:48.451 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-18T03:18:48.520 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-18T03:18:48.607 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2023-07-18T03:18:48.612 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2023-07-18T03:18:48.645 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-18T03:18:48.650 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-18T03:18:48.740 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-18T03:18:49.096 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-18T03:18:49.133 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2023-07-18T03:18:49.185 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-18T03:18:49.283 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-18T03:18:49.365 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-18T03:18:49.443 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-18T03:18:49.564 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-18T03:18:49.762 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-18T03:18:49.764 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-18T03:18:49.856 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-18T03:18:49.976 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-18T03:18:49.988 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-18T03:18:50.358 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-18T03:18:50.380 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-18T03:18:50.742 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-18T03:18:50.973 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-18T03:18:51.156 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-18T03:18:51.166 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-18T03:18:51.203 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-18T03:18:51.360 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-18T03:18:51.622 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-18T03:18:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-18T03:18:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-18T03:18:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-18T03:18:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-18T03:18:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-18T03:18:51.623 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-18T03:18:51.624 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-18T03:18:51.624 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-18T03:18:51.624 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-18T03:18:52.078 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-18T03:18:52.124 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-18T03:18:52.363 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-18T03:18:52.400 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-18T03:18:52.512 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-18T03:18:52.575 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-18T03:18:52.671 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-18T03:18:52.742 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-18T03:18:52.816 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-18T03:18:52.816 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:02:06 ago on Tue 18 Jul 2023 03:16:46 AM UTC. 2023-07-18T03:18:52.952 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-18T03:18:52.952 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-18T03:18:52.952 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-18T03:18:52.952 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-18T03:18:52.953 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-18T03:18:52.953 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-18T03:18:52.953 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-18T03:18:52.953 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-18T03:18:52.953 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-18T03:18:52.953 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-18T03:18:53.004 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-18T03:18:53.005 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-18T03:18:53.005 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-18T03:18:53.005 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-18T03:18:53.005 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-18T03:18:53.005 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-18T03:18:53.005 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-18T03:18:53.005 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-18T03:18:53.006 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-18T03:18:53.006 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-18T03:18:54.109 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-18T03:18:54.110 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:43 ago on Tue 18 Jul 2023 03:17:10 AM UTC. 2023-07-18T03:18:54.125 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-18T03:18:54.125 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:01:58 ago on Tue 18 Jul 2023 03:16:55 AM UTC. 2023-07-18T03:18:55.429 INFO:teuthology.orchestra.run.smithi134.stdout:Installed products updated. 2023-07-18T03:18:55.623 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-18T03:18:55.624 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2023-07-18T03:18:55.624 INFO:teuthology.orchestra.run.smithi134.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T03:18:55.624 INFO:teuthology.orchestra.run.smithi134.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-18T03:18:55.624 INFO:teuthology.orchestra.run.smithi134.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T03:18:55.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-18T03:18:55.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-18T03:18:55.624 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-18T03:18:55.625 INFO:teuthology.orchestra.run.smithi134.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T03:18:55.625 INFO:teuthology.orchestra.run.smithi134.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T03:18:55.625 INFO:teuthology.orchestra.run.smithi134.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-18T03:18:55.625 INFO:teuthology.orchestra.run.smithi134.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-18T03:18:55.625 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-18T03:18:55.625 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2023-07-18T03:18:55.777 DEBUG:teuthology.parallel:result is None 2023-07-18T03:18:57.171 INFO:teuthology.orchestra.run.smithi064.stdout:Installed products updated. 2023-07-18T03:18:57.193 INFO:teuthology.orchestra.run.smithi040.stdout:Installed products updated. 2023-07-18T03:18:57.388 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-07-18T03:18:57.388 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2023-07-18T03:18:57.388 INFO:teuthology.orchestra.run.smithi064.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T03:18:57.388 INFO:teuthology.orchestra.run.smithi064.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-18T03:18:57.388 INFO:teuthology.orchestra.run.smithi064.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T03:18:57.388 INFO:teuthology.orchestra.run.smithi064.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-18T03:18:57.389 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-18T03:18:57.389 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-18T03:18:57.389 INFO:teuthology.orchestra.run.smithi064.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T03:18:57.389 INFO:teuthology.orchestra.run.smithi064.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T03:18:57.389 INFO:teuthology.orchestra.run.smithi064.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-18T03:18:57.389 INFO:teuthology.orchestra.run.smithi064.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-18T03:18:57.389 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-07-18T03:18:57.390 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2023-07-18T03:18:57.408 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:18:57.408 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-18T03:18:57.408 INFO:teuthology.orchestra.run.smithi040.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T03:18:57.408 INFO:teuthology.orchestra.run.smithi040.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-18T03:18:57.408 INFO:teuthology.orchestra.run.smithi040.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T03:18:57.409 INFO:teuthology.orchestra.run.smithi040.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-18T03:18:57.409 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-18T03:18:57.409 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-18T03:18:57.409 INFO:teuthology.orchestra.run.smithi040.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T03:18:57.409 INFO:teuthology.orchestra.run.smithi040.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-18T03:18:57.409 INFO:teuthology.orchestra.run.smithi040.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-18T03:18:57.409 INFO:teuthology.orchestra.run.smithi040.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-18T03:18:57.410 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:18:57.410 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-18T03:18:57.535 DEBUG:teuthology.parallel:result is None 2023-07-18T03:18:57.555 DEBUG:teuthology.parallel:result is None 2023-07-18T03:18:57.556 INFO:teuthology.run_tasks:Running task install... 2023-07-18T03:18:57.565 DEBUG:teuthology.task.install:project ceph 2023-07-18T03:18:57.566 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9'}} 2023-07-18T03:18:57.566 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9'} 2023-07-18T03:18:57.566 INFO:teuthology.task.install:Using flavor: default 2023-07-18T03:18:57.572 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-18T03:18:57.573 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-18T03:18:57.573 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'tag': None, 'wait_for_package': False} 2023-07-18T03:18:57.574 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:18:57.576 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'tag': None, 'wait_for_package': False} 2023-07-18T03:18:57.576 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:18:57.577 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'tag': None, 'wait_for_package': False} 2023-07-18T03:18:57.578 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:18:57.865 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/ 2023-07-18T03:18:57.865 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4746.g81cf785a 2023-07-18T03:18:57.871 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/ 2023-07-18T03:18:57.871 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4746.g81cf785a 2023-07-18T03:18:57.875 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/ 2023-07-18T03:18:57.875 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4746.g81cf785a 2023-07-18T03:18:58.103 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-18T03:18:58.103 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:18:58.103 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-18T03:18:58.107 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-18T03:18:58.108 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-18T03:18:58.108 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-18T03:18:58.113 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-18T03:18:58.113 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-07-18T03:18:58.114 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-18T03:18:58.134 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-18T03:18:58.134 DEBUG:teuthology.orchestra.run.smithi040:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81cf785aa7781c4e08381c540f890cb9bc987ef9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-18T03:18:58.140 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-18T03:18:58.140 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81cf785aa7781c4e08381c540f890cb9bc987ef9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-18T03:18:58.142 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-18T03:18:58.142 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81cf785aa7781c4e08381c540f890cb9bc987ef9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-18T03:18:58.206 DEBUG:teuthology.orchestra.run.smithi040:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-18T03:18:58.211 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-18T03:18:58.212 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-18T03:18:58.289 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-18T03:18:58.290 DEBUG:teuthology.orchestra.run.smithi040:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-18T03:18:58.295 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-18T03:18:58.356 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2023-07-18T03:18:58.357 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2023-07-18T03:18:58.361 INFO:teuthology.orchestra.run.smithi040.stdout:check_obsoletes = 1 2023-07-18T03:18:58.362 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum clean all 2023-07-18T03:18:58.364 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2023-07-18T03:18:58.366 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2023-07-18T03:18:58.739 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2023-07-18T03:18:58.748 INFO:teuthology.orchestra.run.smithi064.stdout:Updating Subscription Management repositories. 2023-07-18T03:18:58.758 INFO:teuthology.orchestra.run.smithi040.stdout:Updating Subscription Management repositories. 2023-07-18T03:18:59.000 INFO:teuthology.orchestra.run.smithi134.stdout:44 files removed 2023-07-18T03:18:59.028 INFO:teuthology.orchestra.run.smithi064.stdout:44 files removed 2023-07-18T03:18:59.045 INFO:teuthology.orchestra.run.smithi040.stdout:44 files removed 2023-07-18T03:18:59.047 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-18T03:18:59.076 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-18T03:18:59.092 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-18T03:18:59.377 INFO:teuthology.orchestra.run.smithi134.stdout:Updating Subscription Management repositories. 2023-07-18T03:18:59.428 INFO:teuthology.orchestra.run.smithi064.stdout:Updating Subscription Management repositories. 2023-07-18T03:18:59.442 INFO:teuthology.orchestra.run.smithi040.stdout:Updating Subscription Management repositories. 2023-07-18T03:19:00.331 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 140 kB/s | 81 kB 00:00 2023-07-18T03:19:00.455 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 132 kB/s | 81 kB 00:00 2023-07-18T03:19:00.509 INFO:teuthology.orchestra.run.smithi040.stdout:ceph packages for x86_64 131 kB/s | 81 kB 00:00 2023-07-18T03:19:00.823 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 20 kB/s | 9.4 kB 00:00 2023-07-18T03:19:00.881 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 24 kB/s | 9.4 kB 00:00 2023-07-18T03:19:00.926 INFO:teuthology.orchestra.run.smithi040.stdout:ceph noarch packages 25 kB/s | 9.4 kB 00:00 2023-07-18T03:19:01.324 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 3.1 kB/s | 1.5 kB 00:00 2023-07-18T03:19:01.415 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 2.9 kB/s | 1.5 kB 00:00 2023-07-18T03:19:01.435 INFO:teuthology.orchestra.run.smithi040.stdout:ceph source packages 3.1 kB/s | 1.5 kB 00:00 2023-07-18T03:19:01.583 INFO:teuthology.orchestra.run.smithi134.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-07-18T03:19:01.640 INFO:teuthology.orchestra.run.smithi064.stdout:Copr repo for python3-asyncssh owned by ceph 17 kB/s | 3.5 kB 00:00 2023-07-18T03:19:01.660 INFO:teuthology.orchestra.run.smithi040.stdout:Copr repo for python3-asyncssh owned by ceph 17 kB/s | 3.5 kB 00:00 2023-07-18T03:19:05.307 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 4.3 MB/s | 16 MB 00:03 2023-07-18T03:19:07.850 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 2.5 MB/s | 16 MB 00:06 2023-07-18T03:19:08.991 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 431 kB/s | 24 kB 00:00 2023-07-18T03:19:10.301 INFO:teuthology.orchestra.run.smithi064.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 43 MB/s | 53 MB 00:01 2023-07-18T03:19:10.994 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 1.7 MB/s | 16 MB 00:09 2023-07-18T03:19:11.283 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 590 kB/s | 24 kB 00:00 2023-07-18T03:19:12.211 INFO:teuthology.orchestra.run.smithi134.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 MB/s | 53 MB 00:00 2023-07-18T03:19:14.636 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 387 kB/s | 24 kB 00:00 2023-07-18T03:19:15.604 INFO:teuthology.orchestra.run.smithi040.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 59 MB/s | 53 MB 00:00 2023-07-18T03:19:18.860 INFO:teuthology.orchestra.run.smithi064.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 59 MB/s | 47 MB 00:00 2023-07-18T03:19:20.311 INFO:teuthology.orchestra.run.smithi134.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-07-18T03:19:24.264 INFO:teuthology.orchestra.run.smithi040.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 56 MB/s | 47 MB 00:00 2023-07-18T03:19:30.518 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-18T03:19:30.519 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-18T03:19:30.534 INFO:teuthology.orchestra.run.smithi064.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-18T03:19:30.535 INFO:teuthology.orchestra.run.smithi064.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-18T03:19:30.537 INFO:teuthology.orchestra.run.smithi064.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-18T03:19:30.538 INFO:teuthology.orchestra.run.smithi064.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-18T03:19:30.540 INFO:teuthology.orchestra.run.smithi064.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-18T03:19:30.543 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-18T03:19:30.544 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-18T03:19:30.688 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2023-07-18T03:19:30.701 INFO:teuthology.orchestra.run.smithi064.stdout:=========================================================================================================================== 2023-07-18T03:19:30.702 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2023-07-18T03:19:30.702 INFO:teuthology.orchestra.run.smithi064.stdout:=========================================================================================================================== 2023-07-18T03:19:30.702 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2023-07-18T03:19:30.702 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-18T03:19:30.702 INFO:teuthology.orchestra.run.smithi064.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-18T03:19:30.702 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.4 k 2023-07-18T03:19:30.703 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 5.4 M 2023-07-18T03:19:30.703 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 848 k 2023-07-18T03:19:30.703 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 165 k 2023-07-18T03:19:30.703 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 1.5 M 2023-07-18T03:19:30.703 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 129 k 2023-07-18T03:19:30.703 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 1.7 M 2023-07-18T03:19:30.703 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 7.4 M 2023-07-18T03:19:30.704 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 47 k 2023-07-18T03:19:30.704 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 13 M 2023-07-18T03:19:30.704 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 48 M 2023-07-18T03:19:30.704 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 258 k 2023-07-18T03:19:30.704 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 166 k 2023-07-18T03:19:30.704 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 258 k 2023-07-18T03:19:30.704 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 18 k 2023-07-18T03:19:30.704 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-18T03:19:30.705 INFO:teuthology.orchestra.run.smithi064.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-18T03:19:30.705 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-18T03:19:30.705 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 31 k 2023-07-18T03:19:30.705 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 772 k 2023-07-18T03:19:30.705 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 139 k 2023-07-18T03:19:30.706 INFO:teuthology.orchestra.run.smithi064.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-18T03:19:30.706 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-18T03:19:30.706 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-18T03:19:30.706 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-18T03:19:30.707 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-18T03:19:30.707 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-18T03:19:30.707 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 197 k 2023-07-18T03:19:30.707 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 370 k 2023-07-18T03:19:30.707 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 364 k 2023-07-18T03:19:30.707 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 107 k 2023-07-18T03:19:30.707 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 94 k 2023-07-18T03:19:30.707 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.4 M 2023-07-18T03:19:30.708 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 179 k 2023-07-18T03:19:30.708 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-18T03:19:30.708 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2023-07-18T03:19:30.708 INFO:teuthology.orchestra.run.smithi064.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-18T03:19:30.708 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.7 M 2023-07-18T03:19:30.708 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.6 M 2023-07-18T03:19:30.708 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2023-07-18T03:19:30.709 INFO:teuthology.orchestra.run.smithi064.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-18T03:19:30.709 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 22 M 2023-07-18T03:19:30.709 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 23 k 2023-07-18T03:19:30.709 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 2.3 M 2023-07-18T03:19:30.709 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 240 k 2023-07-18T03:19:30.709 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 4.6 M 2023-07-18T03:19:30.709 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 18 M 2023-07-18T03:19:30.710 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 15 k 2023-07-18T03:19:30.710 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 24 k 2023-07-18T03:19:30.710 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-18T03:19:30.710 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-18T03:19:30.710 INFO:teuthology.orchestra.run.smithi064.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-18T03:19:30.710 INFO:teuthology.orchestra.run.smithi064.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-18T03:19:30.710 INFO:teuthology.orchestra.run.smithi064.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-18T03:19:30.710 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 137 k 2023-07-18T03:19:30.711 INFO:teuthology.orchestra.run.smithi064.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-18T03:19:30.711 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-18T03:19:30.711 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-18T03:19:30.711 INFO:teuthology.orchestra.run.smithi064.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-18T03:19:30.711 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-18T03:19:30.711 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-18T03:19:30.711 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 473 k 2023-07-18T03:19:30.712 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-18T03:19:30.712 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.7 M 2023-07-18T03:19:30.712 INFO:teuthology.orchestra.run.smithi064.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-18T03:19:30.712 INFO:teuthology.orchestra.run.smithi064.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-18T03:19:30.712 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-18T03:19:30.712 INFO:teuthology.orchestra.run.smithi064.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-18T03:19:30.712 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-18T03:19:30.713 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-18T03:19:30.713 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-18T03:19:30.713 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-18T03:19:30.713 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-18T03:19:30.713 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-18T03:19:30.713 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-18T03:19:30.713 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-18T03:19:30.714 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-18T03:19:30.714 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-18T03:19:30.714 INFO:teuthology.orchestra.run.smithi064.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-18T03:19:30.714 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-18T03:19:30.714 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 45 k 2023-07-18T03:19:30.714 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 119 k 2023-07-18T03:19:30.714 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-18T03:19:30.714 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-18T03:19:30.715 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-18T03:19:30.715 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-18T03:19:30.715 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-18T03:19:30.715 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-18T03:19:30.715 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-18T03:19:30.715 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-18T03:19:30.715 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-18T03:19:30.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-18T03:19:30.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-18T03:19:30.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-18T03:19:30.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-18T03:19:30.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-18T03:19:30.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-18T03:19:30.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-18T03:19:30.716 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-18T03:19:30.717 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-18T03:19:30.717 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-18T03:19:30.717 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-18T03:19:30.717 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-18T03:19:30.717 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-18T03:19:30.717 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-18T03:19:30.717 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-18T03:19:30.718 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-18T03:19:30.718 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-18T03:19:30.718 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-18T03:19:30.718 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-18T03:19:30.718 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-18T03:19:30.718 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-18T03:19:30.718 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-18T03:19:30.718 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-18T03:19:30.719 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-18T03:19:30.719 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-18T03:19:30.719 INFO:teuthology.orchestra.run.smithi064.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-18T03:19:30.719 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-18T03:19:30.719 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-18T03:19:30.719 INFO:teuthology.orchestra.run.smithi064.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-18T03:19:30.719 INFO:teuthology.orchestra.run.smithi064.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-18T03:19:30.719 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-18T03:19:30.720 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-18T03:19:30.720 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-18T03:19:30.720 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-18T03:19:30.720 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-18T03:19:30.720 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-18T03:19:30.720 INFO:teuthology.orchestra.run.smithi064.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-18T03:19:30.720 INFO:teuthology.orchestra.run.smithi064.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-18T03:19:30.721 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-18T03:19:30.721 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-18T03:19:30.721 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling module streams: 2023-07-18T03:19:30.721 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql 10 2023-07-18T03:19:30.721 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-07-18T03:19:30.721 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2023-07-18T03:19:30.721 INFO:teuthology.orchestra.run.smithi064.stdout:=========================================================================================================================== 2023-07-18T03:19:30.721 INFO:teuthology.orchestra.run.smithi064.stdout:Install 128 Packages 2023-07-18T03:19:30.722 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 3 Packages 2023-07-18T03:19:30.722 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-07-18T03:19:30.722 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 205 M 2023-07-18T03:19:30.723 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2023-07-18T03:19:31.186 INFO:teuthology.orchestra.run.smithi064.stdout:(1/131): ceph-18.0.0-4746.g81cf785a.el8.x86_64. 24 kB/s | 6.4 kB 00:00 2023-07-18T03:19:31.234 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-18T03:19:31.235 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-18T03:19:31.250 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-18T03:19:31.251 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-18T03:19:31.252 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-18T03:19:31.253 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-18T03:19:31.256 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-18T03:19:31.259 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-18T03:19:31.260 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-18T03:19:31.393 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2023-07-18T03:19:31.405 INFO:teuthology.orchestra.run.smithi134.stdout:=========================================================================================================================== 2023-07-18T03:19:31.406 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2023-07-18T03:19:31.406 INFO:teuthology.orchestra.run.smithi134.stdout:=========================================================================================================================== 2023-07-18T03:19:31.406 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2023-07-18T03:19:31.406 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-18T03:19:31.406 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-18T03:19:31.406 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.4 k 2023-07-18T03:19:31.407 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 5.4 M 2023-07-18T03:19:31.407 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 848 k 2023-07-18T03:19:31.407 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 165 k 2023-07-18T03:19:31.407 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 1.5 M 2023-07-18T03:19:31.407 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 129 k 2023-07-18T03:19:31.407 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 1.7 M 2023-07-18T03:19:31.407 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 7.4 M 2023-07-18T03:19:31.407 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 47 k 2023-07-18T03:19:31.408 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 13 M 2023-07-18T03:19:31.408 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 48 M 2023-07-18T03:19:31.408 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 258 k 2023-07-18T03:19:31.408 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 166 k 2023-07-18T03:19:31.408 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 258 k 2023-07-18T03:19:31.408 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 18 k 2023-07-18T03:19:31.408 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-18T03:19:31.409 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-18T03:19:31.409 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-18T03:19:31.409 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 31 k 2023-07-18T03:19:31.409 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 772 k 2023-07-18T03:19:31.409 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 139 k 2023-07-18T03:19:31.409 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-18T03:19:31.409 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-18T03:19:31.410 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-18T03:19:31.410 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-18T03:19:31.410 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-18T03:19:31.410 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-18T03:19:31.410 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 197 k 2023-07-18T03:19:31.410 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 370 k 2023-07-18T03:19:31.410 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 364 k 2023-07-18T03:19:31.411 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 107 k 2023-07-18T03:19:31.411 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 94 k 2023-07-18T03:19:31.411 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.4 M 2023-07-18T03:19:31.411 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 179 k 2023-07-18T03:19:31.411 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-18T03:19:31.411 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2023-07-18T03:19:31.411 INFO:teuthology.orchestra.run.smithi134.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-18T03:19:31.411 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.7 M 2023-07-18T03:19:31.412 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.6 M 2023-07-18T03:19:31.412 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2023-07-18T03:19:31.412 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-18T03:19:31.412 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 22 M 2023-07-18T03:19:31.412 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 23 k 2023-07-18T03:19:31.412 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 2.3 M 2023-07-18T03:19:31.412 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 240 k 2023-07-18T03:19:31.413 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 4.6 M 2023-07-18T03:19:31.413 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 18 M 2023-07-18T03:19:31.413 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 15 k 2023-07-18T03:19:31.413 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 24 k 2023-07-18T03:19:31.413 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-18T03:19:31.413 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-18T03:19:31.413 INFO:teuthology.orchestra.run.smithi134.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-18T03:19:31.413 INFO:teuthology.orchestra.run.smithi134.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-18T03:19:31.414 INFO:teuthology.orchestra.run.smithi134.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-18T03:19:31.414 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 137 k 2023-07-18T03:19:31.414 INFO:teuthology.orchestra.run.smithi134.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-18T03:19:31.414 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-18T03:19:31.414 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-18T03:19:31.414 INFO:teuthology.orchestra.run.smithi134.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-18T03:19:31.414 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-18T03:19:31.414 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-18T03:19:31.415 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 473 k 2023-07-18T03:19:31.415 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-18T03:19:31.415 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.7 M 2023-07-18T03:19:31.415 INFO:teuthology.orchestra.run.smithi134.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-18T03:19:31.415 INFO:teuthology.orchestra.run.smithi134.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-18T03:19:31.415 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-18T03:19:31.415 INFO:teuthology.orchestra.run.smithi134.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-18T03:19:31.416 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-18T03:19:31.416 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-18T03:19:31.416 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-18T03:19:31.416 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-18T03:19:31.416 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-18T03:19:31.416 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-18T03:19:31.416 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-18T03:19:31.417 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-18T03:19:31.417 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-18T03:19:31.417 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-18T03:19:31.417 INFO:teuthology.orchestra.run.smithi134.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-18T03:19:31.417 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-18T03:19:31.417 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 45 k 2023-07-18T03:19:31.417 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 119 k 2023-07-18T03:19:31.417 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-18T03:19:31.418 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-18T03:19:31.418 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-18T03:19:31.418 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-18T03:19:31.418 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-18T03:19:31.418 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-18T03:19:31.418 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-18T03:19:31.419 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-18T03:19:31.419 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-18T03:19:31.419 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-18T03:19:31.419 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-18T03:19:31.419 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-18T03:19:31.419 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-18T03:19:31.419 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-18T03:19:31.420 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-18T03:19:31.420 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-18T03:19:31.420 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-18T03:19:31.420 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-18T03:19:31.420 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-18T03:19:31.420 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-18T03:19:31.420 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-18T03:19:31.421 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-18T03:19:31.421 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-18T03:19:31.421 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-18T03:19:31.421 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-18T03:19:31.421 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-18T03:19:31.421 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-18T03:19:31.421 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-18T03:19:31.421 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-18T03:19:31.422 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-18T03:19:31.422 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-18T03:19:31.422 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-18T03:19:31.422 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-18T03:19:31.422 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-18T03:19:31.422 INFO:teuthology.orchestra.run.smithi134.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-18T03:19:31.422 INFO:teuthology.orchestra.run.smithi134.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-18T03:19:31.422 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-18T03:19:31.423 INFO:teuthology.orchestra.run.smithi134.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-18T03:19:31.423 INFO:teuthology.orchestra.run.smithi134.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-18T03:19:31.423 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-18T03:19:31.423 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-18T03:19:31.423 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-18T03:19:31.423 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-18T03:19:31.423 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-18T03:19:31.423 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-18T03:19:31.424 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-18T03:19:31.424 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-18T03:19:31.424 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-18T03:19:31.424 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-18T03:19:31.424 INFO:teuthology.orchestra.run.smithi134.stdout:Enabling module streams: 2023-07-18T03:19:31.424 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql 10 2023-07-18T03:19:31.424 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-18T03:19:31.425 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2023-07-18T03:19:31.425 INFO:teuthology.orchestra.run.smithi134.stdout:=========================================================================================================================== 2023-07-18T03:19:31.425 INFO:teuthology.orchestra.run.smithi134.stdout:Install 128 Packages 2023-07-18T03:19:31.425 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 3 Packages 2023-07-18T03:19:31.425 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-18T03:19:31.426 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 205 M 2023-07-18T03:19:31.426 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2023-07-18T03:19:31.836 INFO:teuthology.orchestra.run.smithi134.stdout:(1/131): ceph-18.0.0-4746.g81cf785a.el8.x86_64. 24 kB/s | 6.4 kB 00:00 2023-07-18T03:19:31.895 INFO:teuthology.orchestra.run.smithi064.stdout:(2/131): ceph-fuse-18.0.0-4746.g81cf785a.el8.x8 1.2 MB/s | 848 kB 00:00 2023-07-18T03:19:32.028 INFO:teuthology.orchestra.run.smithi064.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 1.2 MB/s | 165 kB 00:00 2023-07-18T03:19:32.553 INFO:teuthology.orchestra.run.smithi134.stdout:(2/131): ceph-fuse-18.0.0-4746.g81cf785a.el8.x8 1.2 MB/s | 848 kB 00:00 2023-07-18T03:19:32.712 INFO:teuthology.orchestra.run.smithi134.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 1.0 MB/s | 165 kB 00:00 2023-07-18T03:19:33.020 INFO:teuthology.orchestra.run.smithi064.stdout:(4/131): ceph-mds-18.0.0-4746.g81cf785a.el8.x86 2.3 MB/s | 2.3 MB 00:00 2023-07-18T03:19:33.779 INFO:teuthology.orchestra.run.smithi064.stdout:(5/131): ceph-mgr-18.0.0-4746.g81cf785a.el8.x86 2.0 MB/s | 1.5 MB 00:00 2023-07-18T03:19:33.912 INFO:teuthology.orchestra.run.smithi064.stdout:(6/131): ceph-base-18.0.0-4746.g81cf785a.el8.x8 1.8 MB/s | 5.4 MB 00:02 2023-07-18T03:19:34.863 INFO:teuthology.orchestra.run.smithi134.stdout:(4/131): ceph-mds-18.0.0-4746.g81cf785a.el8.x86 1.1 MB/s | 2.3 MB 00:02 2023-07-18T03:19:35.889 INFO:teuthology.orchestra.run.smithi040.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-18T03:19:35.890 INFO:teuthology.orchestra.run.smithi040.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-18T03:19:35.905 INFO:teuthology.orchestra.run.smithi040.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-18T03:19:35.906 INFO:teuthology.orchestra.run.smithi040.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-18T03:19:35.908 INFO:teuthology.orchestra.run.smithi040.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-18T03:19:35.909 INFO:teuthology.orchestra.run.smithi040.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-18T03:19:35.911 INFO:teuthology.orchestra.run.smithi040.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-18T03:19:35.914 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-18T03:19:35.915 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-18T03:19:36.060 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-07-18T03:19:36.073 INFO:teuthology.orchestra.run.smithi040.stdout:=========================================================================================================================== 2023-07-18T03:19:36.074 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2023-07-18T03:19:36.074 INFO:teuthology.orchestra.run.smithi040.stdout:=========================================================================================================================== 2023-07-18T03:19:36.074 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-07-18T03:19:36.074 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-18T03:19:36.075 INFO:teuthology.orchestra.run.smithi040.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-18T03:19:36.075 INFO:teuthology.orchestra.run.smithi040.stdout: ceph x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.4 k 2023-07-18T03:19:36.075 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 5.4 M 2023-07-18T03:19:36.075 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 848 k 2023-07-18T03:19:36.075 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 165 k 2023-07-18T03:19:36.075 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 1.5 M 2023-07-18T03:19:36.075 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 129 k 2023-07-18T03:19:36.076 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 1.7 M 2023-07-18T03:19:36.076 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 7.4 M 2023-07-18T03:19:36.076 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 47 k 2023-07-18T03:19:36.076 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 13 M 2023-07-18T03:19:36.076 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 48 M 2023-07-18T03:19:36.076 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 258 k 2023-07-18T03:19:36.076 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 166 k 2023-07-18T03:19:36.077 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 258 k 2023-07-18T03:19:36.077 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 18 k 2023-07-18T03:19:36.077 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-18T03:19:36.077 INFO:teuthology.orchestra.run.smithi040.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-18T03:19:36.077 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-18T03:19:36.077 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 31 k 2023-07-18T03:19:36.077 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 772 k 2023-07-18T03:19:36.078 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 139 k 2023-07-18T03:19:36.078 INFO:teuthology.orchestra.run.smithi040.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-18T03:19:36.078 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-18T03:19:36.078 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-18T03:19:36.078 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-18T03:19:36.078 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-18T03:19:36.079 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-18T03:19:36.079 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 197 k 2023-07-18T03:19:36.079 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 370 k 2023-07-18T03:19:36.079 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 364 k 2023-07-18T03:19:36.079 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 107 k 2023-07-18T03:19:36.079 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 94 k 2023-07-18T03:19:36.080 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.4 M 2023-07-18T03:19:36.080 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 179 k 2023-07-18T03:19:36.080 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-18T03:19:36.080 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrading: 2023-07-18T03:19:36.080 INFO:teuthology.orchestra.run.smithi040.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-18T03:19:36.080 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.7 M 2023-07-18T03:19:36.081 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.6 M 2023-07-18T03:19:36.081 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2023-07-18T03:19:36.081 INFO:teuthology.orchestra.run.smithi040.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-18T03:19:36.081 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 22 M 2023-07-18T03:19:36.082 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 23 k 2023-07-18T03:19:36.082 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 2.3 M 2023-07-18T03:19:36.082 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 240 k 2023-07-18T03:19:36.082 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 4.6 M 2023-07-18T03:19:36.082 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 18 M 2023-07-18T03:19:36.082 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 15 k 2023-07-18T03:19:36.083 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 24 k 2023-07-18T03:19:36.083 INFO:teuthology.orchestra.run.smithi040.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-18T03:19:36.083 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-18T03:19:36.083 INFO:teuthology.orchestra.run.smithi040.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-18T03:19:36.083 INFO:teuthology.orchestra.run.smithi040.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-18T03:19:36.083 INFO:teuthology.orchestra.run.smithi040.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-18T03:19:36.084 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 137 k 2023-07-18T03:19:36.084 INFO:teuthology.orchestra.run.smithi040.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-18T03:19:36.084 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-18T03:19:36.084 INFO:teuthology.orchestra.run.smithi040.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-18T03:19:36.084 INFO:teuthology.orchestra.run.smithi040.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-18T03:19:36.084 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-18T03:19:36.084 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-18T03:19:36.085 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 473 k 2023-07-18T03:19:36.085 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-18T03:19:36.085 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.7 M 2023-07-18T03:19:36.085 INFO:teuthology.orchestra.run.smithi040.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-18T03:19:36.085 INFO:teuthology.orchestra.run.smithi040.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-18T03:19:36.085 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-18T03:19:36.085 INFO:teuthology.orchestra.run.smithi040.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-18T03:19:36.086 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-18T03:19:36.086 INFO:teuthology.orchestra.run.smithi040.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-18T03:19:36.086 INFO:teuthology.orchestra.run.smithi040.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-18T03:19:36.086 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-18T03:19:36.086 INFO:teuthology.orchestra.run.smithi040.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-18T03:19:36.086 INFO:teuthology.orchestra.run.smithi040.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-18T03:19:36.086 INFO:teuthology.orchestra.run.smithi040.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-18T03:19:36.087 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-18T03:19:36.087 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-18T03:19:36.087 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-18T03:19:36.087 INFO:teuthology.orchestra.run.smithi040.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-18T03:19:36.087 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-18T03:19:36.087 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 45 k 2023-07-18T03:19:36.087 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 119 k 2023-07-18T03:19:36.087 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-18T03:19:36.088 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-18T03:19:36.088 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-18T03:19:36.088 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-18T03:19:36.088 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-18T03:19:36.088 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-18T03:19:36.088 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-18T03:19:36.088 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-18T03:19:36.089 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-18T03:19:36.089 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-18T03:19:36.089 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-18T03:19:36.089 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-18T03:19:36.089 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-18T03:19:36.089 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-18T03:19:36.090 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-18T03:19:36.090 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-18T03:19:36.090 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-18T03:19:36.090 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-18T03:19:36.090 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-18T03:19:36.090 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-18T03:19:36.090 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-18T03:19:36.090 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-18T03:19:36.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-18T03:19:36.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-18T03:19:36.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-18T03:19:36.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-18T03:19:36.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-18T03:19:36.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-18T03:19:36.092 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-18T03:19:36.092 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-18T03:19:36.092 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-18T03:19:36.092 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-18T03:19:36.092 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-18T03:19:36.093 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-18T03:19:36.093 INFO:teuthology.orchestra.run.smithi040.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-18T03:19:36.093 INFO:teuthology.orchestra.run.smithi040.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-18T03:19:36.093 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-18T03:19:36.093 INFO:teuthology.orchestra.run.smithi040.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-18T03:19:36.093 INFO:teuthology.orchestra.run.smithi040.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-18T03:19:36.093 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-18T03:19:36.093 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-18T03:19:36.094 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-18T03:19:36.094 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-18T03:19:36.094 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-18T03:19:36.094 INFO:teuthology.orchestra.run.smithi040.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-18T03:19:36.094 INFO:teuthology.orchestra.run.smithi040.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-18T03:19:36.094 INFO:teuthology.orchestra.run.smithi040.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-18T03:19:36.094 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-18T03:19:36.094 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-18T03:19:36.095 INFO:teuthology.orchestra.run.smithi040.stdout:Enabling module streams: 2023-07-18T03:19:36.095 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql 10 2023-07-18T03:19:36.095 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:19:36.095 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-07-18T03:19:36.095 INFO:teuthology.orchestra.run.smithi040.stdout:=========================================================================================================================== 2023-07-18T03:19:36.095 INFO:teuthology.orchestra.run.smithi040.stdout:Install 128 Packages 2023-07-18T03:19:36.095 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrade 3 Packages 2023-07-18T03:19:36.096 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:19:36.096 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 205 M 2023-07-18T03:19:36.096 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-07-18T03:19:36.255 INFO:teuthology.orchestra.run.smithi134.stdout:(5/131): ceph-mgr-18.0.0-4746.g81cf785a.el8.x86 1.1 MB/s | 1.5 MB 00:01 2023-07-18T03:19:36.398 INFO:teuthology.orchestra.run.smithi040.stdout:(1/131): ceph-18.0.0-4746.g81cf785a.el8.x86_64. 25 kB/s | 6.4 kB 00:00 2023-07-18T03:19:36.880 INFO:teuthology.orchestra.run.smithi064.stdout:(7/131): ceph-mon-18.0.0-4746.g81cf785a.el8.x86 1.5 MB/s | 4.6 MB 00:03 2023-07-18T03:19:37.004 INFO:teuthology.orchestra.run.smithi134.stdout:(6/131): ceph-base-18.0.0-4746.g81cf785a.el8.x8 1.0 MB/s | 5.4 MB 00:05 2023-07-18T03:19:37.164 INFO:teuthology.orchestra.run.smithi040.stdout:(2/131): ceph-fuse-18.0.0-4746.g81cf785a.el8.x8 1.1 MB/s | 848 kB 00:00 2023-07-18T03:19:37.331 INFO:teuthology.orchestra.run.smithi040.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 993 kB/s | 165 kB 00:00 2023-07-18T03:19:38.348 INFO:teuthology.orchestra.run.smithi040.stdout:(4/131): ceph-mds-18.0.0-4746.g81cf785a.el8.x86 2.3 MB/s | 2.3 MB 00:01 2023-07-18T03:19:38.649 INFO:teuthology.orchestra.run.smithi040.stdout:(5/131): ceph-base-18.0.0-4746.g81cf785a.el8.x8 2.1 MB/s | 5.4 MB 00:02 2023-07-18T03:19:39.099 INFO:teuthology.orchestra.run.smithi040.stdout:(6/131): ceph-mgr-18.0.0-4746.g81cf785a.el8.x86 2.1 MB/s | 1.5 MB 00:00 2023-07-18T03:19:39.416 INFO:teuthology.orchestra.run.smithi040.stdout:(7/131): ceph-mon-18.0.0-4746.g81cf785a.el8.x86 6.0 MB/s | 4.6 MB 00:00 2023-07-18T03:19:40.221 INFO:teuthology.orchestra.run.smithi134.stdout:(7/131): ceph-mon-18.0.0-4746.g81cf785a.el8.x86 1.2 MB/s | 4.6 MB 00:03 2023-07-18T03:19:40.624 INFO:teuthology.orchestra.run.smithi040.stdout:(8/131): ceph-radosgw-18.0.0-4746.g81cf785a.el8 11 MB/s | 13 MB 00:01 2023-07-18T03:19:40.700 INFO:teuthology.orchestra.run.smithi040.stdout:(9/131): ceph-selinux-18.0.0-4746.g81cf785a.el8 326 kB/s | 24 kB 00:00 2023-07-18T03:19:43.608 INFO:teuthology.orchestra.run.smithi040.stdout:(10/131): ceph-test-18.0.0-4746.g81cf785a.el8.x 16 MB/s | 48 MB 00:02 2023-07-18T03:19:44.033 INFO:teuthology.orchestra.run.smithi040.stdout:(11/131): cephfs-mirror-18.0.0-4746.g81cf785a.e 607 kB/s | 258 kB 00:00 2023-07-18T03:19:44.100 INFO:teuthology.orchestra.run.smithi040.stdout:(12/131): libcephfs-devel-18.0.0-4746.g81cf785a 467 kB/s | 31 kB 00:00 2023-07-18T03:19:44.267 INFO:teuthology.orchestra.run.smithi040.stdout:(13/131): libcephfs2-18.0.0-4746.g81cf785a.el8. 4.5 MB/s | 772 kB 00:00 2023-07-18T03:19:44.376 INFO:teuthology.orchestra.run.smithi040.stdout:(14/131): libcephsqlite-18.0.0-4746.g81cf785a.e 1.2 MB/s | 137 kB 00:00 2023-07-18T03:19:44.476 INFO:teuthology.orchestra.run.smithi040.stdout:(15/131): librados-devel-18.0.0-4746.g81cf785a. 1.4 MB/s | 139 kB 00:00 2023-07-18T03:19:44.625 INFO:teuthology.orchestra.run.smithi040.stdout:(16/131): libradosstriper1-18.0.0-4746.g81cf785 3.1 MB/s | 473 kB 00:00 2023-07-18T03:19:45.485 INFO:teuthology.orchestra.run.smithi040.stdout:(17/131): librgw2-18.0.0-4746.g81cf785a.el8.x86 7.8 MB/s | 6.7 MB 00:00 2023-07-18T03:19:45.569 INFO:teuthology.orchestra.run.smithi040.stdout:(18/131): python3-ceph-argparse-18.0.0-4746.g81 538 kB/s | 45 kB 00:00 2023-07-18T03:19:45.645 INFO:teuthology.orchestra.run.smithi064.stdout:(8/131): ceph-osd-18.0.0-4746.g81cf785a.el8.x86 1.5 MB/s | 18 MB 00:11 2023-07-18T03:19:45.652 INFO:teuthology.orchestra.run.smithi040.stdout:(19/131): python3-ceph-common-18.0.0-4746.g81cf 1.4 MB/s | 119 kB 00:00 2023-07-18T03:19:45.738 INFO:teuthology.orchestra.run.smithi064.stdout:(9/131): ceph-selinux-18.0.0-4746.g81cf785a.el8 266 kB/s | 24 kB 00:00 2023-07-18T03:19:45.744 INFO:teuthology.orchestra.run.smithi040.stdout:(20/131): python3-cephfs-18.0.0-4746.g81cf785a. 2.1 MB/s | 197 kB 00:00 2023-07-18T03:19:45.845 INFO:teuthology.orchestra.run.smithi040.stdout:(21/131): python3-rados-18.0.0-4746.g81cf785a.e 3.6 MB/s | 370 kB 00:00 2023-07-18T03:19:45.945 INFO:teuthology.orchestra.run.smithi040.stdout:(22/131): python3-rbd-18.0.0-4746.g81cf785a.el8 3.6 MB/s | 364 kB 00:00 2023-07-18T03:19:46.037 INFO:teuthology.orchestra.run.smithi040.stdout:(23/131): python3-rgw-18.0.0-4746.g81cf785a.el8 1.1 MB/s | 107 kB 00:00 2023-07-18T03:19:46.137 INFO:teuthology.orchestra.run.smithi040.stdout:(24/131): rbd-fuse-18.0.0-4746.g81cf785a.el8.x8 946 kB/s | 94 kB 00:00 2023-07-18T03:19:46.538 INFO:teuthology.orchestra.run.smithi064.stdout:(10/131): ceph-radosgw-18.0.0-4746.g81cf785a.el 1.4 MB/s | 13 MB 00:09 2023-07-18T03:19:46.763 INFO:teuthology.orchestra.run.smithi064.stdout:(11/131): ceph-common-18.0.0-4746.g81cf785a.el8 1.4 MB/s | 22 MB 00:15 2023-07-18T03:19:46.855 INFO:teuthology.orchestra.run.smithi064.stdout:(12/131): cephfs-mirror-18.0.0-4746.g81cf785a.e 814 kB/s | 258 kB 00:00 2023-07-18T03:19:46.880 INFO:teuthology.orchestra.run.smithi064.stdout:(13/131): libcephfs-devel-18.0.0-4746.g81cf785a 266 kB/s | 31 kB 00:00 2023-07-18T03:19:46.995 INFO:teuthology.orchestra.run.smithi040.stdout:(25/131): rbd-mirror-18.0.0-4746.g81cf785a.el8. 4.0 MB/s | 3.4 MB 00:00 2023-07-18T03:19:47.079 INFO:teuthology.orchestra.run.smithi040.stdout:(26/131): rbd-nbd-18.0.0-4746.g81cf785a.el8.x86 2.1 MB/s | 179 kB 00:00 2023-07-18T03:19:47.105 INFO:teuthology.orchestra.run.smithi064.stdout:(14/131): libcephsqlite-18.0.0-4746.g81cf785a.e 610 kB/s | 137 kB 00:00 2023-07-18T03:19:47.155 INFO:teuthology.orchestra.run.smithi040.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4746.g 307 kB/s | 23 kB 00:00 2023-07-18T03:19:47.255 INFO:teuthology.orchestra.run.smithi040.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4746.g81cf785 1.3 MB/s | 129 kB 00:00 2023-07-18T03:19:47.330 INFO:teuthology.orchestra.run.smithi064.stdout:(15/131): librados-devel-18.0.0-4746.g81cf785a. 618 kB/s | 139 kB 00:00 2023-07-18T03:19:47.547 INFO:teuthology.orchestra.run.smithi040.stdout:(29/131): ceph-osd-18.0.0-4746.g81cf785a.el8.x8 2.1 MB/s | 18 MB 00:08 2023-07-18T03:19:47.572 INFO:teuthology.orchestra.run.smithi064.stdout:(16/131): libcephfs2-18.0.0-4746.g81cf785a.el8. 1.1 MB/s | 772 kB 00:00 2023-07-18T03:19:47.839 INFO:teuthology.orchestra.run.smithi040.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4746.g81cf7 2.9 MB/s | 1.7 MB 00:00 2023-07-18T03:19:47.906 INFO:teuthology.orchestra.run.smithi064.stdout:(17/131): libradosstriper1-18.0.0-4746.g81cf785 822 kB/s | 473 kB 00:00 2023-07-18T03:19:47.922 INFO:teuthology.orchestra.run.smithi040.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4746.g81 2.8 MB/s | 240 kB 00:00 2023-07-18T03:19:47.989 INFO:teuthology.orchestra.run.smithi064.stdout:(18/131): python3-ceph-argparse-18.0.0-4746.g81 537 kB/s | 45 kB 00:00 2023-07-18T03:19:48.006 INFO:teuthology.orchestra.run.smithi040.stdout:(32/131): ceph-mgr-rook-18.0.0-4746.g81cf785a.e 569 kB/s | 47 kB 00:00 2023-07-18T03:19:48.081 INFO:teuthology.orchestra.run.smithi040.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4746.g8 194 kB/s | 15 kB 00:00 2023-07-18T03:19:48.123 INFO:teuthology.orchestra.run.smithi064.stdout:(19/131): python3-ceph-common-18.0.0-4746.g81cf 892 kB/s | 119 kB 00:00 2023-07-18T03:19:48.181 INFO:teuthology.orchestra.run.smithi040.stdout:(34/131): ceph-volume-18.0.0-4746.g81cf785a.el8 2.5 MB/s | 258 kB 00:00 2023-07-18T03:19:48.265 INFO:teuthology.orchestra.run.smithi040.stdout:(35/131): cephadm-18.0.0-4746.g81cf785a.el8.noa 1.9 MB/s | 166 kB 00:00 2023-07-18T03:19:48.323 INFO:teuthology.orchestra.run.smithi064.stdout:(20/131): python3-cephfs-18.0.0-4746.g81cf785a. 986 kB/s | 197 kB 00:00 2023-07-18T03:19:48.332 INFO:teuthology.orchestra.run.smithi040.stdout:(36/131): cephfs-top-18.0.0-4746.g81cf785a.el8. 270 kB/s | 18 kB 00:00 2023-07-18T03:19:48.615 INFO:teuthology.orchestra.run.smithi064.stdout:(21/131): python3-rados-18.0.0-4746.g81cf785a.e 1.2 MB/s | 370 kB 00:00 2023-07-18T03:19:48.729 INFO:teuthology.orchestra.run.smithi134.stdout:(8/131): ceph-common-18.0.0-4746.g81cf785a.el8. 1.3 MB/s | 22 MB 00:17 2023-07-18T03:19:48.829 INFO:teuthology.orchestra.run.smithi134.stdout:(9/131): ceph-selinux-18.0.0-4746.g81cf785a.el8 244 kB/s | 24 kB 00:00 2023-07-18T03:19:48.857 INFO:teuthology.orchestra.run.smithi064.stdout:(22/131): python3-rbd-18.0.0-4746.g81cf785a.el8 1.5 MB/s | 364 kB 00:00 2023-07-18T03:19:48.866 INFO:teuthology.orchestra.run.smithi040.stdout:(37/131): ceph-common-18.0.0-4746.g81cf785a.el8 1.7 MB/s | 22 MB 00:12 2023-07-18T03:19:49.016 INFO:teuthology.orchestra.run.smithi064.stdout:(23/131): python3-rgw-18.0.0-4746.g81cf785a.el8 672 kB/s | 107 kB 00:00 2023-07-18T03:19:49.033 INFO:teuthology.orchestra.run.smithi040.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 437 kB/s | 306 kB 00:00 2023-07-18T03:19:49.158 INFO:teuthology.orchestra.run.smithi040.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 601 kB/s | 75 kB 00:00 2023-07-18T03:19:49.183 INFO:teuthology.orchestra.run.smithi064.stdout:(24/131): rbd-fuse-18.0.0-4746.g81cf785a.el8.x8 567 kB/s | 94 kB 00:00 2023-07-18T03:19:49.358 INFO:teuthology.orchestra.run.smithi040.stdout:(40/131): liboath-2.6.2-3.el8.x86_64.rpm 119 kB/s | 59 kB 00:00 2023-07-18T03:19:49.475 INFO:teuthology.orchestra.run.smithi040.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.5 MB/s | 476 kB 00:00 2023-07-18T03:19:49.501 INFO:teuthology.orchestra.run.smithi040.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 307 kB/s | 44 kB 00:00 2023-07-18T03:19:49.592 INFO:teuthology.orchestra.run.smithi040.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.5 MB/s | 185 kB 00:00 2023-07-18T03:19:49.604 INFO:teuthology.orchestra.run.smithi134.stdout:(10/131): ceph-radosgw-18.0.0-4746.g81cf785a.el 1.4 MB/s | 13 MB 00:09 2023-07-18T03:19:49.618 INFO:teuthology.orchestra.run.smithi040.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 282 kB/s | 33 kB 00:00 2023-07-18T03:19:49.710 INFO:teuthology.orchestra.run.smithi040.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 139 kB/s | 16 kB 00:00 2023-07-18T03:19:49.804 INFO:teuthology.orchestra.run.smithi134.stdout:(11/131): cephfs-mirror-18.0.0-4746.g81cf785a.e 1.3 MB/s | 258 kB 00:00 2023-07-18T03:19:49.818 INFO:teuthology.orchestra.run.smithi040.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 862 kB/s | 173 kB 00:00 2023-07-18T03:19:49.880 INFO:teuthology.orchestra.run.smithi134.stdout:(12/131): libcephfs-devel-18.0.0-4746.g81cf785a 414 kB/s | 31 kB 00:00 2023-07-18T03:19:49.919 INFO:teuthology.orchestra.run.smithi040.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.8 MB/s | 384 kB 00:00 2023-07-18T03:19:49.944 INFO:teuthology.orchestra.run.smithi040.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 649 kB/s | 81 kB 00:00 2023-07-18T03:19:50.036 INFO:teuthology.orchestra.run.smithi040.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-07-18T03:19:50.061 INFO:teuthology.orchestra.run.smithi040.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 150 kB/s | 18 kB 00:00 2023-07-18T03:19:50.195 INFO:teuthology.orchestra.run.smithi040.stdout:(51/131): ceph-mgr-diskprediction-local-18.0.0- 2.8 MB/s | 7.4 MB 00:02 2023-07-18T03:19:50.239 INFO:teuthology.orchestra.run.smithi040.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 274 kB/s | 49 kB 00:00 2023-07-18T03:19:50.280 INFO:teuthology.orchestra.run.smithi134.stdout:(13/131): libcephfs2-18.0.0-4746.g81cf785a.el8. 1.9 MB/s | 772 kB 00:00 2023-07-18T03:19:50.354 INFO:teuthology.orchestra.run.smithi040.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 527 kB/s | 60 kB 00:00 2023-07-18T03:19:50.430 INFO:teuthology.orchestra.run.smithi134.stdout:(14/131): libcephsqlite-18.0.0-4746.g81cf785a.e 915 kB/s | 137 kB 00:00 2023-07-18T03:19:50.462 INFO:teuthology.orchestra.run.smithi040.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.6 MB/s | 283 kB 00:00 2023-07-18T03:19:50.546 INFO:teuthology.orchestra.run.smithi040.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 2.3 MB/s | 1.2 MB 00:00 2023-07-18T03:19:50.572 INFO:teuthology.orchestra.run.smithi134.stdout:(15/131): librados-devel-18.0.0-4746.g81cf785a. 984 kB/s | 139 kB 00:00 2023-07-18T03:19:50.580 INFO:teuthology.orchestra.run.smithi040.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 133 kB/s | 16 kB 00:00 2023-07-18T03:19:50.605 INFO:teuthology.orchestra.run.smithi040.stdout:(57/131): python3-more-itertools-7.2.0-3.el8.no 143 kB/s | 59 kB 00:00 2023-07-18T03:19:50.655 INFO:teuthology.orchestra.run.smithi040.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 308 kB/s | 34 kB 00:00 2023-07-18T03:19:50.697 INFO:teuthology.orchestra.run.smithi040.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.6 MB/s | 196 kB 00:00 2023-07-18T03:19:50.722 INFO:teuthology.orchestra.run.smithi040.stdout:(60/131): python3-rsa-4.8-1.el8.noarch.rpm 524 kB/s | 61 kB 00:00 2023-07-18T03:19:50.764 INFO:teuthology.orchestra.run.smithi040.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2023-07-18T03:19:50.806 INFO:teuthology.orchestra.run.smithi040.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 220 kB/s | 24 kB 00:00 2023-07-18T03:19:50.822 INFO:teuthology.orchestra.run.smithi134.stdout:(16/131): ceph-osd-18.0.0-4746.g81cf785a.el8.x8 1.3 MB/s | 18 MB 00:13 2023-07-18T03:19:50.840 INFO:teuthology.orchestra.run.smithi040.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 249 kB/s | 29 kB 00:00 2023-07-18T03:19:50.874 INFO:teuthology.orchestra.run.smithi040.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 252 kB/s | 27 kB 00:00 2023-07-18T03:19:50.897 INFO:teuthology.orchestra.run.smithi134.stdout:(17/131): libradosstriper1-18.0.0-4746.g81cf785 1.4 MB/s | 473 kB 00:00 2023-07-18T03:19:50.941 INFO:teuthology.orchestra.run.smithi040.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2023-07-18T03:19:50.981 INFO:teuthology.orchestra.run.smithi134.stdout:(18/131): python3-ceph-argparse-18.0.0-4746.g81 538 kB/s | 45 kB 00:00 2023-07-18T03:19:50.999 INFO:teuthology.orchestra.run.smithi040.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 488 kB/s | 61 kB 00:00 2023-07-18T03:19:51.058 INFO:teuthology.orchestra.run.smithi040.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 730 kB/s | 85 kB 00:00 2023-07-18T03:19:51.065 INFO:teuthology.orchestra.run.smithi134.stdout:(19/131): python3-ceph-common-18.0.0-4746.g81cf 1.4 MB/s | 119 kB 00:00 2023-07-18T03:19:51.083 INFO:teuthology.orchestra.run.smithi040.stdout:(68/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.0 MB/s | 251 kB 00:00 2023-07-18T03:19:51.108 INFO:teuthology.orchestra.run.smithi040.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-18T03:19:51.134 INFO:teuthology.orchestra.run.smithi040.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 171 kB/s | 23 kB 00:00 2023-07-18T03:19:51.201 INFO:teuthology.orchestra.run.smithi040.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 510 kB/s | 47 kB 00:00 2023-07-18T03:19:51.206 INFO:teuthology.orchestra.run.smithi134.stdout:(20/131): python3-cephfs-18.0.0-4746.g81cf785a. 1.4 MB/s | 197 kB 00:00 2023-07-18T03:19:51.226 INFO:teuthology.orchestra.run.smithi040.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.1 MB/s | 200 kB 00:00 2023-07-18T03:19:51.251 INFO:teuthology.orchestra.run.smithi040.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 849 kB/s | 43 kB 00:00 2023-07-18T03:19:51.276 INFO:teuthology.orchestra.run.smithi040.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-07-18T03:19:51.302 INFO:teuthology.orchestra.run.smithi040.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 956 kB/s | 48 kB 00:00 2023-07-18T03:19:51.327 INFO:teuthology.orchestra.run.smithi040.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-07-18T03:19:51.353 INFO:teuthology.orchestra.run.smithi040.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 851 kB/s | 43 kB 00:00 2023-07-18T03:19:51.366 INFO:teuthology.orchestra.run.smithi064.stdout:(25/131): librgw2-18.0.0-4746.g81cf785a.el8.x86 1.8 MB/s | 6.7 MB 00:03 2023-07-18T03:19:51.377 INFO:teuthology.orchestra.run.smithi040.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-07-18T03:19:51.407 INFO:teuthology.orchestra.run.smithi134.stdout:(21/131): python3-rados-18.0.0-4746.g81cf785a.e 1.8 MB/s | 370 kB 00:00 2023-07-18T03:19:51.420 INFO:teuthology.orchestra.run.smithi040.stdout:(79/131): thrift-0.13.0-2.el8.x86_64.rpm 4.8 MB/s | 1.7 MB 00:00 2023-07-18T03:19:51.445 INFO:teuthology.orchestra.run.smithi040.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.4 MB/s | 229 kB 00:00 2023-07-18T03:19:51.475 INFO:teuthology.orchestra.run.smithi064.stdout:(26/131): rbd-mirror-18.0.0-4746.g81cf785a.el8. 1.5 MB/s | 3.4 MB 00:02 2023-07-18T03:19:51.487 INFO:teuthology.orchestra.run.smithi040.stdout:(81/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.0 MB/s | 560 kB 00:00 2023-07-18T03:19:51.512 INFO:teuthology.orchestra.run.smithi040.stdout:(82/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 203 kB/s | 19 kB 00:00 2023-07-18T03:19:51.533 INFO:teuthology.orchestra.run.smithi064.stdout:(27/131): rbd-nbd-18.0.0-4746.g81cf785a.el8.x86 1.1 MB/s | 179 kB 00:00 2023-07-18T03:19:51.537 INFO:teuthology.orchestra.run.smithi040.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.8 MB/s | 155 kB 00:00 2023-07-18T03:19:51.554 INFO:teuthology.orchestra.run.smithi040.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 426 kB/s | 18 kB 00:00 2023-07-18T03:19:51.559 INFO:teuthology.orchestra.run.smithi064.stdout:(28/131): ceph-grafana-dashboards-18.0.0-4746.g 276 kB/s | 23 kB 00:00 2023-07-18T03:19:51.588 INFO:teuthology.orchestra.run.smithi040.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 5.4 MB/s | 559 kB 00:00 2023-07-18T03:19:51.590 INFO:teuthology.orchestra.run.smithi134.stdout:(22/131): python3-rbd-18.0.0-4746.g81cf785a.el8 1.9 MB/s | 364 kB 00:00 2023-07-18T03:19:51.645 INFO:teuthology.orchestra.run.smithi040.stdout:(86/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.5 MB/s | 605 kB 00:00 2023-07-18T03:19:51.666 INFO:teuthology.orchestra.run.smithi134.stdout:(23/131): python3-rgw-18.0.0-4746.g81cf785a.el8 1.4 MB/s | 107 kB 00:00 2023-07-18T03:19:51.667 INFO:teuthology.orchestra.run.smithi064.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4746.g81cf785 966 kB/s | 129 kB 00:00 2023-07-18T03:19:51.680 INFO:teuthology.orchestra.run.smithi040.stdout:(87/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.9 MB/s | 238 kB 00:00 2023-07-18T03:19:51.705 INFO:teuthology.orchestra.run.smithi040.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.2 MB/s | 72 kB 00:00 2023-07-18T03:19:51.731 INFO:teuthology.orchestra.run.smithi040.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-07-18T03:19:51.748 INFO:teuthology.orchestra.run.smithi040.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.4 MB/s | 60 kB 00:00 2023-07-18T03:19:51.749 INFO:teuthology.orchestra.run.smithi134.stdout:(24/131): rbd-fuse-18.0.0-4746.g81cf785a.el8.x8 1.1 MB/s | 94 kB 00:00 2023-07-18T03:19:51.773 INFO:teuthology.orchestra.run.smithi040.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-07-18T03:19:51.798 INFO:teuthology.orchestra.run.smithi040.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-07-18T03:19:51.832 INFO:teuthology.orchestra.run.smithi040.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 656 kB/s | 39 kB 00:00 2023-07-18T03:19:51.882 INFO:teuthology.orchestra.run.smithi040.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 7.5 MB/s | 644 kB 00:00 2023-07-18T03:19:51.908 INFO:teuthology.orchestra.run.smithi040.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.2 MB/s | 171 kB 00:00 2023-07-18T03:19:51.933 INFO:teuthology.orchestra.run.smithi040.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-07-18T03:19:51.991 INFO:teuthology.orchestra.run.smithi040.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 5.8 MB/s | 2.3 MB 00:00 2023-07-18T03:19:52.017 INFO:teuthology.orchestra.run.smithi040.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-07-18T03:19:52.042 INFO:teuthology.orchestra.run.smithi040.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-07-18T03:19:52.067 INFO:teuthology.orchestra.run.smithi040.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.0 MB/s | 259 kB 00:00 2023-07-18T03:19:52.101 INFO:teuthology.orchestra.run.smithi040.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-07-18T03:19:52.126 INFO:teuthology.orchestra.run.smithi040.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 662 kB/s | 39 kB 00:00 2023-07-18T03:19:52.177 INFO:teuthology.orchestra.run.smithi040.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 8.9 MB/s | 688 kB 00:00 2023-07-18T03:19:52.210 INFO:teuthology.orchestra.run.smithi040.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 6.3 MB/s | 538 kB 00:00 2023-07-18T03:19:52.235 INFO:teuthology.orchestra.run.smithi040.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-07-18T03:19:52.286 INFO:teuthology.orchestra.run.smithi040.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2023-07-18T03:19:52.356 INFO:teuthology.orchestra.run.smithi040.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 5.8 MB/s | 713 kB 00:00 2023-07-18T03:19:52.428 INFO:teuthology.orchestra.run.smithi040.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 372 kB/s | 27 kB 00:00 2023-07-18T03:19:52.486 INFO:teuthology.orchestra.run.smithi040.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 5.4 MB/s | 320 kB 00:00 2023-07-18T03:19:52.570 INFO:teuthology.orchestra.run.smithi040.stdout:(110/131): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-07-18T03:19:52.595 INFO:teuthology.orchestra.run.smithi040.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 405 kB/s | 44 kB 00:00 2023-07-18T03:19:52.643 INFO:teuthology.orchestra.run.smithi064.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4746.g81cf7 1.6 MB/s | 1.7 MB 00:01 2023-07-18T03:19:52.646 INFO:teuthology.orchestra.run.smithi040.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-07-18T03:19:52.796 INFO:teuthology.orchestra.run.smithi040.stdout:(113/131): python3-scipy-1.0.0-21.module+el8.5. 15 MB/s | 14 MB 00:00 2023-07-18T03:19:52.821 INFO:teuthology.orchestra.run.smithi040.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.3 MB/s | 225 kB 00:00 2023-07-18T03:19:52.846 INFO:teuthology.orchestra.run.smithi040.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2023-07-18T03:19:52.849 INFO:teuthology.orchestra.run.smithi064.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4746.g81 1.1 MB/s | 240 kB 00:00 2023-07-18T03:19:52.872 INFO:teuthology.orchestra.run.smithi040.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-07-18T03:19:52.939 INFO:teuthology.orchestra.run.smithi040.stdout:(117/131): python3-numpy-1.14.3-10.el8.x86_64.r 10 MB/s | 3.7 MB 00:00 2023-07-18T03:19:52.943 INFO:teuthology.orchestra.run.smithi064.stdout:(32/131): ceph-mgr-rook-18.0.0-4746.g81cf785a.e 505 kB/s | 47 kB 00:00 2023-07-18T03:19:52.964 INFO:teuthology.orchestra.run.smithi040.stdout:(118/131): python3-jsonpointer-1.10-11.el8.noar 170 kB/s | 20 kB 00:00 2023-07-18T03:19:52.989 INFO:teuthology.orchestra.run.smithi040.stdout:(119/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 882 kB/s | 103 kB 00:00 2023-07-18T03:19:53.035 INFO:teuthology.orchestra.run.smithi064.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4746.g8 158 kB/s | 15 kB 00:00 2023-07-18T03:19:53.051 INFO:teuthology.orchestra.run.smithi040.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 6.2 MB/s | 709 kB 00:00 2023-07-18T03:19:53.081 INFO:teuthology.orchestra.run.smithi040.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 2.1 MB/s | 198 kB 00:00 2023-07-18T03:19:53.157 INFO:teuthology.orchestra.run.smithi040.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 2.6 MB/s | 202 kB 00:00 2023-07-18T03:19:53.227 INFO:teuthology.orchestra.run.smithi064.stdout:(34/131): ceph-volume-18.0.0-4746.g81cf785a.el8 1.3 MB/s | 258 kB 00:00 2023-07-18T03:19:53.241 INFO:teuthology.orchestra.run.smithi040.stdout:(123/131): openblas-threads-0.3.15-3.el8.x86_64 18 MB/s | 4.9 MB 00:00 2023-07-18T03:19:53.283 INFO:teuthology.orchestra.run.smithi040.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.8 MB/s | 354 kB 00:00 2023-07-18T03:19:53.366 INFO:teuthology.orchestra.run.smithi040.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-07-18T03:19:53.400 INFO:teuthology.orchestra.run.smithi040.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.7 MB/s | 808 kB 00:00 2023-07-18T03:19:53.450 INFO:teuthology.orchestra.run.smithi040.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 18 MB/s | 1.5 MB 00:00 2023-07-18T03:19:53.534 INFO:teuthology.orchestra.run.smithi040.stdout:(128/131): postgresql-server-10.21-2.module+el8 17 MB/s | 5.1 MB 00:00 2023-07-18T03:19:53.584 INFO:teuthology.orchestra.run.smithi040.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-07-18T03:19:53.677 INFO:teuthology.orchestra.run.smithi064.stdout:(35/131): cephadm-18.0.0-4746.g81cf785a.el8.noa 370 kB/s | 166 kB 00:00 2023-07-18T03:19:53.761 INFO:teuthology.orchestra.run.smithi064.stdout:(36/131): cephfs-top-18.0.0-4746.g81cf785a.el8. 215 kB/s | 18 kB 00:00 2023-07-18T03:19:54.136 INFO:teuthology.orchestra.run.smithi064.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 817 kB/s | 306 kB 00:00 2023-07-18T03:19:54.211 INFO:teuthology.orchestra.run.smithi064.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 786 kB/s | 59 kB 00:00 2023-07-18T03:19:54.295 INFO:teuthology.orchestra.run.smithi064.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 901 kB/s | 75 kB 00:00 2023-07-18T03:19:54.454 INFO:teuthology.orchestra.run.smithi064.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.9 MB/s | 476 kB 00:00 2023-07-18T03:19:54.529 INFO:teuthology.orchestra.run.smithi064.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 583 kB/s | 44 kB 00:00 2023-07-18T03:19:54.567 INFO:teuthology.orchestra.run.smithi040.stdout:(130/131): librbd1-18.0.0-4746.g81cf785a.el8.x8 3.2 MB/s | 3.6 MB 00:01 2023-07-18T03:19:54.604 INFO:teuthology.orchestra.run.smithi064.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2023-07-18T03:19:54.671 INFO:teuthology.orchestra.run.smithi064.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 497 kB/s | 33 kB 00:00 2023-07-18T03:19:54.738 INFO:teuthology.orchestra.run.smithi064.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 244 kB/s | 16 kB 00:00 2023-07-18T03:19:54.768 INFO:teuthology.orchestra.run.smithi040.stdout:(131/131): librados2-18.0.0-4746.g81cf785a.el8. 2.7 MB/s | 3.7 MB 00:01 2023-07-18T03:19:54.771 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-18T03:19:54.771 INFO:teuthology.orchestra.run.smithi040.stdout:Total 11 MB/s | 205 MB 00:18 2023-07-18T03:19:54.808 INFO:teuthology.orchestra.run.smithi134.stdout:(25/131): rbd-mirror-18.0.0-4746.g81cf785a.el8. 1.1 MB/s | 3.4 MB 00:03 2023-07-18T03:19:54.822 INFO:teuthology.orchestra.run.smithi064.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2023-07-18T03:19:54.905 INFO:teuthology.orchestra.run.smithi064.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 4.5 MB/s | 384 kB 00:00 2023-07-18T03:19:54.981 INFO:teuthology.orchestra.run.smithi064.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2023-07-18T03:19:55.008 INFO:teuthology.orchestra.run.smithi134.stdout:(26/131): rbd-nbd-18.0.0-4746.g81cf785a.el8.x86 899 kB/s | 179 kB 00:00 2023-07-18T03:19:55.048 INFO:teuthology.orchestra.run.smithi064.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 167 kB/s | 11 kB 00:00 2023-07-18T03:19:55.092 INFO:teuthology.orchestra.run.smithi134.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4746.g 276 kB/s | 23 kB 00:00 2023-07-18T03:19:55.148 INFO:teuthology.orchestra.run.smithi064.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 176 kB/s | 18 kB 00:00 2023-07-18T03:19:55.242 INFO:teuthology.orchestra.run.smithi134.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4746.g81cf785 859 kB/s | 129 kB 00:00 2023-07-18T03:19:55.306 INFO:teuthology.orchestra.run.smithi064.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 7.4 MB/s | 1.2 MB 00:00 2023-07-18T03:19:55.390 INFO:teuthology.orchestra.run.smithi064.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 582 kB/s | 49 kB 00:00 2023-07-18T03:19:55.466 INFO:teuthology.orchestra.run.smithi064.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 783 kB/s | 59 kB 00:00 2023-07-18T03:19:55.541 INFO:teuthology.orchestra.run.smithi064.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 805 kB/s | 60 kB 00:00 2023-07-18T03:19:55.619 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-07-18T03:19:55.624 INFO:teuthology.orchestra.run.smithi064.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-07-18T03:19:55.700 INFO:teuthology.orchestra.run.smithi064.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 208 kB/s | 16 kB 00:00 2023-07-18T03:19:55.751 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-07-18T03:19:55.751 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-07-18T03:19:55.758 INFO:teuthology.orchestra.run.smithi064.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 579 kB/s | 34 kB 00:00 2023-07-18T03:19:55.825 INFO:teuthology.orchestra.run.smithi064.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 2.9 MB/s | 196 kB 00:00 2023-07-18T03:19:55.876 INFO:teuthology.orchestra.run.smithi134.stdout:(29/131): librgw2-18.0.0-4746.g81cf785a.el8.x86 1.3 MB/s | 6.7 MB 00:05 2023-07-18T03:19:55.917 INFO:teuthology.orchestra.run.smithi064.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 676 kB/s | 61 kB 00:00 2023-07-18T03:19:55.984 INFO:teuthology.orchestra.run.smithi064.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 279 kB/s | 19 kB 00:00 2023-07-18T03:19:56.043 INFO:teuthology.orchestra.run.smithi064.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 410 kB/s | 24 kB 00:00 2023-07-18T03:19:56.110 INFO:teuthology.orchestra.run.smithi064.stdout:(61/131): python3-tempora-1.14.1-5.el8.noarch.r 440 kB/s | 29 kB 00:00 2023-07-18T03:19:56.177 INFO:teuthology.orchestra.run.smithi064.stdout:(62/131): python3-trustme-0.6.0-4.el8.noarch.rp 413 kB/s | 27 kB 00:00 2023-07-18T03:19:56.252 INFO:teuthology.orchestra.run.smithi064.stdout:(63/131): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2023-07-18T03:19:56.327 INFO:teuthology.orchestra.run.smithi064.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.3 MB/s | 251 kB 00:00 2023-07-18T03:19:56.386 INFO:teuthology.orchestra.run.smithi064.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-07-18T03:19:56.445 INFO:teuthology.orchestra.run.smithi064.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2023-07-18T03:19:56.495 INFO:teuthology.orchestra.run.smithi064.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 459 kB/s | 23 kB 00:00 2023-07-18T03:19:56.576 INFO:teuthology.orchestra.run.smithi134.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4746.g81cf7 1.3 MB/s | 1.7 MB 00:01 2023-07-18T03:19:56.629 INFO:teuthology.orchestra.run.smithi064.stdout:(68/131): thrift-0.13.0-2.el8.x86_64.rpm 13 MB/s | 1.7 MB 00:00 2023-07-18T03:19:56.768 INFO:teuthology.orchestra.run.smithi134.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4746.g81 1.2 MB/s | 240 kB 00:00 2023-07-18T03:19:56.851 INFO:teuthology.orchestra.run.smithi134.stdout:(32/131): ceph-mgr-rook-18.0.0-4746.g81cf785a.e 569 kB/s | 47 kB 00:00 2023-07-18T03:19:56.933 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-07-18T03:19:56.935 INFO:teuthology.orchestra.run.smithi134.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4746.g8 175 kB/s | 15 kB 00:00 2023-07-18T03:19:56.938 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-07-18T03:19:56.962 INFO:teuthology.orchestra.run.smithi064.stdout:(69/131): ceph-mgr-diskprediction-local-18.0.0- 1.4 MB/s | 7.4 MB 00:05 2023-07-18T03:19:57.021 INFO:teuthology.orchestra.run.smithi064.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 177 kB/s | 70 kB 00:00 2023-07-18T03:19:57.079 INFO:teuthology.orchestra.run.smithi064.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 402 kB/s | 47 kB 00:00 2023-07-18T03:19:57.105 INFO:teuthology.orchestra.run.smithi064.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.6 MB/s | 200 kB 00:00 2023-07-18T03:19:57.130 INFO:teuthology.orchestra.run.smithi064.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 847 kB/s | 43 kB 00:00 2023-07-18T03:19:57.143 INFO:teuthology.orchestra.run.smithi134.stdout:(34/131): ceph-volume-18.0.0-4746.g81cf785a.el8 1.2 MB/s | 258 kB 00:00 2023-07-18T03:19:57.155 INFO:teuthology.orchestra.run.smithi064.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-07-18T03:19:57.181 INFO:teuthology.orchestra.run.smithi064.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 959 kB/s | 48 kB 00:00 2023-07-18T03:19:57.206 INFO:teuthology.orchestra.run.smithi064.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-07-18T03:19:57.231 INFO:teuthology.orchestra.run.smithi064.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 853 kB/s | 43 kB 00:00 2023-07-18T03:19:57.256 INFO:teuthology.orchestra.run.smithi064.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-07-18T03:19:57.282 INFO:teuthology.orchestra.run.smithi064.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-07-18T03:19:57.285 INFO:teuthology.orchestra.run.smithi134.stdout:(35/131): cephadm-18.0.0-4746.g81cf785a.el8.noa 1.1 MB/s | 166 kB 00:00 2023-07-18T03:19:57.316 INFO:teuthology.orchestra.run.smithi064.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-07-18T03:19:57.341 INFO:teuthology.orchestra.run.smithi064.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 319 kB/s | 19 kB 00:00 2023-07-18T03:19:57.361 INFO:teuthology.orchestra.run.smithi134.stdout:(36/131): cephfs-top-18.0.0-4746.g81cf785a.el8. 240 kB/s | 18 kB 00:00 2023-07-18T03:19:57.366 INFO:teuthology.orchestra.run.smithi064.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-07-18T03:19:57.400 INFO:teuthology.orchestra.run.smithi064.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-07-18T03:19:57.446 INFO:teuthology.orchestra.run.smithi064.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 225 kB/s | 18 kB 00:00 2023-07-18T03:19:57.484 INFO:teuthology.orchestra.run.smithi064.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.1 MB/s | 605 kB 00:00 2023-07-18T03:19:57.509 INFO:teuthology.orchestra.run.smithi064.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.7 MB/s | 238 kB 00:00 2023-07-18T03:19:57.559 INFO:teuthology.orchestra.run.smithi064.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-07-18T03:19:57.609 INFO:teuthology.orchestra.run.smithi064.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 19 MB/s | 2.3 MB 00:00 2023-07-18T03:19:57.635 INFO:teuthology.orchestra.run.smithi064.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-07-18T03:19:57.660 INFO:teuthology.orchestra.run.smithi064.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-07-18T03:19:57.685 INFO:teuthology.orchestra.run.smithi064.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-07-18T03:19:57.711 INFO:teuthology.orchestra.run.smithi064.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-07-18T03:19:57.736 INFO:teuthology.orchestra.run.smithi064.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 770 kB/s | 39 kB 00:00 2023-07-18T03:19:57.744 INFO:teuthology.orchestra.run.smithi134.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 799 kB/s | 306 kB 00:00 2023-07-18T03:19:57.769 INFO:teuthology.orchestra.run.smithi064.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-07-18T03:19:57.803 INFO:teuthology.orchestra.run.smithi064.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-07-18T03:19:57.820 INFO:teuthology.orchestra.run.smithi134.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 785 kB/s | 59 kB 00:00 2023-07-18T03:19:57.821 INFO:teuthology.orchestra.run.smithi064.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-07-18T03:19:57.870 INFO:teuthology.orchestra.run.smithi064.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.2 MB/s | 110 kB 00:00 2023-07-18T03:19:57.904 INFO:teuthology.orchestra.run.smithi134.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 901 kB/s | 75 kB 00:00 2023-07-18T03:19:57.912 INFO:teuthology.orchestra.run.smithi064.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-18T03:19:57.963 INFO:teuthology.orchestra.run.smithi064.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-07-18T03:19:58.013 INFO:teuthology.orchestra.run.smithi064.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-07-18T03:19:58.046 INFO:teuthology.orchestra.run.smithi064.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 1.1 MB/s | 39 kB 00:00 2023-07-18T03:19:58.105 INFO:teuthology.orchestra.run.smithi064.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-07-18T03:19:58.128 INFO:teuthology.orchestra.run.smithi134.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.1 MB/s | 476 kB 00:00 2023-07-18T03:19:58.155 INFO:teuthology.orchestra.run.smithi064.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 11 MB/s | 538 kB 00:00 2023-07-18T03:19:58.212 INFO:teuthology.orchestra.run.smithi134.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 524 kB/s | 44 kB 00:00 2023-07-18T03:19:58.314 INFO:teuthology.orchestra.run.smithi064.stdout:(104/131): python3-scipy-1.0.0-21.module+el8.5. 27 MB/s | 14 MB 00:00 2023-07-18T03:19:58.339 INFO:teuthology.orchestra.run.smithi064.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 854 kB/s | 157 kB 00:00 2023-07-18T03:19:58.362 INFO:teuthology.orchestra.run.smithi134.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.2 MB/s | 185 kB 00:00 2023-07-18T03:19:58.364 INFO:teuthology.orchestra.run.smithi064.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-07-18T03:19:58.398 INFO:teuthology.orchestra.run.smithi064.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-07-18T03:19:58.438 INFO:teuthology.orchestra.run.smithi134.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 440 kB/s | 33 kB 00:00 2023-07-18T03:19:58.440 INFO:teuthology.orchestra.run.smithi064.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 637 kB/s | 27 kB 00:00 2023-07-18T03:19:58.515 INFO:teuthology.orchestra.run.smithi064.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 4.2 MB/s | 320 kB 00:00 2023-07-18T03:19:58.521 INFO:teuthology.orchestra.run.smithi134.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 196 kB/s | 16 kB 00:00 2023-07-18T03:19:58.591 INFO:teuthology.orchestra.run.smithi064.stdout:(110/131): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-07-18T03:19:58.613 INFO:teuthology.orchestra.run.smithi134.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.8 MB/s | 173 kB 00:00 2023-07-18T03:19:58.616 INFO:teuthology.orchestra.run.smithi064.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 438 kB/s | 44 kB 00:00 2023-07-18T03:19:58.666 INFO:teuthology.orchestra.run.smithi064.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-07-18T03:19:58.733 INFO:teuthology.orchestra.run.smithi064.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 26 MB/s | 3.7 MB 00:00 2023-07-18T03:19:58.758 INFO:teuthology.orchestra.run.smithi064.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.4 MB/s | 225 kB 00:00 2023-07-18T03:19:58.764 INFO:teuthology.orchestra.run.smithi134.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.5 MB/s | 384 kB 00:00 2023-07-18T03:19:58.784 INFO:teuthology.orchestra.run.smithi064.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-07-18T03:19:58.809 INFO:teuthology.orchestra.run.smithi064.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-07-18T03:19:58.834 INFO:teuthology.orchestra.run.smithi064.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-07-18T03:19:58.859 INFO:teuthology.orchestra.run.smithi064.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-07-18T03:19:58.872 INFO:teuthology.orchestra.run.smithi134.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 751 kB/s | 81 kB 00:00 2023-07-18T03:19:58.893 INFO:teuthology.orchestra.run.smithi064.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-07-18T03:19:58.935 INFO:teuthology.orchestra.run.smithi064.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-07-18T03:19:58.981 INFO:teuthology.orchestra.run.smithi134.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 103 kB/s | 11 kB 00:00 2023-07-18T03:19:59.027 INFO:teuthology.orchestra.run.smithi064.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 29 MB/s | 4.9 MB 00:00 2023-07-18T03:19:59.069 INFO:teuthology.orchestra.run.smithi064.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-18T03:19:59.073 INFO:teuthology.orchestra.run.smithi134.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 192 kB/s | 18 kB 00:00 2023-07-18T03:19:59.145 INFO:teuthology.orchestra.run.smithi064.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 23 MB/s | 4.7 MB 00:00 2023-07-18T03:19:59.169 INFO:teuthology.orchestra.run.smithi064.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.4 MB/s | 354 kB 00:00 2023-07-18T03:19:59.228 INFO:teuthology.orchestra.run.smithi064.stdout:(125/131): postgresql-contrib-10.21-2.module+el 14 MB/s | 808 kB 00:00 2023-07-18T03:19:59.328 INFO:teuthology.orchestra.run.smithi064.stdout:(126/131): postgresql-server-10.21-2.module+el8 28 MB/s | 5.1 MB 00:00 2023-07-18T03:19:59.370 INFO:teuthology.orchestra.run.smithi064.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-07-18T03:19:59.540 INFO:teuthology.orchestra.run.smithi134.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 2.5 MB/s | 1.2 MB 00:00 2023-07-18T03:19:59.648 INFO:teuthology.orchestra.run.smithi134.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 451 kB/s | 49 kB 00:00 2023-07-18T03:19:59.749 INFO:teuthology.orchestra.run.smithi134.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 587 kB/s | 59 kB 00:00 2023-07-18T03:19:59.849 INFO:teuthology.orchestra.run.smithi134.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 602 kB/s | 60 kB 00:00 2023-07-18T03:20:00.016 INFO:teuthology.orchestra.run.smithi134.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2023-07-18T03:20:00.116 INFO:teuthology.orchestra.run.smithi134.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 156 kB/s | 16 kB 00:00 2023-07-18T03:20:00.208 INFO:teuthology.orchestra.run.smithi134.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 367 kB/s | 34 kB 00:00 2023-07-18T03:20:00.334 INFO:teuthology.orchestra.run.smithi134.stdout:(57/131): ceph-mgr-diskprediction-local-18.0.0- 1.7 MB/s | 7.4 MB 00:04 2023-07-18T03:20:00.409 INFO:teuthology.orchestra.run.smithi134.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 981 kB/s | 196 kB 00:00 2023-07-18T03:20:00.493 INFO:teuthology.orchestra.run.smithi134.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 225 kB/s | 19 kB 00:00 2023-07-18T03:20:00.584 INFO:teuthology.orchestra.run.smithi134.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 261 kB/s | 24 kB 00:00 2023-07-18T03:20:00.618 INFO:teuthology.orchestra.run.smithi134.stdout:(61/131): python3-rsa-4.8-1.el8.noarch.rpm 217 kB/s | 61 kB 00:00 2023-07-18T03:20:00.677 INFO:teuthology.orchestra.run.smithi134.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 318 kB/s | 29 kB 00:00 2023-07-18T03:20:00.710 INFO:teuthology.orchestra.run.smithi134.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 300 kB/s | 27 kB 00:00 2023-07-18T03:20:00.811 INFO:teuthology.orchestra.run.smithi134.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2023-07-18T03:20:00.928 INFO:teuthology.orchestra.run.smithi134.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 525 kB/s | 61 kB 00:00 2023-07-18T03:20:00.953 INFO:teuthology.orchestra.run.smithi134.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.0 MB/s | 251 kB 00:00 2023-07-18T03:20:01.028 INFO:teuthology.orchestra.run.smithi134.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 853 kB/s | 85 kB 00:00 2023-07-18T03:20:01.053 INFO:teuthology.orchestra.run.smithi134.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 229 kB/s | 23 kB 00:00 2023-07-18T03:20:01.079 INFO:teuthology.orchestra.run.smithi134.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-18T03:20:01.137 INFO:teuthology.orchestra.run.smithi134.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 804 kB/s | 47 kB 00:00 2023-07-18T03:20:01.179 INFO:teuthology.orchestra.run.smithi134.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-07-18T03:20:01.221 INFO:teuthology.orchestra.run.smithi134.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-07-18T03:20:01.263 INFO:teuthology.orchestra.run.smithi134.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-07-18T03:20:01.314 INFO:teuthology.orchestra.run.smithi134.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 963 kB/s | 48 kB 00:00 2023-07-18T03:20:01.389 INFO:teuthology.orchestra.run.smithi134.stdout:(75/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.1 MB/s | 87 kB 00:00 2023-07-18T03:20:01.439 INFO:teuthology.orchestra.run.smithi134.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 854 kB/s | 43 kB 00:00 2023-07-18T03:20:01.498 INFO:teuthology.orchestra.run.smithi134.stdout:(77/131): python3-pycparser-2.14-14.el8.noarch. 1.8 MB/s | 109 kB 00:00 2023-07-18T03:20:01.529 INFO:teuthology.orchestra.run.smithi064.stdout:(128/131): librados2-18.0.0-4746.g81cf785a.el8. 1.7 MB/s | 3.7 MB 00:02 2023-07-18T03:20:01.612 INFO:teuthology.orchestra.run.smithi064.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 9.4 MB/s | 798 kB 00:00 2023-07-18T03:20:01.665 INFO:teuthology.orchestra.run.smithi134.stdout:(78/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.3 MB/s | 229 kB 00:00 2023-07-18T03:20:01.823 INFO:teuthology.orchestra.run.smithi134.stdout:(79/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.5 MB/s | 560 kB 00:00 2023-07-18T03:20:01.854 INFO:teuthology.orchestra.run.smithi064.stdout:(130/131): librbd1-18.0.0-4746.g81cf785a.el8.x8 1.4 MB/s | 3.6 MB 00:02 2023-07-18T03:20:01.857 INFO:teuthology.orchestra.run.smithi134.stdout:(80/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 560 kB/s | 19 kB 00:00 2023-07-18T03:20:01.865 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-07-18T03:20:01.899 INFO:teuthology.orchestra.run.smithi134.stdout:(81/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.7 MB/s | 155 kB 00:00 2023-07-18T03:20:01.941 INFO:teuthology.orchestra.run.smithi134.stdout:(82/131): thrift-0.13.0-2.el8.x86_64.rpm 1.9 MB/s | 1.7 MB 00:00 2023-07-18T03:20:01.983 INFO:teuthology.orchestra.run.smithi134.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 6.5 MB/s | 559 kB 00:00 2023-07-18T03:20:02.025 INFO:teuthology.orchestra.run.smithi134.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 214 kB/s | 18 kB 00:00 2023-07-18T03:20:02.037 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-18T03:20:02.059 INFO:teuthology.orchestra.run.smithi134.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 8.5 MB/s | 605 kB 00:00 2023-07-18T03:20:02.084 INFO:teuthology.orchestra.run.smithi134.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-07-18T03:20:02.126 INFO:teuthology.orchestra.run.smithi134.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-07-18T03:20:02.218 INFO:teuthology.orchestra.run.smithi134.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 15 MB/s | 2.3 MB 00:00 2023-07-18T03:20:02.243 INFO:teuthology.orchestra.run.smithi134.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 709 kB/s | 83 kB 00:00 2023-07-18T03:20:02.256 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-18T03:20:02.268 INFO:teuthology.orchestra.run.smithi134.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-07-18T03:20:02.294 INFO:teuthology.orchestra.run.smithi134.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-07-18T03:20:02.319 INFO:teuthology.orchestra.run.smithi134.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-07-18T03:20:02.353 INFO:teuthology.orchestra.run.smithi134.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 661 kB/s | 39 kB 00:00 2023-07-18T03:20:02.386 INFO:teuthology.orchestra.run.smithi134.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-07-18T03:20:02.412 INFO:teuthology.orchestra.run.smithi134.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-07-18T03:20:02.422 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-18T03:20:02.437 INFO:teuthology.orchestra.run.smithi134.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-07-18T03:20:02.470 INFO:teuthology.orchestra.run.smithi134.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 3.2 MB/s | 110 kB 00:00 2023-07-18T03:20:02.512 INFO:teuthology.orchestra.run.smithi134.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-18T03:20:02.540 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-18T03:20:02.554 INFO:teuthology.orchestra.run.smithi134.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-18T03:20:02.605 INFO:teuthology.orchestra.run.smithi134.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-07-18T03:20:02.671 INFO:teuthology.orchestra.run.smithi134.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 583 kB/s | 39 kB 00:00 2023-07-18T03:20:02.723 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-18T03:20:02.839 INFO:teuthology.orchestra.run.smithi134.stdout:(102/131): python3-scipy-1.0.0-21.module+el8.5. 32 MB/s | 14 MB 00:00 2023-07-18T03:20:02.872 INFO:teuthology.orchestra.run.smithi134.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 3.4 MB/s | 688 kB 00:00 2023-07-18T03:20:02.906 INFO:teuthology.orchestra.run.smithi134.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.9 MB/s | 538 kB 00:00 2023-07-18T03:20:02.931 INFO:teuthology.orchestra.run.smithi134.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-07-18T03:20:02.956 INFO:teuthology.orchestra.run.smithi134.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-07-18T03:20:03.023 INFO:teuthology.orchestra.run.smithi134.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 7.6 MB/s | 713 kB 00:00 2023-07-18T03:20:03.124 INFO:teuthology.orchestra.run.smithi134.stdout:(108/131): python3-babel-2.5.1-7.el8.noarch.rpm 29 MB/s | 4.8 MB 00:00 2023-07-18T03:20:03.144 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-18T03:20:03.157 INFO:teuthology.orchestra.run.smithi134.stdout:(109/131): python3-jsonpatch-1.21-2.el8.noarch. 199 kB/s | 27 kB 00:00 2023-07-18T03:20:03.182 INFO:teuthology.orchestra.run.smithi134.stdout:(110/131): flex-2.6.1-9.el8.x86_64.rpm 5.4 MB/s | 320 kB 00:00 2023-07-18T03:20:03.201 INFO:teuthology.orchestra.run.smithi134.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 999 kB/s | 44 kB 00:00 2023-07-18T03:20:03.252 INFO:teuthology.orchestra.run.smithi134.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 8.9 MB/s | 457 kB 00:00 2023-07-18T03:20:03.319 INFO:teuthology.orchestra.run.smithi134.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 27 MB/s | 3.7 MB 00:00 2023-07-18T03:20:03.322 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-18T03:20:03.344 INFO:teuthology.orchestra.run.smithi134.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.4 MB/s | 225 kB 00:00 2023-07-18T03:20:03.369 INFO:teuthology.orchestra.run.smithi134.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-07-18T03:20:03.428 INFO:teuthology.orchestra.run.smithi134.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.5 MB/s | 126 kB 00:00 2023-07-18T03:20:03.478 INFO:teuthology.orchestra.run.smithi134.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 184 kB/s | 20 kB 00:00 2023-07-18T03:20:03.495 INFO:teuthology.orchestra.run.smithi134.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.5 MB/s | 103 kB 00:00 2023-07-18T03:20:03.537 INFO:teuthology.orchestra.run.smithi134.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-07-18T03:20:03.579 INFO:teuthology.orchestra.run.smithi134.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-07-18T03:20:03.671 INFO:teuthology.orchestra.run.smithi134.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 28 MB/s | 4.9 MB 00:00 2023-07-18T03:20:03.730 INFO:teuthology.orchestra.run.smithi134.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2023-07-18T03:20:03.738 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-18T03:20:03.805 INFO:teuthology.orchestra.run.smithi134.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2023-07-18T03:20:03.830 INFO:teuthology.orchestra.run.smithi134.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.5 MB/s | 354 kB 00:00 2023-07-18T03:20:03.897 INFO:teuthology.orchestra.run.smithi134.stdout:(125/131): postgresql-contrib-10.21-2.module+el 12 MB/s | 808 kB 00:00 2023-07-18T03:20:03.898 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T03:20:03.964 INFO:teuthology.orchestra.run.smithi134.stdout:(126/131): postgresql-server-10.21-2.module+el8 32 MB/s | 5.1 MB 00:00 2023-07-18T03:20:04.013 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T03:20:04.014 INFO:teuthology.orchestra.run.smithi134.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-07-18T03:20:04.189 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T03:20:04.274 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T03:20:04.461 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T03:20:04.630 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-18T03:20:05.370 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-18T03:20:05.560 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-18T03:20:05.984 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-18T03:20:06.209 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-18T03:20:06.347 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-18T03:20:06.476 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-18T03:20:06.639 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-18T03:20:06.777 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-18T03:20:06.956 INFO:teuthology.orchestra.run.smithi134.stdout:(128/131): librbd1-18.0.0-4746.g81cf785a.el8.x8 1.2 MB/s | 3.6 MB 00:02 2023-07-18T03:20:06.985 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-18T03:20:07.023 INFO:teuthology.orchestra.run.smithi134.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 12 MB/s | 798 kB 00:00 2023-07-18T03:20:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-18T03:20:07.256 INFO:teuthology.orchestra.run.smithi134.stdout:(130/131): librados2-18.0.0-4746.g81cf785a.el8. 1.1 MB/s | 3.7 MB 00:03 2023-07-18T03:20:07.345 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-18T03:20:07.480 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-18T03:20:07.624 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-18T03:20:07.780 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-18T03:20:08.456 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-18T03:20:08.647 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-18T03:20:09.280 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-18T03:20:09.463 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-18T03:20:10.188 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-18T03:20:10.422 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-18T03:20:10.578 INFO:teuthology.orchestra.run.smithi064.stdout:(131/131): ceph-test-18.0.0-4746.g81cf785a.el8. 1.9 MB/s | 48 MB 00:24 2023-07-18T03:20:10.581 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2023-07-18T03:20:10.582 INFO:teuthology.orchestra.run.smithi064.stdout:Total 5.1 MB/s | 205 MB 00:39 2023-07-18T03:20:10.607 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-18T03:20:10.868 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-18T03:20:11.045 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 26/134 2023-07-18T03:20:11.192 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 27/134 2023-07-18T03:20:11.422 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2023-07-18T03:20:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 28/134 2023-07-18T03:20:11.554 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2023-07-18T03:20:11.555 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2023-07-18T03:20:11.646 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-18T03:20:12.737 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2023-07-18T03:20:12.741 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2023-07-18T03:20:13.535 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-18T03:20:14.592 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-18T03:20:14.887 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-18T03:20:15.144 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-18T03:20:15.378 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-18T03:20:15.588 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-18T03:20:15.786 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-18T03:20:16.086 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 36/134 2023-07-18T03:20:16.261 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 37/134 2023-07-18T03:20:16.446 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-18T03:20:16.607 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-18T03:20:16.779 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-18T03:20:16.856 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-18T03:20:17.116 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-18T03:20:17.326 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-18T03:20:17.479 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-18T03:20:17.559 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-18T03:20:17.745 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2023-07-18T03:20:17.797 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-18T03:20:17.923 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-18T03:20:17.967 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-18T03:20:18.100 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-18T03:20:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-18T03:20:18.283 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-18T03:20:18.299 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-18T03:20:18.355 INFO:teuthology.orchestra.run.smithi134.stdout:(131/131): ceph-test-18.0.0-4746.g81cf785a.el8. 1.6 MB/s | 48 MB 00:29 2023-07-18T03:20:18.358 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2023-07-18T03:20:18.358 INFO:teuthology.orchestra.run.smithi134.stdout:Total 4.4 MB/s | 205 MB 00:46 2023-07-18T03:20:18.401 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-18T03:20:18.482 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-18T03:20:18.584 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-18T03:20:18.617 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-18T03:20:18.815 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-18T03:20:18.998 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-18T03:20:19.008 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-18T03:20:19.184 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2023-07-18T03:20:19.259 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-18T03:20:19.303 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2023-07-18T03:20:19.304 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2023-07-18T03:20:19.527 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-18T03:20:19.675 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-18T03:20:19.840 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-18T03:20:19.867 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T03:20:19.999 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T03:20:20.075 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-18T03:20:20.167 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T03:20:20.268 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T03:20:20.289 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-18T03:20:20.405 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2023-07-18T03:20:20.409 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2023-07-18T03:20:20.422 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T03:20:20.511 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-18T03:20:20.602 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-18T03:20:20.800 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-18T03:20:20.850 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-18T03:20:21.045 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-18T03:20:21.104 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-18T03:20:21.239 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-18T03:20:21.478 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-18T03:20:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-18T03:20:21.754 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-18T03:20:21.759 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-18T03:20:21.883 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-18T03:20:21.942 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-18T03:20:21.974 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-18T03:20:22.150 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-18T03:20:22.270 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-18T03:20:22.295 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-18T03:20:22.496 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-18T03:20:22.537 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-18T03:20:22.676 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-18T03:20:22.858 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-18T03:20:22.905 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-18T03:20:22.966 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-18T03:20:23.089 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-18T03:20:23.113 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-18T03:20:23.229 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-18T03:20:23.241 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-18T03:20:23.426 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-18T03:20:23.691 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-18T03:20:23.892 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-18T03:20:23.894 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-18T03:20:24.033 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-18T03:20:24.042 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-18T03:20:24.241 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-18T03:20:24.442 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-18T03:20:24.652 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-18T03:20:24.666 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-18T03:20:24.816 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-18T03:20:24.842 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-18T03:20:25.015 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-18T03:20:25.143 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-18T03:20:25.228 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2023-07-18T03:20:25.364 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-18T03:20:25.446 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-18T03:20:25.557 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-18T03:20:25.600 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-18T03:20:25.654 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-18T03:20:25.773 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-18T03:20:25.809 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-18T03:20:25.823 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-18T03:20:25.912 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-18T03:20:26.074 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-18T03:20:26.094 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-18T03:20:26.148 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-18T03:20:26.303 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-18T03:20:26.311 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-18T03:20:26.456 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 26/134 2023-07-18T03:20:26.479 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-18T03:20:26.550 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-18T03:20:26.586 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 27/134 2023-07-18T03:20:26.691 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-18T03:20:26.790 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-18T03:20:26.809 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 28/134 2023-07-18T03:20:26.982 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-18T03:20:27.187 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-18T03:20:27.382 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T03:20:27.513 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T03:20:27.698 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T03:20:27.792 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T03:20:27.853 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-18T03:20:28.019 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T03:20:28.117 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-18T03:20:28.198 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-18T03:20:28.354 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-18T03:20:28.427 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-18T03:20:28.676 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-18T03:20:28.689 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-18T03:20:28.863 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-18T03:20:28.973 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-18T03:20:29.065 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-18T03:20:29.213 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-18T03:20:29.301 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-18T03:20:29.456 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-18T03:20:29.548 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-18T03:20:29.739 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-18T03:20:29.865 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-18T03:20:29.946 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-18T03:20:30.034 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-18T03:20:30.085 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-18T03:20:30.177 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-18T03:20:30.239 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-18T03:20:30.278 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-18T03:20:30.366 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-18T03:20:30.411 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 93/134 2023-07-18T03:20:30.476 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-18T03:20:30.614 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-18T03:20:30.632 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-18T03:20:30.756 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-18T03:20:30.822 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-18T03:20:30.931 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-18T03:20:31.021 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-18T03:20:31.212 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-18T03:20:31.397 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 36/134 2023-07-18T03:20:31.539 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 37/134 2023-07-18T03:20:31.547 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-18T03:20:31.674 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-18T03:20:31.758 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-18T03:20:31.810 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-18T03:20:31.973 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-18T03:20:32.048 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-18T03:20:32.269 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-18T03:20:32.355 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-18T03:20:32.462 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-18T03:20:32.572 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-18T03:20:32.607 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-18T03:20:32.685 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-18T03:20:32.916 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-18T03:20:33.070 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-18T03:20:33.224 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-18T03:20:33.261 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-18T03:20:33.402 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-18T03:20:33.495 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-18T03:20:33.593 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-18T03:20:33.696 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-18T03:20:33.746 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-18T03:20:33.940 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-18T03:20:33.943 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-18T03:20:34.137 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 26/134 2023-07-18T03:20:34.142 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-18T03:20:34.275 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 27/134 2023-07-18T03:20:34.303 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T03:20:34.519 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-18T03:20:34.535 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T03:20:34.544 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 28/134 2023-07-18T03:20:34.755 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-18T03:20:34.802 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-18T03:20:35.011 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-18T03:20:35.150 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T03:20:35.249 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-18T03:20:35.388 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-18T03:20:35.475 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-18T03:20:35.580 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-18T03:20:35.652 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-18T03:20:35.900 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-18T03:20:36.073 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-18T03:20:36.266 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-18T03:20:36.463 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-18T03:20:36.499 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-18T03:20:36.644 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-18T03:20:36.810 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-18T03:20:37.074 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-18T03:20:37.312 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-18T03:20:37.496 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-18T03:20:37.666 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-18T03:20:37.819 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-18T03:20:37.875 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-18T03:20:38.006 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-18T03:20:38.090 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-18T03:20:38.160 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-18T03:20:38.341 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-18T03:20:38.430 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-18T03:20:38.566 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-18T03:20:38.622 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-18T03:20:38.748 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-18T03:20:38.752 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-18T03:20:38.892 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-18T03:20:38.952 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 36/134 2023-07-18T03:20:39.102 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 37/134 2023-07-18T03:20:39.107 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-18T03:20:39.262 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-18T03:20:39.285 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-18T03:20:39.442 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-18T03:20:39.448 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-18T03:20:39.592 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-18T03:20:39.621 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-18T03:20:39.698 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-18T03:20:39.712 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-18T03:20:39.923 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-18T03:20:39.951 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-18T03:20:40.169 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-18T03:20:40.183 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-18T03:20:40.354 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-18T03:20:40.355 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-18T03:20:40.435 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-18T03:20:40.621 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-18T03:20:40.652 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-18T03:20:40.805 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-18T03:20:40.842 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-18T03:20:40.956 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-18T03:20:40.988 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-18T03:20:41.148 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-18T03:20:41.160 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-18T03:20:41.324 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-18T03:20:41.508 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-18T03:20:41.712 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-18T03:20:41.911 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-18T03:20:42.286 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-18T03:20:42.337 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-18T03:20:42.555 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-18T03:20:42.571 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-18T03:20:42.781 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-18T03:20:42.852 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-18T03:20:43.007 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-18T03:20:43.160 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-18T03:20:43.250 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-18T03:20:43.447 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-18T03:20:43.545 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-18T03:20:43.705 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-18T03:20:43.770 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-18T03:20:43.989 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-18T03:20:44.217 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-18T03:20:44.400 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-18T03:20:44.504 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-18T03:20:44.616 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-18T03:20:44.647 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-18T03:20:44.721 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-18T03:20:44.881 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 93/134 2023-07-18T03:20:44.934 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-18T03:20:45.192 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-18T03:20:45.433 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-18T03:20:45.680 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-18T03:20:46.018 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-18T03:20:46.359 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-18T03:20:46.652 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-18T03:20:46.861 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-18T03:20:47.025 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-18T03:20:47.215 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-18T03:20:47.430 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-18T03:20:47.642 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-18T03:20:47.849 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-18T03:20:48.048 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-18T03:20:48.218 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-18T03:20:48.479 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-18T03:20:48.724 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-18T03:20:48.757 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T03:20:48.926 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-18T03:20:48.989 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T03:20:49.199 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-18T03:20:49.436 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-18T03:20:49.602 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T03:20:49.618 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-18T03:20:49.876 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-18T03:20:49.890 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-18T03:20:49.960 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-18T03:20:50.096 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-18T03:20:50.956 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-18T03:20:51.248 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-18T03:20:51.552 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-18T03:20:51.894 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-18T03:20:52.163 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-18T03:20:52.410 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-18T03:20:53.194 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-18T03:20:53.361 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-18T03:20:53.422 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-18T03:20:53.619 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 93/134 2023-07-18T03:20:55.234 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-18T03:20:55.234 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /sys 2023-07-18T03:20:55.235 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /proc 2023-07-18T03:20:55.235 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /dev 2023-07-18T03:20:55.235 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /run 2023-07-18T03:20:55.235 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /mnt 2023-07-18T03:20:55.235 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /var/tmp 2023-07-18T03:20:55.235 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /home 2023-07-18T03:20:55.235 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /tmp 2023-07-18T03:20:55.236 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /dev 2023-07-18T03:20:55.236 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:20:55.566 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-18T03:20:57.242 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-18T03:20:57.322 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T03:20:57.435 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-18T03:20:57.552 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T03:20:57.667 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-18T03:20:57.840 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-18T03:20:58.129 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-18T03:20:58.324 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-18T03:20:58.412 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-18T03:20:58.457 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-18T03:20:58.475 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-18T03:20:58.482 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-18T03:20:58.652 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-18T03:21:00.306 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-18T03:21:00.496 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-18T03:21:00.630 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 102/134 2023-07-18T03:21:00.832 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 103/134 2023-07-18T03:21:01.015 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-18T03:21:01.238 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-18T03:21:01.526 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-18T03:21:02.547 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-18T03:21:02.689 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-18T03:21:02.786 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-18T03:21:03.196 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 107/134 2023-07-18T03:21:03.403 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-18T03:21:03.439 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-18T03:21:03.590 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-18T03:21:03.601 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-18T03:21:05.002 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 110/134 2023-07-18T03:21:05.186 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-18T03:21:05.224 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-18T03:21:05.356 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-18T03:21:05.399 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-18T03:21:05.531 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-18T03:21:05.898 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-18T03:21:06.056 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-18T03:21:09.483 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-18T03:21:09.483 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2023-07-18T03:21:09.483 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2023-07-18T03:21:09.483 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /dev 2023-07-18T03:21:09.483 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /run 2023-07-18T03:21:09.484 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2023-07-18T03:21:09.484 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2023-07-18T03:21:09.484 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2023-07-18T03:21:09.484 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2023-07-18T03:21:09.484 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /dev 2023-07-18T03:21:09.484 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-07-18T03:21:09.828 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-18T03:21:10.902 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-18T03:21:11.500 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-18T03:21:11.679 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-18T03:21:11.832 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 115/134 2023-07-18T03:21:11.910 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-18T03:21:12.077 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-18T03:21:12.149 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-18T03:21:12.325 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 117/134 2023-07-18T03:21:12.559 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-18T03:21:12.567 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-18T03:21:12.694 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-18T03:21:12.711 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-18T03:21:12.713 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-18T03:21:12.917 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-18T03:21:13.151 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-18T03:21:13.318 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 121/134 2023-07-18T03:21:13.464 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-18T03:21:14.437 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-18T03:21:14.657 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-18T03:21:14.775 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 102/134 2023-07-18T03:21:14.960 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 103/134 2023-07-18T03:21:15.143 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-18T03:21:15.368 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-18T03:21:15.662 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-18T03:21:15.812 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T03:21:16.063 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T03:21:16.082 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T03:21:16.239 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 124/134 2023-07-18T03:21:16.539 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 125/134 2023-07-18T03:21:16.682 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-18T03:21:16.826 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-18T03:21:16.852 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 126/134 2023-07-18T03:21:16.923 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-18T03:21:17.184 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-18T03:21:17.273 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 107/134 2023-07-18T03:21:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-18T03:21:17.454 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-18T03:21:17.456 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-18T03:21:17.493 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-18T03:21:17.573 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-18T03:21:17.574 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2023-07-18T03:21:17.574 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2023-07-18T03:21:17.574 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /dev 2023-07-18T03:21:17.574 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /run 2023-07-18T03:21:17.574 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2023-07-18T03:21:17.574 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2023-07-18T03:21:17.574 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2023-07-18T03:21:17.575 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2023-07-18T03:21:17.575 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /dev 2023-07-18T03:21:17.575 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-18T03:21:17.626 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-18T03:21:17.638 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-18T03:21:17.646 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-18T03:21:17.659 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-18T03:21:17.854 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-18T03:21:17.863 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-18T03:21:17.966 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-18T03:21:17.998 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-18T03:21:17.999 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T03:21:18.183 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T03:21:18.183 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-18T03:21:18.420 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-18T03:21:18.654 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T03:21:19.036 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 110/134 2023-07-18T03:21:19.201 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-18T03:21:19.211 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T03:21:19.211 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 1/134 2023-07-18T03:21:19.212 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2/134 2023-07-18T03:21:19.212 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 3/134 2023-07-18T03:21:19.212 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 4/134 2023-07-18T03:21:19.212 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 5/134 2023-07-18T03:21:19.212 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T03:21:19.212 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T03:21:19.212 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T03:21:19.213 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-18T03:21:19.213 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 10/134 2023-07-18T03:21:19.213 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 11/134 2023-07-18T03:21:19.213 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 12/134 2023-07-18T03:21:19.213 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 13/134 2023-07-18T03:21:19.213 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 14/134 2023-07-18T03:21:19.213 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 15/134 2023-07-18T03:21:19.213 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 16/134 2023-07-18T03:21:19.214 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 17/134 2023-07-18T03:21:19.214 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 18/134 2023-07-18T03:21:19.214 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 19/134 2023-07-18T03:21:19.214 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 20/134 2023-07-18T03:21:19.215 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 21/134 2023-07-18T03:21:19.215 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 22/134 2023-07-18T03:21:19.215 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 23/134 2023-07-18T03:21:19.215 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 24/134 2023-07-18T03:21:19.216 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 25/134 2023-07-18T03:21:19.216 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 26/134 2023-07-18T03:21:19.216 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 27/134 2023-07-18T03:21:19.216 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 28/134 2023-07-18T03:21:19.216 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 29/134 2023-07-18T03:21:19.216 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 30/134 2023-07-18T03:21:19.223 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 31/134 2023-07-18T03:21:19.223 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 32/134 2023-07-18T03:21:19.223 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 33/134 2023-07-18T03:21:19.223 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 34/134 2023-07-18T03:21:19.223 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 35/134 2023-07-18T03:21:19.223 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 36/134 2023-07-18T03:21:19.224 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 37/134 2023-07-18T03:21:19.224 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 38/134 2023-07-18T03:21:19.224 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-18T03:21:19.224 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-18T03:21:19.224 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-18T03:21:19.224 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-18T03:21:19.224 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-18T03:21:19.225 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-18T03:21:19.225 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-18T03:21:19.225 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-18T03:21:19.225 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-18T03:21:19.225 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-18T03:21:19.225 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-18T03:21:19.225 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-18T03:21:19.225 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-18T03:21:19.226 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-18T03:21:19.226 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-18T03:21:19.226 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-18T03:21:19.226 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-18T03:21:19.226 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-18T03:21:19.226 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-18T03:21:19.226 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-18T03:21:19.226 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-18T03:21:19.227 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-18T03:21:19.227 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-18T03:21:19.227 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-18T03:21:19.227 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-18T03:21:19.227 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-18T03:21:19.227 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-18T03:21:19.227 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-18T03:21:19.227 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-18T03:21:19.228 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-18T03:21:19.228 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-18T03:21:19.228 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-18T03:21:19.228 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-18T03:21:19.228 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-18T03:21:19.228 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-18T03:21:19.228 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-18T03:21:19.229 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-18T03:21:19.229 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-18T03:21:19.229 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-18T03:21:19.229 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-18T03:21:19.229 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-18T03:21:19.229 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-18T03:21:19.229 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-18T03:21:19.229 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-18T03:21:19.230 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-18T03:21:19.230 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-18T03:21:19.230 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-18T03:21:19.230 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-18T03:21:19.230 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-18T03:21:19.230 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-18T03:21:19.230 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-18T03:21:19.230 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-18T03:21:19.231 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-18T03:21:19.231 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-18T03:21:19.231 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-18T03:21:19.231 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-18T03:21:19.231 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-18T03:21:19.231 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-18T03:21:19.231 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-18T03:21:19.231 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-18T03:21:19.232 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-18T03:21:19.232 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-18T03:21:19.232 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-18T03:21:19.232 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-18T03:21:19.232 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-18T03:21:19.232 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-18T03:21:19.232 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-18T03:21:19.233 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-18T03:21:19.233 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-18T03:21:19.233 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-18T03:21:19.233 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-18T03:21:19.233 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-18T03:21:19.233 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-18T03:21:19.233 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-18T03:21:19.233 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-18T03:21:19.234 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-18T03:21:19.234 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-18T03:21:19.234 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-18T03:21:19.234 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-18T03:21:19.234 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-18T03:21:19.234 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-18T03:21:19.234 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-18T03:21:19.234 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-18T03:21:19.235 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-18T03:21:19.235 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-18T03:21:19.235 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-18T03:21:19.235 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-18T03:21:19.235 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-18T03:21:19.235 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-18T03:21:19.235 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-18T03:21:19.235 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 129/134 2023-07-18T03:21:19.236 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-18T03:21:19.236 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 131/134 2023-07-18T03:21:19.236 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T03:21:19.236 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-18T03:21:19.251 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-18T03:21:19.384 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-18T03:21:19.427 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-18T03:21:19.547 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-18T03:21:19.551 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-18T03:21:19.734 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-18T03:21:19.917 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-18T03:21:19.952 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-18T03:21:20.093 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-18T03:21:20.132 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-18T03:21:20.593 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-18T03:21:20.732 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-18T03:21:20.749 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-18T03:21:21.726 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T03:21:21.727 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:58 ago on Tue 18 Jul 2023 03:19:23 AM UTC. 2023-07-18T03:21:22.479 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-18T03:21:22.686 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-18T03:21:22.804 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 102/134 2023-07-18T03:21:22.995 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 103/134 2023-07-18T03:21:23.189 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-18T03:21:23.402 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-18T03:21:23.708 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-18T03:21:24.493 INFO:teuthology.orchestra.run.smithi040.stdout:Installed products updated. 2023-07-18T03:21:24.669 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-18T03:21:24.732 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:21:24.732 INFO:teuthology.orchestra.run.smithi040.stdout:Upgraded: 2023-07-18T03:21:24.732 INFO:teuthology.orchestra.run.smithi040.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-18T03:21:24.733 INFO:teuthology.orchestra.run.smithi040.stdout: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.733 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.733 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-07-18T03:21:24.733 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-18T03:21:24.733 INFO:teuthology.orchestra.run.smithi040.stdout: automake-1.16.1-7.el8.noarch 2023-07-18T03:21:24.733 INFO:teuthology.orchestra.run.smithi040.stdout: bison-3.0.4-10.el8.x86_64 2023-07-18T03:21:24.733 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.733 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.734 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.734 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.734 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:24.734 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.734 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.734 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.734 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:24.735 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:24.735 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:24.735 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:24.735 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:24.735 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.735 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.735 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:24.735 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.736 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.736 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.736 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:24.736 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:24.736 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.736 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:24.736 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-18T03:21:24.737 INFO:teuthology.orchestra.run.smithi040.stdout: flex-2.6.1-9.el8.x86_64 2023-07-18T03:21:24.737 INFO:teuthology.orchestra.run.smithi040.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-18T03:21:24.737 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-18T03:21:24.737 INFO:teuthology.orchestra.run.smithi040.stdout: jq-1.6-3.el8.x86_64 2023-07-18T03:21:24.737 INFO:teuthology.orchestra.run.smithi040.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-18T03:21:24.737 INFO:teuthology.orchestra.run.smithi040.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-18T03:21:24.737 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-18T03:21:24.737 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.738 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.738 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.738 INFO:teuthology.orchestra.run.smithi040.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-18T03:21:24.738 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-18T03:21:24.738 INFO:teuthology.orchestra.run.smithi040.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-18T03:21:24.738 INFO:teuthology.orchestra.run.smithi040.stdout: libpq-13.5-1.el8.x86_64 2023-07-18T03:21:24.738 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-18T03:21:24.739 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-18T03:21:24.739 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.739 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.739 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-18T03:21:24.739 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.739 INFO:teuthology.orchestra.run.smithi040.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-18T03:21:24.739 INFO:teuthology.orchestra.run.smithi040.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-18T03:21:24.739 INFO:teuthology.orchestra.run.smithi040.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-18T03:21:24.740 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-18T03:21:24.740 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-18T03:21:24.740 INFO:teuthology.orchestra.run.smithi040.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-18T03:21:24.740 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-18T03:21:24.740 INFO:teuthology.orchestra.run.smithi040.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-18T03:21:24.740 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-18T03:21:24.740 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-18T03:21:24.741 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-18T03:21:24.741 INFO:teuthology.orchestra.run.smithi040.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-18T03:21:24.741 INFO:teuthology.orchestra.run.smithi040.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-18T03:21:24.742 INFO:teuthology.orchestra.run.smithi040.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-18T03:21:24.742 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T03:21:24.742 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T03:21:24.742 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T03:21:24.743 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-18T03:21:24.743 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-18T03:21:24.743 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-18T03:21:24.743 INFO:teuthology.orchestra.run.smithi040.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-18T03:21:24.744 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-18T03:21:24.744 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.744 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.744 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.745 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-18T03:21:24.745 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-18T03:21:24.745 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-18T03:21:24.746 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-18T03:21:24.746 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-18T03:21:24.746 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-18T03:21:24.746 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-18T03:21:24.747 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-18T03:21:24.747 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-18T03:21:24.747 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-18T03:21:24.747 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-18T03:21:24.748 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-18T03:21:24.748 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-18T03:21:24.748 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-18T03:21:24.748 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-18T03:21:24.748 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-18T03:21:24.748 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-18T03:21:24.748 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-18T03:21:24.749 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-18T03:21:24.749 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-18T03:21:24.749 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-18T03:21:24.749 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-18T03:21:24.749 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend-2.6-1.el8.noarch 2023-07-18T03:21:24.749 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-18T03:21:24.749 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-18T03:21:24.750 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-18T03:21:24.750 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-18T03:21:24.750 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-18T03:21:24.750 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-18T03:21:24.750 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.750 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.750 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-18T03:21:24.750 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-18T03:21:24.751 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.751 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-18T03:21:24.751 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-18T03:21:24.751 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-18T03:21:24.751 INFO:teuthology.orchestra.run.smithi040.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-18T03:21:24.751 INFO:teuthology.orchestra.run.smithi040.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-18T03:21:24.751 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-18T03:21:24.751 INFO:teuthology.orchestra.run.smithi040.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-18T03:21:24.752 INFO:teuthology.orchestra.run.smithi040.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-18T03:21:24.752 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-18T03:21:24.752 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-18T03:21:24.752 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-18T03:21:24.752 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-18T03:21:24.752 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-18T03:21:24.752 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.753 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.753 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:24.753 INFO:teuthology.orchestra.run.smithi040.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-18T03:21:24.753 INFO:teuthology.orchestra.run.smithi040.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-18T03:21:24.753 INFO:teuthology.orchestra.run.smithi040.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-18T03:21:24.753 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-18T03:21:24.754 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-18T03:21:24.754 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-18T03:21:24.755 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:21:24.755 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-07-18T03:21:24.839 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-18T03:21:24.927 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-18T03:21:24.928 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-18T03:21:25.069 DEBUG:teuthology.parallel:result is None 2023-07-18T03:21:25.319 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 107/134 2023-07-18T03:21:25.525 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-18T03:21:25.560 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-18T03:21:25.764 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-18T03:21:25.775 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-18T03:21:25.852 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 115/134 2023-07-18T03:21:26.128 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-18T03:21:26.312 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 117/134 2023-07-18T03:21:26.534 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-18T03:21:26.666 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-18T03:21:26.877 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-18T03:21:27.074 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 110/134 2023-07-18T03:21:27.121 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-18T03:21:27.258 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-18T03:21:27.296 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 121/134 2023-07-18T03:21:27.297 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-18T03:21:27.442 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-18T03:21:27.447 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-18T03:21:27.489 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-18T03:21:27.664 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-18T03:21:28.007 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-18T03:21:28.164 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-18T03:21:29.743 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T03:21:29.997 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T03:21:30.016 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T03:21:30.194 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 124/134 2023-07-18T03:21:30.433 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 125/134 2023-07-18T03:21:30.700 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 126/134 2023-07-18T03:21:31.054 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-18T03:21:31.215 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-18T03:21:31.300 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-18T03:21:31.491 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-18T03:21:31.503 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-18T03:21:31.674 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-18T03:21:31.682 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-18T03:21:31.810 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-18T03:21:31.810 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T03:21:31.994 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T03:21:31.994 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-18T03:21:32.230 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-18T03:21:32.440 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T03:21:32.683 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-18T03:21:32.970 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T03:21:32.970 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 1/134 2023-07-18T03:21:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2/134 2023-07-18T03:21:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 3/134 2023-07-18T03:21:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 4/134 2023-07-18T03:21:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 5/134 2023-07-18T03:21:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T03:21:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T03:21:32.971 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T03:21:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-18T03:21:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 10/134 2023-07-18T03:21:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 11/134 2023-07-18T03:21:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 12/134 2023-07-18T03:21:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 13/134 2023-07-18T03:21:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 14/134 2023-07-18T03:21:32.972 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 15/134 2023-07-18T03:21:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 16/134 2023-07-18T03:21:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 17/134 2023-07-18T03:21:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 18/134 2023-07-18T03:21:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 19/134 2023-07-18T03:21:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 20/134 2023-07-18T03:21:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 21/134 2023-07-18T03:21:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 22/134 2023-07-18T03:21:32.973 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 23/134 2023-07-18T03:21:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 24/134 2023-07-18T03:21:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 25/134 2023-07-18T03:21:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 26/134 2023-07-18T03:21:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 27/134 2023-07-18T03:21:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 28/134 2023-07-18T03:21:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 29/134 2023-07-18T03:21:32.974 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 30/134 2023-07-18T03:21:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 31/134 2023-07-18T03:21:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 32/134 2023-07-18T03:21:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 33/134 2023-07-18T03:21:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 34/134 2023-07-18T03:21:32.981 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 35/134 2023-07-18T03:21:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 36/134 2023-07-18T03:21:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 37/134 2023-07-18T03:21:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 38/134 2023-07-18T03:21:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-18T03:21:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-18T03:21:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-18T03:21:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-18T03:21:32.982 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-18T03:21:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-18T03:21:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-18T03:21:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-18T03:21:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-18T03:21:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-18T03:21:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-18T03:21:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-18T03:21:32.983 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-18T03:21:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-18T03:21:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-18T03:21:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-18T03:21:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-18T03:21:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-18T03:21:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-18T03:21:32.984 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-18T03:21:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-18T03:21:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-18T03:21:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-18T03:21:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-18T03:21:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-18T03:21:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-18T03:21:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-18T03:21:32.985 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-18T03:21:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-18T03:21:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-18T03:21:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-18T03:21:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-18T03:21:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-18T03:21:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-18T03:21:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-18T03:21:32.986 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-18T03:21:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-18T03:21:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-18T03:21:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-18T03:21:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-18T03:21:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-18T03:21:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-18T03:21:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-18T03:21:32.987 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-18T03:21:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-18T03:21:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-18T03:21:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-18T03:21:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-18T03:21:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-18T03:21:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-18T03:21:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-18T03:21:32.988 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-18T03:21:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-18T03:21:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-18T03:21:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-18T03:21:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-18T03:21:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-18T03:21:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-18T03:21:32.989 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-18T03:21:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-18T03:21:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-18T03:21:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-18T03:21:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-18T03:21:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-18T03:21:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-18T03:21:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-18T03:21:32.990 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-18T03:21:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-18T03:21:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-18T03:21:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-18T03:21:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-18T03:21:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-18T03:21:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-18T03:21:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-18T03:21:32.991 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-18T03:21:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-18T03:21:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-18T03:21:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-18T03:21:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-18T03:21:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-18T03:21:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-18T03:21:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-18T03:21:32.992 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-18T03:21:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-18T03:21:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-18T03:21:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-18T03:21:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-18T03:21:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-18T03:21:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-18T03:21:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-18T03:21:32.993 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 129/134 2023-07-18T03:21:32.994 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-18T03:21:32.994 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 131/134 2023-07-18T03:21:32.994 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T03:21:32.994 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-18T03:21:33.588 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 115/134 2023-07-18T03:21:33.919 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-18T03:21:34.132 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 117/134 2023-07-18T03:21:34.397 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-18T03:21:34.562 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-18T03:21:34.751 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-18T03:21:34.992 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-18T03:21:35.183 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 121/134 2023-07-18T03:21:35.354 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T03:21:35.355 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:02:17 ago on Tue 18 Jul 2023 03:19:18 AM UTC. 2023-07-18T03:21:35.372 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-18T03:21:37.556 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T03:21:37.812 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T03:21:37.829 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-18T03:21:37.992 INFO:teuthology.orchestra.run.smithi064.stdout:Installed products updated. 2023-07-18T03:21:38.038 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 124/134 2023-07-18T03:21:38.225 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-07-18T03:21:38.226 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2023-07-18T03:21:38.226 INFO:teuthology.orchestra.run.smithi064.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-18T03:21:38.226 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.226 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.226 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2023-07-18T03:21:38.226 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-18T03:21:38.226 INFO:teuthology.orchestra.run.smithi064.stdout: automake-1.16.1-7.el8.noarch 2023-07-18T03:21:38.227 INFO:teuthology.orchestra.run.smithi064.stdout: bison-3.0.4-10.el8.x86_64 2023-07-18T03:21:38.227 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.227 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.227 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.227 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.227 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:38.227 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.227 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.228 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.228 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:38.228 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:38.228 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:38.228 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:38.228 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:38.229 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.229 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.229 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:38.229 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.229 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.229 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.229 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:38.229 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:38.230 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.230 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:38.230 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-18T03:21:38.230 INFO:teuthology.orchestra.run.smithi064.stdout: flex-2.6.1-9.el8.x86_64 2023-07-18T03:21:38.230 INFO:teuthology.orchestra.run.smithi064.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-18T03:21:38.230 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-18T03:21:38.230 INFO:teuthology.orchestra.run.smithi064.stdout: jq-1.6-3.el8.x86_64 2023-07-18T03:21:38.230 INFO:teuthology.orchestra.run.smithi064.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-18T03:21:38.231 INFO:teuthology.orchestra.run.smithi064.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-18T03:21:38.231 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-18T03:21:38.231 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.231 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.231 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.231 INFO:teuthology.orchestra.run.smithi064.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-18T03:21:38.231 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-18T03:21:38.232 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-18T03:21:38.232 INFO:teuthology.orchestra.run.smithi064.stdout: libpq-13.5-1.el8.x86_64 2023-07-18T03:21:38.232 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-18T03:21:38.232 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-18T03:21:38.232 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.232 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.232 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-18T03:21:38.232 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.233 INFO:teuthology.orchestra.run.smithi064.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-18T03:21:38.233 INFO:teuthology.orchestra.run.smithi064.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-18T03:21:38.233 INFO:teuthology.orchestra.run.smithi064.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-18T03:21:38.233 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-18T03:21:38.233 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-18T03:21:38.233 INFO:teuthology.orchestra.run.smithi064.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-18T03:21:38.233 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-18T03:21:38.233 INFO:teuthology.orchestra.run.smithi064.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-18T03:21:38.234 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-18T03:21:38.234 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-18T03:21:38.234 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-18T03:21:38.234 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-18T03:21:38.234 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-18T03:21:38.234 INFO:teuthology.orchestra.run.smithi064.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-18T03:21:38.234 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T03:21:38.235 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T03:21:38.235 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T03:21:38.235 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-18T03:21:38.235 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-18T03:21:38.235 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-18T03:21:38.235 INFO:teuthology.orchestra.run.smithi064.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-18T03:21:38.235 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-18T03:21:38.235 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.236 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.236 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.236 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-18T03:21:38.236 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-18T03:21:38.236 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-18T03:21:38.236 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-18T03:21:38.236 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-18T03:21:38.237 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-18T03:21:38.237 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-18T03:21:38.237 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-18T03:21:38.237 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-18T03:21:38.237 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-18T03:21:38.237 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-18T03:21:38.237 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-18T03:21:38.237 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-18T03:21:38.238 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-18T03:21:38.238 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-18T03:21:38.238 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-18T03:21:38.238 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-18T03:21:38.238 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-18T03:21:38.238 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-18T03:21:38.238 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-18T03:21:38.239 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-18T03:21:38.239 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-18T03:21:38.239 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-2.6-1.el8.noarch 2023-07-18T03:21:38.239 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-18T03:21:38.239 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-18T03:21:38.239 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-18T03:21:38.239 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-18T03:21:38.240 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-18T03:21:38.240 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-18T03:21:38.240 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.240 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.240 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-18T03:21:38.240 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-18T03:21:38.240 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.241 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-18T03:21:38.241 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-18T03:21:38.241 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-18T03:21:38.241 INFO:teuthology.orchestra.run.smithi064.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-18T03:21:38.241 INFO:teuthology.orchestra.run.smithi064.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-18T03:21:38.241 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-18T03:21:38.241 INFO:teuthology.orchestra.run.smithi064.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-18T03:21:38.242 INFO:teuthology.orchestra.run.smithi064.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-18T03:21:38.242 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-18T03:21:38.242 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-18T03:21:38.242 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-18T03:21:38.242 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-18T03:21:38.242 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-18T03:21:38.242 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.243 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.243 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:38.243 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-18T03:21:38.243 INFO:teuthology.orchestra.run.smithi064.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-18T03:21:38.243 INFO:teuthology.orchestra.run.smithi064.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-18T03:21:38.243 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-18T03:21:38.244 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-18T03:21:38.244 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-18T03:21:38.244 INFO:teuthology.orchestra.run.smithi064.stdout: 2023-07-18T03:21:38.245 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2023-07-18T03:21:38.328 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 125/134 2023-07-18T03:21:38.580 DEBUG:teuthology.parallel:result is None 2023-07-18T03:21:38.664 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 126/134 2023-07-18T03:21:39.008 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-18T03:21:39.212 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-18T03:21:39.298 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-18T03:21:39.495 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-18T03:21:39.507 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-18T03:21:39.678 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-18T03:21:39.686 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-18T03:21:39.864 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-18T03:21:39.864 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T03:21:40.057 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T03:21:40.057 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-18T03:21:40.292 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-18T03:21:40.519 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T03:21:41.039 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T03:21:41.040 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 1/134 2023-07-18T03:21:41.040 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2/134 2023-07-18T03:21:41.040 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 3/134 2023-07-18T03:21:41.040 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 4/134 2023-07-18T03:21:41.040 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 5/134 2023-07-18T03:21:41.040 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-18T03:21:41.041 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-18T03:21:41.041 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-18T03:21:41.041 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-18T03:21:41.041 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 10/134 2023-07-18T03:21:41.041 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 11/134 2023-07-18T03:21:41.041 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 12/134 2023-07-18T03:21:41.041 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 13/134 2023-07-18T03:21:41.042 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 14/134 2023-07-18T03:21:41.042 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 15/134 2023-07-18T03:21:41.042 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 16/134 2023-07-18T03:21:41.042 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 17/134 2023-07-18T03:21:41.042 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 18/134 2023-07-18T03:21:41.042 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 19/134 2023-07-18T03:21:41.042 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 20/134 2023-07-18T03:21:41.043 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 21/134 2023-07-18T03:21:41.043 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 22/134 2023-07-18T03:21:41.043 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 23/134 2023-07-18T03:21:41.043 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 24/134 2023-07-18T03:21:41.043 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 25/134 2023-07-18T03:21:41.043 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 26/134 2023-07-18T03:21:41.043 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 27/134 2023-07-18T03:21:41.043 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 28/134 2023-07-18T03:21:41.044 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 29/134 2023-07-18T03:21:41.044 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 30/134 2023-07-18T03:21:41.044 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 31/134 2023-07-18T03:21:41.051 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 32/134 2023-07-18T03:21:41.051 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 33/134 2023-07-18T03:21:41.051 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 34/134 2023-07-18T03:21:41.051 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 35/134 2023-07-18T03:21:41.051 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 36/134 2023-07-18T03:21:41.051 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 37/134 2023-07-18T03:21:41.052 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 38/134 2023-07-18T03:21:41.052 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-18T03:21:41.052 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-18T03:21:41.052 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-18T03:21:41.052 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-18T03:21:41.052 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-18T03:21:41.052 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-18T03:21:41.053 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-18T03:21:41.053 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-18T03:21:41.053 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-18T03:21:41.053 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-18T03:21:41.053 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-18T03:21:41.053 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-18T03:21:41.053 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-18T03:21:41.054 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-18T03:21:41.054 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-18T03:21:41.054 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-18T03:21:41.054 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-18T03:21:41.054 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-18T03:21:41.054 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-18T03:21:41.055 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-18T03:21:41.055 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-18T03:21:41.055 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-18T03:21:41.055 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-18T03:21:41.056 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-18T03:21:41.056 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-18T03:21:41.056 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-18T03:21:41.056 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-18T03:21:41.056 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-18T03:21:41.056 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-18T03:21:41.057 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-18T03:21:41.057 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-18T03:21:41.057 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-18T03:21:41.057 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-18T03:21:41.057 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-18T03:21:41.057 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-18T03:21:41.057 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-18T03:21:41.057 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-18T03:21:41.058 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-18T03:21:41.058 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-18T03:21:41.058 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-18T03:21:41.058 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-18T03:21:41.058 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-18T03:21:41.058 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-18T03:21:41.059 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-18T03:21:41.059 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-18T03:21:41.059 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-18T03:21:41.059 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-18T03:21:41.059 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-18T03:21:41.060 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-18T03:21:41.060 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-18T03:21:41.060 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-18T03:21:41.060 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-18T03:21:41.060 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-18T03:21:41.060 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-18T03:21:41.060 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-18T03:21:41.061 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-18T03:21:41.061 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-18T03:21:41.061 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-18T03:21:41.061 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-18T03:21:41.061 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-18T03:21:41.061 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-18T03:21:41.061 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-18T03:21:41.062 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-18T03:21:41.062 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-18T03:21:41.062 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-18T03:21:41.062 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-18T03:21:41.062 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-18T03:21:41.062 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-18T03:21:41.062 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-18T03:21:41.062 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-18T03:21:41.063 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-18T03:21:41.063 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-18T03:21:41.063 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-18T03:21:41.063 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-18T03:21:41.063 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-18T03:21:41.063 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-18T03:21:41.063 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-18T03:21:41.063 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-18T03:21:41.064 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-18T03:21:41.064 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-18T03:21:41.064 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-18T03:21:41.064 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-18T03:21:41.064 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-18T03:21:41.064 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-18T03:21:41.064 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-18T03:21:41.065 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-18T03:21:41.065 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-18T03:21:41.065 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-18T03:21:41.065 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-18T03:21:41.065 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-18T03:21:41.065 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 129/134 2023-07-18T03:21:41.065 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-18T03:21:41.065 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 131/134 2023-07-18T03:21:41.066 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-18T03:21:41.066 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-18T03:21:43.459 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-18T03:21:43.460 INFO:teuthology.orchestra.run.smithi134.stdout:Last metadata expiration check: 0:02:24 ago on Tue 18 Jul 2023 03:19:19 AM UTC. 2023-07-18T03:21:45.990 INFO:teuthology.orchestra.run.smithi134.stdout:Installed products updated. 2023-07-18T03:21:46.201 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-18T03:21:46.202 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2023-07-18T03:21:46.202 INFO:teuthology.orchestra.run.smithi134.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-18T03:21:46.202 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.202 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.203 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2023-07-18T03:21:46.203 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-18T03:21:46.203 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.1-7.el8.noarch 2023-07-18T03:21:46.203 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.0.4-10.el8.x86_64 2023-07-18T03:21:46.203 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.203 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.203 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.204 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.204 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:46.204 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.204 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.204 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.204 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:46.205 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:46.205 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:46.205 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:46.205 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:46.205 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.205 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.205 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:46.206 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.206 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.206 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.206 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:46.206 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:46.206 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.206 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-18T03:21:46.207 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-18T03:21:46.207 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.1-9.el8.x86_64 2023-07-18T03:21:46.207 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-18T03:21:46.207 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-18T03:21:46.207 INFO:teuthology.orchestra.run.smithi134.stdout: jq-1.6-3.el8.x86_64 2023-07-18T03:21:46.207 INFO:teuthology.orchestra.run.smithi134.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-18T03:21:46.208 INFO:teuthology.orchestra.run.smithi134.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-18T03:21:46.208 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-18T03:21:46.208 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.208 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.208 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.208 INFO:teuthology.orchestra.run.smithi134.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-18T03:21:46.208 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-18T03:21:46.209 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-18T03:21:46.209 INFO:teuthology.orchestra.run.smithi134.stdout: libpq-13.5-1.el8.x86_64 2023-07-18T03:21:46.209 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-18T03:21:46.209 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-18T03:21:46.209 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.209 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.209 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-18T03:21:46.209 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.210 INFO:teuthology.orchestra.run.smithi134.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-18T03:21:46.210 INFO:teuthology.orchestra.run.smithi134.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-18T03:21:46.210 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-18T03:21:46.210 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-18T03:21:46.210 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-18T03:21:46.210 INFO:teuthology.orchestra.run.smithi134.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-18T03:21:46.210 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-18T03:21:46.211 INFO:teuthology.orchestra.run.smithi134.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-18T03:21:46.211 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-18T03:21:46.211 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-18T03:21:46.211 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-18T03:21:46.211 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-18T03:21:46.211 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-18T03:21:46.211 INFO:teuthology.orchestra.run.smithi134.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-18T03:21:46.211 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T03:21:46.212 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T03:21:46.212 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-18T03:21:46.212 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-18T03:21:46.212 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-18T03:21:46.212 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-18T03:21:46.212 INFO:teuthology.orchestra.run.smithi134.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-18T03:21:46.212 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-18T03:21:46.213 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.213 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.213 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.213 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-18T03:21:46.213 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-18T03:21:46.214 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-18T03:21:46.214 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-18T03:21:46.214 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-18T03:21:46.214 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-18T03:21:46.214 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-18T03:21:46.214 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-18T03:21:46.215 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-18T03:21:46.215 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-18T03:21:46.215 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-18T03:21:46.215 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-18T03:21:46.215 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-18T03:21:46.215 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-18T03:21:46.215 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-18T03:21:46.215 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-18T03:21:46.216 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-18T03:21:46.216 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-18T03:21:46.216 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-18T03:21:46.216 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-18T03:21:46.216 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-18T03:21:46.216 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-18T03:21:46.217 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-2.6-1.el8.noarch 2023-07-18T03:21:46.217 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-18T03:21:46.217 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-18T03:21:46.217 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-18T03:21:46.217 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-18T03:21:46.217 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-18T03:21:46.217 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-18T03:21:46.218 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.218 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.218 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-18T03:21:46.218 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-18T03:21:46.218 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.218 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-18T03:21:46.219 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-18T03:21:46.219 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-18T03:21:46.219 INFO:teuthology.orchestra.run.smithi134.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-18T03:21:46.219 INFO:teuthology.orchestra.run.smithi134.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-18T03:21:46.219 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-18T03:21:46.219 INFO:teuthology.orchestra.run.smithi134.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-18T03:21:46.219 INFO:teuthology.orchestra.run.smithi134.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-18T03:21:46.219 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-18T03:21:46.220 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-18T03:21:46.220 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-18T03:21:46.220 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-18T03:21:46.220 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-18T03:21:46.220 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.220 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.220 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-18T03:21:46.221 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-18T03:21:46.221 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-18T03:21:46.221 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-18T03:21:46.221 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-18T03:21:46.221 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-18T03:21:46.221 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-18T03:21:46.222 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-18T03:21:46.222 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2023-07-18T03:21:46.429 DEBUG:teuthology.parallel:result is None 2023-07-18T03:21:46.429 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:21:46.660 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-18T03:21:46.688 INFO:teuthology.orchestra.run.smithi040.stdout:18.0.0-4746.g81cf785a.el8 2023-07-18T03:21:46.689 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4746.g81cf785a.el8 2023-07-18T03:21:46.689 INFO:teuthology.task.install:The correct ceph version 18.0.0-4746.g81cf785a is installed. 2023-07-18T03:21:46.690 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:21:46.925 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-18T03:21:46.953 INFO:teuthology.orchestra.run.smithi064.stdout:18.0.0-4746.g81cf785a.el8 2023-07-18T03:21:46.954 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4746.g81cf785a.el8 2023-07-18T03:21:46.954 INFO:teuthology.task.install:The correct ceph version 18.0.0-4746.g81cf785a is installed. 2023-07-18T03:21:46.955 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:21:47.181 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-18T03:21:47.206 INFO:teuthology.orchestra.run.smithi134.stdout:18.0.0-4746.g81cf785a.el8 2023-07-18T03:21:47.206 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4746.g81cf785a.el8 2023-07-18T03:21:47.206 INFO:teuthology.task.install:The correct ceph version 18.0.0-4746.g81cf785a is installed. 2023-07-18T03:21:47.207 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-18T03:21:47.207 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:21:47.207 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-18T03:21:47.256 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-07-18T03:21:47.256 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-18T03:21:47.306 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-18T03:21:47.307 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-18T03:21:47.348 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-18T03:21:47.349 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:21:47.349 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2023-07-18T03:21:47.381 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-18T03:21:47.451 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-07-18T03:21:47.452 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2023-07-18T03:21:47.485 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-18T03:21:47.556 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-18T03:21:47.556 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2023-07-18T03:21:47.584 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-18T03:21:47.652 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-18T03:21:47.652 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:21:47.652 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-18T03:21:47.685 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-18T03:21:47.757 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-07-18T03:21:47.757 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-18T03:21:47.790 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-18T03:21:47.860 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-18T03:21:47.860 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-18T03:21:47.887 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-18T03:21:47.954 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-18T03:21:47.954 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:21:47.955 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2023-07-18T03:21:47.987 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-18T03:21:48.059 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-07-18T03:21:48.059 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2023-07-18T03:21:48.092 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-18T03:21:48.163 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-18T03:21:48.164 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2023-07-18T03:21:48.192 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-18T03:21:48.258 INFO:teuthology.run_tasks:Running task cephadm... 2023-07-18T03:21:48.339 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-07-18T03:21:48.339 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:21:48.350 INFO:tasks.cephadm:Cluster fsid is 3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:21:48.351 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-07-18T03:21:48.351 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.40', 'mon.b': '172.21.15.64', 'mon.c': '172.21.15.134'} 2023-07-18T03:21:48.351 INFO:tasks.cephadm:First mon is mon.a on smithi040 2023-07-18T03:21:48.352 INFO:tasks.cephadm:First mgr is x 2023-07-18T03:21:48.352 INFO:tasks.cephadm:Normalizing hostnames... 2023-07-18T03:21:48.353 DEBUG:teuthology.orchestra.run.smithi040:> sudo hostname $(hostname -s) 2023-07-18T03:21:48.387 DEBUG:teuthology.orchestra.run.smithi064:> sudo hostname $(hostname -s) 2023-07-18T03:21:48.420 DEBUG:teuthology.orchestra.run.smithi134:> sudo hostname $(hostname -s) 2023-07-18T03:21:48.448 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-07-18T03:21:48.449 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:21:48.680 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/71509/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-4746-g81cf785a', 'node_name': '172.21.2.5+braggi05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-4746.g81cf785a'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-07-18 00:38:12.274423', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-2023Jul13-m-quota', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-07-18T03:21:48.903 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-2023Jul13-m-quota, sha1 81cf785aa7781c4e08381c540f890cb9bc987ef9 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:21:48.903 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/x86_64/flavors/default/cephadm 2023-07-18T03:21:48.904 DEBUG:teuthology.orchestra.run.smithi040:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-18T03:21:49.608 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 18 03:21 /home/ubuntu/cephtest/cephadm 2023-07-18T03:21:49.609 DEBUG:teuthology.orchestra.run.smithi064:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-18T03:21:50.162 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 18 03:21 /home/ubuntu/cephtest/cephadm 2023-07-18T03:21:50.162 DEBUG:teuthology.orchestra.run.smithi134:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-18T03:21:50.718 INFO:teuthology.orchestra.run.smithi134.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 18 03:21 /home/ubuntu/cephtest/cephadm 2023-07-18T03:21:50.719 DEBUG:teuthology.orchestra.run.smithi040:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-18T03:21:50.741 DEBUG:teuthology.orchestra.run.smithi064:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-18T03:21:50.763 DEBUG:teuthology.orchestra.run.smithi134:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-18T03:21:50.798 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 on all hosts... 2023-07-18T03:21:50.799 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 pull 2023-07-18T03:21:50.802 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 pull 2023-07-18T03:21:50.805 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 pull 2023-07-18T03:21:52.037 INFO:teuthology.orchestra.run.smithi040.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-18T03:21:55.036 INFO:teuthology.orchestra.run.smithi064.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-18T03:21:56.635 INFO:teuthology.orchestra.run.smithi134.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-18T03:22:21.319 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2023-07-18T03:22:21.320 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_version": "ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev)", 2023-07-18T03:22:21.320 INFO:teuthology.orchestra.run.smithi134.stdout: "image_id": "66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421", 2023-07-18T03:22:21.320 INFO:teuthology.orchestra.run.smithi134.stdout: "repo_digests": [ 2023-07-18T03:22:21.320 INFO:teuthology.orchestra.run.smithi134.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b4b7dd677a0f5b8e56a57eadd762824d684980cb80d319543e86b72e549ef9c" 2023-07-18T03:22:21.320 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2023-07-18T03:22:21.321 INFO:teuthology.orchestra.run.smithi134.stdout:} 2023-07-18T03:22:21.825 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-07-18T03:22:21.825 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev)", 2023-07-18T03:22:21.825 INFO:teuthology.orchestra.run.smithi064.stdout: "image_id": "66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421", 2023-07-18T03:22:21.826 INFO:teuthology.orchestra.run.smithi064.stdout: "repo_digests": [ 2023-07-18T03:22:21.826 INFO:teuthology.orchestra.run.smithi064.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b4b7dd677a0f5b8e56a57eadd762824d684980cb80d319543e86b72e549ef9c" 2023-07-18T03:22:21.826 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2023-07-18T03:22:21.826 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-07-18T03:22:22.423 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:22:22.423 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev)", 2023-07-18T03:22:22.423 INFO:teuthology.orchestra.run.smithi040.stdout: "image_id": "66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421", 2023-07-18T03:22:22.424 INFO:teuthology.orchestra.run.smithi040.stdout: "repo_digests": [ 2023-07-18T03:22:22.424 INFO:teuthology.orchestra.run.smithi040.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b4b7dd677a0f5b8e56a57eadd762824d684980cb80d319543e86b72e549ef9c" 2023-07-18T03:22:22.424 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-07-18T03:22:22.424 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:22:22.443 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph 2023-07-18T03:22:22.474 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph 2023-07-18T03:22:22.513 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph 2023-07-18T03:22:22.551 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /etc/ceph 2023-07-18T03:22:22.578 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /etc/ceph 2023-07-18T03:22:22.609 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /etc/ceph 2023-07-18T03:22:22.641 INFO:tasks.cephadm:Writing seed config... 2023-07-18T03:22:22.643 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-07-18T03:22:22.644 INFO:tasks.cephadm: override: [client] debug client = 20 2023-07-18T03:22:22.644 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-07-18T03:22:22.644 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-07-18T03:22:22.645 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-07-18T03:22:22.645 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-07-18T03:22:22.645 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-07-18T03:22:22.645 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-07-18T03:22:22.646 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-07-18T03:22:22.646 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-07-18T03:22:22.646 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-07-18T03:22:22.646 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-07-18T03:22:22.646 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-07-18T03:22:22.647 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-07-18T03:22:22.647 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-07-18T03:22:22.647 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-07-18T03:22:22.647 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-07-18T03:22:22.647 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-07-18T03:22:22.648 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-07-18T03:22:22.648 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-07-18T03:22:22.648 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-07-18T03:22:22.648 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-07-18T03:22:22.648 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-07-18T03:22:22.649 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-07-18T03:22:22.649 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-07-18T03:22:22.649 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-07-18T03:22:22.649 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-07-18T03:22:22.649 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2023-07-18T03:22:22.650 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2023-07-18T03:22:22.650 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2023-07-18T03:22:22.650 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-07-18T03:22:22.650 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-07-18T03:22:22.650 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2023-07-18T03:22:22.651 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2023-07-18T03:22:22.651 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-07-18T03:22:22.651 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-07-18T03:22:22.651 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2023-07-18T03:22:22.651 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-07-18T03:22:22.652 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-07-18T03:22:22.652 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-07-18T03:22:22.652 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-07-18T03:22:22.652 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-07-18T03:22:22.652 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-07-18T03:22:22.653 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-07-18T03:22:22.653 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-07-18T03:22:22.654 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:22:22.654 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-07-18T03:22:22.671 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3af3dba8-251a-11ee-9b34-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-07-18T03:22:22.671 DEBUG:teuthology.orchestra.run.smithi040:mon.a> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mon.a.service 2023-07-18T03:22:22.713 DEBUG:teuthology.orchestra.run.smithi040:mgr.x> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mgr.x.service 2023-07-18T03:22:22.739 INFO:journalctl@ceph.mon.a.smithi040.stdout:-- Logs begin at Tue 2023-07-18 03:07:07 UTC. -- 2023-07-18T03:22:22.740 INFO:tasks.cephadm:Bootstrapping... 2023-07-18T03:22:22.741 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 -v bootstrap --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.40 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-07-18T03:22:22.766 INFO:journalctl@ceph.mgr.x.smithi040.stdout:-- Logs begin at Tue 2023-07-18 03:07:07 UTC. -- 2023-07-18T03:22:22.905 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-07-18T03:22:22.905 INFO:teuthology.orchestra.run.smithi040.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9', '-v', 'bootstrap', '--fsid', '3af3dba8-251a-11ee-9b34-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.40', '--skip-admin-label'] 2023-07-18T03:22:23.054 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stdout 4.1.1 2023-07-18T03:22:23.055 INFO:teuthology.orchestra.run.smithi040.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-07-18T03:22:23.055 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying podman|docker is present... 2023-07-18T03:22:23.200 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stdout 4.1.1 2023-07-18T03:22:23.201 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying lvm2 is present... 2023-07-18T03:22:23.201 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying time synchronization is in place... 2023-07-18T03:22:23.208 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-18T03:22:23.208 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-18T03:22:23.214 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-18T03:22:23.214 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2023-07-18T03:22:23.220 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout enabled 2023-07-18T03:22:23.226 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout active 2023-07-18T03:22:23.226 INFO:teuthology.orchestra.run.smithi040.stdout:Unit chronyd.service is enabled and running 2023-07-18T03:22:23.226 INFO:teuthology.orchestra.run.smithi040.stdout:Repeating the final host check... 2023-07-18T03:22:23.371 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stdout 4.1.1 2023-07-18T03:22:23.372 INFO:teuthology.orchestra.run.smithi040.stdout:podman (/bin/podman) version 4.1.1 is present 2023-07-18T03:22:23.372 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl is present 2023-07-18T03:22:23.372 INFO:teuthology.orchestra.run.smithi040.stdout:lvcreate is present 2023-07-18T03:22:23.378 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-18T03:22:23.378 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-18T03:22:23.384 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-18T03:22:23.384 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2023-07-18T03:22:23.390 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout enabled 2023-07-18T03:22:23.395 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout active 2023-07-18T03:22:23.396 INFO:teuthology.orchestra.run.smithi040.stdout:Unit chronyd.service is enabled and running 2023-07-18T03:22:23.396 INFO:teuthology.orchestra.run.smithi040.stdout:Host looks OK 2023-07-18T03:22:23.396 INFO:teuthology.orchestra.run.smithi040.stdout:Cluster fsid: 3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:22:23.397 INFO:teuthology.orchestra.run.smithi040.stdout:Acquiring lock 139874038196712 on /run/cephadm/3af3dba8-251a-11ee-9b34-001a4aab830c.lock 2023-07-18T03:22:23.397 INFO:teuthology.orchestra.run.smithi040.stdout:Lock 139874038196712 acquired on /run/cephadm/3af3dba8-251a-11ee-9b34-001a4aab830c.lock 2023-07-18T03:22:23.397 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying IP 172.21.15.40 port 3300 ... 2023-07-18T03:22:23.397 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying IP 172.21.15.40 port 6789 ... 2023-07-18T03:22:23.397 INFO:teuthology.orchestra.run.smithi040.stdout:Base mon IP(s) is [172.21.15.40:3300, 172.21.15.40:6789], mon addrv is [v2:172.21.15.40:3300,v1:172.21.15.40:6789] 2023-07-18T03:22:23.400 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-07-18T03:22:23.401 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.40 metric 100 2023-07-18T03:22:23.403 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-07-18T03:22:23.403 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-07-18T03:22:23.403 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1765sec hoplimit 64 pref medium 2023-07-18T03:22:23.406 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-07-18T03:22:23.406 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-07-18T03:22:23.406 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-18T03:22:23.406 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout 3: ens1f0: mtu 1500 state UP qlen 1000 2023-07-18T03:22:23.406 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:12cc/64 scope link 2023-07-18T03:22:23.406 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-18T03:22:23.407 INFO:teuthology.orchestra.run.smithi040.stdout:Mon IP `172.21.15.40` is in CIDR network `172.21.0.0/20` 2023-07-18T03:22:23.407 INFO:teuthology.orchestra.run.smithi040.stdout:Mon IP `172.21.15.40` is in CIDR network `172.21.0.0/20` 2023-07-18T03:22:23.407 INFO:teuthology.orchestra.run.smithi040.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-07-18T03:22:23.408 INFO:teuthology.orchestra.run.smithi040.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-07-18T03:22:23.409 INFO:teuthology.orchestra.run.smithi040.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-18T03:22:23.958 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stdout 66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421 2023-07-18T03:22:23.959 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-18T03:22:23.959 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Getting image source signatures 2023-07-18T03:22:23.959 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Copying blob sha256:711183bdaeabd1b922e81a2784cd69d11b05a304985585795e1ae029bf69712a 2023-07-18T03:22:23.959 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Copying blob sha256:6022c611d474f2b306ddd1edfc8360123ece5266119096f25fde843d7967e636 2023-07-18T03:22:23.959 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Copying config sha256:66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421 2023-07-18T03:22:23.960 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Writing manifest to image destination 2023-07-18T03:22:23.960 INFO:teuthology.orchestra.run.smithi040.stdout:/bin/podman: stderr Storing signatures 2023-07-18T03:22:24.812 INFO:teuthology.orchestra.run.smithi040.stdout:ceph: stdout ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev) 2023-07-18T03:22:24.812 INFO:teuthology.orchestra.run.smithi040.stdout:Ceph version: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev) 2023-07-18T03:22:24.812 INFO:teuthology.orchestra.run.smithi040.stdout:Extracting ceph user uid/gid from container image... 2023-07-18T03:22:25.564 INFO:teuthology.orchestra.run.smithi040.stdout:stat: stdout 167 167 2023-07-18T03:22:25.564 INFO:teuthology.orchestra.run.smithi040.stdout:Creating initial keys... 2023-07-18T03:22:26.414 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph-authtool: stdout AQBxBbZkd9ZMNxAAWt06JXV7OLYfVQA1urwFtg== 2023-07-18T03:22:27.161 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph-authtool: stdout AQByBbZkyZu8LhAACxFXteqhWyyym24e65r6BQ== 2023-07-18T03:22:27.905 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph-authtool: stdout AQBzBbZkUesnHhAANGnpIcHUkEdrHOj/rBteGQ== 2023-07-18T03:22:27.906 INFO:teuthology.orchestra.run.smithi040.stdout:Creating initial monmap... 2023-07-18T03:22:28.558 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-18T03:22:28.558 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-07-18T03:22:28.558 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:22:28.558 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-18T03:22:28.559 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool for a [v2:172.21.15.40:3300,v1:172.21.15.40:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-18T03:22:28.559 INFO:teuthology.orchestra.run.smithi040.stdout:setting min_mon_release = pacific 2023-07-18T03:22:28.559 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: set fsid to 3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:22:28.559 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-18T03:22:28.559 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:22:28.559 INFO:teuthology.orchestra.run.smithi040.stdout:Creating mon... 2023-07-18T03:22:29.305 INFO:teuthology.orchestra.run.smithi040.stdout:create mon.a on 2023-07-18T03:22:29.446 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-07-18T03:22:29.539 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-07-18T03:22:29.654 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3af3dba8-251a-11ee-9b34-001a4aab830c.target → /etc/systemd/system/ceph-3af3dba8-251a-11ee-9b34-001a4aab830c.target. 2023-07-18T03:22:29.654 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3af3dba8-251a-11ee-9b34-001a4aab830c.target → /etc/systemd/system/ceph-3af3dba8-251a-11ee-9b34-001a4aab830c.target. 2023-07-18T03:22:29.836 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mon.a 2023-07-18T03:22:29.837 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mon.a.service: Unit ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mon.a.service not loaded. 2023-07-18T03:22:29.929 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3af3dba8-251a-11ee-9b34-001a4aab830c.target.wants/ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@.service. 2023-07-18T03:22:30.235 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:29 smithi040 systemd[1]: Starting Ceph mon.a for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:22:30.527 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:30 smithi040 podman[95896]: 2023-07-18T03:22:30.527 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:30 smithi040 bash[95896]: 862743260e72bbccdc12535d3b5a03e72185130fea0095d6445f9370baf1ada5 2023-07-18T03:22:30.527 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:30 smithi040 ceph-mon[95936]: mkfs 3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:22:30.527 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:30 smithi040 ceph-mon[95936]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-18T03:22:30.539 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-18T03:22:30.539 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout disabled 2023-07-18T03:22:30.547 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-18T03:22:30.547 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2023-07-18T03:22:30.547 INFO:teuthology.orchestra.run.smithi040.stdout:firewalld.service is not enabled 2023-07-18T03:22:30.547 INFO:teuthology.orchestra.run.smithi040.stdout:Not possible to enable service . firewalld.service is not available 2023-07-18T03:22:30.548 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mon to start... 2023-07-18T03:22:30.548 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mon... 2023-07-18T03:22:30.796 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:30 smithi040 systemd[1]: Started Ceph mon.a for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:22:31.597 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout cluster: 2023-07-18T03:22:31.598 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout id: 3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:22:31.598 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-07-18T03:22:31.598 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2023-07-18T03:22:31.598 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout services: 2023-07-18T03:22:31.598 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.731697s) 2023-07-18T03:22:31.599 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-07-18T03:22:31.599 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-07-18T03:22:31.599 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2023-07-18T03:22:31.599 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout data: 2023-07-18T03:22:31.599 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-07-18T03:22:31.599 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-07-18T03:22:31.599 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-07-18T03:22:31.600 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout pgs: 2023-07-18T03:22:31.600 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2023-07-18T03:22:31.600 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.053+0000 7f95416c9700 1 Processor -- start 2023-07-18T03:22:31.600 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.053+0000 7f95416c9700 1 -- start start 2023-07-18T03:22:31.600 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.053+0000 7f95416c9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953c1053b0 0x7f953c1057c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:31.600 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.053+0000 7f95416c9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c105d00 con 0x7f953c1053b0 2023-07-18T03:22:31.600 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.054+0000 7f953affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953c1053b0 0x7f953c1057c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:31.601 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.054+0000 7f953affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953c1053b0 0x7f953c1057c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54758/0 (socket says 172.21.15.40:54758) 2023-07-18T03:22:31.601 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.054+0000 7f953affd700 1 -- 172.21.15.40:0/627161228 learned_addr learned my addr 172.21.15.40:0/627161228 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:31.601 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.054+0000 7f953affd700 1 -- 172.21.15.40:0/627161228 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c105e40 con 0x7f953c1053b0 2023-07-18T03:22:31.601 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.054+0000 7f953affd700 1 --2- 172.21.15.40:0/627161228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953c1053b0 0x7f953c1057c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f95300098a0 tx=0x7f9530009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8da1862698934f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:31.601 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.054+0000 7f9539ffb700 1 -- 172.21.15.40:0/627161228 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9530004160 con 0x7f953c1053b0 2023-07-18T03:22:31.602 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.054+0000 7f9539ffb700 1 -- 172.21.15.40:0/627161228 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95300042c0 con 0x7f953c1053b0 2023-07-18T03:22:31.602 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.054+0000 7f9539ffb700 1 -- 172.21.15.40:0/627161228 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95300045a0 con 0x7f953c1053b0 2023-07-18T03:22:31.602 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.055+0000 7f95416c9700 1 -- 172.21.15.40:0/627161228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953c1053b0 msgr2=0x7f953c1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:31.602 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.055+0000 7f95416c9700 1 --2- 172.21.15.40:0/627161228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953c1053b0 0x7f953c1057c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f95300098a0 tx=0x7f9530009bb0 comp rx=0 tx=0).stop 2023-07-18T03:22:31.602 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.055+0000 7f95416c9700 1 -- 172.21.15.40:0/627161228 shutdown_connections 2023-07-18T03:22:31.602 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.055+0000 7f95416c9700 1 --2- 172.21.15.40:0/627161228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953c1053b0 0x7f953c1057c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:31.603 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.055+0000 7f95416c9700 1 -- 172.21.15.40:0/627161228 >> 172.21.15.40:0/627161228 conn(0x7f953c100a20 msgr2=0x7f953c102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:31.603 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.055+0000 7f95416c9700 1 -- 172.21.15.40:0/627161228 shutdown_connections 2023-07-18T03:22:31.603 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.055+0000 7f95416c9700 1 -- 172.21.15.40:0/627161228 wait complete. 2023-07-18T03:22:31.603 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.055+0000 7f95416c9700 1 Processor -- start 2023-07-18T03:22:31.603 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.056+0000 7f95416c9700 1 -- start start 2023-07-18T03:22:31.603 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.056+0000 7f95416c9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953c1053b0 0x7f953c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:31.603 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.056+0000 7f95416c9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c198e30 con 0x7f953c1053b0 2023-07-18T03:22:31.604 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.056+0000 7f953affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953c1053b0 0x7f953c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:31.604 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.056+0000 7f953affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953c1053b0 0x7f953c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54760/0 (socket says 172.21.15.40:54760) 2023-07-18T03:22:31.604 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.056+0000 7f953affd700 1 -- 172.21.15.40:0/1993516604 learned_addr learned my addr 172.21.15.40:0/1993516604 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:31.604 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.056+0000 7f953affd700 1 -- 172.21.15.40:0/1993516604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9530009550 con 0x7f953c1053b0 2023-07-18T03:22:31.604 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.056+0000 7f953affd700 1 --2- 172.21.15.40:0/1993516604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953c1053b0 0x7f953c1988f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f953c1060e0 tx=0x7f9530004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:31.604 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.057+0000 7f9523fff700 1 -- 172.21.15.40:0/1993516604 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9530011630 con 0x7f953c1053b0 2023-07-18T03:22:31.604 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.057+0000 7f9523fff700 1 -- 172.21.15.40:0/1993516604 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9530011790 con 0x7f953c1053b0 2023-07-18T03:22:31.604 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.057+0000 7f95416c9700 1 -- 172.21.15.40:0/1993516604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953c199030 con 0x7f953c1053b0 2023-07-18T03:22:31.605 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.057+0000 7f9523fff700 1 -- 172.21.15.40:0/1993516604 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9530011a30 con 0x7f953c1053b0 2023-07-18T03:22:31.605 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.057+0000 7f95416c9700 1 -- 172.21.15.40:0/1993516604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f953c199450 con 0x7f953c1053b0 2023-07-18T03:22:31.605 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.057+0000 7f9523fff700 1 -- 172.21.15.40:0/1993516604 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9530022070 con 0x7f953c1053b0 2023-07-18T03:22:31.605 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.057+0000 7f9523fff700 1 -- 172.21.15.40:0/1993516604 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f953001c450 con 0x7f953c1053b0 2023-07-18T03:22:31.605 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.058+0000 7f95416c9700 1 -- 172.21.15.40:0/1993516604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f953c1920d0 con 0x7f953c1053b0 2023-07-18T03:22:31.605 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.060+0000 7f9523fff700 1 -- 172.21.15.40:0/1993516604 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f953001b450 con 0x7f953c1053b0 2023-07-18T03:22:31.605 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.191+0000 7f95416c9700 1 -- 172.21.15.40:0/1993516604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f953c02c850 con 0x7f953c1053b0 2023-07-18T03:22:31.606 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.191+0000 7f9523fff700 1 -- 172.21.15.40:0/1993516604 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f953001b5f0 con 0x7f953c1053b0 2023-07-18T03:22:31.606 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.192+0000 7f95416c9700 1 -- 172.21.15.40:0/1993516604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953c1053b0 msgr2=0x7f953c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:31.606 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.192+0000 7f95416c9700 1 --2- 172.21.15.40:0/1993516604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953c1053b0 0x7f953c1988f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f953c1060e0 tx=0x7f9530004750 comp rx=0 tx=0).stop 2023-07-18T03:22:31.606 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.192+0000 7f95416c9700 1 -- 172.21.15.40:0/1993516604 shutdown_connections 2023-07-18T03:22:31.606 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.192+0000 7f95416c9700 1 --2- 172.21.15.40:0/1993516604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f953c1053b0 0x7f953c1988f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:31.606 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.192+0000 7f95416c9700 1 -- 172.21.15.40:0/1993516604 >> 172.21.15.40:0/1993516604 conn(0x7f953c100a20 msgr2=0x7f953c18ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:31.606 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.192+0000 7f95416c9700 1 -- 172.21.15.40:0/1993516604 shutdown_connections 2023-07-18T03:22:31.606 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:31.192+0000 7f95416c9700 1 -- 172.21.15.40:0/1993516604 wait complete. 2023-07-18T03:22:31.607 INFO:teuthology.orchestra.run.smithi040.stdout:mon is available 2023-07-18T03:22:31.607 INFO:teuthology.orchestra.run.smithi040.stdout:Assimilating anything we can from ceph.conf... 2023-07-18T03:22:31.804 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:31 smithi040 ceph-mon[95936]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-18T03:22:31.805 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:31 smithi040 ceph-mon[95936]: monmap e1: 1 mons at {a=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0]} removed_ranks: {} 2023-07-18T03:22:31.805 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:31 smithi040 ceph-mon[95936]: fsmap 2023-07-18T03:22:31.805 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:31 smithi040 ceph-mon[95936]: osdmap e1: 0 total, 0 up, 0 in 2023-07-18T03:22:31.805 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:31 smithi040 ceph-mon[95936]: mgrmap e1: no daemons active 2023-07-18T03:22:31.805 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:31 smithi040 ceph-mon[95936]: from='client.? 172.21.15.40:0/1993516604' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-07-18T03:22:32.544 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2023-07-18T03:22:32.545 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [global] 2023-07-18T03:22:32.545 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout fsid = 3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:22:32.545 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.40:3300,v1:172.21.15.40:6789] 2023-07-18T03:22:32.545 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-18T03:22:32.545 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-18T03:22:32.545 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-18T03:22:32.546 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-18T03:22:32.546 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2023-07-18T03:22:32.546 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-18T03:22:32.546 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-18T03:22:32.546 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2023-07-18T03:22:32.546 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [osd] 2023-07-18T03:22:32.547 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-18T03:22:32.547 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-18T03:22:32.547 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-18T03:22:32.547 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-18T03:22:32.547 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-18T03:22:32.547 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.089+0000 7ff21bf2e700 1 Processor -- start 2023-07-18T03:22:32.547 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.089+0000 7ff21bf2e700 1 -- start start 2023-07-18T03:22:32.548 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.089+0000 7ff21bf2e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff214105e90 0x7ff214108270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:32.548 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.089+0000 7ff21bf2e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff214073c20 con 0x7ff214105e90 2023-07-18T03:22:32.548 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.090+0000 7ff219cca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff214105e90 0x7ff214108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:32.548 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.090+0000 7ff219cca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff214105e90 0x7ff214108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54762/0 (socket says 172.21.15.40:54762) 2023-07-18T03:22:32.548 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.090+0000 7ff219cca700 1 -- 172.21.15.40:0/443664382 learned_addr learned my addr 172.21.15.40:0/443664382 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:32.548 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.090+0000 7ff219cca700 1 -- 172.21.15.40:0/443664382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff214073d60 con 0x7ff214105e90 2023-07-18T03:22:32.548 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.091+0000 7ff219cca700 1 --2- 172.21.15.40:0/443664382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff214105e90 0x7ff214108270 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff2040098a0 tx=0x7ff204009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56e1691c7630a17 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:32.549 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.091+0000 7ff218cc8700 1 -- 172.21.15.40:0/443664382 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff204004160 con 0x7ff214105e90 2023-07-18T03:22:32.549 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.091+0000 7ff218cc8700 1 -- 172.21.15.40:0/443664382 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2040042c0 con 0x7ff214105e90 2023-07-18T03:22:32.549 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.091+0000 7ff218cc8700 1 -- 172.21.15.40:0/443664382 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2040045a0 con 0x7ff214105e90 2023-07-18T03:22:32.549 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.091+0000 7ff21bf2e700 1 -- 172.21.15.40:0/443664382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff214105e90 msgr2=0x7ff214108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:32.549 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.091+0000 7ff21bf2e700 1 --2- 172.21.15.40:0/443664382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff214105e90 0x7ff214108270 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff2040098a0 tx=0x7ff204009bb0 comp rx=0 tx=0).stop 2023-07-18T03:22:32.549 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.091+0000 7ff21bf2e700 1 -- 172.21.15.40:0/443664382 shutdown_connections 2023-07-18T03:22:32.549 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.091+0000 7ff21bf2e700 1 --2- 172.21.15.40:0/443664382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff214105e90 0x7ff214108270 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:32.550 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.091+0000 7ff21bf2e700 1 -- 172.21.15.40:0/443664382 >> 172.21.15.40:0/443664382 conn(0x7ff2140ff8d0 msgr2=0x7ff214101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:32.550 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.092+0000 7ff21bf2e700 1 -- 172.21.15.40:0/443664382 shutdown_connections 2023-07-18T03:22:32.550 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.092+0000 7ff21bf2e700 1 -- 172.21.15.40:0/443664382 wait complete. 2023-07-18T03:22:32.550 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.092+0000 7ff21bf2e700 1 Processor -- start 2023-07-18T03:22:32.550 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.092+0000 7ff21bf2e700 1 -- start start 2023-07-18T03:22:32.550 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.092+0000 7ff21bf2e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff214105e90 0x7ff2141988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:32.550 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.092+0000 7ff21bf2e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff214198e10 con 0x7ff214105e90 2023-07-18T03:22:32.550 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.092+0000 7ff219cca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff214105e90 0x7ff2141988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:32.551 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.092+0000 7ff219cca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff214105e90 0x7ff2141988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54764/0 (socket says 172.21.15.40:54764) 2023-07-18T03:22:32.551 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.092+0000 7ff219cca700 1 -- 172.21.15.40:0/3809503648 learned_addr learned my addr 172.21.15.40:0/3809503648 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:32.551 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.093+0000 7ff219cca700 1 -- 172.21.15.40:0/3809503648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff204009550 con 0x7ff214105e90 2023-07-18T03:22:32.551 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.093+0000 7ff219cca700 1 --2- 172.21.15.40:0/3809503648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff214105e90 0x7ff2141988d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff204004000 tx=0x7ff204004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:32.551 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.093+0000 7ff20affd700 1 -- 172.21.15.40:0/3809503648 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff204011630 con 0x7ff214105e90 2023-07-18T03:22:32.551 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.093+0000 7ff20affd700 1 -- 172.21.15.40:0/3809503648 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff204011790 con 0x7ff214105e90 2023-07-18T03:22:32.551 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.093+0000 7ff20affd700 1 -- 172.21.15.40:0/3809503648 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff204011a30 con 0x7ff214105e90 2023-07-18T03:22:32.552 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.093+0000 7ff21bf2e700 1 -- 172.21.15.40:0/3809503648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff214199010 con 0x7ff214105e90 2023-07-18T03:22:32.552 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.093+0000 7ff21bf2e700 1 -- 172.21.15.40:0/3809503648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff214199430 con 0x7ff214105e90 2023-07-18T03:22:32.552 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.093+0000 7ff20affd700 1 -- 172.21.15.40:0/3809503648 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff204022070 con 0x7ff214105e90 2023-07-18T03:22:32.552 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.093+0000 7ff20affd700 1 -- 172.21.15.40:0/3809503648 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff20401c450 con 0x7ff214105e90 2023-07-18T03:22:32.552 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.094+0000 7ff21bf2e700 1 -- 172.21.15.40:0/3809503648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff214192660 con 0x7ff214105e90 2023-07-18T03:22:32.552 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.096+0000 7ff20affd700 1 -- 172.21.15.40:0/3809503648 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7ff20401b450 con 0x7ff214105e90 2023-07-18T03:22:32.552 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.222+0000 7ff21bf2e700 1 -- 172.21.15.40:0/3809503648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff21402cad0 con 0x7ff214105e90 2023-07-18T03:22:32.553 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.226+0000 7ff20affd700 1 -- 172.21.15.40:0/3809503648 <== mon.0 v2:172.21.15.40:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff20401b5f0 con 0x7ff214105e90 2023-07-18T03:22:32.553 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.226+0000 7ff20affd700 1 -- 172.21.15.40:0/3809503648 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7ff204024630 con 0x7ff214105e90 2023-07-18T03:22:32.553 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.227+0000 7ff21bf2e700 1 -- 172.21.15.40:0/3809503648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff214105e90 msgr2=0x7ff2141988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:32.553 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.227+0000 7ff21bf2e700 1 --2- 172.21.15.40:0/3809503648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff214105e90 0x7ff2141988d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff204004000 tx=0x7ff204004750 comp rx=0 tx=0).stop 2023-07-18T03:22:32.553 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.227+0000 7ff21bf2e700 1 -- 172.21.15.40:0/3809503648 shutdown_connections 2023-07-18T03:22:32.553 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.227+0000 7ff21bf2e700 1 --2- 172.21.15.40:0/3809503648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff214105e90 0x7ff2141988d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:32.553 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.227+0000 7ff21bf2e700 1 -- 172.21.15.40:0/3809503648 >> 172.21.15.40:0/3809503648 conn(0x7ff2140ff8d0 msgr2=0x7ff214101240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:32.554 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.227+0000 7ff21bf2e700 1 -- 172.21.15.40:0/3809503648 shutdown_connections 2023-07-18T03:22:32.554 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:32.227+0000 7ff21bf2e700 1 -- 172.21.15.40:0/3809503648 wait complete. 2023-07-18T03:22:32.554 INFO:teuthology.orchestra.run.smithi040.stdout:Generating new minimal ceph.conf... 2023-07-18T03:22:32.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:32 smithi040 ceph-mon[95936]: from='client.? 172.21.15.40:0/3809503648' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-18T03:22:32.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:32 smithi040 ceph-mon[95936]: from='client.? 172.21.15.40:0/3809503648' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-07-18T03:22:33.487 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.020+0000 7f43b6cbd700 1 Processor -- start 2023-07-18T03:22:33.487 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.020+0000 7f43b6cbd700 1 -- start start 2023-07-18T03:22:33.488 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.020+0000 7f43b6cbd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b0104270 0x7f43b0104680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:33.488 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.020+0000 7f43b6cbd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b0104bc0 con 0x7f43b0104270 2023-07-18T03:22:33.488 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.021+0000 7f43b4a59700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b0104270 0x7f43b0104680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:33.488 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.021+0000 7f43b4a59700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b0104270 0x7f43b0104680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54766/0 (socket says 172.21.15.40:54766) 2023-07-18T03:22:33.488 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.021+0000 7f43b4a59700 1 -- 172.21.15.40:0/2670341835 learned_addr learned my addr 172.21.15.40:0/2670341835 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:33.488 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.021+0000 7f43b4a59700 1 -- 172.21.15.40:0/2670341835 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b0104d00 con 0x7f43b0104270 2023-07-18T03:22:33.488 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.021+0000 7f43b4a59700 1 --2- 172.21.15.40:0/2670341835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b0104270 0x7f43b0104680 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f43a400bb40 tx=0x7f43a400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835de128035cdfdb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:33.489 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.021+0000 7f43af7fe700 1 -- 172.21.15.40:0/2670341835 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43a400dbe0 con 0x7f43b0104270 2023-07-18T03:22:33.489 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.021+0000 7f43af7fe700 1 -- 172.21.15.40:0/2670341835 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f43a40044d0 con 0x7f43b0104270 2023-07-18T03:22:33.489 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.021+0000 7f43af7fe700 1 -- 172.21.15.40:0/2670341835 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43a4010470 con 0x7f43b0104270 2023-07-18T03:22:33.489 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.022+0000 7f43b6cbd700 1 -- 172.21.15.40:0/2670341835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b0104270 msgr2=0x7f43b0104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:33.489 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.022+0000 7f43b6cbd700 1 --2- 172.21.15.40:0/2670341835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b0104270 0x7f43b0104680 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f43a400bb40 tx=0x7f43a400be50 comp rx=0 tx=0).stop 2023-07-18T03:22:33.489 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.022+0000 7f43b6cbd700 1 -- 172.21.15.40:0/2670341835 shutdown_connections 2023-07-18T03:22:33.489 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.022+0000 7f43b6cbd700 1 --2- 172.21.15.40:0/2670341835 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b0104270 0x7f43b0104680 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:33.489 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.022+0000 7f43b6cbd700 1 -- 172.21.15.40:0/2670341835 >> 172.21.15.40:0/2670341835 conn(0x7f43b00ff900 msgr2=0x7f43b0101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:33.490 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.022+0000 7f43b6cbd700 1 -- 172.21.15.40:0/2670341835 shutdown_connections 2023-07-18T03:22:33.490 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.022+0000 7f43b6cbd700 1 -- 172.21.15.40:0/2670341835 wait complete. 2023-07-18T03:22:33.490 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.022+0000 7f43b6cbd700 1 Processor -- start 2023-07-18T03:22:33.490 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.023+0000 7f43b6cbd700 1 -- start start 2023-07-18T03:22:33.490 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.023+0000 7f43b6cbd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b0104270 0x7f43b019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:33.490 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.023+0000 7f43b6cbd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b019b870 con 0x7f43b0104270 2023-07-18T03:22:33.490 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.023+0000 7f43b4a59700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b0104270 0x7f43b019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:33.490 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.023+0000 7f43b4a59700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b0104270 0x7f43b019b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54768/0 (socket says 172.21.15.40:54768) 2023-07-18T03:22:33.491 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.023+0000 7f43b4a59700 1 -- 172.21.15.40:0/632203548 learned_addr learned my addr 172.21.15.40:0/632203548 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:33.491 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.023+0000 7f43b4a59700 1 -- 172.21.15.40:0/632203548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a400b7f0 con 0x7f43b0104270 2023-07-18T03:22:33.491 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.023+0000 7f43b4a59700 1 --2- 172.21.15.40:0/632203548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b0104270 0x7f43b019b330 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f43a4003b20 tx=0x7f43a4003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:33.491 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.023+0000 7f43adffb700 1 -- 172.21.15.40:0/632203548 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43a4004340 con 0x7f43b0104270 2023-07-18T03:22:33.491 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.023+0000 7f43adffb700 1 -- 172.21.15.40:0/632203548 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f43a4010b80 con 0x7f43b0104270 2023-07-18T03:22:33.491 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.023+0000 7f43b6cbd700 1 -- 172.21.15.40:0/632203548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b019ba70 con 0x7f43b0104270 2023-07-18T03:22:33.491 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.023+0000 7f43adffb700 1 -- 172.21.15.40:0/632203548 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43a4021bf0 con 0x7f43b0104270 2023-07-18T03:22:33.492 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.024+0000 7f43b6cbd700 1 -- 172.21.15.40:0/632203548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b019be30 con 0x7f43b0104270 2023-07-18T03:22:33.492 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.024+0000 7f43adffb700 1 -- 172.21.15.40:0/632203548 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f43a402a6f0 con 0x7f43b0104270 2023-07-18T03:22:33.492 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.024+0000 7f43adffb700 1 -- 172.21.15.40:0/632203548 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f43a4019740 con 0x7f43b0104270 2023-07-18T03:22:33.492 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.025+0000 7f43b6cbd700 1 -- 172.21.15.40:0/632203548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b0194fb0 con 0x7f43b0104270 2023-07-18T03:22:33.492 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.027+0000 7f43adffb700 1 -- 172.21.15.40:0/632203548 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f43a402f060 con 0x7f43b0104270 2023-07-18T03:22:33.492 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.153+0000 7f43b6cbd700 1 -- 172.21.15.40:0/632203548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f43b0106160 con 0x7f43b0104270 2023-07-18T03:22:33.492 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.153+0000 7f43adffb700 1 -- 172.21.15.40:0/632203548 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f43a4010cf0 con 0x7f43b0104270 2023-07-18T03:22:33.492 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.154+0000 7f43b6cbd700 1 -- 172.21.15.40:0/632203548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b0104270 msgr2=0x7f43b019b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:33.493 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.154+0000 7f43b6cbd700 1 --2- 172.21.15.40:0/632203548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b0104270 0x7f43b019b330 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f43a4003b20 tx=0x7f43a4003980 comp rx=0 tx=0).stop 2023-07-18T03:22:33.493 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.154+0000 7f43b6cbd700 1 -- 172.21.15.40:0/632203548 shutdown_connections 2023-07-18T03:22:33.493 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.154+0000 7f43b6cbd700 1 --2- 172.21.15.40:0/632203548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43b0104270 0x7f43b019b330 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:33.493 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.154+0000 7f43b6cbd700 1 -- 172.21.15.40:0/632203548 >> 172.21.15.40:0/632203548 conn(0x7f43b00ff900 msgr2=0x7f43b01920f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:33.493 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.154+0000 7f43b6cbd700 1 -- 172.21.15.40:0/632203548 shutdown_connections 2023-07-18T03:22:33.493 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:33.154+0000 7f43b6cbd700 1 -- 172.21.15.40:0/632203548 wait complete. 2023-07-18T03:22:33.493 INFO:teuthology.orchestra.run.smithi040.stdout:Restarting the monitor... 2023-07-18T03:22:33.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:33 smithi040 ceph-mon[95936]: from='client.? 172.21.15.40:0/632203548' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:22:33.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:33 smithi040 systemd[1]: Stopping Ceph mon.a for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:22:34.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:34 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mon-a[95912]: 2023-07-18T03:22:34.014+0000 7f39fec3e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:22:34.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:34 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mon-a[95912]: 2023-07-18T03:22:34.014+0000 7f39fec3e700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-07-18T03:22:35.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:35 smithi040 bash[96275]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mon-a 2023-07-18T03:22:35.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:35 smithi040 systemd[1]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mon.a.service: Succeeded. 2023-07-18T03:22:35.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:35 smithi040 systemd[1]: Stopped Ceph mon.a for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:22:35.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:35 smithi040 systemd[1]: Starting Ceph mon.a for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:22:36.557 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 podman[96399]: 2023-07-18T03:22:36.557 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 bash[96399]: c90877839340ffea3a30c63fcdbe39da2893f689b12b9f4a0e434225fcb01cf6 2023-07-18T03:22:36.557 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: set uid:gid to 167:167 (ceph:ceph) 2023-07-18T03:22:36.557 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev), process ceph-mon, pid 7 2023-07-18T03:22:36.558 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: pidfile_write: ignore empty --pid-file 2023-07-18T03:22:36.558 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: load: jerasure load: lrc 2023-07-18T03:22:36.558 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: RocksDB version: 7.9.2 2023-07-18T03:22:36.558 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Git sha 0 2023-07-18T03:22:36.558 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Compile date 2023-07-17 23:00:14 2023-07-18T03:22:36.558 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: DB SUMMARY 2023-07-18T03:22:36.559 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: DB Session ID: UWFS70JZ0KK16YXOK0FL 2023-07-18T03:22:36.559 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: CURRENT file: CURRENT 2023-07-18T03:22:36.559 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: IDENTITY file: IDENTITY 2023-07-18T03:22:36.559 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-07-18T03:22:36.559 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-07-18T03:22:36.559 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99047 ; 2023-07-18T03:22:36.559 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.error_if_exists: 0 2023-07-18T03:22:36.559 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.create_if_missing: 0 2023-07-18T03:22:36.560 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.paranoid_checks: 1 2023-07-18T03:22:36.560 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-18T03:22:36.560 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-18T03:22:36.560 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-18T03:22:36.560 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.env: 0x557e366b1720 2023-07-18T03:22:36.561 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.fs: PosixFileSystem 2023-07-18T03:22:36.561 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.info_log: 0x557e37927280 2023-07-18T03:22:36.561 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_file_opening_threads: 16 2023-07-18T03:22:36.561 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.statistics: (nil) 2023-07-18T03:22:36.561 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.use_fsync: 0 2023-07-18T03:22:36.561 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_log_file_size: 0 2023-07-18T03:22:36.561 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-18T03:22:36.562 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-18T03:22:36.562 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.keep_log_file_num: 1000 2023-07-18T03:22:36.562 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.recycle_log_file_num: 0 2023-07-18T03:22:36.562 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.allow_fallocate: 1 2023-07-18T03:22:36.562 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.allow_mmap_reads: 0 2023-07-18T03:22:36.562 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.allow_mmap_writes: 0 2023-07-18T03:22:36.562 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.use_direct_reads: 0 2023-07-18T03:22:36.563 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-18T03:22:36.563 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.create_missing_column_families: 0 2023-07-18T03:22:36.563 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.db_log_dir: 2023-07-18T03:22:36.564 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.wal_dir: 2023-07-18T03:22:36.564 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-18T03:22:36.564 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-18T03:22:36.564 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-18T03:22:36.565 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-18T03:22:36.565 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-18T03:22:36.565 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-18T03:22:36.565 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.advise_random_on_open: 1 2023-07-18T03:22:36.565 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.db_write_buffer_size: 0 2023-07-18T03:22:36.565 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.write_buffer_manager: 0x557e378483c0 2023-07-18T03:22:36.565 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-18T03:22:36.566 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-18T03:22:36.566 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-18T03:22:36.566 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.rate_limiter: (nil) 2023-07-18T03:22:36.566 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-18T03:22:36.566 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.wal_recovery_mode: 2 2023-07-18T03:22:36.566 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.enable_thread_tracking: 0 2023-07-18T03:22:36.566 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.enable_pipelined_write: 0 2023-07-18T03:22:36.567 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.unordered_write: 0 2023-07-18T03:22:36.567 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-18T03:22:36.567 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-18T03:22:36.567 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-18T03:22:36.567 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-18T03:22:36.567 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.row_cache: None 2023-07-18T03:22:36.567 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.wal_filter: None 2023-07-18T03:22:36.568 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-18T03:22:36.568 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.allow_ingest_behind: 0 2023-07-18T03:22:36.568 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.two_write_queues: 0 2023-07-18T03:22:36.568 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.manual_wal_flush: 0 2023-07-18T03:22:36.568 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.wal_compression: 0 2023-07-18T03:22:36.568 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.atomic_flush: 0 2023-07-18T03:22:36.569 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-18T03:22:36.569 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-18T03:22:36.569 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-18T03:22:36.569 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.log_readahead_size: 0 2023-07-18T03:22:36.569 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-18T03:22:36.569 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.best_efforts_recovery: 0 2023-07-18T03:22:36.569 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-18T03:22:36.569 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-18T03:22:36.570 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.allow_data_in_errors: 0 2023-07-18T03:22:36.570 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.db_host_id: __hostname__ 2023-07-18T03:22:36.570 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-18T03:22:36.570 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_background_jobs: 2 2023-07-18T03:22:36.570 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_background_compactions: -1 2023-07-18T03:22:36.570 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_subcompactions: 1 2023-07-18T03:22:36.570 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-18T03:22:36.570 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-18T03:22:36.571 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-18T03:22:36.571 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_total_wal_size: 0 2023-07-18T03:22:36.571 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-18T03:22:36.571 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-18T03:22:36.571 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-18T03:22:36.571 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-18T03:22:36.572 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_open_files: -1 2023-07-18T03:22:36.572 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.bytes_per_sync: 0 2023-07-18T03:22:36.572 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-18T03:22:36.572 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-18T03:22:36.572 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compaction_readahead_size: 0 2023-07-18T03:22:36.572 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_background_flushes: -1 2023-07-18T03:22:36.573 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Compression algorithms supported: 2023-07-18T03:22:36.573 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-18T03:22:36.573 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: kZSTD supported: 0 2023-07-18T03:22:36.573 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: kXpressCompression supported: 0 2023-07-18T03:22:36.573 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: kLZ4HCCompression supported: 1 2023-07-18T03:22:36.573 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: kZlibCompression supported: 1 2023-07-18T03:22:36.573 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: kSnappyCompression supported: 1 2023-07-18T03:22:36.573 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: kLZ4Compression supported: 1 2023-07-18T03:22:36.574 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: kBZip2Compression supported: 0 2023-07-18T03:22:36.574 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-18T03:22:36.574 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-18T03:22:36.574 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-07-18T03:22:36.574 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-18T03:22:36.574 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-18T03:22:36.574 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.merge_operator: 2023-07-18T03:22:36.574 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compaction_filter: None 2023-07-18T03:22:36.575 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compaction_filter_factory: None 2023-07-18T03:22:36.575 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.sst_partitioner_factory: None 2023-07-18T03:22:36.575 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-18T03:22:36.575 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-18T03:22:36.575 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557e37927380) 2023-07-18T03:22:36.575 INFO:journalctl@ceph.mon.a.smithi040.stdout: cache_index_and_filter_blocks: 1 2023-07-18T03:22:36.575 INFO:journalctl@ceph.mon.a.smithi040.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-18T03:22:36.576 INFO:journalctl@ceph.mon.a.smithi040.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-18T03:22:36.576 INFO:journalctl@ceph.mon.a.smithi040.stdout: pin_top_level_index_and_filter: 1 2023-07-18T03:22:36.576 INFO:journalctl@ceph.mon.a.smithi040.stdout: index_type: 0 2023-07-18T03:22:36.576 INFO:journalctl@ceph.mon.a.smithi040.stdout: data_block_index_type: 0 2023-07-18T03:22:36.576 INFO:journalctl@ceph.mon.a.smithi040.stdout: index_shortening: 1 2023-07-18T03:22:36.576 INFO:journalctl@ceph.mon.a.smithi040.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-18T03:22:36.576 INFO:journalctl@ceph.mon.a.smithi040.stdout: checksum: 4 2023-07-18T03:22:36.577 INFO:journalctl@ceph.mon.a.smithi040.stdout: no_block_cache: 0 2023-07-18T03:22:36.577 INFO:journalctl@ceph.mon.a.smithi040.stdout: block_cache: 0x557e378e71f0 2023-07-18T03:22:36.577 INFO:journalctl@ceph.mon.a.smithi040.stdout: block_cache_name: BinnedLRUCache 2023-07-18T03:22:36.577 INFO:journalctl@ceph.mon.a.smithi040.stdout: block_cache_options: 2023-07-18T03:22:36.577 INFO:journalctl@ceph.mon.a.smithi040.stdout: capacity : 536870912 2023-07-18T03:22:36.577 INFO:journalctl@ceph.mon.a.smithi040.stdout: num_shard_bits : 4 2023-07-18T03:22:36.577 INFO:journalctl@ceph.mon.a.smithi040.stdout: strict_capacity_limit : 0 2023-07-18T03:22:36.577 INFO:journalctl@ceph.mon.a.smithi040.stdout: high_pri_pool_ratio: 0.000 2023-07-18T03:22:36.578 INFO:journalctl@ceph.mon.a.smithi040.stdout: block_cache_compressed: (nil) 2023-07-18T03:22:36.578 INFO:journalctl@ceph.mon.a.smithi040.stdout: persistent_cache: (nil) 2023-07-18T03:22:36.578 INFO:journalctl@ceph.mon.a.smithi040.stdout: block_size: 4096 2023-07-18T03:22:36.578 INFO:journalctl@ceph.mon.a.smithi040.stdout: block_size_deviation: 10 2023-07-18T03:22:36.578 INFO:journalctl@ceph.mon.a.smithi040.stdout: block_restart_interval: 16 2023-07-18T03:22:36.578 INFO:journalctl@ceph.mon.a.smithi040.stdout: index_block_restart_interval: 1 2023-07-18T03:22:36.578 INFO:journalctl@ceph.mon.a.smithi040.stdout: metadata_block_size: 4096 2023-07-18T03:22:36.579 INFO:journalctl@ceph.mon.a.smithi040.stdout: partition_filters: 0 2023-07-18T03:22:36.579 INFO:journalctl@ceph.mon.a.smithi040.stdout: use_delta_encoding: 1 2023-07-18T03:22:36.579 INFO:journalctl@ceph.mon.a.smithi040.stdout: filter_policy: bloomfilter 2023-07-18T03:22:36.579 INFO:journalctl@ceph.mon.a.smithi040.stdout: whole_key_filtering: 1 2023-07-18T03:22:36.579 INFO:journalctl@ceph.mon.a.smithi040.stdout: verify_compression: 0 2023-07-18T03:22:36.579 INFO:journalctl@ceph.mon.a.smithi040.stdout: read_amp_bytes_per_bit: 0 2023-07-18T03:22:36.579 INFO:journalctl@ceph.mon.a.smithi040.stdout: format_version: 5 2023-07-18T03:22:36.580 INFO:journalctl@ceph.mon.a.smithi040.stdout: enable_index_compression: 1 2023-07-18T03:22:36.580 INFO:journalctl@ceph.mon.a.smithi040.stdout: block_align: 0 2023-07-18T03:22:36.580 INFO:journalctl@ceph.mon.a.smithi040.stdout: max_auto_readahead_size: 262144 2023-07-18T03:22:36.580 INFO:journalctl@ceph.mon.a.smithi040.stdout: prepopulate_block_cache: 0 2023-07-18T03:22:36.580 INFO:journalctl@ceph.mon.a.smithi040.stdout: initial_auto_readahead_size: 8192 2023-07-18T03:22:36.580 INFO:journalctl@ceph.mon.a.smithi040.stdout: num_file_reads_for_auto_readahead: 2 2023-07-18T03:22:36.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.write_buffer_size: 33554432 2023-07-18T03:22:36.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_write_buffer_number: 2 2023-07-18T03:22:36.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compression: NoCompression 2023-07-18T03:22:36.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.bottommost_compression: Disabled 2023-07-18T03:22:36.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.prefix_extractor: nullptr 2023-07-18T03:22:36.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-18T03:22:36.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.num_levels: 7 2023-07-18T03:22:36.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-18T03:22:36.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-18T03:22:36.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-18T03:22:36.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-18T03:22:36.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-18T03:22:36.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-18T03:22:36.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-18T03:22:36.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-18T03:22:36.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-18T03:22:36.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-18T03:22:36.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-18T03:22:36.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-18T03:22:36.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-18T03:22:36.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compression_opts.level: 32767 2023-07-18T03:22:36.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compression_opts.strategy: 0 2023-07-18T03:22:36.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-18T03:22:36.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-18T03:22:36.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-18T03:22:36.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-18T03:22:36.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compression_opts.enabled: false 2023-07-18T03:22:36.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-18T03:22:36.585 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-18T03:22:36.585 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-18T03:22:36.585 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-18T03:22:36.585 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.target_file_size_base: 67108864 2023-07-18T03:22:36.585 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-18T03:22:36.585 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-18T03:22:36.586 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-18T03:22:36.586 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-18T03:22:36.586 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-18T03:22:36.586 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-18T03:22:36.586 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-18T03:22:36.586 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-18T03:22:36.586 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-18T03:22:36.587 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-18T03:22:36.587 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-18T03:22:36.587 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-18T03:22:36.587 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-18T03:22:36.587 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-18T03:22:36.587 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.arena_block_size: 1048576 2023-07-18T03:22:36.587 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-18T03:22:36.587 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-18T03:22:36.588 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.disable_auto_compactions: 0 2023-07-18T03:22:36.588 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-18T03:22:36.588 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-18T03:22:36.588 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-18T03:22:36.588 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-18T03:22:36.588 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-18T03:22:36.588 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-18T03:22:36.589 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-18T03:22:36.589 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-18T03:22:36.589 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-18T03:22:36.589 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-18T03:22:36.589 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.table_properties_collectors: 2023-07-18T03:22:36.589 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.inplace_update_support: 0 2023-07-18T03:22:36.589 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-18T03:22:36.590 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-18T03:22:36.590 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-18T03:22:36.590 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-18T03:22:36.590 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.bloom_locality: 0 2023-07-18T03:22:36.590 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.max_successive_merges: 0 2023-07-18T03:22:36.590 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-18T03:22:36.590 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.paranoid_file_checks: 0 2023-07-18T03:22:36.590 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.force_consistency_checks: 1 2023-07-18T03:22:36.591 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.report_bg_io_stats: 0 2023-07-18T03:22:36.591 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.ttl: 2592000 2023-07-18T03:22:36.591 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-18T03:22:36.591 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-18T03:22:36.591 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-18T03:22:36.591 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.enable_blob_files: false 2023-07-18T03:22:36.591 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.min_blob_size: 0 2023-07-18T03:22:36.591 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.blob_file_size: 268435456 2023-07-18T03:22:36.592 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-18T03:22:36.592 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-18T03:22:36.592 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-18T03:22:36.592 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-18T03:22:36.592 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-18T03:22:36.593 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.blob_file_starting_level: 0 2023-07-18T03:22:36.593 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-18T03:22:36.593 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-07-18T03:22:36.593 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-07-18T03:22:36.593 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 952625f0-2349-434d-af5e-03863d7339b8 2023-07-18T03:22:36.593 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689650556486258, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-07-18T03:22:36.593 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-07-18T03:22:36.593 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689650556487411, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94266, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 92264, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13747, "raw_average_key_size": 49, "raw_value_size": 84761, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689650556, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "952625f0-2349-434d-af5e-03863d7339b8", "db_session_id": "UWFS70JZ0KK16YXOK0FL", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-07-18T03:22:36.594 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689650556487497, "job": 1, "event": "recovery_finished"} 2023-07-18T03:22:36.594 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-07-18T03:22:36.594 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-18T03:22:36.594 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557e37980000 2023-07-18T03:22:36.594 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: DB pointer 0x557e3796c000 2023-07-18T03:22:36.594 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-18T03:22:36.595 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-18T03:22:36.595 INFO:journalctl@ceph.mon.a.smithi040.stdout: ** DB Stats ** 2023-07-18T03:22:36.595 INFO:journalctl@ceph.mon.a.smithi040.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-18T03:22:36.595 INFO:journalctl@ceph.mon.a.smithi040.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-18T03:22:36.595 INFO:journalctl@ceph.mon.a.smithi040.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-18T03:22:36.595 INFO:journalctl@ceph.mon.a.smithi040.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-18T03:22:36.596 INFO:journalctl@ceph.mon.a.smithi040.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-18T03:22:36.596 INFO:journalctl@ceph.mon.a.smithi040.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-18T03:22:36.596 INFO:journalctl@ceph.mon.a.smithi040.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-18T03:22:36.596 INFO:journalctl@ceph.mon.a.smithi040.stdout: 2023-07-18T03:22:36.596 INFO:journalctl@ceph.mon.a.smithi040.stdout: ** Compaction Stats [default] ** 2023-07-18T03:22:36.596 INFO:journalctl@ceph.mon.a.smithi040.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-18T03:22:36.597 INFO:journalctl@ceph.mon.a.smithi040.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-18T03:22:36.597 INFO:journalctl@ceph.mon.a.smithi040.stdout: L0 2/0 93.89 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-18T03:22:36.597 INFO:journalctl@ceph.mon.a.smithi040.stdout: Sum 2/0 93.89 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-18T03:22:36.597 INFO:journalctl@ceph.mon.a.smithi040.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-18T03:22:36.597 INFO:journalctl@ceph.mon.a.smithi040.stdout: 2023-07-18T03:22:36.597 INFO:journalctl@ceph.mon.a.smithi040.stdout: ** Compaction Stats [default] ** 2023-07-18T03:22:36.597 INFO:journalctl@ceph.mon.a.smithi040.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-18T03:22:36.598 INFO:journalctl@ceph.mon.a.smithi040.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-18T03:22:36.598 INFO:journalctl@ceph.mon.a.smithi040.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 127.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-18T03:22:36.598 INFO:journalctl@ceph.mon.a.smithi040.stdout: 2023-07-18T03:22:36.598 INFO:journalctl@ceph.mon.a.smithi040.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-18T03:22:36.598 INFO:journalctl@ceph.mon.a.smithi040.stdout: 2023-07-18T03:22:36.598 INFO:journalctl@ceph.mon.a.smithi040.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-18T03:22:36.598 INFO:journalctl@ceph.mon.a.smithi040.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-18T03:22:36.598 INFO:journalctl@ceph.mon.a.smithi040.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-18T03:22:36.599 INFO:journalctl@ceph.mon.a.smithi040.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-18T03:22:36.599 INFO:journalctl@ceph.mon.a.smithi040.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-18T03:22:36.599 INFO:journalctl@ceph.mon.a.smithi040.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-18T03:22:36.599 INFO:journalctl@ceph.mon.a.smithi040.stdout: Cumulative compaction: 0.00 GB write, 25.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-18T03:22:36.599 INFO:journalctl@ceph.mon.a.smithi040.stdout: Interval compaction: 0.00 GB write, 25.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-18T03:22:36.599 INFO:journalctl@ceph.mon.a.smithi040.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-18T03:22:36.599 INFO:journalctl@ceph.mon.a.smithi040.stdout: Block cache BinnedLRUCache@0x557e378e71f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-07-18T03:22:36.599 INFO:journalctl@ceph.mon.a.smithi040.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-07-18T03:22:36.600 INFO:journalctl@ceph.mon.a.smithi040.stdout: 2023-07-18T03:22:36.600 INFO:journalctl@ceph.mon.a.smithi040.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-18T03:22:36.600 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: starting mon.a rank 0 at public addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] at bind addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:22:36.600 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: mon.a@-1(???) e1 preinit fsid 3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:22:36.600 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: mon.a@-1(???).mds e1 new map 2023-07-18T03:22:36.601 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: mon.a@-1(???).mds e1 print_map 2023-07-18T03:22:36.601 INFO:journalctl@ceph.mon.a.smithi040.stdout: e1 2023-07-18T03:22:36.601 INFO:journalctl@ceph.mon.a.smithi040.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-18T03:22:36.601 INFO:journalctl@ceph.mon.a.smithi040.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-18T03:22:36.601 INFO:journalctl@ceph.mon.a.smithi040.stdout: legacy client fscid: -1 2023-07-18T03:22:36.601 INFO:journalctl@ceph.mon.a.smithi040.stdout: 2023-07-18T03:22:36.601 INFO:journalctl@ceph.mon.a.smithi040.stdout: No filesystems configured 2023-07-18T03:22:36.602 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-18T03:22:36.602 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T03:22:36.602 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T03:22:36.602 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T03:22:36.602 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-07-18T03:22:36.602 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: expand_channel_meta expand map: {default=false} 2023-07-18T03:22:36.603 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: expand_channel_meta from 'false' to 'false' 2023-07-18T03:22:36.603 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: expand_channel_meta expanded map: {default=false} 2023-07-18T03:22:36.603 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: expand_channel_meta expand map: {default=info} 2023-07-18T03:22:36.603 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: expand_channel_meta from 'info' to 'info' 2023-07-18T03:22:36.603 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: expand_channel_meta expanded map: {default=info} 2023-07-18T03:22:36.603 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: expand_channel_meta expand map: {default=daemon} 2023-07-18T03:22:36.603 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-18T03:22:36.604 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: expand_channel_meta expanded map: {default=daemon} 2023-07-18T03:22:36.604 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: expand_channel_meta expand map: {default=debug} 2023-07-18T03:22:36.604 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: expand_channel_meta from 'debug' to 'debug' 2023-07-18T03:22:36.604 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: expand_channel_meta expanded map: {default=debug} 2023-07-18T03:22:36.604 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-18T03:22:36.604 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: monmap e1: 1 mons at {a=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0]} removed_ranks: {} 2023-07-18T03:22:36.604 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: fsmap 2023-07-18T03:22:36.605 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: osdmap e1: 0 total, 0 up, 0 in 2023-07-18T03:22:36.605 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 ceph-mon[96439]: mgrmap e1: no daemons active 2023-07-18T03:22:36.605 INFO:teuthology.orchestra.run.smithi040.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-07-18T03:22:36.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:36 smithi040 systemd[1]: Started Ceph mon.a for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:22:37.732 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.217+0000 7f5dd6d13700 1 Processor -- start 2023-07-18T03:22:37.732 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.217+0000 7f5dd6d13700 1 -- start start 2023-07-18T03:22:37.732 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.218+0000 7f5dd6d13700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd0103ca0 0x7f5dd01040b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:37.732 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.218+0000 7f5dd6d13700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd0079d20 con 0x7f5dd0103ca0 2023-07-18T03:22:37.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.218+0000 7f5dd4aaf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd0103ca0 0x7f5dd01040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:37.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.218+0000 7f5dd4aaf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd0103ca0 0x7f5dd01040b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54770/0 (socket says 172.21.15.40:54770) 2023-07-18T03:22:37.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.218+0000 7f5dd4aaf700 1 -- 172.21.15.40:0/2827932657 learned_addr learned my addr 172.21.15.40:0/2827932657 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:37.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.218+0000 7f5dd4aaf700 1 -- 172.21.15.40:0/2827932657 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd0079e60 con 0x7f5dd0103ca0 2023-07-18T03:22:37.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.219+0000 7f5dd4aaf700 1 --2- 172.21.15.40:0/2827932657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd0103ca0 0x7f5dd01040b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40098a0 tx=0x7f5dc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1e53d8c3ad0bc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:37.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.219+0000 7f5dcf7fe700 1 -- 172.21.15.40:0/2827932657 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5dc4004160 con 0x7f5dd0103ca0 2023-07-18T03:22:37.733 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.219+0000 7f5dcf7fe700 1 -- 172.21.15.40:0/2827932657 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5dc40042c0 con 0x7f5dd0103ca0 2023-07-18T03:22:37.734 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.219+0000 7f5dcf7fe700 1 -- 172.21.15.40:0/2827932657 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5dc4011a00 con 0x7f5dd0103ca0 2023-07-18T03:22:37.734 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.219+0000 7f5dd6d13700 1 -- 172.21.15.40:0/2827932657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd0103ca0 msgr2=0x7f5dd01040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:37.734 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.219+0000 7f5dd6d13700 1 --2- 172.21.15.40:0/2827932657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd0103ca0 0x7f5dd01040b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40098a0 tx=0x7f5dc4009bb0 comp rx=0 tx=0).stop 2023-07-18T03:22:37.734 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.219+0000 7f5dd6d13700 1 -- 172.21.15.40:0/2827932657 shutdown_connections 2023-07-18T03:22:37.734 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.219+0000 7f5dd6d13700 1 --2- 172.21.15.40:0/2827932657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd0103ca0 0x7f5dd01040b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:37.734 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.220+0000 7f5dd6d13700 1 -- 172.21.15.40:0/2827932657 >> 172.21.15.40:0/2827932657 conn(0x7f5dd00ff8d0 msgr2=0x7f5dd0101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:37.734 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.220+0000 7f5dd6d13700 1 -- 172.21.15.40:0/2827932657 shutdown_connections 2023-07-18T03:22:37.735 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.220+0000 7f5dd6d13700 1 -- 172.21.15.40:0/2827932657 wait complete. 2023-07-18T03:22:37.735 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.220+0000 7f5dd6d13700 1 Processor -- start 2023-07-18T03:22:37.735 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.220+0000 7f5dd6d13700 1 -- start start 2023-07-18T03:22:37.735 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.220+0000 7f5dd6d13700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd0103ca0 0x7f5dd019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:37.735 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.220+0000 7f5dd6d13700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd019b7d0 con 0x7f5dd0103ca0 2023-07-18T03:22:37.735 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.221+0000 7f5dd4aaf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd0103ca0 0x7f5dd019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:37.735 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.221+0000 7f5dd4aaf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd0103ca0 0x7f5dd019b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54772/0 (socket says 172.21.15.40:54772) 2023-07-18T03:22:37.736 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.221+0000 7f5dd4aaf700 1 -- 172.21.15.40:0/3000262076 learned_addr learned my addr 172.21.15.40:0/3000262076 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:37.736 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.221+0000 7f5dd4aaf700 1 -- 172.21.15.40:0/3000262076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc4009550 con 0x7f5dd0103ca0 2023-07-18T03:22:37.736 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.221+0000 7f5dd4aaf700 1 --2- 172.21.15.40:0/3000262076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd0103ca0 0x7f5dd019b290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4006af0 tx=0x7f5dc4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:37.736 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.221+0000 7f5dcdffb700 1 -- 172.21.15.40:0/3000262076 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5dc4003a40 con 0x7f5dd0103ca0 2023-07-18T03:22:37.736 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.221+0000 7f5dcdffb700 1 -- 172.21.15.40:0/3000262076 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5dc4003ba0 con 0x7f5dd0103ca0 2023-07-18T03:22:37.736 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.221+0000 7f5dd6d13700 1 -- 172.21.15.40:0/3000262076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd019ba30 con 0x7f5dd0103ca0 2023-07-18T03:22:37.736 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.221+0000 7f5dcdffb700 1 -- 172.21.15.40:0/3000262076 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5dc401a700 con 0x7f5dd0103ca0 2023-07-18T03:22:37.737 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.221+0000 7f5dd6d13700 1 -- 172.21.15.40:0/3000262076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd019beb0 con 0x7f5dd0103ca0 2023-07-18T03:22:37.737 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.222+0000 7f5dcdffb700 1 -- 172.21.15.40:0/3000262076 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5dc401ac30 con 0x7f5dd0103ca0 2023-07-18T03:22:37.737 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.222+0000 7f5dcdffb700 1 -- 172.21.15.40:0/3000262076 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5dc4019700 con 0x7f5dd0103ca0 2023-07-18T03:22:37.737 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.222+0000 7f5dd6d13700 1 -- 172.21.15.40:0/3000262076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db4005320 con 0x7f5dd0103ca0 2023-07-18T03:22:37.737 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.225+0000 7f5dcdffb700 1 -- 172.21.15.40:0/3000262076 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f5dc402ddf0 con 0x7f5dd0103ca0 2023-07-18T03:22:37.737 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.351+0000 7f5dd6d13700 1 -- 172.21.15.40:0/3000262076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f5db4005cc0 con 0x7f5dd0103ca0 2023-07-18T03:22:37.737 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.353+0000 7f5dcdffb700 1 -- 172.21.15.40:0/3000262076 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5dc40198c0 con 0x7f5dd0103ca0 2023-07-18T03:22:37.738 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.354+0000 7f5dd6d13700 1 -- 172.21.15.40:0/3000262076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd0103ca0 msgr2=0x7f5dd019b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:37.738 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.354+0000 7f5dd6d13700 1 --2- 172.21.15.40:0/3000262076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd0103ca0 0x7f5dd019b290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4006af0 tx=0x7f5dc4011ad0 comp rx=0 tx=0).stop 2023-07-18T03:22:37.738 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.354+0000 7f5dd6d13700 1 -- 172.21.15.40:0/3000262076 shutdown_connections 2023-07-18T03:22:37.738 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.354+0000 7f5dd6d13700 1 --2- 172.21.15.40:0/3000262076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd0103ca0 0x7f5dd019b290 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:37.738 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.354+0000 7f5dd6d13700 1 -- 172.21.15.40:0/3000262076 >> 172.21.15.40:0/3000262076 conn(0x7f5dd00ff8d0 msgr2=0x7f5dd018f860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:37.738 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.354+0000 7f5dd6d13700 1 -- 172.21.15.40:0/3000262076 shutdown_connections 2023-07-18T03:22:37.738 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:37.354+0000 7f5dd6d13700 1 -- 172.21.15.40:0/3000262076 wait complete. 2023-07-18T03:22:37.738 INFO:teuthology.orchestra.run.smithi040.stdout:Wrote config to /etc/ceph/ceph.conf 2023-07-18T03:22:37.744 INFO:teuthology.orchestra.run.smithi040.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-07-18T03:22:37.744 INFO:teuthology.orchestra.run.smithi040.stdout:Creating mgr... 2023-07-18T03:22:37.744 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying port 9283 ... 2023-07-18T03:22:37.745 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying port 8765 ... 2023-07-18T03:22:37.923 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mgr.x 2023-07-18T03:22:37.923 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mgr.x.service: Unit ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mgr.x.service not loaded. 2023-07-18T03:22:38.019 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3af3dba8-251a-11ee-9b34-001a4aab830c.target.wants/ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@.service. 2023-07-18T03:22:38.078 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:38 smithi040 systemd[1]: Starting Ceph mgr.x for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:22:38.711 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:38 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3000262076' entity='client.admin' 2023-07-18T03:22:38.712 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:38 smithi040 podman[96670]: 2023-07-18T03:22:38.828 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-18T03:22:38.829 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout disabled 2023-07-18T03:22:38.836 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-18T03:22:38.836 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2023-07-18T03:22:38.837 INFO:teuthology.orchestra.run.smithi040.stdout:firewalld.service is not enabled 2023-07-18T03:22:38.837 INFO:teuthology.orchestra.run.smithi040.stdout:Not possible to enable service . firewalld.service is not available 2023-07-18T03:22:38.846 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-18T03:22:38.846 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout disabled 2023-07-18T03:22:38.854 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-18T03:22:38.854 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2023-07-18T03:22:38.855 INFO:teuthology.orchestra.run.smithi040.stdout:firewalld.service is not enabled 2023-07-18T03:22:38.855 INFO:teuthology.orchestra.run.smithi040.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-07-18T03:22:38.855 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mgr to start... 2023-07-18T03:22:38.855 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mgr... 2023-07-18T03:22:38.967 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:38 smithi040 bash[96670]: 91c5a2cd4a9955cfa721cb23d296773c4dbf6fca4a1ee89ce0a35951afce927c 2023-07-18T03:22:38.967 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:38 smithi040 systemd[1]: Started Ceph mgr.x for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:22:38.967 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:38 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:38.861+0000 7fa9f17a2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-18T03:22:39.328 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:38 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:38.970+0000 7fa9f17a2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-18T03:22:39.329 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:39 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:39.081+0000 7fa9f17a2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-18T03:22:39.329 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:39 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:39.199+0000 7fa9f17a2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-18T03:22:39.737 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:39 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2528181934' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-18T03:22:39.738 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:39 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:39.408+0000 7fa9f17a2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-18T03:22:40.023 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2023-07-18T03:22:40.024 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2023-07-18T03:22:40.024 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsid": "3af3dba8-251a-11ee-9b34-001a4aab830c", 2023-07-18T03:22:40.024 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "health": { 2023-07-18T03:22:40.025 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-18T03:22:40.025 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-18T03:22:40.025 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-18T03:22:40.025 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:40.025 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-18T03:22:40.025 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-18T03:22:40.025 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 0 2023-07-18T03:22:40.026 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2023-07-18T03:22:40.026 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-18T03:22:40.026 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "a" 2023-07-18T03:22:40.026 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2023-07-18T03:22:40.026 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-07-18T03:22:40.026 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-18T03:22:40.026 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T03:22:40.026 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-18T03:22:40.027 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-18T03:22:40.027 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:40.027 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-18T03:22:40.027 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T03:22:40.027 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-18T03:22:40.027 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-18T03:22:40.027 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-18T03:22:40.027 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-18T03:22:40.028 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-18T03:22:40.028 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-18T03:22:40.028 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:40.028 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-18T03:22:40.028 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-18T03:22:40.028 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-18T03:22:40.028 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-18T03:22:40.029 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-18T03:22:40.029 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-18T03:22:40.029 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-18T03:22:40.029 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-18T03:22:40.029 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-18T03:22:40.029 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:40.029 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-18T03:22:40.029 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T03:22:40.030 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-18T03:22:40.030 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-18T03:22:40.030 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:40.030 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-18T03:22:40.030 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-18T03:22:40.030 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-18T03:22:40.030 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-18T03:22:40.031 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-18T03:22:40.031 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-18T03:22:40.031 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "restful" 2023-07-18T03:22:40.031 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2023-07-18T03:22:40.031 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-18T03:22:40.031 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:40.031 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-18T03:22:40.031 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T03:22:40.032 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modified": "2023-07-18T03:22:30.461984+0000", 2023-07-18T03:22:40.032 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-18T03:22:40.032 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:40.032 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-18T03:22:40.032 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2023-07-18T03:22:40.032 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.414+0000 7f17e8ab2700 1 Processor -- start 2023-07-18T03:22:40.033 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.414+0000 7f17e8ab2700 1 -- start start 2023-07-18T03:22:40.033 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.415+0000 7f17e8ab2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e407acd0 0x7f17e40791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:40.033 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.415+0000 7f17e8ab2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e4079710 con 0x7f17e407acd0 2023-07-18T03:22:40.033 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.415+0000 7f17e259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e407acd0 0x7f17e40791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:40.033 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.415+0000 7f17e259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e407acd0 0x7f17e40791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54778/0 (socket says 172.21.15.40:54778) 2023-07-18T03:22:40.033 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.415+0000 7f17e259c700 1 -- 172.21.15.40:0/2277970430 learned_addr learned my addr 172.21.15.40:0/2277970430 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:40.033 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.415+0000 7f17e259c700 1 -- 172.21.15.40:0/2277970430 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e4079850 con 0x7f17e407acd0 2023-07-18T03:22:40.033 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.415+0000 7f17e259c700 1 --2- 172.21.15.40:0/2277970430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e407acd0 0x7f17e40791d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f17d40098a0 tx=0x7f17d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4bfdb25ccd0d18 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:40.034 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.416+0000 7f17e159a700 1 -- 172.21.15.40:0/2277970430 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17d4004160 con 0x7f17e407acd0 2023-07-18T03:22:40.034 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.416+0000 7f17e159a700 1 -- 172.21.15.40:0/2277970430 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f17d40042c0 con 0x7f17e407acd0 2023-07-18T03:22:40.034 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.416+0000 7f17e159a700 1 -- 172.21.15.40:0/2277970430 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17d4011a00 con 0x7f17e407acd0 2023-07-18T03:22:40.034 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.416+0000 7f17e8ab2700 1 -- 172.21.15.40:0/2277970430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e407acd0 msgr2=0x7f17e40791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:40.034 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.416+0000 7f17e8ab2700 1 --2- 172.21.15.40:0/2277970430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e407acd0 0x7f17e40791d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f17d40098a0 tx=0x7f17d4009bb0 comp rx=0 tx=0).stop 2023-07-18T03:22:40.034 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.416+0000 7f17e8ab2700 1 -- 172.21.15.40:0/2277970430 shutdown_connections 2023-07-18T03:22:40.034 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.416+0000 7f17e8ab2700 1 --2- 172.21.15.40:0/2277970430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e407acd0 0x7f17e40791d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:40.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.416+0000 7f17e8ab2700 1 -- 172.21.15.40:0/2277970430 >> 172.21.15.40:0/2277970430 conn(0x7f17e41009f0 msgr2=0x7f17e4102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:40.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.417+0000 7f17e8ab2700 1 -- 172.21.15.40:0/2277970430 shutdown_connections 2023-07-18T03:22:40.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.417+0000 7f17e8ab2700 1 -- 172.21.15.40:0/2277970430 wait complete. 2023-07-18T03:22:40.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.417+0000 7f17e8ab2700 1 Processor -- start 2023-07-18T03:22:40.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.417+0000 7f17e8ab2700 1 -- start start 2023-07-18T03:22:40.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.418+0000 7f17e8ab2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e407acd0 0x7f17e419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:40.035 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.418+0000 7f17e8ab2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e419b8b0 con 0x7f17e407acd0 2023-07-18T03:22:40.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.418+0000 7f17e259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e407acd0 0x7f17e419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:40.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.418+0000 7f17e259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e407acd0 0x7f17e419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54780/0 (socket says 172.21.15.40:54780) 2023-07-18T03:22:40.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.418+0000 7f17e259c700 1 -- 172.21.15.40:0/2528181934 learned_addr learned my addr 172.21.15.40:0/2528181934 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:40.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.418+0000 7f17e259c700 1 -- 172.21.15.40:0/2528181934 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d4009550 con 0x7f17e407acd0 2023-07-18T03:22:40.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.418+0000 7f17e259c700 1 --2- 172.21.15.40:0/2528181934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e407acd0 0x7f17e419b370 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f17d4006b20 tx=0x7f17d4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:40.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.418+0000 7f17d37fe700 1 -- 172.21.15.40:0/2528181934 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17d4003b70 con 0x7f17e407acd0 2023-07-18T03:22:40.036 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.418+0000 7f17d37fe700 1 -- 172.21.15.40:0/2528181934 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f17d4003cd0 con 0x7f17e407acd0 2023-07-18T03:22:40.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.418+0000 7f17e8ab2700 1 -- 172.21.15.40:0/2528181934 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e419bab0 con 0x7f17e407acd0 2023-07-18T03:22:40.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.418+0000 7f17d37fe700 1 -- 172.21.15.40:0/2528181934 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17d40247c0 con 0x7f17e407acd0 2023-07-18T03:22:40.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.419+0000 7f17e8ab2700 1 -- 172.21.15.40:0/2528181934 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e419bed0 con 0x7f17e407acd0 2023-07-18T03:22:40.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.419+0000 7f17d37fe700 1 -- 172.21.15.40:0/2528181934 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f17d40193c0 con 0x7f17e407acd0 2023-07-18T03:22:40.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.419+0000 7f17d37fe700 1 -- 172.21.15.40:0/2528181934 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f17d4019670 con 0x7f17e407acd0 2023-07-18T03:22:40.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.420+0000 7f17e8ab2700 1 -- 172.21.15.40:0/2528181934 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e4061b40 con 0x7f17e407acd0 2023-07-18T03:22:40.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.422+0000 7f17d37fe700 1 -- 172.21.15.40:0/2528181934 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f17d401ab40 con 0x7f17e407acd0 2023-07-18T03:22:40.037 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.553+0000 7f17e8ab2700 1 -- 172.21.15.40:0/2528181934 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f17e419e910 con 0x7f17e407acd0 2023-07-18T03:22:40.038 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.553+0000 7f17d37fe700 1 -- 172.21.15.40:0/2528181934 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f17d401a430 con 0x7f17e407acd0 2023-07-18T03:22:40.038 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.554+0000 7f17e8ab2700 1 -- 172.21.15.40:0/2528181934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e407acd0 msgr2=0x7f17e419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:40.038 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.554+0000 7f17e8ab2700 1 --2- 172.21.15.40:0/2528181934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e407acd0 0x7f17e419b370 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f17d4006b20 tx=0x7f17d4011ad0 comp rx=0 tx=0).stop 2023-07-18T03:22:40.038 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.554+0000 7f17e8ab2700 1 -- 172.21.15.40:0/2528181934 shutdown_connections 2023-07-18T03:22:40.038 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.554+0000 7f17e8ab2700 1 --2- 172.21.15.40:0/2528181934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17e407acd0 0x7f17e419b370 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:40.038 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.554+0000 7f17e8ab2700 1 -- 172.21.15.40:0/2528181934 >> 172.21.15.40:0/2528181934 conn(0x7f17e41009f0 msgr2=0x7f17e41020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:40.038 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.554+0000 7f17e8ab2700 1 -- 172.21.15.40:0/2528181934 shutdown_connections 2023-07-18T03:22:40.039 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:39.555+0000 7f17e8ab2700 1 -- 172.21.15.40:0/2528181934 wait complete. 2023-07-18T03:22:40.039 INFO:teuthology.orchestra.run.smithi040.stdout:mgr not available, waiting (1/15)... 2023-07-18T03:22:40.051 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:39 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:39.812+0000 7fa9f17a2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-18T03:22:40.051 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:39 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:39.921+0000 7fa9f17a2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-18T03:22:40.328 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:40 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:40.079+0000 7fa9f17a2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-18T03:22:40.328 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:40 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:40.193+0000 7fa9f17a2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-18T03:22:41.078 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:40 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:40.703+0000 7fa9f17a2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-18T03:22:41.828 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:41 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:41.409+0000 7fa9f17a2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-18T03:22:41.829 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:41 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:41.601+0000 7fa9f17a2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-18T03:22:42.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:42 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2234958939' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-18T03:22:42.829 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:42 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:42.574+0000 7fa9f17a2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:22:42.829 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:42 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:42.684+0000 7fa9f17a2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-18T03:22:43.071 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2023-07-18T03:22:43.071 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2023-07-18T03:22:43.072 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsid": "3af3dba8-251a-11ee-9b34-001a4aab830c", 2023-07-18T03:22:43.072 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "health": { 2023-07-18T03:22:43.072 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-18T03:22:43.073 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-18T03:22:43.073 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-18T03:22:43.073 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:43.073 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-18T03:22:43.074 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-18T03:22:43.074 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 0 2023-07-18T03:22:43.074 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2023-07-18T03:22:43.074 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-18T03:22:43.074 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "a" 2023-07-18T03:22:43.074 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2023-07-18T03:22:43.075 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-07-18T03:22:43.075 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-18T03:22:43.075 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T03:22:43.075 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-18T03:22:43.075 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-18T03:22:43.075 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:43.076 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-18T03:22:43.076 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T03:22:43.076 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-18T03:22:43.076 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-18T03:22:43.076 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-18T03:22:43.084 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-18T03:22:43.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-18T03:22:43.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-18T03:22:43.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:43.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-18T03:22:43.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-18T03:22:43.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-18T03:22:43.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-18T03:22:43.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-18T03:22:43.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-18T03:22:43.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-18T03:22:43.088 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-18T03:22:43.088 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-18T03:22:43.088 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:43.089 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-18T03:22:43.089 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T03:22:43.089 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-18T03:22:43.089 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-18T03:22:43.089 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:43.090 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-18T03:22:43.090 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-18T03:22:43.090 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-18T03:22:43.090 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-18T03:22:43.090 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-18T03:22:43.090 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-18T03:22:43.090 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "restful" 2023-07-18T03:22:43.091 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2023-07-18T03:22:43.091 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-18T03:22:43.091 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:43.091 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-18T03:22:43.091 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T03:22:43.091 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modified": "2023-07-18T03:22:30.461984+0000", 2023-07-18T03:22:43.091 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-18T03:22:43.092 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:43.092 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-18T03:22:43.092 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2023-07-18T03:22:43.092 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.561+0000 7f2f69dd7700 1 Processor -- start 2023-07-18T03:22:43.092 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.561+0000 7f2f69dd7700 1 -- start start 2023-07-18T03:22:43.092 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.561+0000 7f2f69dd7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f64104280 0x7f2f64104690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:43.093 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.561+0000 7f2f69dd7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f64104bd0 con 0x7f2f64104280 2023-07-18T03:22:43.093 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.561+0000 7f2f637fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f64104280 0x7f2f64104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:43.093 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.562+0000 7f2f637fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f64104280 0x7f2f64104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54782/0 (socket says 172.21.15.40:54782) 2023-07-18T03:22:43.093 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.562+0000 7f2f637fe700 1 -- 172.21.15.40:0/3424690346 learned_addr learned my addr 172.21.15.40:0/3424690346 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:43.093 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.562+0000 7f2f637fe700 1 -- 172.21.15.40:0/3424690346 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f64104d10 con 0x7f2f64104280 2023-07-18T03:22:43.093 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.562+0000 7f2f637fe700 1 --2- 172.21.15.40:0/3424690346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f64104280 0x7f2f64104690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f540098a0 tx=0x7f2f54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea25f39f1ab9c6f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:43.094 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.562+0000 7f2f627fc700 1 -- 172.21.15.40:0/3424690346 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f54004160 con 0x7f2f64104280 2023-07-18T03:22:43.094 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.562+0000 7f2f627fc700 1 -- 172.21.15.40:0/3424690346 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2f540042c0 con 0x7f2f64104280 2023-07-18T03:22:43.094 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.562+0000 7f2f627fc700 1 -- 172.21.15.40:0/3424690346 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f54011a00 con 0x7f2f64104280 2023-07-18T03:22:43.094 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.563+0000 7f2f69dd7700 1 -- 172.21.15.40:0/3424690346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f64104280 msgr2=0x7f2f64104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:43.094 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.563+0000 7f2f69dd7700 1 --2- 172.21.15.40:0/3424690346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f64104280 0x7f2f64104690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f540098a0 tx=0x7f2f54009bb0 comp rx=0 tx=0).stop 2023-07-18T03:22:43.095 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.563+0000 7f2f69dd7700 1 -- 172.21.15.40:0/3424690346 shutdown_connections 2023-07-18T03:22:43.095 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.563+0000 7f2f69dd7700 1 --2- 172.21.15.40:0/3424690346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f64104280 0x7f2f64104690 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:43.095 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.563+0000 7f2f69dd7700 1 -- 172.21.15.40:0/3424690346 >> 172.21.15.40:0/3424690346 conn(0x7f2f640ff8d0 msgr2=0x7f2f64101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:43.095 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.563+0000 7f2f69dd7700 1 -- 172.21.15.40:0/3424690346 shutdown_connections 2023-07-18T03:22:43.096 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.563+0000 7f2f69dd7700 1 -- 172.21.15.40:0/3424690346 wait complete. 2023-07-18T03:22:43.096 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.564+0000 7f2f69dd7700 1 Processor -- start 2023-07-18T03:22:43.096 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.564+0000 7f2f69dd7700 1 -- start start 2023-07-18T03:22:43.096 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.564+0000 7f2f69dd7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f64104280 0x7f2f641990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:43.096 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.564+0000 7f2f69dd7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f64199620 con 0x7f2f64104280 2023-07-18T03:22:43.096 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.564+0000 7f2f637fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f64104280 0x7f2f641990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:43.096 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.564+0000 7f2f637fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f64104280 0x7f2f641990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54784/0 (socket says 172.21.15.40:54784) 2023-07-18T03:22:43.097 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.564+0000 7f2f637fe700 1 -- 172.21.15.40:0/2234958939 learned_addr learned my addr 172.21.15.40:0/2234958939 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:43.097 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.564+0000 7f2f637fe700 1 -- 172.21.15.40:0/2234958939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f54009550 con 0x7f2f64104280 2023-07-18T03:22:43.097 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.565+0000 7f2f637fe700 1 --2- 172.21.15.40:0/2234958939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f64104280 0x7f2f641990e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2f54006af0 tx=0x7f2f54011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:43.097 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.565+0000 7f2f60ff9700 1 -- 172.21.15.40:0/2234958939 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f54003a40 con 0x7f2f64104280 2023-07-18T03:22:43.097 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.565+0000 7f2f60ff9700 1 -- 172.21.15.40:0/2234958939 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2f54003ba0 con 0x7f2f64104280 2023-07-18T03:22:43.097 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.565+0000 7f2f60ff9700 1 -- 172.21.15.40:0/2234958939 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f5401a6e0 con 0x7f2f64104280 2023-07-18T03:22:43.098 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.565+0000 7f2f69dd7700 1 -- 172.21.15.40:0/2234958939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f64199820 con 0x7f2f64104280 2023-07-18T03:22:43.098 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.565+0000 7f2f69dd7700 1 -- 172.21.15.40:0/2234958939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f64199c40 con 0x7f2f64104280 2023-07-18T03:22:43.098 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.565+0000 7f2f60ff9700 1 -- 172.21.15.40:0/2234958939 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2f5401ac50 con 0x7f2f64104280 2023-07-18T03:22:43.098 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.565+0000 7f2f60ff9700 1 -- 172.21.15.40:0/2234958939 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2f54019700 con 0x7f2f64104280 2023-07-18T03:22:43.098 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.566+0000 7f2f69dd7700 1 -- 172.21.15.40:0/2234958939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f64061b40 con 0x7f2f64104280 2023-07-18T03:22:43.098 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.568+0000 7f2f60ff9700 1 -- 172.21.15.40:0/2234958939 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f2f5402de30 con 0x7f2f64104280 2023-07-18T03:22:43.098 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.701+0000 7f2f69dd7700 1 -- 172.21.15.40:0/2234958939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2f6419cbd0 con 0x7f2f64104280 2023-07-18T03:22:43.099 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.701+0000 7f2f60ff9700 1 -- 172.21.15.40:0/2234958939 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f2f54024a60 con 0x7f2f64104280 2023-07-18T03:22:43.099 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.702+0000 7f2f69dd7700 1 -- 172.21.15.40:0/2234958939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f64104280 msgr2=0x7f2f641990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:43.099 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.702+0000 7f2f69dd7700 1 --2- 172.21.15.40:0/2234958939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f64104280 0x7f2f641990e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2f54006af0 tx=0x7f2f54011ad0 comp rx=0 tx=0).stop 2023-07-18T03:22:43.099 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.702+0000 7f2f69dd7700 1 -- 172.21.15.40:0/2234958939 shutdown_connections 2023-07-18T03:22:43.099 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.702+0000 7f2f69dd7700 1 --2- 172.21.15.40:0/2234958939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f64104280 0x7f2f641990e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:43.099 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.702+0000 7f2f69dd7700 1 -- 172.21.15.40:0/2234958939 >> 172.21.15.40:0/2234958939 conn(0x7f2f640ff8d0 msgr2=0x7f2f64073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:43.099 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.702+0000 7f2f69dd7700 1 -- 172.21.15.40:0/2234958939 shutdown_connections 2023-07-18T03:22:43.100 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:42.702+0000 7f2f69dd7700 1 -- 172.21.15.40:0/2234958939 wait complete. 2023-07-18T03:22:43.100 INFO:teuthology.orchestra.run.smithi040.stdout:mgr not available, waiting (2/15)... 2023-07-18T03:22:43.251 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:42 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:42.908+0000 7fa9f17a2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:22:43.502 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:43 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:43.255+0000 7fa9f17a2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-18T03:22:43.828 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:43 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:43.507+0000 7fa9f17a2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-18T03:22:43.829 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:43 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:43.687+0000 7fa9f17a2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-18T03:22:44.328 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:44 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:44.011+0000 7fa9f17a2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-18T03:22:45.078 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:44 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:44.827+0000 7fa9f17a2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-18T03:22:45.078 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:44 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:44.984+0000 7fa9f17a2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-18T03:22:45.578 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:45 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:45.236+0000 7fa9f17a2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-18T03:22:45.579 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:45 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:45.376+0000 7fa9f17a2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-18T03:22:46.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:45 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2631944911' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-18T03:22:46.079 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:45 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:45.650+0000 7fa9f17a2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-18T03:22:46.432 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2023-07-18T03:22:46.433 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2023-07-18T03:22:46.433 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsid": "3af3dba8-251a-11ee-9b34-001a4aab830c", 2023-07-18T03:22:46.433 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "health": { 2023-07-18T03:22:46.433 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-18T03:22:46.433 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-18T03:22:46.433 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-18T03:22:46.433 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:46.434 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-18T03:22:46.434 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-18T03:22:46.434 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 0 2023-07-18T03:22:46.434 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2023-07-18T03:22:46.434 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-18T03:22:46.434 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "a" 2023-07-18T03:22:46.434 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2023-07-18T03:22:46.434 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-07-18T03:22:46.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-18T03:22:46.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T03:22:46.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-18T03:22:46.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-18T03:22:46.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:46.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-18T03:22:46.435 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T03:22:46.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-18T03:22:46.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-18T03:22:46.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-18T03:22:46.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-18T03:22:46.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-18T03:22:46.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-18T03:22:46.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:46.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-18T03:22:46.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-18T03:22:46.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-18T03:22:46.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-18T03:22:46.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-18T03:22:46.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-18T03:22:46.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-18T03:22:46.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-18T03:22:46.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-18T03:22:46.438 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:46.438 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-18T03:22:46.438 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T03:22:46.438 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-18T03:22:46.438 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-18T03:22:46.438 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:46.438 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-18T03:22:46.438 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-18T03:22:46.439 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-18T03:22:46.439 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-18T03:22:46.439 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-18T03:22:46.439 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-18T03:22:46.439 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "restful" 2023-07-18T03:22:46.439 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2023-07-18T03:22:46.439 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-18T03:22:46.439 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:46.440 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-18T03:22:46.440 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T03:22:46.440 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modified": "2023-07-18T03:22:30.461984+0000", 2023-07-18T03:22:46.440 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-18T03:22:46.440 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:46.440 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-18T03:22:46.440 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2023-07-18T03:22:46.441 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.578+0000 7f4516773700 1 Processor -- start 2023-07-18T03:22:46.441 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.578+0000 7f4516773700 1 -- start start 2023-07-18T03:22:46.441 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.578+0000 7f4516773700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4510104280 0x7f4510104690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:46.441 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.578+0000 7f4516773700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4510104bd0 con 0x7f4510104280 2023-07-18T03:22:46.441 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.578+0000 7f450ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4510104280 0x7f4510104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:46.441 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.578+0000 7f450ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4510104280 0x7f4510104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54786/0 (socket says 172.21.15.40:54786) 2023-07-18T03:22:46.441 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.578+0000 7f450ffff700 1 -- 172.21.15.40:0/4203387990 learned_addr learned my addr 172.21.15.40:0/4203387990 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:46.441 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.579+0000 7f450ffff700 1 -- 172.21.15.40:0/4203387990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4510104d10 con 0x7f4510104280 2023-07-18T03:22:46.442 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.579+0000 7f450ffff700 1 --2- 172.21.15.40:0/4203387990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4510104280 0x7f4510104690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f45000098a0 tx=0x7f4500009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf8501970c4933d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:46.442 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.579+0000 7f450effd700 1 -- 172.21.15.40:0/4203387990 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4500004160 con 0x7f4510104280 2023-07-18T03:22:46.442 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.579+0000 7f450effd700 1 -- 172.21.15.40:0/4203387990 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f45000042c0 con 0x7f4510104280 2023-07-18T03:22:46.442 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.579+0000 7f450effd700 1 -- 172.21.15.40:0/4203387990 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4500011a00 con 0x7f4510104280 2023-07-18T03:22:46.442 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.580+0000 7f4516773700 1 -- 172.21.15.40:0/4203387990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4510104280 msgr2=0x7f4510104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:46.442 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.580+0000 7f4516773700 1 --2- 172.21.15.40:0/4203387990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4510104280 0x7f4510104690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f45000098a0 tx=0x7f4500009bb0 comp rx=0 tx=0).stop 2023-07-18T03:22:46.442 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.580+0000 7f4516773700 1 -- 172.21.15.40:0/4203387990 shutdown_connections 2023-07-18T03:22:46.443 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.580+0000 7f4516773700 1 --2- 172.21.15.40:0/4203387990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4510104280 0x7f4510104690 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:46.443 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.580+0000 7f4516773700 1 -- 172.21.15.40:0/4203387990 >> 172.21.15.40:0/4203387990 conn(0x7f45100ff8d0 msgr2=0x7f4510101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:46.443 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.580+0000 7f4516773700 1 -- 172.21.15.40:0/4203387990 shutdown_connections 2023-07-18T03:22:46.443 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.580+0000 7f4516773700 1 -- 172.21.15.40:0/4203387990 wait complete. 2023-07-18T03:22:46.443 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.580+0000 7f4516773700 1 Processor -- start 2023-07-18T03:22:46.443 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.581+0000 7f4516773700 1 -- start start 2023-07-18T03:22:46.443 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.581+0000 7f4516773700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4510104280 0x7f45100739d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:46.443 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.581+0000 7f4516773700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451019be20 con 0x7f4510104280 2023-07-18T03:22:46.444 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.581+0000 7f450ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4510104280 0x7f45100739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:46.444 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.581+0000 7f450ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4510104280 0x7f45100739d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54788/0 (socket says 172.21.15.40:54788) 2023-07-18T03:22:46.444 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.581+0000 7f450ffff700 1 -- 172.21.15.40:0/2631944911 learned_addr learned my addr 172.21.15.40:0/2631944911 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:46.444 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.581+0000 7f450ffff700 1 -- 172.21.15.40:0/2631944911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4500009550 con 0x7f4510104280 2023-07-18T03:22:46.444 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.581+0000 7f450ffff700 1 --2- 172.21.15.40:0/2631944911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4510104280 0x7f45100739d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4500009fb0 tx=0x7f4500011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:46.444 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.582+0000 7f450d7fa700 1 -- 172.21.15.40:0/2631944911 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4500003930 con 0x7f4510104280 2023-07-18T03:22:46.444 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.582+0000 7f4516773700 1 -- 172.21.15.40:0/2631944911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f451019bf60 con 0x7f4510104280 2023-07-18T03:22:46.445 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.582+0000 7f450d7fa700 1 -- 172.21.15.40:0/2631944911 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4500003a90 con 0x7f4510104280 2023-07-18T03:22:46.445 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.582+0000 7f450d7fa700 1 -- 172.21.15.40:0/2631944911 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4500019610 con 0x7f4510104280 2023-07-18T03:22:46.445 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.582+0000 7f4516773700 1 -- 172.21.15.40:0/2631944911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451019c320 con 0x7f4510104280 2023-07-18T03:22:46.445 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.582+0000 7f450d7fa700 1 -- 172.21.15.40:0/2631944911 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4500019c10 con 0x7f4510104280 2023-07-18T03:22:46.445 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.582+0000 7f450d7fa700 1 -- 172.21.15.40:0/2631944911 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f45000243f0 con 0x7f4510104280 2023-07-18T03:22:46.445 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.583+0000 7f4516773700 1 -- 172.21.15.40:0/2631944911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4510061b40 con 0x7f4510104280 2023-07-18T03:22:46.445 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.585+0000 7f450d7fa700 1 -- 172.21.15.40:0/2631944911 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f4500003c00 con 0x7f4510104280 2023-07-18T03:22:46.445 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.717+0000 7f4516773700 1 -- 172.21.15.40:0/2631944911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f451019cc40 con 0x7f4510104280 2023-07-18T03:22:46.446 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.718+0000 7f450d7fa700 1 -- 172.21.15.40:0/2631944911 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f450001a9d0 con 0x7f4510104280 2023-07-18T03:22:46.446 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.719+0000 7f4516773700 1 -- 172.21.15.40:0/2631944911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4510104280 msgr2=0x7f45100739d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:46.446 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.719+0000 7f4516773700 1 --2- 172.21.15.40:0/2631944911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4510104280 0x7f45100739d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4500009fb0 tx=0x7f4500011ad0 comp rx=0 tx=0).stop 2023-07-18T03:22:46.446 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.719+0000 7f4516773700 1 -- 172.21.15.40:0/2631944911 shutdown_connections 2023-07-18T03:22:46.446 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.719+0000 7f4516773700 1 --2- 172.21.15.40:0/2631944911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4510104280 0x7f45100739d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:46.446 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.719+0000 7f4516773700 1 -- 172.21.15.40:0/2631944911 >> 172.21.15.40:0/2631944911 conn(0x7f45100ff8d0 msgr2=0x7f4510100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:46.446 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.719+0000 7f4516773700 1 -- 172.21.15.40:0/2631944911 shutdown_connections 2023-07-18T03:22:46.447 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:45.719+0000 7f4516773700 1 -- 172.21.15.40:0/2631944911 wait complete. 2023-07-18T03:22:46.447 INFO:teuthology.orchestra.run.smithi040.stdout:mgr not available, waiting (3/15)... 2023-07-18T03:22:47.078 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:46.694+0000 7fa9f17a2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-18T03:22:47.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:47 smithi040 ceph-mon[96439]: Activating manager daemon x 2023-07-18T03:22:47.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:47 smithi040 ceph-mon[96439]: mgrmap e2: x(active, starting, since 0.0030829s) 2023-07-18T03:22:47.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:47 smithi040 ceph-mon[96439]: from='mgr.14100 172.21.15.40:0/140627799' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-18T03:22:47.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:47 smithi040 ceph-mon[96439]: from='mgr.14100 172.21.15.40:0/140627799' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-18T03:22:47.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:47 smithi040 ceph-mon[96439]: from='mgr.14100 172.21.15.40:0/140627799' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-18T03:22:47.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:47 smithi040 ceph-mon[96439]: from='mgr.14100 172.21.15.40:0/140627799' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:22:47.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:47 smithi040 ceph-mon[96439]: from='mgr.14100 172.21.15.40:0/140627799' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-18T03:22:47.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:47 smithi040 ceph-mon[96439]: Manager daemon x is now available 2023-07-18T03:22:47.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:47 smithi040 ceph-mon[96439]: from='mgr.14100 172.21.15.40:0/140627799' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-18T03:22:47.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:47 smithi040 ceph-mon[96439]: from='mgr.14100 172.21.15.40:0/140627799' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-18T03:22:47.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:47 smithi040 ceph-mon[96439]: from='mgr.14100 172.21.15.40:0/140627799' entity='mgr.x' 2023-07-18T03:22:47.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:47 smithi040 ceph-mon[96439]: from='mgr.14100 172.21.15.40:0/140627799' entity='mgr.x' 2023-07-18T03:22:47.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:47 smithi040 ceph-mon[96439]: from='mgr.14100 172.21.15.40:0/140627799' entity='mgr.x' 2023-07-18T03:22:49.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:48 smithi040 ceph-mon[96439]: mgrmap e3: x(active, since 1.00919s) 2023-07-18T03:22:50.975 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:50 smithi040 ceph-mon[96439]: mgrmap e4: x(active, since 3s) 2023-07-18T03:22:52.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:52 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/736560222' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-18T03:22:54.600 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2023-07-18T03:22:54.600 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2023-07-18T03:22:54.600 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsid": "3af3dba8-251a-11ee-9b34-001a4aab830c", 2023-07-18T03:22:54.600 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "health": { 2023-07-18T03:22:54.601 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-18T03:22:54.601 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-18T03:22:54.601 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-18T03:22:54.602 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:54.602 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-18T03:22:54.602 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-18T03:22:54.603 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 0 2023-07-18T03:22:54.603 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2023-07-18T03:22:54.603 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-18T03:22:54.603 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "a" 2023-07-18T03:22:54.604 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2023-07-18T03:22:54.604 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2023-07-18T03:22:54.604 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-18T03:22:54.604 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T03:22:54.605 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-18T03:22:54.605 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-18T03:22:54.605 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:54.606 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-18T03:22:54.606 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T03:22:54.606 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-18T03:22:54.606 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-18T03:22:54.607 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-18T03:22:54.607 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-18T03:22:54.607 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-18T03:22:54.607 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-18T03:22:54.608 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:54.608 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-18T03:22:54.608 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-18T03:22:54.608 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-18T03:22:54.608 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-18T03:22:54.609 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-18T03:22:54.609 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-18T03:22:54.609 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-18T03:22:54.609 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-18T03:22:54.609 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-18T03:22:54.609 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:54.609 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-18T03:22:54.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T03:22:54.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-18T03:22:54.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-18T03:22:54.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:54.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-18T03:22:54.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-18T03:22:54.610 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-18T03:22:54.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-18T03:22:54.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-18T03:22:54.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-18T03:22:54.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "restful" 2023-07-18T03:22:54.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ], 2023-07-18T03:22:54.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-18T03:22:54.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:54.611 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-18T03:22:54.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-18T03:22:54.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "modified": "2023-07-18T03:22:30.461984+0000", 2023-07-18T03:22:54.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-18T03:22:54.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout }, 2023-07-18T03:22:54.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-18T03:22:54.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2023-07-18T03:22:54.612 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.459+0000 7f3f16c53700 1 Processor -- start 2023-07-18T03:22:54.613 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.459+0000 7f3f16c53700 1 -- start start 2023-07-18T03:22:54.613 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.459+0000 7f3f16c53700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f1007bdc0 0x7f3f1007a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:54.613 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.459+0000 7f3f16c53700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1007a800 con 0x7f3f1007bdc0 2023-07-18T03:22:54.613 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.459+0000 7f3f149ef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f1007bdc0 0x7f3f1007a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:54.613 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.459+0000 7f3f149ef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f1007bdc0 0x7f3f1007a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54814/0 (socket says 172.21.15.40:54814) 2023-07-18T03:22:54.613 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.459+0000 7f3f149ef700 1 -- 172.21.15.40:0/2841184168 learned_addr learned my addr 172.21.15.40:0/2841184168 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:54.613 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.460+0000 7f3f149ef700 1 -- 172.21.15.40:0/2841184168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1007a940 con 0x7f3f1007bdc0 2023-07-18T03:22:54.614 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.460+0000 7f3f149ef700 1 --2- 172.21.15.40:0/2841184168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f1007bdc0 0x7f3f1007a2c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3f040098a0 tx=0x7f3f04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67e6eda7349a9a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:54.614 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.460+0000 7f3f0f7fe700 1 -- 172.21.15.40:0/2841184168 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f04004160 con 0x7f3f1007bdc0 2023-07-18T03:22:54.614 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.460+0000 7f3f0f7fe700 1 -- 172.21.15.40:0/2841184168 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3f040042c0 con 0x7f3f1007bdc0 2023-07-18T03:22:54.614 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.460+0000 7f3f0f7fe700 1 -- 172.21.15.40:0/2841184168 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f04011a00 con 0x7f3f1007bdc0 2023-07-18T03:22:54.614 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.461+0000 7f3f16c53700 1 -- 172.21.15.40:0/2841184168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f1007bdc0 msgr2=0x7f3f1007a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:54.614 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.461+0000 7f3f16c53700 1 --2- 172.21.15.40:0/2841184168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f1007bdc0 0x7f3f1007a2c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3f040098a0 tx=0x7f3f04009bb0 comp rx=0 tx=0).stop 2023-07-18T03:22:54.614 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.461+0000 7f3f16c53700 1 -- 172.21.15.40:0/2841184168 shutdown_connections 2023-07-18T03:22:54.614 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.461+0000 7f3f16c53700 1 --2- 172.21.15.40:0/2841184168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f1007bdc0 0x7f3f1007a2c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:54.615 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.461+0000 7f3f16c53700 1 -- 172.21.15.40:0/2841184168 >> 172.21.15.40:0/2841184168 conn(0x7f3f10100a20 msgr2=0x7f3f10102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:54.615 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.461+0000 7f3f16c53700 1 -- 172.21.15.40:0/2841184168 shutdown_connections 2023-07-18T03:22:54.615 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.461+0000 7f3f16c53700 1 -- 172.21.15.40:0/2841184168 wait complete. 2023-07-18T03:22:54.615 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.461+0000 7f3f16c53700 1 Processor -- start 2023-07-18T03:22:54.615 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.461+0000 7f3f16c53700 1 -- start start 2023-07-18T03:22:54.615 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.462+0000 7f3f16c53700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f1007bdc0 0x7f3f1019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:54.615 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.462+0000 7f3f16c53700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1019b900 con 0x7f3f1007bdc0 2023-07-18T03:22:54.616 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.462+0000 7f3f149ef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f1007bdc0 0x7f3f1019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:54.616 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.462+0000 7f3f149ef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f1007bdc0 0x7f3f1019b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54816/0 (socket says 172.21.15.40:54816) 2023-07-18T03:22:54.616 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.462+0000 7f3f149ef700 1 -- 172.21.15.40:0/736560222 learned_addr learned my addr 172.21.15.40:0/736560222 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:54.616 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.462+0000 7f3f149ef700 1 -- 172.21.15.40:0/736560222 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f04009550 con 0x7f3f1007bdc0 2023-07-18T03:22:54.616 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.462+0000 7f3f149ef700 1 --2- 172.21.15.40:0/736560222 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f1007bdc0 0x7f3f1019b3c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009fb0 tx=0x7f3f04011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:54.616 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.462+0000 7f3f0dffb700 1 -- 172.21.15.40:0/736560222 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f04003a40 con 0x7f3f1007bdc0 2023-07-18T03:22:54.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.462+0000 7f3f0dffb700 1 -- 172.21.15.40:0/736560222 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3f04003ba0 con 0x7f3f1007bdc0 2023-07-18T03:22:54.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.462+0000 7f3f16c53700 1 -- 172.21.15.40:0/736560222 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1019bb00 con 0x7f3f1007bdc0 2023-07-18T03:22:54.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.462+0000 7f3f0dffb700 1 -- 172.21.15.40:0/736560222 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f0401a750 con 0x7f3f1007bdc0 2023-07-18T03:22:54.617 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.462+0000 7f3f16c53700 1 -- 172.21.15.40:0/736560222 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1019bf20 con 0x7f3f1007bdc0 2023-07-18T03:22:54.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.463+0000 7f3f0dffb700 1 -- 172.21.15.40:0/736560222 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f3f0401a980 con 0x7f3f1007bdc0 2023-07-18T03:22:54.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.464+0000 7f3f16c53700 1 -- 172.21.15.40:0/736560222 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f10195380 con 0x7f3f1007bdc0 2023-07-18T03:22:54.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.464+0000 7f3f0dffb700 1 --2- 172.21.15.40:0/736560222 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f3f00038710 0x7f3f0003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:54.618 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.464+0000 7f3f0dffb700 1 -- 172.21.15.40:0/736560222 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3f0404c4a0 con 0x7f3f1007bdc0 2023-07-18T03:22:54.619 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.464+0000 7f3f0ffff700 1 --2- 172.21.15.40:0/736560222 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f3f00038710 0x7f3f0003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:54.619 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.464+0000 7f3f0ffff700 1 --2- 172.21.15.40:0/736560222 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f3f00038710 0x7f3f0003abc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3efc006fd0 tx=0x7f3efc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:54.619 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.469+0000 7f3f0dffb700 1 -- 172.21.15.40:0/736560222 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3f0401ac30 con 0x7f3f1007bdc0 2023-07-18T03:22:54.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.987+0000 7f3f16c53700 1 -- 172.21.15.40:0/736560222 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3f1002c850 con 0x7f3f1007bdc0 2023-07-18T03:22:54.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.987+0000 7f3f0dffb700 1 -- 172.21.15.40:0/736560222 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f3f04018350 con 0x7f3f1007bdc0 2023-07-18T03:22:54.621 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.990+0000 7f3f16c53700 1 -- 172.21.15.40:0/736560222 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f3f00038710 msgr2=0x7f3f0003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:54.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.990+0000 7f3f16c53700 1 --2- 172.21.15.40:0/736560222 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f3f00038710 0x7f3f0003abc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3efc006fd0 tx=0x7f3efc006e40 comp rx=0 tx=0).stop 2023-07-18T03:22:54.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.990+0000 7f3f16c53700 1 -- 172.21.15.40:0/736560222 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f1007bdc0 msgr2=0x7f3f1019b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:54.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.990+0000 7f3f16c53700 1 --2- 172.21.15.40:0/736560222 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f1007bdc0 0x7f3f1019b3c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009fb0 tx=0x7f3f04011ad0 comp rx=0 tx=0).stop 2023-07-18T03:22:54.622 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.990+0000 7f3f16c53700 1 -- 172.21.15.40:0/736560222 shutdown_connections 2023-07-18T03:22:54.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.990+0000 7f3f16c53700 1 --2- 172.21.15.40:0/736560222 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f3f00038710 0x7f3f0003abc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:54.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.990+0000 7f3f16c53700 1 --2- 172.21.15.40:0/736560222 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f1007bdc0 0x7f3f1019b3c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:54.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.990+0000 7f3f16c53700 1 -- 172.21.15.40:0/736560222 >> 172.21.15.40:0/736560222 conn(0x7f3f10100a20 msgr2=0x7f3f10102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:54.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.990+0000 7f3f16c53700 1 -- 172.21.15.40:0/736560222 shutdown_connections 2023-07-18T03:22:54.623 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:51.990+0000 7f3f16c53700 1 -- 172.21.15.40:0/736560222 wait complete. 2023-07-18T03:22:54.624 INFO:teuthology.orchestra.run.smithi040.stdout:mgr is available 2023-07-18T03:22:55.990 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:55 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2132201283' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-18T03:22:56.052 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2023-07-18T03:22:56.052 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [global] 2023-07-18T03:22:56.052 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout fsid = 3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:22:56.053 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-18T03:22:56.053 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-18T03:22:56.053 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-18T03:22:56.053 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-18T03:22:56.053 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2023-07-18T03:22:56.053 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-18T03:22:56.053 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-18T03:22:56.054 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 2023-07-18T03:22:56.054 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout [osd] 2023-07-18T03:22:56.054 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-18T03:22:56.054 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-18T03:22:56.054 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-18T03:22:56.054 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-18T03:22:56.054 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-18T03:22:56.054 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.254+0000 7f2804fdd700 1 Processor -- start 2023-07-18T03:22:56.055 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.254+0000 7f2804fdd700 1 -- start start 2023-07-18T03:22:56.055 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.255+0000 7f2804fdd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2800104280 0x7f2800104690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.055 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.255+0000 7f2804fdd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2800104bd0 con 0x7f2800104280 2023-07-18T03:22:56.055 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.255+0000 7f27fe59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2800104280 0x7f2800104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.055 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.255+0000 7f27fe59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2800104280 0x7f2800104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54820/0 (socket says 172.21.15.40:54820) 2023-07-18T03:22:56.055 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.255+0000 7f27fe59c700 1 -- 172.21.15.40:0/3977495683 learned_addr learned my addr 172.21.15.40:0/3977495683 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:56.055 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.255+0000 7f27fe59c700 1 -- 172.21.15.40:0/3977495683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2800104d10 con 0x7f2800104280 2023-07-18T03:22:56.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.255+0000 7f27fe59c700 1 --2- 172.21.15.40:0/3977495683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2800104280 0x7f2800104690 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f27f00098a0 tx=0x7f27f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242da93852cb3a98 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:56.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.255+0000 7f27fd59a700 1 -- 172.21.15.40:0/3977495683 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27f0004160 con 0x7f2800104280 2023-07-18T03:22:56.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.255+0000 7f27fd59a700 1 -- 172.21.15.40:0/3977495683 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f27f00042c0 con 0x7f2800104280 2023-07-18T03:22:56.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.255+0000 7f27fd59a700 1 -- 172.21.15.40:0/3977495683 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27f0011a00 con 0x7f2800104280 2023-07-18T03:22:56.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.256+0000 7f2804fdd700 1 -- 172.21.15.40:0/3977495683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2800104280 msgr2=0x7f2800104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:56.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.256+0000 7f2804fdd700 1 --2- 172.21.15.40:0/3977495683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2800104280 0x7f2800104690 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f27f00098a0 tx=0x7f27f0009bb0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.056 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.256+0000 7f2804fdd700 1 -- 172.21.15.40:0/3977495683 shutdown_connections 2023-07-18T03:22:56.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.256+0000 7f2804fdd700 1 --2- 172.21.15.40:0/3977495683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2800104280 0x7f2800104690 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.256+0000 7f2804fdd700 1 -- 172.21.15.40:0/3977495683 >> 172.21.15.40:0/3977495683 conn(0x7f28000ff8d0 msgr2=0x7f2800101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:56.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.256+0000 7f2804fdd700 1 -- 172.21.15.40:0/3977495683 shutdown_connections 2023-07-18T03:22:56.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.256+0000 7f2804fdd700 1 -- 172.21.15.40:0/3977495683 wait complete. 2023-07-18T03:22:56.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.257+0000 7f2804fdd700 1 Processor -- start 2023-07-18T03:22:56.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.257+0000 7f2804fdd700 1 -- start start 2023-07-18T03:22:56.057 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.257+0000 7f2804fdd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2800104280 0x7f28001990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.257+0000 7f2804fdd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2800199620 con 0x7f2800104280 2023-07-18T03:22:56.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.257+0000 7f27fe59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2800104280 0x7f28001990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.257+0000 7f27fe59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2800104280 0x7f28001990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54822/0 (socket says 172.21.15.40:54822) 2023-07-18T03:22:56.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.257+0000 7f27fe59c700 1 -- 172.21.15.40:0/2132201283 learned_addr learned my addr 172.21.15.40:0/2132201283 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:56.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.258+0000 7f27fe59c700 1 -- 172.21.15.40:0/2132201283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f0009550 con 0x7f2800104280 2023-07-18T03:22:56.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.258+0000 7f27fe59c700 1 --2- 172.21.15.40:0/2132201283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2800104280 0x7f28001990e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27f0006af0 tx=0x7f27f0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:56.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.258+0000 7f27f77fe700 1 -- 172.21.15.40:0/2132201283 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27f0003a40 con 0x7f2800104280 2023-07-18T03:22:56.058 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.258+0000 7f27f77fe700 1 -- 172.21.15.40:0/2132201283 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f27f0003ba0 con 0x7f2800104280 2023-07-18T03:22:56.059 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.258+0000 7f2804fdd700 1 -- 172.21.15.40:0/2132201283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2800199820 con 0x7f2800104280 2023-07-18T03:22:56.059 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.258+0000 7f27f77fe700 1 -- 172.21.15.40:0/2132201283 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27f001a6e0 con 0x7f2800104280 2023-07-18T03:22:56.059 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.258+0000 7f2804fdd700 1 -- 172.21.15.40:0/2132201283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2800199c40 con 0x7f2800104280 2023-07-18T03:22:56.059 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.259+0000 7f2804fdd700 1 -- 172.21.15.40:0/2132201283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2800061b40 con 0x7f2800104280 2023-07-18T03:22:56.059 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.259+0000 7f27f77fe700 1 -- 172.21.15.40:0/2132201283 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f27f0024a60 con 0x7f2800104280 2023-07-18T03:22:56.059 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.259+0000 7f27f77fe700 1 --2- 172.21.15.40:0/2132201283 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f27ec0386c0 0x7f27ec03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:56.059 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.259+0000 7f27f77fe700 1 -- 172.21.15.40:0/2132201283 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f27f004c7c0 con 0x7f2800104280 2023-07-18T03:22:56.060 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.260+0000 7f27fdd9b700 1 --2- 172.21.15.40:0/2132201283 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f27ec0386c0 0x7f27ec03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:56.060 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.260+0000 7f27fdd9b700 1 --2- 172.21.15.40:0/2132201283 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f27ec0386c0 0x7f27ec03ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f27e8006fd0 tx=0x7f27e8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:56.060 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.264+0000 7f27f77fe700 1 -- 172.21.15.40:0/2132201283 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f27f0018940 con 0x7f2800104280 2023-07-18T03:22:56.060 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.620+0000 7f2804fdd700 1 -- 172.21.15.40:0/2132201283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f2800106140 con 0x7f2800104280 2023-07-18T03:22:56.060 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.621+0000 7f27f77fe700 1 -- 172.21.15.40:0/2132201283 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f27f0018350 con 0x7f2800104280 2023-07-18T03:22:56.060 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.623+0000 7f2804fdd700 1 -- 172.21.15.40:0/2132201283 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f27ec0386c0 msgr2=0x7f27ec03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:56.060 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.623+0000 7f2804fdd700 1 --2- 172.21.15.40:0/2132201283 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f27ec0386c0 0x7f27ec03ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f27e8006fd0 tx=0x7f27e8006e40 comp rx=0 tx=0).stop 2023-07-18T03:22:56.061 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.623+0000 7f2804fdd700 1 -- 172.21.15.40:0/2132201283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2800104280 msgr2=0x7f28001990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:56.061 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.623+0000 7f2804fdd700 1 --2- 172.21.15.40:0/2132201283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2800104280 0x7f28001990e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27f0006af0 tx=0x7f27f0011ad0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.061 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.623+0000 7f2804fdd700 1 -- 172.21.15.40:0/2132201283 shutdown_connections 2023-07-18T03:22:56.061 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.623+0000 7f2804fdd700 1 --2- 172.21.15.40:0/2132201283 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f27ec0386c0 0x7f27ec03ab70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.061 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.623+0000 7f2804fdd700 1 --2- 172.21.15.40:0/2132201283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2800104280 0x7f28001990e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:56.061 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.623+0000 7f2804fdd700 1 -- 172.21.15.40:0/2132201283 >> 172.21.15.40:0/2132201283 conn(0x7f28000ff8d0 msgr2=0x7f2800072fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:56.061 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.624+0000 7f2804fdd700 1 -- 172.21.15.40:0/2132201283 shutdown_connections 2023-07-18T03:22:56.062 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:55.624+0000 7f2804fdd700 1 -- 172.21.15.40:0/2132201283 wait complete. 2023-07-18T03:22:56.062 INFO:teuthology.orchestra.run.smithi040.stdout:Enabling cephadm module... 2023-07-18T03:22:57.790 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:57 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1273827272' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-07-18T03:22:57.791 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:57 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: ignoring --setuser ceph since I am not root 2023-07-18T03:22:57.791 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:57 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: ignoring --setgroup ceph since I am not root 2023-07-18T03:22:57.791 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:57 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:57.699+0000 7f0ed82b8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-18T03:22:57.998 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.572+0000 7f70d6e5e700 1 Processor -- start 2023-07-18T03:22:57.999 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.572+0000 7f70d6e5e700 1 -- start start 2023-07-18T03:22:57.999 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.572+0000 7f70d6e5e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70d0105380 0x7f70d0105790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:57.999 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.572+0000 7f70d6e5e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d0105cd0 con 0x7f70d0105380 2023-07-18T03:22:57.999 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.572+0000 7f70d4bfa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70d0105380 0x7f70d0105790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:57.999 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.572+0000 7f70d4bfa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70d0105380 0x7f70d0105790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54826/0 (socket says 172.21.15.40:54826) 2023-07-18T03:22:58.000 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.572+0000 7f70d4bfa700 1 -- 172.21.15.40:0/3421440331 learned_addr learned my addr 172.21.15.40:0/3421440331 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:58.000 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.573+0000 7f70d4bfa700 1 -- 172.21.15.40:0/3421440331 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d0105e10 con 0x7f70d0105380 2023-07-18T03:22:58.000 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.573+0000 7f70d4bfa700 1 --2- 172.21.15.40:0/3421440331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70d0105380 0x7f70d0105790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f70c40098a0 tx=0x7f70c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43d86c5f5c2fce5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.000 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.573+0000 7f70cf7fe700 1 -- 172.21.15.40:0/3421440331 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70c4004160 con 0x7f70d0105380 2023-07-18T03:22:58.000 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.573+0000 7f70cf7fe700 1 -- 172.21.15.40:0/3421440331 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f70c40042c0 con 0x7f70d0105380 2023-07-18T03:22:58.000 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.573+0000 7f70cf7fe700 1 -- 172.21.15.40:0/3421440331 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70c4011a00 con 0x7f70d0105380 2023-07-18T03:22:58.000 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.573+0000 7f70d6e5e700 1 -- 172.21.15.40:0/3421440331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70d0105380 msgr2=0x7f70d0105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.000 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.573+0000 7f70d6e5e700 1 --2- 172.21.15.40:0/3421440331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70d0105380 0x7f70d0105790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f70c40098a0 tx=0x7f70c4009bb0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.001 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.574+0000 7f70d6e5e700 1 -- 172.21.15.40:0/3421440331 shutdown_connections 2023-07-18T03:22:58.001 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.574+0000 7f70d6e5e700 1 --2- 172.21.15.40:0/3421440331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70d0105380 0x7f70d0105790 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.001 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.574+0000 7f70d6e5e700 1 -- 172.21.15.40:0/3421440331 >> 172.21.15.40:0/3421440331 conn(0x7f70d01009f0 msgr2=0x7f70d0102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:58.001 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.574+0000 7f70d6e5e700 1 -- 172.21.15.40:0/3421440331 shutdown_connections 2023-07-18T03:22:58.001 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.574+0000 7f70d6e5e700 1 -- 172.21.15.40:0/3421440331 wait complete. 2023-07-18T03:22:58.001 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.574+0000 7f70d6e5e700 1 Processor -- start 2023-07-18T03:22:58.001 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.574+0000 7f70d6e5e700 1 -- start start 2023-07-18T03:22:58.002 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.574+0000 7f70d6e5e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70d0105380 0x7f70d01990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.002 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.575+0000 7f70d6e5e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d0199610 con 0x7f70d0105380 2023-07-18T03:22:58.002 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.575+0000 7f70d4bfa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70d0105380 0x7f70d01990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.002 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.575+0000 7f70d4bfa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70d0105380 0x7f70d01990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54828/0 (socket says 172.21.15.40:54828) 2023-07-18T03:22:58.002 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.575+0000 7f70d4bfa700 1 -- 172.21.15.40:0/1273827272 learned_addr learned my addr 172.21.15.40:0/1273827272 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:58.002 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.575+0000 7f70d4bfa700 1 -- 172.21.15.40:0/1273827272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c4009550 con 0x7f70d0105380 2023-07-18T03:22:58.002 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.575+0000 7f70d4bfa700 1 --2- 172.21.15.40:0/1273827272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70d0105380 0x7f70d01990d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009fb0 tx=0x7f70c4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.002 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.575+0000 7f70cdffb700 1 -- 172.21.15.40:0/1273827272 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70c4003930 con 0x7f70d0105380 2023-07-18T03:22:58.003 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.575+0000 7f70cdffb700 1 -- 172.21.15.40:0/1273827272 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f70c4003a90 con 0x7f70d0105380 2023-07-18T03:22:58.003 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.575+0000 7f70d6e5e700 1 -- 172.21.15.40:0/1273827272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d0199810 con 0x7f70d0105380 2023-07-18T03:22:58.003 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.575+0000 7f70cdffb700 1 -- 172.21.15.40:0/1273827272 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70c4019610 con 0x7f70d0105380 2023-07-18T03:22:58.003 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.575+0000 7f70d6e5e700 1 -- 172.21.15.40:0/1273827272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d0199c30 con 0x7f70d0105380 2023-07-18T03:22:58.003 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.576+0000 7f70cdffb700 1 -- 172.21.15.40:0/1273827272 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f70c4019880 con 0x7f70d0105380 2023-07-18T03:22:58.003 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.576+0000 7f70cdffb700 1 --2- 172.21.15.40:0/1273827272 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f70c00386c0 0x7f70c003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:58.003 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.577+0000 7f70cdffb700 1 -- 172.21.15.40:0/1273827272 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f70c404c460 con 0x7f70d0105380 2023-07-18T03:22:58.004 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.577+0000 7f70cffff700 1 --2- 172.21.15.40:0/1273827272 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f70c00386c0 0x7f70c003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:58.004 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.577+0000 7f70d6e5e700 1 -- 172.21.15.40:0/1273827272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d0109810 con 0x7f70d0105380 2023-07-18T03:22:58.004 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.577+0000 7f70cffff700 1 --2- 172.21.15.40:0/1273827272 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f70c00386c0 0x7f70c003ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f70bc006fd0 tx=0x7f70bc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:58.004 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:56.581+0000 7f70cdffb700 1 -- 172.21.15.40:0/1273827272 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f70c4019b30 con 0x7f70d0105380 2023-07-18T03:22:58.004 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:57.046+0000 7f70d6e5e700 1 -- 172.21.15.40:0/1273827272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f70d0107240 con 0x7f70d0105380 2023-07-18T03:22:58.004 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:57.500+0000 7f70cdffb700 1 -- 172.21.15.40:0/1273827272 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f70c402d960 con 0x7f70d0105380 2023-07-18T03:22:58.004 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:57.502+0000 7f70cdffb700 1 -- 172.21.15.40:0/1273827272 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f70c404c1c0 con 0x7f70d0105380 2023-07-18T03:22:58.005 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:57.504+0000 7f70d6e5e700 1 -- 172.21.15.40:0/1273827272 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f70c00386c0 msgr2=0x7f70c003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.005 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:57.504+0000 7f70d6e5e700 1 --2- 172.21.15.40:0/1273827272 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f70c00386c0 0x7f70c003ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f70bc006fd0 tx=0x7f70bc006e40 comp rx=0 tx=0).stop 2023-07-18T03:22:58.005 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:57.504+0000 7f70d6e5e700 1 -- 172.21.15.40:0/1273827272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70d0105380 msgr2=0x7f70d01990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:58.005 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:57.504+0000 7f70d6e5e700 1 --2- 172.21.15.40:0/1273827272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70d0105380 0x7f70d01990d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009fb0 tx=0x7f70c4011ad0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.005 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:57.505+0000 7f70d6e5e700 1 -- 172.21.15.40:0/1273827272 shutdown_connections 2023-07-18T03:22:58.005 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:57.505+0000 7f70d6e5e700 1 --2- 172.21.15.40:0/1273827272 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f70c00386c0 0x7f70c003ab70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.005 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:57.505+0000 7f70d6e5e700 1 --2- 172.21.15.40:0/1273827272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70d0105380 0x7f70d01990d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:58.006 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:57.505+0000 7f70d6e5e700 1 -- 172.21.15.40:0/1273827272 >> 172.21.15.40:0/1273827272 conn(0x7f70d01009f0 msgr2=0x7f70d0102050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:58.006 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:57.505+0000 7f70d6e5e700 1 -- 172.21.15.40:0/1273827272 shutdown_connections 2023-07-18T03:22:58.006 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:57.505+0000 7f70d6e5e700 1 -- 172.21.15.40:0/1273827272 wait complete. 2023-07-18T03:22:58.078 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:57 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:57.794+0000 7f0ed82b8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-18T03:22:58.079 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:57 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:57.904+0000 7f0ed82b8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-18T03:22:58.079 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:58 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:58.016+0000 7f0ed82b8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-18T03:22:58.494 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:58 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:58.233+0000 7f0ed82b8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-18T03:22:58.828 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:58 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:58.636+0000 7f0ed82b8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-18T03:22:58.829 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:58 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:58.745+0000 7f0ed82b8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-18T03:22:58.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:58 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1273827272' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-07-18T03:22:58.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:58 smithi040 ceph-mon[96439]: mgrmap e5: x(active, since 10s) 2023-07-18T03:22:59.328 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:58 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:58.896+0000 7f0ed82b8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-18T03:22:59.329 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:59 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:59.013+0000 7f0ed82b8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-18T03:22:59.346 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2023-07-18T03:22:59.347 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-07-18T03:22:59.347 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-18T03:22:59.347 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-18T03:22:59.347 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-18T03:22:59.347 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2023-07-18T03:22:59.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.501+0000 7f47148cb700 1 Processor -- start 2023-07-18T03:22:59.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.501+0000 7f47148cb700 1 -- start start 2023-07-18T03:22:59.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.502+0000 7f47148cb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4710105ed0 0x7f47101082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:59.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.502+0000 7f47148cb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4710079d00 con 0x7f4710105ed0 2023-07-18T03:22:59.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.502+0000 7f470e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4710105ed0 0x7f47101082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:59.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.502+0000 7f470e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4710105ed0 0x7f47101082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54842/0 (socket says 172.21.15.40:54842) 2023-07-18T03:22:59.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.502+0000 7f470e59c700 1 -- 172.21.15.40:0/433125764 learned_addr learned my addr 172.21.15.40:0/433125764 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:59.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.502+0000 7f470e59c700 1 -- 172.21.15.40:0/433125764 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4710079e40 con 0x7f4710105ed0 2023-07-18T03:22:59.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.502+0000 7f470e59c700 1 --2- 172.21.15.40:0/433125764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4710105ed0 0x7f47101082b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f47000098a0 tx=0x7f4700009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e02f469eff1805 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:59.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.502+0000 7f470d59a700 1 -- 172.21.15.40:0/433125764 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4700004160 con 0x7f4710105ed0 2023-07-18T03:22:59.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.503+0000 7f470d59a700 1 -- 172.21.15.40:0/433125764 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f47000042c0 con 0x7f4710105ed0 2023-07-18T03:22:59.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.503+0000 7f470d59a700 1 -- 172.21.15.40:0/433125764 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4700011a00 con 0x7f4710105ed0 2023-07-18T03:22:59.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.503+0000 7f47148cb700 1 -- 172.21.15.40:0/433125764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4710105ed0 msgr2=0x7f47101082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:59.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.503+0000 7f47148cb700 1 --2- 172.21.15.40:0/433125764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4710105ed0 0x7f47101082b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f47000098a0 tx=0x7f4700009bb0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.503+0000 7f47148cb700 1 -- 172.21.15.40:0/433125764 shutdown_connections 2023-07-18T03:22:59.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.503+0000 7f47148cb700 1 --2- 172.21.15.40:0/433125764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4710105ed0 0x7f47101082b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.503+0000 7f47148cb700 1 -- 172.21.15.40:0/433125764 >> 172.21.15.40:0/433125764 conn(0x7f47100ff8d0 msgr2=0x7f4710101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:59.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.503+0000 7f47148cb700 1 -- 172.21.15.40:0/433125764 shutdown_connections 2023-07-18T03:22:59.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.503+0000 7f47148cb700 1 -- 172.21.15.40:0/433125764 wait complete. 2023-07-18T03:22:59.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.504+0000 7f47148cb700 1 Processor -- start 2023-07-18T03:22:59.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.504+0000 7f47148cb700 1 -- start start 2023-07-18T03:22:59.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.505+0000 7f47148cb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4710105ed0 0x7f4710105830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:59.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.505+0000 7f47148cb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4710105d70 con 0x7f4710105ed0 2023-07-18T03:22:59.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.505+0000 7f470e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4710105ed0 0x7f4710105830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:22:59.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.505+0000 7f470e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4710105ed0 0x7f4710105830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54844/0 (socket says 172.21.15.40:54844) 2023-07-18T03:22:59.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.505+0000 7f470e59c700 1 -- 172.21.15.40:0/4036658113 learned_addr learned my addr 172.21.15.40:0/4036658113 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:22:59.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.505+0000 7f470e59c700 1 -- 172.21.15.40:0/4036658113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4700009550 con 0x7f4710105ed0 2023-07-18T03:22:59.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.505+0000 7f470e59c700 1 --2- 172.21.15.40:0/4036658113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4710105ed0 0x7f4710105830 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4700009fb0 tx=0x7f4700011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:22:59.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.505+0000 7f46ff7fe700 1 -- 172.21.15.40:0/4036658113 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4700003930 con 0x7f4710105ed0 2023-07-18T03:22:59.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.506+0000 7f46ff7fe700 1 -- 172.21.15.40:0/4036658113 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4700003a90 con 0x7f4710105ed0 2023-07-18T03:22:59.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.506+0000 7f47148cb700 1 -- 172.21.15.40:0/4036658113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4710103e80 con 0x7f4710105ed0 2023-07-18T03:22:59.355 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.506+0000 7f46ff7fe700 1 -- 172.21.15.40:0/4036658113 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47000195e0 con 0x7f4710105ed0 2023-07-18T03:22:59.355 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.506+0000 7f47148cb700 1 -- 172.21.15.40:0/4036658113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47101042a0 con 0x7f4710105ed0 2023-07-18T03:22:59.355 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.507+0000 7f47148cb700 1 -- 172.21.15.40:0/4036658113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4710109460 con 0x7f4710105ed0 2023-07-18T03:22:59.355 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.507+0000 7f46ff7fe700 1 -- 172.21.15.40:0/4036658113 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f47000198a0 con 0x7f4710105ed0 2023-07-18T03:22:59.356 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.507+0000 7f46ff7fe700 1 --2- 172.21.15.40:0/4036658113 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f46f80386c0 0x7f46f803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:22:59.356 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.507+0000 7f46ff7fe700 1 -- 172.21.15.40:0/4036658113 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f470004c680 con 0x7f4710105ed0 2023-07-18T03:22:59.356 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.507+0000 7f470dd9b700 1 -- 172.21.15.40:0/4036658113 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f46f80386c0 msgr2=0x7f46f803ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1353038297 2023-07-18T03:22:59.356 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.507+0000 7f470dd9b700 1 --2- 172.21.15.40:0/4036658113 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f46f80386c0 0x7f46f803ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T03:22:59.356 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.512+0000 7f46ff7fe700 1 -- 172.21.15.40:0/4036658113 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4700018530 con 0x7f4710105ed0 2023-07-18T03:22:59.356 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.708+0000 7f470dd9b700 1 -- 172.21.15.40:0/4036658113 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f46f80386c0 msgr2=0x7f46f803ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1353038297 2023-07-18T03:22:59.357 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.708+0000 7f470dd9b700 1 --2- 172.21.15.40:0/4036658113 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f46f80386c0 0x7f46f803ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-18T03:22:59.357 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.976+0000 7f47148cb700 1 -- 172.21.15.40:0/4036658113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f4710109680 con 0x7f4710105ed0 2023-07-18T03:22:59.357 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.977+0000 7f46ff7fe700 1 -- 172.21.15.40:0/4036658113 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f4700018350 con 0x7f4710105ed0 2023-07-18T03:22:59.357 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.979+0000 7f47148cb700 1 -- 172.21.15.40:0/4036658113 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f46f80386c0 msgr2=0x7f46f803ab70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:22:59.357 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.979+0000 7f47148cb700 1 --2- 172.21.15.40:0/4036658113 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f46f80386c0 0x7f46f803ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.358 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.979+0000 7f47148cb700 1 -- 172.21.15.40:0/4036658113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4710105ed0 msgr2=0x7f4710105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:22:59.358 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.979+0000 7f47148cb700 1 --2- 172.21.15.40:0/4036658113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4710105ed0 0x7f4710105830 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4700009fb0 tx=0x7f4700011ad0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.358 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.979+0000 7f47148cb700 1 -- 172.21.15.40:0/4036658113 shutdown_connections 2023-07-18T03:22:59.358 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.979+0000 7f47148cb700 1 --2- 172.21.15.40:0/4036658113 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f46f80386c0 0x7f46f803ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.358 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.979+0000 7f47148cb700 1 --2- 172.21.15.40:0/4036658113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4710105ed0 0x7f4710105830 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:22:59.358 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.979+0000 7f47148cb700 1 -- 172.21.15.40:0/4036658113 >> 172.21.15.40:0/4036658113 conn(0x7f47100ff8d0 msgr2=0x7f4710100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:22:59.358 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.979+0000 7f47148cb700 1 -- 172.21.15.40:0/4036658113 shutdown_connections 2023-07-18T03:22:59.359 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:58.980+0000 7f47148cb700 1 -- 172.21.15.40:0/4036658113 wait complete. 2023-07-18T03:22:59.359 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for the mgr to restart... 2023-07-18T03:22:59.359 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mgr epoch 5... 2023-07-18T03:22:59.619 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:22:59 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/4036658113' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-18T03:22:59.619 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:22:59 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:22:59.535+0000 7f0ed82b8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-18T03:23:00.578 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:00 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:00.237+0000 7f0ed82b8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-18T03:23:00.579 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:00 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:00.428+0000 7f0ed82b8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-18T03:23:01.710 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:01 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:01.386+0000 7f0ed82b8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:23:01.710 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:01 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:01.497+0000 7f0ed82b8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-18T03:23:02.055 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:01 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:01.714+0000 7f0ed82b8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:23:02.328 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:02 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:02.060+0000 7f0ed82b8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-18T03:23:02.329 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:02 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:02.310+0000 7f0ed82b8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-18T03:23:02.813 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:02 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:02.491+0000 7f0ed82b8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-18T03:23:03.078 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:02 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:02.817+0000 7f0ed82b8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-18T03:23:04.027 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:03 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:03.633+0000 7f0ed82b8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-18T03:23:04.028 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:03 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:03.790+0000 7f0ed82b8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-18T03:23:04.328 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:04 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:04.032+0000 7f0ed82b8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-18T03:23:04.329 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:04 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:04.169+0000 7f0ed82b8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-18T03:23:04.828 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:04 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:04.438+0000 7f0ed82b8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-18T03:23:05.828 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:05 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:05.469+0000 7f0ed82b8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-18T03:23:05.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:05 smithi040 ceph-mon[96439]: Active manager daemon x restarted 2023-07-18T03:23:05.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:05 smithi040 ceph-mon[96439]: Activating manager daemon x 2023-07-18T03:23:05.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:05 smithi040 ceph-mon[96439]: osdmap e2: 0 total, 0 up, 0 in 2023-07-18T03:23:05.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:05 smithi040 ceph-mon[96439]: mgrmap e6: x(active, starting, since 0.0032811s) 2023-07-18T03:23:05.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:05 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:23:05.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:05 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-18T03:23:05.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:05 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-18T03:23:05.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:05 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-18T03:23:05.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:05 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-18T03:23:05.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:05 smithi040 ceph-mon[96439]: Manager daemon x is now available 2023-07-18T03:23:05.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:05 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' 2023-07-18T03:23:05.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:05 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' 2023-07-18T03:23:05.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:05 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:23:05.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:05 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:23:06.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:06 smithi040 ceph-mon[96439]: Found migration_current of "None". Setting to last migration. 2023-07-18T03:23:06.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:06 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:23:06.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:06 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-18T03:23:06.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:06 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-18T03:23:06.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:06 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' 2023-07-18T03:23:06.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:06 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' 2023-07-18T03:23:06.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:06 smithi040 ceph-mon[96439]: mgrmap e7: x(active, since 1.00533s) 2023-07-18T03:23:06.905 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2023-07-18T03:23:06.905 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-07-18T03:23:06.905 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-18T03:23:06.905 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2023-07-18T03:23:06.906 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.849+0000 7f6c5f607700 1 Processor -- start 2023-07-18T03:23:06.906 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.849+0000 7f6c5f607700 1 -- start start 2023-07-18T03:23:06.906 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.849+0000 7f6c5f607700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c58104280 0x7f6c58104690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:06.906 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.849+0000 7f6c5f607700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c58104bd0 con 0x7f6c58104280 2023-07-18T03:23:06.906 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.850+0000 7f6c5d3a3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c58104280 0x7f6c58104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:06.906 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.850+0000 7f6c5d3a3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c58104280 0x7f6c58104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54852/0 (socket says 172.21.15.40:54852) 2023-07-18T03:23:06.906 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.850+0000 7f6c5d3a3700 1 -- 172.21.15.40:0/3982823265 learned_addr learned my addr 172.21.15.40:0/3982823265 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:06.907 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.850+0000 7f6c5d3a3700 1 -- 172.21.15.40:0/3982823265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c58104d10 con 0x7f6c58104280 2023-07-18T03:23:06.907 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.850+0000 7f6c5d3a3700 1 --2- 172.21.15.40:0/3982823265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c58104280 0x7f6c58104690 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6c540098a0 tx=0x7f6c54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8938b199ad473b2b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:06.907 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.850+0000 7f6c4ffff700 1 -- 172.21.15.40:0/3982823265 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c54004160 con 0x7f6c58104280 2023-07-18T03:23:06.907 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.850+0000 7f6c4ffff700 1 -- 172.21.15.40:0/3982823265 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6c540042c0 con 0x7f6c58104280 2023-07-18T03:23:06.907 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.850+0000 7f6c4ffff700 1 -- 172.21.15.40:0/3982823265 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c54011a00 con 0x7f6c58104280 2023-07-18T03:23:06.907 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.851+0000 7f6c5f607700 1 -- 172.21.15.40:0/3982823265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c58104280 msgr2=0x7f6c58104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:06.907 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.851+0000 7f6c5f607700 1 --2- 172.21.15.40:0/3982823265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c58104280 0x7f6c58104690 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6c540098a0 tx=0x7f6c54009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:06.908 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.851+0000 7f6c5f607700 1 -- 172.21.15.40:0/3982823265 shutdown_connections 2023-07-18T03:23:06.908 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.851+0000 7f6c5f607700 1 --2- 172.21.15.40:0/3982823265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c58104280 0x7f6c58104690 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:06.908 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.851+0000 7f6c5f607700 1 -- 172.21.15.40:0/3982823265 >> 172.21.15.40:0/3982823265 conn(0x7f6c580ff8d0 msgr2=0x7f6c58101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:06.908 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.851+0000 7f6c5f607700 1 -- 172.21.15.40:0/3982823265 shutdown_connections 2023-07-18T03:23:06.908 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.851+0000 7f6c5f607700 1 -- 172.21.15.40:0/3982823265 wait complete. 2023-07-18T03:23:06.908 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.852+0000 7f6c5f607700 1 Processor -- start 2023-07-18T03:23:06.908 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.852+0000 7f6c5f607700 1 -- start start 2023-07-18T03:23:06.908 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.852+0000 7f6c5f607700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c58104280 0x7f6c5818f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:06.909 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.852+0000 7f6c5f607700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5818f940 con 0x7f6c58104280 2023-07-18T03:23:06.909 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.852+0000 7f6c5d3a3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c58104280 0x7f6c5818f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:06.909 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.852+0000 7f6c5d3a3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c58104280 0x7f6c5818f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54854/0 (socket says 172.21.15.40:54854) 2023-07-18T03:23:06.909 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.852+0000 7f6c5d3a3700 1 -- 172.21.15.40:0/1573736304 learned_addr learned my addr 172.21.15.40:0/1573736304 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:06.909 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.852+0000 7f6c5d3a3700 1 -- 172.21.15.40:0/1573736304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c54009550 con 0x7f6c58104280 2023-07-18T03:23:06.909 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.853+0000 7f6c5d3a3700 1 --2- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c58104280 0x7f6c5818f400 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009fb0 tx=0x7f6c54011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:06.909 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.853+0000 7f6c4e7fc700 1 -- 172.21.15.40:0/1573736304 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c54003930 con 0x7f6c58104280 2023-07-18T03:23:06.910 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.853+0000 7f6c4e7fc700 1 -- 172.21.15.40:0/1573736304 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6c54003a90 con 0x7f6c58104280 2023-07-18T03:23:06.910 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.853+0000 7f6c5f607700 1 -- 172.21.15.40:0/1573736304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5818da50 con 0x7f6c58104280 2023-07-18T03:23:06.910 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.853+0000 7f6c4e7fc700 1 -- 172.21.15.40:0/1573736304 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c54019610 con 0x7f6c58104280 2023-07-18T03:23:06.910 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.853+0000 7f6c5f607700 1 -- 172.21.15.40:0/1573736304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5818de70 con 0x7f6c58104280 2023-07-18T03:23:06.910 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.854+0000 7f6c4e7fc700 1 -- 172.21.15.40:0/1573736304 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f6c5401a9d0 con 0x7f6c58104280 2023-07-18T03:23:06.910 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.854+0000 7f6c4e7fc700 1 --2- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f6c440386c0 0x7f6c4403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:06.910 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.854+0000 7f6c4e7fc700 1 -- 172.21.15.40:0/1573736304 --> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6c4403b280 con 0x7f6c440386c0 2023-07-18T03:23:06.910 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.854+0000 7f6c4e7fc700 1 -- 172.21.15.40:0/1573736304 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6c5404c490 con 0x7f6c58104280 2023-07-18T03:23:06.911 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.854+0000 7f6c5cba2700 1 -- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f6c440386c0 msgr2=0x7f6c4403ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1353038297 2023-07-18T03:23:06.911 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:22:59.854+0000 7f6c5cba2700 1 --2- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f6c440386c0 0x7f6c4403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T03:23:06.911 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:00.055+0000 7f6c5cba2700 1 -- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f6c440386c0 msgr2=0x7f6c4403ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1353038297 2023-07-18T03:23:06.911 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:00.055+0000 7f6c5cba2700 1 --2- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f6c440386c0 0x7f6c4403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-18T03:23:06.911 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:00.456+0000 7f6c5cba2700 1 -- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f6c440386c0 msgr2=0x7f6c4403ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1353038297 2023-07-18T03:23:06.911 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:00.456+0000 7f6c5cba2700 1 --2- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f6c440386c0 0x7f6c4403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-18T03:23:06.911 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:01.257+0000 7f6c5cba2700 1 -- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f6c440386c0 msgr2=0x7f6c4403ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1353038297 2023-07-18T03:23:06.911 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:01.257+0000 7f6c5cba2700 1 --2- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f6c440386c0 0x7f6c4403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-18T03:23:06.912 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:02.858+0000 7f6c5cba2700 1 -- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f6c440386c0 msgr2=0x7f6c4403ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1353038297 2023-07-18T03:23:06.912 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:02.858+0000 7f6c5cba2700 1 --2- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f6c440386c0 0x7f6c4403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-18T03:23:06.912 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:05.484+0000 7f6c4e7fc700 1 -- 172.21.15.40:0/1573736304 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7f6c5401a430 con 0x7f6c58104280 2023-07-18T03:23:06.912 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:05.484+0000 7f6c4e7fc700 1 -- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f6c440386c0 msgr2=0x7f6c4403ab70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:23:06.912 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:05.484+0000 7f6c4e7fc700 1 --2- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f6c440386c0 0x7f6c4403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:06.912 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.485+0000 7f6c4e7fc700 1 -- 172.21.15.40:0/1573736304 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f6c5404db90 con 0x7f6c58104280 2023-07-18T03:23:06.912 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.485+0000 7f6c4e7fc700 1 --2- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f6c4403bfc0 0x7f6c4403e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:06.913 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.485+0000 7f6c4e7fc700 1 -- 172.21.15.40:0/1573736304 --> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6c4403b280 con 0x7f6c4403bfc0 2023-07-18T03:23:06.913 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.486+0000 7f6c5cba2700 1 --2- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f6c4403bfc0 0x7f6c4403e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:06.913 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.486+0000 7f6c5cba2700 1 --2- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f6c4403bfc0 0x7f6c4403e3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6c48003a10 tx=0x7f6c48009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:06.913 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.487+0000 7f6c4e7fc700 1 -- 172.21.15.40:0/1573736304 <== mgr.14120 v2:172.21.15.40:6800/1112125515 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f6c4403b280 con 0x7f6c4403bfc0 2023-07-18T03:23:06.913 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.498+0000 7f6c5f607700 1 -- 172.21.15.40:0/1573736304 --> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f6c5818efa0 con 0x7f6c4403bfc0 2023-07-18T03:23:06.913 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.498+0000 7f6c4e7fc700 1 -- 172.21.15.40:0/1573736304 <== mgr.14120 v2:172.21.15.40:6800/1112125515 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f6c5818efa0 con 0x7f6c4403bfc0 2023-07-18T03:23:06.913 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.498+0000 7f6c5f607700 1 -- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f6c4403bfc0 msgr2=0x7f6c4403e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:06.914 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.498+0000 7f6c5f607700 1 --2- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f6c4403bfc0 0x7f6c4403e3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6c48003a10 tx=0x7f6c48009290 comp rx=0 tx=0).stop 2023-07-18T03:23:06.914 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.498+0000 7f6c5f607700 1 -- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c58104280 msgr2=0x7f6c5818f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:06.914 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.498+0000 7f6c5f607700 1 --2- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c58104280 0x7f6c5818f400 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009fb0 tx=0x7f6c54011ad0 comp rx=0 tx=0).stop 2023-07-18T03:23:06.914 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.498+0000 7f6c5f607700 1 -- 172.21.15.40:0/1573736304 shutdown_connections 2023-07-18T03:23:06.914 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.498+0000 7f6c5f607700 1 --2- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f6c4403bfc0 0x7f6c4403e3a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:06.914 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.498+0000 7f6c5f607700 1 --2- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c58104280 0x7f6c5818f400 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:06.914 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.498+0000 7f6c5f607700 1 --2- 172.21.15.40:0/1573736304 >> [v2:172.21.15.40:6800/1353038297,v1:172.21.15.40:6801/1353038297] conn(0x7f6c440386c0 0x7f6c4403ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:06.914 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.499+0000 7f6c5f607700 1 -- 172.21.15.40:0/1573736304 >> 172.21.15.40:0/1573736304 conn(0x7f6c580ff8d0 msgr2=0x7f6c58072fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:06.915 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.499+0000 7f6c5f607700 1 -- 172.21.15.40:0/1573736304 shutdown_connections 2023-07-18T03:23:06.915 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:06.499+0000 7f6c5f607700 1 -- 172.21.15.40:0/1573736304 wait complete. 2023-07-18T03:23:06.915 INFO:teuthology.orchestra.run.smithi040.stdout:mgr epoch 5 is available 2023-07-18T03:23:06.915 INFO:teuthology.orchestra.run.smithi040.stdout:Setting orchestrator backend to cephadm... 2023-07-18T03:23:07.790 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:07 smithi040 ceph-mon[96439]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-18T03:23:07.790 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:07 smithi040 ceph-mon[96439]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-18T03:23:08.149 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.350+0000 7f881a063700 1 Processor -- start 2023-07-18T03:23:08.149 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.351+0000 7f881a063700 1 -- start start 2023-07-18T03:23:08.149 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.351+0000 7f881a063700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8814104280 0x7f8814104690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:08.149 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.351+0000 7f881a063700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8814104bd0 con 0x7f8814104280 2023-07-18T03:23:08.149 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.351+0000 7f88137fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8814104280 0x7f8814104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:08.150 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.351+0000 7f88137fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8814104280 0x7f8814104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54898/0 (socket says 172.21.15.40:54898) 2023-07-18T03:23:08.150 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.351+0000 7f88137fe700 1 -- 172.21.15.40:0/3558535785 learned_addr learned my addr 172.21.15.40:0/3558535785 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:08.150 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.352+0000 7f88137fe700 1 -- 172.21.15.40:0/3558535785 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8814104d10 con 0x7f8814104280 2023-07-18T03:23:08.150 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.352+0000 7f88137fe700 1 --2- 172.21.15.40:0/3558535785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8814104280 0x7f8814104690 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f88040098a0 tx=0x7f8804009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae097d6aa9ee7ffe server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:08.150 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.352+0000 7f88127fc700 1 -- 172.21.15.40:0/3558535785 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8804004160 con 0x7f8814104280 2023-07-18T03:23:08.150 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.352+0000 7f88127fc700 1 -- 172.21.15.40:0/3558535785 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f88040042c0 con 0x7f8814104280 2023-07-18T03:23:08.151 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.352+0000 7f88127fc700 1 -- 172.21.15.40:0/3558535785 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8804011a00 con 0x7f8814104280 2023-07-18T03:23:08.151 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.352+0000 7f881a063700 1 -- 172.21.15.40:0/3558535785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8814104280 msgr2=0x7f8814104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:08.151 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.352+0000 7f881a063700 1 --2- 172.21.15.40:0/3558535785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8814104280 0x7f8814104690 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f88040098a0 tx=0x7f8804009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:08.151 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.353+0000 7f881a063700 1 -- 172.21.15.40:0/3558535785 shutdown_connections 2023-07-18T03:23:08.151 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.353+0000 7f881a063700 1 --2- 172.21.15.40:0/3558535785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8814104280 0x7f8814104690 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:08.151 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.353+0000 7f881a063700 1 -- 172.21.15.40:0/3558535785 >> 172.21.15.40:0/3558535785 conn(0x7f88140ff8d0 msgr2=0x7f8814101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:08.151 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.353+0000 7f881a063700 1 -- 172.21.15.40:0/3558535785 shutdown_connections 2023-07-18T03:23:08.151 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.353+0000 7f881a063700 1 -- 172.21.15.40:0/3558535785 wait complete. 2023-07-18T03:23:08.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.353+0000 7f881a063700 1 Processor -- start 2023-07-18T03:23:08.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.353+0000 7f881a063700 1 -- start start 2023-07-18T03:23:08.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.354+0000 7f881a063700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8814104280 0x7f881418f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:08.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.354+0000 7f881a063700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881418f940 con 0x7f8814104280 2023-07-18T03:23:08.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.354+0000 7f88137fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8814104280 0x7f881418f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:08.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.354+0000 7f88137fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8814104280 0x7f881418f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54900/0 (socket says 172.21.15.40:54900) 2023-07-18T03:23:08.152 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.354+0000 7f88137fe700 1 -- 172.21.15.40:0/3841861097 learned_addr learned my addr 172.21.15.40:0/3841861097 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:08.153 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.354+0000 7f88137fe700 1 -- 172.21.15.40:0/3841861097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8804009550 con 0x7f8814104280 2023-07-18T03:23:08.153 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.354+0000 7f88137fe700 1 --2- 172.21.15.40:0/3841861097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8814104280 0x7f881418f400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8804006af0 tx=0x7f8804011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:08.153 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.354+0000 7f8810ff9700 1 -- 172.21.15.40:0/3841861097 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8804003a40 con 0x7f8814104280 2023-07-18T03:23:08.153 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.354+0000 7f8810ff9700 1 -- 172.21.15.40:0/3841861097 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8804003ba0 con 0x7f8814104280 2023-07-18T03:23:08.153 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.354+0000 7f881a063700 1 -- 172.21.15.40:0/3841861097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881418da50 con 0x7f8814104280 2023-07-18T03:23:08.153 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.354+0000 7f8810ff9700 1 -- 172.21.15.40:0/3841861097 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f880401a6e0 con 0x7f8814104280 2023-07-18T03:23:08.153 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.355+0000 7f881a063700 1 -- 172.21.15.40:0/3841861097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881418de70 con 0x7f8814104280 2023-07-18T03:23:08.154 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.355+0000 7f8810ff9700 1 -- 172.21.15.40:0/3841861097 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f880401a860 con 0x7f8814104280 2023-07-18T03:23:08.154 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.356+0000 7f8810ff9700 1 --2- 172.21.15.40:0/3841861097 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f87fc038680 0x7f87fc03ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:08.154 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.356+0000 7f8810ff9700 1 -- 172.21.15.40:0/3841861097 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f880404c7a0 con 0x7f8814104280 2023-07-18T03:23:08.154 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.356+0000 7f8812ffd700 1 --2- 172.21.15.40:0/3841861097 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f87fc038680 0x7f87fc03ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:08.155 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.356+0000 7f881a063700 1 -- 172.21.15.40:0/3841861097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8814061b40 con 0x7f8814104280 2023-07-18T03:23:08.155 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.356+0000 7f8812ffd700 1 --2- 172.21.15.40:0/3841861097 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f87fc038680 0x7f87fc03ab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8808006fd0 tx=0x7f8808006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:08.155 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.361+0000 7f8810ff9700 1 -- 172.21.15.40:0/3841861097 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f880401ab10 con 0x7f8814104280 2023-07-18T03:23:08.155 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.537+0000 7f8810ff9700 1 -- 172.21.15.40:0/3841861097 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f880401acf0 con 0x7f8814104280 2023-07-18T03:23:08.155 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.755+0000 7f881a063700 1 -- 172.21.15.40:0/3841861097 --> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f881418e7f0 con 0x7f87fc038680 2023-07-18T03:23:08.155 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.762+0000 7f8810ff9700 1 -- 172.21.15.40:0/3841861097 <== mgr.14120 v2:172.21.15.40:6800/1112125515 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f881418e7f0 con 0x7f87fc038680 2023-07-18T03:23:08.155 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.764+0000 7f881a063700 1 -- 172.21.15.40:0/3841861097 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f87fc038680 msgr2=0x7f87fc03ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:08.156 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.764+0000 7f881a063700 1 --2- 172.21.15.40:0/3841861097 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f87fc038680 0x7f87fc03ab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8808006fd0 tx=0x7f8808006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:08.156 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.764+0000 7f881a063700 1 -- 172.21.15.40:0/3841861097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8814104280 msgr2=0x7f881418f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:08.156 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.764+0000 7f881a063700 1 --2- 172.21.15.40:0/3841861097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8814104280 0x7f881418f400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8804006af0 tx=0x7f8804011ad0 comp rx=0 tx=0).stop 2023-07-18T03:23:08.156 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.764+0000 7f881a063700 1 -- 172.21.15.40:0/3841861097 shutdown_connections 2023-07-18T03:23:08.156 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.764+0000 7f881a063700 1 --2- 172.21.15.40:0/3841861097 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f87fc038680 0x7f87fc03ab30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:08.156 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.764+0000 7f881a063700 1 --2- 172.21.15.40:0/3841861097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8814104280 0x7f881418f400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:08.156 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.764+0000 7f881a063700 1 -- 172.21.15.40:0/3841861097 >> 172.21.15.40:0/3841861097 conn(0x7f88140ff8d0 msgr2=0x7f88140737c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:08.157 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.764+0000 7f881a063700 1 -- 172.21.15.40:0/3841861097 shutdown_connections 2023-07-18T03:23:08.157 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:07.765+0000 7f881a063700 1 -- 172.21.15.40:0/3841861097 wait complete. 2023-07-18T03:23:08.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:08 smithi040 ceph-mon[96439]: mgrmap e8: x(active, since 2s) 2023-07-18T03:23:08.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:08 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' 2023-07-18T03:23:08.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:08 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:23:08.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:08 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:23:09.495 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout value unchanged 2023-07-18T03:23:09.495 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.693+0000 7f7d08ad7700 1 Processor -- start 2023-07-18T03:23:09.495 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.693+0000 7f7d08ad7700 1 -- start start 2023-07-18T03:23:09.495 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.693+0000 7f7d08ad7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d04105380 0x7f7d04105790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:09.495 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.693+0000 7f7d08ad7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d04105cd0 con 0x7f7d04105380 2023-07-18T03:23:09.495 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.693+0000 7f7d0259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d04105380 0x7f7d04105790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:09.496 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.693+0000 7f7d0259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d04105380 0x7f7d04105790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54912/0 (socket says 172.21.15.40:54912) 2023-07-18T03:23:09.496 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.693+0000 7f7d0259c700 1 -- 172.21.15.40:0/3981773347 learned_addr learned my addr 172.21.15.40:0/3981773347 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:09.496 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.694+0000 7f7d0259c700 1 -- 172.21.15.40:0/3981773347 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d04105e10 con 0x7f7d04105380 2023-07-18T03:23:09.496 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.694+0000 7f7d0259c700 1 --2- 172.21.15.40:0/3981773347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d04105380 0x7f7d04105790 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40098a0 tx=0x7f7cf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ec740e4bbfbe77 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:09.496 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.694+0000 7f7d0159a700 1 -- 172.21.15.40:0/3981773347 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cf4004160 con 0x7f7d04105380 2023-07-18T03:23:09.496 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.694+0000 7f7d0159a700 1 -- 172.21.15.40:0/3981773347 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7cf40042c0 con 0x7f7d04105380 2023-07-18T03:23:09.496 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.694+0000 7f7d0159a700 1 -- 172.21.15.40:0/3981773347 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cf4011a00 con 0x7f7d04105380 2023-07-18T03:23:09.497 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.694+0000 7f7d08ad7700 1 -- 172.21.15.40:0/3981773347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d04105380 msgr2=0x7f7d04105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:09.497 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.694+0000 7f7d08ad7700 1 --2- 172.21.15.40:0/3981773347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d04105380 0x7f7d04105790 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40098a0 tx=0x7f7cf4009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:09.497 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.695+0000 7f7d08ad7700 1 -- 172.21.15.40:0/3981773347 shutdown_connections 2023-07-18T03:23:09.497 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.695+0000 7f7d08ad7700 1 --2- 172.21.15.40:0/3981773347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d04105380 0x7f7d04105790 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:09.497 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.695+0000 7f7d08ad7700 1 -- 172.21.15.40:0/3981773347 >> 172.21.15.40:0/3981773347 conn(0x7f7d041009f0 msgr2=0x7f7d04102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:09.497 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.695+0000 7f7d08ad7700 1 -- 172.21.15.40:0/3981773347 shutdown_connections 2023-07-18T03:23:09.497 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.695+0000 7f7d08ad7700 1 -- 172.21.15.40:0/3981773347 wait complete. 2023-07-18T03:23:09.498 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.695+0000 7f7d08ad7700 1 Processor -- start 2023-07-18T03:23:09.498 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.695+0000 7f7d08ad7700 1 -- start start 2023-07-18T03:23:09.498 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.696+0000 7f7d08ad7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d04105380 0x7f7d041990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:09.498 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.696+0000 7f7d08ad7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d04199610 con 0x7f7d04105380 2023-07-18T03:23:09.498 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.696+0000 7f7d0259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d04105380 0x7f7d041990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:09.498 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.696+0000 7f7d0259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d04105380 0x7f7d041990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54914/0 (socket says 172.21.15.40:54914) 2023-07-18T03:23:09.498 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.696+0000 7f7d0259c700 1 -- 172.21.15.40:0/1761921770 learned_addr learned my addr 172.21.15.40:0/1761921770 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:09.498 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.696+0000 7f7d0259c700 1 -- 172.21.15.40:0/1761921770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf4009550 con 0x7f7d04105380 2023-07-18T03:23:09.499 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.696+0000 7f7d0259c700 1 --2- 172.21.15.40:0/1761921770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d04105380 0x7f7d041990d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009fb0 tx=0x7f7cf4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:09.499 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.696+0000 7f7cf37fe700 1 -- 172.21.15.40:0/1761921770 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cf4003930 con 0x7f7d04105380 2023-07-18T03:23:09.499 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.696+0000 7f7cf37fe700 1 -- 172.21.15.40:0/1761921770 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7cf4003a90 con 0x7f7d04105380 2023-07-18T03:23:09.499 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.696+0000 7f7cf37fe700 1 -- 172.21.15.40:0/1761921770 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cf4019610 con 0x7f7d04105380 2023-07-18T03:23:09.499 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.696+0000 7f7d08ad7700 1 -- 172.21.15.40:0/1761921770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d04199810 con 0x7f7d04105380 2023-07-18T03:23:09.499 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.696+0000 7f7d08ad7700 1 -- 172.21.15.40:0/1761921770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d04199c30 con 0x7f7d04105380 2023-07-18T03:23:09.499 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.697+0000 7f7cf37fe700 1 -- 172.21.15.40:0/1761921770 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f7cf40198d0 con 0x7f7d04105380 2023-07-18T03:23:09.499 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.697+0000 7f7cf37fe700 1 --2- 172.21.15.40:0/1761921770 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f7cec0386c0 0x7f7cec03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:09.500 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.697+0000 7f7cf37fe700 1 -- 172.21.15.40:0/1761921770 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7cf404cae0 con 0x7f7d04105380 2023-07-18T03:23:09.500 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.698+0000 7f7d08ad7700 1 -- 172.21.15.40:0/1761921770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d04109810 con 0x7f7d04105380 2023-07-18T03:23:09.500 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.698+0000 7f7d01d9b700 1 --2- 172.21.15.40:0/1761921770 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f7cec0386c0 0x7f7cec03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:09.500 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.698+0000 7f7d01d9b700 1 --2- 172.21.15.40:0/1761921770 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f7cec0386c0 0x7f7cec03ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8006fd0 tx=0x7f7cf8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:09.500 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:08.703+0000 7f7cf37fe700 1 -- 172.21.15.40:0/1761921770 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7cf4019bd0 con 0x7f7d04105380 2023-07-18T03:23:09.500 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:09.061+0000 7f7d08ad7700 1 -- 172.21.15.40:0/1761921770 --> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d04102cb0 con 0x7f7cec0386c0 2023-07-18T03:23:09.500 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:09.062+0000 7f7cf37fe700 1 -- 172.21.15.40:0/1761921770 <== mgr.14120 v2:172.21.15.40:6800/1112125515 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f7d04102cb0 con 0x7f7cec0386c0 2023-07-18T03:23:09.501 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:09.064+0000 7f7d08ad7700 1 -- 172.21.15.40:0/1761921770 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f7cec0386c0 msgr2=0x7f7cec03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:09.501 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:09.064+0000 7f7d08ad7700 1 --2- 172.21.15.40:0/1761921770 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f7cec0386c0 0x7f7cec03ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8006fd0 tx=0x7f7cf8006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:09.501 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:09.064+0000 7f7d08ad7700 1 -- 172.21.15.40:0/1761921770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d04105380 msgr2=0x7f7d041990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:09.501 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:09.064+0000 7f7d08ad7700 1 --2- 172.21.15.40:0/1761921770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d04105380 0x7f7d041990d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009fb0 tx=0x7f7cf4011ad0 comp rx=0 tx=0).stop 2023-07-18T03:23:09.501 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:09.064+0000 7f7d08ad7700 1 -- 172.21.15.40:0/1761921770 shutdown_connections 2023-07-18T03:23:09.501 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:09.064+0000 7f7d08ad7700 1 --2- 172.21.15.40:0/1761921770 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f7cec0386c0 0x7f7cec03ab70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:09.501 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:09.064+0000 7f7d08ad7700 1 --2- 172.21.15.40:0/1761921770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d04105380 0x7f7d041990d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:09.502 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:09.064+0000 7f7d08ad7700 1 -- 172.21.15.40:0/1761921770 >> 172.21.15.40:0/1761921770 conn(0x7f7d041009f0 msgr2=0x7f7d04102050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:09.502 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:09.064+0000 7f7d08ad7700 1 -- 172.21.15.40:0/1761921770 shutdown_connections 2023-07-18T03:23:09.502 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:09.064+0000 7f7d08ad7700 1 -- 172.21.15.40:0/1761921770 wait complete. 2023-07-18T03:23:09.502 INFO:teuthology.orchestra.run.smithi040.stdout:Generating ssh key... 2023-07-18T03:23:09.789 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:09 smithi040 ceph-mon[96439]: [18/Jul/2023:03:23:07] ENGINE Bus STARTING 2023-07-18T03:23:09.789 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:09 smithi040 ceph-mon[96439]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:23:09.790 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:09 smithi040 ceph-mon[96439]: [18/Jul/2023:03:23:07] ENGINE Serving on https://172.21.15.40:7150 2023-07-18T03:23:09.790 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:09 smithi040 ceph-mon[96439]: [18/Jul/2023:03:23:07] ENGINE Serving on http://172.21.15.40:8765 2023-07-18T03:23:09.790 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:09 smithi040 ceph-mon[96439]: [18/Jul/2023:03:23:07] ENGINE Bus STARTED 2023-07-18T03:23:10.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-mon[96439]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:23:10.829 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: Generating public/private rsa key pair. 2023-07-18T03:23:10.830 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: Your identification has been saved in /tmp/tmp3zpsh7r9/key. 2023-07-18T03:23:10.830 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: Your public key has been saved in /tmp/tmp3zpsh7r9/key.pub. 2023-07-18T03:23:10.830 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: The key fingerprint is: 2023-07-18T03:23:10.831 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: SHA256:7Zo+wq7NdnIxSdcoMyp5bMc4bDjXfSmFRGm7XGNw9lQ ceph-3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:23:10.831 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: The key's randomart image is: 2023-07-18T03:23:10.831 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: +---[RSA 3072]----+ 2023-07-18T03:23:10.832 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: | ... .E| 2023-07-18T03:23:10.832 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: | = o . | 2023-07-18T03:23:10.832 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: | o O o | 2023-07-18T03:23:10.832 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: | =.= * . | 2023-07-18T03:23:10.832 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: | = *SO.= o | 2023-07-18T03:23:10.833 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: | = @ B.= o | 2023-07-18T03:23:10.833 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: | O o o.o | 2023-07-18T03:23:10.833 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: | o= +o | 2023-07-18T03:23:10.833 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: | .++*+. | 2023-07-18T03:23:10.834 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:10 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: +----[SHA256]-----+ 2023-07-18T03:23:10.942 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.058+0000 7f923e33b700 1 Processor -- start 2023-07-18T03:23:10.942 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.058+0000 7f923e33b700 1 -- start start 2023-07-18T03:23:10.943 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.058+0000 7f923e33b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9238079ce0 0x7f923810a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:10.943 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.058+0000 7f923e33b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9238105cf0 con 0x7f9238079ce0 2023-07-18T03:23:10.943 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.058+0000 7f9237fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9238079ce0 0x7f923810a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:10.944 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.058+0000 7f9237fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9238079ce0 0x7f923810a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54918/0 (socket says 172.21.15.40:54918) 2023-07-18T03:23:10.944 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.058+0000 7f9237fff700 1 -- 172.21.15.40:0/3065733843 learned_addr learned my addr 172.21.15.40:0/3065733843 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:10.944 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.058+0000 7f9237fff700 1 -- 172.21.15.40:0/3065733843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923810a840 con 0x7f9238079ce0 2023-07-18T03:23:10.944 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.059+0000 7f9237fff700 1 --2- 172.21.15.40:0/3065733843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9238079ce0 0x7f923810a300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f92280098a0 tx=0x7f9228009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76ee793a6932aaa server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:10.944 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.059+0000 7f9236ffd700 1 -- 172.21.15.40:0/3065733843 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9228004160 con 0x7f9238079ce0 2023-07-18T03:23:10.945 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.059+0000 7f9236ffd700 1 -- 172.21.15.40:0/3065733843 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f92280042c0 con 0x7f9238079ce0 2023-07-18T03:23:10.945 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.059+0000 7f9236ffd700 1 -- 172.21.15.40:0/3065733843 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9228011a00 con 0x7f9238079ce0 2023-07-18T03:23:10.945 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.059+0000 7f923e33b700 1 -- 172.21.15.40:0/3065733843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9238079ce0 msgr2=0x7f923810a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:10.945 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.059+0000 7f923e33b700 1 --2- 172.21.15.40:0/3065733843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9238079ce0 0x7f923810a300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f92280098a0 tx=0x7f9228009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:10.945 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.059+0000 7f923e33b700 1 -- 172.21.15.40:0/3065733843 shutdown_connections 2023-07-18T03:23:10.945 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.059+0000 7f923e33b700 1 --2- 172.21.15.40:0/3065733843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9238079ce0 0x7f923810a300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:10.945 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.059+0000 7f923e33b700 1 -- 172.21.15.40:0/3065733843 >> 172.21.15.40:0/3065733843 conn(0x7f92380ff8d0 msgr2=0x7f9238101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:10.946 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.059+0000 7f923e33b700 1 -- 172.21.15.40:0/3065733843 shutdown_connections 2023-07-18T03:23:10.946 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.060+0000 7f923e33b700 1 -- 172.21.15.40:0/3065733843 wait complete. 2023-07-18T03:23:10.946 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.060+0000 7f923e33b700 1 Processor -- start 2023-07-18T03:23:10.946 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.060+0000 7f923e33b700 1 -- start start 2023-07-18T03:23:10.946 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.061+0000 7f923e33b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923819b2b0 0x7f923819b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:10.946 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.061+0000 7f923e33b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923819bc00 con 0x7f923819b2b0 2023-07-18T03:23:10.947 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.061+0000 7f9237fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923819b2b0 0x7f923819b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:10.947 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.061+0000 7f9237fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923819b2b0 0x7f923819b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54920/0 (socket says 172.21.15.40:54920) 2023-07-18T03:23:10.947 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.061+0000 7f9237fff700 1 -- 172.21.15.40:0/3491288517 learned_addr learned my addr 172.21.15.40:0/3491288517 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:10.947 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.061+0000 7f9237fff700 1 -- 172.21.15.40:0/3491288517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9228009550 con 0x7f923819b2b0 2023-07-18T03:23:10.947 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.061+0000 7f9237fff700 1 --2- 172.21.15.40:0/3491288517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923819b2b0 0x7f923819b6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9228006af0 tx=0x7f9228011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:10.947 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.061+0000 7f92357fa700 1 -- 172.21.15.40:0/3491288517 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9228003a40 con 0x7f923819b2b0 2023-07-18T03:23:10.947 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.061+0000 7f923e33b700 1 -- 172.21.15.40:0/3491288517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f923819be00 con 0x7f923819b2b0 2023-07-18T03:23:10.948 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.061+0000 7f92357fa700 1 -- 172.21.15.40:0/3491288517 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9228003ba0 con 0x7f923819b2b0 2023-07-18T03:23:10.948 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.062+0000 7f92357fa700 1 -- 172.21.15.40:0/3491288517 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f922801a700 con 0x7f923819b2b0 2023-07-18T03:23:10.948 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.062+0000 7f923e33b700 1 -- 172.21.15.40:0/3491288517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923819ea60 con 0x7f923819b2b0 2023-07-18T03:23:10.948 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.062+0000 7f92357fa700 1 -- 172.21.15.40:0/3491288517 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f922801a980 con 0x7f923819b2b0 2023-07-18T03:23:10.948 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.063+0000 7f92357fa700 1 --2- 172.21.15.40:0/3491288517 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f9220038710 0x7f922003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:10.948 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.063+0000 7f92357fa700 1 -- 172.21.15.40:0/3491288517 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f922804c6f0 con 0x7f923819b2b0 2023-07-18T03:23:10.948 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.063+0000 7f92377fe700 1 --2- 172.21.15.40:0/3491288517 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f9220038710 0x7f922003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:10.949 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.063+0000 7f92377fe700 1 --2- 172.21.15.40:0/3491288517 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f9220038710 0x7f922003abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f922c006fd0 tx=0x7f922c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:10.949 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.063+0000 7f923e33b700 1 -- 172.21.15.40:0/3491288517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9238061b40 con 0x7f923819b2b0 2023-07-18T03:23:10.949 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.068+0000 7f92357fa700 1 -- 172.21.15.40:0/3491288517 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9228023390 con 0x7f923819b2b0 2023-07-18T03:23:10.949 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.426+0000 7f923e33b700 1 -- 172.21.15.40:0/3491288517 --> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f9238101bc0 con 0x7f9220038710 2023-07-18T03:23:10.949 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.555+0000 7f92357fa700 1 -- 172.21.15.40:0/3491288517 <== mgr.14120 v2:172.21.15.40:6800/1112125515 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9238101bc0 con 0x7f9220038710 2023-07-18T03:23:10.949 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.557+0000 7f923e33b700 1 -- 172.21.15.40:0/3491288517 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f9220038710 msgr2=0x7f922003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:10.949 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.557+0000 7f923e33b700 1 --2- 172.21.15.40:0/3491288517 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f9220038710 0x7f922003abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f922c006fd0 tx=0x7f922c006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:10.950 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.557+0000 7f923e33b700 1 -- 172.21.15.40:0/3491288517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923819b2b0 msgr2=0x7f923819b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:10.950 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.557+0000 7f923e33b700 1 --2- 172.21.15.40:0/3491288517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923819b2b0 0x7f923819b6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9228006af0 tx=0x7f9228011ad0 comp rx=0 tx=0).stop 2023-07-18T03:23:10.950 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.557+0000 7f923e33b700 1 -- 172.21.15.40:0/3491288517 shutdown_connections 2023-07-18T03:23:10.950 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.557+0000 7f923e33b700 1 --2- 172.21.15.40:0/3491288517 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f9220038710 0x7f922003abc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:10.950 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.557+0000 7f923e33b700 1 --2- 172.21.15.40:0/3491288517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f923819b2b0 0x7f923819b6c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:10.950 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.557+0000 7f923e33b700 1 -- 172.21.15.40:0/3491288517 >> 172.21.15.40:0/3491288517 conn(0x7f92380ff8d0 msgr2=0x7f9238100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:10.950 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.557+0000 7f923e33b700 1 -- 172.21.15.40:0/3491288517 shutdown_connections 2023-07-18T03:23:10.950 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:10.557+0000 7f923e33b700 1 -- 172.21.15.40:0/3491288517 wait complete. 2023-07-18T03:23:11.823 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:11 smithi040 ceph-mon[96439]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:23:11.823 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:11 smithi040 ceph-mon[96439]: Generating ssh key... 2023-07-18T03:23:11.823 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:11 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' 2023-07-18T03:23:11.823 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:11 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' 2023-07-18T03:23:12.284 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZMp7JKokJelbqW4Vgy8MiDhicnmIkxl2sVzIsSkUNwlmOUDo8quH/haUHYaou75IXaBcwTZCUdPBjPROyYS4c09r/whjxEpeE3xfKS1Ua+lhUdUmUV+gJj41Jmx1xXHKm4zS9ZXRkfbQGAyQVG/sQn9v3vDlOMqxMXgsrjSnG5u19YZjx3QcQM9ixY6Fa0B2X+G+jONQWipBE3zQHr1pdNsCAPPNcnQElJvzSWOSLLag0NSUb2vNmcTdGKTRxShooiZo9TBK5m2c96IvBclw1Q9iXtU44ot+D52aigoeWIYBLhaTohz69ezIC/kYSye03eYerLq9e8GHeHmitRV8hMF8E6WNO2iWN95LJqojOLvIzfeusweVAwyixzokvuOe5osrzvS/Q3Wi/CA7fqwpIza9mzkJ7HzSkyV65IrutPcfeNtLwzT9UJPSl6XeFgDhLJDMd8QhkWvleTSQiSd4vgPrYy7hZRFp/qkTdoUU0u94f+Ul5ab+o1w5Da0jjXX0= ceph-3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:23:12.284 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.431+0000 7fe61d467700 1 Processor -- start 2023-07-18T03:23:12.284 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.431+0000 7fe61d467700 1 -- start start 2023-07-18T03:23:12.284 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.431+0000 7fe61d467700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe618105380 0x7fe618105790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:12.284 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.432+0000 7fe61d467700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe618105cd0 con 0x7fe618105380 2023-07-18T03:23:12.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.432+0000 7fe616ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe618105380 0x7fe618105790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:12.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.432+0000 7fe616ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe618105380 0x7fe618105790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54924/0 (socket says 172.21.15.40:54924) 2023-07-18T03:23:12.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.432+0000 7fe616ffd700 1 -- 172.21.15.40:0/3364702250 learned_addr learned my addr 172.21.15.40:0/3364702250 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:12.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.432+0000 7fe616ffd700 1 -- 172.21.15.40:0/3364702250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe618105e10 con 0x7fe618105380 2023-07-18T03:23:12.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.432+0000 7fe616ffd700 1 --2- 172.21.15.40:0/3364702250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe618105380 0x7fe618105790 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe6080098a0 tx=0x7fe608009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b792434eeb957f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:12.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.432+0000 7fe615ffb700 1 -- 172.21.15.40:0/3364702250 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe608004160 con 0x7fe618105380 2023-07-18T03:23:12.285 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.432+0000 7fe615ffb700 1 -- 172.21.15.40:0/3364702250 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe6080042c0 con 0x7fe618105380 2023-07-18T03:23:12.286 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.432+0000 7fe615ffb700 1 -- 172.21.15.40:0/3364702250 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe608011a00 con 0x7fe618105380 2023-07-18T03:23:12.286 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.433+0000 7fe61d467700 1 -- 172.21.15.40:0/3364702250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe618105380 msgr2=0x7fe618105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:12.286 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.433+0000 7fe61d467700 1 --2- 172.21.15.40:0/3364702250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe618105380 0x7fe618105790 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe6080098a0 tx=0x7fe608009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:12.286 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.433+0000 7fe61d467700 1 -- 172.21.15.40:0/3364702250 shutdown_connections 2023-07-18T03:23:12.286 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.433+0000 7fe61d467700 1 --2- 172.21.15.40:0/3364702250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe618105380 0x7fe618105790 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:12.286 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.433+0000 7fe61d467700 1 -- 172.21.15.40:0/3364702250 >> 172.21.15.40:0/3364702250 conn(0x7fe6181009f0 msgr2=0x7fe618102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:12.286 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.433+0000 7fe61d467700 1 -- 172.21.15.40:0/3364702250 shutdown_connections 2023-07-18T03:23:12.286 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.433+0000 7fe61d467700 1 -- 172.21.15.40:0/3364702250 wait complete. 2023-07-18T03:23:12.287 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.434+0000 7fe61d467700 1 Processor -- start 2023-07-18T03:23:12.287 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.434+0000 7fe61d467700 1 -- start start 2023-07-18T03:23:12.287 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.434+0000 7fe61d467700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe618105380 0x7fe61807ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:12.287 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.434+0000 7fe61d467700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61807b060 con 0x7fe618105380 2023-07-18T03:23:12.287 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.435+0000 7fe616ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe618105380 0x7fe61807ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:12.287 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.435+0000 7fe616ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe618105380 0x7fe61807ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54926/0 (socket says 172.21.15.40:54926) 2023-07-18T03:23:12.287 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.435+0000 7fe616ffd700 1 -- 172.21.15.40:0/3606015787 learned_addr learned my addr 172.21.15.40:0/3606015787 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:12.288 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.435+0000 7fe616ffd700 1 -- 172.21.15.40:0/3606015787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe608009550 con 0x7fe618105380 2023-07-18T03:23:12.288 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.435+0000 7fe616ffd700 1 --2- 172.21.15.40:0/3606015787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe618105380 0x7fe61807ab20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe608009fb0 tx=0x7fe608011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:12.288 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.435+0000 7fe60ffff700 1 -- 172.21.15.40:0/3606015787 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe608003930 con 0x7fe618105380 2023-07-18T03:23:12.288 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.435+0000 7fe60ffff700 1 -- 172.21.15.40:0/3606015787 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe608003a90 con 0x7fe618105380 2023-07-18T03:23:12.288 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.435+0000 7fe60ffff700 1 -- 172.21.15.40:0/3606015787 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe608019610 con 0x7fe618105380 2023-07-18T03:23:12.288 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.435+0000 7fe61d467700 1 -- 172.21.15.40:0/3606015787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe618079170 con 0x7fe618105380 2023-07-18T03:23:12.288 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.435+0000 7fe61d467700 1 -- 172.21.15.40:0/3606015787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe618079590 con 0x7fe618105380 2023-07-18T03:23:12.288 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.436+0000 7fe60ffff700 1 -- 172.21.15.40:0/3606015787 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fe6080198d0 con 0x7fe618105380 2023-07-18T03:23:12.289 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.436+0000 7fe60ffff700 1 --2- 172.21.15.40:0/3606015787 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7fe6040386c0 0x7fe60403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:12.289 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.436+0000 7fe61d467700 1 -- 172.21.15.40:0/3606015787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe618061b40 con 0x7fe618105380 2023-07-18T03:23:12.289 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.436+0000 7fe60ffff700 1 -- 172.21.15.40:0/3606015787 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe60804caf0 con 0x7fe618105380 2023-07-18T03:23:12.289 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.437+0000 7fe6167fc700 1 --2- 172.21.15.40:0/3606015787 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7fe6040386c0 0x7fe60403ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:12.289 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.437+0000 7fe6167fc700 1 --2- 172.21.15.40:0/3606015787 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7fe6040386c0 0x7fe60403ab70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe600006fd0 tx=0x7fe600006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:12.289 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.441+0000 7fe60ffff700 1 -- 172.21.15.40:0/3606015787 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe608018530 con 0x7fe618105380 2023-07-18T03:23:12.289 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.801+0000 7fe61d467700 1 -- 172.21.15.40:0/3606015787 --> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fe618102cb0 con 0x7fe6040386c0 2023-07-18T03:23:12.290 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.802+0000 7fe60ffff700 1 -- 172.21.15.40:0/3606015787 <== mgr.14120 v2:172.21.15.40:6800/1112125515 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fe618102cb0 con 0x7fe6040386c0 2023-07-18T03:23:12.290 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.804+0000 7fe61d467700 1 -- 172.21.15.40:0/3606015787 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7fe6040386c0 msgr2=0x7fe60403ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:12.290 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.804+0000 7fe61d467700 1 --2- 172.21.15.40:0/3606015787 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7fe6040386c0 0x7fe60403ab70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe600006fd0 tx=0x7fe600006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:12.290 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.804+0000 7fe61d467700 1 -- 172.21.15.40:0/3606015787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe618105380 msgr2=0x7fe61807ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:12.290 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.804+0000 7fe61d467700 1 --2- 172.21.15.40:0/3606015787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe618105380 0x7fe61807ab20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe608009fb0 tx=0x7fe608011ad0 comp rx=0 tx=0).stop 2023-07-18T03:23:12.290 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.805+0000 7fe61d467700 1 -- 172.21.15.40:0/3606015787 shutdown_connections 2023-07-18T03:23:12.290 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.805+0000 7fe61d467700 1 --2- 172.21.15.40:0/3606015787 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7fe6040386c0 0x7fe60403ab70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:12.290 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.805+0000 7fe61d467700 1 --2- 172.21.15.40:0/3606015787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe618105380 0x7fe61807ab20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:12.291 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.805+0000 7fe61d467700 1 -- 172.21.15.40:0/3606015787 >> 172.21.15.40:0/3606015787 conn(0x7fe6181009f0 msgr2=0x7fe618102050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:12.291 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.805+0000 7fe61d467700 1 -- 172.21.15.40:0/3606015787 shutdown_connections 2023-07-18T03:23:12.291 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:11.805+0000 7fe61d467700 1 -- 172.21.15.40:0/3606015787 wait complete. 2023-07-18T03:23:12.291 INFO:teuthology.orchestra.run.smithi040.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-07-18T03:23:12.291 INFO:teuthology.orchestra.run.smithi040.stdout:Adding key to root@localhost authorized_keys... 2023-07-18T03:23:12.292 INFO:teuthology.orchestra.run.smithi040.stdout:Adding host smithi040... 2023-07-18T03:23:12.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:12 smithi040 ceph-mon[96439]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:23:13.694 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:13 smithi040 ceph-mon[96439]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi040", "addr": "172.21.15.40", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:23:15.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:14 smithi040 ceph-mon[96439]: Deploying cephadm binary to smithi040 2023-07-18T03:23:16.340 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout Added host 'smithi040' with addr '172.21.15.40' 2023-07-18T03:23:16.341 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.785+0000 7f3f0aee9700 1 Processor -- start 2023-07-18T03:23:16.341 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.786+0000 7f3f0aee9700 1 -- start start 2023-07-18T03:23:16.341 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.786+0000 7f3f0aee9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f04105380 0x7f3f04105790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:16.342 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.786+0000 7f3f0aee9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f04105cd0 con 0x7f3f04105380 2023-07-18T03:23:16.342 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.786+0000 7f3f08c85700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f04105380 0x7f3f04105790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:16.342 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.786+0000 7f3f08c85700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f04105380 0x7f3f04105790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54930/0 (socket says 172.21.15.40:54930) 2023-07-18T03:23:16.343 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.786+0000 7f3f08c85700 1 -- 172.21.15.40:0/3985221169 learned_addr learned my addr 172.21.15.40:0/3985221169 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:16.343 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.786+0000 7f3f08c85700 1 -- 172.21.15.40:0/3985221169 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f04105e10 con 0x7f3f04105380 2023-07-18T03:23:16.343 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.787+0000 7f3f08c85700 1 --2- 172.21.15.40:0/3985221169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f04105380 0x7f3f04105790 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80098a0 tx=0x7f3ef8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c77b373fb70a792 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:16.343 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.787+0000 7f3f037fe700 1 -- 172.21.15.40:0/3985221169 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ef8004160 con 0x7f3f04105380 2023-07-18T03:23:16.344 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.787+0000 7f3f037fe700 1 -- 172.21.15.40:0/3985221169 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3ef80042c0 con 0x7f3f04105380 2023-07-18T03:23:16.344 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.787+0000 7f3f037fe700 1 -- 172.21.15.40:0/3985221169 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ef8011a00 con 0x7f3f04105380 2023-07-18T03:23:16.344 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.787+0000 7f3f0aee9700 1 -- 172.21.15.40:0/3985221169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f04105380 msgr2=0x7f3f04105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:16.345 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.787+0000 7f3f0aee9700 1 --2- 172.21.15.40:0/3985221169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f04105380 0x7f3f04105790 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80098a0 tx=0x7f3ef8009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:16.345 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.787+0000 7f3f0aee9700 1 -- 172.21.15.40:0/3985221169 shutdown_connections 2023-07-18T03:23:16.345 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.787+0000 7f3f0aee9700 1 --2- 172.21.15.40:0/3985221169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f04105380 0x7f3f04105790 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:16.346 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.787+0000 7f3f0aee9700 1 -- 172.21.15.40:0/3985221169 >> 172.21.15.40:0/3985221169 conn(0x7f3f041009f0 msgr2=0x7f3f04102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:16.346 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.788+0000 7f3f0aee9700 1 -- 172.21.15.40:0/3985221169 shutdown_connections 2023-07-18T03:23:16.346 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.788+0000 7f3f0aee9700 1 -- 172.21.15.40:0/3985221169 wait complete. 2023-07-18T03:23:16.346 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.788+0000 7f3f0aee9700 1 Processor -- start 2023-07-18T03:23:16.346 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.788+0000 7f3f0aee9700 1 -- start start 2023-07-18T03:23:16.347 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.788+0000 7f3f0aee9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f04105380 0x7f3f0407ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:16.347 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.788+0000 7f3f0aee9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0407b060 con 0x7f3f04105380 2023-07-18T03:23:16.347 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.788+0000 7f3f08c85700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f04105380 0x7f3f0407ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:16.347 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.788+0000 7f3f08c85700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f04105380 0x7f3f0407ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54932/0 (socket says 172.21.15.40:54932) 2023-07-18T03:23:16.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.788+0000 7f3f08c85700 1 -- 172.21.15.40:0/2477972791 learned_addr learned my addr 172.21.15.40:0/2477972791 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:16.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.789+0000 7f3f08c85700 1 -- 172.21.15.40:0/2477972791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef8009550 con 0x7f3f04105380 2023-07-18T03:23:16.348 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.789+0000 7f3f08c85700 1 --2- 172.21.15.40:0/2477972791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f04105380 0x7f3f0407ab20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009fb0 tx=0x7f3ef8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:16.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.789+0000 7f3f01ffb700 1 -- 172.21.15.40:0/2477972791 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ef8003930 con 0x7f3f04105380 2023-07-18T03:23:16.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.789+0000 7f3f01ffb700 1 -- 172.21.15.40:0/2477972791 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3ef8003a90 con 0x7f3f04105380 2023-07-18T03:23:16.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.789+0000 7f3f0aee9700 1 -- 172.21.15.40:0/2477972791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f04079170 con 0x7f3f04105380 2023-07-18T03:23:16.349 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.789+0000 7f3f01ffb700 1 -- 172.21.15.40:0/2477972791 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ef8019610 con 0x7f3f04105380 2023-07-18T03:23:16.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.789+0000 7f3f0aee9700 1 -- 172.21.15.40:0/2477972791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f04079590 con 0x7f3f04105380 2023-07-18T03:23:16.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.790+0000 7f3f01ffb700 1 -- 172.21.15.40:0/2477972791 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f3ef80198d0 con 0x7f3f04105380 2023-07-18T03:23:16.350 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.790+0000 7f3f01ffb700 1 --2- 172.21.15.40:0/2477972791 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f3ef40386c0 0x7f3ef403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:16.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.790+0000 7f3f01ffb700 1 -- 172.21.15.40:0/2477972791 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3ef804c950 con 0x7f3f04105380 2023-07-18T03:23:16.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.790+0000 7f3f0aee9700 1 -- 172.21.15.40:0/2477972791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f04061b40 con 0x7f3f04105380 2023-07-18T03:23:16.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.791+0000 7f3f03fff700 1 --2- 172.21.15.40:0/2477972791 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f3ef40386c0 0x7f3ef403ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:16.351 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.791+0000 7f3f03fff700 1 --2- 172.21.15.40:0/2477972791 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f3ef40386c0 0x7f3ef403ab70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0006fd0 tx=0x7f3ef0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:16.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:12.795+0000 7f3f01ffb700 1 -- 172.21.15.40:0/2477972791 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3ef8019bd0 con 0x7f3f04105380 2023-07-18T03:23:16.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:13.158+0000 7f3f0aee9700 1 -- 172.21.15.40:0/2477972791 --> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi040", "addr": "172.21.15.40", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f04079fa0 con 0x7f3ef40386c0 2023-07-18T03:23:16.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:15.902+0000 7f3f01ffb700 1 -- 172.21.15.40:0/2477972791 <== mgr.14120 v2:172.21.15.40:6800/1112125515 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f3f04079fa0 con 0x7f3ef40386c0 2023-07-18T03:23:16.352 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:15.905+0000 7f3f0aee9700 1 -- 172.21.15.40:0/2477972791 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f3ef40386c0 msgr2=0x7f3ef403ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:16.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:15.905+0000 7f3f0aee9700 1 --2- 172.21.15.40:0/2477972791 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f3ef40386c0 0x7f3ef403ab70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0006fd0 tx=0x7f3ef0006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:16.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:15.905+0000 7f3f0aee9700 1 -- 172.21.15.40:0/2477972791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f04105380 msgr2=0x7f3f0407ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:16.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:15.905+0000 7f3f0aee9700 1 --2- 172.21.15.40:0/2477972791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f04105380 0x7f3f0407ab20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009fb0 tx=0x7f3ef8011ad0 comp rx=0 tx=0).stop 2023-07-18T03:23:16.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:15.905+0000 7f3f0aee9700 1 -- 172.21.15.40:0/2477972791 shutdown_connections 2023-07-18T03:23:16.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:15.905+0000 7f3f0aee9700 1 --2- 172.21.15.40:0/2477972791 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f3ef40386c0 0x7f3ef403ab70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:16.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:15.905+0000 7f3f0aee9700 1 --2- 172.21.15.40:0/2477972791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3f04105380 0x7f3f0407ab20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:16.353 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:15.905+0000 7f3f0aee9700 1 -- 172.21.15.40:0/2477972791 >> 172.21.15.40:0/2477972791 conn(0x7f3f041009f0 msgr2=0x7f3f04102050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:16.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:15.905+0000 7f3f0aee9700 1 -- 172.21.15.40:0/2477972791 shutdown_connections 2023-07-18T03:23:16.354 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:15.905+0000 7f3f0aee9700 1 -- 172.21.15.40:0/2477972791 wait complete. 2023-07-18T03:23:16.354 INFO:teuthology.orchestra.run.smithi040.stdout:Deploying unmanaged mon service... 2023-07-18T03:23:17.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:16 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' 2023-07-18T03:23:17.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:16 smithi040 ceph-mon[96439]: Added host smithi040 2023-07-18T03:23:17.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:16 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:23:18.087 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-07-18T03:23:18.088 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.995+0000 7f0079287700 1 Processor -- start 2023-07-18T03:23:18.088 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.995+0000 7f0079287700 1 -- start start 2023-07-18T03:23:18.088 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.995+0000 7f0079287700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0074105ed0 0x7f00741082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:18.088 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.995+0000 7f0079287700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0074073be0 con 0x7f0074105ed0 2023-07-18T03:23:18.088 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.995+0000 7f0072ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0074105ed0 0x7f00741082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:18.089 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.995+0000 7f0072ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0074105ed0 0x7f00741082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54966/0 (socket says 172.21.15.40:54966) 2023-07-18T03:23:18.089 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.995+0000 7f0072ffd700 1 -- 172.21.15.40:0/2691530584 learned_addr learned my addr 172.21.15.40:0/2691530584 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:18.089 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.996+0000 7f0072ffd700 1 -- 172.21.15.40:0/2691530584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0074073d20 con 0x7f0074105ed0 2023-07-18T03:23:18.089 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.996+0000 7f0072ffd700 1 --2- 172.21.15.40:0/2691530584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0074105ed0 0x7f00741082b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f00640098a0 tx=0x7f0064009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f54db4cdac1233b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:18.089 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.996+0000 7f0071ffb700 1 -- 172.21.15.40:0/2691530584 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0064004160 con 0x7f0074105ed0 2023-07-18T03:23:18.089 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.996+0000 7f0071ffb700 1 -- 172.21.15.40:0/2691530584 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f00640042c0 con 0x7f0074105ed0 2023-07-18T03:23:18.089 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.996+0000 7f0071ffb700 1 -- 172.21.15.40:0/2691530584 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0064011a00 con 0x7f0074105ed0 2023-07-18T03:23:18.090 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.997+0000 7f0079287700 1 -- 172.21.15.40:0/2691530584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0074105ed0 msgr2=0x7f00741082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:18.090 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.997+0000 7f0079287700 1 --2- 172.21.15.40:0/2691530584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0074105ed0 0x7f00741082b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f00640098a0 tx=0x7f0064009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:18.090 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.997+0000 7f0079287700 1 -- 172.21.15.40:0/2691530584 shutdown_connections 2023-07-18T03:23:18.090 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.997+0000 7f0079287700 1 --2- 172.21.15.40:0/2691530584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0074105ed0 0x7f00741082b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:18.090 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.997+0000 7f0079287700 1 -- 172.21.15.40:0/2691530584 >> 172.21.15.40:0/2691530584 conn(0x7f00740ff8d0 msgr2=0x7f0074101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:18.090 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.997+0000 7f0079287700 1 -- 172.21.15.40:0/2691530584 shutdown_connections 2023-07-18T03:23:18.090 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.997+0000 7f0079287700 1 -- 172.21.15.40:0/2691530584 wait complete. 2023-07-18T03:23:18.091 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.997+0000 7f0079287700 1 Processor -- start 2023-07-18T03:23:18.091 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.998+0000 7f0079287700 1 -- start start 2023-07-18T03:23:18.091 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.998+0000 7f0079287700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0074105ed0 0x7f0074105830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:18.091 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.998+0000 7f0079287700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0074105d70 con 0x7f0074105ed0 2023-07-18T03:23:18.091 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.998+0000 7f0072ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0074105ed0 0x7f0074105830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:18.091 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.998+0000 7f0072ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0074105ed0 0x7f0074105830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54968/0 (socket says 172.21.15.40:54968) 2023-07-18T03:23:18.091 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.998+0000 7f0072ffd700 1 -- 172.21.15.40:0/106817476 learned_addr learned my addr 172.21.15.40:0/106817476 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:18.091 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.998+0000 7f0072ffd700 1 -- 172.21.15.40:0/106817476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0064009550 con 0x7f0074105ed0 2023-07-18T03:23:18.092 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.999+0000 7f0072ffd700 1 --2- 172.21.15.40:0/106817476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0074105ed0 0x7f0074105830 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0064006af0 tx=0x7f0064011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:18.092 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.999+0000 7f005bfff700 1 -- 172.21.15.40:0/106817476 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0064003a40 con 0x7f0074105ed0 2023-07-18T03:23:18.092 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.999+0000 7f005bfff700 1 -- 172.21.15.40:0/106817476 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0064003ba0 con 0x7f0074105ed0 2023-07-18T03:23:18.092 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.999+0000 7f005bfff700 1 -- 172.21.15.40:0/106817476 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f006401a700 con 0x7f0074105ed0 2023-07-18T03:23:18.092 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.999+0000 7f0079287700 1 -- 172.21.15.40:0/106817476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0074103ee0 con 0x7f0074105ed0 2023-07-18T03:23:18.092 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:16.999+0000 7f0079287700 1 -- 172.21.15.40:0/106817476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0074104360 con 0x7f0074105ed0 2023-07-18T03:23:18.093 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.000+0000 7f005bfff700 1 -- 172.21.15.40:0/106817476 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f006401a930 con 0x7f0074105ed0 2023-07-18T03:23:18.093 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.000+0000 7f0079287700 1 -- 172.21.15.40:0/106817476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00741093f0 con 0x7f0074105ed0 2023-07-18T03:23:18.093 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.000+0000 7f005bfff700 1 --2- 172.21.15.40:0/106817476 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f005c038760 0x7f005c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:18.093 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.000+0000 7f005bfff700 1 -- 172.21.15.40:0/106817476 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f006404c670 con 0x7f0074105ed0 2023-07-18T03:23:18.093 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.001+0000 7f00727fc700 1 --2- 172.21.15.40:0/106817476 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f005c038760 0x7f005c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:18.093 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.001+0000 7f00727fc700 1 --2- 172.21.15.40:0/106817476 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f005c038760 0x7f005c03ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0068006fd0 tx=0x7f0068006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:18.093 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.006+0000 7f005bfff700 1 -- 172.21.15.40:0/106817476 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f006401abe0 con 0x7f0074105ed0 2023-07-18T03:23:18.093 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.393+0000 7f0079287700 1 -- 172.21.15.40:0/106817476 --> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0074101bd0 con 0x7f005c038760 2023-07-18T03:23:18.094 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.401+0000 7f005bfff700 1 -- 172.21.15.40:0/106817476 <== mgr.14120 v2:172.21.15.40:6800/1112125515 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0074101bd0 con 0x7f005c038760 2023-07-18T03:23:18.094 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.403+0000 7f0079287700 1 -- 172.21.15.40:0/106817476 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f005c038760 msgr2=0x7f005c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:18.094 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.403+0000 7f0079287700 1 --2- 172.21.15.40:0/106817476 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f005c038760 0x7f005c03ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0068006fd0 tx=0x7f0068006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:18.094 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.403+0000 7f0079287700 1 -- 172.21.15.40:0/106817476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0074105ed0 msgr2=0x7f0074105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:18.094 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.403+0000 7f0079287700 1 --2- 172.21.15.40:0/106817476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0074105ed0 0x7f0074105830 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0064006af0 tx=0x7f0064011ad0 comp rx=0 tx=0).stop 2023-07-18T03:23:18.094 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.403+0000 7f0079287700 1 -- 172.21.15.40:0/106817476 shutdown_connections 2023-07-18T03:23:18.094 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.403+0000 7f0079287700 1 --2- 172.21.15.40:0/106817476 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f005c038760 0x7f005c03ac10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:18.095 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.403+0000 7f0079287700 1 --2- 172.21.15.40:0/106817476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0074105ed0 0x7f0074105830 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:18.095 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.403+0000 7f0079287700 1 -- 172.21.15.40:0/106817476 >> 172.21.15.40:0/106817476 conn(0x7f00740ff8d0 msgr2=0x7f0074100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:18.095 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.403+0000 7f0079287700 1 -- 172.21.15.40:0/106817476 shutdown_connections 2023-07-18T03:23:18.095 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:17.403+0000 7f0079287700 1 -- 172.21.15.40:0/106817476 wait complete. 2023-07-18T03:23:18.095 INFO:teuthology.orchestra.run.smithi040.stdout:Deploying unmanaged mgr service... 2023-07-18T03:23:18.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:18 smithi040 ceph-mon[96439]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:23:18.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:18 smithi040 ceph-mon[96439]: Saving service mon spec with placement count:5 2023-07-18T03:23:18.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:18 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' 2023-07-18T03:23:19.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-07-18T03:23:19.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.659+0000 7fc4ee189700 1 Processor -- start 2023-07-18T03:23:19.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.659+0000 7fc4ee189700 1 -- start start 2023-07-18T03:23:19.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.659+0000 7fc4ee189700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e8103660 0x7fc4e8103a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:19.436 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.659+0000 7fc4ee189700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e8103fb0 con 0x7fc4e8103660 2023-07-18T03:23:19.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.660+0000 7fc4e77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e8103660 0x7fc4e8103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:19.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.660+0000 7fc4e77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e8103660 0x7fc4e8103a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54972/0 (socket says 172.21.15.40:54972) 2023-07-18T03:23:19.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.660+0000 7fc4e77fe700 1 -- 172.21.15.40:0/533788251 learned_addr learned my addr 172.21.15.40:0/533788251 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:19.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.660+0000 7fc4e77fe700 1 -- 172.21.15.40:0/533788251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e81040f0 con 0x7fc4e8103660 2023-07-18T03:23:19.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.660+0000 7fc4e77fe700 1 --2- 172.21.15.40:0/533788251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e8103660 0x7fc4e8103a70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80098a0 tx=0x7fc4d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369422b81496ec78 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:19.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.660+0000 7fc4e67fc700 1 -- 172.21.15.40:0/533788251 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4d8004160 con 0x7fc4e8103660 2023-07-18T03:23:19.437 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.660+0000 7fc4e67fc700 1 -- 172.21.15.40:0/533788251 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc4d80042c0 con 0x7fc4e8103660 2023-07-18T03:23:19.438 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.660+0000 7fc4e67fc700 1 -- 172.21.15.40:0/533788251 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4d8011a00 con 0x7fc4e8103660 2023-07-18T03:23:19.438 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.661+0000 7fc4ee189700 1 -- 172.21.15.40:0/533788251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e8103660 msgr2=0x7fc4e8103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:19.438 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.661+0000 7fc4ee189700 1 --2- 172.21.15.40:0/533788251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e8103660 0x7fc4e8103a70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80098a0 tx=0x7fc4d8009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:19.438 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.661+0000 7fc4ee189700 1 -- 172.21.15.40:0/533788251 shutdown_connections 2023-07-18T03:23:19.438 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.661+0000 7fc4ee189700 1 --2- 172.21.15.40:0/533788251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e8103660 0x7fc4e8103a70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:19.438 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.661+0000 7fc4ee189700 1 -- 172.21.15.40:0/533788251 >> 172.21.15.40:0/533788251 conn(0x7fc4e80ff040 msgr2=0x7fc4e8101470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:19.438 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.661+0000 7fc4ee189700 1 -- 172.21.15.40:0/533788251 shutdown_connections 2023-07-18T03:23:19.439 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.661+0000 7fc4ee189700 1 -- 172.21.15.40:0/533788251 wait complete. 2023-07-18T03:23:19.439 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.662+0000 7fc4ee189700 1 Processor -- start 2023-07-18T03:23:19.439 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.662+0000 7fc4ee189700 1 -- start start 2023-07-18T03:23:19.439 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.662+0000 7fc4ee189700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e8103660 0x7fc4e8199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:19.439 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.662+0000 7fc4ee189700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e8199640 con 0x7fc4e8103660 2023-07-18T03:23:19.439 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.662+0000 7fc4e77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e8103660 0x7fc4e8199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:19.439 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.662+0000 7fc4e77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e8103660 0x7fc4e8199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54974/0 (socket says 172.21.15.40:54974) 2023-07-18T03:23:19.439 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.662+0000 7fc4e77fe700 1 -- 172.21.15.40:0/1399601325 learned_addr learned my addr 172.21.15.40:0/1399601325 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:19.440 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.663+0000 7fc4e77fe700 1 -- 172.21.15.40:0/1399601325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d8009550 con 0x7fc4e8103660 2023-07-18T03:23:19.440 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.663+0000 7fc4e77fe700 1 --2- 172.21.15.40:0/1399601325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e8103660 0x7fc4e8199100 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009fb0 tx=0x7fc4d8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:19.440 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.663+0000 7fc4e4ff9700 1 -- 172.21.15.40:0/1399601325 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4d8003930 con 0x7fc4e8103660 2023-07-18T03:23:19.440 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.663+0000 7fc4e4ff9700 1 -- 172.21.15.40:0/1399601325 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc4d8003a90 con 0x7fc4e8103660 2023-07-18T03:23:19.440 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.663+0000 7fc4ee189700 1 -- 172.21.15.40:0/1399601325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e8199840 con 0x7fc4e8103660 2023-07-18T03:23:19.440 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.663+0000 7fc4e4ff9700 1 -- 172.21.15.40:0/1399601325 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4d80195e0 con 0x7fc4e8103660 2023-07-18T03:23:19.440 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.663+0000 7fc4ee189700 1 -- 172.21.15.40:0/1399601325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e8199c60 con 0x7fc4e8103660 2023-07-18T03:23:19.441 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.664+0000 7fc4ee189700 1 -- 172.21.15.40:0/1399601325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e8061b40 con 0x7fc4e8103660 2023-07-18T03:23:19.441 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.664+0000 7fc4e4ff9700 1 -- 172.21.15.40:0/1399601325 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fc4d801a9d0 con 0x7fc4e8103660 2023-07-18T03:23:19.441 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.665+0000 7fc4e4ff9700 1 --2- 172.21.15.40:0/1399601325 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7fc4d0038710 0x7fc4d003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:19.441 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.665+0000 7fc4e4ff9700 1 -- 172.21.15.40:0/1399601325 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc4d804c790 con 0x7fc4e8103660 2023-07-18T03:23:19.441 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.665+0000 7fc4e6ffd700 1 --2- 172.21.15.40:0/1399601325 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7fc4d0038710 0x7fc4d003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:19.441 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.665+0000 7fc4e6ffd700 1 --2- 172.21.15.40:0/1399601325 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7fc4d0038710 0x7fc4d003abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc006fd0 tx=0x7fc4dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:19.441 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.670+0000 7fc4e4ff9700 1 -- 172.21.15.40:0/1399601325 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc4d8003c00 con 0x7fc4e8103660 2023-07-18T03:23:19.441 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:18.700+0000 7fc4e4ff9700 1 -- 172.21.15.40:0/1399601325 <== mon.0 v2:172.21.15.40:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4d804c1a0 con 0x7fc4e8103660 2023-07-18T03:23:19.442 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.054+0000 7fc4ee189700 1 -- 172.21.15.40:0/1399601325 --> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fc4e8101360 con 0x7fc4d0038710 2023-07-18T03:23:19.442 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.062+0000 7fc4e4ff9700 1 -- 172.21.15.40:0/1399601325 <== mgr.14120 v2:172.21.15.40:6800/1112125515 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc4e8101360 con 0x7fc4d0038710 2023-07-18T03:23:19.442 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.065+0000 7fc4ee189700 1 -- 172.21.15.40:0/1399601325 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7fc4d0038710 msgr2=0x7fc4d003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:19.442 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.065+0000 7fc4ee189700 1 --2- 172.21.15.40:0/1399601325 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7fc4d0038710 0x7fc4d003abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc006fd0 tx=0x7fc4dc006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:19.442 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.065+0000 7fc4ee189700 1 -- 172.21.15.40:0/1399601325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e8103660 msgr2=0x7fc4e8199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:19.442 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.065+0000 7fc4ee189700 1 --2- 172.21.15.40:0/1399601325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e8103660 0x7fc4e8199100 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009fb0 tx=0x7fc4d8011ad0 comp rx=0 tx=0).stop 2023-07-18T03:23:19.442 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.065+0000 7fc4ee189700 1 -- 172.21.15.40:0/1399601325 shutdown_connections 2023-07-18T03:23:19.443 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.065+0000 7fc4ee189700 1 --2- 172.21.15.40:0/1399601325 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7fc4d0038710 0x7fc4d003abc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:19.443 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.065+0000 7fc4ee189700 1 --2- 172.21.15.40:0/1399601325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e8103660 0x7fc4e8199100 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:19.443 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.065+0000 7fc4ee189700 1 -- 172.21.15.40:0/1399601325 >> 172.21.15.40:0/1399601325 conn(0x7fc4e80ff040 msgr2=0x7fc4e8100700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:19.443 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.065+0000 7fc4ee189700 1 -- 172.21.15.40:0/1399601325 shutdown_connections 2023-07-18T03:23:19.443 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.065+0000 7fc4ee189700 1 -- 172.21.15.40:0/1399601325 wait complete. 2023-07-18T03:23:20.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:19 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' 2023-07-18T03:23:20.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:19 smithi040 ceph-mon[96439]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:23:20.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:19 smithi040 ceph-mon[96439]: Saving service mgr spec with placement count:2 2023-07-18T03:23:20.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:19 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' 2023-07-18T03:23:20.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:19 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' 2023-07-18T03:23:20.781 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.933+0000 7f821c113700 1 Processor -- start 2023-07-18T03:23:20.782 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.933+0000 7f821c113700 1 -- start start 2023-07-18T03:23:20.782 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.934+0000 7f821c113700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8214079ce0 0x7f8214107f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:20.782 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.934+0000 7f821c113700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8214108440 con 0x7f8214079ce0 2023-07-18T03:23:20.782 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.934+0000 7f8219eaf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8214079ce0 0x7f8214107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:20.782 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.934+0000 7f8219eaf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8214079ce0 0x7f8214107f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54978/0 (socket says 172.21.15.40:54978) 2023-07-18T03:23:20.783 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.934+0000 7f8219eaf700 1 -- 172.21.15.40:0/3769614584 learned_addr learned my addr 172.21.15.40:0/3769614584 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:20.783 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.934+0000 7f8219eaf700 1 -- 172.21.15.40:0/3769614584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8214108580 con 0x7f8214079ce0 2023-07-18T03:23:20.783 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.934+0000 7f8219eaf700 1 --2- 172.21.15.40:0/3769614584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8214079ce0 0x7f8214107f00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f82100098a0 tx=0x7f8210009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d50157c742651cd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:20.783 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.935+0000 7f8218ead700 1 -- 172.21.15.40:0/3769614584 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8210004160 con 0x7f8214079ce0 2023-07-18T03:23:20.783 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.935+0000 7f8218ead700 1 -- 172.21.15.40:0/3769614584 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82100042c0 con 0x7f8214079ce0 2023-07-18T03:23:20.783 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.935+0000 7f8218ead700 1 -- 172.21.15.40:0/3769614584 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82100119b0 con 0x7f8214079ce0 2023-07-18T03:23:20.783 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.935+0000 7f821c113700 1 -- 172.21.15.40:0/3769614584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8214079ce0 msgr2=0x7f8214107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:20.784 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.935+0000 7f821c113700 1 --2- 172.21.15.40:0/3769614584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8214079ce0 0x7f8214107f00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f82100098a0 tx=0x7f8210009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:20.784 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.935+0000 7f821c113700 1 -- 172.21.15.40:0/3769614584 shutdown_connections 2023-07-18T03:23:20.784 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.935+0000 7f821c113700 1 --2- 172.21.15.40:0/3769614584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8214079ce0 0x7f8214107f00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:20.784 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.935+0000 7f821c113700 1 -- 172.21.15.40:0/3769614584 >> 172.21.15.40:0/3769614584 conn(0x7f82140ff8d0 msgr2=0x7f8214101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:20.784 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.935+0000 7f821c113700 1 -- 172.21.15.40:0/3769614584 shutdown_connections 2023-07-18T03:23:20.784 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.936+0000 7f821c113700 1 -- 172.21.15.40:0/3769614584 wait complete. 2023-07-18T03:23:20.784 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.936+0000 7f821c113700 1 Processor -- start 2023-07-18T03:23:20.785 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.936+0000 7f821c113700 1 -- start start 2023-07-18T03:23:20.785 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.936+0000 7f821c113700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f821419b2b0 0x7f821419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:20.785 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.936+0000 7f821c113700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821419bc00 con 0x7f821419b2b0 2023-07-18T03:23:20.785 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.936+0000 7f8219eaf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f821419b2b0 0x7f821419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:20.785 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.936+0000 7f8219eaf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f821419b2b0 0x7f821419b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54980/0 (socket says 172.21.15.40:54980) 2023-07-18T03:23:20.785 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.936+0000 7f8219eaf700 1 -- 172.21.15.40:0/2206754587 learned_addr learned my addr 172.21.15.40:0/2206754587 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:20.785 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.937+0000 7f8219eaf700 1 -- 172.21.15.40:0/2206754587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8210009550 con 0x7f821419b2b0 2023-07-18T03:23:20.785 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.937+0000 7f8219eaf700 1 --2- 172.21.15.40:0/2206754587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f821419b2b0 0x7f821419b6c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8210003e60 tx=0x7f8210011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:20.786 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.937+0000 7f820affd700 1 -- 172.21.15.40:0/2206754587 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8210003970 con 0x7f821419b2b0 2023-07-18T03:23:20.786 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.937+0000 7f821c113700 1 -- 172.21.15.40:0/2206754587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821419be00 con 0x7f821419b2b0 2023-07-18T03:23:20.786 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.937+0000 7f821c113700 1 -- 172.21.15.40:0/2206754587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821419ea60 con 0x7f821419b2b0 2023-07-18T03:23:20.786 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.937+0000 7f820affd700 1 -- 172.21.15.40:0/2206754587 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8210003ad0 con 0x7f821419b2b0 2023-07-18T03:23:20.786 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.937+0000 7f820affd700 1 -- 172.21.15.40:0/2206754587 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82100195e0 con 0x7f821419b2b0 2023-07-18T03:23:20.786 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.938+0000 7f820affd700 1 -- 172.21.15.40:0/2206754587 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f82100198a0 con 0x7f821419b2b0 2023-07-18T03:23:20.786 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.938+0000 7f820affd700 1 --2- 172.21.15.40:0/2206754587 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f82000386c0 0x7f820003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:20.787 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.938+0000 7f820affd700 1 -- 172.21.15.40:0/2206754587 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f821004c8f0 con 0x7f821419b2b0 2023-07-18T03:23:20.787 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.938+0000 7f82196ae700 1 --2- 172.21.15.40:0/2206754587 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f82000386c0 0x7f820003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:20.787 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.939+0000 7f821c113700 1 -- 172.21.15.40:0/2206754587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f8005320 con 0x7f821419b2b0 2023-07-18T03:23:20.787 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.939+0000 7f82196ae700 1 --2- 172.21.15.40:0/2206754587 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f82000386c0 0x7f820003ab70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8204006fd0 tx=0x7f8204006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:20.787 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:19.943+0000 7f820affd700 1 -- 172.21.15.40:0/2206754587 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8210019bf0 con 0x7f821419b2b0 2023-07-18T03:23:20.787 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:20.307+0000 7f821c113700 1 -- 172.21.15.40:0/2206754587 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f81f8005190 con 0x7f821419b2b0 2023-07-18T03:23:20.787 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:20.310+0000 7f820affd700 1 -- 172.21.15.40:0/2206754587 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f8210018b40 con 0x7f821419b2b0 2023-07-18T03:23:20.788 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:20.313+0000 7f821c113700 1 -- 172.21.15.40:0/2206754587 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f82000386c0 msgr2=0x7f820003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:20.788 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:20.313+0000 7f821c113700 1 --2- 172.21.15.40:0/2206754587 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f82000386c0 0x7f820003ab70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8204006fd0 tx=0x7f8204006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:20.788 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:20.313+0000 7f821c113700 1 -- 172.21.15.40:0/2206754587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f821419b2b0 msgr2=0x7f821419b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:20.788 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:20.313+0000 7f821c113700 1 --2- 172.21.15.40:0/2206754587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f821419b2b0 0x7f821419b6c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8210003e60 tx=0x7f8210011aa0 comp rx=0 tx=0).stop 2023-07-18T03:23:20.788 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:20.313+0000 7f821c113700 1 -- 172.21.15.40:0/2206754587 shutdown_connections 2023-07-18T03:23:20.788 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:20.313+0000 7f821c113700 1 --2- 172.21.15.40:0/2206754587 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f82000386c0 0x7f820003ab70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:20.788 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:20.313+0000 7f821c113700 1 --2- 172.21.15.40:0/2206754587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f821419b2b0 0x7f821419b6c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:20.789 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:20.313+0000 7f821c113700 1 -- 172.21.15.40:0/2206754587 >> 172.21.15.40:0/2206754587 conn(0x7f82140ff8d0 msgr2=0x7f8214100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:20.789 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:20.313+0000 7f821c113700 1 -- 172.21.15.40:0/2206754587 shutdown_connections 2023-07-18T03:23:20.789 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:20.313+0000 7f821c113700 1 -- 172.21.15.40:0/2206754587 wait complete. 2023-07-18T03:23:21.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:21 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2206754587' entity='client.admin' 2023-07-18T03:23:21.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:21 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' 2023-07-18T03:23:21.924 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.297+0000 7f6bc601a700 1 Processor -- start 2023-07-18T03:23:21.925 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.297+0000 7f6bc601a700 1 -- start start 2023-07-18T03:23:21.925 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.297+0000 7f6bc601a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bc0105ed0 0x7f6bc01082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:21.925 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.297+0000 7f6bc601a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc0079d00 con 0x7f6bc0105ed0 2023-07-18T03:23:21.925 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.297+0000 7f6bbf7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bc0105ed0 0x7f6bc01082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:21.925 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.297+0000 7f6bbf7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bc0105ed0 0x7f6bc01082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54984/0 (socket says 172.21.15.40:54984) 2023-07-18T03:23:21.925 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.297+0000 7f6bbf7fe700 1 -- 172.21.15.40:0/1543768866 learned_addr learned my addr 172.21.15.40:0/1543768866 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:21.926 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.297+0000 7f6bbf7fe700 1 -- 172.21.15.40:0/1543768866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc0079e40 con 0x7f6bc0105ed0 2023-07-18T03:23:21.926 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.298+0000 7f6bbf7fe700 1 --2- 172.21.15.40:0/1543768866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bc0105ed0 0x7f6bc01082b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00098a0 tx=0x7f6bb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3dc689e674b292 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:21.926 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.298+0000 7f6bbe7fc700 1 -- 172.21.15.40:0/1543768866 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6bb0004160 con 0x7f6bc0105ed0 2023-07-18T03:23:21.926 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.298+0000 7f6bbe7fc700 1 -- 172.21.15.40:0/1543768866 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bb00042c0 con 0x7f6bc0105ed0 2023-07-18T03:23:21.926 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.298+0000 7f6bbe7fc700 1 -- 172.21.15.40:0/1543768866 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6bb00119b0 con 0x7f6bc0105ed0 2023-07-18T03:23:21.926 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.298+0000 7f6bc601a700 1 -- 172.21.15.40:0/1543768866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bc0105ed0 msgr2=0x7f6bc01082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:21.926 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.298+0000 7f6bc601a700 1 --2- 172.21.15.40:0/1543768866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bc0105ed0 0x7f6bc01082b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00098a0 tx=0x7f6bb0009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:21.927 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.299+0000 7f6bc601a700 1 -- 172.21.15.40:0/1543768866 shutdown_connections 2023-07-18T03:23:21.927 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.299+0000 7f6bc601a700 1 --2- 172.21.15.40:0/1543768866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bc0105ed0 0x7f6bc01082b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:21.927 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.299+0000 7f6bc601a700 1 -- 172.21.15.40:0/1543768866 >> 172.21.15.40:0/1543768866 conn(0x7f6bc00ff8d0 msgr2=0x7f6bc0101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:21.927 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.299+0000 7f6bc601a700 1 -- 172.21.15.40:0/1543768866 shutdown_connections 2023-07-18T03:23:21.927 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.299+0000 7f6bc601a700 1 -- 172.21.15.40:0/1543768866 wait complete. 2023-07-18T03:23:21.927 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.299+0000 7f6bc601a700 1 Processor -- start 2023-07-18T03:23:21.927 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.299+0000 7f6bc601a700 1 -- start start 2023-07-18T03:23:21.928 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.300+0000 7f6bc601a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bc0105ed0 0x7f6bc0199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:21.928 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.300+0000 7f6bc601a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc0199670 con 0x7f6bc0105ed0 2023-07-18T03:23:21.928 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.300+0000 7f6bbf7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bc0105ed0 0x7f6bc0199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:21.928 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.300+0000 7f6bbf7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bc0105ed0 0x7f6bc0199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54986/0 (socket says 172.21.15.40:54986) 2023-07-18T03:23:21.928 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.300+0000 7f6bbf7fe700 1 -- 172.21.15.40:0/1214878516 learned_addr learned my addr 172.21.15.40:0/1214878516 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:21.928 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.300+0000 7f6bbf7fe700 1 -- 172.21.15.40:0/1214878516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb0009550 con 0x7f6bc0105ed0 2023-07-18T03:23:21.928 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.300+0000 7f6bbf7fe700 1 --2- 172.21.15.40:0/1214878516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bc0105ed0 0x7f6bc0199130 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0000c00 tx=0x7f6bb0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:21.928 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.300+0000 7f6bbcff9700 1 -- 172.21.15.40:0/1214878516 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6bb00039f0 con 0x7f6bc0105ed0 2023-07-18T03:23:21.929 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.300+0000 7f6bc601a700 1 -- 172.21.15.40:0/1214878516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc0199870 con 0x7f6bc0105ed0 2023-07-18T03:23:21.929 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.300+0000 7f6bbcff9700 1 -- 172.21.15.40:0/1214878516 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bb0003b50 con 0x7f6bc0105ed0 2023-07-18T03:23:21.929 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.300+0000 7f6bbcff9700 1 -- 172.21.15.40:0/1214878516 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6bb001a780 con 0x7f6bc0105ed0 2023-07-18T03:23:21.929 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.301+0000 7f6bc601a700 1 -- 172.21.15.40:0/1214878516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc0199c90 con 0x7f6bc0105ed0 2023-07-18T03:23:21.929 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.302+0000 7f6bbcff9700 1 -- 172.21.15.40:0/1214878516 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f6bb001aa40 con 0x7f6bc0105ed0 2023-07-18T03:23:21.929 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.302+0000 7f6bbcff9700 1 --2- 172.21.15.40:0/1214878516 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f6ba8038710 0x7f6ba803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:21.929 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.302+0000 7f6bbcff9700 1 -- 172.21.15.40:0/1214878516 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6bb004c760 con 0x7f6bc0105ed0 2023-07-18T03:23:21.930 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.302+0000 7f6bc601a700 1 -- 172.21.15.40:0/1214878516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc0061b40 con 0x7f6bc0105ed0 2023-07-18T03:23:21.930 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.302+0000 7f6bbeffd700 1 --2- 172.21.15.40:0/1214878516 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f6ba8038710 0x7f6ba803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:21.930 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.302+0000 7f6bbeffd700 1 --2- 172.21.15.40:0/1214878516 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f6ba8038710 0x7f6ba803abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4006fd0 tx=0x7f6bb4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:21.930 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.311+0000 7f6bbcff9700 1 -- 172.21.15.40:0/1214878516 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6bb001acf0 con 0x7f6bc0105ed0 2023-07-18T03:23:21.930 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.672+0000 7f6bc601a700 1 -- 172.21.15.40:0/1214878516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f6bc019c6d0 con 0x7f6bc0105ed0 2023-07-18T03:23:21.930 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.678+0000 7f6bbcff9700 1 -- 172.21.15.40:0/1214878516 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f6bb0018350 con 0x7f6bc0105ed0 2023-07-18T03:23:21.930 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.680+0000 7f6bc601a700 1 -- 172.21.15.40:0/1214878516 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f6ba8038710 msgr2=0x7f6ba803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:21.930 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.680+0000 7f6bc601a700 1 --2- 172.21.15.40:0/1214878516 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f6ba8038710 0x7f6ba803abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4006fd0 tx=0x7f6bb4006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:21.931 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.680+0000 7f6bc601a700 1 -- 172.21.15.40:0/1214878516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bc0105ed0 msgr2=0x7f6bc0199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:21.931 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.680+0000 7f6bc601a700 1 --2- 172.21.15.40:0/1214878516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bc0105ed0 0x7f6bc0199130 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0000c00 tx=0x7f6bb0011a30 comp rx=0 tx=0).stop 2023-07-18T03:23:21.931 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.681+0000 7f6bc601a700 1 -- 172.21.15.40:0/1214878516 shutdown_connections 2023-07-18T03:23:21.931 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.681+0000 7f6bc601a700 1 --2- 172.21.15.40:0/1214878516 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f6ba8038710 0x7f6ba803abc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:21.931 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.681+0000 7f6bc601a700 1 --2- 172.21.15.40:0/1214878516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6bc0105ed0 0x7f6bc0199130 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:21.931 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.681+0000 7f6bc601a700 1 -- 172.21.15.40:0/1214878516 >> 172.21.15.40:0/1214878516 conn(0x7f6bc00ff8d0 msgr2=0x7f6bc0100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:21.931 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.681+0000 7f6bc601a700 1 -- 172.21.15.40:0/1214878516 shutdown_connections 2023-07-18T03:23:21.932 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:21.681+0000 7f6bc601a700 1 -- 172.21.15.40:0/1214878516 wait complete. 2023-07-18T03:23:21.932 INFO:teuthology.orchestra.run.smithi040.stdout:Enabling the dashboard module... 2023-07-18T03:23:23.021 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:22 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1214878516' entity='client.admin' 2023-07-18T03:23:23.021 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:22 smithi040 ceph-mon[96439]: from='mgr.14120 172.21.15.40:0/254978580' entity='mgr.x' 2023-07-18T03:23:23.981 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:23 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3337209980' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-07-18T03:23:23.981 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:23 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: ignoring --setuser ceph since I am not root 2023-07-18T03:23:23.982 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:23 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: ignoring --setgroup ceph since I am not root 2023-07-18T03:23:23.982 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:23 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:23.888+0000 7fe1517a2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-18T03:23:23.982 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:23 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:23.984+0000 7fe1517a2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-18T03:23:24.075 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.361+0000 7f85e0d97700 1 Processor -- start 2023-07-18T03:23:24.075 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.361+0000 7f85e0d97700 1 -- start start 2023-07-18T03:23:24.075 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.362+0000 7f85e0d97700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc104280 0x7f85dc104690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:24.075 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.362+0000 7f85e0d97700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc104bd0 con 0x7f85dc104280 2023-07-18T03:23:24.075 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.362+0000 7f85da59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc104280 0x7f85dc104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:24.076 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.362+0000 7f85da59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc104280 0x7f85dc104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54990/0 (socket says 172.21.15.40:54990) 2023-07-18T03:23:24.076 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.362+0000 7f85da59c700 1 -- 172.21.15.40:0/2546759062 learned_addr learned my addr 172.21.15.40:0/2546759062 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:24.076 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.362+0000 7f85da59c700 1 -- 172.21.15.40:0/2546759062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc104d10 con 0x7f85dc104280 2023-07-18T03:23:24.076 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.362+0000 7f85da59c700 1 --2- 172.21.15.40:0/2546759062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc104280 0x7f85dc104690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f85cc0098a0 tx=0x7f85cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa2581db8063b5e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:24.076 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.363+0000 7f85d959a700 1 -- 172.21.15.40:0/2546759062 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85cc004160 con 0x7f85dc104280 2023-07-18T03:23:24.076 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.363+0000 7f85d959a700 1 -- 172.21.15.40:0/2546759062 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85cc0042c0 con 0x7f85dc104280 2023-07-18T03:23:24.077 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.363+0000 7f85d959a700 1 -- 172.21.15.40:0/2546759062 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85cc0119b0 con 0x7f85dc104280 2023-07-18T03:23:24.077 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.363+0000 7f85e0d97700 1 -- 172.21.15.40:0/2546759062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc104280 msgr2=0x7f85dc104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:24.077 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.363+0000 7f85e0d97700 1 --2- 172.21.15.40:0/2546759062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc104280 0x7f85dc104690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f85cc0098a0 tx=0x7f85cc009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:24.077 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.363+0000 7f85e0d97700 1 -- 172.21.15.40:0/2546759062 shutdown_connections 2023-07-18T03:23:24.077 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.363+0000 7f85e0d97700 1 --2- 172.21.15.40:0/2546759062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc104280 0x7f85dc104690 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:24.077 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.363+0000 7f85e0d97700 1 -- 172.21.15.40:0/2546759062 >> 172.21.15.40:0/2546759062 conn(0x7f85dc0ff8d0 msgr2=0x7f85dc101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:24.077 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.363+0000 7f85e0d97700 1 -- 172.21.15.40:0/2546759062 shutdown_connections 2023-07-18T03:23:24.078 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.363+0000 7f85e0d97700 1 -- 172.21.15.40:0/2546759062 wait complete. 2023-07-18T03:23:24.078 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.364+0000 7f85e0d97700 1 Processor -- start 2023-07-18T03:23:24.078 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.364+0000 7f85e0d97700 1 -- start start 2023-07-18T03:23:24.078 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.364+0000 7f85e0d97700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc104280 0x7f85dc18f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:24.078 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.364+0000 7f85e0d97700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc18f960 con 0x7f85dc104280 2023-07-18T03:23:24.078 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.364+0000 7f85da59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc104280 0x7f85dc18f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:24.079 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.364+0000 7f85da59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc104280 0x7f85dc18f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54992/0 (socket says 172.21.15.40:54992) 2023-07-18T03:23:24.079 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.365+0000 7f85da59c700 1 -- 172.21.15.40:0/3337209980 learned_addr learned my addr 172.21.15.40:0/3337209980 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:24.079 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.365+0000 7f85da59c700 1 -- 172.21.15.40:0/3337209980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85cc009550 con 0x7f85dc104280 2023-07-18T03:23:24.079 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.365+0000 7f85da59c700 1 --2- 172.21.15.40:0/3337209980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc104280 0x7f85dc18f420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f85cc003e60 tx=0x7f85cc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:24.079 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.365+0000 7f85cb7fe700 1 -- 172.21.15.40:0/3337209980 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85cc003970 con 0x7f85dc104280 2023-07-18T03:23:24.079 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.365+0000 7f85cb7fe700 1 -- 172.21.15.40:0/3337209980 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85cc003ad0 con 0x7f85dc104280 2023-07-18T03:23:24.080 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.365+0000 7f85e0d97700 1 -- 172.21.15.40:0/3337209980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85dc18da70 con 0x7f85dc104280 2023-07-18T03:23:24.080 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.365+0000 7f85cb7fe700 1 -- 172.21.15.40:0/3337209980 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85cc0195e0 con 0x7f85dc104280 2023-07-18T03:23:24.080 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.365+0000 7f85e0d97700 1 -- 172.21.15.40:0/3337209980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85dc18de90 con 0x7f85dc104280 2023-07-18T03:23:24.080 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.366+0000 7f85cb7fe700 1 -- 172.21.15.40:0/3337209980 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f85cc0198a0 con 0x7f85dc104280 2023-07-18T03:23:24.081 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.366+0000 7f85cb7fe700 1 --2- 172.21.15.40:0/3337209980 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f85c40386c0 0x7f85c403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:24.081 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.366+0000 7f85e0d97700 1 -- 172.21.15.40:0/3337209980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85dc061b40 con 0x7f85dc104280 2023-07-18T03:23:24.081 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.366+0000 7f85cb7fe700 1 -- 172.21.15.40:0/3337209980 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f85cc04c8e0 con 0x7f85dc104280 2023-07-18T03:23:24.082 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.367+0000 7f85d9d9b700 1 --2- 172.21.15.40:0/3337209980 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f85c40386c0 0x7f85c403ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:24.082 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.367+0000 7f85d9d9b700 1 --2- 172.21.15.40:0/3337209980 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f85c40386c0 0x7f85c403ab70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f85d0006fd0 tx=0x7f85d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:24.082 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.371+0000 7f85cb7fe700 1 -- 172.21.15.40:0/3337209980 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f85cc003c40 con 0x7f85dc104280 2023-07-18T03:23:24.082 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:22.846+0000 7f85e0d97700 1 -- 172.21.15.40:0/3337209980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f85dc106180 con 0x7f85dc104280 2023-07-18T03:23:24.083 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:23.684+0000 7f85cb7fe700 1 -- 172.21.15.40:0/3337209980 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f85cc01a430 con 0x7f85dc104280 2023-07-18T03:23:24.083 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:23.685+0000 7f85cb7fe700 1 -- 172.21.15.40:0/3337209980 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f85cc018740 con 0x7f85dc104280 2023-07-18T03:23:24.083 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:23.688+0000 7f85e0d97700 1 -- 172.21.15.40:0/3337209980 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f85c40386c0 msgr2=0x7f85c403ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:24.083 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:23.688+0000 7f85e0d97700 1 --2- 172.21.15.40:0/3337209980 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f85c40386c0 0x7f85c403ab70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f85d0006fd0 tx=0x7f85d0006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:24.084 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:23.688+0000 7f85e0d97700 1 -- 172.21.15.40:0/3337209980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc104280 msgr2=0x7f85dc18f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:24.084 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:23.688+0000 7f85e0d97700 1 --2- 172.21.15.40:0/3337209980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc104280 0x7f85dc18f420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f85cc003e60 tx=0x7f85cc011aa0 comp rx=0 tx=0).stop 2023-07-18T03:23:24.084 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:23.688+0000 7f85e0d97700 1 -- 172.21.15.40:0/3337209980 shutdown_connections 2023-07-18T03:23:24.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:23.688+0000 7f85e0d97700 1 --2- 172.21.15.40:0/3337209980 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f85c40386c0 0x7f85c403ab70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:24.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:23.688+0000 7f85e0d97700 1 --2- 172.21.15.40:0/3337209980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc104280 0x7f85dc18f420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:24.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:23.688+0000 7f85e0d97700 1 -- 172.21.15.40:0/3337209980 >> 172.21.15.40:0/3337209980 conn(0x7f85dc0ff8d0 msgr2=0x7f85dc072fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:24.085 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:23.688+0000 7f85e0d97700 1 -- 172.21.15.40:0/3337209980 shutdown_connections 2023-07-18T03:23:24.086 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:23.688+0000 7f85e0d97700 1 -- 172.21.15.40:0/3337209980 wait complete. 2023-07-18T03:23:24.255 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:24 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:24.097+0000 7fe1517a2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-18T03:23:24.255 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:24 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:24.211+0000 7fe1517a2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-18T03:23:24.578 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:24 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:24.426+0000 7fe1517a2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-18T03:23:24.942 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:24 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:24.836+0000 7fe1517a2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-18T03:23:24.942 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:24 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3337209980' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-07-18T03:23:24.942 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:24 smithi040 ceph-mon[96439]: mgrmap e9: x(active, since 18s) 2023-07-18T03:23:25.216 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:24 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:24.945+0000 7fe1517a2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-18T03:23:25.216 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:25 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:25.102+0000 7fe1517a2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-18T03:23:25.315 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2023-07-18T03:23:25.320 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-07-18T03:23:25.320 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-18T03:23:25.320 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-18T03:23:25.321 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-18T03:23:25.321 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2023-07-18T03:23:25.321 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.530+0000 7f9df1a1a700 1 Processor -- start 2023-07-18T03:23:25.321 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.530+0000 7f9df1a1a700 1 -- start start 2023-07-18T03:23:25.321 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.531+0000 7f9df1a1a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec105370 0x7f9dec105780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:25.321 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.531+0000 7f9df1a1a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec105cc0 con 0x7f9dec105370 2023-07-18T03:23:25.321 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.531+0000 7f9deaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec105370 0x7f9dec105780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:25.322 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.531+0000 7f9deaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec105370 0x7f9dec105780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55006/0 (socket says 172.21.15.40:55006) 2023-07-18T03:23:25.322 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.531+0000 7f9deaffd700 1 -- 172.21.15.40:0/2976261826 learned_addr learned my addr 172.21.15.40:0/2976261826 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:25.322 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.531+0000 7f9deaffd700 1 -- 172.21.15.40:0/2976261826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dec105e00 con 0x7f9dec105370 2023-07-18T03:23:25.322 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.531+0000 7f9deaffd700 1 --2- 172.21.15.40:0/2976261826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec105370 0x7f9dec105780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0098a0 tx=0x7f9ddc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84433fb9250e69b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:25.322 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.532+0000 7f9de9ffb700 1 -- 172.21.15.40:0/2976261826 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ddc004160 con 0x7f9dec105370 2023-07-18T03:23:25.323 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.532+0000 7f9de9ffb700 1 -- 172.21.15.40:0/2976261826 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ddc0042c0 con 0x7f9dec105370 2023-07-18T03:23:25.323 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.532+0000 7f9de9ffb700 1 -- 172.21.15.40:0/2976261826 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ddc0119b0 con 0x7f9dec105370 2023-07-18T03:23:25.323 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.532+0000 7f9df1a1a700 1 -- 172.21.15.40:0/2976261826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec105370 msgr2=0x7f9dec105780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:25.323 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.532+0000 7f9df1a1a700 1 --2- 172.21.15.40:0/2976261826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec105370 0x7f9dec105780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0098a0 tx=0x7f9ddc009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:25.323 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.532+0000 7f9df1a1a700 1 -- 172.21.15.40:0/2976261826 shutdown_connections 2023-07-18T03:23:25.323 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.532+0000 7f9df1a1a700 1 --2- 172.21.15.40:0/2976261826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec105370 0x7f9dec105780 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:25.323 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.532+0000 7f9df1a1a700 1 -- 172.21.15.40:0/2976261826 >> 172.21.15.40:0/2976261826 conn(0x7f9dec1009e0 msgr2=0x7f9dec102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:25.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.532+0000 7f9df1a1a700 1 -- 172.21.15.40:0/2976261826 shutdown_connections 2023-07-18T03:23:25.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.532+0000 7f9df1a1a700 1 -- 172.21.15.40:0/2976261826 wait complete. 2023-07-18T03:23:25.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.533+0000 7f9df1a1a700 1 Processor -- start 2023-07-18T03:23:25.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.533+0000 7f9df1a1a700 1 -- start start 2023-07-18T03:23:25.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.533+0000 7f9df1a1a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec105370 0x7f9dec07bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:25.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.533+0000 7f9df1a1a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec07c170 con 0x7f9dec105370 2023-07-18T03:23:25.324 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.533+0000 7f9deaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec105370 0x7f9dec07bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:25.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.534+0000 7f9deaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec105370 0x7f9dec07bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55008/0 (socket says 172.21.15.40:55008) 2023-07-18T03:23:25.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.534+0000 7f9deaffd700 1 -- 172.21.15.40:0/1269550201 learned_addr learned my addr 172.21.15.40:0/1269550201 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:25.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.534+0000 7f9deaffd700 1 -- 172.21.15.40:0/1269550201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc009550 con 0x7f9dec105370 2023-07-18T03:23:25.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.534+0000 7f9deaffd700 1 --2- 172.21.15.40:0/1269550201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec105370 0x7f9dec07bc30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc006af0 tx=0x7f9ddc011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:25.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.534+0000 7f9df0a18700 1 -- 172.21.15.40:0/1269550201 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ddc003a40 con 0x7f9dec105370 2023-07-18T03:23:25.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.534+0000 7f9df1a1a700 1 -- 172.21.15.40:0/1269550201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dec07a280 con 0x7f9dec105370 2023-07-18T03:23:25.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.534+0000 7f9df0a18700 1 -- 172.21.15.40:0/1269550201 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ddc003ba0 con 0x7f9dec105370 2023-07-18T03:23:25.325 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.534+0000 7f9df0a18700 1 -- 172.21.15.40:0/1269550201 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ddc0196c0 con 0x7f9dec105370 2023-07-18T03:23:25.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.534+0000 7f9df1a1a700 1 -- 172.21.15.40:0/1269550201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dec07a6a0 con 0x7f9dec105370 2023-07-18T03:23:25.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.535+0000 7f9df0a18700 1 -- 172.21.15.40:0/1269550201 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f9ddc019980 con 0x7f9dec105370 2023-07-18T03:23:25.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.536+0000 7f9df0a18700 1 --2- 172.21.15.40:0/1269550201 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f9dd8038710 0x7f9dd803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:25.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.536+0000 7f9dea7fc700 1 -- 172.21.15.40:0/1269550201 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f9dd8038710 msgr2=0x7f9dd803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1112125515 2023-07-18T03:23:25.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.536+0000 7f9df0a18700 1 -- 172.21.15.40:0/1269550201 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9ddc04c6b0 con 0x7f9dec105370 2023-07-18T03:23:25.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.536+0000 7f9dea7fc700 1 --2- 172.21.15.40:0/1269550201 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f9dd8038710 0x7f9dd803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T03:23:25.326 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.536+0000 7f9df1a1a700 1 -- 172.21.15.40:0/1269550201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dec061b40 con 0x7f9dec105370 2023-07-18T03:23:25.327 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.541+0000 7f9df0a18700 1 -- 172.21.15.40:0/1269550201 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9ddc0243f0 con 0x7f9dec105370 2023-07-18T03:23:25.327 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.737+0000 7f9dea7fc700 1 -- 172.21.15.40:0/1269550201 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f9dd8038710 msgr2=0x7f9dd803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1112125515 2023-07-18T03:23:25.327 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:24.737+0000 7f9dea7fc700 1 --2- 172.21.15.40:0/1269550201 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f9dd8038710 0x7f9dd803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-18T03:23:25.327 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:25.013+0000 7f9df1a1a700 1 -- 172.21.15.40:0/1269550201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f9dec07af50 con 0x7f9dec105370 2023-07-18T03:23:25.327 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:25.014+0000 7f9df0a18700 1 -- 172.21.15.40:0/1269550201 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f9ddc04a030 con 0x7f9dec105370 2023-07-18T03:23:25.327 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:25.016+0000 7f9df1a1a700 1 -- 172.21.15.40:0/1269550201 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f9dd8038710 msgr2=0x7f9dd803abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:23:25.327 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:25.016+0000 7f9df1a1a700 1 --2- 172.21.15.40:0/1269550201 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f9dd8038710 0x7f9dd803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:25.327 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:25.016+0000 7f9df1a1a700 1 -- 172.21.15.40:0/1269550201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec105370 msgr2=0x7f9dec07bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:25.328 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:25.016+0000 7f9df1a1a700 1 --2- 172.21.15.40:0/1269550201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec105370 0x7f9dec07bc30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc006af0 tx=0x7f9ddc011a10 comp rx=0 tx=0).stop 2023-07-18T03:23:25.328 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:25.016+0000 7f9df1a1a700 1 -- 172.21.15.40:0/1269550201 shutdown_connections 2023-07-18T03:23:25.328 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:25.016+0000 7f9df1a1a700 1 --2- 172.21.15.40:0/1269550201 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f9dd8038710 0x7f9dd803abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:25.328 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:25.016+0000 7f9df1a1a700 1 --2- 172.21.15.40:0/1269550201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dec105370 0x7f9dec07bc30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:25.328 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:25.016+0000 7f9df1a1a700 1 -- 172.21.15.40:0/1269550201 >> 172.21.15.40:0/1269550201 conn(0x7f9dec1009e0 msgr2=0x7f9dec1020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:25.328 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:25.017+0000 7f9df1a1a700 1 -- 172.21.15.40:0/1269550201 shutdown_connections 2023-07-18T03:23:25.328 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:25.017+0000 7f9df1a1a700 1 -- 172.21.15.40:0/1269550201 wait complete. 2023-07-18T03:23:25.329 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for the mgr to restart... 2023-07-18T03:23:25.329 INFO:teuthology.orchestra.run.smithi040.stdout:Waiting for mgr epoch 9... 2023-07-18T03:23:25.578 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:25 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:25.219+0000 7fe1517a2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-18T03:23:26.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:25 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1269550201' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-18T03:23:26.079 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:25 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:25.739+0000 7fe1517a2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-18T03:23:26.828 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:26 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:26.436+0000 7fe1517a2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-18T03:23:26.829 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:26 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:26.628+0000 7fe1517a2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-18T03:23:27.920 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:27 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:27.593+0000 7fe1517a2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:23:27.920 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:27 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:27.704+0000 7fe1517a2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-18T03:23:28.270 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:27 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:27.923+0000 7fe1517a2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:23:28.521 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:28 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:28.273+0000 7fe1517a2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-18T03:23:28.828 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:28 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:28.524+0000 7fe1517a2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-18T03:23:28.829 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:28 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:28.707+0000 7fe1517a2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-18T03:23:29.328 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:29 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:29.036+0000 7fe1517a2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-18T03:23:30.258 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:29 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:29.860+0000 7fe1517a2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-18T03:23:30.259 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:30 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:30.018+0000 7fe1517a2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-18T03:23:30.578 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:30 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:30.261+0000 7fe1517a2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-18T03:23:30.579 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:30 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:30.399+0000 7fe1517a2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-18T03:23:31.078 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:30 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:30.671+0000 7fe1517a2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-18T03:23:32.078 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:23:31 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:23:31.720+0000 7fe1517a2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-18T03:23:32.774 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:32 smithi040 ceph-mon[96439]: Active manager daemon x restarted 2023-07-18T03:23:32.774 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:32 smithi040 ceph-mon[96439]: Activating manager daemon x 2023-07-18T03:23:32.774 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:32 smithi040 ceph-mon[96439]: osdmap e3: 0 total, 0 up, 0 in 2023-07-18T03:23:32.774 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:32 smithi040 ceph-mon[96439]: mgrmap e10: x(active, starting, since 0.00334992s) 2023-07-18T03:23:32.774 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:23:32.774 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-18T03:23:32.775 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-18T03:23:32.775 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-18T03:23:32.775 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-18T03:23:32.775 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:32 smithi040 ceph-mon[96439]: Manager daemon x is now available 2023-07-18T03:23:32.775 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:23:32.775 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:23:32.775 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-18T03:23:32.776 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-18T03:23:33.176 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout { 2023-07-18T03:23:33.184 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-07-18T03:23:33.184 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-18T03:23:33.184 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout } 2023-07-18T03:23:33.185 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.206+0000 7f0456217700 1 Processor -- start 2023-07-18T03:23:33.185 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.207+0000 7f0456217700 1 -- start start 2023-07-18T03:23:33.185 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.207+0000 7f0456217700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0450105380 0x7f0450105790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.185 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.207+0000 7f0456217700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0450105cd0 con 0x7f0450105380 2023-07-18T03:23:33.185 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.207+0000 7f044f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0450105380 0x7f0450105790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.185 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.207+0000 7f044f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0450105380 0x7f0450105790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55016/0 (socket says 172.21.15.40:55016) 2023-07-18T03:23:33.185 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.207+0000 7f044f7fe700 1 -- 172.21.15.40:0/362197828 learned_addr learned my addr 172.21.15.40:0/362197828 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:33.186 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.207+0000 7f044f7fe700 1 -- 172.21.15.40:0/362197828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0450105e10 con 0x7f0450105380 2023-07-18T03:23:33.186 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.207+0000 7f044f7fe700 1 --2- 172.21.15.40:0/362197828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0450105380 0x7f0450105790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f04400098a0 tx=0x7f0440009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387fdceb43031bd4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:33.186 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.208+0000 7f044e7fc700 1 -- 172.21.15.40:0/362197828 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0440004160 con 0x7f0450105380 2023-07-18T03:23:33.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.208+0000 7f044e7fc700 1 -- 172.21.15.40:0/362197828 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04400042c0 con 0x7f0450105380 2023-07-18T03:23:33.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.208+0000 7f044e7fc700 1 -- 172.21.15.40:0/362197828 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04400119b0 con 0x7f0450105380 2023-07-18T03:23:33.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.208+0000 7f0456217700 1 -- 172.21.15.40:0/362197828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0450105380 msgr2=0x7f0450105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.208+0000 7f0456217700 1 --2- 172.21.15.40:0/362197828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0450105380 0x7f0450105790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f04400098a0 tx=0x7f0440009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.208+0000 7f0456217700 1 -- 172.21.15.40:0/362197828 shutdown_connections 2023-07-18T03:23:33.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.208+0000 7f0456217700 1 --2- 172.21.15.40:0/362197828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0450105380 0x7f0450105790 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.187 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.208+0000 7f0456217700 1 -- 172.21.15.40:0/362197828 >> 172.21.15.40:0/362197828 conn(0x7f04501009f0 msgr2=0x7f0450102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:33.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.208+0000 7f0456217700 1 -- 172.21.15.40:0/362197828 shutdown_connections 2023-07-18T03:23:33.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.208+0000 7f0456217700 1 -- 172.21.15.40:0/362197828 wait complete. 2023-07-18T03:23:33.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.209+0000 7f0456217700 1 Processor -- start 2023-07-18T03:23:33.188 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.209+0000 7f0456217700 1 -- start start 2023-07-18T03:23:33.189 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.209+0000 7f0456217700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0450105380 0x7f04501990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.189 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.209+0000 7f0456217700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0450199630 con 0x7f0450105380 2023-07-18T03:23:33.189 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.210+0000 7f044f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0450105380 0x7f04501990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.189 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.210+0000 7f044f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0450105380 0x7f04501990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55018/0 (socket says 172.21.15.40:55018) 2023-07-18T03:23:33.189 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.210+0000 7f044f7fe700 1 -- 172.21.15.40:0/4274886813 learned_addr learned my addr 172.21.15.40:0/4274886813 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:33.189 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.210+0000 7f044f7fe700 1 -- 172.21.15.40:0/4274886813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0440009550 con 0x7f0450105380 2023-07-18T03:23:33.190 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.210+0000 7f044f7fe700 1 --2- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0450105380 0x7f04501990f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0440003e60 tx=0x7f0440011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:33.190 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.210+0000 7f044cff9700 1 -- 172.21.15.40:0/4274886813 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0440003970 con 0x7f0450105380 2023-07-18T03:23:33.190 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.210+0000 7f044cff9700 1 -- 172.21.15.40:0/4274886813 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0440003ad0 con 0x7f0450105380 2023-07-18T03:23:33.190 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.210+0000 7f0456217700 1 -- 172.21.15.40:0/4274886813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0450199830 con 0x7f0450105380 2023-07-18T03:23:33.190 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.210+0000 7f044cff9700 1 -- 172.21.15.40:0/4274886813 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04400195e0 con 0x7f0450105380 2023-07-18T03:23:33.190 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.210+0000 7f0456217700 1 -- 172.21.15.40:0/4274886813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0450199c50 con 0x7f0450105380 2023-07-18T03:23:33.190 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.212+0000 7f044cff9700 1 -- 172.21.15.40:0/4274886813 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f04400198a0 con 0x7f0450105380 2023-07-18T03:23:33.191 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.212+0000 7f044cff9700 1 --2- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f0438038710 0x7f043803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.191 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.212+0000 7f044cff9700 1 -- 172.21.15.40:0/4274886813 --> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f043803b2d0 con 0x7f0438038710 2023-07-18T03:23:33.191 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.212+0000 7f044cff9700 1 -- 172.21.15.40:0/4274886813 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f044004c920 con 0x7f0450105380 2023-07-18T03:23:33.191 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.212+0000 7f044effd700 1 -- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f0438038710 msgr2=0x7f043803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1112125515 2023-07-18T03:23:33.191 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.212+0000 7f044effd700 1 --2- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f0438038710 0x7f043803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T03:23:33.191 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.413+0000 7f044effd700 1 -- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f0438038710 msgr2=0x7f043803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1112125515 2023-07-18T03:23:33.191 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.413+0000 7f044effd700 1 --2- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f0438038710 0x7f043803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-18T03:23:33.191 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.814+0000 7f044effd700 1 -- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f0438038710 msgr2=0x7f043803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1112125515 2023-07-18T03:23:33.192 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:26.814+0000 7f044effd700 1 --2- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f0438038710 0x7f043803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-18T03:23:33.192 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:27.615+0000 7f044effd700 1 -- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f0438038710 msgr2=0x7f043803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1112125515 2023-07-18T03:23:33.192 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:27.615+0000 7f044effd700 1 --2- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f0438038710 0x7f043803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-18T03:23:33.192 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:29.216+0000 7f044effd700 1 -- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f0438038710 msgr2=0x7f043803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:6800/1112125515 2023-07-18T03:23:33.192 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:29.216+0000 7f044effd700 1 --2- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f0438038710 0x7f043803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-18T03:23:33.192 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:31.735+0000 7f044cff9700 1 -- 172.21.15.40:0/4274886813 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7f0440019ba0 con 0x7f0450105380 2023-07-18T03:23:33.193 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:31.735+0000 7f044cff9700 1 -- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f0438038710 msgr2=0x7f043803abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:23:33.193 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:31.735+0000 7f044cff9700 1 --2- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f0438038710 0x7f043803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.193 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.736+0000 7f044cff9700 1 -- 172.21.15.40:0/4274886813 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f044004de30 con 0x7f0450105380 2023-07-18T03:23:33.193 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.736+0000 7f044cff9700 1 --2- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f043803bff0 0x7f043803e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:33.193 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.737+0000 7f044cff9700 1 -- 172.21.15.40:0/4274886813 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f043803b2d0 con 0x7f043803bff0 2023-07-18T03:23:33.193 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.737+0000 7f044effd700 1 --2- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f043803bff0 0x7f043803e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:33.193 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.738+0000 7f044effd700 1 --2- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f043803bff0 0x7f043803e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0444003a10 tx=0x7f04440092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:33.193 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.738+0000 7f044cff9700 1 -- 172.21.15.40:0/4274886813 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f043803b2d0 con 0x7f043803bff0 2023-07-18T03:23:33.194 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.749+0000 7f0456217700 1 -- 172.21.15.40:0/4274886813 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f045019a430 con 0x7f043803bff0 2023-07-18T03:23:33.194 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.750+0000 7f044cff9700 1 -- 172.21.15.40:0/4274886813 <== mgr.14152 v2:172.21.15.40:6800/3612011194 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f045019a430 con 0x7f043803bff0 2023-07-18T03:23:33.194 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.750+0000 7f0456217700 1 -- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f043803bff0 msgr2=0x7f043803e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.194 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.750+0000 7f0456217700 1 --2- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f043803bff0 0x7f043803e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0444003a10 tx=0x7f04440092b0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.194 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.750+0000 7f0456217700 1 -- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0450105380 msgr2=0x7f04501990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:33.194 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.750+0000 7f0456217700 1 --2- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0450105380 0x7f04501990f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0440003e60 tx=0x7f0440011aa0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.194 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.750+0000 7f0456217700 1 -- 172.21.15.40:0/4274886813 shutdown_connections 2023-07-18T03:23:33.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.750+0000 7f0456217700 1 --2- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f043803bff0 0x7f043803e3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.750+0000 7f0456217700 1 --2- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0450105380 0x7f04501990f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.750+0000 7f0456217700 1 --2- 172.21.15.40:0/4274886813 >> [v2:172.21.15.40:6800/1112125515,v1:172.21.15.40:6801/1112125515] conn(0x7f0438038710 0x7f043803abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:33.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.750+0000 7f0456217700 1 -- 172.21.15.40:0/4274886813 >> 172.21.15.40:0/4274886813 conn(0x7f04501009f0 msgr2=0x7f0450102050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:33.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.750+0000 7f0456217700 1 -- 172.21.15.40:0/4274886813 shutdown_connections 2023-07-18T03:23:33.195 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:32.750+0000 7f0456217700 1 -- 172.21.15.40:0/4274886813 wait complete. 2023-07-18T03:23:33.195 INFO:teuthology.orchestra.run.smithi040.stdout:mgr epoch 9 is available 2023-07-18T03:23:33.196 INFO:teuthology.orchestra.run.smithi040.stdout:Generating a dashboard self-signed certificate... 2023-07-18T03:23:34.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:33 smithi040 ceph-mon[96439]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-18T03:23:34.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:33 smithi040 ceph-mon[96439]: mgrmap e11: x(active, since 1.00624s) 2023-07-18T03:23:34.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:33 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:34.531 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-07-18T03:23:34.531 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.722+0000 7f933f1e3700 1 Processor -- start 2023-07-18T03:23:34.531 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.722+0000 7f933f1e3700 1 -- start start 2023-07-18T03:23:34.531 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.722+0000 7f933f1e3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9338105880 0x7f9338105c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:34.531 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.722+0000 7f933f1e3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93381061d0 con 0x7f9338105880 2023-07-18T03:23:34.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.723+0000 7f933cf7f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9338105880 0x7f9338105c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:34.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.723+0000 7f933cf7f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9338105880 0x7f9338105c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55072/0 (socket says 172.21.15.40:55072) 2023-07-18T03:23:34.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.723+0000 7f933cf7f700 1 -- 172.21.15.40:0/3046594394 learned_addr learned my addr 172.21.15.40:0/3046594394 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:34.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.723+0000 7f933cf7f700 1 -- 172.21.15.40:0/3046594394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9338106310 con 0x7f9338105880 2023-07-18T03:23:34.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.723+0000 7f933cf7f700 1 --2- 172.21.15.40:0/3046594394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9338105880 0x7f9338105c90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f932c0098a0 tx=0x7f932c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e25b2b5e15f426 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:34.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.723+0000 7f93377fe700 1 -- 172.21.15.40:0/3046594394 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f932c004160 con 0x7f9338105880 2023-07-18T03:23:34.532 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.723+0000 7f93377fe700 1 -- 172.21.15.40:0/3046594394 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f932c0042c0 con 0x7f9338105880 2023-07-18T03:23:34.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.723+0000 7f93377fe700 1 -- 172.21.15.40:0/3046594394 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f932c0119b0 con 0x7f9338105880 2023-07-18T03:23:34.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.724+0000 7f933f1e3700 1 -- 172.21.15.40:0/3046594394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9338105880 msgr2=0x7f9338105c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:34.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.724+0000 7f933f1e3700 1 --2- 172.21.15.40:0/3046594394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9338105880 0x7f9338105c90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f932c0098a0 tx=0x7f932c009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:34.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.724+0000 7f933f1e3700 1 -- 172.21.15.40:0/3046594394 shutdown_connections 2023-07-18T03:23:34.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.724+0000 7f933f1e3700 1 --2- 172.21.15.40:0/3046594394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9338105880 0x7f9338105c90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:34.533 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.724+0000 7f933f1e3700 1 -- 172.21.15.40:0/3046594394 >> 172.21.15.40:0/3046594394 conn(0x7f93380ff040 msgr2=0x7f9338101470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:34.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.724+0000 7f933f1e3700 1 -- 172.21.15.40:0/3046594394 shutdown_connections 2023-07-18T03:23:34.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.724+0000 7f933f1e3700 1 -- 172.21.15.40:0/3046594394 wait complete. 2023-07-18T03:23:34.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.725+0000 7f933f1e3700 1 Processor -- start 2023-07-18T03:23:34.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.725+0000 7f933f1e3700 1 -- start start 2023-07-18T03:23:34.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.725+0000 7f933f1e3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9338105880 0x7f933818f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:34.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.725+0000 7f933f1e3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933818f950 con 0x7f9338105880 2023-07-18T03:23:34.534 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.725+0000 7f933cf7f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9338105880 0x7f933818f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:34.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.725+0000 7f933cf7f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9338105880 0x7f933818f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55074/0 (socket says 172.21.15.40:55074) 2023-07-18T03:23:34.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.725+0000 7f933cf7f700 1 -- 172.21.15.40:0/209331330 learned_addr learned my addr 172.21.15.40:0/209331330 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:34.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.725+0000 7f933cf7f700 1 -- 172.21.15.40:0/209331330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c009550 con 0x7f9338105880 2023-07-18T03:23:34.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.725+0000 7f933cf7f700 1 --2- 172.21.15.40:0/209331330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9338105880 0x7f933818f410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f932c003e60 tx=0x7f932c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:34.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.726+0000 7f9335ffb700 1 -- 172.21.15.40:0/209331330 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f932c003970 con 0x7f9338105880 2023-07-18T03:23:34.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.726+0000 7f9335ffb700 1 -- 172.21.15.40:0/209331330 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f932c003ad0 con 0x7f9338105880 2023-07-18T03:23:34.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.726+0000 7f933f1e3700 1 -- 172.21.15.40:0/209331330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933818da60 con 0x7f9338105880 2023-07-18T03:23:34.535 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.726+0000 7f9335ffb700 1 -- 172.21.15.40:0/209331330 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f932c0195e0 con 0x7f9338105880 2023-07-18T03:23:34.536 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.726+0000 7f933f1e3700 1 -- 172.21.15.40:0/209331330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933818de80 con 0x7f9338105880 2023-07-18T03:23:34.536 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.727+0000 7f933f1e3700 1 -- 172.21.15.40:0/209331330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9338061b40 con 0x7f9338105880 2023-07-18T03:23:34.536 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.728+0000 7f9335ffb700 1 -- 172.21.15.40:0/209331330 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f932c0198a0 con 0x7f9338105880 2023-07-18T03:23:34.536 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.728+0000 7f9335ffb700 1 --2- 172.21.15.40:0/209331330 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f93280385f0 0x7f932803aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:34.536 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.728+0000 7f9335ffb700 1 -- 172.21.15.40:0/209331330 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f932c04caf0 con 0x7f9338105880 2023-07-18T03:23:34.536 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.728+0000 7f9337fff700 1 --2- 172.21.15.40:0/209331330 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f93280385f0 0x7f932803aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:34.537 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.728+0000 7f9337fff700 1 --2- 172.21.15.40:0/209331330 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f93280385f0 0x7f932803aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9324006fd0 tx=0x7f9324006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:34.537 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:33.732+0000 7f9335ffb700 1 -- 172.21.15.40:0/209331330 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f932c003c40 con 0x7f9338105880 2023-07-18T03:23:34.537 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:34.105+0000 7f933f1e3700 1 -- 172.21.15.40:0/209331330 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f9338103cc0 con 0x7f93280385f0 2023-07-18T03:23:34.538 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:34.180+0000 7f9335ffb700 1 -- 172.21.15.40:0/209331330 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f9338103cc0 con 0x7f93280385f0 2023-07-18T03:23:34.538 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:34.182+0000 7f933f1e3700 1 -- 172.21.15.40:0/209331330 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f93280385f0 msgr2=0x7f932803aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:34.538 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:34.182+0000 7f933f1e3700 1 --2- 172.21.15.40:0/209331330 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f93280385f0 0x7f932803aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9324006fd0 tx=0x7f9324006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:34.538 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:34.182+0000 7f933f1e3700 1 -- 172.21.15.40:0/209331330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9338105880 msgr2=0x7f933818f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:34.538 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:34.182+0000 7f933f1e3700 1 --2- 172.21.15.40:0/209331330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9338105880 0x7f933818f410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f932c003e60 tx=0x7f932c011aa0 comp rx=0 tx=0).stop 2023-07-18T03:23:34.538 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:34.182+0000 7f933f1e3700 1 -- 172.21.15.40:0/209331330 shutdown_connections 2023-07-18T03:23:34.539 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:34.182+0000 7f933f1e3700 1 --2- 172.21.15.40:0/209331330 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f93280385f0 0x7f932803aaa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:34.539 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:34.182+0000 7f933f1e3700 1 --2- 172.21.15.40:0/209331330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9338105880 0x7f933818f410 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:34.539 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:34.182+0000 7f933f1e3700 1 -- 172.21.15.40:0/209331330 >> 172.21.15.40:0/209331330 conn(0x7f93380ff040 msgr2=0x7f93381035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:34.539 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:34.182+0000 7f933f1e3700 1 -- 172.21.15.40:0/209331330 shutdown_connections 2023-07-18T03:23:34.539 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:34.182+0000 7f933f1e3700 1 -- 172.21.15.40:0/209331330 wait complete. 2023-07-18T03:23:34.539 INFO:teuthology.orchestra.run.smithi040.stdout:Creating initial admin user... 2023-07-18T03:23:34.746 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:34 smithi040 ceph-mon[96439]: [18/Jul/2023:03:23:32] ENGINE Bus STARTING 2023-07-18T03:23:34.746 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:34 smithi040 ceph-mon[96439]: [18/Jul/2023:03:23:33] ENGINE Serving on http://172.21.15.40:8765 2023-07-18T03:23:34.746 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:34 smithi040 ceph-mon[96439]: [18/Jul/2023:03:23:33] ENGINE Serving on https://172.21.15.40:7150 2023-07-18T03:23:34.746 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:34 smithi040 ceph-mon[96439]: [18/Jul/2023:03:23:33] ENGINE Bus STARTED 2023-07-18T03:23:34.746 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:34 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:34.746 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:34 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:34.747 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:34 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:36.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:35 smithi040 ceph-mon[96439]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:23:36.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:35 smithi040 ceph-mon[96439]: mgrmap e12: x(active, since 3s) 2023-07-18T03:23:36.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:36.379 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$zBQ2YnDGDVKljtH.BS7Zvezp61t7Rto6s1dMi0PBaInKL83XYy3/G", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1689650615, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-07-18T03:23:36.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.044+0000 7f77a400c700 1 Processor -- start 2023-07-18T03:23:36.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.044+0000 7f77a400c700 1 -- start start 2023-07-18T03:23:36.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.044+0000 7f77a400c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f779c07bdc0 0x7f779c07a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:36.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.044+0000 7f77a400c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c07a800 con 0x7f779c07bdc0 2023-07-18T03:23:36.380 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.045+0000 7f77a1da8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f779c07bdc0 0x7f779c07a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:36.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.045+0000 7f77a1da8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f779c07bdc0 0x7f779c07a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55078/0 (socket says 172.21.15.40:55078) 2023-07-18T03:23:36.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.045+0000 7f77a1da8700 1 -- 172.21.15.40:0/1698796264 learned_addr learned my addr 172.21.15.40:0/1698796264 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:36.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.045+0000 7f77a1da8700 1 -- 172.21.15.40:0/1698796264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779c07a940 con 0x7f779c07bdc0 2023-07-18T03:23:36.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.045+0000 7f77a1da8700 1 --2- 172.21.15.40:0/1698796264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f779c07bdc0 0x7f779c07a2c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f77980098a0 tx=0x7f7798009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f9f91da6bfe665 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:36.381 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.045+0000 7f77a0da6700 1 -- 172.21.15.40:0/1698796264 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7798004160 con 0x7f779c07bdc0 2023-07-18T03:23:36.382 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.045+0000 7f77a0da6700 1 -- 172.21.15.40:0/1698796264 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77980042c0 con 0x7f779c07bdc0 2023-07-18T03:23:36.382 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.045+0000 7f77a0da6700 1 -- 172.21.15.40:0/1698796264 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77980119b0 con 0x7f779c07bdc0 2023-07-18T03:23:36.382 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.046+0000 7f77a400c700 1 -- 172.21.15.40:0/1698796264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f779c07bdc0 msgr2=0x7f779c07a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:36.382 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.046+0000 7f77a400c700 1 --2- 172.21.15.40:0/1698796264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f779c07bdc0 0x7f779c07a2c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f77980098a0 tx=0x7f7798009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:36.382 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.046+0000 7f77a400c700 1 -- 172.21.15.40:0/1698796264 shutdown_connections 2023-07-18T03:23:36.382 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.046+0000 7f77a400c700 1 --2- 172.21.15.40:0/1698796264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f779c07bdc0 0x7f779c07a2c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:36.382 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.046+0000 7f77a400c700 1 -- 172.21.15.40:0/1698796264 >> 172.21.15.40:0/1698796264 conn(0x7f779c1009e0 msgr2=0x7f779c102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:36.382 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.046+0000 7f77a400c700 1 -- 172.21.15.40:0/1698796264 shutdown_connections 2023-07-18T03:23:36.383 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.046+0000 7f77a400c700 1 -- 172.21.15.40:0/1698796264 wait complete. 2023-07-18T03:23:36.383 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.046+0000 7f77a400c700 1 Processor -- start 2023-07-18T03:23:36.383 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.046+0000 7f77a400c700 1 -- start start 2023-07-18T03:23:36.383 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.047+0000 7f77a400c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f779c07bdc0 0x7f779c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:36.383 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.047+0000 7f77a400c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c19b900 con 0x7f779c07bdc0 2023-07-18T03:23:36.383 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.047+0000 7f77a1da8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f779c07bdc0 0x7f779c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:36.383 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.047+0000 7f77a1da8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f779c07bdc0 0x7f779c19b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55080/0 (socket says 172.21.15.40:55080) 2023-07-18T03:23:36.383 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.047+0000 7f77a1da8700 1 -- 172.21.15.40:0/1930786203 learned_addr learned my addr 172.21.15.40:0/1930786203 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:36.384 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.047+0000 7f77a1da8700 1 -- 172.21.15.40:0/1930786203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7798009550 con 0x7f779c07bdc0 2023-07-18T03:23:36.384 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.047+0000 7f77a1da8700 1 --2- 172.21.15.40:0/1930786203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f779c07bdc0 0x7f779c19b3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7798003e60 tx=0x7f7798011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:36.384 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.047+0000 7f7792ffd700 1 -- 172.21.15.40:0/1930786203 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7798003a60 con 0x7f779c07bdc0 2023-07-18T03:23:36.384 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.047+0000 7f7792ffd700 1 -- 172.21.15.40:0/1930786203 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7798003bc0 con 0x7f779c07bdc0 2023-07-18T03:23:36.384 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.047+0000 7f77a400c700 1 -- 172.21.15.40:0/1930786203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779c19bb00 con 0x7f779c07bdc0 2023-07-18T03:23:36.384 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.047+0000 7f7792ffd700 1 -- 172.21.15.40:0/1930786203 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77980196c0 con 0x7f779c07bdc0 2023-07-18T03:23:36.384 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.047+0000 7f77a400c700 1 -- 172.21.15.40:0/1930786203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779c19bf20 con 0x7f779c07bdc0 2023-07-18T03:23:36.385 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.048+0000 7f7792ffd700 1 -- 172.21.15.40:0/1930786203 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f7798019930 con 0x7f779c07bdc0 2023-07-18T03:23:36.385 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.048+0000 7f7792ffd700 1 --2- 172.21.15.40:0/1930786203 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7788038790 0x7f778803ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:36.385 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.048+0000 7f7792ffd700 1 -- 172.21.15.40:0/1930786203 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f779804ca70 con 0x7f779c07bdc0 2023-07-18T03:23:36.385 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.049+0000 7f77a15a7700 1 --2- 172.21.15.40:0/1930786203 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7788038790 0x7f778803ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:36.385 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.049+0000 7f77a400c700 1 -- 172.21.15.40:0/1930786203 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f779c195370 con 0x7f779c07bdc0 2023-07-18T03:23:36.385 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.049+0000 7f77a15a7700 1 --2- 172.21.15.40:0/1930786203 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7788038790 0x7f778803ac40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f778c006fd0 tx=0x7f778c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:36.385 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.054+0000 7f7792ffd700 1 -- 172.21.15.40:0/1930786203 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7798019e60 con 0x7f779c07bdc0 2023-07-18T03:23:36.385 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.450+0000 7f77a400c700 1 -- 172.21.15.40:0/1930786203 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f779c061960 con 0x7f7788038790 2023-07-18T03:23:36.386 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.700+0000 7f7792ffd700 1 -- 172.21.15.40:0/1930786203 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f779c061960 con 0x7f7788038790 2023-07-18T03:23:36.386 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.702+0000 7f77a400c700 1 -- 172.21.15.40:0/1930786203 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7788038790 msgr2=0x7f778803ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:36.386 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.702+0000 7f77a400c700 1 --2- 172.21.15.40:0/1930786203 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7788038790 0x7f778803ac40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f778c006fd0 tx=0x7f778c006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:36.386 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.702+0000 7f77a400c700 1 -- 172.21.15.40:0/1930786203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f779c07bdc0 msgr2=0x7f779c19b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:36.386 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.702+0000 7f77a400c700 1 --2- 172.21.15.40:0/1930786203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f779c07bdc0 0x7f779c19b3c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7798003e60 tx=0x7f7798011a30 comp rx=0 tx=0).stop 2023-07-18T03:23:36.386 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.702+0000 7f77a400c700 1 -- 172.21.15.40:0/1930786203 shutdown_connections 2023-07-18T03:23:36.386 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.702+0000 7f77a400c700 1 --2- 172.21.15.40:0/1930786203 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7788038790 0x7f778803ac40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:36.387 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.702+0000 7f77a400c700 1 --2- 172.21.15.40:0/1930786203 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f779c07bdc0 0x7f779c19b3c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:36.387 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.702+0000 7f77a400c700 1 -- 172.21.15.40:0/1930786203 >> 172.21.15.40:0/1930786203 conn(0x7f779c1009e0 msgr2=0x7f779c102140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:36.387 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.702+0000 7f77a400c700 1 -- 172.21.15.40:0/1930786203 shutdown_connections 2023-07-18T03:23:36.387 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:35.703+0000 7f77a400c700 1 -- 172.21.15.40:0/1930786203 wait complete. 2023-07-18T03:23:36.387 INFO:teuthology.orchestra.run.smithi040.stdout:Fetching dashboard port number... 2023-07-18T03:23:37.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:36 smithi040 ceph-mon[96439]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:23:37.620 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stdout 8443 2023-07-18T03:23:37.624 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.978+0000 7f3a6bf4f700 1 Processor -- start 2023-07-18T03:23:37.624 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.978+0000 7f3a6bf4f700 1 -- start start 2023-07-18T03:23:37.624 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.978+0000 7f3a6bf4f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a64104280 0x7f3a64104690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:37.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.978+0000 7f3a6bf4f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a64104bd0 con 0x7f3a64104280 2023-07-18T03:23:37.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.978+0000 7f3a69ceb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a64104280 0x7f3a64104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:37.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.978+0000 7f3a69ceb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a64104280 0x7f3a64104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55084/0 (socket says 172.21.15.40:55084) 2023-07-18T03:23:37.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.978+0000 7f3a69ceb700 1 -- 172.21.15.40:0/181893702 learned_addr learned my addr 172.21.15.40:0/181893702 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:37.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.979+0000 7f3a69ceb700 1 -- 172.21.15.40:0/181893702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a64104d10 con 0x7f3a64104280 2023-07-18T03:23:37.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.979+0000 7f3a69ceb700 1 --2- 172.21.15.40:0/181893702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a64104280 0x7f3a64104690 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3a600098a0 tx=0x7f3a60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8741069f5f66b28 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:37.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.979+0000 7f3a68ce9700 1 -- 172.21.15.40:0/181893702 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a60004160 con 0x7f3a64104280 2023-07-18T03:23:37.625 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.979+0000 7f3a68ce9700 1 -- 172.21.15.40:0/181893702 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a600042c0 con 0x7f3a64104280 2023-07-18T03:23:37.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.979+0000 7f3a68ce9700 1 -- 172.21.15.40:0/181893702 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a600119b0 con 0x7f3a64104280 2023-07-18T03:23:37.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.980+0000 7f3a6bf4f700 1 -- 172.21.15.40:0/181893702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a64104280 msgr2=0x7f3a64104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:37.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.980+0000 7f3a6bf4f700 1 --2- 172.21.15.40:0/181893702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a64104280 0x7f3a64104690 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3a600098a0 tx=0x7f3a60009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:37.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.980+0000 7f3a6bf4f700 1 -- 172.21.15.40:0/181893702 shutdown_connections 2023-07-18T03:23:37.626 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.980+0000 7f3a6bf4f700 1 --2- 172.21.15.40:0/181893702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a64104280 0x7f3a64104690 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:37.627 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.980+0000 7f3a6bf4f700 1 -- 172.21.15.40:0/181893702 >> 172.21.15.40:0/181893702 conn(0x7f3a640ff8d0 msgr2=0x7f3a64101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:37.627 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.980+0000 7f3a6bf4f700 1 -- 172.21.15.40:0/181893702 shutdown_connections 2023-07-18T03:23:37.627 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.980+0000 7f3a6bf4f700 1 -- 172.21.15.40:0/181893702 wait complete. 2023-07-18T03:23:37.627 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.980+0000 7f3a6bf4f700 1 Processor -- start 2023-07-18T03:23:37.627 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.980+0000 7f3a6bf4f700 1 -- start start 2023-07-18T03:23:37.627 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.981+0000 7f3a6bf4f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a64104280 0x7f3a6419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:37.628 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.981+0000 7f3a6bf4f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6419b840 con 0x7f3a64104280 2023-07-18T03:23:37.628 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.981+0000 7f3a69ceb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a64104280 0x7f3a6419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:37.628 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.981+0000 7f3a69ceb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a64104280 0x7f3a6419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55086/0 (socket says 172.21.15.40:55086) 2023-07-18T03:23:37.628 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.981+0000 7f3a69ceb700 1 -- 172.21.15.40:0/3230909469 learned_addr learned my addr 172.21.15.40:0/3230909469 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:37.628 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.981+0000 7f3a69ceb700 1 -- 172.21.15.40:0/3230909469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a60009550 con 0x7f3a64104280 2023-07-18T03:23:37.628 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.981+0000 7f3a69ceb700 1 --2- 172.21.15.40:0/3230909469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a64104280 0x7f3a6419b300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a60011980 tx=0x7f3a60011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:37.628 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.981+0000 7f3a5affd700 1 -- 172.21.15.40:0/3230909469 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a60003a60 con 0x7f3a64104280 2023-07-18T03:23:37.629 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.982+0000 7f3a5affd700 1 -- 172.21.15.40:0/3230909469 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a60003bc0 con 0x7f3a64104280 2023-07-18T03:23:37.629 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.982+0000 7f3a6bf4f700 1 -- 172.21.15.40:0/3230909469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6419ba40 con 0x7f3a64104280 2023-07-18T03:23:37.629 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.982+0000 7f3a5affd700 1 -- 172.21.15.40:0/3230909469 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a6001a670 con 0x7f3a64104280 2023-07-18T03:23:37.629 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.982+0000 7f3a6bf4f700 1 -- 172.21.15.40:0/3230909469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6419be60 con 0x7f3a64104280 2023-07-18T03:23:37.629 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.982+0000 7f3a5affd700 1 -- 172.21.15.40:0/3230909469 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f3a6001a930 con 0x7f3a64104280 2023-07-18T03:23:37.629 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.983+0000 7f3a5affd700 1 --2- 172.21.15.40:0/3230909469 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3a50038760 0x7f3a5003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:37.629 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.983+0000 7f3a5affd700 1 -- 172.21.15.40:0/3230909469 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3a6004c900 con 0x7f3a64104280 2023-07-18T03:23:37.630 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.983+0000 7f3a694ea700 1 --2- 172.21.15.40:0/3230909469 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3a50038760 0x7f3a5003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:37.630 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.983+0000 7f3a6bf4f700 1 -- 172.21.15.40:0/3230909469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a64061b40 con 0x7f3a64104280 2023-07-18T03:23:37.630 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.983+0000 7f3a694ea700 1 --2- 172.21.15.40:0/3230909469 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3a50038760 0x7f3a5003ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3a54006fd0 tx=0x7f3a54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:37.630 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:36.988+0000 7f3a5affd700 1 -- 172.21.15.40:0/3230909469 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3a6001ac30 con 0x7f3a64104280 2023-07-18T03:23:37.630 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:37.343+0000 7f3a6bf4f700 1 -- 172.21.15.40:0/3230909469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f3a6419ed60 con 0x7f3a64104280 2023-07-18T03:23:37.630 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:37.344+0000 7f3a5affd700 1 -- 172.21.15.40:0/3230909469 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f3a60018350 con 0x7f3a64104280 2023-07-18T03:23:37.630 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:37.346+0000 7f3a6bf4f700 1 -- 172.21.15.40:0/3230909469 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3a50038760 msgr2=0x7f3a5003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:37.632 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:37.346+0000 7f3a6bf4f700 1 --2- 172.21.15.40:0/3230909469 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3a50038760 0x7f3a5003ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3a54006fd0 tx=0x7f3a54006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:37.632 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:37.346+0000 7f3a6bf4f700 1 -- 172.21.15.40:0/3230909469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a64104280 msgr2=0x7f3a6419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:37.632 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:37.346+0000 7f3a6bf4f700 1 --2- 172.21.15.40:0/3230909469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a64104280 0x7f3a6419b300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a60011980 tx=0x7f3a60011a60 comp rx=0 tx=0).stop 2023-07-18T03:23:37.632 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:37.346+0000 7f3a6bf4f700 1 -- 172.21.15.40:0/3230909469 shutdown_connections 2023-07-18T03:23:37.632 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:37.346+0000 7f3a6bf4f700 1 --2- 172.21.15.40:0/3230909469 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3a50038760 0x7f3a5003ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:37.632 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:37.346+0000 7f3a6bf4f700 1 --2- 172.21.15.40:0/3230909469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a64104280 0x7f3a6419b300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:37.633 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:37.346+0000 7f3a6bf4f700 1 -- 172.21.15.40:0/3230909469 >> 172.21.15.40:0/3230909469 conn(0x7f3a640ff8d0 msgr2=0x7f3a64072fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:37.633 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:37.346+0000 7f3a6bf4f700 1 -- 172.21.15.40:0/3230909469 shutdown_connections 2023-07-18T03:23:37.634 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:37.346+0000 7f3a6bf4f700 1 -- 172.21.15.40:0/3230909469 wait complete. 2023-07-18T03:23:37.634 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-18T03:23:37.635 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout disabled 2023-07-18T03:23:37.640 INFO:teuthology.orchestra.run.smithi040.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-18T03:23:37.640 INFO:teuthology.orchestra.run.smithi040.stdout:systemctl: stdout inactive 2023-07-18T03:23:37.641 INFO:teuthology.orchestra.run.smithi040.stdout:firewalld.service is not enabled 2023-07-18T03:23:37.641 INFO:teuthology.orchestra.run.smithi040.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-07-18T03:23:37.642 INFO:teuthology.orchestra.run.smithi040.stdout:Ceph Dashboard is now available at: 2023-07-18T03:23:37.642 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:23:37.642 INFO:teuthology.orchestra.run.smithi040.stdout: URL: https://smithi040:8443/ 2023-07-18T03:23:37.642 INFO:teuthology.orchestra.run.smithi040.stdout: User: admin 2023-07-18T03:23:37.643 INFO:teuthology.orchestra.run.smithi040.stdout: Password: 795u2iplcd 2023-07-18T03:23:37.643 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:23:37.643 INFO:teuthology.orchestra.run.smithi040.stdout:Saving cluster configuration to /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config directory 2023-07-18T03:23:37.643 INFO:teuthology.orchestra.run.smithi040.stdout:Enabling autotune for osd_memory_target 2023-07-18T03:23:38.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:37 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3230909469' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-07-18T03:23:38.890 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.095+0000 7fbcfe9c0700 1 Processor -- start 2023-07-18T03:23:38.891 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.095+0000 7fbcfe9c0700 1 -- start start 2023-07-18T03:23:38.891 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.095+0000 7fbcfe9c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf8105370 0x7fbcf8105780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:38.891 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.095+0000 7fbcfe9c0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf8105cc0 con 0x7fbcf8105370 2023-07-18T03:23:38.891 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.095+0000 7fbcf7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf8105370 0x7fbcf8105780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:38.891 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.095+0000 7fbcf7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf8105370 0x7fbcf8105780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55094/0 (socket says 172.21.15.40:55094) 2023-07-18T03:23:38.891 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.095+0000 7fbcf7fff700 1 -- 172.21.15.40:0/777458163 learned_addr learned my addr 172.21.15.40:0/777458163 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:38.891 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.096+0000 7fbcf7fff700 1 -- 172.21.15.40:0/777458163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf8105e00 con 0x7fbcf8105370 2023-07-18T03:23:38.892 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.096+0000 7fbcf7fff700 1 --2- 172.21.15.40:0/777458163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf8105370 0x7fbcf8105780 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbce80098a0 tx=0x7fbce8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dd11282fb0c57e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:38.892 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.096+0000 7fbcf6ffd700 1 -- 172.21.15.40:0/777458163 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbce8004160 con 0x7fbcf8105370 2023-07-18T03:23:38.892 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.096+0000 7fbcf6ffd700 1 -- 172.21.15.40:0/777458163 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbce80042c0 con 0x7fbcf8105370 2023-07-18T03:23:38.892 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.096+0000 7fbcf6ffd700 1 -- 172.21.15.40:0/777458163 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbce80119b0 con 0x7fbcf8105370 2023-07-18T03:23:38.892 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.096+0000 7fbcfe9c0700 1 -- 172.21.15.40:0/777458163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf8105370 msgr2=0x7fbcf8105780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:38.892 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.097+0000 7fbcfe9c0700 1 --2- 172.21.15.40:0/777458163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf8105370 0x7fbcf8105780 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbce80098a0 tx=0x7fbce8009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:38.892 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.097+0000 7fbcfe9c0700 1 -- 172.21.15.40:0/777458163 shutdown_connections 2023-07-18T03:23:38.893 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.097+0000 7fbcfe9c0700 1 --2- 172.21.15.40:0/777458163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf8105370 0x7fbcf8105780 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:38.893 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.097+0000 7fbcfe9c0700 1 -- 172.21.15.40:0/777458163 >> 172.21.15.40:0/777458163 conn(0x7fbcf81009e0 msgr2=0x7fbcf8102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:38.893 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.097+0000 7fbcfe9c0700 1 -- 172.21.15.40:0/777458163 shutdown_connections 2023-07-18T03:23:38.893 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.097+0000 7fbcfe9c0700 1 -- 172.21.15.40:0/777458163 wait complete. 2023-07-18T03:23:38.893 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.097+0000 7fbcfe9c0700 1 Processor -- start 2023-07-18T03:23:38.893 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.097+0000 7fbcfe9c0700 1 -- start start 2023-07-18T03:23:38.893 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.098+0000 7fbcfe9c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf8105370 0x7fbcf807bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:38.894 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.098+0000 7fbcfe9c0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf807c170 con 0x7fbcf8105370 2023-07-18T03:23:38.894 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.098+0000 7fbcf7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf8105370 0x7fbcf807bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:38.894 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.098+0000 7fbcf7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf8105370 0x7fbcf807bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55096/0 (socket says 172.21.15.40:55096) 2023-07-18T03:23:38.894 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.098+0000 7fbcf7fff700 1 -- 172.21.15.40:0/4227852101 learned_addr learned my addr 172.21.15.40:0/4227852101 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:38.894 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.098+0000 7fbcf7fff700 1 -- 172.21.15.40:0/4227852101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce8009550 con 0x7fbcf8105370 2023-07-18T03:23:38.894 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.098+0000 7fbcf7fff700 1 --2- 172.21.15.40:0/4227852101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf8105370 0x7fbcf807bc30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbce8006af0 tx=0x7fbce8011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:38.895 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.098+0000 7fbcf57fa700 1 -- 172.21.15.40:0/4227852101 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbce8003a40 con 0x7fbcf8105370 2023-07-18T03:23:38.895 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.098+0000 7fbcf57fa700 1 -- 172.21.15.40:0/4227852101 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbce8003ba0 con 0x7fbcf8105370 2023-07-18T03:23:38.895 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.098+0000 7fbcfe9c0700 1 -- 172.21.15.40:0/4227852101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf807a280 con 0x7fbcf8105370 2023-07-18T03:23:38.895 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.098+0000 7fbcf57fa700 1 -- 172.21.15.40:0/4227852101 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbce80196c0 con 0x7fbcf8105370 2023-07-18T03:23:38.895 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.099+0000 7fbcfe9c0700 1 -- 172.21.15.40:0/4227852101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf807a6a0 con 0x7fbcf8105370 2023-07-18T03:23:38.895 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.099+0000 7fbcf57fa700 1 -- 172.21.15.40:0/4227852101 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fbce8019980 con 0x7fbcf8105370 2023-07-18T03:23:38.895 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.099+0000 7fbcf57fa700 1 --2- 172.21.15.40:0/4227852101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbce00387b0 0x7fbce003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:38.896 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.100+0000 7fbcf57fa700 1 -- 172.21.15.40:0/4227852101 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbce804cc50 con 0x7fbcf8105370 2023-07-18T03:23:38.896 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.100+0000 7fbcfe9c0700 1 -- 172.21.15.40:0/4227852101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf8061b40 con 0x7fbcf8105370 2023-07-18T03:23:38.896 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.100+0000 7fbcf77fe700 1 --2- 172.21.15.40:0/4227852101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbce00387b0 0x7fbce003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:38.896 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.100+0000 7fbcf77fe700 1 --2- 172.21.15.40:0/4227852101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbce00387b0 0x7fbce003ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbcec006fd0 tx=0x7fbcec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:38.896 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.105+0000 7fbcf57fa700 1 -- 172.21.15.40:0/4227852101 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbce80243f0 con 0x7fbcf8105370 2023-07-18T03:23:38.897 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.461+0000 7fbcfe9c0700 1 -- 172.21.15.40:0/4227852101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fbcf807afc0 con 0x7fbcf8105370 2023-07-18T03:23:38.897 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.461+0000 7fbcf57fa700 1 -- 172.21.15.40:0/4227852101 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fbce804a020 con 0x7fbcf8105370 2023-07-18T03:23:38.897 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.464+0000 7fbcfe9c0700 1 -- 172.21.15.40:0/4227852101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbce00387b0 msgr2=0x7fbce003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:38.897 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.464+0000 7fbcfe9c0700 1 --2- 172.21.15.40:0/4227852101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbce00387b0 0x7fbce003ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbcec006fd0 tx=0x7fbcec006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:38.897 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.464+0000 7fbcfe9c0700 1 -- 172.21.15.40:0/4227852101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf8105370 msgr2=0x7fbcf807bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:38.897 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.464+0000 7fbcfe9c0700 1 --2- 172.21.15.40:0/4227852101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf8105370 0x7fbcf807bc30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbce8006af0 tx=0x7fbce8011a10 comp rx=0 tx=0).stop 2023-07-18T03:23:38.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.464+0000 7fbcfe9c0700 1 -- 172.21.15.40:0/4227852101 shutdown_connections 2023-07-18T03:23:38.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.464+0000 7fbcfe9c0700 1 --2- 172.21.15.40:0/4227852101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbce00387b0 0x7fbce003ac60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:38.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.464+0000 7fbcfe9c0700 1 --2- 172.21.15.40:0/4227852101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcf8105370 0x7fbcf807bc30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:38.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.464+0000 7fbcfe9c0700 1 -- 172.21.15.40:0/4227852101 >> 172.21.15.40:0/4227852101 conn(0x7fbcf81009e0 msgr2=0x7fbcf81020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:38.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.464+0000 7fbcfe9c0700 1 -- 172.21.15.40:0/4227852101 shutdown_connections 2023-07-18T03:23:38.898 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:38.464+0000 7fbcfe9c0700 1 -- 172.21.15.40:0/4227852101 wait complete. 2023-07-18T03:23:39.069 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:38 smithi040 ceph-mon[96439]: mgrmap e13: x(active, since 6s) 2023-07-18T03:23:40.240 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.326+0000 7fb6e0ee6700 1 Processor -- start 2023-07-18T03:23:40.240 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.326+0000 7fb6e0ee6700 1 -- start start 2023-07-18T03:23:40.240 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.326+0000 7fb6e0ee6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6dc104280 0x7fb6dc104690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:40.240 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.326+0000 7fb6e0ee6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc104bd0 con 0x7fb6dc104280 2023-07-18T03:23:40.240 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.326+0000 7fb6da59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6dc104280 0x7fb6dc104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:40.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.326+0000 7fb6da59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6dc104280 0x7fb6dc104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55100/0 (socket says 172.21.15.40:55100) 2023-07-18T03:23:40.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.326+0000 7fb6da59c700 1 -- 172.21.15.40:0/1989310658 learned_addr learned my addr 172.21.15.40:0/1989310658 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:40.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.327+0000 7fb6da59c700 1 -- 172.21.15.40:0/1989310658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6dc104d10 con 0x7fb6dc104280 2023-07-18T03:23:40.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.327+0000 7fb6da59c700 1 --2- 172.21.15.40:0/1989310658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6dc104280 0x7fb6dc104690 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00098a0 tx=0x7fb6d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b99bdb226838b10 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:40.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.327+0000 7fb6d959a700 1 -- 172.21.15.40:0/1989310658 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6d0004160 con 0x7fb6dc104280 2023-07-18T03:23:40.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.327+0000 7fb6d959a700 1 -- 172.21.15.40:0/1989310658 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6d00042c0 con 0x7fb6dc104280 2023-07-18T03:23:40.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.327+0000 7fb6d959a700 1 -- 172.21.15.40:0/1989310658 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6d00119b0 con 0x7fb6dc104280 2023-07-18T03:23:40.241 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.328+0000 7fb6e0ee6700 1 -- 172.21.15.40:0/1989310658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6dc104280 msgr2=0x7fb6dc104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:40.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.328+0000 7fb6e0ee6700 1 --2- 172.21.15.40:0/1989310658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6dc104280 0x7fb6dc104690 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00098a0 tx=0x7fb6d0009bb0 comp rx=0 tx=0).stop 2023-07-18T03:23:40.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.328+0000 7fb6e0ee6700 1 -- 172.21.15.40:0/1989310658 shutdown_connections 2023-07-18T03:23:40.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.328+0000 7fb6e0ee6700 1 --2- 172.21.15.40:0/1989310658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6dc104280 0x7fb6dc104690 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:40.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.328+0000 7fb6e0ee6700 1 -- 172.21.15.40:0/1989310658 >> 172.21.15.40:0/1989310658 conn(0x7fb6dc0ff8d0 msgr2=0x7fb6dc101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:40.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.328+0000 7fb6e0ee6700 1 -- 172.21.15.40:0/1989310658 shutdown_connections 2023-07-18T03:23:40.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.328+0000 7fb6e0ee6700 1 -- 172.21.15.40:0/1989310658 wait complete. 2023-07-18T03:23:40.242 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.329+0000 7fb6e0ee6700 1 Processor -- start 2023-07-18T03:23:40.243 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.329+0000 7fb6e0ee6700 1 -- start start 2023-07-18T03:23:40.243 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.329+0000 7fb6e0ee6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6dc104280 0x7fb6dc197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:40.243 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.329+0000 7fb6e0ee6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6dc198530 con 0x7fb6dc104280 2023-07-18T03:23:40.243 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.329+0000 7fb6da59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6dc104280 0x7fb6dc197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:40.243 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.329+0000 7fb6da59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6dc104280 0x7fb6dc197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55102/0 (socket says 172.21.15.40:55102) 2023-07-18T03:23:40.243 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.329+0000 7fb6da59c700 1 -- 172.21.15.40:0/3043601927 learned_addr learned my addr 172.21.15.40:0/3043601927 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:40.243 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.330+0000 7fb6da59c700 1 -- 172.21.15.40:0/3043601927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d0009550 con 0x7fb6dc104280 2023-07-18T03:23:40.243 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.330+0000 7fb6da59c700 1 --2- 172.21.15.40:0/3043601927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6dc104280 0x7fb6dc197ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0011980 tx=0x7fb6d0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:40.244 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.330+0000 7fb6cb7fe700 1 -- 172.21.15.40:0/3043601927 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6d0003a60 con 0x7fb6dc104280 2023-07-18T03:23:40.244 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.330+0000 7fb6e0ee6700 1 -- 172.21.15.40:0/3043601927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6dc198730 con 0x7fb6dc104280 2023-07-18T03:23:40.244 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.330+0000 7fb6cb7fe700 1 -- 172.21.15.40:0/3043601927 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6d0003bc0 con 0x7fb6dc104280 2023-07-18T03:23:40.244 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.330+0000 7fb6cb7fe700 1 -- 172.21.15.40:0/3043601927 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6d001a670 con 0x7fb6dc104280 2023-07-18T03:23:40.244 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.330+0000 7fb6e0ee6700 1 -- 172.21.15.40:0/3043601927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6dc198b50 con 0x7fb6dc104280 2023-07-18T03:23:40.244 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.331+0000 7fb6cb7fe700 1 -- 172.21.15.40:0/3043601927 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fb6d0024a00 con 0x7fb6dc104280 2023-07-18T03:23:40.245 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.331+0000 7fb6e0ee6700 1 -- 172.21.15.40:0/3043601927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6dc061b40 con 0x7fb6dc104280 2023-07-18T03:23:40.245 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.331+0000 7fb6cb7fe700 1 --2- 172.21.15.40:0/3043601927 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb6c4038800 0x7fb6c403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:40.245 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.331+0000 7fb6cb7fe700 1 -- 172.21.15.40:0/3043601927 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb6d004ca00 con 0x7fb6dc104280 2023-07-18T03:23:40.245 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.332+0000 7fb6d9d9b700 1 --2- 172.21.15.40:0/3043601927 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb6c4038800 0x7fb6c403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:40.245 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.332+0000 7fb6d9d9b700 1 --2- 172.21.15.40:0/3043601927 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb6c4038800 0x7fb6c403acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc006fd0 tx=0x7fb6cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:40.245 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.336+0000 7fb6cb7fe700 1 -- 172.21.15.40:0/3043601927 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb6d0024440 con 0x7fb6dc104280 2023-07-18T03:23:40.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.910+0000 7fb6e0ee6700 1 -- 172.21.15.40:0/3043601927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fb6dc19bbc0 con 0x7fb6dc104280 2023-07-18T03:23:40.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.912+0000 7fb6cb7fe700 1 -- 172.21.15.40:0/3043601927 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fb6d0018350 con 0x7fb6dc104280 2023-07-18T03:23:40.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.914+0000 7fb6e0ee6700 1 -- 172.21.15.40:0/3043601927 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb6c4038800 msgr2=0x7fb6c403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:40.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.914+0000 7fb6e0ee6700 1 --2- 172.21.15.40:0/3043601927 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb6c4038800 0x7fb6c403acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc006fd0 tx=0x7fb6cc006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:40.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.914+0000 7fb6e0ee6700 1 -- 172.21.15.40:0/3043601927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6dc104280 msgr2=0x7fb6dc197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:40.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.914+0000 7fb6e0ee6700 1 --2- 172.21.15.40:0/3043601927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6dc104280 0x7fb6dc197ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0011980 tx=0x7fb6d0011a60 comp rx=0 tx=0).stop 2023-07-18T03:23:40.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.915+0000 7fb6e0ee6700 1 -- 172.21.15.40:0/3043601927 shutdown_connections 2023-07-18T03:23:40.246 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.915+0000 7fb6e0ee6700 1 --2- 172.21.15.40:0/3043601927 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb6c4038800 0x7fb6c403acb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:40.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.915+0000 7fb6e0ee6700 1 --2- 172.21.15.40:0/3043601927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6dc104280 0x7fb6dc197ff0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:40.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.915+0000 7fb6e0ee6700 1 -- 172.21.15.40:0/3043601927 >> 172.21.15.40:0/3043601927 conn(0x7fb6dc0ff8d0 msgr2=0x7fb6dc100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:40.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.915+0000 7fb6e0ee6700 1 -- 172.21.15.40:0/3043601927 shutdown_connections 2023-07-18T03:23:40.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr 2023-07-18T03:23:39.915+0000 7fb6e0ee6700 1 -- 172.21.15.40:0/3043601927 wait complete. 2023-07-18T03:23:40.247 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-07-18T03:23:40.247 INFO:teuthology.orchestra.run.smithi040.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-07-18T03:23:40.247 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:23:40.248 INFO:teuthology.orchestra.run.smithi040.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-07-18T03:23:40.248 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:23:40.248 INFO:teuthology.orchestra.run.smithi040.stdout:Or, if you are only running a single cluster on this host: 2023-07-18T03:23:40.248 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:23:40.248 INFO:teuthology.orchestra.run.smithi040.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-07-18T03:23:40.248 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:23:40.248 INFO:teuthology.orchestra.run.smithi040.stdout:Please consider enabling telemetry to help improve Ceph: 2023-07-18T03:23:40.248 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:23:40.249 INFO:teuthology.orchestra.run.smithi040.stdout: ceph telemetry on 2023-07-18T03:23:40.249 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:23:40.249 INFO:teuthology.orchestra.run.smithi040.stdout:For more information see: 2023-07-18T03:23:40.249 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:23:40.249 INFO:teuthology.orchestra.run.smithi040.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-07-18T03:23:40.249 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:23:40.249 INFO:teuthology.orchestra.run.smithi040.stdout:Bootstrap complete. 2023-07-18T03:23:40.273 INFO:tasks.cephadm:Fetching config... 2023-07-18T03:23:40.273 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:23:40.274 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-07-18T03:23:40.291 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-07-18T03:23:40.291 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:23:40.292 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-07-18T03:23:40.351 INFO:tasks.cephadm:Fetching mon keyring... 2023-07-18T03:23:40.351 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:23:40.352 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-07-18T03:23:40.419 INFO:tasks.cephadm:Fetching pub ssh key... 2023-07-18T03:23:40.420 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:23:40.420 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-07-18T03:23:40.478 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-07-18T03:23:40.478 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZMp7JKokJelbqW4Vgy8MiDhicnmIkxl2sVzIsSkUNwlmOUDo8quH/haUHYaou75IXaBcwTZCUdPBjPROyYS4c09r/whjxEpeE3xfKS1Ua+lhUdUmUV+gJj41Jmx1xXHKm4zS9ZXRkfbQGAyQVG/sQn9v3vDlOMqxMXgsrjSnG5u19YZjx3QcQM9ixY6Fa0B2X+G+jONQWipBE3zQHr1pdNsCAPPNcnQElJvzSWOSLLag0NSUb2vNmcTdGKTRxShooiZo9TBK5m2c96IvBclw1Q9iXtU44ot+D52aigoeWIYBLhaTohz69ezIC/kYSye03eYerLq9e8GHeHmitRV8hMF8E6WNO2iWN95LJqojOLvIzfeusweVAwyixzokvuOe5osrzvS/Q3Wi/CA7fqwpIza9mzkJ7HzSkyV65IrutPcfeNtLwzT9UJPSl6XeFgDhLJDMd8QhkWvleTSQiSd4vgPrYy7hZRFp/qkTdoUU0u94f+Ul5ab+o1w5Da0jjXX0= ceph-3af3dba8-251a-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-18T03:23:40.558 INFO:teuthology.orchestra.run.smithi040.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZMp7JKokJelbqW4Vgy8MiDhicnmIkxl2sVzIsSkUNwlmOUDo8quH/haUHYaou75IXaBcwTZCUdPBjPROyYS4c09r/whjxEpeE3xfKS1Ua+lhUdUmUV+gJj41Jmx1xXHKm4zS9ZXRkfbQGAyQVG/sQn9v3vDlOMqxMXgsrjSnG5u19YZjx3QcQM9ixY6Fa0B2X+G+jONQWipBE3zQHr1pdNsCAPPNcnQElJvzSWOSLLag0NSUb2vNmcTdGKTRxShooiZo9TBK5m2c96IvBclw1Q9iXtU44ot+D52aigoeWIYBLhaTohz69ezIC/kYSye03eYerLq9e8GHeHmitRV8hMF8E6WNO2iWN95LJqojOLvIzfeusweVAwyixzokvuOe5osrzvS/Q3Wi/CA7fqwpIza9mzkJ7HzSkyV65IrutPcfeNtLwzT9UJPSl6XeFgDhLJDMd8QhkWvleTSQiSd4vgPrYy7hZRFp/qkTdoUU0u94f+Ul5ab+o1w5Da0jjXX0= ceph-3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:23:40.571 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZMp7JKokJelbqW4Vgy8MiDhicnmIkxl2sVzIsSkUNwlmOUDo8quH/haUHYaou75IXaBcwTZCUdPBjPROyYS4c09r/whjxEpeE3xfKS1Ua+lhUdUmUV+gJj41Jmx1xXHKm4zS9ZXRkfbQGAyQVG/sQn9v3vDlOMqxMXgsrjSnG5u19YZjx3QcQM9ixY6Fa0B2X+G+jONQWipBE3zQHr1pdNsCAPPNcnQElJvzSWOSLLag0NSUb2vNmcTdGKTRxShooiZo9TBK5m2c96IvBclw1Q9iXtU44ot+D52aigoeWIYBLhaTohz69ezIC/kYSye03eYerLq9e8GHeHmitRV8hMF8E6WNO2iWN95LJqojOLvIzfeusweVAwyixzokvuOe5osrzvS/Q3Wi/CA7fqwpIza9mzkJ7HzSkyV65IrutPcfeNtLwzT9UJPSl6XeFgDhLJDMd8QhkWvleTSQiSd4vgPrYy7hZRFp/qkTdoUU0u94f+Ul5ab+o1w5Da0jjXX0= ceph-3af3dba8-251a-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-18T03:23:40.619 INFO:teuthology.orchestra.run.smithi064.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZMp7JKokJelbqW4Vgy8MiDhicnmIkxl2sVzIsSkUNwlmOUDo8quH/haUHYaou75IXaBcwTZCUdPBjPROyYS4c09r/whjxEpeE3xfKS1Ua+lhUdUmUV+gJj41Jmx1xXHKm4zS9ZXRkfbQGAyQVG/sQn9v3vDlOMqxMXgsrjSnG5u19YZjx3QcQM9ixY6Fa0B2X+G+jONQWipBE3zQHr1pdNsCAPPNcnQElJvzSWOSLLag0NSUb2vNmcTdGKTRxShooiZo9TBK5m2c96IvBclw1Q9iXtU44ot+D52aigoeWIYBLhaTohz69ezIC/kYSye03eYerLq9e8GHeHmitRV8hMF8E6WNO2iWN95LJqojOLvIzfeusweVAwyixzokvuOe5osrzvS/Q3Wi/CA7fqwpIza9mzkJ7HzSkyV65IrutPcfeNtLwzT9UJPSl6XeFgDhLJDMd8QhkWvleTSQiSd4vgPrYy7hZRFp/qkTdoUU0u94f+Ul5ab+o1w5Da0jjXX0= ceph-3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:23:40.632 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZMp7JKokJelbqW4Vgy8MiDhicnmIkxl2sVzIsSkUNwlmOUDo8quH/haUHYaou75IXaBcwTZCUdPBjPROyYS4c09r/whjxEpeE3xfKS1Ua+lhUdUmUV+gJj41Jmx1xXHKm4zS9ZXRkfbQGAyQVG/sQn9v3vDlOMqxMXgsrjSnG5u19YZjx3QcQM9ixY6Fa0B2X+G+jONQWipBE3zQHr1pdNsCAPPNcnQElJvzSWOSLLag0NSUb2vNmcTdGKTRxShooiZo9TBK5m2c96IvBclw1Q9iXtU44ot+D52aigoeWIYBLhaTohz69ezIC/kYSye03eYerLq9e8GHeHmitRV8hMF8E6WNO2iWN95LJqojOLvIzfeusweVAwyixzokvuOe5osrzvS/Q3Wi/CA7fqwpIza9mzkJ7HzSkyV65IrutPcfeNtLwzT9UJPSl6XeFgDhLJDMd8QhkWvleTSQiSd4vgPrYy7hZRFp/qkTdoUU0u94f+Ul5ab+o1w5Da0jjXX0= ceph-3af3dba8-251a-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-18T03:23:40.676 INFO:teuthology.orchestra.run.smithi134.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZMp7JKokJelbqW4Vgy8MiDhicnmIkxl2sVzIsSkUNwlmOUDo8quH/haUHYaou75IXaBcwTZCUdPBjPROyYS4c09r/whjxEpeE3xfKS1Ua+lhUdUmUV+gJj41Jmx1xXHKm4zS9ZXRkfbQGAyQVG/sQn9v3vDlOMqxMXgsrjSnG5u19YZjx3QcQM9ixY6Fa0B2X+G+jONQWipBE3zQHr1pdNsCAPPNcnQElJvzSWOSLLag0NSUb2vNmcTdGKTRxShooiZo9TBK5m2c96IvBclw1Q9iXtU44ot+D52aigoeWIYBLhaTohz69ezIC/kYSye03eYerLq9e8GHeHmitRV8hMF8E6WNO2iWN95LJqojOLvIzfeusweVAwyixzokvuOe5osrzvS/Q3Wi/CA7fqwpIza9mzkJ7HzSkyV65IrutPcfeNtLwzT9UJPSl6XeFgDhLJDMd8QhkWvleTSQiSd4vgPrYy7hZRFp/qkTdoUU0u94f+Ul5ab+o1w5Da0jjXX0= ceph-3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:23:40.689 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-07-18T03:23:41.188 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:40 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3043601927' entity='client.admin' 2023-07-18T03:23:41.500 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:23:42.779 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:42.779 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:42.780 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi040", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:23:42.780 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:42.780 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:23:42.780 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:42.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.804+0000 7fd0d35a2700 1 -- 172.21.15.40:0/3928812527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc1000b0 msgr2=0x7fd0cc102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:42.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.804+0000 7fd0d35a2700 1 --2- 172.21.15.40:0/3928812527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc1000b0 0x7fd0cc102490 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400b0a0 tx=0x7fd0c400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:23:42.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.804+0000 7fd0d35a2700 1 -- 172.21.15.40:0/3928812527 shutdown_connections 2023-07-18T03:23:42.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.804+0000 7fd0d35a2700 1 --2- 172.21.15.40:0/3928812527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc1000b0 0x7fd0cc102490 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:42.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.804+0000 7fd0d35a2700 1 -- 172.21.15.40:0/3928812527 >> 172.21.15.40:0/3928812527 conn(0x7fd0cc0f9a70 msgr2=0x7fd0cc0fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:42.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.805+0000 7fd0d35a2700 1 -- 172.21.15.40:0/3928812527 shutdown_connections 2023-07-18T03:23:42.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.805+0000 7fd0d35a2700 1 -- 172.21.15.40:0/3928812527 wait complete. 2023-07-18T03:23:42.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.805+0000 7fd0d35a2700 1 Processor -- start 2023-07-18T03:23:42.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.805+0000 7fd0d35a2700 1 -- start start 2023-07-18T03:23:42.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.805+0000 7fd0d35a2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc1000b0 0x7fd0cc0ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:42.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.805+0000 7fd0d35a2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0fff50 con 0x7fd0cc1000b0 2023-07-18T03:23:42.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.805+0000 7fd0d133e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc1000b0 0x7fd0cc0ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:42.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.805+0000 7fd0d133e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc1000b0 0x7fd0cc0ffa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55108/0 (socket says 172.21.15.40:55108) 2023-07-18T03:23:42.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.805+0000 7fd0d133e700 1 -- 172.21.15.40:0/4138000525 learned_addr learned my addr 172.21.15.40:0/4138000525 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:42.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.806+0000 7fd0d133e700 1 -- 172.21.15.40:0/4138000525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c4009d00 con 0x7fd0cc1000b0 2023-07-18T03:23:42.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.806+0000 7fd0d133e700 1 --2- 172.21.15.40:0/4138000525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc1000b0 0x7fd0cc0ffa10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400b830 tx=0x7fd0c400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:42.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.806+0000 7fd0ca7fc700 1 -- 172.21.15.40:0/4138000525 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0c4021070 con 0x7fd0cc1000b0 2023-07-18T03:23:42.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.806+0000 7fd0ca7fc700 1 -- 172.21.15.40:0/4138000525 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0c4003e00 con 0x7fd0cc1000b0 2023-07-18T03:23:42.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.807+0000 7fd0d35a2700 1 -- 172.21.15.40:0/4138000525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0cc0fe060 con 0x7fd0cc1000b0 2023-07-18T03:23:42.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.807+0000 7fd0ca7fc700 1 -- 172.21.15.40:0/4138000525 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0c400e040 con 0x7fd0cc1000b0 2023-07-18T03:23:42.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.807+0000 7fd0d35a2700 1 -- 172.21.15.40:0/4138000525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc0fe500 con 0x7fd0cc1000b0 2023-07-18T03:23:42.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.807+0000 7fd0ca7fc700 1 -- 172.21.15.40:0/4138000525 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fd0c4019070 con 0x7fd0cc1000b0 2023-07-18T03:23:42.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.807+0000 7fd0d35a2700 1 -- 172.21.15.40:0/4138000525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0cc061b40 con 0x7fd0cc1000b0 2023-07-18T03:23:42.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.808+0000 7fd0ca7fc700 1 --2- 172.21.15.40:0/4138000525 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd0b4038850 0x7fd0b403ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:42.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.808+0000 7fd0ca7fc700 1 -- 172.21.15.40:0/4138000525 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd0c404c530 con 0x7fd0cc1000b0 2023-07-18T03:23:42.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.808+0000 7fd0d0b3d700 1 --2- 172.21.15.40:0/4138000525 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd0b4038850 0x7fd0b403ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:42.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.808+0000 7fd0d0b3d700 1 --2- 172.21.15.40:0/4138000525 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd0b4038850 0x7fd0b403ad00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0006fd0 tx=0x7fd0c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:42.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:42.812+0000 7fd0ca7fc700 1 -- 172.21.15.40:0/4138000525 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd0c401a950 con 0x7fd0cc1000b0 2023-07-18T03:23:43.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:43.011+0000 7fd0d35a2700 1 -- 172.21.15.40:0/4138000525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fd0cc18e050 con 0x7fd0cc1000b0 2023-07-18T03:23:43.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:43.014+0000 7fd0ca7fc700 1 -- 172.21.15.40:0/4138000525 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd0c4017070 con 0x7fd0cc1000b0 2023-07-18T03:23:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:43.016+0000 7fd0d35a2700 1 -- 172.21.15.40:0/4138000525 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd0b4038850 msgr2=0x7fd0b403ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:43.016+0000 7fd0d35a2700 1 --2- 172.21.15.40:0/4138000525 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd0b4038850 0x7fd0b403ad00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0006fd0 tx=0x7fd0c0006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:43.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:43.016+0000 7fd0d35a2700 1 -- 172.21.15.40:0/4138000525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc1000b0 msgr2=0x7fd0cc0ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:43.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:43.016+0000 7fd0d35a2700 1 --2- 172.21.15.40:0/4138000525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc1000b0 0x7fd0cc0ffa10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400b830 tx=0x7fd0c400bd40 comp rx=0 tx=0).stop 2023-07-18T03:23:43.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:43.016+0000 7fd0d35a2700 1 -- 172.21.15.40:0/4138000525 shutdown_connections 2023-07-18T03:23:43.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:43.016+0000 7fd0d35a2700 1 --2- 172.21.15.40:0/4138000525 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd0b4038850 0x7fd0b403ad00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:43.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:43.016+0000 7fd0d35a2700 1 --2- 172.21.15.40:0/4138000525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0cc1000b0 0x7fd0cc0ffa10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:43.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:43.016+0000 7fd0d35a2700 1 -- 172.21.15.40:0/4138000525 >> 172.21.15.40:0/4138000525 conn(0x7fd0cc0f9a70 msgr2=0x7fd0cc0fa760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:43.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:43.017+0000 7fd0d35a2700 1 -- 172.21.15.40:0/4138000525 shutdown_connections 2023-07-18T03:23:43.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:43.017+0000 7fd0d35a2700 1 -- 172.21.15.40:0/4138000525 wait complete. 2023-07-18T03:23:43.356 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-07-18T03:23:43.356 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-07-18T03:23:43.913 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:23:44.281 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:44 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/4138000525' entity='client.admin' 2023-07-18T03:23:44.281 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:23:44.281 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:45.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.070+0000 7f4017d8b700 1 -- 172.21.15.40:0/385290362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40100fe4e0 msgr2=0x7f40100fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:45.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.070+0000 7f4017d8b700 1 --2- 172.21.15.40:0/385290362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40100fe4e0 0x7f40100fe8f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f400c009910 tx=0x7f400c009c20 comp rx=0 tx=0).stop 2023-07-18T03:23:45.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.070+0000 7f4017d8b700 1 -- 172.21.15.40:0/385290362 shutdown_connections 2023-07-18T03:23:45.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.070+0000 7f4017d8b700 1 --2- 172.21.15.40:0/385290362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40100fe4e0 0x7f40100fe8f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:45.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.070+0000 7f4017d8b700 1 -- 172.21.15.40:0/385290362 >> 172.21.15.40:0/385290362 conn(0x7f40100f9a70 msgr2=0x7f40100fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:45.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.070+0000 7f4017d8b700 1 -- 172.21.15.40:0/385290362 shutdown_connections 2023-07-18T03:23:45.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.070+0000 7f4017d8b700 1 -- 172.21.15.40:0/385290362 wait complete. 2023-07-18T03:23:45.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.070+0000 7f4017d8b700 1 Processor -- start 2023-07-18T03:23:45.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.070+0000 7f4017d8b700 1 -- start start 2023-07-18T03:23:45.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.071+0000 7f4017d8b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40100fe4e0 0x7f40100fba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:45.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.071+0000 7f4017d8b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40100fbf60 con 0x7f40100fe4e0 2023-07-18T03:23:45.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.071+0000 7f4015b27700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40100fe4e0 0x7f40100fba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:45.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.071+0000 7f4015b27700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40100fe4e0 0x7f40100fba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55114/0 (socket says 172.21.15.40:55114) 2023-07-18T03:23:45.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.071+0000 7f4015b27700 1 -- 172.21.15.40:0/681188448 learned_addr learned my addr 172.21.15.40:0/681188448 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:45.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.071+0000 7f4015b27700 1 -- 172.21.15.40:0/681188448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c0095f0 con 0x7f40100fe4e0 2023-07-18T03:23:45.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.071+0000 7f4015b27700 1 --2- 172.21.15.40:0/681188448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40100fe4e0 0x7f40100fba20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f400c006010 tx=0x7f400c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.071+0000 7f4006ffd700 1 -- 172.21.15.40:0/681188448 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f400c01d070 con 0x7f40100fe4e0 2023-07-18T03:23:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.071+0000 7f4006ffd700 1 -- 172.21.15.40:0/681188448 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f400c0044e0 con 0x7f40100fe4e0 2023-07-18T03:23:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.071+0000 7f4017d8b700 1 -- 172.21.15.40:0/681188448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40100fa070 con 0x7f40100fe4e0 2023-07-18T03:23:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.071+0000 7f4006ffd700 1 -- 172.21.15.40:0/681188448 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f400c0039b0 con 0x7f40100fe4e0 2023-07-18T03:23:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.071+0000 7f4017d8b700 1 -- 172.21.15.40:0/681188448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40100fa4b0 con 0x7f40100fe4e0 2023-07-18T03:23:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.072+0000 7f4006ffd700 1 -- 172.21.15.40:0/681188448 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f400c00baf0 con 0x7f40100fe4e0 2023-07-18T03:23:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.072+0000 7f4017d8b700 1 -- 172.21.15.40:0/681188448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4010061b40 con 0x7f40100fe4e0 2023-07-18T03:23:45.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.072+0000 7f4006ffd700 1 --2- 172.21.15.40:0/681188448 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3ffc0387b0 0x7f3ffc03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.072+0000 7f4006ffd700 1 -- 172.21.15.40:0/681188448 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f400c04c810 con 0x7f40100fe4e0 2023-07-18T03:23:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.073+0000 7f4015326700 1 --2- 172.21.15.40:0/681188448 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3ffc0387b0 0x7f3ffc03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:45.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.073+0000 7f4015326700 1 --2- 172.21.15.40:0/681188448 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3ffc0387b0 0x7f3ffc03ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4000006fd0 tx=0x7f4000006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:45.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.078+0000 7f4006ffd700 1 -- 172.21.15.40:0/681188448 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f400c02a930 con 0x7f40100fe4e0 2023-07-18T03:23:45.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.279+0000 7f4017d8b700 1 -- 172.21.15.40:0/681188448 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f40100fa750 con 0x7f3ffc0387b0 2023-07-18T03:23:45.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.282+0000 7f4006ffd700 1 -- 172.21.15.40:0/681188448 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f40100fa750 con 0x7f3ffc0387b0 2023-07-18T03:23:45.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.284+0000 7f4017d8b700 1 -- 172.21.15.40:0/681188448 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3ffc0387b0 msgr2=0x7f3ffc03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:45.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.284+0000 7f4017d8b700 1 --2- 172.21.15.40:0/681188448 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3ffc0387b0 0x7f3ffc03ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4000006fd0 tx=0x7f4000006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:45.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.284+0000 7f4017d8b700 1 -- 172.21.15.40:0/681188448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40100fe4e0 msgr2=0x7f40100fba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:45.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.284+0000 7f4017d8b700 1 --2- 172.21.15.40:0/681188448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40100fe4e0 0x7f40100fba20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f400c006010 tx=0x7f400c0057c0 comp rx=0 tx=0).stop 2023-07-18T03:23:45.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.284+0000 7f4017d8b700 1 -- 172.21.15.40:0/681188448 shutdown_connections 2023-07-18T03:23:45.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.284+0000 7f4017d8b700 1 --2- 172.21.15.40:0/681188448 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3ffc0387b0 0x7f3ffc03ac60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:45.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.284+0000 7f4017d8b700 1 --2- 172.21.15.40:0/681188448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40100fe4e0 0x7f40100fba20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:45.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.284+0000 7f4017d8b700 1 -- 172.21.15.40:0/681188448 >> 172.21.15.40:0/681188448 conn(0x7f40100f9a70 msgr2=0x7f4010067fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:45.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.284+0000 7f4017d8b700 1 -- 172.21.15.40:0/681188448 shutdown_connections 2023-07-18T03:23:45.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:45.284+0000 7f4017d8b700 1 -- 172.21.15.40:0/681188448 wait complete. 2023-07-18T03:23:45.666 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi064 2023-07-18T03:23:45.667 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-07-18T03:23:45.668 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/etc/ceph/ceph.conf 2023-07-18T03:23:45.689 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-07-18T03:23:45.689 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:23:45.749 INFO:tasks.cephadm:Adding host smithi064 to orchestrator... 2023-07-18T03:23:45.750 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch host add smithi064 2023-07-18T03:23:46.317 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:23:46.560 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:46 smithi040 ceph-mon[96439]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:23:46.560 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:46 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:46.560 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:46 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:23:46.560 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:46 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:23:46.560 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:46 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:23:46.561 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:46 smithi040 ceph-mon[96439]: Updating smithi040:/etc/ceph/ceph.conf 2023-07-18T03:23:46.561 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:46 smithi040 ceph-mon[96439]: Updating smithi040:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:23:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.497+0000 7f2d600c3700 1 -- 172.21.15.40:0/3093363163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d581044f0 msgr2=0x7f2d581048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.497+0000 7f2d600c3700 1 --2- 172.21.15.40:0/3093363163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d581044f0 0x7f2d581048c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009910 tx=0x7f2d54009c20 comp rx=0 tx=0).stop 2023-07-18T03:23:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.497+0000 7f2d600c3700 1 -- 172.21.15.40:0/3093363163 shutdown_connections 2023-07-18T03:23:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.497+0000 7f2d600c3700 1 --2- 172.21.15.40:0/3093363163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d581044f0 0x7f2d581048c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.497+0000 7f2d600c3700 1 -- 172.21.15.40:0/3093363163 >> 172.21.15.40:0/3093363163 conn(0x7f2d580742b0 msgr2=0x7f2d580746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.497+0000 7f2d600c3700 1 -- 172.21.15.40:0/3093363163 shutdown_connections 2023-07-18T03:23:47.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.498+0000 7f2d600c3700 1 -- 172.21.15.40:0/3093363163 wait complete. 2023-07-18T03:23:47.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.498+0000 7f2d600c3700 1 Processor -- start 2023-07-18T03:23:47.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.498+0000 7f2d600c3700 1 -- start start 2023-07-18T03:23:47.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.498+0000 7f2d600c3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d581044f0 0x7f2d5819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.498+0000 7f2d600c3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5819ae80 con 0x7f2d581044f0 2023-07-18T03:23:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.498+0000 7f2d5de5f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d581044f0 0x7f2d5819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.498+0000 7f2d5de5f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d581044f0 0x7f2d5819a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55120/0 (socket says 172.21.15.40:55120) 2023-07-18T03:23:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.498+0000 7f2d5de5f700 1 -- 172.21.15.40:0/2958369662 learned_addr learned my addr 172.21.15.40:0/2958369662 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.499+0000 7f2d5de5f700 1 -- 172.21.15.40:0/2958369662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d540095f0 con 0x7f2d581044f0 2023-07-18T03:23:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.499+0000 7f2d5de5f700 1 --2- 172.21.15.40:0/2958369662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d581044f0 0x7f2d5819a940 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2d54004f40 tx=0x7f2d54005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.499+0000 7f2d4affd700 1 -- 172.21.15.40:0/2958369662 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d5401d070 con 0x7f2d581044f0 2023-07-18T03:23:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.499+0000 7f2d4affd700 1 -- 172.21.15.40:0/2958369662 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d54004370 con 0x7f2d581044f0 2023-07-18T03:23:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.499+0000 7f2d600c3700 1 -- 172.21.15.40:0/2958369662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5819b080 con 0x7f2d581044f0 2023-07-18T03:23:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.499+0000 7f2d4affd700 1 -- 172.21.15.40:0/2958369662 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d540039b0 con 0x7f2d581044f0 2023-07-18T03:23:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.499+0000 7f2d600c3700 1 -- 172.21.15.40:0/2958369662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5819b4a0 con 0x7f2d581044f0 2023-07-18T03:23:47.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.500+0000 7f2d4affd700 1 -- 172.21.15.40:0/2958369662 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f2d54003c20 con 0x7f2d581044f0 2023-07-18T03:23:47.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.500+0000 7f2d600c3700 1 -- 172.21.15.40:0/2958369662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d58061b40 con 0x7f2d581044f0 2023-07-18T03:23:47.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.500+0000 7f2d4affd700 1 --2- 172.21.15.40:0/2958369662 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2d44038800 0x7f2d4403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.500+0000 7f2d4affd700 1 -- 172.21.15.40:0/2958369662 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2d5404c970 con 0x7f2d581044f0 2023-07-18T03:23:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.500+0000 7f2d5d65e700 1 --2- 172.21.15.40:0/2958369662 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2d44038800 0x7f2d4403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.501+0000 7f2d5d65e700 1 --2- 172.21.15.40:0/2958369662 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2d44038800 0x7f2d4403acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c006fd0 tx=0x7f2d4c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.505+0000 7f2d4affd700 1 -- 172.21.15.40:0/2958369662 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2d54029360 con 0x7f2d581044f0 2023-07-18T03:23:47.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:47 smithi040 ceph-mon[96439]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:23:47.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:47 smithi040 ceph-mon[96439]: Updating smithi040:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T03:23:47.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:47 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:47.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:47 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:47.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:47 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:47.707+0000 7f2d600c3700 1 -- 172.21.15.40:0/2958369662 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi064", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d580fd3e0 con 0x7f2d44038800 2023-07-18T03:23:48.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:48 smithi040 ceph-mon[96439]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi064", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:23:49.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:49 smithi040 ceph-mon[96439]: Deploying cephadm binary to smithi064 2023-07-18T03:23:50.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:50.478+0000 7f2d4affd700 1 -- 172.21.15.40:0/2958369662 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f2d580fd3e0 con 0x7f2d44038800 2023-07-18T03:23:50.474 INFO:teuthology.orchestra.run.smithi040.stdout:Added host 'smithi064' with addr '172.21.15.64' 2023-07-18T03:23:50.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:50.480+0000 7f2d600c3700 1 -- 172.21.15.40:0/2958369662 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2d44038800 msgr2=0x7f2d4403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:50.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:50.480+0000 7f2d600c3700 1 --2- 172.21.15.40:0/2958369662 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2d44038800 0x7f2d4403acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c006fd0 tx=0x7f2d4c006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:50.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:50.480+0000 7f2d600c3700 1 -- 172.21.15.40:0/2958369662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d581044f0 msgr2=0x7f2d5819a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:50.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:50.480+0000 7f2d600c3700 1 --2- 172.21.15.40:0/2958369662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d581044f0 0x7f2d5819a940 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2d54004f40 tx=0x7f2d54005020 comp rx=0 tx=0).stop 2023-07-18T03:23:50.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:50.480+0000 7f2d600c3700 1 -- 172.21.15.40:0/2958369662 shutdown_connections 2023-07-18T03:23:50.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:50.480+0000 7f2d600c3700 1 --2- 172.21.15.40:0/2958369662 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2d44038800 0x7f2d4403acb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:50.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:50.480+0000 7f2d600c3700 1 --2- 172.21.15.40:0/2958369662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d581044f0 0x7f2d5819a940 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:50.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:50.480+0000 7f2d600c3700 1 -- 172.21.15.40:0/2958369662 >> 172.21.15.40:0/2958369662 conn(0x7f2d580742b0 msgr2=0x7f2d58067fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:50.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:50.480+0000 7f2d600c3700 1 -- 172.21.15.40:0/2958369662 shutdown_connections 2023-07-18T03:23:50.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:50.480+0000 7f2d600c3700 1 -- 172.21.15.40:0/2958369662 wait complete. 2023-07-18T03:23:50.785 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch host ls --format=json 2023-07-18T03:23:51.344 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:23:51.811 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:51 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:51.811 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:51 smithi040 ceph-mon[96439]: Added host smithi064 2023-07-18T03:23:51.811 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:51 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:23:51.811 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:51 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:52.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.421+0000 7ff8683dd700 1 -- 172.21.15.40:0/1889426534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8601055d0 msgr2=0x7ff8601059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.421+0000 7ff8683dd700 1 --2- 172.21.15.40:0/1889426534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8601055d0 0x7ff8601059a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009910 tx=0x7ff85c009c20 comp rx=0 tx=0).stop 2023-07-18T03:23:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.421+0000 7ff8683dd700 1 -- 172.21.15.40:0/1889426534 shutdown_connections 2023-07-18T03:23:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.421+0000 7ff8683dd700 1 --2- 172.21.15.40:0/1889426534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8601055d0 0x7ff8601059a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.421+0000 7ff8683dd700 1 -- 172.21.15.40:0/1889426534 >> 172.21.15.40:0/1889426534 conn(0x7ff8600742b0 msgr2=0x7ff8600746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.421+0000 7ff8683dd700 1 -- 172.21.15.40:0/1889426534 shutdown_connections 2023-07-18T03:23:52.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.421+0000 7ff8683dd700 1 -- 172.21.15.40:0/1889426534 wait complete. 2023-07-18T03:23:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.422+0000 7ff8683dd700 1 Processor -- start 2023-07-18T03:23:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.422+0000 7ff8683dd700 1 -- start start 2023-07-18T03:23:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.422+0000 7ff8683dd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8601055d0 0x7ff86019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.422+0000 7ff8683dd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86019ad50 con 0x7ff8601055d0 2023-07-18T03:23:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.422+0000 7ff866179700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8601055d0 0x7ff86019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.422+0000 7ff866179700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8601055d0 0x7ff86019a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55146/0 (socket says 172.21.15.40:55146) 2023-07-18T03:23:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.422+0000 7ff866179700 1 -- 172.21.15.40:0/3597585641 learned_addr learned my addr 172.21.15.40:0/3597585641 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:52.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.423+0000 7ff866179700 1 -- 172.21.15.40:0/3597585641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85c0095f0 con 0x7ff8601055d0 2023-07-18T03:23:52.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.423+0000 7ff866179700 1 --2- 172.21.15.40:0/3597585641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8601055d0 0x7ff86019a810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff85c004f40 tx=0x7ff85c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:52.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.423+0000 7ff8577fe700 1 -- 172.21.15.40:0/3597585641 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff85c01d070 con 0x7ff8601055d0 2023-07-18T03:23:52.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.423+0000 7ff8683dd700 1 -- 172.21.15.40:0/3597585641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff86019af50 con 0x7ff8601055d0 2023-07-18T03:23:52.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.423+0000 7ff8577fe700 1 -- 172.21.15.40:0/3597585641 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff85c004370 con 0x7ff8601055d0 2023-07-18T03:23:52.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.423+0000 7ff8577fe700 1 -- 172.21.15.40:0/3597585641 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff85c0039b0 con 0x7ff8601055d0 2023-07-18T03:23:52.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.423+0000 7ff8683dd700 1 -- 172.21.15.40:0/3597585641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff860194890 con 0x7ff8601055d0 2023-07-18T03:23:52.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.424+0000 7ff8683dd700 1 -- 172.21.15.40:0/3597585641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff860061b40 con 0x7ff8601055d0 2023-07-18T03:23:52.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.425+0000 7ff8577fe700 1 -- 172.21.15.40:0/3597585641 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7ff85c003c20 con 0x7ff8601055d0 2023-07-18T03:23:52.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.425+0000 7ff8577fe700 1 --2- 172.21.15.40:0/3597585641 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff84c038800 0x7ff84c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:52.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.425+0000 7ff8577fe700 1 -- 172.21.15.40:0/3597585641 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff85c04c8f0 con 0x7ff8601055d0 2023-07-18T03:23:52.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.425+0000 7ff865978700 1 --2- 172.21.15.40:0/3597585641 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff84c038800 0x7ff84c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:52.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.425+0000 7ff865978700 1 --2- 172.21.15.40:0/3597585641 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff84c038800 0x7ff84c03acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff850006fd0 tx=0x7ff850006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:52.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.434+0000 7ff8577fe700 1 -- 172.21.15.40:0/3597585641 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff85c02a980 con 0x7ff8601055d0 2023-07-18T03:23:52.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.637+0000 7ff8683dd700 1 -- 172.21.15.40:0/3597585641 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8601952c0 con 0x7ff84c038800 2023-07-18T03:23:52.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.638+0000 7ff8577fe700 1 -- 172.21.15.40:0/3597585641 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7ff8601952c0 con 0x7ff84c038800 2023-07-18T03:23:52.634 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:23:52.634 INFO:teuthology.orchestra.run.smithi040.stdout:[{"addr": "172.21.15.40", "hostname": "smithi040", "labels": [], "status": ""}, {"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}] 2023-07-18T03:23:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.640+0000 7ff8683dd700 1 -- 172.21.15.40:0/3597585641 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff84c038800 msgr2=0x7ff84c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.640+0000 7ff8683dd700 1 --2- 172.21.15.40:0/3597585641 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff84c038800 0x7ff84c03acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff850006fd0 tx=0x7ff850006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.640+0000 7ff8683dd700 1 -- 172.21.15.40:0/3597585641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8601055d0 msgr2=0x7ff86019a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.640+0000 7ff8683dd700 1 --2- 172.21.15.40:0/3597585641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8601055d0 0x7ff86019a810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff85c004f40 tx=0x7ff85c005020 comp rx=0 tx=0).stop 2023-07-18T03:23:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.640+0000 7ff8683dd700 1 -- 172.21.15.40:0/3597585641 shutdown_connections 2023-07-18T03:23:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.640+0000 7ff8683dd700 1 --2- 172.21.15.40:0/3597585641 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff84c038800 0x7ff84c03acb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.640+0000 7ff8683dd700 1 --2- 172.21.15.40:0/3597585641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8601055d0 0x7ff86019a810 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.640+0000 7ff8683dd700 1 -- 172.21.15.40:0/3597585641 >> 172.21.15.40:0/3597585641 conn(0x7ff8600742b0 msgr2=0x7ff860106bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.640+0000 7ff8683dd700 1 -- 172.21.15.40:0/3597585641 shutdown_connections 2023-07-18T03:23:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:52.640+0000 7ff8683dd700 1 -- 172.21.15.40:0/3597585641 wait complete. 2023-07-18T03:23:52.926 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:52 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:52.926 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:52 smithi040 ceph-mon[96439]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:23:52.990 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi134 2023-07-18T03:23:52.991 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-18T03:23:52.991 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.conf 2023-07-18T03:23:53.011 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-18T03:23:53.012 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:23:53.070 INFO:tasks.cephadm:Adding host smithi134 to orchestrator... 2023-07-18T03:23:53.071 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch host add smithi134 2023-07-18T03:23:53.626 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:23:53.857 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:53 smithi040 ceph-mon[96439]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:23:53.857 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:53 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.774+0000 7f6d0fb43700 1 -- 172.21.15.40:0/559561862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d081044f0 msgr2=0x7f6d081048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.774+0000 7f6d0fb43700 1 --2- 172.21.15.40:0/559561862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d081044f0 0x7f6d081048c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009910 tx=0x7f6d04009c20 comp rx=0 tx=0).stop 2023-07-18T03:23:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.774+0000 7f6d0fb43700 1 -- 172.21.15.40:0/559561862 shutdown_connections 2023-07-18T03:23:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.774+0000 7f6d0fb43700 1 --2- 172.21.15.40:0/559561862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d081044f0 0x7f6d081048c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.774+0000 7f6d0fb43700 1 -- 172.21.15.40:0/559561862 >> 172.21.15.40:0/559561862 conn(0x7f6d080fa010 msgr2=0x7f6d080fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.774+0000 7f6d0fb43700 1 -- 172.21.15.40:0/559561862 shutdown_connections 2023-07-18T03:23:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.774+0000 7f6d0fb43700 1 -- 172.21.15.40:0/559561862 wait complete. 2023-07-18T03:23:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.775+0000 7f6d0fb43700 1 Processor -- start 2023-07-18T03:23:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.775+0000 7f6d0fb43700 1 -- start start 2023-07-18T03:23:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.775+0000 7f6d0fb43700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d081044f0 0x7f6d080fa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.775+0000 7f6d0fb43700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080fc430 con 0x7f6d081044f0 2023-07-18T03:23:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.775+0000 7f6d0d8df700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d081044f0 0x7f6d080fa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.775+0000 7f6d0d8df700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d081044f0 0x7f6d080fa4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55152/0 (socket says 172.21.15.40:55152) 2023-07-18T03:23:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.775+0000 7f6d0d8df700 1 -- 172.21.15.40:0/577581618 learned_addr learned my addr 172.21.15.40:0/577581618 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.776+0000 7f6d0d8df700 1 -- 172.21.15.40:0/577581618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d040095f0 con 0x7f6d081044f0 2023-07-18T03:23:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.776+0000 7f6d0d8df700 1 --2- 172.21.15.40:0/577581618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d081044f0 0x7f6d080fa4b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d04004f40 tx=0x7f6d04005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.776+0000 7f6cfeffd700 1 -- 172.21.15.40:0/577581618 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d0401d070 con 0x7f6d081044f0 2023-07-18T03:23:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.776+0000 7f6cfeffd700 1 -- 172.21.15.40:0/577581618 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d04004370 con 0x7f6d081044f0 2023-07-18T03:23:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.776+0000 7f6d0fb43700 1 -- 172.21.15.40:0/577581618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d080fa9f0 con 0x7f6d081044f0 2023-07-18T03:23:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.776+0000 7f6cfeffd700 1 -- 172.21.15.40:0/577581618 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d040039b0 con 0x7f6d081044f0 2023-07-18T03:23:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.776+0000 7f6d0fb43700 1 -- 172.21.15.40:0/577581618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d080fae70 con 0x7f6d081044f0 2023-07-18T03:23:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.777+0000 7f6cfeffd700 1 -- 172.21.15.40:0/577581618 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f6d04003c20 con 0x7f6d081044f0 2023-07-18T03:23:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.777+0000 7f6d0fb43700 1 -- 172.21.15.40:0/577581618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d08061b40 con 0x7f6d081044f0 2023-07-18T03:23:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.777+0000 7f6cfeffd700 1 --2- 172.21.15.40:0/577581618 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6cf4038800 0x7f6cf403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:54.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.777+0000 7f6cfeffd700 1 -- 172.21.15.40:0/577581618 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6d0404c750 con 0x7f6d081044f0 2023-07-18T03:23:54.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.777+0000 7f6d0d0de700 1 --2- 172.21.15.40:0/577581618 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6cf4038800 0x7f6cf403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:54.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.777+0000 7f6d0d0de700 1 --2- 172.21.15.40:0/577581618 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6cf4038800 0x7f6cf403acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8006fd0 tx=0x7f6cf8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:54.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.782+0000 7f6cfeffd700 1 -- 172.21.15.40:0/577581618 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6d04029360 con 0x7f6d081044f0 2023-07-18T03:23:54.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:54 smithi040 ceph-mon[96439]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:23:54.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:54.984+0000 7f6d0fb43700 1 -- 172.21.15.40:0/577581618 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d080fb260 con 0x7f6cf4038800 2023-07-18T03:23:56.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:55 smithi040 ceph-mon[96439]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:23:56.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:55 smithi040 ceph-mon[96439]: Deploying cephadm binary to smithi134 2023-07-18T03:23:56.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:55 smithi040 ceph-mon[96439]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:23:57.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:56 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:57.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:56 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:57.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:56 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:57.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:56 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:57.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:56 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:23:57.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:56 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:23:57.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:56 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:23:57.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:56 smithi040 ceph-mon[96439]: Updating smithi064:/etc/ceph/ceph.conf 2023-07-18T03:23:57.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:57.776+0000 7f6cfeffd700 1 -- 172.21.15.40:0/577581618 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f6d080fb260 con 0x7f6cf4038800 2023-07-18T03:23:57.772 INFO:teuthology.orchestra.run.smithi040.stdout:Added host 'smithi134' with addr '172.21.15.134' 2023-07-18T03:23:57.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:57.778+0000 7f6d0fb43700 1 -- 172.21.15.40:0/577581618 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6cf4038800 msgr2=0x7f6cf403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:57.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:57.778+0000 7f6d0fb43700 1 --2- 172.21.15.40:0/577581618 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6cf4038800 0x7f6cf403acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8006fd0 tx=0x7f6cf8006e40 comp rx=0 tx=0).stop 2023-07-18T03:23:57.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:57.778+0000 7f6d0fb43700 1 -- 172.21.15.40:0/577581618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d081044f0 msgr2=0x7f6d080fa4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:57.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:57.778+0000 7f6d0fb43700 1 --2- 172.21.15.40:0/577581618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d081044f0 0x7f6d080fa4b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d04004f40 tx=0x7f6d04005020 comp rx=0 tx=0).stop 2023-07-18T03:23:57.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:57.778+0000 7f6d0fb43700 1 -- 172.21.15.40:0/577581618 shutdown_connections 2023-07-18T03:23:57.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:57.778+0000 7f6d0fb43700 1 --2- 172.21.15.40:0/577581618 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6cf4038800 0x7f6cf403acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:57.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:57.778+0000 7f6d0fb43700 1 --2- 172.21.15.40:0/577581618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d081044f0 0x7f6d080fa4b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:57.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:57.778+0000 7f6d0fb43700 1 -- 172.21.15.40:0/577581618 >> 172.21.15.40:0/577581618 conn(0x7f6d080fa010 msgr2=0x7f6d08067fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:57.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:57.778+0000 7f6d0fb43700 1 -- 172.21.15.40:0/577581618 shutdown_connections 2023-07-18T03:23:57.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:57.778+0000 7f6d0fb43700 1 -- 172.21.15.40:0/577581618 wait complete. 2023-07-18T03:23:58.068 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:57 smithi040 ceph-mon[96439]: Updating smithi064:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:23:58.068 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:57 smithi040 ceph-mon[96439]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:23:58.068 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:57 smithi040 ceph-mon[96439]: Updating smithi064:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T03:23:58.069 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:57 smithi040 ceph-mon[96439]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:23:58.069 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:57 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:58.084 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch host ls --format=json 2023-07-18T03:23:58.642 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:23:59.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:58 smithi040 ceph-mon[96439]: Added host smithi134 2023-07-18T03:23:59.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:59.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:59.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:59.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:23:59.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:23:59.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.824+0000 7f20a0576700 1 -- 172.21.15.40:0/3691826581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20981006a0 msgr2=0x7f2098100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:23:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.824+0000 7f20a0576700 1 --2- 172.21.15.40:0/3691826581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20981006a0 0x7f2098100a70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f208800b0a0 tx=0x7f208800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:23:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.824+0000 7f20a0576700 1 -- 172.21.15.40:0/3691826581 shutdown_connections 2023-07-18T03:23:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.824+0000 7f20a0576700 1 --2- 172.21.15.40:0/3691826581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20981006a0 0x7f2098100a70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:23:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.824+0000 7f20a0576700 1 -- 172.21.15.40:0/3691826581 >> 172.21.15.40:0/3691826581 conn(0x7f20980fa050 msgr2=0x7f20980fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:23:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.824+0000 7f20a0576700 1 -- 172.21.15.40:0/3691826581 shutdown_connections 2023-07-18T03:23:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.824+0000 7f20a0576700 1 -- 172.21.15.40:0/3691826581 wait complete. 2023-07-18T03:23:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.824+0000 7f20a0576700 1 Processor -- start 2023-07-18T03:23:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.825+0000 7f20a0576700 1 -- start start 2023-07-18T03:23:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.825+0000 7f20a0576700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20981006a0 0x7f20980fe4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.825+0000 7f20a0576700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2098100450 con 0x7f20981006a0 2023-07-18T03:23:59.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.825+0000 7f209e312700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20981006a0 0x7f20980fe4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:59.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.825+0000 7f209e312700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20981006a0 0x7f20980fe4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55178/0 (socket says 172.21.15.40:55178) 2023-07-18T03:23:59.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.825+0000 7f209e312700 1 -- 172.21.15.40:0/1960726432 learned_addr learned my addr 172.21.15.40:0/1960726432 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:23:59.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.825+0000 7f209e312700 1 -- 172.21.15.40:0/1960726432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2088009d00 con 0x7f20981006a0 2023-07-18T03:23:59.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.825+0000 7f209e312700 1 --2- 172.21.15.40:0/1960726432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20981006a0 0x7f20980fe4d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2088009fd0 tx=0x7f2088003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:59.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.825+0000 7f20977fe700 1 -- 172.21.15.40:0/1960726432 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f208800e040 con 0x7f20981006a0 2023-07-18T03:23:59.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.825+0000 7f20a0576700 1 -- 172.21.15.40:0/1960726432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20980fea10 con 0x7f20981006a0 2023-07-18T03:23:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.825+0000 7f20977fe700 1 -- 172.21.15.40:0/1960726432 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2088007ca0 con 0x7f20981006a0 2023-07-18T03:23:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.825+0000 7f20977fe700 1 -- 172.21.15.40:0/1960726432 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f208801d5f0 con 0x7f20981006a0 2023-07-18T03:23:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.826+0000 7f20a0576700 1 -- 172.21.15.40:0/1960726432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20980fee90 con 0x7f20981006a0 2023-07-18T03:23:59.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.826+0000 7f20a0576700 1 -- 172.21.15.40:0/1960726432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2098106d80 con 0x7f20981006a0 2023-07-18T03:23:59.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.827+0000 7f20977fe700 1 -- 172.21.15.40:0/1960726432 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f2088004040 con 0x7f20981006a0 2023-07-18T03:23:59.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.827+0000 7f20977fe700 1 --2- 172.21.15.40:0/1960726432 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f20800387a0 0x7f208003ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:23:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.827+0000 7f20977fe700 1 -- 172.21.15.40:0/1960726432 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f208804cbf0 con 0x7f20981006a0 2023-07-18T03:23:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.827+0000 7f209db11700 1 --2- 172.21.15.40:0/1960726432 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f20800387a0 0x7f208003ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:23:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.828+0000 7f209db11700 1 --2- 172.21.15.40:0/1960726432 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f20800387a0 0x7f208003ac50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f208c006fd0 tx=0x7f208c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:23:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:23:59.832+0000 7f20977fe700 1 -- 172.21.15.40:0/1960726432 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2088025360 con 0x7f20981006a0 2023-07-18T03:24:00.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:00.037+0000 7f20a0576700 1 -- 172.21.15.40:0/1960726432 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f20980ff140 con 0x7f20800387a0 2023-07-18T03:24:00.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:00.038+0000 7f20977fe700 1 -- 172.21.15.40:0/1960726432 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f20980ff140 con 0x7f20800387a0 2023-07-18T03:24:00.035 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:24:00.035 INFO:teuthology.orchestra.run.smithi040.stdout:[{"addr": "172.21.15.40", "hostname": "smithi040", "labels": [], "status": ""}, {"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}] 2023-07-18T03:24:00.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:00.040+0000 7f20a0576700 1 -- 172.21.15.40:0/1960726432 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f20800387a0 msgr2=0x7f208003ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:00.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:00.040+0000 7f20a0576700 1 --2- 172.21.15.40:0/1960726432 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f20800387a0 0x7f208003ac50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f208c006fd0 tx=0x7f208c006e40 comp rx=0 tx=0).stop 2023-07-18T03:24:00.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:00.040+0000 7f20a0576700 1 -- 172.21.15.40:0/1960726432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20981006a0 msgr2=0x7f20980fe4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:00.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:00.040+0000 7f20a0576700 1 --2- 172.21.15.40:0/1960726432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20981006a0 0x7f20980fe4d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2088009fd0 tx=0x7f2088003f30 comp rx=0 tx=0).stop 2023-07-18T03:24:00.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:00.040+0000 7f20a0576700 1 -- 172.21.15.40:0/1960726432 shutdown_connections 2023-07-18T03:24:00.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:00.040+0000 7f20a0576700 1 --2- 172.21.15.40:0/1960726432 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f20800387a0 0x7f208003ac50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:00.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:00.040+0000 7f20a0576700 1 --2- 172.21.15.40:0/1960726432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20981006a0 0x7f20980fe4d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:00.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:00.040+0000 7f20a0576700 1 -- 172.21.15.40:0/1960726432 >> 172.21.15.40:0/1960726432 conn(0x7f20980fa050 msgr2=0x7f2098103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:00.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:00.041+0000 7f20a0576700 1 -- 172.21.15.40:0/1960726432 shutdown_connections 2023-07-18T03:24:00.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:00.041+0000 7f20a0576700 1 -- 172.21.15.40:0/1960726432 wait complete. 2023-07-18T03:24:00.322 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:00.322 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:23:59 smithi040 ceph-mon[96439]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:00.390 INFO:tasks.cephadm:Setting crush tunables to default 2023-07-18T03:24:00.391 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd crush tunables default 2023-07-18T03:24:00.950 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:24:01.459 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:01 smithi040 ceph-mon[96439]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:24:01.460 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:01 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:02.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.116+0000 7f6d1c0a8700 1 -- 172.21.15.40:0/2260750912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d14105630 msgr2=0x7f6d14105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:02.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.116+0000 7f6d1c0a8700 1 --2- 172.21.15.40:0/2260750912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d14105630 0x7f6d14105a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009910 tx=0x7f6d04009c20 comp rx=0 tx=0).stop 2023-07-18T03:24:02.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.116+0000 7f6d1c0a8700 1 -- 172.21.15.40:0/2260750912 shutdown_connections 2023-07-18T03:24:02.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.116+0000 7f6d1c0a8700 1 --2- 172.21.15.40:0/2260750912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d14105630 0x7f6d14105a00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:02.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.116+0000 7f6d1c0a8700 1 -- 172.21.15.40:0/2260750912 >> 172.21.15.40:0/2260750912 conn(0x7f6d140fb0f0 msgr2=0x7f6d140fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:02.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.116+0000 7f6d1c0a8700 1 -- 172.21.15.40:0/2260750912 shutdown_connections 2023-07-18T03:24:02.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.116+0000 7f6d1c0a8700 1 -- 172.21.15.40:0/2260750912 wait complete. 2023-07-18T03:24:02.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.116+0000 7f6d1c0a8700 1 Processor -- start 2023-07-18T03:24:02.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.116+0000 7f6d1c0a8700 1 -- start start 2023-07-18T03:24:02.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.117+0000 7f6d1c0a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d14105630 0x7f6d141949f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:02.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.117+0000 7f6d1c0a8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d14194f30 con 0x7f6d14105630 2023-07-18T03:24:02.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.117+0000 7f6d19e44700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d14105630 0x7f6d141949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:02.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.117+0000 7f6d19e44700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d14105630 0x7f6d141949f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55184/0 (socket says 172.21.15.40:55184) 2023-07-18T03:24:02.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.117+0000 7f6d19e44700 1 -- 172.21.15.40:0/3668808298 learned_addr learned my addr 172.21.15.40:0/3668808298 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:24:02.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.117+0000 7f6d19e44700 1 -- 172.21.15.40:0/3668808298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d040095f0 con 0x7f6d14105630 2023-07-18T03:24:02.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.117+0000 7f6d19e44700 1 --2- 172.21.15.40:0/3668808298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d14105630 0x7f6d141949f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6d04006010 tx=0x7f6d04004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:02.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.117+0000 7f6d0affd700 1 -- 172.21.15.40:0/3668808298 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d0401d070 con 0x7f6d14105630 2023-07-18T03:24:02.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.117+0000 7f6d1c0a8700 1 -- 172.21.15.40:0/3668808298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d14195190 con 0x7f6d14105630 2023-07-18T03:24:02.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.117+0000 7f6d0affd700 1 -- 172.21.15.40:0/3668808298 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d0400bc10 con 0x7f6d14105630 2023-07-18T03:24:02.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.117+0000 7f6d0affd700 1 -- 172.21.15.40:0/3668808298 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d04003910 con 0x7f6d14105630 2023-07-18T03:24:02.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.117+0000 7f6d1c0a8700 1 -- 172.21.15.40:0/3668808298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d14195610 con 0x7f6d14105630 2023-07-18T03:24:02.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.118+0000 7f6d0affd700 1 -- 172.21.15.40:0/3668808298 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f6d04003bd0 con 0x7f6d14105630 2023-07-18T03:24:02.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.118+0000 7f6d0affd700 1 --2- 172.21.15.40:0/3668808298 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d000387b0 0x7f6d0003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:02.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.118+0000 7f6d0affd700 1 -- 172.21.15.40:0/3668808298 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6d0404c510 con 0x7f6d14105630 2023-07-18T03:24:02.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.118+0000 7f6d1c0a8700 1 -- 172.21.15.40:0/3668808298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d14061b40 con 0x7f6d14105630 2023-07-18T03:24:02.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.118+0000 7f6d19643700 1 --2- 172.21.15.40:0/3668808298 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d000387b0 0x7f6d0003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:02.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.119+0000 7f6d19643700 1 --2- 172.21.15.40:0/3668808298 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d000387b0 0x7f6d0003ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6d10006fd0 tx=0x7f6d10006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:02.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.124+0000 7f6d0affd700 1 -- 172.21.15.40:0/3668808298 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6d04026020 con 0x7f6d14105630 2023-07-18T03:24:02.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.326+0000 7f6d1c0a8700 1 -- 172.21.15.40:0/3668808298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f6d14104350 con 0x7f6d14105630 2023-07-18T03:24:02.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.515+0000 7f6d0affd700 1 -- 172.21.15.40:0/3668808298 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6d0402b440 con 0x7f6d14105630 2023-07-18T03:24:02.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.517+0000 7f6d1c0a8700 1 -- 172.21.15.40:0/3668808298 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d000387b0 msgr2=0x7f6d0003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:02.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.517+0000 7f6d1c0a8700 1 --2- 172.21.15.40:0/3668808298 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d000387b0 0x7f6d0003ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6d10006fd0 tx=0x7f6d10006e40 comp rx=0 tx=0).stop 2023-07-18T03:24:02.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.517+0000 7f6d1c0a8700 1 -- 172.21.15.40:0/3668808298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d14105630 msgr2=0x7f6d141949f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:02.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.517+0000 7f6d1c0a8700 1 --2- 172.21.15.40:0/3668808298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d14105630 0x7f6d141949f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6d04006010 tx=0x7f6d04004ff0 comp rx=0 tx=0).stop 2023-07-18T03:24:02.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.517+0000 7f6d1c0a8700 1 -- 172.21.15.40:0/3668808298 shutdown_connections 2023-07-18T03:24:02.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.517+0000 7f6d1c0a8700 1 --2- 172.21.15.40:0/3668808298 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d000387b0 0x7f6d0003ac60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:02.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.517+0000 7f6d1c0a8700 1 --2- 172.21.15.40:0/3668808298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d14105630 0x7f6d141949f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:02.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.517+0000 7f6d1c0a8700 1 -- 172.21.15.40:0/3668808298 >> 172.21.15.40:0/3668808298 conn(0x7f6d140fb0f0 msgr2=0x7f6d140fbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:02.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.517+0000 7f6d1c0a8700 1 -- 172.21.15.40:0/3668808298 shutdown_connections 2023-07-18T03:24:02.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:02.517+0000 7f6d1c0a8700 1 -- 172.21.15.40:0/3668808298 wait complete. 2023-07-18T03:24:02.515 INFO:teuthology.orchestra.run.smithi040.stderr:adjusted tunables profile to default 2023-07-18T03:24:02.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:02 smithi040 ceph-mon[96439]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:02.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:02 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3668808298' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-18T03:24:02.913 INFO:tasks.cephadm:Adding mon.a on smithi040 2023-07-18T03:24:02.913 INFO:tasks.cephadm:Adding mon.b on smithi064 2023-07-18T03:24:02.914 INFO:tasks.cephadm:Adding mon.c on smithi134 2023-07-18T03:24:02.914 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch apply mon '3;smithi040:172.21.15.40=a;smithi064:172.21.15.64=b;smithi134:172.21.15.134=c' 2023-07-18T03:24:03.508 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:24:03.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:03 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3668808298' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-18T03:24:03.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:03 smithi040 ceph-mon[96439]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T03:24:03.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:03.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:03.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:03.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:03.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:24:03.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:03.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:03.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:03 smithi040 ceph-mon[96439]: Updating smithi134:/etc/ceph/ceph.conf 2023-07-18T03:24:04.699 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.702+0000 7f6dbf0b4700 1 -- 172.21.15.134:0/728783749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6db80ff560 msgr2=0x7f6db80ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:04.699 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.702+0000 7f6dbf0b4700 1 --2- 172.21.15.134:0/728783749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6db80ff560 0x7f6db80ff930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009910 tx=0x7f6dac009c20 comp rx=0 tx=0).stop 2023-07-18T03:24:04.699 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.702+0000 7f6dbf0b4700 1 -- 172.21.15.134:0/728783749 shutdown_connections 2023-07-18T03:24:04.699 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.702+0000 7f6dbf0b4700 1 --2- 172.21.15.134:0/728783749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6db80ff560 0x7f6db80ff930 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:04.700 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.702+0000 7f6dbf0b4700 1 -- 172.21.15.134:0/728783749 >> 172.21.15.134:0/728783749 conn(0x7f6db8074f10 msgr2=0x7f6db8075310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:04.700 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.702+0000 7f6dbf0b4700 1 -- 172.21.15.134:0/728783749 shutdown_connections 2023-07-18T03:24:04.700 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.702+0000 7f6dbf0b4700 1 -- 172.21.15.134:0/728783749 wait complete. 2023-07-18T03:24:04.701 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.703+0000 7f6dbf0b4700 1 Processor -- start 2023-07-18T03:24:04.701 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.703+0000 7f6dbf0b4700 1 -- start start 2023-07-18T03:24:04.701 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.703+0000 7f6dbf0b4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6db80ff560 0x7f6db81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:04.701 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.703+0000 7f6dbf0b4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db8199d00 con 0x7f6db80ff560 2023-07-18T03:24:04.701 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.703+0000 7f6dbce50700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6db80ff560 0x7f6db81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:04.702 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.703+0000 7f6dbce50700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6db80ff560 0x7f6db81997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.134:50370/0 (socket says 172.21.15.134:50370) 2023-07-18T03:24:04.702 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.703+0000 7f6dbce50700 1 -- 172.21.15.134:0/3641321850 learned_addr learned my addr 172.21.15.134:0/3641321850 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-07-18T03:24:04.702 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.704+0000 7f6dbce50700 1 -- 172.21.15.134:0/3641321850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dac0095f0 con 0x7f6db80ff560 2023-07-18T03:24:04.702 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.704+0000 7f6dbce50700 1 --2- 172.21.15.134:0/3641321850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6db80ff560 0x7f6db81997c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0092e0 tx=0x7f6dac004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:04.702 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.704+0000 7f6db5ffb700 1 -- 172.21.15.134:0/3641321850 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6dac01d070 con 0x7f6db80ff560 2023-07-18T03:24:04.702 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.704+0000 7f6dbf0b4700 1 -- 172.21.15.134:0/3641321850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db8199f00 con 0x7f6db80ff560 2023-07-18T03:24:04.702 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.704+0000 7f6db5ffb700 1 -- 172.21.15.134:0/3641321850 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dac00bc10 con 0x7f6db80ff560 2023-07-18T03:24:04.703 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.704+0000 7f6db5ffb700 1 -- 172.21.15.134:0/3641321850 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6dac003930 con 0x7f6db80ff560 2023-07-18T03:24:04.703 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.704+0000 7f6dbf0b4700 1 -- 172.21.15.134:0/3641321850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db819a320 con 0x7f6db80ff560 2023-07-18T03:24:04.703 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.705+0000 7f6dbf0b4700 1 -- 172.21.15.134:0/3641321850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db8061b40 con 0x7f6db80ff560 2023-07-18T03:24:04.704 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.705+0000 7f6db5ffb700 1 -- 172.21.15.134:0/3641321850 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f6dac003bb0 con 0x7f6db80ff560 2023-07-18T03:24:04.704 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.705+0000 7f6db5ffb700 1 --2- 172.21.15.134:0/3641321850 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6da8038800 0x7f6da803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:04.704 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.705+0000 7f6db5ffb700 1 -- 172.21.15.134:0/3641321850 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6dac04c520 con 0x7f6db80ff560 2023-07-18T03:24:04.704 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.706+0000 7f6db7fff700 1 --2- 172.21.15.134:0/3641321850 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6da8038800 0x7f6da803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:04.704 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.706+0000 7f6db7fff700 1 --2- 172.21.15.134:0/3641321850 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6da8038800 0x7f6da803acb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6da4006fd0 tx=0x7f6da4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:04.706 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.710+0000 7f6db5ffb700 1 -- 172.21.15.134:0/3641321850 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6dac00ba70 con 0x7f6db80ff560 2023-07-18T03:24:04.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:04 smithi040 ceph-mon[96439]: Updating smithi134:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:24:04.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:04 smithi040 ceph-mon[96439]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:24:04.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:04 smithi040 ceph-mon[96439]: Updating smithi134:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T03:24:04.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:04 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:04.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:04 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:04.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:04 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:04.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:04 smithi040 ceph-mon[96439]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:04.901 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.904+0000 7f6dbf0b4700 1 -- 172.21.15.134:0/3641321850 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi040:172.21.15.40=a;smithi064:172.21.15.64=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f6db8103710 con 0x7f6da8038800 2023-07-18T03:24:04.910 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.913+0000 7f6db5ffb700 1 -- 172.21.15.134:0/3641321850 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6db8103710 con 0x7f6da8038800 2023-07-18T03:24:04.911 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mon update... 2023-07-18T03:24:04.912 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.915+0000 7f6dbf0b4700 1 -- 172.21.15.134:0/3641321850 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6da8038800 msgr2=0x7f6da803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:04.912 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.915+0000 7f6dbf0b4700 1 --2- 172.21.15.134:0/3641321850 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6da8038800 0x7f6da803acb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6da4006fd0 tx=0x7f6da4006e40 comp rx=0 tx=0).stop 2023-07-18T03:24:04.913 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.915+0000 7f6dbf0b4700 1 -- 172.21.15.134:0/3641321850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6db80ff560 msgr2=0x7f6db81997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:04.913 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.915+0000 7f6dbf0b4700 1 --2- 172.21.15.134:0/3641321850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6db80ff560 0x7f6db81997c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0092e0 tx=0x7f6dac004ff0 comp rx=0 tx=0).stop 2023-07-18T03:24:04.913 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.915+0000 7f6dbf0b4700 1 -- 172.21.15.134:0/3641321850 shutdown_connections 2023-07-18T03:24:04.913 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.915+0000 7f6dbf0b4700 1 --2- 172.21.15.134:0/3641321850 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6da8038800 0x7f6da803acb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:04.914 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.915+0000 7f6dbf0b4700 1 --2- 172.21.15.134:0/3641321850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6db80ff560 0x7f6db81997c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:04.914 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.915+0000 7f6dbf0b4700 1 -- 172.21.15.134:0/3641321850 >> 172.21.15.134:0/3641321850 conn(0x7f6db8074f10 msgr2=0x7f6db80fc060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:04.914 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.915+0000 7f6dbf0b4700 1 -- 172.21.15.134:0/3641321850 shutdown_connections 2023-07-18T03:24:04.915 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:04.915+0000 7f6dbf0b4700 1 -- 172.21.15.134:0/3641321850 wait complete. 2023-07-18T03:24:05.292 DEBUG:teuthology.orchestra.run.smithi064:mon.b> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mon.b.service 2023-07-18T03:24:05.295 DEBUG:teuthology.orchestra.run.smithi134:mon.c> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mon.c.service 2023-07-18T03:24:05.297 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-18T03:24:05.297 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-18T03:24:05.328 INFO:journalctl@ceph.mon.c.smithi134.stdout:-- Logs begin at Tue 2023-07-18 03:07:14 UTC. -- 2023-07-18T03:24:05.331 INFO:journalctl@ceph.mon.b.smithi064.stdout:-- Logs begin at Tue 2023-07-18 03:07:19 UTC. -- 2023-07-18T03:24:05.880 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:24:06.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:05 smithi040 ceph-mon[96439]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi040:172.21.15.40=a;smithi064:172.21.15.64=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:24:06.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:05 smithi040 ceph-mon[96439]: Saving service mon spec with placement smithi040:172.21.15.40=a;smithi064:172.21.15.64=b;smithi134:172.21.15.134=c;count:3 2023-07-18T03:24:06.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:05 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:06.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:05 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:06.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:05 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:06.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:05 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:06.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:05 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:06.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:05 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T03:24:06.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:05 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:06.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:05 smithi040 ceph-mon[96439]: Deploying daemon mon.c on smithi134 2023-07-18T03:24:06.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:05 smithi040 ceph-mon[96439]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:07.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:07 smithi134 systemd[1]: Starting Ceph mon.c for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:24:07.856 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.859+0000 7f0fdca89700 1 -- 172.21.15.134:0/2770641120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80ff560 msgr2=0x7f0fd80ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:07.856 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.859+0000 7f0fdca89700 1 --2- 172.21.15.134:0/2770641120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80ff560 0x7f0fd80ff930 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8009910 tx=0x7f0fc8009c20 comp rx=0 tx=0).stop 2023-07-18T03:24:07.857 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.859+0000 7f0fdca89700 1 -- 172.21.15.134:0/2770641120 shutdown_connections 2023-07-18T03:24:07.857 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.859+0000 7f0fdca89700 1 --2- 172.21.15.134:0/2770641120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80ff560 0x7f0fd80ff930 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:07.857 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.859+0000 7f0fdca89700 1 -- 172.21.15.134:0/2770641120 >> 172.21.15.134:0/2770641120 conn(0x7f0fd8074f10 msgr2=0x7f0fd8075310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:07.857 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.859+0000 7f0fdca89700 1 -- 172.21.15.134:0/2770641120 shutdown_connections 2023-07-18T03:24:07.857 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.859+0000 7f0fdca89700 1 -- 172.21.15.134:0/2770641120 wait complete. 2023-07-18T03:24:07.858 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.860+0000 7f0fdca89700 1 Processor -- start 2023-07-18T03:24:07.859 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.860+0000 7f0fdca89700 1 -- start start 2023-07-18T03:24:07.859 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.860+0000 7f0fdca89700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80ff560 0x7f0fd8195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.859 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.860+0000 7f0fdca89700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd8195e80 con 0x7f0fd80ff560 2023-07-18T03:24:07.859 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.861+0000 7f0fd659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80ff560 0x7f0fd8195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.859 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.861+0000 7f0fd659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80ff560 0x7f0fd8195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.134:50376/0 (socket says 172.21.15.134:50376) 2023-07-18T03:24:07.859 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.861+0000 7f0fd659c700 1 -- 172.21.15.134:0/4089275105 learned_addr learned my addr 172.21.15.134:0/4089275105 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-07-18T03:24:07.860 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.861+0000 7f0fd659c700 1 -- 172.21.15.134:0/4089275105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc80095f0 con 0x7f0fd80ff560 2023-07-18T03:24:07.860 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.861+0000 7f0fd659c700 1 --2- 172.21.15.134:0/4089275105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80ff560 0x7f0fd8195940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8004f40 tx=0x7f0fc8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.860 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.861+0000 7f0fc77fe700 1 -- 172.21.15.134:0/4089275105 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fc801d070 con 0x7f0fd80ff560 2023-07-18T03:24:07.860 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.862+0000 7f0fdca89700 1 -- 172.21.15.134:0/4089275105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd8196080 con 0x7f0fd80ff560 2023-07-18T03:24:07.860 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.862+0000 7f0fc77fe700 1 -- 172.21.15.134:0/4089275105 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fc800be20 con 0x7f0fd80ff560 2023-07-18T03:24:07.860 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.862+0000 7f0fc77fe700 1 -- 172.21.15.134:0/4089275105 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fc8022d00 con 0x7f0fd80ff560 2023-07-18T03:24:07.860 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.862+0000 7f0fdca89700 1 -- 172.21.15.134:0/4089275105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd81964a0 con 0x7f0fd80ff560 2023-07-18T03:24:07.861 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.862+0000 7f0fc77fe700 1 -- 172.21.15.134:0/4089275105 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f0fc800f460 con 0x7f0fd80ff560 2023-07-18T03:24:07.861 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.862+0000 7f0fdca89700 1 -- 172.21.15.134:0/4089275105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd8103ac0 con 0x7f0fd80ff560 2023-07-18T03:24:07.861 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.863+0000 7f0fc77fe700 1 --2- 172.21.15.134:0/4089275105 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0fc00387b0 0x7f0fc003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:07.862 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.863+0000 7f0fc77fe700 1 -- 172.21.15.134:0/4089275105 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0fc804c7e0 con 0x7f0fd80ff560 2023-07-18T03:24:07.862 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.863+0000 7f0fd5d9b700 1 --2- 172.21.15.134:0/4089275105 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0fc00387b0 0x7f0fc003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:07.862 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.863+0000 7f0fd5d9b700 1 --2- 172.21.15.134:0/4089275105 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0fc00387b0 0x7f0fc003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc006fd0 tx=0x7f0fcc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:07.864 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:07.868+0000 7f0fc77fe700 1 -- 172.21.15.134:0/4089275105 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0fc8017440 con 0x7f0fd80ff560 2023-07-18T03:24:08.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:07 smithi040 ceph-mon[96439]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:08.142 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:08.146+0000 7f0fdca89700 1 -- 172.21.15.134:0/4089275105 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0fd80fe800 con 0x7f0fd80ff560 2023-07-18T03:24:08.143 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:08.147+0000 7f0fc77fe700 1 -- 172.21.15.134:0/4089275105 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f0fc8027030 con 0x7f0fd80ff560 2023-07-18T03:24:08.143 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-18T03:24:08.144 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":1,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","modified":"2023-07-18T03:22:28.250962Z","created":"2023-07-18T03:22:28.250962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-07-18T03:24:08.145 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:08.149+0000 7f0fdca89700 1 -- 172.21.15.134:0/4089275105 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0fc00387b0 msgr2=0x7f0fc003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:08.145 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:08.149+0000 7f0fdca89700 1 --2- 172.21.15.134:0/4089275105 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0fc00387b0 0x7f0fc003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc006fd0 tx=0x7f0fcc006e40 comp rx=0 tx=0).stop 2023-07-18T03:24:08.146 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:08.149+0000 7f0fdca89700 1 -- 172.21.15.134:0/4089275105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80ff560 msgr2=0x7f0fd8195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:08.146 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:08.149+0000 7f0fdca89700 1 --2- 172.21.15.134:0/4089275105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80ff560 0x7f0fd8195940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8004f40 tx=0x7f0fc8005020 comp rx=0 tx=0).stop 2023-07-18T03:24:08.146 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:08.149+0000 7f0fdca89700 1 -- 172.21.15.134:0/4089275105 shutdown_connections 2023-07-18T03:24:08.146 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:08.149+0000 7f0fdca89700 1 --2- 172.21.15.134:0/4089275105 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0fc00387b0 0x7f0fc003ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:08.146 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:08.149+0000 7f0fdca89700 1 --2- 172.21.15.134:0/4089275105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fd80ff560 0x7f0fd8195940 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:08.146 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:08.149+0000 7f0fdca89700 1 -- 172.21.15.134:0/4089275105 >> 172.21.15.134:0/4089275105 conn(0x7f0fd8074f10 msgr2=0x7f0fd80fc060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:08.146 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:08.149+0000 7f0fdca89700 1 -- 172.21.15.134:0/4089275105 shutdown_connections 2023-07-18T03:24:08.147 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:08.149+0000 7f0fdca89700 1 -- 172.21.15.134:0/4089275105 wait complete. 2023-07-18T03:24:08.147 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 1 2023-07-18T03:24:08.282 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:07 smithi134 podman[98153]: 2023-07-18T03:24:08.779 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: set uid:gid to 167:167 (ceph:ceph) 2023-07-18T03:24:08.779 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev), process ceph-mon, pid 7 2023-07-18T03:24:08.780 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: pidfile_write: ignore empty --pid-file 2023-07-18T03:24:08.780 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: load: jerasure load: lrc 2023-07-18T03:24:08.780 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: RocksDB version: 7.9.2 2023-07-18T03:24:08.780 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Git sha 0 2023-07-18T03:24:08.780 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Compile date 2023-07-17 23:00:14 2023-07-18T03:24:08.780 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: DB SUMMARY 2023-07-18T03:24:08.780 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: DB Session ID: P2TBD2QN1VAOWPV5NTYJ 2023-07-18T03:24:08.781 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: CURRENT file: CURRENT 2023-07-18T03:24:08.781 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: IDENTITY file: IDENTITY 2023-07-18T03:24:08.781 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-18T03:24:08.781 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-07-18T03:24:08.781 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-07-18T03:24:08.781 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.error_if_exists: 0 2023-07-18T03:24:08.781 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.create_if_missing: 0 2023-07-18T03:24:08.782 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.paranoid_checks: 1 2023-07-18T03:24:08.782 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-18T03:24:08.782 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-18T03:24:08.782 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 bash[98153]: 0a3fcd58217265b17a5142d2f441db982fa019c3ea19e24e3232a15fee146e48 2023-07-18T03:24:08.782 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-18T03:24:08.782 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.env: 0x55bddc848720 2023-07-18T03:24:08.782 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.fs: PosixFileSystem 2023-07-18T03:24:08.783 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.info_log: 0x55bddd7c5280 2023-07-18T03:24:08.783 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_file_opening_threads: 16 2023-07-18T03:24:08.783 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.statistics: (nil) 2023-07-18T03:24:08.783 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.use_fsync: 0 2023-07-18T03:24:08.783 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_log_file_size: 0 2023-07-18T03:24:08.783 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-18T03:24:08.783 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-18T03:24:08.784 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.keep_log_file_num: 1000 2023-07-18T03:24:08.784 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.recycle_log_file_num: 0 2023-07-18T03:24:08.784 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.allow_fallocate: 1 2023-07-18T03:24:08.784 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.allow_mmap_reads: 0 2023-07-18T03:24:08.784 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.allow_mmap_writes: 0 2023-07-18T03:24:08.784 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.use_direct_reads: 0 2023-07-18T03:24:08.784 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-18T03:24:08.784 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.create_missing_column_families: 0 2023-07-18T03:24:08.785 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.db_log_dir: 2023-07-18T03:24:08.785 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.wal_dir: 2023-07-18T03:24:08.785 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-18T03:24:08.785 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-18T03:24:08.785 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-18T03:24:08.785 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-18T03:24:08.785 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-18T03:24:08.785 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-18T03:24:08.786 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.advise_random_on_open: 1 2023-07-18T03:24:08.786 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.db_write_buffer_size: 0 2023-07-18T03:24:08.786 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.write_buffer_manager: 0x55bddd6e63c0 2023-07-18T03:24:08.786 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-18T03:24:08.786 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-18T03:24:08.786 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-18T03:24:08.786 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.rate_limiter: (nil) 2023-07-18T03:24:08.787 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-18T03:24:08.787 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.wal_recovery_mode: 2 2023-07-18T03:24:08.787 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.enable_thread_tracking: 0 2023-07-18T03:24:08.787 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.enable_pipelined_write: 0 2023-07-18T03:24:08.787 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.unordered_write: 0 2023-07-18T03:24:08.787 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-18T03:24:08.787 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-18T03:24:08.788 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-18T03:24:08.788 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-18T03:24:08.788 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.row_cache: None 2023-07-18T03:24:08.788 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.wal_filter: None 2023-07-18T03:24:08.788 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-18T03:24:08.788 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.allow_ingest_behind: 0 2023-07-18T03:24:08.788 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.two_write_queues: 0 2023-07-18T03:24:08.788 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.manual_wal_flush: 0 2023-07-18T03:24:08.789 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.wal_compression: 0 2023-07-18T03:24:08.789 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.atomic_flush: 0 2023-07-18T03:24:08.789 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-18T03:24:08.789 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-18T03:24:08.789 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-18T03:24:08.789 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.log_readahead_size: 0 2023-07-18T03:24:08.789 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-18T03:24:08.789 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.best_efforts_recovery: 0 2023-07-18T03:24:08.790 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-18T03:24:08.790 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-18T03:24:08.790 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.allow_data_in_errors: 0 2023-07-18T03:24:08.790 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.db_host_id: __hostname__ 2023-07-18T03:24:08.790 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-18T03:24:08.790 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_background_jobs: 2 2023-07-18T03:24:08.790 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_background_compactions: -1 2023-07-18T03:24:08.791 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_subcompactions: 1 2023-07-18T03:24:08.791 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-18T03:24:08.791 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-18T03:24:08.791 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-18T03:24:08.791 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_total_wal_size: 0 2023-07-18T03:24:08.791 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-18T03:24:08.791 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-18T03:24:08.791 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-18T03:24:08.792 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-18T03:24:08.792 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_open_files: -1 2023-07-18T03:24:08.792 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.bytes_per_sync: 0 2023-07-18T03:24:08.792 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-18T03:24:08.792 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-18T03:24:08.792 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compaction_readahead_size: 0 2023-07-18T03:24:08.792 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_background_flushes: -1 2023-07-18T03:24:08.793 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Compression algorithms supported: 2023-07-18T03:24:08.793 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-18T03:24:08.793 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: kZSTD supported: 0 2023-07-18T03:24:08.793 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: kXpressCompression supported: 0 2023-07-18T03:24:08.793 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: kLZ4HCCompression supported: 1 2023-07-18T03:24:08.793 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: kZlibCompression supported: 1 2023-07-18T03:24:08.793 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: kSnappyCompression supported: 1 2023-07-18T03:24:08.793 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: kLZ4Compression supported: 1 2023-07-18T03:24:08.794 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: kBZip2Compression supported: 0 2023-07-18T03:24:08.794 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-18T03:24:08.794 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-18T03:24:08.794 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-07-18T03:24:08.794 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-18T03:24:08.794 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-18T03:24:08.794 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.merge_operator: 2023-07-18T03:24:08.794 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compaction_filter: None 2023-07-18T03:24:08.795 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compaction_filter_factory: None 2023-07-18T03:24:08.795 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.sst_partitioner_factory: None 2023-07-18T03:24:08.795 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-18T03:24:08.795 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-18T03:24:08.795 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bddd7c5380) 2023-07-18T03:24:08.795 INFO:journalctl@ceph.mon.c.smithi134.stdout: cache_index_and_filter_blocks: 1 2023-07-18T03:24:08.795 INFO:journalctl@ceph.mon.c.smithi134.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-18T03:24:08.796 INFO:journalctl@ceph.mon.c.smithi134.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-18T03:24:08.796 INFO:journalctl@ceph.mon.c.smithi134.stdout: pin_top_level_index_and_filter: 1 2023-07-18T03:24:08.796 INFO:journalctl@ceph.mon.c.smithi134.stdout: index_type: 0 2023-07-18T03:24:08.796 INFO:journalctl@ceph.mon.c.smithi134.stdout: data_block_index_type: 0 2023-07-18T03:24:08.796 INFO:journalctl@ceph.mon.c.smithi134.stdout: index_shortening: 1 2023-07-18T03:24:08.796 INFO:journalctl@ceph.mon.c.smithi134.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-18T03:24:08.796 INFO:journalctl@ceph.mon.c.smithi134.stdout: checksum: 4 2023-07-18T03:24:08.796 INFO:journalctl@ceph.mon.c.smithi134.stdout: no_block_cache: 0 2023-07-18T03:24:08.797 INFO:journalctl@ceph.mon.c.smithi134.stdout: block_cache: 0x55bddd7851f0 2023-07-18T03:24:08.797 INFO:journalctl@ceph.mon.c.smithi134.stdout: block_cache_name: BinnedLRUCache 2023-07-18T03:24:08.797 INFO:journalctl@ceph.mon.c.smithi134.stdout: block_cache_options: 2023-07-18T03:24:08.797 INFO:journalctl@ceph.mon.c.smithi134.stdout: capacity : 536870912 2023-07-18T03:24:08.797 INFO:journalctl@ceph.mon.c.smithi134.stdout: num_shard_bits : 4 2023-07-18T03:24:08.797 INFO:journalctl@ceph.mon.c.smithi134.stdout: strict_capacity_limit : 0 2023-07-18T03:24:08.797 INFO:journalctl@ceph.mon.c.smithi134.stdout: high_pri_pool_ratio: 0.000 2023-07-18T03:24:08.798 INFO:journalctl@ceph.mon.c.smithi134.stdout: block_cache_compressed: (nil) 2023-07-18T03:24:08.798 INFO:journalctl@ceph.mon.c.smithi134.stdout: persistent_cache: (nil) 2023-07-18T03:24:08.798 INFO:journalctl@ceph.mon.c.smithi134.stdout: block_size: 4096 2023-07-18T03:24:08.798 INFO:journalctl@ceph.mon.c.smithi134.stdout: block_size_deviation: 10 2023-07-18T03:24:08.798 INFO:journalctl@ceph.mon.c.smithi134.stdout: block_restart_interval: 16 2023-07-18T03:24:08.798 INFO:journalctl@ceph.mon.c.smithi134.stdout: index_block_restart_interval: 1 2023-07-18T03:24:08.798 INFO:journalctl@ceph.mon.c.smithi134.stdout: metadata_block_size: 4096 2023-07-18T03:24:08.798 INFO:journalctl@ceph.mon.c.smithi134.stdout: partition_filters: 0 2023-07-18T03:24:08.799 INFO:journalctl@ceph.mon.c.smithi134.stdout: use_delta_encoding: 1 2023-07-18T03:24:08.799 INFO:journalctl@ceph.mon.c.smithi134.stdout: filter_policy: bloomfilter 2023-07-18T03:24:08.799 INFO:journalctl@ceph.mon.c.smithi134.stdout: whole_key_filtering: 1 2023-07-18T03:24:08.799 INFO:journalctl@ceph.mon.c.smithi134.stdout: verify_compression: 0 2023-07-18T03:24:08.799 INFO:journalctl@ceph.mon.c.smithi134.stdout: read_amp_bytes_per_bit: 0 2023-07-18T03:24:08.799 INFO:journalctl@ceph.mon.c.smithi134.stdout: format_version: 5 2023-07-18T03:24:08.799 INFO:journalctl@ceph.mon.c.smithi134.stdout: enable_index_compression: 1 2023-07-18T03:24:08.799 INFO:journalctl@ceph.mon.c.smithi134.stdout: block_align: 0 2023-07-18T03:24:08.800 INFO:journalctl@ceph.mon.c.smithi134.stdout: max_auto_readahead_size: 262144 2023-07-18T03:24:08.800 INFO:journalctl@ceph.mon.c.smithi134.stdout: prepopulate_block_cache: 0 2023-07-18T03:24:08.800 INFO:journalctl@ceph.mon.c.smithi134.stdout: initial_auto_readahead_size: 8192 2023-07-18T03:24:08.800 INFO:journalctl@ceph.mon.c.smithi134.stdout: num_file_reads_for_auto_readahead: 2 2023-07-18T03:24:08.800 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.write_buffer_size: 33554432 2023-07-18T03:24:08.800 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_write_buffer_number: 2 2023-07-18T03:24:08.800 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compression: NoCompression 2023-07-18T03:24:08.801 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.bottommost_compression: Disabled 2023-07-18T03:24:08.801 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.prefix_extractor: nullptr 2023-07-18T03:24:08.801 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-18T03:24:08.801 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.num_levels: 7 2023-07-18T03:24:08.801 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-18T03:24:08.801 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-18T03:24:08.801 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-18T03:24:08.801 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-18T03:24:08.802 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-18T03:24:08.802 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-18T03:24:08.802 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-18T03:24:08.802 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-18T03:24:08.802 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-18T03:24:08.802 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-18T03:24:08.802 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-18T03:24:08.803 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-18T03:24:08.803 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-18T03:24:08.803 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compression_opts.level: 32767 2023-07-18T03:24:08.803 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compression_opts.strategy: 0 2023-07-18T03:24:08.803 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-18T03:24:08.803 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-18T03:24:08.803 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-18T03:24:08.803 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-18T03:24:08.804 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compression_opts.enabled: false 2023-07-18T03:24:08.804 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-18T03:24:08.804 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-18T03:24:08.804 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-18T03:24:08.804 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-18T03:24:08.804 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.target_file_size_base: 67108864 2023-07-18T03:24:08.804 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-18T03:24:08.805 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-18T03:24:08.805 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-18T03:24:08.805 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-18T03:24:08.805 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-18T03:24:08.805 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-18T03:24:08.805 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-18T03:24:08.805 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-18T03:24:08.806 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-18T03:24:08.806 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-18T03:24:08.806 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-18T03:24:08.806 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-18T03:24:08.806 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-18T03:24:08.806 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-18T03:24:08.806 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.arena_block_size: 1048576 2023-07-18T03:24:08.806 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-18T03:24:08.807 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-18T03:24:08.807 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.disable_auto_compactions: 0 2023-07-18T03:24:08.807 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-18T03:24:08.807 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-18T03:24:08.807 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-18T03:24:08.807 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-18T03:24:08.807 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-18T03:24:08.808 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-18T03:24:08.808 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-18T03:24:08.808 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-18T03:24:08.808 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-18T03:24:08.808 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-18T03:24:08.808 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.table_properties_collectors: 2023-07-18T03:24:08.808 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.inplace_update_support: 0 2023-07-18T03:24:08.808 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-18T03:24:08.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-18T03:24:08.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-18T03:24:08.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-18T03:24:08.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.bloom_locality: 0 2023-07-18T03:24:08.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.max_successive_merges: 0 2023-07-18T03:24:08.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-18T03:24:08.810 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.paranoid_file_checks: 0 2023-07-18T03:24:08.810 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.force_consistency_checks: 1 2023-07-18T03:24:08.810 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.report_bg_io_stats: 0 2023-07-18T03:24:08.810 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.ttl: 2592000 2023-07-18T03:24:08.810 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-18T03:24:08.810 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-18T03:24:08.810 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-18T03:24:08.811 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.enable_blob_files: false 2023-07-18T03:24:08.811 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.min_blob_size: 0 2023-07-18T03:24:08.811 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.blob_file_size: 268435456 2023-07-18T03:24:08.811 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-18T03:24:08.811 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-18T03:24:08.811 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-18T03:24:08.811 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-18T03:24:08.811 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-18T03:24:08.812 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.blob_file_starting_level: 0 2023-07-18T03:24:08.812 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-18T03:24:08.812 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-18T03:24:08.812 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-18T03:24:08.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 86d6c05c-2c1d-4fcb-ab2f-012a0d535135 2023-07-18T03:24:08.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689650648413560, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-18T03:24:08.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-18T03:24:08.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689650648414107, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689650648, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "86d6c05c-2c1d-4fcb-ab2f-012a0d535135", "db_session_id": "P2TBD2QN1VAOWPV5NTYJ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-18T03:24:08.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689650648414197, "job": 1, "event": "recovery_finished"} 2023-07-18T03:24:08.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-18T03:24:08.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-18T03:24:08.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bddd81e000 2023-07-18T03:24:08.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: DB pointer 0x55bddd80a000 2023-07-18T03:24:08.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-18T03:24:08.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-18T03:24:08.815 INFO:journalctl@ceph.mon.c.smithi134.stdout: ** DB Stats ** 2023-07-18T03:24:08.815 INFO:journalctl@ceph.mon.c.smithi134.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-18T03:24:08.815 INFO:journalctl@ceph.mon.c.smithi134.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-18T03:24:08.815 INFO:journalctl@ceph.mon.c.smithi134.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-18T03:24:08.815 INFO:journalctl@ceph.mon.c.smithi134.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-18T03:24:08.815 INFO:journalctl@ceph.mon.c.smithi134.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-18T03:24:08.816 INFO:journalctl@ceph.mon.c.smithi134.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-18T03:24:08.816 INFO:journalctl@ceph.mon.c.smithi134.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-18T03:24:08.816 INFO:journalctl@ceph.mon.c.smithi134.stdout: 2023-07-18T03:24:08.816 INFO:journalctl@ceph.mon.c.smithi134.stdout: ** Compaction Stats [default] ** 2023-07-18T03:24:08.816 INFO:journalctl@ceph.mon.c.smithi134.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-18T03:24:08.816 INFO:journalctl@ceph.mon.c.smithi134.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-18T03:24:08.816 INFO:journalctl@ceph.mon.c.smithi134.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-18T03:24:08.816 INFO:journalctl@ceph.mon.c.smithi134.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-18T03:24:08.817 INFO:journalctl@ceph.mon.c.smithi134.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-18T03:24:08.817 INFO:journalctl@ceph.mon.c.smithi134.stdout: 2023-07-18T03:24:08.817 INFO:journalctl@ceph.mon.c.smithi134.stdout: ** Compaction Stats [default] ** 2023-07-18T03:24:08.817 INFO:journalctl@ceph.mon.c.smithi134.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-18T03:24:08.817 INFO:journalctl@ceph.mon.c.smithi134.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-18T03:24:08.817 INFO:journalctl@ceph.mon.c.smithi134.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-18T03:24:08.817 INFO:journalctl@ceph.mon.c.smithi134.stdout: 2023-07-18T03:24:08.818 INFO:journalctl@ceph.mon.c.smithi134.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-18T03:24:08.818 INFO:journalctl@ceph.mon.c.smithi134.stdout: 2023-07-18T03:24:08.818 INFO:journalctl@ceph.mon.c.smithi134.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-18T03:24:08.818 INFO:journalctl@ceph.mon.c.smithi134.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-18T03:24:08.818 INFO:journalctl@ceph.mon.c.smithi134.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-18T03:24:08.818 INFO:journalctl@ceph.mon.c.smithi134.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-18T03:24:08.818 INFO:journalctl@ceph.mon.c.smithi134.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-18T03:24:08.819 INFO:journalctl@ceph.mon.c.smithi134.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-18T03:24:08.819 INFO:journalctl@ceph.mon.c.smithi134.stdout: Cumulative compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-18T03:24:08.819 INFO:journalctl@ceph.mon.c.smithi134.stdout: Interval compaction: 0.00 GB write, 0.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-18T03:24:08.819 INFO:journalctl@ceph.mon.c.smithi134.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-18T03:24:08.819 INFO:journalctl@ceph.mon.c.smithi134.stdout: Block cache BinnedLRUCache@0x55bddd7851f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-07-18T03:24:08.819 INFO:journalctl@ceph.mon.c.smithi134.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-18T03:24:08.819 INFO:journalctl@ceph.mon.c.smithi134.stdout: 2023-07-18T03:24:08.819 INFO:journalctl@ceph.mon.c.smithi134.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-18T03:24:08.820 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-07-18T03:24:08.820 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: using public_addr v2:172.21.15.134:0/0 -> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2023-07-18T03:24:08.820 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: starting mon.c rank -1 at public addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] at bind addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:24:08.820 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: mon.c@-1(???) e0 preinit fsid 3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:24:08.820 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: mon.c@-1(synchronizing).mds e1 new map 2023-07-18T03:24:08.820 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: mon.c@-1(synchronizing).mds e1 print_map 2023-07-18T03:24:08.820 INFO:journalctl@ceph.mon.c.smithi134.stdout: e1 2023-07-18T03:24:08.821 INFO:journalctl@ceph.mon.c.smithi134.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-18T03:24:08.821 INFO:journalctl@ceph.mon.c.smithi134.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-18T03:24:08.821 INFO:journalctl@ceph.mon.c.smithi134.stdout: legacy client fscid: -1 2023-07-18T03:24:08.821 INFO:journalctl@ceph.mon.c.smithi134.stdout: 2023-07-18T03:24:08.821 INFO:journalctl@ceph.mon.c.smithi134.stdout: No filesystems configured 2023-07-18T03:24:08.821 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-18T03:24:08.821 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-18T03:24:08.821 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-18T03:24:08.822 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-18T03:24:08.822 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-18T03:24:08.822 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-18T03:24:08.822 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-18T03:24:08.822 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T03:24:08.822 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T03:24:08.822 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T03:24:08.823 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.823 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:08.823 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:24:08.823 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.823 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:08.823 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:24:08.823 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: Deploying cephadm binary to smithi134 2023-07-18T03:24:08.823 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:08.824 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.824 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.824 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.824 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.824 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:24:08.824 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:08.824 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:08.825 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: Updating smithi064:/etc/ceph/ceph.conf 2023-07-18T03:24:08.825 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: Updating smithi064:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:24:08.825 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:24:08.825 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: Updating smithi064:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T03:24:08.825 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:08.825 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.825 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: Added host smithi134 2023-07-18T03:24:08.825 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.826 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.826 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.826 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:08.826 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.826 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.826 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:08.826 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:24:08.827 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.827 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:08.827 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3668808298' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-18T03:24:08.827 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3668808298' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-18T03:24:08.827 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T03:24:08.827 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.827 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.827 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.828 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.828 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:24:08.828 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:08.828 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:08.828 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: Updating smithi134:/etc/ceph/ceph.conf 2023-07-18T03:24:08.828 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: Updating smithi134:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:24:08.828 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:24:08.828 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: Updating smithi134:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T03:24:08.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:08.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi040:172.21.15.40=a;smithi064:172.21.15.64=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:24:08.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: Saving service mon spec with placement smithi040:172.21.15.40=a;smithi064:172.21.15.64=b;smithi134:172.21.15.134=c;count:3 2023-07-18T03:24:08.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.830 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:08.830 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:08.830 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:08.830 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:08.830 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T03:24:08.830 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:08.830 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: Deploying daemon mon.c on smithi134 2023-07-18T03:24:08.830 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:08.831 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:08.831 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-18T03:24:08.831 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-07-18T03:24:08.831 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: expand_channel_meta expand map: {default=false} 2023-07-18T03:24:08.831 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: expand_channel_meta from 'false' to 'false' 2023-07-18T03:24:08.831 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: expand_channel_meta expanded map: {default=false} 2023-07-18T03:24:08.831 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: expand_channel_meta expand map: {default=info} 2023-07-18T03:24:08.831 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: expand_channel_meta from 'info' to 'info' 2023-07-18T03:24:08.832 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: expand_channel_meta expanded map: {default=info} 2023-07-18T03:24:08.832 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: expand_channel_meta expand map: {default=daemon} 2023-07-18T03:24:08.832 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-18T03:24:08.832 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: expand_channel_meta expanded map: {default=daemon} 2023-07-18T03:24:08.832 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: expand_channel_meta expand map: {default=debug} 2023-07-18T03:24:08.832 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: expand_channel_meta from 'debug' to 'debug' 2023-07-18T03:24:08.832 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 ceph-mon[98216]: expand_channel_meta expanded map: {default=debug} 2023-07-18T03:24:08.832 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:08 smithi134 systemd[1]: Started Ceph mon.c for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:24:09.891 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-18T03:24:09.891 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-18T03:24:10.456 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:24:13.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:24:13.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: mon.a calling monitor election 2023-07-18T03:24:13.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:13.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:13.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:13.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:13.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: mon.c calling monitor election 2023-07-18T03:24:13.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:13.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:13.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:13.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:13.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-18T03:24:13.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: monmap e2: 2 mons at {a=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} 2023-07-18T03:24:13.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: fsmap 2023-07-18T03:24:13.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T03:24:13.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: mgrmap e13: x(active, since 41s) 2023-07-18T03:24:13.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: overall HEALTH_OK 2023-07-18T03:24:13.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:13.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:13 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:13.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:24:13.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: mon.a calling monitor election 2023-07-18T03:24:13.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:13.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:13.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:13.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:13.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: mon.c calling monitor election 2023-07-18T03:24:13.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:13.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:13.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:13.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:13.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-18T03:24:13.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: monmap e2: 2 mons at {a=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} 2023-07-18T03:24:13.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: fsmap 2023-07-18T03:24:13.833 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T03:24:13.833 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: mgrmap e13: x(active, since 41s) 2023-07-18T03:24:13.833 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: overall HEALTH_OK 2023-07-18T03:24:13.833 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:13.833 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:13 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:14.653 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.656+0000 7fb89a50c700 1 -- 172.21.15.134:0/3833870042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0035c0 msgr2=0x7fb87c005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:14.653 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.656+0000 7fb89a50c700 1 --2- 172.21.15.134:0/3833870042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0035c0 0x7fb87c005a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb894100bc0 tx=0x7fb88400fe60 comp rx=0 tx=0).stop 2023-07-18T03:24:14.653 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.656+0000 7fb89a50c700 1 -- 172.21.15.134:0/3833870042 shutdown_connections 2023-07-18T03:24:14.653 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.656+0000 7fb89a50c700 1 --2- 172.21.15.134:0/3833870042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0035c0 0x7fb87c005a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:14.653 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.656+0000 7fb89a50c700 1 -- 172.21.15.134:0/3833870042 >> 172.21.15.134:0/3833870042 conn(0x7fb8940fb210 msgr2=0x7fb8940fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:14.654 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.656+0000 7fb89a50c700 1 -- 172.21.15.134:0/3833870042 shutdown_connections 2023-07-18T03:24:14.654 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.656+0000 7fb89a50c700 1 -- 172.21.15.134:0/3833870042 wait complete. 2023-07-18T03:24:14.654 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.657+0000 7fb89a50c700 1 Processor -- start 2023-07-18T03:24:14.654 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.658+0000 7fb89a50c700 1 -- start start 2023-07-18T03:24:14.655 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.658+0000 7fb89a50c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0035c0 0x7fb89419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:14.656 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.658+0000 7fb89a50c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb89419aed0 0x7fb894194a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:14.656 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.658+0000 7fb89a50c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89419b400 con 0x7fb87c0035c0 2023-07-18T03:24:14.656 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.658+0000 7fb89a50c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89419b570 con 0x7fb89419aed0 2023-07-18T03:24:14.656 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.659+0000 7fb8937fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb89419aed0 0x7fb894194a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:14.656 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.659+0000 7fb8937fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb89419aed0 0x7fb894194a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:47964/0 (socket says 172.21.15.134:47964) 2023-07-18T03:24:14.657 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.659+0000 7fb8937fe700 1 -- 172.21.15.134:0/1894313842 learned_addr learned my addr 172.21.15.134:0/1894313842 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-07-18T03:24:14.657 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.659+0000 7fb893fff700 1 --2- 172.21.15.134:0/1894313842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0035c0 0x7fb89419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:14.657 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.659+0000 7fb8937fe700 1 -- 172.21.15.134:0/1894313842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb89419aed0 msgr2=0x7fb894194a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-07-18T03:24:14.657 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.659+0000 7fb8937fe700 1 -- 172.21.15.134:0/1894313842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb89419aed0 msgr2=0x7fb894194a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-18T03:24:14.657 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.659+0000 7fb8937fe700 1 --2- 172.21.15.134:0/1894313842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb89419aed0 0x7fb894194a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T03:24:14.657 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.659+0000 7fb8937fe700 1 --2- 172.21.15.134:0/1894313842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb89419aed0 0x7fb894194a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T03:24:14.657 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.659+0000 7fb893fff700 1 -- 172.21.15.134:0/1894313842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb89419aed0 msgr2=0x7fb894194a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:24:14.658 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.659+0000 7fb893fff700 1 --2- 172.21.15.134:0/1894313842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb89419aed0 0x7fb894194a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:14.658 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.659+0000 7fb893fff700 1 -- 172.21.15.134:0/1894313842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88400f8b0 con 0x7fb87c0035c0 2023-07-18T03:24:14.658 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.659+0000 7fb893fff700 1 --2- 172.21.15.134:0/1894313842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0035c0 0x7fb89419a990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb884005cd0 tx=0x7fb884013c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:14.659 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.659+0000 7fb8917fa700 1 -- 172.21.15.134:0/1894313842 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fb884023070 con 0x7fb87c0035c0 2023-07-18T03:24:14.659 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.659+0000 7fb8917fa700 1 -- 172.21.15.134:0/1894313842 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb884004040 con 0x7fb87c0035c0 2023-07-18T03:24:14.659 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.659+0000 7fb89a50c700 1 -- 172.21.15.134:0/1894313842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb894195010 con 0x7fb87c0035c0 2023-07-18T03:24:14.659 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.660+0000 7fb8917fa700 1 -- 172.21.15.134:0/1894313842 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fb88401cc90 con 0x7fb87c0035c0 2023-07-18T03:24:14.659 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.660+0000 7fb89a50c700 1 -- 172.21.15.134:0/1894313842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8941955e0 con 0x7fb87c0035c0 2023-07-18T03:24:14.660 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.661+0000 7fb8917fa700 1 -- 172.21.15.134:0/1894313842 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fb884011680 con 0x7fb87c0035c0 2023-07-18T03:24:14.660 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.661+0000 7fb8917fa700 1 --2- 172.21.15.134:0/1894313842 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb8800385f0 0x7fb88003aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:14.660 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.661+0000 7fb89a50c700 1 -- 172.21.15.134:0/1894313842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb894061b40 con 0x7fb87c0035c0 2023-07-18T03:24:14.660 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.661+0000 7fb8917fa700 1 -- 172.21.15.134:0/1894313842 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb88404d540 con 0x7fb87c0035c0 2023-07-18T03:24:14.660 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.661+0000 7fb8937fe700 1 --2- 172.21.15.134:0/1894313842 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb8800385f0 0x7fb88003aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:14.660 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.662+0000 7fb8937fe700 1 --2- 172.21.15.134:0/1894313842 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb8800385f0 0x7fb88003aaa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb888009180 tx=0x7fb888008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:14.663 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.666+0000 7fb8917fa700 1 -- 172.21.15.134:0/1894313842 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb8840103d0 con 0x7fb87c0035c0 2023-07-18T03:24:14.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:14 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:14.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:14 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T03:24:14.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:14 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:14.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:14 smithi134 ceph-mon[98216]: Deploying daemon mon.b on smithi064 2023-07-18T03:24:14.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:14 smithi134 ceph-mon[98216]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:14.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:14 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:14.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:14 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:14.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:14 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T03:24:14.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:14 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:14.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:14 smithi040 ceph-mon[96439]: Deploying daemon mon.b on smithi064 2023-07-18T03:24:14.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:14 smithi040 ceph-mon[96439]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:14.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:14 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:14.926 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.930+0000 7fb89a50c700 1 -- 172.21.15.134:0/1894313842 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb89419f7a0 con 0x7fb87c0035c0 2023-07-18T03:24:14.927 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.930+0000 7fb8917fa700 1 -- 172.21.15.134:0/1894313842 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7fb88402c210 con 0x7fb87c0035c0 2023-07-18T03:24:14.927 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-18T03:24:14.927 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":2,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","modified":"2023-07-18T03:24:08.445513Z","created":"2023-07-18T03:22:28.250962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-18T03:24:14.929 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.932+0000 7fb89a50c700 1 -- 172.21.15.134:0/1894313842 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb8800385f0 msgr2=0x7fb88003aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:14.929 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.932+0000 7fb89a50c700 1 --2- 172.21.15.134:0/1894313842 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb8800385f0 0x7fb88003aaa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb888009180 tx=0x7fb888008040 comp rx=0 tx=0).stop 2023-07-18T03:24:14.930 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.932+0000 7fb89a50c700 1 -- 172.21.15.134:0/1894313842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0035c0 msgr2=0x7fb89419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:14.930 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.932+0000 7fb89a50c700 1 --2- 172.21.15.134:0/1894313842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0035c0 0x7fb89419a990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb884005cd0 tx=0x7fb884013c40 comp rx=0 tx=0).stop 2023-07-18T03:24:14.930 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.932+0000 7fb89a50c700 1 -- 172.21.15.134:0/1894313842 shutdown_connections 2023-07-18T03:24:14.930 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.932+0000 7fb89a50c700 1 --2- 172.21.15.134:0/1894313842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0035c0 0x7fb89419a990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:14.930 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.932+0000 7fb89a50c700 1 --2- 172.21.15.134:0/1894313842 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb8800385f0 0x7fb88003aaa0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:14.930 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.932+0000 7fb89a50c700 1 --2- 172.21.15.134:0/1894313842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb89419aed0 0x7fb894194a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:14.931 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.932+0000 7fb89a50c700 1 -- 172.21.15.134:0/1894313842 >> 172.21.15.134:0/1894313842 conn(0x7fb8940fb210 msgr2=0x7fb894103280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:14.931 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.933+0000 7fb89a50c700 1 -- 172.21.15.134:0/1894313842 shutdown_connections 2023-07-18T03:24:14.931 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:14.933+0000 7fb89a50c700 1 -- 172.21.15.134:0/1894313842 wait complete. 2023-07-18T03:24:14.931 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 2 2023-07-18T03:24:15.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:15 smithi134 ceph-mon[98216]: from='client.? 172.21.15.134:0/1894313842' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-18T03:24:15.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:15 smithi040 ceph-mon[96439]: from='client.? 172.21.15.134:0/1894313842' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-18T03:24:16.320 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-18T03:24:16.321 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-18T03:24:16.413 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 systemd[1]: Starting Ceph mon.b for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:24:16.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:16 smithi134 ceph-mon[98216]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:16.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:16 smithi040 ceph-mon[96439]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:16.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 podman[98019]: 2023-07-18T03:24:16.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: set uid:gid to 167:167 (ceph:ceph) 2023-07-18T03:24:16.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev), process ceph-mon, pid 7 2023-07-18T03:24:16.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 bash[98019]: 06bbfda0118c37af0492d44670b96f3d947d9e7042688e3e84728137c06ff463 2023-07-18T03:24:16.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: pidfile_write: ignore empty --pid-file 2023-07-18T03:24:16.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: load: jerasure load: lrc 2023-07-18T03:24:16.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: RocksDB version: 7.9.2 2023-07-18T03:24:16.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Git sha 0 2023-07-18T03:24:16.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Compile date 2023-07-17 23:00:14 2023-07-18T03:24:16.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: DB SUMMARY 2023-07-18T03:24:16.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: DB Session ID: OE8UT6YZUA4K6NBDY3FB 2023-07-18T03:24:16.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: CURRENT file: CURRENT 2023-07-18T03:24:16.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: IDENTITY file: IDENTITY 2023-07-18T03:24:16.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-18T03:24:16.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-07-18T03:24:16.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-07-18T03:24:16.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.error_if_exists: 0 2023-07-18T03:24:16.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.create_if_missing: 0 2023-07-18T03:24:16.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.paranoid_checks: 1 2023-07-18T03:24:16.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-18T03:24:16.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-18T03:24:16.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-18T03:24:16.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.env: 0x5639d3451720 2023-07-18T03:24:16.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.fs: PosixFileSystem 2023-07-18T03:24:16.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.info_log: 0x5639d4841280 2023-07-18T03:24:16.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_file_opening_threads: 16 2023-07-18T03:24:16.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.statistics: (nil) 2023-07-18T03:24:16.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.use_fsync: 0 2023-07-18T03:24:16.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_log_file_size: 0 2023-07-18T03:24:16.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-18T03:24:16.855 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-18T03:24:16.855 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.keep_log_file_num: 1000 2023-07-18T03:24:16.855 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.recycle_log_file_num: 0 2023-07-18T03:24:16.855 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.allow_fallocate: 1 2023-07-18T03:24:16.855 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.allow_mmap_reads: 0 2023-07-18T03:24:16.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.allow_mmap_writes: 0 2023-07-18T03:24:16.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.use_direct_reads: 0 2023-07-18T03:24:16.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-18T03:24:16.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.create_missing_column_families: 0 2023-07-18T03:24:16.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.db_log_dir: 2023-07-18T03:24:16.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.wal_dir: 2023-07-18T03:24:16.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-18T03:24:16.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-18T03:24:16.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-18T03:24:16.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-18T03:24:16.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-18T03:24:16.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-18T03:24:16.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.advise_random_on_open: 1 2023-07-18T03:24:16.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.db_write_buffer_size: 0 2023-07-18T03:24:16.857 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.write_buffer_manager: 0x5639d47623c0 2023-07-18T03:24:16.858 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-18T03:24:16.858 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-18T03:24:16.858 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-18T03:24:16.858 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.rate_limiter: (nil) 2023-07-18T03:24:16.858 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-18T03:24:16.858 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.wal_recovery_mode: 2 2023-07-18T03:24:16.858 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.enable_thread_tracking: 0 2023-07-18T03:24:16.858 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.enable_pipelined_write: 0 2023-07-18T03:24:16.859 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.unordered_write: 0 2023-07-18T03:24:16.859 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-18T03:24:16.859 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-18T03:24:16.859 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-18T03:24:16.859 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-18T03:24:16.859 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.row_cache: None 2023-07-18T03:24:16.859 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.wal_filter: None 2023-07-18T03:24:16.860 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-18T03:24:16.860 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.allow_ingest_behind: 0 2023-07-18T03:24:16.860 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.two_write_queues: 0 2023-07-18T03:24:16.860 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.manual_wal_flush: 0 2023-07-18T03:24:16.860 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.wal_compression: 0 2023-07-18T03:24:16.860 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.atomic_flush: 0 2023-07-18T03:24:16.860 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-18T03:24:16.860 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-18T03:24:16.861 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-18T03:24:16.861 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.log_readahead_size: 0 2023-07-18T03:24:16.861 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-18T03:24:16.861 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.best_efforts_recovery: 0 2023-07-18T03:24:16.861 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-18T03:24:16.861 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-18T03:24:16.861 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.allow_data_in_errors: 0 2023-07-18T03:24:16.863 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.db_host_id: __hostname__ 2023-07-18T03:24:16.863 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-18T03:24:16.863 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_background_jobs: 2 2023-07-18T03:24:16.863 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_background_compactions: -1 2023-07-18T03:24:16.863 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_subcompactions: 1 2023-07-18T03:24:16.864 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-18T03:24:16.864 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-18T03:24:16.864 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-18T03:24:16.864 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_total_wal_size: 0 2023-07-18T03:24:16.864 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-18T03:24:16.864 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-18T03:24:16.864 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-18T03:24:16.865 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-18T03:24:16.865 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_open_files: -1 2023-07-18T03:24:16.865 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.bytes_per_sync: 0 2023-07-18T03:24:16.865 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-18T03:24:16.865 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-18T03:24:16.865 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compaction_readahead_size: 0 2023-07-18T03:24:16.865 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_background_flushes: -1 2023-07-18T03:24:16.865 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Compression algorithms supported: 2023-07-18T03:24:16.866 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-18T03:24:16.866 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: kZSTD supported: 0 2023-07-18T03:24:16.866 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: kXpressCompression supported: 0 2023-07-18T03:24:16.866 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: kLZ4HCCompression supported: 1 2023-07-18T03:24:16.866 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: kZlibCompression supported: 1 2023-07-18T03:24:16.866 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: kSnappyCompression supported: 1 2023-07-18T03:24:16.866 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: kLZ4Compression supported: 1 2023-07-18T03:24:16.867 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: kBZip2Compression supported: 0 2023-07-18T03:24:16.867 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-18T03:24:16.867 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-18T03:24:16.867 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-07-18T03:24:16.867 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-18T03:24:16.867 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-18T03:24:16.867 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.merge_operator: 2023-07-18T03:24:16.867 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compaction_filter: None 2023-07-18T03:24:16.868 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compaction_filter_factory: None 2023-07-18T03:24:16.868 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.sst_partitioner_factory: None 2023-07-18T03:24:16.868 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-18T03:24:16.868 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-18T03:24:16.868 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5639d4841380) 2023-07-18T03:24:16.868 INFO:journalctl@ceph.mon.b.smithi064.stdout: cache_index_and_filter_blocks: 1 2023-07-18T03:24:16.868 INFO:journalctl@ceph.mon.b.smithi064.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-18T03:24:16.869 INFO:journalctl@ceph.mon.b.smithi064.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-18T03:24:16.869 INFO:journalctl@ceph.mon.b.smithi064.stdout: pin_top_level_index_and_filter: 1 2023-07-18T03:24:16.869 INFO:journalctl@ceph.mon.b.smithi064.stdout: index_type: 0 2023-07-18T03:24:16.869 INFO:journalctl@ceph.mon.b.smithi064.stdout: data_block_index_type: 0 2023-07-18T03:24:16.869 INFO:journalctl@ceph.mon.b.smithi064.stdout: index_shortening: 1 2023-07-18T03:24:16.869 INFO:journalctl@ceph.mon.b.smithi064.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-18T03:24:16.869 INFO:journalctl@ceph.mon.b.smithi064.stdout: checksum: 4 2023-07-18T03:24:16.869 INFO:journalctl@ceph.mon.b.smithi064.stdout: no_block_cache: 0 2023-07-18T03:24:16.870 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_cache: 0x5639d48011f0 2023-07-18T03:24:16.870 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_cache_name: BinnedLRUCache 2023-07-18T03:24:16.870 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_cache_options: 2023-07-18T03:24:16.870 INFO:journalctl@ceph.mon.b.smithi064.stdout: capacity : 536870912 2023-07-18T03:24:16.870 INFO:journalctl@ceph.mon.b.smithi064.stdout: num_shard_bits : 4 2023-07-18T03:24:16.870 INFO:journalctl@ceph.mon.b.smithi064.stdout: strict_capacity_limit : 0 2023-07-18T03:24:16.870 INFO:journalctl@ceph.mon.b.smithi064.stdout: high_pri_pool_ratio: 0.000 2023-07-18T03:24:16.870 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_cache_compressed: (nil) 2023-07-18T03:24:16.871 INFO:journalctl@ceph.mon.b.smithi064.stdout: persistent_cache: (nil) 2023-07-18T03:24:16.871 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_size: 4096 2023-07-18T03:24:16.871 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_size_deviation: 10 2023-07-18T03:24:16.871 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_restart_interval: 16 2023-07-18T03:24:16.871 INFO:journalctl@ceph.mon.b.smithi064.stdout: index_block_restart_interval: 1 2023-07-18T03:24:16.871 INFO:journalctl@ceph.mon.b.smithi064.stdout: metadata_block_size: 4096 2023-07-18T03:24:16.871 INFO:journalctl@ceph.mon.b.smithi064.stdout: partition_filters: 0 2023-07-18T03:24:16.871 INFO:journalctl@ceph.mon.b.smithi064.stdout: use_delta_encoding: 1 2023-07-18T03:24:16.872 INFO:journalctl@ceph.mon.b.smithi064.stdout: filter_policy: bloomfilter 2023-07-18T03:24:16.872 INFO:journalctl@ceph.mon.b.smithi064.stdout: whole_key_filtering: 1 2023-07-18T03:24:16.872 INFO:journalctl@ceph.mon.b.smithi064.stdout: verify_compression: 0 2023-07-18T03:24:16.872 INFO:journalctl@ceph.mon.b.smithi064.stdout: read_amp_bytes_per_bit: 0 2023-07-18T03:24:16.872 INFO:journalctl@ceph.mon.b.smithi064.stdout: format_version: 5 2023-07-18T03:24:16.872 INFO:journalctl@ceph.mon.b.smithi064.stdout: enable_index_compression: 1 2023-07-18T03:24:16.872 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_align: 0 2023-07-18T03:24:16.872 INFO:journalctl@ceph.mon.b.smithi064.stdout: max_auto_readahead_size: 262144 2023-07-18T03:24:16.873 INFO:journalctl@ceph.mon.b.smithi064.stdout: prepopulate_block_cache: 0 2023-07-18T03:24:16.873 INFO:journalctl@ceph.mon.b.smithi064.stdout: initial_auto_readahead_size: 8192 2023-07-18T03:24:16.873 INFO:journalctl@ceph.mon.b.smithi064.stdout: num_file_reads_for_auto_readahead: 2 2023-07-18T03:24:16.873 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.write_buffer_size: 33554432 2023-07-18T03:24:16.873 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_write_buffer_number: 2 2023-07-18T03:24:16.873 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compression: NoCompression 2023-07-18T03:24:16.873 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.bottommost_compression: Disabled 2023-07-18T03:24:16.874 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.prefix_extractor: nullptr 2023-07-18T03:24:16.874 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-18T03:24:16.874 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.num_levels: 7 2023-07-18T03:24:16.874 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-18T03:24:16.874 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-18T03:24:16.874 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-18T03:24:16.874 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-18T03:24:16.874 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-18T03:24:16.875 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-18T03:24:16.875 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-18T03:24:16.875 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-18T03:24:16.875 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-18T03:24:16.875 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-18T03:24:16.875 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-18T03:24:16.875 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-18T03:24:16.875 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-18T03:24:16.876 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compression_opts.level: 32767 2023-07-18T03:24:16.876 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compression_opts.strategy: 0 2023-07-18T03:24:16.876 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-18T03:24:16.876 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-18T03:24:16.876 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-18T03:24:16.876 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-18T03:24:16.876 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compression_opts.enabled: false 2023-07-18T03:24:16.877 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-18T03:24:16.877 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-18T03:24:16.877 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-18T03:24:16.877 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-18T03:24:16.877 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.target_file_size_base: 67108864 2023-07-18T03:24:16.877 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-18T03:24:16.877 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-18T03:24:16.877 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-18T03:24:16.878 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-18T03:24:16.878 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-18T03:24:16.878 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-18T03:24:16.878 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-18T03:24:16.878 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-18T03:24:16.878 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-18T03:24:16.878 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-18T03:24:16.878 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-18T03:24:16.879 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-18T03:24:16.879 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-18T03:24:16.879 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-18T03:24:16.879 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.arena_block_size: 1048576 2023-07-18T03:24:16.879 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-18T03:24:16.879 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-18T03:24:16.879 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.disable_auto_compactions: 0 2023-07-18T03:24:16.879 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-18T03:24:16.880 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-18T03:24:16.880 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-18T03:24:16.880 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-18T03:24:16.880 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-18T03:24:16.880 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-18T03:24:16.880 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-18T03:24:16.880 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-18T03:24:16.881 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-18T03:24:16.881 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-18T03:24:16.881 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.table_properties_collectors: 2023-07-18T03:24:16.881 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.inplace_update_support: 0 2023-07-18T03:24:16.881 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-18T03:24:16.881 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-18T03:24:16.881 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-18T03:24:16.881 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-18T03:24:16.882 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.bloom_locality: 0 2023-07-18T03:24:16.882 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.max_successive_merges: 0 2023-07-18T03:24:16.882 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-18T03:24:16.882 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.paranoid_file_checks: 0 2023-07-18T03:24:16.882 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.force_consistency_checks: 1 2023-07-18T03:24:16.882 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.report_bg_io_stats: 0 2023-07-18T03:24:16.882 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.ttl: 2592000 2023-07-18T03:24:16.882 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-18T03:24:16.883 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-18T03:24:16.883 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-18T03:24:16.883 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.enable_blob_files: false 2023-07-18T03:24:16.883 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.min_blob_size: 0 2023-07-18T03:24:16.883 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.blob_file_size: 268435456 2023-07-18T03:24:16.883 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-18T03:24:16.883 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-18T03:24:16.883 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-18T03:24:16.884 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-18T03:24:16.884 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-18T03:24:16.884 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.blob_file_starting_level: 0 2023-07-18T03:24:16.884 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-18T03:24:16.884 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-18T03:24:16.884 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-18T03:24:16.884 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7902afd3-77d0-4f49-a3d6-881dece60f73 2023-07-18T03:24:16.885 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689650656609725, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-18T03:24:16.885 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-18T03:24:16.885 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689650656610134, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689650656, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7902afd3-77d0-4f49-a3d6-881dece60f73", "db_session_id": "OE8UT6YZUA4K6NBDY3FB", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-18T03:24:16.885 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689650656610213, "job": 1, "event": "recovery_finished"} 2023-07-18T03:24:16.885 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-18T03:24:16.885 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-18T03:24:16.885 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5639d489a000 2023-07-18T03:24:16.885 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: DB pointer 0x5639d4886000 2023-07-18T03:24:16.886 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-18T03:24:16.886 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-18T03:24:16.886 INFO:journalctl@ceph.mon.b.smithi064.stdout: ** DB Stats ** 2023-07-18T03:24:16.886 INFO:journalctl@ceph.mon.b.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-18T03:24:16.886 INFO:journalctl@ceph.mon.b.smithi064.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-18T03:24:16.886 INFO:journalctl@ceph.mon.b.smithi064.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-18T03:24:16.886 INFO:journalctl@ceph.mon.b.smithi064.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-18T03:24:16.887 INFO:journalctl@ceph.mon.b.smithi064.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-18T03:24:16.887 INFO:journalctl@ceph.mon.b.smithi064.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-18T03:24:16.887 INFO:journalctl@ceph.mon.b.smithi064.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-18T03:24:16.887 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2023-07-18T03:24:16.887 INFO:journalctl@ceph.mon.b.smithi064.stdout: ** Compaction Stats [default] ** 2023-07-18T03:24:16.887 INFO:journalctl@ceph.mon.b.smithi064.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-18T03:24:16.887 INFO:journalctl@ceph.mon.b.smithi064.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-18T03:24:16.887 INFO:journalctl@ceph.mon.b.smithi064.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-18T03:24:16.888 INFO:journalctl@ceph.mon.b.smithi064.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-18T03:24:16.888 INFO:journalctl@ceph.mon.b.smithi064.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-18T03:24:16.888 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2023-07-18T03:24:16.888 INFO:journalctl@ceph.mon.b.smithi064.stdout: ** Compaction Stats [default] ** 2023-07-18T03:24:16.888 INFO:journalctl@ceph.mon.b.smithi064.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-18T03:24:16.888 INFO:journalctl@ceph.mon.b.smithi064.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-18T03:24:16.888 INFO:journalctl@ceph.mon.b.smithi064.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-18T03:24:16.888 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2023-07-18T03:24:16.889 INFO:journalctl@ceph.mon.b.smithi064.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-18T03:24:16.889 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2023-07-18T03:24:16.889 INFO:journalctl@ceph.mon.b.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-18T03:24:16.889 INFO:journalctl@ceph.mon.b.smithi064.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-18T03:24:16.889 INFO:journalctl@ceph.mon.b.smithi064.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-18T03:24:16.889 INFO:journalctl@ceph.mon.b.smithi064.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-18T03:24:16.889 INFO:journalctl@ceph.mon.b.smithi064.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-18T03:24:16.890 INFO:journalctl@ceph.mon.b.smithi064.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-18T03:24:16.890 INFO:journalctl@ceph.mon.b.smithi064.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-18T03:24:16.890 INFO:journalctl@ceph.mon.b.smithi064.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-18T03:24:16.890 INFO:journalctl@ceph.mon.b.smithi064.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-18T03:24:16.890 INFO:journalctl@ceph.mon.b.smithi064.stdout: Block cache BinnedLRUCache@0x5639d48011f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-07-18T03:24:16.890 INFO:journalctl@ceph.mon.b.smithi064.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-18T03:24:16.890 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2023-07-18T03:24:16.890 INFO:journalctl@ceph.mon.b.smithi064.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-18T03:24:16.891 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-07-18T03:24:16.891 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: using public_addr v2:172.21.15.64:0/0 -> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] 2023-07-18T03:24:16.891 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: starting mon.b rank -1 at public addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] at bind addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:24:16.891 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.b@-1(???) e0 preinit fsid 3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:24:16.891 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.b@-1(synchronizing).mds e1 new map 2023-07-18T03:24:16.891 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.b@-1(synchronizing).mds e1 print_map 2023-07-18T03:24:16.891 INFO:journalctl@ceph.mon.b.smithi064.stdout: e1 2023-07-18T03:24:16.891 INFO:journalctl@ceph.mon.b.smithi064.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-18T03:24:16.892 INFO:journalctl@ceph.mon.b.smithi064.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-18T03:24:16.892 INFO:journalctl@ceph.mon.b.smithi064.stdout: legacy client fscid: -1 2023-07-18T03:24:16.892 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2023-07-18T03:24:16.892 INFO:journalctl@ceph.mon.b.smithi064.stdout: No filesystems configured 2023-07-18T03:24:16.892 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-18T03:24:16.892 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 systemd[1]: Started Ceph mon.b for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:24:16.892 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-18T03:24:16.893 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-18T03:24:16.893 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-18T03:24:16.893 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-18T03:24:16.893 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-18T03:24:16.893 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-18T03:24:16.893 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T03:24:16.893 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T03:24:16.893 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-18T03:24:16.894 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.894 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:16.894 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:24:16.894 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.894 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:16.894 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:24:16.894 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: Deploying cephadm binary to smithi134 2023-07-18T03:24:16.894 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:16.895 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.895 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.895 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.895 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.895 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:24:16.895 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:16.895 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:16.896 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: Updating smithi064:/etc/ceph/ceph.conf 2023-07-18T03:24:16.896 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: Updating smithi064:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:24:16.896 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:24:16.896 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: Updating smithi064:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T03:24:16.896 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:16.896 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.896 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: Added host smithi134 2023-07-18T03:24:16.896 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.897 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.897 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.897 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:16.897 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.897 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.897 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:16.897 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:24:16.897 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.898 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:16.898 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3668808298' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-18T03:24:16.898 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3668808298' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-18T03:24:16.898 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T03:24:16.898 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.898 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.898 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.899 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.899 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:24:16.899 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:16.899 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:16.899 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: Updating smithi134:/etc/ceph/ceph.conf 2023-07-18T03:24:16.899 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: Updating smithi134:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:24:16.899 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:24:16.899 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: Updating smithi134:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-18T03:24:16.900 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.900 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.900 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.900 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:16.900 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi040:172.21.15.40=a;smithi064:172.21.15.64=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:24:16.900 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: Saving service mon spec with placement smithi040:172.21.15.40=a;smithi064:172.21.15.64=b;smithi134:172.21.15.134=c;count:3 2023-07-18T03:24:16.900 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.901 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:16.901 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:16.901 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:16.901 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.901 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T03:24:16.901 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:16.901 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: Deploying daemon mon.c on smithi134 2023-07-18T03:24:16.901 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:16.902 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:16.902 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:24:16.902 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.a calling monitor election 2023-07-18T03:24:16.902 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:16.902 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:16.902 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:16.902 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:16.902 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.c calling monitor election 2023-07-18T03:24:16.903 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:16.903 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:16.903 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:16.903 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:16.903 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-18T03:24:16.903 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: monmap e2: 2 mons at {a=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} 2023-07-18T03:24:16.903 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: fsmap 2023-07-18T03:24:16.904 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T03:24:16.904 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mgrmap e13: x(active, since 41s) 2023-07-18T03:24:16.904 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: overall HEALTH_OK 2023-07-18T03:24:16.904 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.904 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.904 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:16.904 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T03:24:16.904 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:16.905 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: Deploying daemon mon.b on smithi064 2023-07-18T03:24:16.905 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:16.905 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:16.905 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: from='client.? 172.21.15.134:0/1894313842' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-18T03:24:16.905 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:16.905 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-18T03:24:16.905 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:16 smithi064 ceph-mon[98059]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-07-18T03:24:16.906 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:24:22.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:24:22.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: mon.a calling monitor election 2023-07-18T03:24:22.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: mon.c calling monitor election 2023-07-18T03:24:22.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:22.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:22.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:22.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:22.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:22.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:22.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:22.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:22.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:22.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-18T03:24:22.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: monmap e3: 3 mons at {a=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0],b=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} 2023-07-18T03:24:22.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: fsmap 2023-07-18T03:24:22.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T03:24:22.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: mgrmap e13: x(active, since 49s) 2023-07-18T03:24:22.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-18T03:24:22.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-18T03:24:22.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-18T03:24:22.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: mon.b (rank 2) addr [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] is down (out of quorum) 2023-07-18T03:24:22.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:22.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:22.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:22.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:22.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:24:22.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: mon.a calling monitor election 2023-07-18T03:24:22.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: mon.c calling monitor election 2023-07-18T03:24:22.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:22.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:22.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:22.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:22.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:22.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:22.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:22.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:22.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:22.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-18T03:24:22.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: monmap e3: 3 mons at {a=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0],b=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} 2023-07-18T03:24:22.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: fsmap 2023-07-18T03:24:22.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T03:24:22.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: mgrmap e13: x(active, since 49s) 2023-07-18T03:24:22.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-18T03:24:22.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-18T03:24:22.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-18T03:24:22.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: mon.b (rank 2) addr [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] is down (out of quorum) 2023-07-18T03:24:22.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:22.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:22.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:22.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:22.354 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.357+0000 7f9a7c053700 1 -- 172.21.15.134:0/820263769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a74104650 msgr2=0x7f9a60007820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:22.355 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.357+0000 7f9a7c053700 1 --2- 172.21.15.134:0/820263769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a74104650 0x7f9a60007820 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9a64005fa0 tx=0x7f9a6400ff70 comp rx=0 tx=0).stop 2023-07-18T03:24:22.355 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.357+0000 7f9a7c053700 1 -- 172.21.15.134:0/820263769 shutdown_connections 2023-07-18T03:24:22.355 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.357+0000 7f9a7c053700 1 --2- 172.21.15.134:0/820263769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a74104650 0x7f9a60007820 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:22.355 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.357+0000 7f9a7c053700 1 -- 172.21.15.134:0/820263769 >> 172.21.15.134:0/820263769 conn(0x7f9a740fa130 msgr2=0x7f9a740fc540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:22.355 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.358+0000 7f9a7c053700 1 -- 172.21.15.134:0/820263769 shutdown_connections 2023-07-18T03:24:22.355 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.358+0000 7f9a7c053700 1 -- 172.21.15.134:0/820263769 wait complete. 2023-07-18T03:24:22.358 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.359+0000 7f9a7c053700 1 Processor -- start 2023-07-18T03:24:22.358 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.359+0000 7f9a7c053700 1 -- start start 2023-07-18T03:24:22.358 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.359+0000 7f9a7c053700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a74104650 0x7f9a74198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:22.358 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.359+0000 7f9a7c053700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a74198c50 0x7f9a74192790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:22.358 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.359+0000 7f9a7c053700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a74192cd0 0x7f9a74193140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:22.359 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.359+0000 7f9a7c053700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a741992b0 con 0x7f9a74192cd0 2023-07-18T03:24:22.359 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.359+0000 7f9a7c053700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7419f530 con 0x7f9a74198c50 2023-07-18T03:24:22.359 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.359+0000 7f9a7c053700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7419f670 con 0x7f9a74104650 2023-07-18T03:24:22.359 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a79def700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a74104650 0x7f9a74198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:22.359 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a79def700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a74104650 0x7f9a74198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:47986/0 (socket says 172.21.15.134:47986) 2023-07-18T03:24:22.359 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a79def700 1 -- 172.21.15.134:0/1989698106 learned_addr learned my addr 172.21.15.134:0/1989698106 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-07-18T03:24:22.359 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a79def700 1 -- 172.21.15.134:0/1989698106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a74104650 msgr2=0x7f9a74198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-07-18T03:24:22.360 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a79def700 1 -- 172.21.15.134:0/1989698106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a74104650 msgr2=0x7f9a74198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-18T03:24:22.360 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a79def700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a74104650 0x7f9a74198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T03:24:22.360 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a795ee700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a74198c50 0x7f9a74192790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:22.360 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a79def700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a74104650 0x7f9a74198710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T03:24:22.360 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a7a5f0700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a74192cd0 0x7f9a74193140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:22.360 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a795ee700 1 -- 172.21.15.134:0/1989698106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a74198c50 msgr2=0x7f9a74192790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-07-18T03:24:22.360 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a795ee700 1 -- 172.21.15.134:0/1989698106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a74198c50 msgr2=0x7f9a74192790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-18T03:24:22.361 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a795ee700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a74198c50 0x7f9a74192790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-18T03:24:22.361 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a795ee700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a74198c50 0x7f9a74192790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-18T03:24:22.361 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a7a5f0700 1 -- 172.21.15.134:0/1989698106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a74104650 msgr2=0x7f9a74198710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:24:22.361 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a7a5f0700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a74104650 0x7f9a74198710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:22.361 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a7a5f0700 1 -- 172.21.15.134:0/1989698106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a74198c50 msgr2=0x7f9a74192790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:24:22.361 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a7a5f0700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a74198c50 0x7f9a74192790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:22.362 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a7a5f0700 1 -- 172.21.15.134:0/1989698106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6400f970 con 0x7f9a74192cd0 2023-07-18T03:24:22.362 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.360+0000 7f9a7a5f0700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a74192cd0 0x7f9a74193140 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00bd50 tx=0x7f9a6c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:22.362 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.361+0000 7f9a6affd700 1 -- 172.21.15.134:0/1989698106 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c00ccb0 con 0x7f9a74192cd0 2023-07-18T03:24:22.362 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.361+0000 7f9a6affd700 1 -- 172.21.15.134:0/1989698106 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a6c00ce10 con 0x7f9a74192cd0 2023-07-18T03:24:22.362 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.361+0000 7f9a7c053700 1 -- 172.21.15.134:0/1989698106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7419f950 con 0x7f9a74192cd0 2023-07-18T03:24:22.362 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.361+0000 7f9a6affd700 1 -- 172.21.15.134:0/1989698106 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0127e0 con 0x7f9a74192cd0 2023-07-18T03:24:22.363 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.361+0000 7f9a7c053700 1 -- 172.21.15.134:0/1989698106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7419fed0 con 0x7f9a74192cd0 2023-07-18T03:24:22.364 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.362+0000 7f9a6affd700 1 -- 172.21.15.134:0/1989698106 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f9a6c014a70 con 0x7f9a74192cd0 2023-07-18T03:24:22.364 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.362+0000 7f9a7c053700 1 -- 172.21.15.134:0/1989698106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a74061b40 con 0x7f9a74192cd0 2023-07-18T03:24:22.364 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.362+0000 7f9a6affd700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9a58038c80 0x7f9a5803b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:22.364 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.362+0000 7f9a6affd700 1 -- 172.21.15.134:0/1989698106 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9a6c04ae70 con 0x7f9a74192cd0 2023-07-18T03:24:22.364 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.363+0000 7f9a79def700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9a58038c80 0x7f9a5803b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:22.364 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.363+0000 7f9a79def700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9a58038c80 0x7f9a5803b130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9a640127e0 tx=0x7f9a64005690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:22.365 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.368+0000 7f9a6affd700 1 -- 172.21.15.134:0/1989698106 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9a6c01b710 con 0x7f9a74192cd0 2023-07-18T03:24:22.638 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.641+0000 7f9a7c053700 1 -- 172.21.15.134:0/1989698106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9a74105490 con 0x7f9a74192cd0 2023-07-18T03:24:22.638 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.642+0000 7f9a6affd700 1 -- 172.21.15.134:0/1989698106 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1270 (secure 0 0 0) 0x7f9a6c019030 con 0x7f9a74192cd0 2023-07-18T03:24:22.639 INFO:teuthology.orchestra.run.smithi134.stdout: 2023-07-18T03:24:22.639 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":3,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","modified":"2023-07-18T03:24:16.638173Z","created":"2023-07-18T03:22:28.250962Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-18T03:24:22.648 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.644+0000 7f9a7c053700 1 -- 172.21.15.134:0/1989698106 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9a58038c80 msgr2=0x7f9a5803b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:22.648 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.644+0000 7f9a7c053700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9a58038c80 0x7f9a5803b130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9a640127e0 tx=0x7f9a64005690 comp rx=0 tx=0).stop 2023-07-18T03:24:22.648 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.644+0000 7f9a7c053700 1 -- 172.21.15.134:0/1989698106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a74192cd0 msgr2=0x7f9a74193140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:22.648 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.644+0000 7f9a7c053700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a74192cd0 0x7f9a74193140 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00bd50 tx=0x7f9a6c00be30 comp rx=0 tx=0).stop 2023-07-18T03:24:22.648 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.644+0000 7f9a7c053700 1 -- 172.21.15.134:0/1989698106 shutdown_connections 2023-07-18T03:24:22.648 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.644+0000 7f9a7c053700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a74192cd0 0x7f9a74193140 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:22.649 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.644+0000 7f9a7c053700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9a58038c80 0x7f9a5803b130 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:22.649 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.644+0000 7f9a7c053700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a74104650 0x7f9a74198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:22.649 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.644+0000 7f9a7c053700 1 --2- 172.21.15.134:0/1989698106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a74198c50 0x7f9a74192790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:22.649 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.644+0000 7f9a7c053700 1 -- 172.21.15.134:0/1989698106 >> 172.21.15.134:0/1989698106 conn(0x7f9a740fa130 msgr2=0x7f9a7418b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:22.649 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.644+0000 7f9a7c053700 1 -- 172.21.15.134:0/1989698106 shutdown_connections 2023-07-18T03:24:22.649 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:22.644+0000 7f9a7c053700 1 -- 172.21.15.134:0/1989698106 wait complete. 2023-07-18T03:24:22.649 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 3 2023-07-18T03:24:22.936 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:22 smithi134 ceph-mon[98216]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:22.936 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:22 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:22.936 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:22 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:22.937 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:22 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:22.937 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:22 smithi134 ceph-mon[98216]: from='client.? 172.21.15.134:0/1989698106' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-18T03:24:22.949 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-07-18T03:24:22.949 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph config generate-minimal-conf 2023-07-18T03:24:22.982 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:22 smithi040 ceph-mon[96439]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:22.982 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:22 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:22.983 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:22 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:22.983 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:22 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:22.983 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:22 smithi040 ceph-mon[96439]: from='client.? 172.21.15.134:0/1989698106' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-18T03:24:23.527 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:24:24.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:24:24.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: mon.a calling monitor election 2023-07-18T03:24:24.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: mon.c calling monitor election 2023-07-18T03:24:24.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:24.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:24:24.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:24.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:24.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:24.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-18T03:24:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: monmap e3: 3 mons at {a=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0],b=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} 2023-07-18T03:24:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: fsmap 2023-07-18T03:24:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T03:24:24.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: mgrmap e13: x(active, since 49s) 2023-07-18T03:24:24.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-18T03:24:24.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-18T03:24:24.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-18T03:24:24.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: mon.b (rank 2) addr [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] is down (out of quorum) 2023-07-18T03:24:24.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:24.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:24.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:24.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:23 smithi064 ceph-mon[98059]: from='client.? 172.21.15.134:0/1989698106' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-18T03:24:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.612+0000 7f9858776700 1 -- 172.21.15.40:0/2810664462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9850105630 msgr2=0x7f9850105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.612+0000 7f9858776700 1 --2- 172.21.15.40:0/2810664462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9850105630 0x7f9850105a00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9840009910 tx=0x7f9840009c20 comp rx=0 tx=0).stop 2023-07-18T03:24:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.612+0000 7f9858776700 1 -- 172.21.15.40:0/2810664462 shutdown_connections 2023-07-18T03:24:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.612+0000 7f9858776700 1 --2- 172.21.15.40:0/2810664462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9850105630 0x7f9850105a00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.612+0000 7f9858776700 1 -- 172.21.15.40:0/2810664462 >> 172.21.15.40:0/2810664462 conn(0x7f98500fb130 msgr2=0x7f98500fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.612+0000 7f9858776700 1 -- 172.21.15.40:0/2810664462 shutdown_connections 2023-07-18T03:24:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.612+0000 7f9858776700 1 -- 172.21.15.40:0/2810664462 wait complete. 2023-07-18T03:24:24.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.613+0000 7f9858776700 1 Processor -- start 2023-07-18T03:24:24.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.613+0000 7f9858776700 1 -- start start 2023-07-18T03:24:24.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.613+0000 7f9858776700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9850105630 0x7f985019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:24.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.613+0000 7f9858776700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985019ae30 0x7f98501949e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:24.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.613+0000 7f9858776700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850194f20 0x7f98501953c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:24.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.613+0000 7f9858776700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985019b440 con 0x7f985019ae30 2023-07-18T03:24:24.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.613+0000 7f9858776700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98501a1710 con 0x7f9850105630 2023-07-18T03:24:24.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.613+0000 7f9858776700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98501a1850 con 0x7f9850194f20 2023-07-18T03:24:24.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.613+0000 7f9855d11700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985019ae30 0x7f98501949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:24.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.613+0000 7f9855d11700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985019ae30 0x7f98501949e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55192/0 (socket says 172.21.15.40:55192) 2023-07-18T03:24:24.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.613+0000 7f9855d11700 1 -- 172.21.15.40:0/3941892000 learned_addr learned my addr 172.21.15.40:0/3941892000 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:24:24.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.613+0000 7f9856512700 1 --2- 172.21.15.40:0/3941892000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9850105630 0x7f985019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:24.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.614+0000 7f9856d13700 1 --2- 172.21.15.40:0/3941892000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850194f20 0x7f98501953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.614+0000 7f9855d11700 1 -- 172.21.15.40:0/3941892000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850194f20 msgr2=0x7f98501953c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.614+0000 7f9855d11700 1 --2- 172.21.15.40:0/3941892000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850194f20 0x7f98501953c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.614+0000 7f9855d11700 1 -- 172.21.15.40:0/3941892000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9850105630 msgr2=0x7f985019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.614+0000 7f9855d11700 1 --2- 172.21.15.40:0/3941892000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9850105630 0x7f985019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.614+0000 7f9855d11700 1 -- 172.21.15.40:0/3941892000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98400095f0 con 0x7f985019ae30 2023-07-18T03:24:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.614+0000 7f9856d13700 1 --2- 172.21.15.40:0/3941892000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850194f20 0x7f98501953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:24:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.614+0000 7f9855d11700 1 --2- 172.21.15.40:0/3941892000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985019ae30 0x7f98501949e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f984c00b780 tx=0x7f984c00ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:24.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.614+0000 7f98477fe700 1 -- 172.21.15.40:0/3941892000 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c004230 con 0x7f985019ae30 2023-07-18T03:24:24.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.614+0000 7f98477fe700 1 -- 172.21.15.40:0/3941892000 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f984c004390 con 0x7f985019ae30 2023-07-18T03:24:24.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.614+0000 7f9858776700 1 -- 172.21.15.40:0/3941892000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98501a1b30 con 0x7f985019ae30 2023-07-18T03:24:24.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.614+0000 7f98477fe700 1 -- 172.21.15.40:0/3941892000 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c011ad0 con 0x7f985019ae30 2023-07-18T03:24:24.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.614+0000 7f9858776700 1 -- 172.21.15.40:0/3941892000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98501a2080 con 0x7f985019ae30 2023-07-18T03:24:24.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.615+0000 7f98477fe700 1 -- 172.21.15.40:0/3941892000 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f984c011c30 con 0x7f985019ae30 2023-07-18T03:24:24.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.615+0000 7f9858776700 1 -- 172.21.15.40:0/3941892000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9850061b40 con 0x7f985019ae30 2023-07-18T03:24:24.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.615+0000 7f98477fe700 1 --2- 172.21.15.40:0/3941892000 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f983c038c80 0x7f983c03b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:24.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.615+0000 7f98477fe700 1 -- 172.21.15.40:0/3941892000 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f984c04aef0 con 0x7f985019ae30 2023-07-18T03:24:24.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.615+0000 7f9856512700 1 --2- 172.21.15.40:0/3941892000 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f983c038c80 0x7f983c03b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:24.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.616+0000 7f9856512700 1 --2- 172.21.15.40:0/3941892000 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f983c038c80 0x7f983c03b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9840009fd0 tx=0x7f98400058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:24.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.620+0000 7f98477fe700 1 -- 172.21.15.40:0/3941892000 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f984c01a400 con 0x7f985019ae30 2023-07-18T03:24:24.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.819+0000 7f9858776700 1 -- 172.21.15.40:0/3941892000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f9850195ef0 con 0x7f985019ae30 2023-07-18T03:24:24.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.819+0000 7f98477fe700 1 -- 172.21.15.40:0/3941892000 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f984c004570 con 0x7f985019ae30 2023-07-18T03:24:24.816 INFO:teuthology.orchestra.run.smithi040.stdout:# minimal ceph.conf for 3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:24:24.816 INFO:teuthology.orchestra.run.smithi040.stdout:[global] 2023-07-18T03:24:24.816 INFO:teuthology.orchestra.run.smithi040.stdout: fsid = 3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:24:24.816 INFO:teuthology.orchestra.run.smithi040.stdout: mon_host = [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2023-07-18T03:24:24.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: mon.b calling monitor election 2023-07-18T03:24:24.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: Updating smithi040:/etc/ceph/ceph.conf 2023-07-18T03:24:24.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: Updating smithi064:/etc/ceph/ceph.conf 2023-07-18T03:24:24.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: Updating smithi134:/etc/ceph/ceph.conf 2023-07-18T03:24:24.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: Updating smithi040:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:24:24.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: mon.b calling monitor election 2023-07-18T03:24:24.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: mon.a calling monitor election 2023-07-18T03:24:24.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: mon.c calling monitor election 2023-07-18T03:24:24.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-18T03:24:24.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: monmap e3: 3 mons at {a=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0],b=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} 2023-07-18T03:24:24.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: Updating smithi064:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:24:24.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: Updating smithi134:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:24:24.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: fsmap 2023-07-18T03:24:24.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T03:24:24.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: mgrmap e13: x(active, since 51s) 2023-07-18T03:24:24.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-18T03:24:24.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: Cluster is now healthy 2023-07-18T03:24:24.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: overall HEALTH_OK 2023-07-18T03:24:24.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:24.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.820 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.820 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.820 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.820 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.820 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.820 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T03:24:24.820 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T03:24:24.821 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:24.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.821+0000 7f9858776700 1 -- 172.21.15.40:0/3941892000 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f983c038c80 msgr2=0x7f983c03b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:24.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.821+0000 7f9858776700 1 --2- 172.21.15.40:0/3941892000 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f983c038c80 0x7f983c03b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9840009fd0 tx=0x7f98400058e0 comp rx=0 tx=0).stop 2023-07-18T03:24:24.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.821+0000 7f9858776700 1 -- 172.21.15.40:0/3941892000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985019ae30 msgr2=0x7f98501949e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.821+0000 7f9858776700 1 --2- 172.21.15.40:0/3941892000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985019ae30 0x7f98501949e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f984c00b780 tx=0x7f984c00ba90 comp rx=0 tx=0).stop 2023-07-18T03:24:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.821+0000 7f9858776700 1 -- 172.21.15.40:0/3941892000 shutdown_connections 2023-07-18T03:24:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.821+0000 7f9858776700 1 --2- 172.21.15.40:0/3941892000 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f983c038c80 0x7f983c03b130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.821+0000 7f9858776700 1 --2- 172.21.15.40:0/3941892000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850194f20 0x7f98501953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.821+0000 7f9858776700 1 --2- 172.21.15.40:0/3941892000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9850105630 0x7f985019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.821+0000 7f9858776700 1 --2- 172.21.15.40:0/3941892000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985019ae30 0x7f98501949e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:24.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.822+0000 7f9858776700 1 -- 172.21.15.40:0/3941892000 >> 172.21.15.40:0/3941892000 conn(0x7f98500fb130 msgr2=0x7f98500fb500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:24.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.822+0000 7f9858776700 1 -- 172.21.15.40:0/3941892000 shutdown_connections 2023-07-18T03:24:24.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:24.822+0000 7f9858776700 1 -- 172.21.15.40:0/3941892000 wait complete. 2023-07-18T03:24:24.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: mon.b calling monitor election 2023-07-18T03:24:24.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: Updating smithi040:/etc/ceph/ceph.conf 2023-07-18T03:24:24.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: Updating smithi064:/etc/ceph/ceph.conf 2023-07-18T03:24:24.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: Updating smithi134:/etc/ceph/ceph.conf 2023-07-18T03:24:24.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: Updating smithi040:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:24:24.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: mon.b calling monitor election 2023-07-18T03:24:24.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: mon.a calling monitor election 2023-07-18T03:24:24.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: mon.c calling monitor election 2023-07-18T03:24:24.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-18T03:24:24.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: monmap e3: 3 mons at {a=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0],b=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} 2023-07-18T03:24:24.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: Updating smithi064:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:24:24.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: Updating smithi134:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:24:24.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: fsmap 2023-07-18T03:24:24.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T03:24:24.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: mgrmap e13: x(active, since 51s) 2023-07-18T03:24:24.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-18T03:24:24.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: Cluster is now healthy 2023-07-18T03:24:24.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: overall HEALTH_OK 2023-07-18T03:24:24.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:24.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T03:24:24.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T03:24:24.833 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:24.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: mon.b calling monitor election 2023-07-18T03:24:24.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: Updating smithi040:/etc/ceph/ceph.conf 2023-07-18T03:24:24.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: Updating smithi064:/etc/ceph/ceph.conf 2023-07-18T03:24:24.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: Updating smithi134:/etc/ceph/ceph.conf 2023-07-18T03:24:24.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: Updating smithi040:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:24:24.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: mon.b calling monitor election 2023-07-18T03:24:24.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: mon.a calling monitor election 2023-07-18T03:24:24.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: mon.c calling monitor election 2023-07-18T03:24:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-18T03:24:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: monmap e3: 3 mons at {a=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0],b=[v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0],c=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} 2023-07-18T03:24:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: Updating smithi064:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:24:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: Updating smithi134:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-18T03:24:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: fsmap 2023-07-18T03:24:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: osdmap e4: 0 total, 0 up, 0 in 2023-07-18T03:24:24.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: mgrmap e13: x(active, since 51s) 2023-07-18T03:24:24.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-18T03:24:24.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: Cluster is now healthy 2023-07-18T03:24:24.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: overall HEALTH_OK 2023-07-18T03:24:24.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:24.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:24.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T03:24:24.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T03:24:24.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:25.188 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-07-18T03:24:25.189 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:24:25.189 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.conf 2023-07-18T03:24:25.260 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:24:25.261 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:24:25.328 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-07-18T03:24:25.328 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.conf 2023-07-18T03:24:25.365 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-07-18T03:24:25.366 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:24:25.438 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-18T03:24:25.438 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.conf 2023-07-18T03:24:25.475 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-18T03:24:25.475 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-18T03:24:25.546 INFO:tasks.cephadm:Adding mgr.x on smithi040 2023-07-18T03:24:25.547 INFO:tasks.cephadm:Adding mgr.y on smithi064 2023-07-18T03:24:25.548 INFO:tasks.cephadm:Adding mgr.z on smithi134 2023-07-18T03:24:25.548 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch apply mgr '3;smithi040=x;smithi064=y;smithi134=z' 2023-07-18T03:24:25.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:25 smithi134 ceph-mon[98216]: Reconfiguring mon.a (unknown last config time)... 2023-07-18T03:24:25.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:25 smithi134 ceph-mon[98216]: Reconfiguring daemon mon.a on smithi040 2023-07-18T03:24:25.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:25 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:25.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:25 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3941892000' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:25.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:25 smithi040 ceph-mon[96439]: Reconfiguring mon.a (unknown last config time)... 2023-07-18T03:24:25.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:25 smithi040 ceph-mon[96439]: Reconfiguring daemon mon.a on smithi040 2023-07-18T03:24:25.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:25 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:25.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:25 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3941892000' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:25.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:25 smithi064 ceph-mon[98059]: Reconfiguring mon.a (unknown last config time)... 2023-07-18T03:24:25.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:25 smithi064 ceph-mon[98059]: Reconfiguring daemon mon.a on smithi040 2023-07-18T03:24:25.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:25 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:24:25.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:25 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3941892000' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:26.139 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:24:26.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:26 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:26.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:26 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:26.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:26 smithi134 ceph-mon[98216]: Reconfiguring mon.b (monmap changed)... 2023-07-18T03:24:26.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:26 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T03:24:26.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:26 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T03:24:26.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:26 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:26.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:26 smithi134 ceph-mon[98216]: Reconfiguring daemon mon.b on smithi064 2023-07-18T03:24:26.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:26 smithi134 ceph-mon[98216]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:26.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:26 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:26.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:26 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:26.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:26 smithi064 ceph-mon[98059]: Reconfiguring mon.b (monmap changed)... 2023-07-18T03:24:26.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:26 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T03:24:26.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:26 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T03:24:26.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:26 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:26.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:26 smithi064 ceph-mon[98059]: Reconfiguring daemon mon.b on smithi064 2023-07-18T03:24:26.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:26 smithi064 ceph-mon[98059]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:26.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:26 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:26.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:26 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:26.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:26 smithi040 ceph-mon[96439]: Reconfiguring mon.b (monmap changed)... 2023-07-18T03:24:26.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:26 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T03:24:26.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:26 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T03:24:26.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:26 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:26.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:26 smithi040 ceph-mon[96439]: Reconfiguring daemon mon.b on smithi064 2023-07-18T03:24:26.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:26 smithi040 ceph-mon[96439]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:27.457 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.460+0000 7fe7c2dc7700 1 -- 172.21.15.134:0/4035884421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7bc0ff700 msgr2=0x7fe7bc0ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:27.458 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.460+0000 7fe7c2dc7700 1 --2- 172.21.15.134:0/4035884421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7bc0ff700 0x7fe7bc0ffb10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009910 tx=0x7fe7b0009c20 comp rx=0 tx=0).stop 2023-07-18T03:24:27.458 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.460+0000 7fe7c2dc7700 1 -- 172.21.15.134:0/4035884421 shutdown_connections 2023-07-18T03:24:27.458 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.460+0000 7fe7c2dc7700 1 --2- 172.21.15.134:0/4035884421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7bc0ff700 0x7fe7bc0ffb10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:27.458 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.460+0000 7fe7c2dc7700 1 -- 172.21.15.134:0/4035884421 >> 172.21.15.134:0/4035884421 conn(0x7fe7bc0facd0 msgr2=0x7fe7bc0fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:27.458 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.460+0000 7fe7c2dc7700 1 -- 172.21.15.134:0/4035884421 shutdown_connections 2023-07-18T03:24:27.458 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.460+0000 7fe7c2dc7700 1 -- 172.21.15.134:0/4035884421 wait complete. 2023-07-18T03:24:27.459 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.461+0000 7fe7c2dc7700 1 Processor -- start 2023-07-18T03:24:27.459 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.461+0000 7fe7c2dc7700 1 -- start start 2023-07-18T03:24:27.460 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.461+0000 7fe7c2dc7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7bc0ff700 0x7fe7bc0742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:27.461 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.461+0000 7fe7c2dc7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7bc072900 0x7fe7bc072d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:27.461 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.461+0000 7fe7c2dc7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7bc0732b0 0x7fe7bc19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:27.461 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.461+0000 7fe7c2dc7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc19ef80 con 0x7fe7bc0732b0 2023-07-18T03:24:27.461 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.461+0000 7fe7c2dc7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc19f0c0 con 0x7fe7bc0ff700 2023-07-18T03:24:27.461 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.461+0000 7fe7c2dc7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc19f200 con 0x7fe7bc072900 2023-07-18T03:24:27.461 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.461+0000 7fe7bbfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7bc072900 0x7fe7bc072d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:27.462 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.461+0000 7fe7bbfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7bc072900 0x7fe7bc072d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:47998/0 (socket says 172.21.15.134:47998) 2023-07-18T03:24:27.462 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.461+0000 7fe7bbfff700 1 -- 172.21.15.134:0/438851731 learned_addr learned my addr 172.21.15.134:0/438851731 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-07-18T03:24:27.462 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.461+0000 7fe7c1364700 1 --2- 172.21.15.134:0/438851731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7bc0732b0 0x7fe7bc19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:27.462 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.462+0000 7fe7c0b63700 1 --2- 172.21.15.134:0/438851731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7bc0ff700 0x7fe7bc0742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:27.462 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.462+0000 7fe7bbfff700 1 -- 172.21.15.134:0/438851731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7bc0ff700 msgr2=0x7fe7bc0742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:27.462 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.462+0000 7fe7bbfff700 1 --2- 172.21.15.134:0/438851731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7bc0ff700 0x7fe7bc0742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:27.462 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.462+0000 7fe7bbfff700 1 -- 172.21.15.134:0/438851731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7bc0732b0 msgr2=0x7fe7bc19ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:27.462 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.462+0000 7fe7bbfff700 1 --2- 172.21.15.134:0/438851731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7bc0732b0 0x7fe7bc19ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:27.463 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.462+0000 7fe7bbfff700 1 -- 172.21.15.134:0/438851731 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b00095f0 con 0x7fe7bc072900 2023-07-18T03:24:27.463 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.462+0000 7fe7c0b63700 1 --2- 172.21.15.134:0/438851731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7bc0ff700 0x7fe7bc0742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:24:27.463 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.462+0000 7fe7c1364700 1 --2- 172.21.15.134:0/438851731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7bc0732b0 0x7fe7bc19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:24:27.463 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.462+0000 7fe7bbfff700 1 --2- 172.21.15.134:0/438851731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7bc072900 0x7fe7bc072d70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800b780 tx=0x7fe7a800ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:27.463 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.462+0000 7fe7b9ffb700 1 -- 172.21.15.134:0/438851731 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8004230 con 0x7fe7bc072900 2023-07-18T03:24:27.463 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.462+0000 7fe7c2dc7700 1 -- 172.21.15.134:0/438851731 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7bc19f4e0 con 0x7fe7bc072900 2023-07-18T03:24:27.463 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.462+0000 7fe7b9ffb700 1 -- 172.21.15.134:0/438851731 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7a8004390 con 0x7fe7bc072900 2023-07-18T03:24:27.464 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.462+0000 7fe7b9ffb700 1 -- 172.21.15.134:0/438851731 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8011ad0 con 0x7fe7bc072900 2023-07-18T03:24:27.464 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.462+0000 7fe7c2dc7700 1 -- 172.21.15.134:0/438851731 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc19fab0 con 0x7fe7bc072900 2023-07-18T03:24:27.464 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.463+0000 7fe7b9ffb700 1 -- 172.21.15.134:0/438851731 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fe7a8011cf0 con 0x7fe7bc072900 2023-07-18T03:24:27.464 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.463+0000 7fe7b9ffb700 1 --2- 172.21.15.134:0/438851731 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe7ac038c30 0x7fe7ac03b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:27.464 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.463+0000 7fe7c2dc7700 1 -- 172.21.15.134:0/438851731 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7bc18e280 con 0x7fe7bc072900 2023-07-18T03:24:27.464 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.464+0000 7fe7b9ffb700 1 -- 172.21.15.134:0/438851731 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe7a804af70 con 0x7fe7bc072900 2023-07-18T03:24:27.464 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.464+0000 7fe7c0b63700 1 --2- 172.21.15.134:0/438851731 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe7ac038c30 0x7fe7ac03b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:27.465 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.464+0000 7fe7c0b63700 1 --2- 172.21.15.134:0/438851731 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe7ac038c30 0x7fe7ac03b0e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0005130 tx=0x7fe7b000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:27.465 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.469+0000 7fe7b9ffb700 1 -- 172.21.15.134:0/438851731 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe7a801a400 con 0x7fe7bc072900 2023-07-18T03:24:27.672 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.674+0000 7fe7c2dc7700 1 -- 172.21.15.134:0/438851731 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi040=x;smithi064=y;smithi134=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7bc0609e0 con 0x7fe7ac038c30 2023-07-18T03:24:27.679 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.682+0000 7fe7b9ffb700 1 -- 172.21.15.134:0/438851731 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe7bc0609e0 con 0x7fe7ac038c30 2023-07-18T03:24:27.679 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mgr update... 2023-07-18T03:24:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.684+0000 7fe7c2dc7700 1 -- 172.21.15.134:0/438851731 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe7ac038c30 msgr2=0x7fe7ac03b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.684+0000 7fe7c2dc7700 1 --2- 172.21.15.134:0/438851731 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe7ac038c30 0x7fe7ac03b0e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0005130 tx=0x7fe7b000b540 comp rx=0 tx=0).stop 2023-07-18T03:24:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.684+0000 7fe7c2dc7700 1 -- 172.21.15.134:0/438851731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7bc072900 msgr2=0x7fe7bc072d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:27.681 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.684+0000 7fe7c2dc7700 1 --2- 172.21.15.134:0/438851731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7bc072900 0x7fe7bc072d70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800b780 tx=0x7fe7a800ba90 comp rx=0 tx=0).stop 2023-07-18T03:24:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.684+0000 7fe7c2dc7700 1 -- 172.21.15.134:0/438851731 shutdown_connections 2023-07-18T03:24:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.684+0000 7fe7c2dc7700 1 --2- 172.21.15.134:0/438851731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7bc0732b0 0x7fe7bc19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.684+0000 7fe7c2dc7700 1 --2- 172.21.15.134:0/438851731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe7bc0ff700 0x7fe7bc0742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.684+0000 7fe7c2dc7700 1 --2- 172.21.15.134:0/438851731 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe7ac038c30 0x7fe7ac03b0e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.684+0000 7fe7c2dc7700 1 --2- 172.21.15.134:0/438851731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7bc072900 0x7fe7bc072d70 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.684+0000 7fe7c2dc7700 1 -- 172.21.15.134:0/438851731 >> 172.21.15.134:0/438851731 conn(0x7fe7bc0facd0 msgr2=0x7fe7bc0fb950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.685+0000 7fe7c2dc7700 1 -- 172.21.15.134:0/438851731 shutdown_connections 2023-07-18T03:24:27.682 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:24:27.685+0000 7fe7c2dc7700 1 -- 172.21.15.134:0/438851731 wait complete. 2023-07-18T03:24:27.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:27 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:27.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:27 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:27.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:27 smithi134 ceph-mon[98216]: Reconfiguring mon.c (monmap changed)... 2023-07-18T03:24:27.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:27 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T03:24:27.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:27 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T03:24:27.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:27 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:27.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:27 smithi134 ceph-mon[98216]: Reconfiguring daemon mon.c on smithi134 2023-07-18T03:24:28.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:27 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:28.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:27 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:28.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:27 smithi040 ceph-mon[96439]: Reconfiguring mon.c (monmap changed)... 2023-07-18T03:24:28.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:27 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T03:24:28.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:27 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T03:24:28.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:27 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:28.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:27 smithi040 ceph-mon[96439]: Reconfiguring daemon mon.c on smithi134 2023-07-18T03:24:28.097 DEBUG:teuthology.orchestra.run.smithi064:mgr.y> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mgr.y.service 2023-07-18T03:24:28.100 DEBUG:teuthology.orchestra.run.smithi134:mgr.z> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mgr.z.service 2023-07-18T03:24:28.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:27 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:28.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:27 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:28.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:27 smithi064 ceph-mon[98059]: Reconfiguring mon.c (monmap changed)... 2023-07-18T03:24:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:27 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-18T03:24:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:27 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-18T03:24:28.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:27 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:28.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:27 smithi064 ceph-mon[98059]: Reconfiguring daemon mon.c on smithi134 2023-07-18T03:24:28.106 INFO:tasks.cephadm:Deploying OSDs... 2023-07-18T03:24:28.106 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:24:28.107 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2023-07-18T03:24:28.130 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-18T03:24:28.130 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2023-07-18T03:24:28.134 INFO:journalctl@ceph.mgr.z.smithi134.stdout:-- Logs begin at Tue 2023-07-18 03:07:14 UTC. -- 2023-07-18T03:24:28.138 INFO:journalctl@ceph.mgr.y.smithi064.stdout:-- Logs begin at Tue 2023-07-18 03:07:19 UTC. -- 2023-07-18T03:24:28.190 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-18T03:24:28.190 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:24:28.191 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 6h/6d Inode: 371687 Links: 1 2023-07-18T03:24:28.191 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:24:28.191 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:24:28.191 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-07-18 03:23:42.553698473 +0000 2023-07-18T03:24:28.191 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-07-18 03:18:07.797837936 +0000 2023-07-18T03:24:28.192 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-07-18 03:18:07.797837936 +0000 2023-07-18T03:24:28.192 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2023-07-18T03:24:28.192 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-18T03:24:28.260 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-07-18T03:24:28.261 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-07-18T03:24:28.261 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000405208 s, 1.3 MB/s 2023-07-18T03:24:28.262 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-18T03:24:28.322 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2023-07-18T03:24:28.381 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-18T03:24:28.382 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:24:28.382 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 6h/6d Inode: 381024 Links: 1 2023-07-18T03:24:28.382 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:24:28.382 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:24:28.382 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-07-18 03:23:42.553698473 +0000 2023-07-18T03:24:28.382 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-07-18 03:18:08.295831781 +0000 2023-07-18T03:24:28.383 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-07-18 03:18:08.295831781 +0000 2023-07-18T03:24:28.383 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2023-07-18T03:24:28.383 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-18T03:24:28.449 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-07-18T03:24:28.450 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-07-18T03:24:28.450 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000231054 s, 2.2 MB/s 2023-07-18T03:24:28.451 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-18T03:24:28.510 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2023-07-18T03:24:28.569 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-18T03:24:28.569 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:24:28.569 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 6h/6d Inode: 378545 Links: 1 2023-07-18T03:24:28.570 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:24:28.570 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:24:28.570 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-07-18 03:23:42.553698473 +0000 2023-07-18T03:24:28.570 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-07-18 03:18:08.819825305 +0000 2023-07-18T03:24:28.570 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-07-18 03:18:08.819825305 +0000 2023-07-18T03:24:28.570 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2023-07-18T03:24:28.571 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-18T03:24:28.636 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-07-18T03:24:28.637 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-07-18T03:24:28.637 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000311689 s, 1.6 MB/s 2023-07-18T03:24:28.638 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-18T03:24:28.698 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2023-07-18T03:24:28.756 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-18T03:24:28.756 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:24:28.756 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 6h/6d Inode: 383143 Links: 1 2023-07-18T03:24:28.756 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:24:28.757 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:24:28.757 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-07-18 03:23:42.553698473 +0000 2023-07-18T03:24:28.757 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-07-18 03:18:09.344818818 +0000 2023-07-18T03:24:28.757 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-07-18 03:18:09.344818818 +0000 2023-07-18T03:24:28.757 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2023-07-18T03:24:28.758 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-18T03:24:28.822 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-07-18T03:24:28.822 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-07-18T03:24:28.822 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000186899 s, 2.7 MB/s 2023-07-18T03:24:28.823 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-18T03:24:28.882 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2023-07-18T03:24:28.882 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2023-07-18T03:24:28.905 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-18T03:24:28.905 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2023-07-18T03:24:28.966 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-18T03:24:28.967 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:24:28.967 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 382511 Links: 1 2023-07-18T03:24:28.967 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:24:28.967 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:24:28.967 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-07-18 03:24:16.088619537 +0000 2023-07-18T03:24:28.967 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-07-18 03:17:53.210170396 +0000 2023-07-18T03:24:28.967 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-07-18 03:17:53.210170396 +0000 2023-07-18T03:24:28.968 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-07-18T03:24:28.968 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-18T03:24:29.023 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:28 smithi134 ceph-mon[98216]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi040=x;smithi064=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:24:29.023 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:28 smithi134 ceph-mon[98216]: Saving service mgr spec with placement smithi040=x;smithi064=y;smithi134=z;count:3 2023-07-18T03:24:29.023 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:29.024 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:28 smithi134 ceph-mon[98216]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:29.024 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:29.024 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:29.024 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:29.024 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:29.024 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:29.024 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:29.025 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T03:24:29.025 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-18T03:24:29.025 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T03:24:29.025 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:29.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:28 smithi064 ceph-mon[98059]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi040=x;smithi064=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:24:29.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:28 smithi064 ceph-mon[98059]: Saving service mgr spec with placement smithi040=x;smithi064=y;smithi134=z;count:3 2023-07-18T03:24:29.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:29.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:28 smithi064 ceph-mon[98059]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:29.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:29.037 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:29.037 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:29.037 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:29.037 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:29.037 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:29.037 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T03:24:29.037 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-18T03:24:29.038 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T03:24:29.038 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:29.038 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-07-18T03:24:29.039 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-07-18T03:24:29.039 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000269635 s, 1.9 MB/s 2023-07-18T03:24:29.040 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-18T03:24:29.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:28 smithi040 ceph-mon[96439]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi040=x;smithi064=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:24:29.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:28 smithi040 ceph-mon[96439]: Saving service mgr spec with placement smithi040=x;smithi064=y;smithi134=z;count:3 2023-07-18T03:24:29.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:29.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:28 smithi040 ceph-mon[96439]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:29.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:29.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:29.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:29.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:29.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:29.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:29.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T03:24:29.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-18T03:24:29.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T03:24:29.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:29.102 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2023-07-18T03:24:29.162 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-18T03:24:29.162 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:24:29.162 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 380046 Links: 1 2023-07-18T03:24:29.162 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:24:29.164 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:24:29.164 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-07-18 03:24:16.088619537 +0000 2023-07-18T03:24:29.164 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-07-18 03:17:53.735159873 +0000 2023-07-18T03:24:29.165 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-07-18 03:17:53.735159873 +0000 2023-07-18T03:24:29.165 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-07-18T03:24:29.165 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-18T03:24:29.230 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-07-18T03:24:29.231 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-07-18T03:24:29.231 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000229856 s, 2.2 MB/s 2023-07-18T03:24:29.232 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-18T03:24:29.291 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2023-07-18T03:24:29.350 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-18T03:24:29.350 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:24:29.350 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 377507 Links: 1 2023-07-18T03:24:29.351 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:24:29.351 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:24:29.351 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-07-18 03:24:16.088619537 +0000 2023-07-18T03:24:29.352 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-07-18 03:17:54.245149649 +0000 2023-07-18T03:24:29.352 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-07-18 03:17:54.245149649 +0000 2023-07-18T03:24:29.353 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-07-18T03:24:29.353 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-18T03:24:29.419 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-07-18T03:24:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-07-18T03:24:29.420 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000222597 s, 2.3 MB/s 2023-07-18T03:24:29.421 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-18T03:24:29.480 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2023-07-18T03:24:29.539 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-18T03:24:29.539 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:24:29.539 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 6h/6d Inode: 384196 Links: 1 2023-07-18T03:24:29.539 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:24:29.539 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:24:29.540 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2023-07-18 03:24:16.088619537 +0000 2023-07-18T03:24:29.540 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2023-07-18 03:17:54.761139305 +0000 2023-07-18T03:24:29.540 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2023-07-18 03:17:54.761139305 +0000 2023-07-18T03:24:29.540 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: - 2023-07-18T03:24:29.540 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-18T03:24:29.606 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2023-07-18T03:24:29.607 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2023-07-18T03:24:29.607 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000193501 s, 2.6 MB/s 2023-07-18T03:24:29.608 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-18T03:24:29.666 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2023-07-18T03:24:29.667 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2023-07-18T03:24:29.689 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-18T03:24:29.689 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2023-07-18T03:24:29.747 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-18T03:24:29.747 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:24:29.748 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 373131 Links: 1 2023-07-18T03:24:29.748 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:24:29.748 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:24:29.748 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2023-07-18 03:24:29.529759015 +0000 2023-07-18T03:24:29.748 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2023-07-18 03:17:39.322721484 +0000 2023-07-18T03:24:29.748 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2023-07-18 03:17:39.322721484 +0000 2023-07-18T03:24:29.748 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2023-07-18T03:24:29.749 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-18T03:24:29.816 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2023-07-18T03:24:29.817 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2023-07-18T03:24:29.817 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000222082 s, 2.3 MB/s 2023-07-18T03:24:29.818 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-18T03:24:29.873 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:29 smithi134 ceph-mon[98216]: Deploying daemon mgr.z on smithi134 2023-07-18T03:24:29.873 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:29 smithi134 systemd[1]: Starting Ceph mgr.z for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:24:29.877 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2023-07-18T03:24:29.934 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-18T03:24:29.935 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:24:29.935 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 375501 Links: 1 2023-07-18T03:24:29.935 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:24:29.935 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:24:29.935 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2023-07-18 03:24:29.529759015 +0000 2023-07-18T03:24:29.935 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2023-07-18 03:17:39.849718814 +0000 2023-07-18T03:24:29.935 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2023-07-18 03:17:39.849718814 +0000 2023-07-18T03:24:29.936 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2023-07-18T03:24:29.936 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-18T03:24:30.000 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2023-07-18T03:24:30.001 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2023-07-18T03:24:30.001 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.00020453 s, 2.5 MB/s 2023-07-18T03:24:30.002 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-18T03:24:30.061 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2023-07-18T03:24:30.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:29 smithi040 ceph-mon[96439]: Deploying daemon mgr.z on smithi134 2023-07-18T03:24:30.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:29 smithi064 ceph-mon[98059]: Deploying daemon mgr.z on smithi134 2023-07-18T03:24:30.119 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-18T03:24:30.119 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:24:30.119 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 381029 Links: 1 2023-07-18T03:24:30.120 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:24:30.120 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:24:30.120 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2023-07-18 03:24:29.529759015 +0000 2023-07-18T03:24:30.120 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2023-07-18 03:17:40.380716125 +0000 2023-07-18T03:24:30.120 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2023-07-18 03:17:40.380716125 +0000 2023-07-18T03:24:30.120 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2023-07-18T03:24:30.121 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-18T03:24:30.139 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:29 smithi134 podman[100083]: 2023-07-18T03:24:30.165 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2023-07-18T03:24:30.165 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2023-07-18T03:24:30.165 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000200843 s, 2.5 MB/s 2023-07-18T03:24:30.166 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-18T03:24:30.224 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2023-07-18T03:24:30.282 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-18T03:24:30.282 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-18T03:24:30.282 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 383217 Links: 1 2023-07-18T03:24:30.283 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:24:30.283 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2023-07-18T03:24:30.283 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2023-07-18 03:24:29.529759015 +0000 2023-07-18T03:24:30.283 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2023-07-18 03:17:40.886713561 +0000 2023-07-18T03:24:30.283 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2023-07-18 03:17:40.886713561 +0000 2023-07-18T03:24:30.283 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2023-07-18T03:24:30.284 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-18T03:24:30.348 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2023-07-18T03:24:30.348 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2023-07-18T03:24:30.349 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.00019417 s, 2.6 MB/s 2023-07-18T03:24:30.350 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-18T03:24:30.409 INFO:tasks.cephadm:Deploying osd.0 on smithi040 with /dev/vg_nvme/lv_4... 2023-07-18T03:24:30.409 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-18T03:24:30.447 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:30 smithi134 ceph-mgr[100262]: -- 172.21.15.134:0/775833821 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558ab05871e0 con 0x558ab055cc00 2023-07-18T03:24:30.447 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:30 smithi134 bash[100083]: a107047838e21c414305ca8934184286cdae59ca4f0eef0b1d28faed8cbf7f7c 2023-07-18T03:24:30.448 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:30 smithi134 systemd[1]: Started Ceph mgr.z for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:24:30.448 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:30.277+0000 7fa662df7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-18T03:24:30.448 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:30.359+0000 7fa662df7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-18T03:24:30.448 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:30.450+0000 7fa662df7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-18T03:24:30.709 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:30.543+0000 7fa662df7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-18T03:24:30.710 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:30 smithi134 ceph-mon[98216]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:30.710 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:30 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:30.710 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:30 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:30.710 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:30 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:30.711 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:30 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T03:24:30.711 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:30 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-18T03:24:30.711 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:30 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T03:24:30.711 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:30 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:30.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:30 smithi040 ceph-mon[96439]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:30.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:30 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:30.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:30 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:30.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:30 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:30.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:30 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T03:24:30.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:30 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-18T03:24:30.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:30 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T03:24:30.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:30 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:30.948 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:30 smithi064 ceph-mon[98059]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:30.948 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:30 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:30.948 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:30 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:30.949 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:30 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:30.949 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:30 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T03:24:30.949 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:30 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-18T03:24:30.949 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:30 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T03:24:30.949 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:30 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:30.985 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:24:31.015 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:30.713+0000 7fa662df7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-18T03:24:31.316 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:31 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:31.018+0000 7fa662df7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-18T03:24:31.316 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:31 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:31.106+0000 7fa662df7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-18T03:24:31.316 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:31 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:31.234+0000 7fa662df7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-18T03:24:31.693 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:31 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:31.327+0000 7fa662df7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-18T03:24:31.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:31 smithi064 ceph-mon[98059]: Deploying daemon mgr.y on smithi064 2023-07-18T03:24:31.851 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:31 smithi064 systemd[1]: Starting Ceph mgr.y for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:24:32.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:31 smithi134 ceph-mon[98216]: Deploying daemon mgr.y on smithi064 2023-07-18T03:24:32.066 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:31 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:31.746+0000 7fa662df7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-18T03:24:32.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:31 smithi040 ceph-mon[96439]: Deploying daemon mgr.y on smithi064 2023-07-18T03:24:32.271 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:32 smithi064 podman[99503]: 2023-07-18T03:24:32.379 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-18T03:24:32.379 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-18T03:24:32.379 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2023-07-18T03:24:32.379 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2023-07-18T03:24:32.380 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0153663 s, 682 MB/s 2023-07-18T03:24:32.380 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2023-07-18T03:24:32.566 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:32 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:32.315+0000 7fa662df7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-18T03:24:32.566 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:32 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:32.487+0000 7fa662df7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-18T03:24:32.600 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:32 smithi064 bash[99503]: 60e8e108da70b02a29ad4f62ac35dc554be0571a2e8cdd12d33a13c599f0881f 2023-07-18T03:24:32.600 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:32 smithi064 ceph-mgr[99543]: -- 172.21.15.64:0/1824835483 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5559e3e591e0 con 0x5559e3e2e800 2023-07-18T03:24:32.600 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:32 smithi064 systemd[1]: Started Ceph mgr.y for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:24:32.601 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:32 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:32.424+0000 7f15e0794200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-18T03:24:32.601 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:32 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:32.520+0000 7f15e0794200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-18T03:24:32.935 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi040:vg_nvme/lv_4 2023-07-18T03:24:32.960 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:32 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:32.630+0000 7f15e0794200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-18T03:24:32.960 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:32 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:32.746+0000 7f15e0794200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-18T03:24:32.961 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:32 smithi064 ceph-mon[98059]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:32.961 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:32.961 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:32.961 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:32.961 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:32.961 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:32.963 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:32 smithi040 ceph-mon[96439]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:32.963 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:32.963 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:32.963 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:32.963 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:32.963 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:33.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:32 smithi134 ceph-mon[98216]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:33.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:33.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:33.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:33.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:33.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:33.350 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:32 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:32.963+0000 7f15e0794200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-18T03:24:33.495 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:24:33.555 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:33 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:33.291+0000 7fa662df7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:24:33.555 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:33 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:33.382+0000 7fa662df7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-18T03:24:33.636 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:33 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:33.371+0000 7f15e0794200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-18T03:24:33.636 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:33 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:33.482+0000 7f15e0794200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-18T03:24:33.805 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:33 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:33.558+0000 7fa662df7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:24:34.055 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:33 smithi040 ceph-mon[96439]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:34.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:33 smithi134 ceph-mon[98216]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:34.067 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:33 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:33.864+0000 7fa662df7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-18T03:24:34.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:33 smithi064 ceph-mon[98059]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:34.100 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:33 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:33.639+0000 7f15e0794200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-18T03:24:34.101 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:33 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:33.754+0000 7f15e0794200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-18T03:24:34.516 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:34 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:34.109+0000 7fa662df7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-18T03:24:34.517 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:34 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:34.257+0000 7fa662df7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-18T03:24:34.600 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:34 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:34.279+0000 7f15e0794200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-18T03:24:34.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.714+0000 7f0d13c88700 1 -- 172.21.15.40:0/2884883712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0c0ffc80 msgr2=0x7f0d0c1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:34.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.714+0000 7f0d13c88700 1 --2- 172.21.15.40:0/2884883712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0c0ffc80 0x7f0d0c1000d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009990 tx=0x7f0d08009ca0 comp rx=0 tx=0).stop 2023-07-18T03:24:34.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.714+0000 7f0d13c88700 1 -- 172.21.15.40:0/2884883712 shutdown_connections 2023-07-18T03:24:34.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.714+0000 7f0d13c88700 1 --2- 172.21.15.40:0/2884883712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d0c100610 0x7f0d0c106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:34.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.714+0000 7f0d13c88700 1 --2- 172.21.15.40:0/2884883712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c0fea80 0x7f0d0c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:34.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.714+0000 7f0d13c88700 1 --2- 172.21.15.40:0/2884883712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0c0ffc80 0x7f0d0c1000d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:34.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.714+0000 7f0d13c88700 1 -- 172.21.15.40:0/2884883712 >> 172.21.15.40:0/2884883712 conn(0x7f0d0c0fa0b0 msgr2=0x7f0d0c0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:34.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.714+0000 7f0d13c88700 1 -- 172.21.15.40:0/2884883712 shutdown_connections 2023-07-18T03:24:34.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.714+0000 7f0d13c88700 1 -- 172.21.15.40:0/2884883712 wait complete. 2023-07-18T03:24:34.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d13c88700 1 Processor -- start 2023-07-18T03:24:34.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d13c88700 1 -- start start 2023-07-18T03:24:34.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d13c88700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c0fea80 0x7f0d0c102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:34.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d13c88700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d0c0ffc80 0x7f0d0c101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:34.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d13c88700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0c100610 0x7f0d0c1015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:34.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d13c88700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c101d20 con 0x7f0d0c100610 2023-07-18T03:24:34.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d13c88700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c101e90 con 0x7f0d0c0fea80 2023-07-18T03:24:34.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d13c88700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c19f8a0 con 0x7f0d0c0ffc80 2023-07-18T03:24:34.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d12225700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0c100610 0x7f0d0c1015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:34.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d11a24700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c0fea80 0x7f0d0c102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:34.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d11223700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d0c0ffc80 0x7f0d0c101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:34.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d11a24700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c0fea80 0x7f0d0c102a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:38528/0 (socket says 172.21.15.40:38528) 2023-07-18T03:24:34.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d12225700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0c100610 0x7f0d0c1015a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55206/0 (socket says 172.21.15.40:55206) 2023-07-18T03:24:34.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d11a24700 1 -- 172.21.15.40:0/1931236992 learned_addr learned my addr 172.21.15.40:0/1931236992 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:24:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d11223700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d0c0ffc80 0x7f0d0c101060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:44232/0 (socket says 172.21.15.40:44232) 2023-07-18T03:24:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d11a24700 1 -- 172.21.15.40:0/1931236992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d0c0ffc80 msgr2=0x7f0d0c101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d11a24700 1 --2- 172.21.15.40:0/1931236992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d0c0ffc80 0x7f0d0c101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d11a24700 1 -- 172.21.15.40:0/1931236992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0c100610 msgr2=0x7f0d0c1015a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:34.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d11a24700 1 --2- 172.21.15.40:0/1931236992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0c100610 0x7f0d0c1015a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:34.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.715+0000 7f0d11a24700 1 -- 172.21.15.40:0/1931236992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d080095f0 con 0x7f0d0c0fea80 2023-07-18T03:24:34.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.716+0000 7f0d11a24700 1 --2- 172.21.15.40:0/1931236992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c0fea80 0x7f0d0c102a10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00e970 tx=0x7f0cfc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:34.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.716+0000 7f0d02ffd700 1 -- 172.21.15.40:0/1931236992 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc009790 con 0x7f0d0c0fea80 2023-07-18T03:24:34.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.716+0000 7f0d02ffd700 1 -- 172.21.15.40:0/1931236992 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cfc00c7e0 con 0x7f0d0c0fea80 2023-07-18T03:24:34.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.716+0000 7f0d13c88700 1 -- 172.21.15.40:0/1931236992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0c19fb80 con 0x7f0d0c0fea80 2023-07-18T03:24:34.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.716+0000 7f0d02ffd700 1 -- 172.21.15.40:0/1931236992 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc003c70 con 0x7f0d0c0fea80 2023-07-18T03:24:34.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.716+0000 7f0d13c88700 1 -- 172.21.15.40:0/1931236992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0c1a00d0 con 0x7f0d0c0fea80 2023-07-18T03:24:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.717+0000 7f0d02ffd700 1 -- 172.21.15.40:0/1931236992 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f0cfc018430 con 0x7f0d0c0fea80 2023-07-18T03:24:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.717+0000 7f0d13c88700 1 -- 172.21.15.40:0/1931236992 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d0c10a240 con 0x7f0d0c0fea80 2023-07-18T03:24:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.717+0000 7f0d02ffd700 1 --2- 172.21.15.40:0/1931236992 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0cf80389e0 0x7f0cf803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:34.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.717+0000 7f0d02ffd700 1 -- 172.21.15.40:0/1931236992 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0cfc014070 con 0x7f0d0c0fea80 2023-07-18T03:24:34.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.717+0000 7f0d11223700 1 --2- 172.21.15.40:0/1931236992 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0cf80389e0 0x7f0cf803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:34.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.717+0000 7f0d11223700 1 --2- 172.21.15.40:0/1931236992 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0cf80389e0 0x7f0cf803ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800b2e0 tx=0x7f0d08005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:34.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.722+0000 7f0d02ffd700 1 -- 172.21.15.40:0/1931236992 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0cfc003dd0 con 0x7f0d0c0fea80 2023-07-18T03:24:34.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:34.934+0000 7f0d13c88700 1 -- 172.21.15.40:0/1931236992 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d0c068740 con 0x7f0cf80389e0 2023-07-18T03:24:35.230 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:35.230 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:35.230 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:35.230 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:35.231 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:35.231 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:35.231 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:35.231 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:35 smithi040 ceph-mon[96439]: Reconfiguring mgr.x (unknown last config time)... 2023-07-18T03:24:35.232 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T03:24:35.232 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T03:24:35.232 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:35.232 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:35 smithi040 ceph-mon[96439]: Reconfiguring daemon mgr.x on smithi040 2023-07-18T03:24:35.233 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:24:35.233 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:24:35.233 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:35.339 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:34 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:34.520+0000 7fa662df7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-18T03:24:35.340 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:35.214+0000 7fa662df7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-18T03:24:35.340 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:35.340 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:35.340 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:35.340 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:35.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:35.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:35.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:35.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-mon[98216]: Reconfiguring mgr.x (unknown last config time)... 2023-07-18T03:24:35.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T03:24:35.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T03:24:35.341 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:35.342 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-mon[98216]: Reconfiguring daemon mgr.x on smithi040 2023-07-18T03:24:35.342 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:24:35.342 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:24:35.342 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:35.350 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:34 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:34.977+0000 7f15e0794200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-18T03:24:35.350 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:35 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:35.168+0000 7f15e0794200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-18T03:24:35.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:35.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:35.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:35.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:35.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:35.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:35.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:35 smithi064 ceph-mon[98059]: Reconfiguring mgr.x (unknown last config time)... 2023-07-18T03:24:35.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-18T03:24:35.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-18T03:24:35.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:35.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:35 smithi064 ceph-mon[98059]: Reconfiguring daemon mgr.x on smithi040 2023-07-18T03:24:35.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:24:35.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:24:35.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:35.641 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:35.343+0000 7fa662df7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-18T03:24:35.642 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:35.532+0000 7fa662df7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-18T03:24:36.066 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:35.645+0000 7fa662df7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-18T03:24:36.066 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:35 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:35.856+0000 7fa662df7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-18T03:24:36.455 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:36 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:36.132+0000 7f15e0794200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:24:36.456 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:36 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:36.242+0000 7f15e0794200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-18T03:24:36.802 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:36 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:36.459+0000 7f15e0794200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-18T03:24:36.803 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:36 smithi064 ceph-mon[98059]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:24:36.803 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:36 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:36.804 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:36 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:36.804 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:36 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:36.805 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:36 smithi064 ceph-mon[98059]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:36.805 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:36 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:36.805 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:36 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:36.805 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:36 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:37.054 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:36 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:36.806+0000 7f15e0794200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-18T03:24:37.066 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:24:36 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z[100180]: 2023-07-18T03:24:36.711+0000 7fa662df7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-18T03:24:37.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:36 smithi134 ceph-mon[98216]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:24:37.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:36 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:37.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:36 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:37.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:36 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:37.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:36 smithi134 ceph-mon[98216]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:37.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:36 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:37.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:36 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:37.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:36 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:37.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:36 smithi040 ceph-mon[96439]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:24:37.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:36 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:37.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:36 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:37.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:36 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:37.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:36 smithi040 ceph-mon[96439]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:37.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:36 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:37.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:36 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:37.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:36 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:37.350 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:37 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:37.057+0000 7f15e0794200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-18T03:24:37.350 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:37 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:37.239+0000 7f15e0794200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-18T03:24:37.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:37.662+0000 7f0d02ffd700 1 -- 172.21.15.40:0/1931236992 <== mon.2 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 14) v1 ==== 90599+0+0 (secure 0 0 0) 0x7f0cfc021430 con 0x7f0d0c0fea80 2023-07-18T03:24:37.850 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:37 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:37.565+0000 7f15e0794200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-18T03:24:37.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:37 smithi064 ceph-mon[98059]: Standby manager daemon z started 2023-07-18T03:24:37.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:37 smithi064 ceph-mon[98059]: from='mgr.? 172.21.15.134:0/1495805477' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-18T03:24:37.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:37 smithi064 ceph-mon[98059]: from='mgr.? 172.21.15.134:0/1495805477' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T03:24:37.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:37 smithi064 ceph-mon[98059]: from='mgr.? 172.21.15.134:0/1495805477' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-18T03:24:37.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:37 smithi064 ceph-mon[98059]: from='mgr.? 172.21.15.134:0/1495805477' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T03:24:37.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:37 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:37.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:37 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/727966498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f924e504-7de2-48fb-820f-7b74c5a46ae3"}]: dispatch 2023-07-18T03:24:37.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:37 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/727966498' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f924e504-7de2-48fb-820f-7b74c5a46ae3"}]': finished 2023-07-18T03:24:37.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:37 smithi064 ceph-mon[98059]: osdmap e5: 1 total, 0 up, 1 in 2023-07-18T03:24:37.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:37 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:38.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:37 smithi134 ceph-mon[98216]: Standby manager daemon z started 2023-07-18T03:24:38.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:37 smithi134 ceph-mon[98216]: from='mgr.? 172.21.15.134:0/1495805477' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-18T03:24:38.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:37 smithi134 ceph-mon[98216]: from='mgr.? 172.21.15.134:0/1495805477' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T03:24:38.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:37 smithi134 ceph-mon[98216]: from='mgr.? 172.21.15.134:0/1495805477' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-18T03:24:38.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:37 smithi134 ceph-mon[98216]: from='mgr.? 172.21.15.134:0/1495805477' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T03:24:38.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:37 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:38.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:37 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/727966498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f924e504-7de2-48fb-820f-7b74c5a46ae3"}]: dispatch 2023-07-18T03:24:38.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:37 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/727966498' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f924e504-7de2-48fb-820f-7b74c5a46ae3"}]': finished 2023-07-18T03:24:38.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:37 smithi134 ceph-mon[98216]: osdmap e5: 1 total, 0 up, 1 in 2023-07-18T03:24:38.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:37 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:38.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:37 smithi040 ceph-mon[96439]: Standby manager daemon z started 2023-07-18T03:24:38.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:37 smithi040 ceph-mon[96439]: from='mgr.? 172.21.15.134:0/1495805477' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-18T03:24:38.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:37 smithi040 ceph-mon[96439]: from='mgr.? 172.21.15.134:0/1495805477' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T03:24:38.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:37 smithi040 ceph-mon[96439]: from='mgr.? 172.21.15.134:0/1495805477' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-18T03:24:38.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:37 smithi040 ceph-mon[96439]: from='mgr.? 172.21.15.134:0/1495805477' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T03:24:38.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:37 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:38.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:37 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/727966498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f924e504-7de2-48fb-820f-7b74c5a46ae3"}]: dispatch 2023-07-18T03:24:38.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:37 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/727966498' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f924e504-7de2-48fb-820f-7b74c5a46ae3"}]': finished 2023-07-18T03:24:38.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:37 smithi040 ceph-mon[96439]: osdmap e5: 1 total, 0 up, 1 in 2023-07-18T03:24:38.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:37 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:38.664 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:38 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:38.384+0000 7f15e0794200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-18T03:24:38.664 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:38 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:38.542+0000 7f15e0794200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-18T03:24:38.918 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:38 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:38.784+0000 7f15e0794200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-18T03:24:38.919 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:38 smithi064 ceph-mon[98059]: mgrmap e14: x(active, since 65s), standbys: z 2023-07-18T03:24:38.920 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:38 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-18T03:24:38.920 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:38 smithi064 ceph-mon[98059]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:38.921 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:38 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/213206435' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:24:39.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:38 smithi134 ceph-mon[98216]: mgrmap e14: x(active, since 65s), standbys: z 2023-07-18T03:24:39.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:38 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-18T03:24:39.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:38 smithi134 ceph-mon[98216]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:39.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:38 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/213206435' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:24:39.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:38 smithi040 ceph-mon[96439]: mgrmap e14: x(active, since 65s), standbys: z 2023-07-18T03:24:39.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:38 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-18T03:24:39.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:38 smithi040 ceph-mon[96439]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:39.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:38 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/213206435' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:24:39.600 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:38 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:38.921+0000 7f15e0794200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-18T03:24:39.600 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:39 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:39.190+0000 7f15e0794200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-18T03:24:40.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:39 smithi134 ceph-mon[98216]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:40.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:39 smithi040 ceph-mon[96439]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:40.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:39 smithi064 ceph-mon[98059]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:40.600 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:24:40 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y[99519]: 2023-07-18T03:24:40.234+0000 7f15e0794200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-18T03:24:40.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:40.822+0000 7f0d02ffd700 1 -- 172.21.15.40:0/1931236992 <== mon.2 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f0cfc07d070 con 0x7f0d0c0fea80 2023-07-18T03:24:41.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:40 smithi134 ceph-mon[98216]: Standby manager daemon y started 2023-07-18T03:24:41.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:40 smithi134 ceph-mon[98216]: from='mgr.? 172.21.15.64:0/1745921527' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-18T03:24:41.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:40 smithi134 ceph-mon[98216]: from='mgr.? 172.21.15.64:0/1745921527' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T03:24:41.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:40 smithi134 ceph-mon[98216]: from='mgr.? 172.21.15.64:0/1745921527' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-18T03:24:41.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:40 smithi134 ceph-mon[98216]: from='mgr.? 172.21.15.64:0/1745921527' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T03:24:41.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:40 smithi040 ceph-mon[96439]: Standby manager daemon y started 2023-07-18T03:24:41.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:40 smithi040 ceph-mon[96439]: from='mgr.? 172.21.15.64:0/1745921527' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-18T03:24:41.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:40 smithi040 ceph-mon[96439]: from='mgr.? 172.21.15.64:0/1745921527' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T03:24:41.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:40 smithi040 ceph-mon[96439]: from='mgr.? 172.21.15.64:0/1745921527' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-18T03:24:41.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:40 smithi040 ceph-mon[96439]: from='mgr.? 172.21.15.64:0/1745921527' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T03:24:41.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:40 smithi064 ceph-mon[98059]: Standby manager daemon y started 2023-07-18T03:24:41.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:40 smithi064 ceph-mon[98059]: from='mgr.? 172.21.15.64:0/1745921527' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-18T03:24:41.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:40 smithi064 ceph-mon[98059]: from='mgr.? 172.21.15.64:0/1745921527' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-18T03:24:41.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:40 smithi064 ceph-mon[98059]: from='mgr.? 172.21.15.64:0/1745921527' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-18T03:24:41.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:40 smithi064 ceph-mon[98059]: from='mgr.? 172.21.15.64:0/1745921527' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-18T03:24:42.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:41 smithi134 ceph-mon[98216]: mgrmap e15: x(active, since 69s), standbys: z, y 2023-07-18T03:24:42.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:41 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-18T03:24:42.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:41 smithi134 ceph-mon[98216]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:42.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:41 smithi040 ceph-mon[96439]: mgrmap e15: x(active, since 69s), standbys: z, y 2023-07-18T03:24:42.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:41 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-18T03:24:42.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:41 smithi040 ceph-mon[96439]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:42.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:41 smithi064 ceph-mon[98059]: mgrmap e15: x(active, since 69s), standbys: z, y 2023-07-18T03:24:42.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:41 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-18T03:24:42.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:41 smithi064 ceph-mon[98059]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:44.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:43 smithi134 ceph-mon[98216]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:44.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:43 smithi040 ceph-mon[96439]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:44.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:43 smithi064 ceph-mon[98059]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:45.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-18T03:24:45.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:45.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:44 smithi134 ceph-mon[98216]: Deploying daemon osd.0 on smithi040 2023-07-18T03:24:45.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-18T03:24:45.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:45.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:44 smithi040 ceph-mon[96439]: Deploying daemon osd.0 on smithi040 2023-07-18T03:24:45.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-18T03:24:45.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:45.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:44 smithi064 ceph-mon[98059]: Deploying daemon osd.0 on smithi040 2023-07-18T03:24:45.937 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:45 smithi040 ceph-mon[96439]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:46.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:45 smithi134 ceph-mon[98216]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:46.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:45 smithi064 ceph-mon[98059]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:48.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:47 smithi134 ceph-mon[98216]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:48.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:47 smithi040 ceph-mon[96439]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:48.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:47 smithi064 ceph-mon[98059]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:49.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:49 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:49.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:49 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:49.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:49 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:49.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:49 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:49.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:49 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:49.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:49 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:49.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:49 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:49.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:49 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:49.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:49 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:50.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:50 smithi134 ceph-mon[98216]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:50.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:50 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:50.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:50 smithi040 ceph-mon[96439]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:50.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:50 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:50.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:50 smithi064 ceph-mon[98059]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:50.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:50 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:51.190 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 0 on host 'smithi040' 2023-07-18T03:24:51.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:51.193+0000 7f0d02ffd700 1 -- 172.21.15.40:0/1931236992 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0d0c068740 con 0x7f0cf80389e0 2023-07-18T03:24:51.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:51.195+0000 7f0d13c88700 1 -- 172.21.15.40:0/1931236992 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0cf80389e0 msgr2=0x7f0cf803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:51.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:51.195+0000 7f0d13c88700 1 --2- 172.21.15.40:0/1931236992 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0cf80389e0 0x7f0cf803ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800b2e0 tx=0x7f0d08005fb0 comp rx=0 tx=0).stop 2023-07-18T03:24:51.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:51.195+0000 7f0d13c88700 1 -- 172.21.15.40:0/1931236992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c0fea80 msgr2=0x7f0d0c102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:51.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:51.195+0000 7f0d13c88700 1 --2- 172.21.15.40:0/1931236992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c0fea80 0x7f0d0c102a10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00e970 tx=0x7f0cfc00ec80 comp rx=0 tx=0).stop 2023-07-18T03:24:51.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:51.195+0000 7f0d13c88700 1 -- 172.21.15.40:0/1931236992 shutdown_connections 2023-07-18T03:24:51.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:51.195+0000 7f0d13c88700 1 --2- 172.21.15.40:0/1931236992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d0c100610 0x7f0d0c1015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:51.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:51.195+0000 7f0d13c88700 1 --2- 172.21.15.40:0/1931236992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d0c0fea80 0x7f0d0c102a10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:51.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:51.195+0000 7f0d13c88700 1 --2- 172.21.15.40:0/1931236992 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0cf80389e0 0x7f0cf803ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:51.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:51.195+0000 7f0d13c88700 1 --2- 172.21.15.40:0/1931236992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d0c0ffc80 0x7f0d0c101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:51.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:51.195+0000 7f0d13c88700 1 -- 172.21.15.40:0/1931236992 >> 172.21.15.40:0/1931236992 conn(0x7f0d0c0fa0b0 msgr2=0x7f0d0c068030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:51.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:51.195+0000 7f0d13c88700 1 -- 172.21.15.40:0/1931236992 shutdown_connections 2023-07-18T03:24:51.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:24:51.195+0000 7f0d13c88700 1 -- 172.21.15.40:0/1931236992 wait complete. 2023-07-18T03:24:51.596 DEBUG:teuthology.orchestra.run.smithi040:osd.0> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.0.service 2023-07-18T03:24:51.628 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:51 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:51.628 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:51 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:51.628 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:51 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:51.628 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:51 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:51.629 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:51 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:51.629 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:51 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:51.629 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:51 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:51.630 INFO:tasks.cephadm:Deploying osd.1 on smithi064 with /dev/vg_nvme/lv_4... 2023-07-18T03:24:51.631 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-18T03:24:51.661 INFO:journalctl@ceph.osd.0.smithi040.stdout:-- Logs begin at Tue 2023-07-18 03:07:07 UTC. -- 2023-07-18T03:24:51.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:51 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:51.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:51 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:51.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:51 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:51.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:51 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:51.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:51 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:51.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:51 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:51.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:51 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:51.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:51 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:51.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:51 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:51.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:51 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:51.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:51 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:51.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:51 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:51.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:51 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:51.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:51 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:24:52.202 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:24:52.737 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:52 smithi064 ceph-mon[98059]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:52.737 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:52 smithi064 ceph-mon[98059]: from='osd.0 [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-18T03:24:52.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:52 smithi134 ceph-mon[98216]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:52.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:52 smithi134 ceph-mon[98216]: from='osd.0 [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-18T03:24:52.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:52 smithi040 ceph-mon[96439]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:52.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:52 smithi040 ceph-mon[96439]: from='osd.0 [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-18T03:24:52.830 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jul 18 03:24:52 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-0[109506]: 2023-07-18T03:24:52.482+0000 7fa7944f2640 -1 osd.0 0 log_to_monitors true 2023-07-18T03:24:53.736 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-18T03:24:53.736 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-18T03:24:53.736 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2023-07-18T03:24:53.736 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2023-07-18T03:24:53.736 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109367 s, 959 MB/s 2023-07-18T03:24:53.737 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2023-07-18T03:24:53.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:53 smithi134 ceph-mon[98216]: from='osd.0 [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-18T03:24:53.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:53 smithi134 ceph-mon[98216]: osdmap e6: 1 total, 0 up, 1 in 2023-07-18T03:24:53.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:53 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:53.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:53 smithi134 ceph-mon[98216]: from='osd.0 [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2023-07-18T03:24:53.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:53 smithi040 ceph-mon[96439]: from='osd.0 [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-18T03:24:53.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:53 smithi040 ceph-mon[96439]: osdmap e6: 1 total, 0 up, 1 in 2023-07-18T03:24:53.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:53 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:53.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:53 smithi040 ceph-mon[96439]: from='osd.0 [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2023-07-18T03:24:53.830 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jul 18 03:24:53 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-0[109506]: 2023-07-18T03:24:53.529+0000 7fa789369700 -1 osd.0 0 waiting for initial osdmap 2023-07-18T03:24:53.830 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jul 18 03:24:53 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-0[109506]: 2023-07-18T03:24:53.538+0000 7fa78395b700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:24:53.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:53 smithi064 ceph-mon[98059]: from='osd.0 [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-18T03:24:53.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:53 smithi064 ceph-mon[98059]: osdmap e6: 1 total, 0 up, 1 in 2023-07-18T03:24:53.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:53 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:53.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:53 smithi064 ceph-mon[98059]: from='osd.0 [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2023-07-18T03:24:54.159 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi064:vg_nvme/lv_4 2023-07-18T03:24:54.716 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:24:54.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:54 smithi134 ceph-mon[98216]: from='osd.0 [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2023-07-18T03:24:54.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:54 smithi134 ceph-mon[98216]: osdmap e7: 1 total, 0 up, 1 in 2023-07-18T03:24:54.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:54.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:54.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:54 smithi134 ceph-mon[98216]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:54.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:54.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:54.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:24:54.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:54.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:54.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:54.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:54.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:54 smithi040 ceph-mon[96439]: from='osd.0 [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2023-07-18T03:24:54.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:54 smithi040 ceph-mon[96439]: osdmap e7: 1 total, 0 up, 1 in 2023-07-18T03:24:54.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:54.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:54.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:54 smithi040 ceph-mon[96439]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:54.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:54.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:54.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:24:54.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:54.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:54.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:54.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:54.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:54 smithi064 ceph-mon[98059]: from='osd.0 [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2023-07-18T03:24:54.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:54 smithi064 ceph-mon[98059]: osdmap e7: 1 total, 0 up, 1 in 2023-07-18T03:24:54.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:54.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:54.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:54 smithi064 ceph-mon[98059]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-18T03:24:54.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:54.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:24:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:54.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:24:54.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:24:55.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:55 smithi134 ceph-mon[98216]: purged_snaps scrub starts 2023-07-18T03:24:55.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:55 smithi134 ceph-mon[98216]: purged_snaps scrub ok 2023-07-18T03:24:55.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:55 smithi134 ceph-mon[98216]: Detected new or changed devices on smithi040 2023-07-18T03:24:55.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:55 smithi134 ceph-mon[98216]: Adjusting osd_memory_target on smithi040 to 17173M 2023-07-18T03:24:55.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:55.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:55 smithi134 ceph-mon[98216]: osd.0 [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254] boot 2023-07-18T03:24:55.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:55 smithi134 ceph-mon[98216]: osdmap e8: 1 total, 1 up, 1 in 2023-07-18T03:24:55.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:55.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:55 smithi040 ceph-mon[96439]: purged_snaps scrub starts 2023-07-18T03:24:55.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:55 smithi040 ceph-mon[96439]: purged_snaps scrub ok 2023-07-18T03:24:55.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:55 smithi040 ceph-mon[96439]: Detected new or changed devices on smithi040 2023-07-18T03:24:55.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:55 smithi040 ceph-mon[96439]: Adjusting osd_memory_target on smithi040 to 17173M 2023-07-18T03:24:55.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:55.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:55 smithi040 ceph-mon[96439]: osd.0 [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254] boot 2023-07-18T03:24:55.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:55 smithi040 ceph-mon[96439]: osdmap e8: 1 total, 1 up, 1 in 2023-07-18T03:24:55.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:55.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:55 smithi064 ceph-mon[98059]: purged_snaps scrub starts 2023-07-18T03:24:55.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:55 smithi064 ceph-mon[98059]: purged_snaps scrub ok 2023-07-18T03:24:55.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:55 smithi064 ceph-mon[98059]: Detected new or changed devices on smithi040 2023-07-18T03:24:55.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:55 smithi064 ceph-mon[98059]: Adjusting osd_memory_target on smithi040 to 17173M 2023-07-18T03:24:55.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:55.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:55 smithi064 ceph-mon[98059]: osd.0 [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254] boot 2023-07-18T03:24:55.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:55 smithi064 ceph-mon[98059]: osdmap e8: 1 total, 1 up, 1 in 2023-07-18T03:24:55.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-18T03:24:55.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.906+0000 7fbe8e26a700 1 -- 172.21.15.64:0/576483380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe880fea80 msgr2=0x7fbe880fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:55.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.906+0000 7fbe8e26a700 1 --2- 172.21.15.64:0/576483380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe880fea80 0x7fbe880fee90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009990 tx=0x7fbe78009ca0 comp rx=0 tx=0).stop 2023-07-18T03:24:55.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.906+0000 7fbe8e26a700 1 -- 172.21.15.64:0/576483380 shutdown_connections 2023-07-18T03:24:55.903 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.906+0000 7fbe8e26a700 1 --2- 172.21.15.64:0/576483380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe88100610 0x7fbe88106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:55.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.906+0000 7fbe8e26a700 1 --2- 172.21.15.64:0/576483380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe880fea80 0x7fbe880fee90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:55.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.906+0000 7fbe8e26a700 1 --2- 172.21.15.64:0/576483380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe880ffc80 0x7fbe881000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:55.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.906+0000 7fbe8e26a700 1 -- 172.21.15.64:0/576483380 >> 172.21.15.64:0/576483380 conn(0x7fbe880fa0b0 msgr2=0x7fbe880fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:24:55.904 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.906+0000 7fbe8e26a700 1 -- 172.21.15.64:0/576483380 shutdown_connections 2023-07-18T03:24:55.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.906+0000 7fbe8e26a700 1 -- 172.21.15.64:0/576483380 wait complete. 2023-07-18T03:24:55.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.906+0000 7fbe8e26a700 1 Processor -- start 2023-07-18T03:24:55.905 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.906+0000 7fbe8e26a700 1 -- start start 2023-07-18T03:24:55.907 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe8e26a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe880fea80 0x7fbe88068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe8e26a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe880ffc80 0x7fbe8819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe8e26a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe88100610 0x7fbe8819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe8e26a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8819d420 con 0x7fbe880fea80 2023-07-18T03:24:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe8e26a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8819d590 con 0x7fbe880ffc80 2023-07-18T03:24:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe8e26a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8819d750 con 0x7fbe88100610 2023-07-18T03:24:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe877fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe880ffc80 0x7fbe8819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:55.908 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe877fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe880ffc80 0x7fbe8819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37372/0 (socket says 172.21.15.64:37372) 2023-07-18T03:24:55.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe877fe700 1 -- 172.21.15.64:0/2758515500 learned_addr learned my addr 172.21.15.64:0/2758515500 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-07-18T03:24:55.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe87fff700 1 --2- 172.21.15.64:0/2758515500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe880fea80 0x7fbe88068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:55.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe8c807700 1 --2- 172.21.15.64:0/2758515500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe88100610 0x7fbe8819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:55.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe877fe700 1 -- 172.21.15.64:0/2758515500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe88100610 msgr2=0x7fbe8819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:55.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe877fe700 1 --2- 172.21.15.64:0/2758515500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe88100610 0x7fbe8819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:55.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe877fe700 1 -- 172.21.15.64:0/2758515500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe880fea80 msgr2=0x7fbe88068b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:24:55.909 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe877fe700 1 --2- 172.21.15.64:0/2758515500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe880fea80 0x7fbe88068b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:24:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe877fe700 1 -- 172.21.15.64:0/2758515500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe780095f0 con 0x7fbe880ffc80 2023-07-18T03:24:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe877fe700 1 --2- 172.21.15.64:0/2758515500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe880ffc80 0x7fbe8819a630 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00d950 tx=0x7fbe7c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe857fa700 1 -- 172.21.15.64:0/2758515500 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c009940 con 0x7fbe880ffc80 2023-07-18T03:24:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.907+0000 7fbe857fa700 1 -- 172.21.15.64:0/2758515500 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe7c010460 con 0x7fbe880ffc80 2023-07-18T03:24:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.908+0000 7fbe8e26a700 1 -- 172.21.15.64:0/2758515500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8819da30 con 0x7fbe880ffc80 2023-07-18T03:24:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.908+0000 7fbe857fa700 1 -- 172.21.15.64:0/2758515500 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c00f620 con 0x7fbe880ffc80 2023-07-18T03:24:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.908+0000 7fbe8e26a700 1 -- 172.21.15.64:0/2758515500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8819df30 con 0x7fbe880ffc80 2023-07-18T03:24:55.910 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.909+0000 7fbe8e26a700 1 -- 172.21.15.64:0/2758515500 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe88106060 con 0x7fbe880ffc80 2023-07-18T03:24:55.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.909+0000 7fbe857fa700 1 -- 172.21.15.64:0/2758515500 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7fbe7c0108d0 con 0x7fbe880ffc80 2023-07-18T03:24:55.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.910+0000 7fbe857fa700 1 --2- 172.21.15.64:0/2758515500 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe700a10c0 0x7fbe700a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:24:55.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.910+0000 7fbe857fa700 1 -- 172.21.15.64:0/2758515500 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fbe7c0cacd0 con 0x7fbe880ffc80 2023-07-18T03:24:55.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.910+0000 7fbe87fff700 1 --2- 172.21.15.64:0/2758515500 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe700a10c0 0x7fbe700a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:24:55.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.911+0000 7fbe87fff700 1 --2- 172.21.15.64:0/2758515500 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe700a10c0 0x7fbe700a3570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800b5c0 tx=0x7fbe78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:24:55.911 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:55.915+0000 7fbe857fa700 1 -- 172.21.15.64:0/2758515500 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbe7c095e90 con 0x7fbe880ffc80 2023-07-18T03:24:56.116 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:24:56.120+0000 7fbe8e26a700 1 -- 172.21.15.64:0/2758515500 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe88060950 con 0x7fbe700a10c0 2023-07-18T03:24:56.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:56 smithi134 ceph-mon[98216]: osdmap e9: 1 total, 1 up, 1 in 2023-07-18T03:24:56.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:56 smithi134 ceph-mon[98216]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:24:56.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:56 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:24:56.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:56 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:24:56.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:56 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:56.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:56 smithi040 ceph-mon[96439]: osdmap e9: 1 total, 1 up, 1 in 2023-07-18T03:24:56.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:56 smithi040 ceph-mon[96439]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:24:56.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:56 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:24:56.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:56 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:24:56.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:56 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:56.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:56 smithi064 ceph-mon[98059]: osdmap e9: 1 total, 1 up, 1 in 2023-07-18T03:24:56.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:56 smithi064 ceph-mon[98059]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:24:56.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:56 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:24:56.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:56 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:24:56.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:56 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:24:57.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:57 smithi134 ceph-mon[98216]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:24:57.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:57 smithi040 ceph-mon[96439]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:24:57.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:57 smithi064 ceph-mon[98059]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:24:58.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:58 smithi134 ceph-mon[98216]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:24:58.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:58 smithi134 ceph-mon[98216]: from='client.? 172.21.15.64:0/1107763399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049"}]: dispatch 2023-07-18T03:24:58.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:58 smithi134 ceph-mon[98216]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049"}]: dispatch 2023-07-18T03:24:58.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:58 smithi134 ceph-mon[98216]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049"}]': finished 2023-07-18T03:24:58.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:58 smithi134 ceph-mon[98216]: osdmap e10: 2 total, 1 up, 2 in 2023-07-18T03:24:58.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:24:58.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:58 smithi040 ceph-mon[96439]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:24:58.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:58 smithi040 ceph-mon[96439]: from='client.? 172.21.15.64:0/1107763399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049"}]: dispatch 2023-07-18T03:24:58.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:58 smithi040 ceph-mon[96439]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049"}]: dispatch 2023-07-18T03:24:58.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:58 smithi040 ceph-mon[96439]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049"}]': finished 2023-07-18T03:24:58.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:58 smithi040 ceph-mon[96439]: osdmap e10: 2 total, 1 up, 2 in 2023-07-18T03:24:58.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:24:58.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:58 smithi064 ceph-mon[98059]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:24:58.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:58 smithi064 ceph-mon[98059]: from='client.? 172.21.15.64:0/1107763399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049"}]: dispatch 2023-07-18T03:24:58.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:58 smithi064 ceph-mon[98059]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049"}]: dispatch 2023-07-18T03:24:58.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:58 smithi064 ceph-mon[98059]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049"}]': finished 2023-07-18T03:24:58.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:58 smithi064 ceph-mon[98059]: osdmap e10: 2 total, 1 up, 2 in 2023-07-18T03:24:58.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:24:59.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:24:59 smithi040 ceph-mon[96439]: from='client.? 172.21.15.64:0/2061828941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:24:59.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:24:59 smithi064 ceph-mon[98059]: from='client.? 172.21.15.64:0/2061828941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:25:00.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:24:59 smithi134 ceph-mon[98216]: from='client.? 172.21.15.64:0/2061828941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:25:00.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:00 smithi064 ceph-mon[98059]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:01.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:00 smithi134 ceph-mon[98216]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:01.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:00 smithi040 ceph-mon[96439]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:02.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:02 smithi134 ceph-mon[98216]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:02.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:02 smithi040 ceph-mon[96439]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:02.830 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:02 smithi064 ceph-mon[98059]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:03.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:03 smithi064 ceph-mon[98059]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:04.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:03 smithi134 ceph-mon[98216]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:04.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:03 smithi040 ceph-mon[96439]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:05.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:05 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-18T03:25:05.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:05 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:05.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:05 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-18T03:25:05.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:05 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:05.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:05 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-18T03:25:05.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:05 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:06.412 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:06 smithi064 ceph-mon[98059]: Deploying daemon osd.1 on smithi064 2023-07-18T03:25:06.413 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:06 smithi064 ceph-mon[98059]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:06.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:06 smithi134 ceph-mon[98216]: Deploying daemon osd.1 on smithi064 2023-07-18T03:25:06.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:06 smithi134 ceph-mon[98216]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:06.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:06 smithi040 ceph-mon[96439]: Deploying daemon osd.1 on smithi064 2023-07-18T03:25:06.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:06 smithi040 ceph-mon[96439]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:08.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:07 smithi134 ceph-mon[98216]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:08.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:07 smithi040 ceph-mon[96439]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:08.091 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:07 smithi064 ceph-mon[98059]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:10.577 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:10 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:10.577 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:10 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:10.578 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:10 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:10.578 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:10 smithi064 ceph-mon[98059]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:10.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:10 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:10.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:10 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:10.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:10 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:10.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:10 smithi134 ceph-mon[98216]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:10.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:10 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:10.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:10 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:10.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:10 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:10.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:10 smithi040 ceph-mon[96439]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:12.192 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:12.195+0000 7fbe857fa700 1 -- 172.21.15.64:0/2758515500 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbe88060950 con 0x7fbe700a10c0 2023-07-18T03:25:12.192 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 1 on host 'smithi064' 2023-07-18T03:25:12.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:12.197+0000 7fbe8e26a700 1 -- 172.21.15.64:0/2758515500 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe700a10c0 msgr2=0x7fbe700a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:12.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:12.197+0000 7fbe8e26a700 1 --2- 172.21.15.64:0/2758515500 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe700a10c0 0x7fbe700a3570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800b5c0 tx=0x7fbe78005fb0 comp rx=0 tx=0).stop 2023-07-18T03:25:12.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:12.197+0000 7fbe8e26a700 1 -- 172.21.15.64:0/2758515500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe880ffc80 msgr2=0x7fbe8819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:12.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:12.197+0000 7fbe8e26a700 1 --2- 172.21.15.64:0/2758515500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe880ffc80 0x7fbe8819a630 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00d950 tx=0x7fbe7c00dc60 comp rx=0 tx=0).stop 2023-07-18T03:25:12.194 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:12.197+0000 7fbe8e26a700 1 -- 172.21.15.64:0/2758515500 shutdown_connections 2023-07-18T03:25:12.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:12.198+0000 7fbe8e26a700 1 --2- 172.21.15.64:0/2758515500 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe700a10c0 0x7fbe700a3570 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:12.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:12.198+0000 7fbe8e26a700 1 --2- 172.21.15.64:0/2758515500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe88100610 0x7fbe8819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:12.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:12.198+0000 7fbe8e26a700 1 --2- 172.21.15.64:0/2758515500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe880fea80 0x7fbe88068b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:12.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:12.198+0000 7fbe8e26a700 1 --2- 172.21.15.64:0/2758515500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe880ffc80 0x7fbe8819a630 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:12.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:12.198+0000 7fbe8e26a700 1 -- 172.21.15.64:0/2758515500 >> 172.21.15.64:0/2758515500 conn(0x7fbe880fa0b0 msgr2=0x7fbe881031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:12.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:12.198+0000 7fbe8e26a700 1 -- 172.21.15.64:0/2758515500 shutdown_connections 2023-07-18T03:25:12.195 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:12.198+0000 7fbe8e26a700 1 -- 172.21.15.64:0/2758515500 wait complete. 2023-07-18T03:25:12.608 DEBUG:teuthology.orchestra.run.smithi064:osd.1> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.1.service 2023-07-18T03:25:12.610 INFO:tasks.cephadm:Deploying osd.2 on smithi134 with /dev/vg_nvme/lv_4... 2023-07-18T03:25:12.611 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-18T03:25:12.642 INFO:journalctl@ceph.osd.1.smithi064.stdout:-- Logs begin at Tue 2023-07-18 03:07:19 UTC. -- 2023-07-18T03:25:12.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:12 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:12.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:12 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:12.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:12 smithi040 ceph-mon[96439]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:12.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:12 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:12.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:12 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:12.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:12 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:12.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:12 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:12.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:12 smithi064 ceph-mon[98059]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:12.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:12 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:12.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:12 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:13.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:12 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:13.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:12 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:13.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:12 smithi134 ceph-mon[98216]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:13.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:12 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:13.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:12 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:13.171 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:25:13.658 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:13 smithi134 ceph-mon[98216]: from='osd.1 [v2:172.21.15.64:6800/2992805245,v1:172.21.15.64:6801/2992805245]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-18T03:25:13.658 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:13 smithi134 ceph-mon[98216]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-18T03:25:13.689 INFO:journalctl@ceph.osd.1.smithi064.stdout:Jul 18 03:25:13 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-1[103095]: 2023-07-18T03:25:13.420+0000 7f203282a640 -1 osd.1 0 log_to_monitors true 2023-07-18T03:25:13.690 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:13 smithi064 ceph-mon[98059]: from='osd.1 [v2:172.21.15.64:6800/2992805245,v1:172.21.15.64:6801/2992805245]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-18T03:25:13.690 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:13 smithi064 ceph-mon[98059]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-18T03:25:14.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:13 smithi040 ceph-mon[96439]: from='osd.1 [v2:172.21.15.64:6800/2992805245,v1:172.21.15.64:6801/2992805245]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-18T03:25:14.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:13 smithi040 ceph-mon[96439]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-18T03:25:14.621 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-18T03:25:14.621 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-18T03:25:14.621 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2023-07-18T03:25:14.622 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2023-07-18T03:25:14.622 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0108082 s, 970 MB/s 2023-07-18T03:25:14.622 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2023-07-18T03:25:14.850 INFO:journalctl@ceph.osd.1.smithi064.stdout:Jul 18 03:25:14 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-1[103095]: 2023-07-18T03:25:14.602+0000 7f20276a1700 -1 osd.1 0 waiting for initial osdmap 2023-07-18T03:25:14.851 INFO:journalctl@ceph.osd.1.smithi064.stdout:Jul 18 03:25:14 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-1[103095]: 2023-07-18T03:25:14.605+0000 7f2023c97700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:25:14.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:14 smithi064 ceph-mon[98059]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-18T03:25:14.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:14 smithi064 ceph-mon[98059]: osdmap e11: 2 total, 1 up, 2 in 2023-07-18T03:25:14.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:14 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:25:14.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:14 smithi064 ceph-mon[98059]: from='osd.1 [v2:172.21.15.64:6800/2992805245,v1:172.21.15.64:6801/2992805245]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:25:14.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:14 smithi064 ceph-mon[98059]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:25:14.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:14 smithi064 ceph-mon[98059]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:14.986 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:14 smithi134 ceph-mon[98216]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-18T03:25:14.986 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:14 smithi134 ceph-mon[98216]: osdmap e11: 2 total, 1 up, 2 in 2023-07-18T03:25:14.986 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:14 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:25:14.987 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:14 smithi134 ceph-mon[98216]: from='osd.1 [v2:172.21.15.64:6800/2992805245,v1:172.21.15.64:6801/2992805245]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:25:14.987 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:14 smithi134 ceph-mon[98216]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:25:14.987 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:14 smithi134 ceph-mon[98216]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:15.011 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi134:vg_nvme/lv_4 2023-07-18T03:25:15.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:14 smithi040 ceph-mon[96439]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-18T03:25:15.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:14 smithi040 ceph-mon[96439]: osdmap e11: 2 total, 1 up, 2 in 2023-07-18T03:25:15.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:14 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:25:15.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:14 smithi040 ceph-mon[96439]: from='osd.1 [v2:172.21.15.64:6800/2992805245,v1:172.21.15.64:6801/2992805245]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:25:15.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:14 smithi040 ceph-mon[96439]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:25:15.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:14 smithi040 ceph-mon[96439]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:15.564 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:25:15.863 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:15 smithi134 ceph-mon[98216]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-07-18T03:25:15.864 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:15 smithi134 ceph-mon[98216]: osdmap e12: 2 total, 1 up, 2 in 2023-07-18T03:25:15.864 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:25:15.864 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:25:15.864 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:15.864 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:15.865 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:25:15.865 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:15.865 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:15.865 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:15.865 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:15.866 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:15.866 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:15.866 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:15.866 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:16.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:15 smithi040 ceph-mon[96439]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-07-18T03:25:16.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:15 smithi040 ceph-mon[96439]: osdmap e12: 2 total, 1 up, 2 in 2023-07-18T03:25:16.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:25:16.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:25:16.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:16.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:16.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:25:16.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:16.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:16.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:16.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:16.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:16.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:16.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:16.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:16.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:15 smithi064 ceph-mon[98059]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-07-18T03:25:16.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:15 smithi064 ceph-mon[98059]: osdmap e12: 2 total, 1 up, 2 in 2023-07-18T03:25:16.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:25:16.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:25:16.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:16.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:16.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:25:16.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:16.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:16.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:16.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:16.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:16.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:16.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:16.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:16.757 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.758+0000 7f6876b2c700 1 -- 172.21.15.134:0/564993640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68700fea80 msgr2=0x7f68700fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:16.757 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.758+0000 7f6876b2c700 1 --2- 172.21.15.134:0/564993640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68700fea80 0x7f68700fee90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6864009990 tx=0x7f6864009ca0 comp rx=0 tx=0).stop 2023-07-18T03:25:16.757 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.758+0000 7f6876b2c700 1 -- 172.21.15.134:0/564993640 shutdown_connections 2023-07-18T03:25:16.757 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.758+0000 7f6876b2c700 1 --2- 172.21.15.134:0/564993640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6870100610 0x7f6870106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:16.757 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.758+0000 7f6876b2c700 1 --2- 172.21.15.134:0/564993640 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68700fea80 0x7f68700fee90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:16.758 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.758+0000 7f6876b2c700 1 --2- 172.21.15.134:0/564993640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68700ffc80 0x7f68701000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:16.758 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.758+0000 7f6876b2c700 1 -- 172.21.15.134:0/564993640 >> 172.21.15.134:0/564993640 conn(0x7f68700fa0b0 msgr2=0x7f68700fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:16.758 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.758+0000 7f6876b2c700 1 -- 172.21.15.134:0/564993640 shutdown_connections 2023-07-18T03:25:16.758 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.759+0000 7f6876b2c700 1 -- 172.21.15.134:0/564993640 wait complete. 2023-07-18T03:25:16.758 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.759+0000 7f6876b2c700 1 Processor -- start 2023-07-18T03:25:16.758 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.759+0000 7f6876b2c700 1 -- start start 2023-07-18T03:25:16.758 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.759+0000 7f6876b2c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68700fea80 0x7f6870068b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:16.759 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.759+0000 7f6876b2c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68700ffc80 0x7f687019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:16.759 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.759+0000 7f6876b2c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6870100610 0x7f687019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:16.759 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.759+0000 7f6876b2c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687019d520 con 0x7f68700ffc80 2023-07-18T03:25:16.759 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.759+0000 7f6876b2c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687019d690 con 0x7f68700fea80 2023-07-18T03:25:16.759 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.759+0000 7f6876b2c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687019d850 con 0x7f6870100610 2023-07-18T03:25:16.759 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.759+0000 7f68750c9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6870100610 0x7f687019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:16.759 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.759+0000 7f68750c9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6870100610 0x7f687019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:48034/0 (socket says 172.21.15.134:48034) 2023-07-18T03:25:16.760 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.759+0000 7f68750c9700 1 -- 172.21.15.134:0/3510575905 learned_addr learned my addr 172.21.15.134:0/3510575905 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-07-18T03:25:16.760 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.759+0000 7f68748c8700 1 --2- 172.21.15.134:0/3510575905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68700fea80 0x7f6870068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:16.760 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.759+0000 7f686ffff700 1 --2- 172.21.15.134:0/3510575905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68700ffc80 0x7f687019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:16.760 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.760+0000 7f68750c9700 1 -- 172.21.15.134:0/3510575905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68700fea80 msgr2=0x7f6870068b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:16.760 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.760+0000 7f68750c9700 1 --2- 172.21.15.134:0/3510575905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68700fea80 0x7f6870068b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:16.760 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.760+0000 7f68750c9700 1 -- 172.21.15.134:0/3510575905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68700ffc80 msgr2=0x7f687019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:16.760 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.760+0000 7f68750c9700 1 --2- 172.21.15.134:0/3510575905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68700ffc80 0x7f687019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:16.761 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.760+0000 7f68750c9700 1 -- 172.21.15.134:0/3510575905 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68640095f0 con 0x7f6870100610 2023-07-18T03:25:16.761 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.760+0000 7f686ffff700 1 --2- 172.21.15.134:0/3510575905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68700ffc80 0x7f687019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:25:16.761 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.760+0000 7f68750c9700 1 --2- 172.21.15.134:0/3510575905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6870100610 0x7f687019cd80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f686800b970 tx=0x7f686800bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:16.761 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.760+0000 7f686dffb700 1 -- 172.21.15.134:0/3510575905 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686800c930 con 0x7f6870100610 2023-07-18T03:25:16.761 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.760+0000 7f686dffb700 1 -- 172.21.15.134:0/3510575905 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6868014440 con 0x7f6870100610 2023-07-18T03:25:16.761 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.760+0000 7f686dffb700 1 -- 172.21.15.134:0/3510575905 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68680075d0 con 0x7f6870100610 2023-07-18T03:25:16.761 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.760+0000 7f6876b2c700 1 -- 172.21.15.134:0/3510575905 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687019db30 con 0x7f6870100610 2023-07-18T03:25:16.761 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.760+0000 7f6876b2c700 1 -- 172.21.15.134:0/3510575905 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687019e060 con 0x7f6870100610 2023-07-18T03:25:16.762 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.761+0000 7f6876b2c700 1 -- 172.21.15.134:0/3510575905 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68701060d0 con 0x7f6870100610 2023-07-18T03:25:16.763 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.762+0000 7f686dffb700 1 -- 172.21.15.134:0/3510575905 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f68680145b0 con 0x7f6870100610 2023-07-18T03:25:16.763 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.763+0000 7f686dffb700 1 --2- 172.21.15.134:0/3510575905 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f68600a11c0 0x7f68600a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:16.763 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.763+0000 7f686dffb700 1 -- 172.21.15.134:0/3510575905 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f6868006e40 con 0x7f6870100610 2023-07-18T03:25:16.763 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.763+0000 7f68748c8700 1 --2- 172.21.15.134:0/3510575905 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f68600a11c0 0x7f68600a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:16.763 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.764+0000 7f68748c8700 1 --2- 172.21.15.134:0/3510575905 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f68600a11c0 0x7f68600a3670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f686400b5c0 tx=0x7f6864005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:16.764 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.767+0000 7f686dffb700 1 -- 172.21.15.134:0/3510575905 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f686804e790 con 0x7f6870100610 2023-07-18T03:25:16.955 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:16.958+0000 7f6876b2c700 1 -- 172.21.15.134:0/3510575905 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6870060950 con 0x7f68600a11c0 2023-07-18T03:25:17.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:16 smithi134 ceph-mon[98216]: purged_snaps scrub starts 2023-07-18T03:25:17.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:16 smithi134 ceph-mon[98216]: purged_snaps scrub ok 2023-07-18T03:25:17.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:16 smithi134 ceph-mon[98216]: Detected new or changed devices on smithi064 2023-07-18T03:25:17.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:16 smithi134 ceph-mon[98216]: Adjusting osd_memory_target on smithi064 to 17168M 2023-07-18T03:25:17.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:16 smithi134 ceph-mon[98216]: osd.1 [v2:172.21.15.64:6800/2992805245,v1:172.21.15.64:6801/2992805245] boot 2023-07-18T03:25:17.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:16 smithi134 ceph-mon[98216]: osdmap e13: 2 total, 2 up, 2 in 2023-07-18T03:25:17.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:16 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:25:17.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:16 smithi134 ceph-mon[98216]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:17.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:16 smithi040 ceph-mon[96439]: purged_snaps scrub starts 2023-07-18T03:25:17.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:16 smithi040 ceph-mon[96439]: purged_snaps scrub ok 2023-07-18T03:25:17.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:16 smithi040 ceph-mon[96439]: Detected new or changed devices on smithi064 2023-07-18T03:25:17.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:16 smithi040 ceph-mon[96439]: Adjusting osd_memory_target on smithi064 to 17168M 2023-07-18T03:25:17.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:16 smithi040 ceph-mon[96439]: osd.1 [v2:172.21.15.64:6800/2992805245,v1:172.21.15.64:6801/2992805245] boot 2023-07-18T03:25:17.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:16 smithi040 ceph-mon[96439]: osdmap e13: 2 total, 2 up, 2 in 2023-07-18T03:25:17.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:16 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:25:17.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:16 smithi040 ceph-mon[96439]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:17.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:16 smithi064 ceph-mon[98059]: purged_snaps scrub starts 2023-07-18T03:25:17.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:16 smithi064 ceph-mon[98059]: purged_snaps scrub ok 2023-07-18T03:25:17.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:16 smithi064 ceph-mon[98059]: Detected new or changed devices on smithi064 2023-07-18T03:25:17.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:16 smithi064 ceph-mon[98059]: Adjusting osd_memory_target on smithi064 to 17168M 2023-07-18T03:25:17.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:16 smithi064 ceph-mon[98059]: osd.1 [v2:172.21.15.64:6800/2992805245,v1:172.21.15.64:6801/2992805245] boot 2023-07-18T03:25:17.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:16 smithi064 ceph-mon[98059]: osdmap e13: 2 total, 2 up, 2 in 2023-07-18T03:25:17.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-18T03:25:17.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:16 smithi064 ceph-mon[98059]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-18T03:25:17.973 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:17 smithi134 ceph-mon[98216]: osdmap e14: 2 total, 2 up, 2 in 2023-07-18T03:25:17.973 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:17 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:25:17.973 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:17 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:25:17.973 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:17 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:18.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:17 smithi040 ceph-mon[96439]: osdmap e14: 2 total, 2 up, 2 in 2023-07-18T03:25:18.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:17 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:25:18.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:17 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:25:18.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:17 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:18.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:17 smithi064 ceph-mon[98059]: osdmap e14: 2 total, 2 up, 2 in 2023-07-18T03:25:18.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:17 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:25:18.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:17 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:25:18.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:17 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:18.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:18 smithi134 ceph-mon[98216]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:25:18.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:18 smithi134 ceph-mon[98216]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:19.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:18 smithi040 ceph-mon[96439]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:25:19.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:18 smithi040 ceph-mon[96439]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:19.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:18 smithi064 ceph-mon[98059]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:25:19.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:18 smithi064 ceph-mon[98059]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:20.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:19 smithi134 ceph-mon[98216]: from='client.? 172.21.15.134:0/3294289924' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e48fbcee-7d01-4dcf-845f-fc64645dca43"}]: dispatch 2023-07-18T03:25:20.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:19 smithi134 ceph-mon[98216]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e48fbcee-7d01-4dcf-845f-fc64645dca43"}]: dispatch 2023-07-18T03:25:20.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:19 smithi134 ceph-mon[98216]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e48fbcee-7d01-4dcf-845f-fc64645dca43"}]': finished 2023-07-18T03:25:20.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:19 smithi134 ceph-mon[98216]: osdmap e15: 3 total, 2 up, 3 in 2023-07-18T03:25:20.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:25:20.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:19 smithi040 ceph-mon[96439]: from='client.? 172.21.15.134:0/3294289924' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e48fbcee-7d01-4dcf-845f-fc64645dca43"}]: dispatch 2023-07-18T03:25:20.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:19 smithi040 ceph-mon[96439]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e48fbcee-7d01-4dcf-845f-fc64645dca43"}]: dispatch 2023-07-18T03:25:20.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:19 smithi040 ceph-mon[96439]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e48fbcee-7d01-4dcf-845f-fc64645dca43"}]': finished 2023-07-18T03:25:20.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:19 smithi040 ceph-mon[96439]: osdmap e15: 3 total, 2 up, 3 in 2023-07-18T03:25:20.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:25:20.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:19 smithi064 ceph-mon[98059]: from='client.? 172.21.15.134:0/3294289924' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e48fbcee-7d01-4dcf-845f-fc64645dca43"}]: dispatch 2023-07-18T03:25:20.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:19 smithi064 ceph-mon[98059]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e48fbcee-7d01-4dcf-845f-fc64645dca43"}]: dispatch 2023-07-18T03:25:20.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:19 smithi064 ceph-mon[98059]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e48fbcee-7d01-4dcf-845f-fc64645dca43"}]': finished 2023-07-18T03:25:20.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:19 smithi064 ceph-mon[98059]: osdmap e15: 3 total, 2 up, 3 in 2023-07-18T03:25:20.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:25:21.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:20 smithi134 ceph-mon[98216]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:21.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:20 smithi134 ceph-mon[98216]: from='client.? 172.21.15.134:0/3551207667' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:25:21.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:20 smithi040 ceph-mon[96439]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:21.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:20 smithi040 ceph-mon[96439]: from='client.? 172.21.15.134:0/3551207667' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:25:21.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:20 smithi064 ceph-mon[98059]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:21.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:20 smithi064 ceph-mon[98059]: from='client.? 172.21.15.134:0/3551207667' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:25:22.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:22 smithi134 ceph-mon[98216]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:22.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:22 smithi040 ceph-mon[96439]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:22.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:22 smithi064 ceph-mon[98059]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:24.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:23 smithi134 ceph-mon[98216]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:24.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:23 smithi040 ceph-mon[96439]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:24.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:23 smithi064 ceph-mon[98059]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:25.890 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:25 smithi134 ceph-mon[98216]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:26.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:25 smithi040 ceph-mon[96439]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:26.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:25 smithi064 ceph-mon[98059]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:27.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:26 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-18T03:25:27.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:26 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:27.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:26 smithi134 ceph-mon[98216]: Deploying daemon osd.2 on smithi134 2023-07-18T03:25:27.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:26 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-18T03:25:27.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:26 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:27.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:26 smithi040 ceph-mon[96439]: Deploying daemon osd.2 on smithi134 2023-07-18T03:25:27.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:26 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-18T03:25:27.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:26 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:27.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:26 smithi064 ceph-mon[98059]: Deploying daemon osd.2 on smithi134 2023-07-18T03:25:28.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:27 smithi134 ceph-mon[98216]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:28.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:27 smithi040 ceph-mon[96439]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:28.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:27 smithi064 ceph-mon[98059]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:30.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:29 smithi040 ceph-mon[96439]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:30.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:29 smithi064 ceph-mon[98059]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:30.116 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:29 smithi134 ceph-mon[98216]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:31.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:31 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:31.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:31 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:31.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:31 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:31.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:31 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:31.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:31 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:31.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:31 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:31.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:31 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:31.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:31 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:31.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:31 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:32.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:32 smithi134 ceph-mon[98216]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:32.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:32.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:32.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:32.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:32.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:32.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:32 smithi040 ceph-mon[96439]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:32.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:32.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:32.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:32.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:32.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:32.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:32 smithi064 ceph-mon[98059]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:32.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:32.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:32.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:32.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:32.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:32.918 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:32.921+0000 7f686dffb700 1 -- 172.21.15.134:0/3510575905 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6870060950 con 0x7f68600a11c0 2023-07-18T03:25:32.919 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 2 on host 'smithi134' 2023-07-18T03:25:32.920 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:32.924+0000 7f6876b2c700 1 -- 172.21.15.134:0/3510575905 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f68600a11c0 msgr2=0x7f68600a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:32.921 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:32.924+0000 7f6876b2c700 1 --2- 172.21.15.134:0/3510575905 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f68600a11c0 0x7f68600a3670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f686400b5c0 tx=0x7f6864005fb0 comp rx=0 tx=0).stop 2023-07-18T03:25:32.921 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:32.924+0000 7f6876b2c700 1 -- 172.21.15.134:0/3510575905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6870100610 msgr2=0x7f687019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:32.921 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:32.924+0000 7f6876b2c700 1 --2- 172.21.15.134:0/3510575905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6870100610 0x7f687019cd80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f686800b970 tx=0x7f686800bc80 comp rx=0 tx=0).stop 2023-07-18T03:25:32.922 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:32.924+0000 7f6876b2c700 1 -- 172.21.15.134:0/3510575905 shutdown_connections 2023-07-18T03:25:32.922 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:32.924+0000 7f6876b2c700 1 --2- 172.21.15.134:0/3510575905 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f68600a11c0 0x7f68600a3670 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:32.922 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:32.924+0000 7f6876b2c700 1 --2- 172.21.15.134:0/3510575905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6870100610 0x7f687019cd80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:32.922 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:32.924+0000 7f6876b2c700 1 --2- 172.21.15.134:0/3510575905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68700fea80 0x7f6870068b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:32.922 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:32.924+0000 7f6876b2c700 1 --2- 172.21.15.134:0/3510575905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68700ffc80 0x7f687019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:32.922 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:32.924+0000 7f6876b2c700 1 -- 172.21.15.134:0/3510575905 >> 172.21.15.134:0/3510575905 conn(0x7f68700fa0b0 msgr2=0x7f6870103210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:32.923 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:32.924+0000 7f6876b2c700 1 -- 172.21.15.134:0/3510575905 shutdown_connections 2023-07-18T03:25:32.923 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:25:32.924+0000 7f6876b2c700 1 -- 172.21.15.134:0/3510575905 wait complete. 2023-07-18T03:25:33.355 DEBUG:teuthology.orchestra.run.smithi134:osd.2> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.2.service 2023-07-18T03:25:33.358 INFO:tasks.cephadm:Deploying osd.3 on smithi040 with /dev/vg_nvme/lv_3... 2023-07-18T03:25:33.359 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-18T03:25:33.391 INFO:journalctl@ceph.osd.2.smithi134.stdout:-- Logs begin at Tue 2023-07-18 03:07:14 UTC. -- 2023-07-18T03:25:33.937 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:25:34.096 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:33 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:34.096 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:33 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:34.097 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:33 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:34.097 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:33 smithi134 ceph-mon[98216]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:34.169 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:33 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:34.170 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:33 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:34.170 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:33 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:34.170 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:33 smithi040 ceph-mon[96439]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:34.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:33 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:34.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:33 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:34.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:33 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:34.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:33 smithi064 ceph-mon[98059]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:34.567 INFO:journalctl@ceph.osd.2.smithi134.stdout:Jul 18 03:25:34 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-2[104040]: 2023-07-18T03:25:34.191+0000 7fba45472640 -1 osd.2 0 log_to_monitors true 2023-07-18T03:25:35.241 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:34 smithi134 ceph-mon[98216]: from='osd.2 [v2:172.21.15.134:6800/1877339886,v1:172.21.15.134:6801/1877339886]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-18T03:25:35.241 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:34 smithi134 ceph-mon[98216]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-18T03:25:35.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:34 smithi040 ceph-mon[96439]: from='osd.2 [v2:172.21.15.134:6800/1877339886,v1:172.21.15.134:6801/1877339886]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-18T03:25:35.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:34 smithi040 ceph-mon[96439]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-18T03:25:35.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:34 smithi064 ceph-mon[98059]: from='osd.2 [v2:172.21.15.134:6800/1877339886,v1:172.21.15.134:6801/1877339886]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-18T03:25:35.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:34 smithi064 ceph-mon[98059]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-18T03:25:35.447 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-18T03:25:35.447 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-18T03:25:35.447 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2023-07-18T03:25:35.448 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2023-07-18T03:25:35.448 INFO:teuthology.orchestra.run.smithi040.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124372 s, 843 MB/s 2023-07-18T03:25:35.448 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 2023-07-18T03:25:35.448 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2023-07-18T03:25:35.885 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi040:vg_nvme/lv_3 2023-07-18T03:25:36.316 INFO:journalctl@ceph.osd.2.smithi134.stdout:Jul 18 03:25:35 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-2[104040]: 2023-07-18T03:25:35.934+0000 7fba3a2e9700 -1 osd.2 0 waiting for initial osdmap 2023-07-18T03:25:36.317 INFO:journalctl@ceph.osd.2.smithi134.stdout:Jul 18 03:25:35 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-2[104040]: 2023-07-18T03:25:35.938+0000 7fba348db700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:25:36.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:35 smithi134 ceph-mon[98216]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-18T03:25:36.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:35 smithi134 ceph-mon[98216]: osdmap e16: 3 total, 2 up, 3 in 2023-07-18T03:25:36.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:25:36.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:35 smithi134 ceph-mon[98216]: from='osd.2 [v2:172.21.15.134:6800/1877339886,v1:172.21.15.134:6801/1877339886]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:25:36.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:35 smithi134 ceph-mon[98216]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:25:36.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:35 smithi134 ceph-mon[98216]: Detected new or changed devices on smithi134 2023-07-18T03:25:36.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:36.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:36.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:25:36.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:35 smithi134 ceph-mon[98216]: Adjusting osd_memory_target on smithi134 to 17168M 2023-07-18T03:25:36.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:36.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:36.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:36.319 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:36.319 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:35 smithi134 ceph-mon[98216]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:36.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:35 smithi040 ceph-mon[96439]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-18T03:25:36.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:35 smithi040 ceph-mon[96439]: osdmap e16: 3 total, 2 up, 3 in 2023-07-18T03:25:36.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:25:36.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:35 smithi040 ceph-mon[96439]: from='osd.2 [v2:172.21.15.134:6800/1877339886,v1:172.21.15.134:6801/1877339886]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:25:36.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:35 smithi040 ceph-mon[96439]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:25:36.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:35 smithi040 ceph-mon[96439]: Detected new or changed devices on smithi134 2023-07-18T03:25:36.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:36.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:36.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:25:36.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:35 smithi040 ceph-mon[96439]: Adjusting osd_memory_target on smithi134 to 17168M 2023-07-18T03:25:36.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:36.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:36.333 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:36.333 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:36.333 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:35 smithi040 ceph-mon[96439]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:36.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:35 smithi064 ceph-mon[98059]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-18T03:25:36.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:35 smithi064 ceph-mon[98059]: osdmap e16: 3 total, 2 up, 3 in 2023-07-18T03:25:36.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:25:36.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:35 smithi064 ceph-mon[98059]: from='osd.2 [v2:172.21.15.134:6800/1877339886,v1:172.21.15.134:6801/1877339886]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:25:36.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:35 smithi064 ceph-mon[98059]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:25:36.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:35 smithi064 ceph-mon[98059]: Detected new or changed devices on smithi134 2023-07-18T03:25:36.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:36.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:36.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:25:36.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:35 smithi064 ceph-mon[98059]: Adjusting osd_memory_target on smithi134 to 17168M 2023-07-18T03:25:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:36.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:35 smithi064 ceph-mon[98059]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:36.445 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:25:37.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:36 smithi134 ceph-mon[98216]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-18T03:25:37.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:36 smithi134 ceph-mon[98216]: osdmap e17: 3 total, 2 up, 3 in 2023-07-18T03:25:37.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:36 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:25:37.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:36 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:25:37.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:36 smithi040 ceph-mon[96439]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-18T03:25:37.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:36 smithi040 ceph-mon[96439]: osdmap e17: 3 total, 2 up, 3 in 2023-07-18T03:25:37.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:36 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:25:37.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:36 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:25:37.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:36 smithi064 ceph-mon[98059]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-18T03:25:37.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:36 smithi064 ceph-mon[98059]: osdmap e17: 3 total, 2 up, 3 in 2023-07-18T03:25:37.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:36 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:25:37.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:36 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:25:37.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.652+0000 7fa030510700 1 -- 172.21.15.40:0/1810364276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa028100610 msgr2=0x7fa028106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:37.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.652+0000 7fa030510700 1 --2- 172.21.15.40:0/1810364276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa028100610 0x7fa028106ed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa02000b0a0 tx=0x7fa02000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:25:37.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.652+0000 7fa030510700 1 -- 172.21.15.40:0/1810364276 shutdown_connections 2023-07-18T03:25:37.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.652+0000 7fa030510700 1 --2- 172.21.15.40:0/1810364276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa028100610 0x7fa028106ed0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.652+0000 7fa030510700 1 --2- 172.21.15.40:0/1810364276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0280fea80 0x7fa0280fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.652+0000 7fa030510700 1 --2- 172.21.15.40:0/1810364276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0280ffc80 0x7fa0281000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.652+0000 7fa030510700 1 -- 172.21.15.40:0/1810364276 >> 172.21.15.40:0/1810364276 conn(0x7fa0280fa0b0 msgr2=0x7fa0280fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.652+0000 7fa030510700 1 -- 172.21.15.40:0/1810364276 shutdown_connections 2023-07-18T03:25:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.652+0000 7fa030510700 1 -- 172.21.15.40:0/1810364276 wait complete. 2023-07-18T03:25:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa030510700 1 Processor -- start 2023-07-18T03:25:37.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa030510700 1 -- start start 2023-07-18T03:25:37.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa030510700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0280fea80 0x7fa0281948b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:37.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa030510700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0280ffc80 0x7fa028194df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:37.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa030510700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa028100610 0x7fa02819be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:37.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa030510700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02819c440 con 0x7fa028100610 2023-07-18T03:25:37.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa030510700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02819c580 con 0x7fa0280fea80 2023-07-18T03:25:37.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa030510700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02819c710 con 0x7fa0280ffc80 2023-07-18T03:25:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa02eaad700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa028100610 0x7fa02819be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa02eaad700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa028100610 0x7fa02819be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55280/0 (socket says 172.21.15.40:55280) 2023-07-18T03:25:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa02eaad700 1 -- 172.21.15.40:0/1124693294 learned_addr learned my addr 172.21.15.40:0/1124693294 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:25:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa02daab700 1 --2- 172.21.15.40:0/1124693294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0280ffc80 0x7fa028194df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa02e2ac700 1 --2- 172.21.15.40:0/1124693294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0280fea80 0x7fa0281948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa02eaad700 1 -- 172.21.15.40:0/1124693294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0280ffc80 msgr2=0x7fa028194df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa02eaad700 1 --2- 172.21.15.40:0/1124693294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0280ffc80 0x7fa028194df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa02eaad700 1 -- 172.21.15.40:0/1124693294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0280fea80 msgr2=0x7fa0281948b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa02eaad700 1 --2- 172.21.15.40:0/1124693294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0280fea80 0x7fa0281948b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa02eaad700 1 -- 172.21.15.40:0/1124693294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa020009d00 con 0x7fa028100610 2023-07-18T03:25:37.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa02e2ac700 1 --2- 172.21.15.40:0/1124693294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0280fea80 0x7fa0281948b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:25:37.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.653+0000 7fa02daab700 1 --2- 172.21.15.40:0/1124693294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0280ffc80 0x7fa028194df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:25:37.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.654+0000 7fa02eaad700 1 --2- 172.21.15.40:0/1124693294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa028100610 0x7fa02819be70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa020006070 tx=0x7fa020004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:37.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.654+0000 7fa01b7fe700 1 -- 172.21.15.40:0/1124693294 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02000e040 con 0x7fa028100610 2023-07-18T03:25:37.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.654+0000 7fa01b7fe700 1 -- 172.21.15.40:0/1124693294 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa020004020 con 0x7fa028100610 2023-07-18T03:25:37.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.654+0000 7fa030510700 1 -- 172.21.15.40:0/1124693294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa02819c990 con 0x7fa028100610 2023-07-18T03:25:37.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.654+0000 7fa01b7fe700 1 -- 172.21.15.40:0/1124693294 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02001b6b0 con 0x7fa028100610 2023-07-18T03:25:37.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.654+0000 7fa030510700 1 -- 172.21.15.40:0/1124693294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02819ce50 con 0x7fa028100610 2023-07-18T03:25:37.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.655+0000 7fa030510700 1 -- 172.21.15.40:0/1124693294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa02810a2a0 con 0x7fa028100610 2023-07-18T03:25:37.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.656+0000 7fa01b7fe700 1 -- 172.21.15.40:0/1124693294 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7fa020004470 con 0x7fa028100610 2023-07-18T03:25:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.657+0000 7fa01b7fe700 1 --2- 172.21.15.40:0/1124693294 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0140a1140 0x7fa0140a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.657+0000 7fa02e2ac700 1 --2- 172.21.15.40:0/1124693294 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0140a1140 0x7fa0140a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.657+0000 7fa01b7fe700 1 -- 172.21.15.40:0/1124693294 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2851+0+0 (secure 0 0 0) 0x7fa0200ccf60 con 0x7fa028100610 2023-07-18T03:25:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.657+0000 7fa02e2ac700 1 --2- 172.21.15.40:0/1124693294 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0140a1140 0x7fa0140a35f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa01c005fd0 tx=0x7fa01c005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:37.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.660+0000 7fa01b7fe700 1 -- 172.21.15.40:0/1124693294 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa020097cd0 con 0x7fa028100610 2023-07-18T03:25:37.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:37.865+0000 7fa030510700 1 -- 172.21.15.40:0/1124693294 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa028067fd0 con 0x7fa0140a1140 2023-07-18T03:25:38.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:38 smithi134 ceph-mon[98216]: purged_snaps scrub starts 2023-07-18T03:25:38.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:38 smithi134 ceph-mon[98216]: purged_snaps scrub ok 2023-07-18T03:25:38.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:38 smithi134 ceph-mon[98216]: osd.2 [v2:172.21.15.134:6800/1877339886,v1:172.21.15.134:6801/1877339886] boot 2023-07-18T03:25:38.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:38 smithi134 ceph-mon[98216]: osdmap e18: 3 total, 3 up, 3 in 2023-07-18T03:25:38.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:38 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:25:38.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:38 smithi134 ceph-mon[98216]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:38.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:38 smithi040 ceph-mon[96439]: purged_snaps scrub starts 2023-07-18T03:25:38.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:38 smithi040 ceph-mon[96439]: purged_snaps scrub ok 2023-07-18T03:25:38.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:38 smithi040 ceph-mon[96439]: osd.2 [v2:172.21.15.134:6800/1877339886,v1:172.21.15.134:6801/1877339886] boot 2023-07-18T03:25:38.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:38 smithi040 ceph-mon[96439]: osdmap e18: 3 total, 3 up, 3 in 2023-07-18T03:25:38.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:38 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:25:38.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:38 smithi040 ceph-mon[96439]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:38.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:38 smithi064 ceph-mon[98059]: purged_snaps scrub starts 2023-07-18T03:25:38.605 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:38 smithi064 ceph-mon[98059]: purged_snaps scrub ok 2023-07-18T03:25:38.605 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:38 smithi064 ceph-mon[98059]: osd.2 [v2:172.21.15.134:6800/1877339886,v1:172.21.15.134:6801/1877339886] boot 2023-07-18T03:25:38.606 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:38 smithi064 ceph-mon[98059]: osdmap e18: 3 total, 3 up, 3 in 2023-07-18T03:25:38.606 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:38 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-18T03:25:38.606 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:38 smithi064 ceph-mon[98059]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-18T03:25:39.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:25:39.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:39 smithi040 ceph-mon[96439]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:25:39.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:25:39.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:25:39.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:39.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-18T03:25:39.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:39 smithi040 ceph-mon[96439]: osdmap e19: 3 total, 3 up, 3 in 2023-07-18T03:25:39.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:25:39.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:25:39.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:39 smithi134 ceph-mon[98216]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:25:39.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:25:39.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:25:39.568 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:39.568 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-18T03:25:39.568 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:39 smithi134 ceph-mon[98216]: osdmap e19: 3 total, 3 up, 3 in 2023-07-18T03:25:39.569 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:25:39.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:25:39.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:39 smithi064 ceph-mon[98059]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:25:39.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:25:39.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:25:39.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:39.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-18T03:25:39.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:39 smithi064 ceph-mon[98059]: osdmap e19: 3 total, 3 up, 3 in 2023-07-18T03:25:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:25:40.079 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jul 18 03:25:39 smithi040 sudo[112502]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-18T03:25:40.079 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jul 18 03:25:39 smithi040 sudo[112502]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:25:40.079 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jul 18 03:25:39 smithi040 sudo[112502]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:25:40.079 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jul 18 03:25:39 smithi040 sudo[112502]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:25:40.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:40 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-18T03:25:40.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:40 smithi134 ceph-mon[98216]: osdmap e20: 3 total, 3 up, 3 in 2023-07-18T03:25:40.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:40 smithi134 ceph-mon[98216]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:25:40.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:40 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-18T03:25:40.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:40 smithi040 ceph-mon[96439]: osdmap e20: 3 total, 3 up, 3 in 2023-07-18T03:25:40.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:40 smithi040 ceph-mon[96439]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:25:40.580 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jul 18 03:25:40 smithi040 sudo[112543]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-18T03:25:40.580 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jul 18 03:25:40 smithi040 sudo[112543]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:25:40.580 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jul 18 03:25:40 smithi040 sudo[112543]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:25:40.580 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jul 18 03:25:40 smithi040 sudo[112543]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:25:40.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:40 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-18T03:25:40.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:40 smithi064 ceph-mon[98059]: osdmap e20: 3 total, 3 up, 3 in 2023-07-18T03:25:40.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:40 smithi064 ceph-mon[98059]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:25:41.100 INFO:journalctl@ceph.osd.1.smithi064.stdout:Jul 18 03:25:40 smithi064 sudo[105439]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-18T03:25:41.100 INFO:journalctl@ceph.osd.1.smithi064.stdout:Jul 18 03:25:40 smithi064 sudo[105439]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:25:41.100 INFO:journalctl@ceph.osd.1.smithi064.stdout:Jul 18 03:25:40 smithi064 sudo[105439]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:25:41.100 INFO:journalctl@ceph.osd.1.smithi064.stdout:Jul 18 03:25:40 smithi064 sudo[105439]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:25:41.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:41.320+0000 7fa01b7fe700 1 -- 172.21.15.40:0/1124693294 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa0200980a0 con 0x7fa028100610 2023-07-18T03:25:41.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:41 smithi134 ceph-mon[98216]: osdmap e21: 3 total, 3 up, 3 in 2023-07-18T03:25:41.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:41 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:41.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:41 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3411251309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "571c77cb-1acd-4d6f-a297-529dd5de162d"}]: dispatch 2023-07-18T03:25:41.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:41 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3411251309' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "571c77cb-1acd-4d6f-a297-529dd5de162d"}]': finished 2023-07-18T03:25:41.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:41 smithi134 ceph-mon[98216]: osdmap e22: 4 total, 3 up, 4 in 2023-07-18T03:25:41.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:41 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:25:41.568 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:41 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:41.568 INFO:journalctl@ceph.osd.2.smithi134.stdout:Jul 18 03:25:41 smithi134 sudo[106305]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-18T03:25:41.568 INFO:journalctl@ceph.osd.2.smithi134.stdout:Jul 18 03:25:41 smithi134 sudo[106305]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:25:41.568 INFO:journalctl@ceph.osd.2.smithi134.stdout:Jul 18 03:25:41 smithi134 sudo[106305]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:25:41.568 INFO:journalctl@ceph.osd.2.smithi134.stdout:Jul 18 03:25:41 smithi134 sudo[106305]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:25:41.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:41 smithi040 ceph-mon[96439]: osdmap e21: 3 total, 3 up, 3 in 2023-07-18T03:25:41.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:41 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:41.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:41 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3411251309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "571c77cb-1acd-4d6f-a297-529dd5de162d"}]: dispatch 2023-07-18T03:25:41.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:41 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3411251309' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "571c77cb-1acd-4d6f-a297-529dd5de162d"}]': finished 2023-07-18T03:25:41.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:41 smithi040 ceph-mon[96439]: osdmap e22: 4 total, 3 up, 4 in 2023-07-18T03:25:41.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:41 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:25:41.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:41 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:41.600 INFO:journalctl@ceph.osd.1.smithi064.stdout:Jul 18 03:25:41 smithi064 sudo[105442]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-18T03:25:41.601 INFO:journalctl@ceph.osd.1.smithi064.stdout:Jul 18 03:25:41 smithi064 sudo[105442]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:25:41.601 INFO:journalctl@ceph.osd.1.smithi064.stdout:Jul 18 03:25:41 smithi064 sudo[105442]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:25:41.601 INFO:journalctl@ceph.osd.1.smithi064.stdout:Jul 18 03:25:41 smithi064 sudo[105442]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:25:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:41 smithi064 ceph-mon[98059]: osdmap e21: 3 total, 3 up, 3 in 2023-07-18T03:25:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:41 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:41.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:41 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3411251309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "571c77cb-1acd-4d6f-a297-529dd5de162d"}]: dispatch 2023-07-18T03:25:41.603 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:41 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3411251309' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "571c77cb-1acd-4d6f-a297-529dd5de162d"}]': finished 2023-07-18T03:25:41.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:41 smithi064 ceph-mon[98059]: osdmap e22: 4 total, 3 up, 4 in 2023-07-18T03:25:41.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:41 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:25:41.604 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:41 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:42.066 INFO:journalctl@ceph.osd.2.smithi134.stdout:Jul 18 03:25:41 smithi134 sudo[106308]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-18T03:25:42.066 INFO:journalctl@ceph.osd.2.smithi134.stdout:Jul 18 03:25:41 smithi134 sudo[106308]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:25:42.067 INFO:journalctl@ceph.osd.2.smithi134.stdout:Jul 18 03:25:41 smithi134 sudo[106308]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:25:42.067 INFO:journalctl@ceph.osd.2.smithi134.stdout:Jul 18 03:25:41 smithi134 sudo[106308]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:25:42.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:42 smithi040 sudo[112844]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-18T03:25:42.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:42 smithi040 sudo[112844]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:25:42.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:42 smithi040 sudo[112844]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:25:42.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:42 smithi040 sudo[112844]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:25:43.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:42 smithi040 sudo[113032]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-18T03:25:43.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:42 smithi040 sudo[113032]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:25:43.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:42 smithi040 sudo[113032]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:25:43.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:42 smithi040 sudo[113032]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:25:43.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:42 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2407715915' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:25:43.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:42 smithi040 ceph-mon[96439]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-18T03:25:43.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:43.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:42 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2407715915' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:25:43.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:42 smithi064 ceph-mon[98059]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-18T03:25:43.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:42 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:43.154 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:42 smithi134 sudo[106311]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-18T03:25:43.154 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:42 smithi134 sudo[106311]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:25:43.154 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:42 smithi134 sudo[106311]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:25:43.154 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:42 smithi134 sudo[106311]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:25:43.568 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:43 smithi134 sudo[106314]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-18T03:25:43.569 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:43 smithi134 sudo[106314]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:25:43.569 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:43 smithi134 sudo[106314]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:25:43.569 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:43 smithi134 sudo[106314]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:25:43.569 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:43 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2407715915' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:25:43.569 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:43 smithi134 ceph-mon[98216]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-18T03:25:43.569 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:43 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:43.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:43 smithi064 sudo[105445]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-18T03:25:43.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:43 smithi064 sudo[105445]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:25:43.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:43 smithi064 sudo[105445]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:25:43.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:43 smithi064 sudo[105445]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:25:44.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:44 smithi134 ceph-mon[98216]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:25:44.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:44 smithi134 ceph-mon[98216]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:25:44.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:44 smithi134 ceph-mon[98216]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:25:44.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:25:44.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:25:44.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:25:44.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:44 smithi134 ceph-mon[98216]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:25:44.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:25:44.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:25:44.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:25:44.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:44 smithi134 ceph-mon[98216]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:25:44.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:25:44.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:25:44.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:25:44.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:44 smithi040 ceph-mon[96439]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:25:44.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:44 smithi040 ceph-mon[96439]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:25:44.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:44 smithi040 ceph-mon[96439]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:25:44.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:25:44.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:25:44.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:25:44.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:44 smithi040 ceph-mon[96439]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:25:44.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:25:44.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:25:44.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:25:44.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:44 smithi040 ceph-mon[96439]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:25:44.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:25:44.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:25:44.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:25:44.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 sudo[105448]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-18T03:25:44.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 sudo[105448]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-18T03:25:44.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 sudo[105448]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-18T03:25:44.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 sudo[105448]: pam_unix(sudo:session): session closed for user root 2023-07-18T03:25:44.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 ceph-mon[98059]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:25:44.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 ceph-mon[98059]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:25:44.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 ceph-mon[98059]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:25:44.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:25:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:25:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:25:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 ceph-mon[98059]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:25:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:25:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:25:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:25:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 ceph-mon[98059]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:25:44.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-18T03:25:44.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-18T03:25:44.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-18T03:25:45.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:45 smithi134 ceph-mon[98216]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:25:45.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:45 smithi134 ceph-mon[98216]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:25:45.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:45 smithi134 ceph-mon[98216]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:25:45.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:45 smithi040 ceph-mon[96439]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:25:45.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:45 smithi040 ceph-mon[96439]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:25:45.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:45 smithi040 ceph-mon[96439]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:25:45.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:45 smithi064 ceph-mon[98059]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-18T03:25:45.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:45 smithi064 ceph-mon[98059]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:25:45.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:45 smithi064 ceph-mon[98059]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-18T03:25:46.128 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:46 smithi040 ceph-mon[96439]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:25:46.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:46 smithi134 ceph-mon[98216]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:25:46.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:46 smithi064 ceph-mon[98059]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:25:47.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:47 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-18T03:25:47.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:47 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:47.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:47 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-18T03:25:47.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:47 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:47.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:47 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-18T03:25:47.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:47 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:48.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:48 smithi134 ceph-mon[98216]: Deploying daemon osd.3 on smithi040 2023-07-18T03:25:48.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:48 smithi134 ceph-mon[98216]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-07-18T03:25:48.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:48 smithi040 ceph-mon[96439]: Deploying daemon osd.3 on smithi040 2023-07-18T03:25:48.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:48 smithi040 ceph-mon[96439]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-07-18T03:25:48.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:48 smithi064 ceph-mon[98059]: Deploying daemon osd.3 on smithi040 2023-07-18T03:25:48.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:48 smithi064 ceph-mon[98059]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-07-18T03:25:50.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:49 smithi040 ceph-mon[96439]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-07-18T03:25:50.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:49 smithi064 ceph-mon[98059]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-07-18T03:25:50.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:49 smithi134 ceph-mon[98216]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2023-07-18T03:25:52.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:52 smithi134 ceph-mon[98216]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-07-18T03:25:52.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:52 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:52.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:52 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:52.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:52 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:52.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:52 smithi040 ceph-mon[96439]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-07-18T03:25:52.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:52 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:52.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:52 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:52.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:52 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:52.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:52 smithi064 ceph-mon[98059]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2023-07-18T03:25:52.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:52 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:52.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:52 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:52.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:52 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:54.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:53 smithi040 ceph-mon[96439]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-18T03:25:54.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:53 smithi064 ceph-mon[98059]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-18T03:25:54.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:53 smithi134 ceph-mon[98216]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-18T03:25:55.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:55.012+0000 7fa01b7fe700 1 -- 172.21.15.40:0/1124693294 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa028067fd0 con 0x7fa0140a1140 2023-07-18T03:25:55.009 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 3 on host 'smithi040' 2023-07-18T03:25:55.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:55.015+0000 7fa030510700 1 -- 172.21.15.40:0/1124693294 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0140a1140 msgr2=0x7fa0140a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:55.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:55.015+0000 7fa030510700 1 --2- 172.21.15.40:0/1124693294 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0140a1140 0x7fa0140a35f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa01c005fd0 tx=0x7fa01c005f00 comp rx=0 tx=0).stop 2023-07-18T03:25:55.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:55.015+0000 7fa030510700 1 -- 172.21.15.40:0/1124693294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa028100610 msgr2=0x7fa02819be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:55.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:55.015+0000 7fa030510700 1 --2- 172.21.15.40:0/1124693294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa028100610 0x7fa02819be70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa020006070 tx=0x7fa020004b00 comp rx=0 tx=0).stop 2023-07-18T03:25:55.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:55.015+0000 7fa030510700 1 -- 172.21.15.40:0/1124693294 shutdown_connections 2023-07-18T03:25:55.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:55.015+0000 7fa030510700 1 --2- 172.21.15.40:0/1124693294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa028100610 0x7fa02819be70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:55.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:55.015+0000 7fa030510700 1 --2- 172.21.15.40:0/1124693294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0280fea80 0x7fa0281948b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:55.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:55.015+0000 7fa030510700 1 --2- 172.21.15.40:0/1124693294 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0140a1140 0x7fa0140a35f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:55.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:55.015+0000 7fa030510700 1 --2- 172.21.15.40:0/1124693294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0280ffc80 0x7fa028194df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:55.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:55.015+0000 7fa030510700 1 -- 172.21.15.40:0/1124693294 >> 172.21.15.40:0/1124693294 conn(0x7fa0280fa0b0 msgr2=0x7fa028103160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:55.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:55.015+0000 7fa030510700 1 -- 172.21.15.40:0/1124693294 shutdown_connections 2023-07-18T03:25:55.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:25:55.015+0000 7fa030510700 1 -- 172.21.15.40:0/1124693294 wait complete. 2023-07-18T03:25:55.350 DEBUG:teuthology.orchestra.run.smithi040:osd.3> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.3.service 2023-07-18T03:25:55.352 INFO:tasks.cephadm:Deploying osd.4 on smithi064 with /dev/vg_nvme/lv_3... 2023-07-18T03:25:55.352 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-18T03:25:55.384 INFO:journalctl@ceph.osd.3.smithi040.stdout:-- Logs begin at Tue 2023-07-18 03:07:07 UTC. -- 2023-07-18T03:25:55.927 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:25:56.033 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:56.034 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:56.034 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:56.034 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:56.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:56.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:56.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:56.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:56.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:56.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:56.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:56.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:56.579 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jul 18 03:25:56 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-3[114842]: 2023-07-18T03:25:56.412+0000 7fbf493ea640 -1 osd.3 0 log_to_monitors true 2023-07-18T03:25:56.947 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:56 smithi064 ceph-mon[98059]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-18T03:25:56.947 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:56 smithi064 ceph-mon[98059]: from='osd.3 [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-18T03:25:57.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:56 smithi134 ceph-mon[98216]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-18T03:25:57.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:56 smithi134 ceph-mon[98216]: from='osd.3 [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-18T03:25:57.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:56 smithi040 ceph-mon[96439]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2023-07-18T03:25:57.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:56 smithi040 ceph-mon[96439]: from='osd.3 [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-18T03:25:57.741 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-18T03:25:57.742 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-18T03:25:57.742 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2023-07-18T03:25:57.743 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2023-07-18T03:25:57.743 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111004 s, 945 MB/s 2023-07-18T03:25:57.743 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2023-07-18T03:25:58.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:57 smithi134 ceph-mon[98216]: from='osd.3 [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-18T03:25:58.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:57 smithi134 ceph-mon[98216]: osdmap e23: 4 total, 3 up, 4 in 2023-07-18T03:25:58.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:57 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:25:58.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:57 smithi134 ceph-mon[98216]: from='osd.3 [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2023-07-18T03:25:58.079 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jul 18 03:25:57 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-3[114842]: 2023-07-18T03:25:57.679+0000 7fbf3e261700 -1 osd.3 0 waiting for initial osdmap 2023-07-18T03:25:58.079 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jul 18 03:25:57 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-3[114842]: 2023-07-18T03:25:57.685+0000 7fbf38052700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:25:58.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:57 smithi040 ceph-mon[96439]: from='osd.3 [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-18T03:25:58.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:57 smithi040 ceph-mon[96439]: osdmap e23: 4 total, 3 up, 4 in 2023-07-18T03:25:58.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:57 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:25:58.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:57 smithi040 ceph-mon[96439]: from='osd.3 [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2023-07-18T03:25:58.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:57 smithi064 ceph-mon[98059]: from='osd.3 [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-18T03:25:58.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:57 smithi064 ceph-mon[98059]: osdmap e23: 4 total, 3 up, 4 in 2023-07-18T03:25:58.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:57 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:25:58.087 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:57 smithi064 ceph-mon[98059]: from='osd.3 [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2023-07-18T03:25:58.088 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi064:vg_nvme/lv_3 2023-07-18T03:25:58.649 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:25:59.043 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: from='osd.3 [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2023-07-18T03:25:59.044 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: osdmap e24: 4 total, 3 up, 4 in 2023-07-18T03:25:59.044 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:25:59.044 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:25:59.044 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: Detected new or changed devices on smithi040 2023-07-18T03:25:59.044 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:59.044 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:25:59.044 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:59.045 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:25:59.045 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:25:59.045 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: Adjusting osd_memory_target on smithi040 to 8586M 2023-07-18T03:25:59.045 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:59.046 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:59.046 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:59.046 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:59.046 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:59.046 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:59.046 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:59.046 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:59.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: from='osd.3 [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2023-07-18T03:25:59.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: osdmap e24: 4 total, 3 up, 4 in 2023-07-18T03:25:59.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:25:59.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:25:59.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: Detected new or changed devices on smithi040 2023-07-18T03:25:59.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:59.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:25:59.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:59.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:25:59.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:25:59.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: Adjusting osd_memory_target on smithi040 to 8586M 2023-07-18T03:25:59.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:59.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:59.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:59.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:59.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:59.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:59.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:59.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:59.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: from='osd.3 [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2023-07-18T03:25:59.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: osdmap e24: 4 total, 3 up, 4 in 2023-07-18T03:25:59.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:25:59.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:25:59.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: Detected new or changed devices on smithi040 2023-07-18T03:25:59.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:59.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-18T03:25:59.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:59.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:25:59.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:25:59.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: Adjusting osd_memory_target on smithi040 to 8586M 2023-07-18T03:25:59.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:59.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:59.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:59.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:59.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:25:59.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:25:59.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:25:59.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:25:59.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.844+0000 7f3c586fd700 1 -- 172.21.15.64:0/1947436735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c50100610 msgr2=0x7f3c50106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:59.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.844+0000 7f3c586fd700 1 --2- 172.21.15.64:0/1947436735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c50100610 0x7f3c50106ed0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800b0a0 tx=0x7f3c4800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:25:59.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.844+0000 7f3c586fd700 1 -- 172.21.15.64:0/1947436735 shutdown_connections 2023-07-18T03:25:59.841 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.844+0000 7f3c586fd700 1 --2- 172.21.15.64:0/1947436735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c50100610 0x7f3c50106ed0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:59.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.844+0000 7f3c586fd700 1 --2- 172.21.15.64:0/1947436735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c500fea80 0x7f3c500fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:59.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.844+0000 7f3c586fd700 1 --2- 172.21.15.64:0/1947436735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c500ffc80 0x7f3c501000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:59.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.844+0000 7f3c586fd700 1 -- 172.21.15.64:0/1947436735 >> 172.21.15.64:0/1947436735 conn(0x7f3c500fa0b0 msgr2=0x7f3c500fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:25:59.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.844+0000 7f3c586fd700 1 -- 172.21.15.64:0/1947436735 shutdown_connections 2023-07-18T03:25:59.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.844+0000 7f3c586fd700 1 -- 172.21.15.64:0/1947436735 wait complete. 2023-07-18T03:25:59.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.844+0000 7f3c586fd700 1 Processor -- start 2023-07-18T03:25:59.842 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c586fd700 1 -- start start 2023-07-18T03:25:59.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c586fd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c500fea80 0x7f3c50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:59.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c586fd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c500ffc80 0x7f3c5019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:59.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c586fd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c50100610 0x7f3c5019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:59.844 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c586fd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5019d460 con 0x7f3c500ffc80 2023-07-18T03:25:59.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c586fd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5019d5d0 con 0x7f3c50100610 2023-07-18T03:25:59.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c586fd700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5019d790 con 0x7f3c500fea80 2023-07-18T03:25:59.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c56c9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c50100610 0x7f3c5019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:59.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c56c9a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c50100610 0x7f3c5019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37450/0 (socket says 172.21.15.64:37450) 2023-07-18T03:25:59.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c56c9a700 1 -- 172.21.15.64:0/1894474334 learned_addr learned my addr 172.21.15.64:0/1894474334 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-07-18T03:25:59.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c56499700 1 --2- 172.21.15.64:0/1894474334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c500fea80 0x7f3c50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:59.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c55c98700 1 --2- 172.21.15.64:0/1894474334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c500ffc80 0x7f3c5019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:59.845 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c56c9a700 1 -- 172.21.15.64:0/1894474334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c500fea80 msgr2=0x7f3c50068b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:59.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c56c9a700 1 --2- 172.21.15.64:0/1894474334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c500fea80 0x7f3c50068b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:59.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c56c9a700 1 -- 172.21.15.64:0/1894474334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c500ffc80 msgr2=0x7f3c5019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:25:59.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c56c9a700 1 --2- 172.21.15.64:0/1894474334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c500ffc80 0x7f3c5019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:25:59.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c56c9a700 1 -- 172.21.15.64:0/1894474334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c48009d00 con 0x7f3c50100610 2023-07-18T03:25:59.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c55c98700 1 --2- 172.21.15.64:0/1894474334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c500ffc80 0x7f3c5019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:25:59.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.845+0000 7f3c56c9a700 1 --2- 172.21.15.64:0/1894474334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c50100610 0x7f3c5019ccc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800b780 tx=0x7f3c4800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:59.846 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.846+0000 7f3c477fe700 1 -- 172.21.15.64:0/1894474334 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4800e040 con 0x7f3c50100610 2023-07-18T03:25:59.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.846+0000 7f3c477fe700 1 -- 172.21.15.64:0/1894474334 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c48004c50 con 0x7f3c50100610 2023-07-18T03:25:59.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.846+0000 7f3c477fe700 1 -- 172.21.15.64:0/1894474334 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c480053b0 con 0x7f3c50100610 2023-07-18T03:25:59.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.846+0000 7f3c586fd700 1 -- 172.21.15.64:0/1894474334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5019da10 con 0x7f3c50100610 2023-07-18T03:25:59.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.846+0000 7f3c586fd700 1 -- 172.21.15.64:0/1894474334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5019de30 con 0x7f3c50100610 2023-07-18T03:25:59.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.847+0000 7f3c586fd700 1 -- 172.21.15.64:0/1894474334 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c50105f80 con 0x7f3c50100610 2023-07-18T03:25:59.847 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.848+0000 7f3c477fe700 1 -- 172.21.15.64:0/1894474334 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3c4801d9c0 con 0x7f3c50100610 2023-07-18T03:25:59.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.848+0000 7f3c477fe700 1 --2- 172.21.15.64:0/1894474334 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c3c0a11d0 0x7f3c3c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:25:59.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.848+0000 7f3c477fe700 1 -- 172.21.15.64:0/1894474334 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f3c480ccd20 con 0x7f3c50100610 2023-07-18T03:25:59.848 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.849+0000 7f3c56499700 1 --2- 172.21.15.64:0/1894474334 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c3c0a11d0 0x7f3c3c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:25:59.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.849+0000 7f3c56499700 1 --2- 172.21.15.64:0/1894474334 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c3c0a11d0 0x7f3c3c0a3680 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c500ffae0 tx=0x7f3c4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:25:59.849 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:25:59.853+0000 7f3c477fe700 1 -- 172.21.15.64:0/1894474334 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3c48097700 con 0x7f3c50100610 2023-07-18T03:26:00.052 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:26:00.055+0000 7f3c586fd700 1 -- 172.21.15.64:0/1894474334 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c50060950 con 0x7f3c3c0a11d0 2023-07-18T03:26:00.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:59 smithi134 ceph-mon[98216]: purged_snaps scrub starts 2023-07-18T03:26:00.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:59 smithi134 ceph-mon[98216]: purged_snaps scrub ok 2023-07-18T03:26:00.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:26:00.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:59 smithi134 ceph-mon[98216]: osd.3 [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213] boot 2023-07-18T03:26:00.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:59 smithi134 ceph-mon[98216]: osdmap e25: 4 total, 4 up, 4 in 2023-07-18T03:26:00.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:25:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:26:00.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:59 smithi040 ceph-mon[96439]: purged_snaps scrub starts 2023-07-18T03:26:00.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:59 smithi040 ceph-mon[96439]: purged_snaps scrub ok 2023-07-18T03:26:00.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:26:00.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:59 smithi040 ceph-mon[96439]: osd.3 [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213] boot 2023-07-18T03:26:00.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:59 smithi040 ceph-mon[96439]: osdmap e25: 4 total, 4 up, 4 in 2023-07-18T03:26:00.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:25:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:26:00.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:59 smithi064 ceph-mon[98059]: purged_snaps scrub starts 2023-07-18T03:26:00.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:59 smithi064 ceph-mon[98059]: purged_snaps scrub ok 2023-07-18T03:26:00.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:26:00.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:59 smithi064 ceph-mon[98059]: osd.3 [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213] boot 2023-07-18T03:26:00.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:59 smithi064 ceph-mon[98059]: osdmap e25: 4 total, 4 up, 4 in 2023-07-18T03:26:00.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:25:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-18T03:26:01.065 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:00 smithi064 ceph-mon[98059]: osdmap e26: 4 total, 4 up, 4 in 2023-07-18T03:26:01.065 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:00 smithi064 ceph-mon[98059]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:01.065 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:00 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:26:01.065 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:00 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:26:01.066 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:00 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:01.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:00 smithi134 ceph-mon[98216]: osdmap e26: 4 total, 4 up, 4 in 2023-07-18T03:26:01.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:00 smithi134 ceph-mon[98216]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:01.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:00 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:26:01.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:00 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:26:01.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:00 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:01.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:00 smithi040 ceph-mon[96439]: osdmap e26: 4 total, 4 up, 4 in 2023-07-18T03:26:01.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:00 smithi040 ceph-mon[96439]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:01.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:00 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:26:01.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:00 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:26:01.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:00 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:02.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:01 smithi134 ceph-mon[98216]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:26:02.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:01 smithi134 ceph-mon[98216]: osdmap e27: 4 total, 4 up, 4 in 2023-07-18T03:26:02.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:01 smithi134 ceph-mon[98216]: osdmap e28: 4 total, 4 up, 4 in 2023-07-18T03:26:02.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:01 smithi040 ceph-mon[96439]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:26:02.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:01 smithi040 ceph-mon[96439]: osdmap e27: 4 total, 4 up, 4 in 2023-07-18T03:26:02.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:01 smithi040 ceph-mon[96439]: osdmap e28: 4 total, 4 up, 4 in 2023-07-18T03:26:02.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:01 smithi064 ceph-mon[98059]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:26:02.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:01 smithi064 ceph-mon[98059]: osdmap e27: 4 total, 4 up, 4 in 2023-07-18T03:26:02.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:01 smithi064 ceph-mon[98059]: osdmap e28: 4 total, 4 up, 4 in 2023-07-18T03:26:03.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:02 smithi134 ceph-mon[98216]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-18T03:26:03.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:02 smithi134 ceph-mon[98216]: from='client.? 172.21.15.64:0/3578490876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b567530-058b-44db-b773-7c6f7224d36a"}]: dispatch 2023-07-18T03:26:03.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:02 smithi134 ceph-mon[98216]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b567530-058b-44db-b773-7c6f7224d36a"}]: dispatch 2023-07-18T03:26:03.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:02 smithi134 ceph-mon[98216]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b567530-058b-44db-b773-7c6f7224d36a"}]': finished 2023-07-18T03:26:03.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:02 smithi134 ceph-mon[98216]: osdmap e29: 5 total, 4 up, 5 in 2023-07-18T03:26:03.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:26:03.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:02 smithi040 ceph-mon[96439]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-18T03:26:03.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:02 smithi040 ceph-mon[96439]: from='client.? 172.21.15.64:0/3578490876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b567530-058b-44db-b773-7c6f7224d36a"}]: dispatch 2023-07-18T03:26:03.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:02 smithi040 ceph-mon[96439]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b567530-058b-44db-b773-7c6f7224d36a"}]: dispatch 2023-07-18T03:26:03.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:02 smithi040 ceph-mon[96439]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b567530-058b-44db-b773-7c6f7224d36a"}]': finished 2023-07-18T03:26:03.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:02 smithi040 ceph-mon[96439]: osdmap e29: 5 total, 4 up, 5 in 2023-07-18T03:26:03.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:26:03.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:02 smithi064 ceph-mon[98059]: pgmap v93: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-18T03:26:03.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:02 smithi064 ceph-mon[98059]: from='client.? 172.21.15.64:0/3578490876' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b567530-058b-44db-b773-7c6f7224d36a"}]: dispatch 2023-07-18T03:26:03.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:02 smithi064 ceph-mon[98059]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b567530-058b-44db-b773-7c6f7224d36a"}]: dispatch 2023-07-18T03:26:03.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:02 smithi064 ceph-mon[98059]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b567530-058b-44db-b773-7c6f7224d36a"}]': finished 2023-07-18T03:26:03.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:02 smithi064 ceph-mon[98059]: osdmap e29: 5 total, 4 up, 5 in 2023-07-18T03:26:03.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:26:04.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:03 smithi134 ceph-mon[98216]: from='client.? 172.21.15.64:0/3718401851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:26:04.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:03 smithi040 ceph-mon[96439]: from='client.? 172.21.15.64:0/3718401851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:26:04.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:03 smithi064 ceph-mon[98059]: from='client.? 172.21.15.64:0/3718401851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:26:05.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:04 smithi134 ceph-mon[98216]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 141 KiB/s, 0 objects/s recovering 2023-07-18T03:26:05.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:04 smithi040 ceph-mon[96439]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 141 KiB/s, 0 objects/s recovering 2023-07-18T03:26:05.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:04 smithi064 ceph-mon[98059]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 141 KiB/s, 0 objects/s recovering 2023-07-18T03:26:06.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:05 smithi040 ceph-mon[96439]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:26:06.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:05 smithi064 ceph-mon[98059]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:26:06.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:05 smithi134 ceph-mon[98216]: pgmap v96: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:26:08.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:07 smithi040 ceph-mon[96439]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-07-18T03:26:08.084 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:07 smithi064 ceph-mon[98059]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-07-18T03:26:08.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:07 smithi134 ceph-mon[98216]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-07-18T03:26:09.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:09 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-18T03:26:09.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:09 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:09.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:09 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-18T03:26:09.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:09 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:09.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:09 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-18T03:26:09.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:09 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:10.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:10 smithi134 ceph-mon[98216]: Deploying daemon osd.4 on smithi064 2023-07-18T03:26:10.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:10 smithi134 ceph-mon[98216]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-07-18T03:26:10.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:10 smithi040 ceph-mon[96439]: Deploying daemon osd.4 on smithi064 2023-07-18T03:26:10.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:10 smithi040 ceph-mon[96439]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-07-18T03:26:10.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:10 smithi064 ceph-mon[98059]: Deploying daemon osd.4 on smithi064 2023-07-18T03:26:10.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:10 smithi064 ceph-mon[98059]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-07-18T03:26:12.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:12 smithi134 ceph-mon[98216]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:12.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:12 smithi040 ceph-mon[96439]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:12.835 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:12 smithi064 ceph-mon[98059]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:14.727 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:14 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:14.727 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:14 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:14.727 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:14 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:14.728 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:14 smithi064 ceph-mon[98059]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:15.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:14 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:15.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:14 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:15.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:14 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:15.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:14 smithi134 ceph-mon[98216]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:15.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:14 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:15.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:14 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:15.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:14 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:15.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:14 smithi040 ceph-mon[96439]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:16.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:15 smithi040 ceph-mon[96439]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:16.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:15 smithi064 ceph-mon[98059]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:16.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:15 smithi134 ceph-mon[98216]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:16.370 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:26:16.373+0000 7f3c477fe700 1 -- 172.21.15.64:0/1894474334 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3c50060950 con 0x7f3c3c0a11d0 2023-07-18T03:26:16.371 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 4 on host 'smithi064' 2023-07-18T03:26:16.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:26:16.376+0000 7f3c586fd700 1 -- 172.21.15.64:0/1894474334 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c3c0a11d0 msgr2=0x7f3c3c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:16.373 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:26:16.376+0000 7f3c586fd700 1 --2- 172.21.15.64:0/1894474334 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c3c0a11d0 0x7f3c3c0a3680 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3c500ffae0 tx=0x7f3c4c009260 comp rx=0 tx=0).stop 2023-07-18T03:26:16.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:26:16.376+0000 7f3c586fd700 1 -- 172.21.15.64:0/1894474334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c50100610 msgr2=0x7f3c5019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:16.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:26:16.376+0000 7f3c586fd700 1 --2- 172.21.15.64:0/1894474334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c50100610 0x7f3c5019ccc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800b780 tx=0x7f3c4800bab0 comp rx=0 tx=0).stop 2023-07-18T03:26:16.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:26:16.377+0000 7f3c586fd700 1 -- 172.21.15.64:0/1894474334 shutdown_connections 2023-07-18T03:26:16.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:26:16.377+0000 7f3c586fd700 1 --2- 172.21.15.64:0/1894474334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c50100610 0x7f3c5019ccc0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:16.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:26:16.377+0000 7f3c586fd700 1 --2- 172.21.15.64:0/1894474334 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c3c0a11d0 0x7f3c3c0a3680 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:16.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:26:16.377+0000 7f3c586fd700 1 --2- 172.21.15.64:0/1894474334 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c500fea80 0x7f3c50068b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:16.374 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:26:16.377+0000 7f3c586fd700 1 --2- 172.21.15.64:0/1894474334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c500ffc80 0x7f3c5019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:16.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:26:16.377+0000 7f3c586fd700 1 -- 172.21.15.64:0/1894474334 >> 172.21.15.64:0/1894474334 conn(0x7f3c500fa0b0 msgr2=0x7f3c501030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:26:16.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:26:16.377+0000 7f3c586fd700 1 -- 172.21.15.64:0/1894474334 shutdown_connections 2023-07-18T03:26:16.375 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:26:16.377+0000 7f3c586fd700 1 -- 172.21.15.64:0/1894474334 wait complete. 2023-07-18T03:26:16.731 DEBUG:teuthology.orchestra.run.smithi064:osd.4> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.4.service 2023-07-18T03:26:16.733 INFO:tasks.cephadm:Deploying osd.5 on smithi134 with /dev/vg_nvme/lv_3... 2023-07-18T03:26:16.734 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-18T03:26:16.764 INFO:journalctl@ceph.osd.4.smithi064.stdout:-- Logs begin at Tue 2023-07-18 03:07:19 UTC. -- 2023-07-18T03:26:17.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:16 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:17.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:16 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:17.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:16 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:17.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:16 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:17.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:17.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:17.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:17.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:17.297 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:26:17.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:16 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:17.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:16 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:17.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:16 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:17.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:16 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:17.850 INFO:journalctl@ceph.osd.4.smithi064.stdout:Jul 18 03:26:17 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-4[108526]: 2023-07-18T03:26:17.585+0000 7f176b37b640 -1 osd.4 0 log_to_monitors true 2023-07-18T03:26:18.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:17 smithi134 ceph-mon[98216]: from='osd.4 [v2:172.21.15.64:6808/788347693,v1:172.21.15.64:6809/788347693]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-18T03:26:18.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:17 smithi134 ceph-mon[98216]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-18T03:26:18.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:17 smithi134 ceph-mon[98216]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:18.129 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:17 smithi064 ceph-mon[98059]: from='osd.4 [v2:172.21.15.64:6808/788347693,v1:172.21.15.64:6809/788347693]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-18T03:26:18.129 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:17 smithi064 ceph-mon[98059]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-18T03:26:18.129 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:17 smithi064 ceph-mon[98059]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:18.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:17 smithi040 ceph-mon[96439]: from='osd.4 [v2:172.21.15.64:6808/788347693,v1:172.21.15.64:6809/788347693]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-18T03:26:18.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:17 smithi040 ceph-mon[96439]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-18T03:26:18.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:17 smithi040 ceph-mon[96439]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:18.746 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-18T03:26:18.746 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-18T03:26:18.747 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2023-07-18T03:26:18.747 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2023-07-18T03:26:18.747 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0152371 s, 688 MB/s 2023-07-18T03:26:18.747 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2023-07-18T03:26:19.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:18 smithi134 ceph-mon[98216]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-18T03:26:19.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:18 smithi134 ceph-mon[98216]: osdmap e30: 5 total, 4 up, 5 in 2023-07-18T03:26:19.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:18 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:26:19.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:18 smithi134 ceph-mon[98216]: from='osd.4 [v2:172.21.15.64:6808/788347693,v1:172.21.15.64:6809/788347693]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:26:19.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:18 smithi134 ceph-mon[98216]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:26:19.071 INFO:journalctl@ceph.osd.4.smithi064.stdout:Jul 18 03:26:18 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-4[108526]: 2023-07-18T03:26:18.923+0000 7f17601f2700 -1 osd.4 0 waiting for initial osdmap 2023-07-18T03:26:19.072 INFO:journalctl@ceph.osd.4.smithi064.stdout:Jul 18 03:26:18 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-4[108526]: 2023-07-18T03:26:18.929+0000 7f175c7e8700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:26:19.072 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:18 smithi064 ceph-mon[98059]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-18T03:26:19.072 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:18 smithi064 ceph-mon[98059]: osdmap e30: 5 total, 4 up, 5 in 2023-07-18T03:26:19.072 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:18 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:26:19.073 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:18 smithi064 ceph-mon[98059]: from='osd.4 [v2:172.21.15.64:6808/788347693,v1:172.21.15.64:6809/788347693]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:26:19.073 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:18 smithi064 ceph-mon[98059]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:26:19.137 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi134:vg_nvme/lv_3 2023-07-18T03:26:19.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:18 smithi040 ceph-mon[96439]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-18T03:26:19.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:18 smithi040 ceph-mon[96439]: osdmap e30: 5 total, 4 up, 5 in 2023-07-18T03:26:19.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:18 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:26:19.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:18 smithi040 ceph-mon[96439]: from='osd.4 [v2:172.21.15.64:6808/788347693,v1:172.21.15.64:6809/788347693]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:26:19.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:18 smithi040 ceph-mon[96439]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:26:19.692 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:26:20.155 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-07-18T03:26:20.156 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: osdmap e31: 5 total, 4 up, 5 in 2023-07-18T03:26:20.156 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:26:20.156 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:26:20.156 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: Detected new or changed devices on smithi064 2023-07-18T03:26:20.157 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:20.157 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:20.157 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:26:20.157 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:26:20.157 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: Adjusting osd_memory_target on smithi064 to 8584M 2023-07-18T03:26:20.157 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:20.157 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:20.158 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:26:20.158 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:20.158 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:20.158 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:20.158 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:26:20.158 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:20.159 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:19 smithi134 ceph-mon[98216]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:20.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-07-18T03:26:20.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: osdmap e31: 5 total, 4 up, 5 in 2023-07-18T03:26:20.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:26:20.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:26:20.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: Detected new or changed devices on smithi064 2023-07-18T03:26:20.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:20.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:20.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:26:20.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:26:20.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: Adjusting osd_memory_target on smithi064 to 8584M 2023-07-18T03:26:20.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:20.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:20.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:26:20.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:20.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:20.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:20.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:26:20.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:20.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:19 smithi040 ceph-mon[96439]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:20.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-07-18T03:26:20.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: osdmap e31: 5 total, 4 up, 5 in 2023-07-18T03:26:20.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:26:20.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:26:20.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: Detected new or changed devices on smithi064 2023-07-18T03:26:20.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:20.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:20.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:26:20.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:26:20.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: Adjusting osd_memory_target on smithi064 to 8584M 2023-07-18T03:26:20.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:20.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:20.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:26:20.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:20.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:20.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:20.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:26:20.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:20.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:19 smithi064 ceph-mon[98059]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-18T03:26:20.878 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.879+0000 7f39a4bf6700 1 -- 172.21.15.134:0/2728081981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39a00fea80 msgr2=0x7f39a00fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:20.878 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.879+0000 7f39a4bf6700 1 --2- 172.21.15.134:0/2728081981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39a00fea80 0x7f39a00fee90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3990009990 tx=0x7f3990009ca0 comp rx=0 tx=0).stop 2023-07-18T03:26:20.879 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.880+0000 7f39a4bf6700 1 -- 172.21.15.134:0/2728081981 shutdown_connections 2023-07-18T03:26:20.879 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.880+0000 7f39a4bf6700 1 --2- 172.21.15.134:0/2728081981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39a0100610 0x7f39a0106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:20.879 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.880+0000 7f39a4bf6700 1 --2- 172.21.15.134:0/2728081981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39a00fea80 0x7f39a00fee90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:20.879 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.880+0000 7f39a4bf6700 1 --2- 172.21.15.134:0/2728081981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a00ffc80 0x7f39a01000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:20.879 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.880+0000 7f39a4bf6700 1 -- 172.21.15.134:0/2728081981 >> 172.21.15.134:0/2728081981 conn(0x7f39a00fa0b0 msgr2=0x7f39a00fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:26:20.879 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.880+0000 7f39a4bf6700 1 -- 172.21.15.134:0/2728081981 shutdown_connections 2023-07-18T03:26:20.879 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.880+0000 7f39a4bf6700 1 -- 172.21.15.134:0/2728081981 wait complete. 2023-07-18T03:26:20.880 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.880+0000 7f39a4bf6700 1 Processor -- start 2023-07-18T03:26:20.880 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.880+0000 7f39a4bf6700 1 -- start start 2023-07-18T03:26:20.880 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.880+0000 7f39a4bf6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39a00fea80 0x7f39a0068b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:20.880 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.880+0000 7f39a4bf6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39a00ffc80 0x7f39a019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:20.880 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f39a4bf6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a0100610 0x7f39a019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:20.880 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f39a4bf6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a019f5e0 con 0x7f39a0100610 2023-07-18T03:26:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f39a4bf6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a019f750 con 0x7f39a00fea80 2023-07-18T03:26:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f39a4bf6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a019f910 con 0x7f39a00ffc80 2023-07-18T03:26:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f399dd9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39a00ffc80 0x7f39a019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f399dd9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39a00ffc80 0x7f39a019c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:48116/0 (socket says 172.21.15.134:48116) 2023-07-18T03:26:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f399dd9b700 1 -- 172.21.15.134:0/2588895103 learned_addr learned my addr 172.21.15.134:0/2588895103 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-07-18T03:26:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f399e59c700 1 --2- 172.21.15.134:0/2588895103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39a00fea80 0x7f39a0068b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:20.881 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f399dd9b700 1 -- 172.21.15.134:0/2588895103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39a00fea80 msgr2=0x7f39a0068b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f399dd9b700 1 --2- 172.21.15.134:0/2588895103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39a00fea80 0x7f39a0068b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f399dd9b700 1 -- 172.21.15.134:0/2588895103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a0100610 msgr2=0x7f39a019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f399dd9b700 1 --2- 172.21.15.134:0/2588895103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a0100610 0x7f39a019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f399dd9b700 1 -- 172.21.15.134:0/2588895103 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39900095f0 con 0x7f39a00ffc80 2023-07-18T03:26:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f399dd9b700 1 --2- 172.21.15.134:0/2588895103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39a00ffc80 0x7f39a019c9e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f399400d950 tx=0x7f399400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:26:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f398f7fe700 1 -- 172.21.15.134:0/2588895103 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994009940 con 0x7f39a00ffc80 2023-07-18T03:26:20.882 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f39a4bf6700 1 -- 172.21.15.134:0/2588895103 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a019fbf0 con 0x7f39a00ffc80 2023-07-18T03:26:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f398f7fe700 1 -- 172.21.15.134:0/2588895103 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3994010460 con 0x7f39a00ffc80 2023-07-18T03:26:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f398f7fe700 1 -- 172.21.15.134:0/2588895103 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399400f620 con 0x7f39a00ffc80 2023-07-18T03:26:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.881+0000 7f39a4bf6700 1 -- 172.21.15.134:0/2588895103 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a01a0140 con 0x7f39a00ffc80 2023-07-18T03:26:20.883 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.882+0000 7f39a4bf6700 1 -- 172.21.15.134:0/2588895103 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a01060c0 con 0x7f39a00ffc80 2023-07-18T03:26:20.884 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.883+0000 7f398f7fe700 1 -- 172.21.15.134:0/2588895103 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f39940108d0 con 0x7f39a00ffc80 2023-07-18T03:26:20.884 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.884+0000 7f398f7fe700 1 --2- 172.21.15.134:0/2588895103 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f39880a1150 0x7f39880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:20.884 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.884+0000 7f398f7fe700 1 -- 172.21.15.134:0/2588895103 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f39940cb720 con 0x7f39a00ffc80 2023-07-18T03:26:20.885 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.884+0000 7f399e59c700 1 --2- 172.21.15.134:0/2588895103 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f39880a1150 0x7f39880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:20.885 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.884+0000 7f399e59c700 1 --2- 172.21.15.134:0/2588895103 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f39880a1150 0x7f39880a3600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f399000b5c0 tx=0x7f3990005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:26:20.885 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:20.888+0000 7f398f7fe700 1 -- 172.21.15.134:0/2588895103 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3994095ee0 con 0x7f39a00ffc80 2023-07-18T03:26:21.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:20 smithi134 ceph-mon[98216]: purged_snaps scrub starts 2023-07-18T03:26:21.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:20 smithi134 ceph-mon[98216]: purged_snaps scrub ok 2023-07-18T03:26:21.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:20 smithi134 ceph-mon[98216]: osd.4 [v2:172.21.15.64:6808/788347693,v1:172.21.15.64:6809/788347693] boot 2023-07-18T03:26:21.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:20 smithi134 ceph-mon[98216]: osdmap e32: 5 total, 5 up, 5 in 2023-07-18T03:26:21.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:20 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:26:21.080 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:21.084+0000 7f39a4bf6700 1 -- 172.21.15.134:0/2588895103 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f39a0060950 con 0x7f39880a1150 2023-07-18T03:26:21.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:20 smithi040 ceph-mon[96439]: purged_snaps scrub starts 2023-07-18T03:26:21.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:20 smithi040 ceph-mon[96439]: purged_snaps scrub ok 2023-07-18T03:26:21.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:20 smithi040 ceph-mon[96439]: osd.4 [v2:172.21.15.64:6808/788347693,v1:172.21.15.64:6809/788347693] boot 2023-07-18T03:26:21.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:20 smithi040 ceph-mon[96439]: osdmap e32: 5 total, 5 up, 5 in 2023-07-18T03:26:21.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:20 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:26:21.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:20 smithi064 ceph-mon[98059]: purged_snaps scrub starts 2023-07-18T03:26:21.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:20 smithi064 ceph-mon[98059]: purged_snaps scrub ok 2023-07-18T03:26:21.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:20 smithi064 ceph-mon[98059]: osd.4 [v2:172.21.15.64:6808/788347693,v1:172.21.15.64:6809/788347693] boot 2023-07-18T03:26:21.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:20 smithi064 ceph-mon[98059]: osdmap e32: 5 total, 5 up, 5 in 2023-07-18T03:26:21.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:20 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-18T03:26:22.138 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:21 smithi134 ceph-mon[98216]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:26:22.139 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:26:22.139 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:26:22.139 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:22.139 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:21 smithi134 ceph-mon[98216]: osdmap e33: 5 total, 5 up, 5 in 2023-07-18T03:26:22.140 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:21 smithi134 ceph-mon[98216]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:22.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:21 smithi040 ceph-mon[96439]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:26:22.335 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:26:22.335 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:26:22.336 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:22.336 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:21 smithi040 ceph-mon[96439]: osdmap e33: 5 total, 5 up, 5 in 2023-07-18T03:26:22.336 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:21 smithi040 ceph-mon[96439]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:22.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:21 smithi064 ceph-mon[98059]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:26:22.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:26:22.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:26:22.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:22.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:21 smithi064 ceph-mon[98059]: osdmap e33: 5 total, 5 up, 5 in 2023-07-18T03:26:22.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:21 smithi064 ceph-mon[98059]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:23.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:23 smithi134 ceph-mon[98216]: from='client.? 172.21.15.134:0/3252556495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37241e63-af2a-400b-b5df-a15096c201e9"}]: dispatch 2023-07-18T03:26:23.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:23 smithi134 ceph-mon[98216]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37241e63-af2a-400b-b5df-a15096c201e9"}]: dispatch 2023-07-18T03:26:23.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:23 smithi134 ceph-mon[98216]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37241e63-af2a-400b-b5df-a15096c201e9"}]': finished 2023-07-18T03:26:23.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:23 smithi134 ceph-mon[98216]: osdmap e34: 6 total, 5 up, 6 in 2023-07-18T03:26:23.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:23 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:26:23.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:23 smithi040 ceph-mon[96439]: from='client.? 172.21.15.134:0/3252556495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37241e63-af2a-400b-b5df-a15096c201e9"}]: dispatch 2023-07-18T03:26:23.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:23 smithi040 ceph-mon[96439]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37241e63-af2a-400b-b5df-a15096c201e9"}]: dispatch 2023-07-18T03:26:23.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:23 smithi040 ceph-mon[96439]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37241e63-af2a-400b-b5df-a15096c201e9"}]': finished 2023-07-18T03:26:23.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:23 smithi040 ceph-mon[96439]: osdmap e34: 6 total, 5 up, 6 in 2023-07-18T03:26:23.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:23 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:26:23.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:23 smithi064 ceph-mon[98059]: from='client.? 172.21.15.134:0/3252556495' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37241e63-af2a-400b-b5df-a15096c201e9"}]: dispatch 2023-07-18T03:26:23.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:23 smithi064 ceph-mon[98059]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "37241e63-af2a-400b-b5df-a15096c201e9"}]: dispatch 2023-07-18T03:26:23.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:23 smithi064 ceph-mon[98059]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "37241e63-af2a-400b-b5df-a15096c201e9"}]': finished 2023-07-18T03:26:23.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:23 smithi064 ceph-mon[98059]: osdmap e34: 6 total, 5 up, 6 in 2023-07-18T03:26:23.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:26:24.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:24 smithi134 ceph-mon[98216]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:24.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:24 smithi134 ceph-mon[98216]: from='client.? 172.21.15.134:0/2977491994' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:26:24.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:24 smithi040 ceph-mon[96439]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:24.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:24 smithi040 ceph-mon[96439]: from='client.? 172.21.15.134:0/2977491994' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:26:24.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:24 smithi064 ceph-mon[98059]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:24.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:24 smithi064 ceph-mon[98059]: from='client.? 172.21.15.134:0/2977491994' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:26:26.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:25 smithi040 ceph-mon[96439]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:26.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:25 smithi064 ceph-mon[98059]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:26.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:25 smithi134 ceph-mon[98216]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:28.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:27 smithi134 ceph-mon[98216]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:28.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:27 smithi040 ceph-mon[96439]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:28.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:27 smithi064 ceph-mon[98059]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:30.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:29 smithi134 ceph-mon[98216]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:30.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:29 smithi040 ceph-mon[96439]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:30.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:29 smithi064 ceph-mon[98059]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:30.962 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:30 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-18T03:26:30.962 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:30 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:30.962 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:30 smithi134 ceph-mon[98216]: Deploying daemon osd.5 on smithi134 2023-07-18T03:26:31.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:30 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-18T03:26:31.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:30 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:31.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:30 smithi040 ceph-mon[96439]: Deploying daemon osd.5 on smithi134 2023-07-18T03:26:31.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:30 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-18T03:26:31.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:30 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:31.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:30 smithi064 ceph-mon[98059]: Deploying daemon osd.5 on smithi134 2023-07-18T03:26:32.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:32 smithi134 ceph-mon[98216]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:32.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:32 smithi040 ceph-mon[96439]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:32.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:32 smithi064 ceph-mon[98059]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:34.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:33 smithi040 ceph-mon[96439]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:34.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:33 smithi064 ceph-mon[98059]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:34.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:33 smithi134 ceph-mon[98216]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:35.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:35.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:35.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:35.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:35.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:35.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:35.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:35.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:35.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:36.761 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:36 smithi134 ceph-mon[98216]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:36.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:36 smithi040 ceph-mon[96439]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:36.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:36 smithi064 ceph-mon[98059]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:37.041 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:37.045+0000 7f398f7fe700 1 -- 172.21.15.134:0/2588895103 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f39a0060950 con 0x7f39880a1150 2023-07-18T03:26:37.042 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 5 on host 'smithi134' 2023-07-18T03:26:37.044 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:37.048+0000 7f39a4bf6700 1 -- 172.21.15.134:0/2588895103 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f39880a1150 msgr2=0x7f39880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:37.044 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:37.048+0000 7f39a4bf6700 1 --2- 172.21.15.134:0/2588895103 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f39880a1150 0x7f39880a3600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f399000b5c0 tx=0x7f3990005fd0 comp rx=0 tx=0).stop 2023-07-18T03:26:37.044 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:37.048+0000 7f39a4bf6700 1 -- 172.21.15.134:0/2588895103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39a00ffc80 msgr2=0x7f39a019c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:37.044 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:37.048+0000 7f39a4bf6700 1 --2- 172.21.15.134:0/2588895103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39a00ffc80 0x7f39a019c9e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f399400d950 tx=0x7f399400dc60 comp rx=0 tx=0).stop 2023-07-18T03:26:37.044 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:37.048+0000 7f39a4bf6700 1 -- 172.21.15.134:0/2588895103 shutdown_connections 2023-07-18T03:26:37.045 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:37.048+0000 7f39a4bf6700 1 --2- 172.21.15.134:0/2588895103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a0100610 0x7f39a019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:37.045 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:37.048+0000 7f39a4bf6700 1 --2- 172.21.15.134:0/2588895103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39a00fea80 0x7f39a0068b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:37.045 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:37.048+0000 7f39a4bf6700 1 --2- 172.21.15.134:0/2588895103 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f39880a1150 0x7f39880a3600 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:37.045 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:37.048+0000 7f39a4bf6700 1 --2- 172.21.15.134:0/2588895103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39a00ffc80 0x7f39a019c9e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:37.045 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:37.048+0000 7f39a4bf6700 1 -- 172.21.15.134:0/2588895103 >> 172.21.15.134:0/2588895103 conn(0x7f39a00fa0b0 msgr2=0x7f39a0103200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:26:37.045 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:37.048+0000 7f39a4bf6700 1 -- 172.21.15.134:0/2588895103 shutdown_connections 2023-07-18T03:26:37.045 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:26:37.048+0000 7f39a4bf6700 1 -- 172.21.15.134:0/2588895103 wait complete. 2023-07-18T03:26:37.384 DEBUG:teuthology.orchestra.run.smithi134:osd.5> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.5.service 2023-07-18T03:26:37.386 INFO:tasks.cephadm:Deploying osd.6 on smithi040 with /dev/vg_nvme/lv_2... 2023-07-18T03:26:37.387 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-18T03:26:37.416 INFO:journalctl@ceph.osd.5.smithi134.stdout:-- Logs begin at Tue 2023-07-18 03:07:14 UTC. -- 2023-07-18T03:26:37.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:37 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:37.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:37 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:37.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:37 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:37.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:37 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:26:37.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:37 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:37.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:37 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:37.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:37 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:37.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:37 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:37.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:37 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:37.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:37 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:37.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:37 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:37.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:37 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:26:37.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:37 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:37.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:37 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:37.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:37 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:37.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:37 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:37.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:37 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:37.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:37 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:37.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:37 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:37.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:37 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:26:37.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:37 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:37.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:37 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:37.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:37 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:37.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:37 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:37.967 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:26:38.513 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:26:38 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:26:38.423+0000 7fb2957ab640 -1 osd.5 0 log_to_monitors true 2023-07-18T03:26:38.764 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:38 smithi134 ceph-mon[98216]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:38.764 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:38 smithi134 ceph-mon[98216]: from='osd.5 [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-18T03:26:38.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:38 smithi040 ceph-mon[96439]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:38.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:38 smithi040 ceph-mon[96439]: from='osd.5 [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-18T03:26:38.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:38 smithi064 ceph-mon[98059]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:38.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:38 smithi064 ceph-mon[98059]: from='osd.5 [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-18T03:26:39.359 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-18T03:26:39.359 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-18T03:26:39.359 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2023-07-18T03:26:39.359 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2023-07-18T03:26:39.360 INFO:teuthology.orchestra.run.smithi040.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0108188 s, 969 MB/s 2023-07-18T03:26:39.360 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2023-07-18T03:26:39.662 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:39 smithi040 ceph-mon[96439]: from='osd.5 [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-18T03:26:39.663 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:39 smithi040 ceph-mon[96439]: osdmap e35: 6 total, 5 up, 6 in 2023-07-18T03:26:39.663 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:26:39.663 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:39 smithi040 ceph-mon[96439]: from='osd.5 [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:26:39.727 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi040:vg_nvme/lv_2 2023-07-18T03:26:39.816 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:26:39 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:26:39.569+0000 7fb28a622700 -1 osd.5 0 waiting for initial osdmap 2023-07-18T03:26:39.817 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:26:39 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:26:39.577+0000 7fb285415700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:26:39.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:39 smithi134 ceph-mon[98216]: from='osd.5 [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-18T03:26:39.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:39 smithi134 ceph-mon[98216]: osdmap e35: 6 total, 5 up, 6 in 2023-07-18T03:26:39.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:26:39.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:39 smithi134 ceph-mon[98216]: from='osd.5 [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:26:39.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:39 smithi064 ceph-mon[98059]: from='osd.5 [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-18T03:26:39.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:39 smithi064 ceph-mon[98059]: osdmap e35: 6 total, 5 up, 6 in 2023-07-18T03:26:39.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:26:39.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:39 smithi064 ceph-mon[98059]: from='osd.5 [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:26:40.288 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:26:40.739 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:40 smithi040 ceph-mon[96439]: from='osd.5 [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-18T03:26:40.739 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:40 smithi040 ceph-mon[96439]: osdmap e36: 6 total, 5 up, 6 in 2023-07-18T03:26:40.739 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:40 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:26:40.739 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:40 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:26:40.739 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:40 smithi040 ceph-mon[96439]: Detected new or changed devices on smithi134 2023-07-18T03:26:40.740 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:40 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:40.740 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:40 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:40.740 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:40 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:26:40.740 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:40 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:26:40.740 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:40 smithi040 ceph-mon[96439]: Adjusting osd_memory_target on smithi134 to 8584M 2023-07-18T03:26:40.740 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:40 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:40.741 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:40 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:40.741 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:40 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:26:40.741 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:40 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:40.741 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:40 smithi040 ceph-mon[96439]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:40.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:40 smithi064 ceph-mon[98059]: from='osd.5 [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-18T03:26:40.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:40 smithi064 ceph-mon[98059]: osdmap e36: 6 total, 5 up, 6 in 2023-07-18T03:26:40.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:40 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:26:40.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:40 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:26:40.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:40 smithi064 ceph-mon[98059]: Detected new or changed devices on smithi134 2023-07-18T03:26:40.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:40 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:40.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:40 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:40.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:40 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:26:40.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:40 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:26:40.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:40 smithi064 ceph-mon[98059]: Adjusting osd_memory_target on smithi134 to 8584M 2023-07-18T03:26:40.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:40 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:40.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:40 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:40.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:40 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:26:40.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:40 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:40.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:40 smithi064 ceph-mon[98059]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:41.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:40 smithi134 ceph-mon[98216]: from='osd.5 [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-18T03:26:41.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:40 smithi134 ceph-mon[98216]: osdmap e36: 6 total, 5 up, 6 in 2023-07-18T03:26:41.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:40 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:26:41.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:40 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:26:41.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:40 smithi134 ceph-mon[98216]: Detected new or changed devices on smithi134 2023-07-18T03:26:41.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:40 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:41.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:40 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:41.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:40 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:26:41.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:40 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:26:41.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:40 smithi134 ceph-mon[98216]: Adjusting osd_memory_target on smithi134 to 8584M 2023-07-18T03:26:41.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:40 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:41.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:40 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:41.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:40 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:26:41.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:40 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:41.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:40 smithi134 ceph-mon[98216]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-18T03:26:41.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.351+0000 7fae6867b700 1 -- 172.21.15.40:0/3583540469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae600ffc80 msgr2=0x7fae601000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:41.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.351+0000 7fae6867b700 1 --2- 172.21.15.40:0/3583540469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae600ffc80 0x7fae601000d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae50009990 tx=0x7fae50009ca0 comp rx=0 tx=0).stop 2023-07-18T03:26:41.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.351+0000 7fae6867b700 1 -- 172.21.15.40:0/3583540469 shutdown_connections 2023-07-18T03:26:41.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.351+0000 7fae6867b700 1 --2- 172.21.15.40:0/3583540469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae60100610 0x7fae60106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:41.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.351+0000 7fae6867b700 1 --2- 172.21.15.40:0/3583540469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae600fea80 0x7fae600fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:41.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.351+0000 7fae6867b700 1 --2- 172.21.15.40:0/3583540469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae600ffc80 0x7fae601000d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:41.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.351+0000 7fae6867b700 1 -- 172.21.15.40:0/3583540469 >> 172.21.15.40:0/3583540469 conn(0x7fae600fa0b0 msgr2=0x7fae600fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:26:41.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.351+0000 7fae6867b700 1 -- 172.21.15.40:0/3583540469 shutdown_connections 2023-07-18T03:26:41.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.351+0000 7fae6867b700 1 -- 172.21.15.40:0/3583540469 wait complete. 2023-07-18T03:26:41.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.352+0000 7fae6867b700 1 Processor -- start 2023-07-18T03:26:41.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.352+0000 7fae6867b700 1 -- start start 2023-07-18T03:26:41.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.352+0000 7fae6867b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae600fea80 0x7fae60068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:41.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.352+0000 7fae6867b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae600ffc80 0x7fae6019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:41.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.352+0000 7fae6867b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae60100610 0x7fae6019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:41.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.352+0000 7fae6867b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6019d440 con 0x7fae60100610 2023-07-18T03:26:41.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.352+0000 7fae6867b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6019d5b0 con 0x7fae600fea80 2023-07-18T03:26:41.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.352+0000 7fae6867b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6019d770 con 0x7fae600ffc80 2023-07-18T03:26:41.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.352+0000 7fae66c18700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae60100610 0x7fae6019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:41.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.352+0000 7fae66c18700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae60100610 0x7fae6019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55408/0 (socket says 172.21.15.40:55408) 2023-07-18T03:26:41.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.352+0000 7fae66c18700 1 -- 172.21.15.40:0/2456836294 learned_addr learned my addr 172.21.15.40:0/2456836294 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:26:41.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.353+0000 7fae65c16700 1 --2- 172.21.15.40:0/2456836294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae600ffc80 0x7fae6019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:41.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.353+0000 7fae66417700 1 --2- 172.21.15.40:0/2456836294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae600fea80 0x7fae60068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:41.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.353+0000 7fae66c18700 1 -- 172.21.15.40:0/2456836294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae600ffc80 msgr2=0x7fae6019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:41.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.353+0000 7fae66c18700 1 --2- 172.21.15.40:0/2456836294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae600ffc80 0x7fae6019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:41.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.353+0000 7fae66c18700 1 -- 172.21.15.40:0/2456836294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae600fea80 msgr2=0x7fae60068b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:41.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.353+0000 7fae66c18700 1 --2- 172.21.15.40:0/2456836294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae600fea80 0x7fae60068b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:41.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.353+0000 7fae66c18700 1 -- 172.21.15.40:0/2456836294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae500095f0 con 0x7fae60100610 2023-07-18T03:26:41.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.353+0000 7fae66c18700 1 --2- 172.21.15.40:0/2456836294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae60100610 0x7fae6019cca0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fae58007ee0 tx=0x7fae5800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:26:41.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.353+0000 7fae577fe700 1 -- 172.21.15.40:0/2456836294 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58017070 con 0x7fae60100610 2023-07-18T03:26:41.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.353+0000 7fae577fe700 1 -- 172.21.15.40:0/2456836294 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae5800f040 con 0x7fae60100610 2023-07-18T03:26:41.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.353+0000 7fae6867b700 1 -- 172.21.15.40:0/2456836294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae60194480 con 0x7fae60100610 2023-07-18T03:26:41.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.353+0000 7fae577fe700 1 -- 172.21.15.40:0/2456836294 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58005580 con 0x7fae60100610 2023-07-18T03:26:41.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.353+0000 7fae6867b700 1 -- 172.21.15.40:0/2456836294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae60194930 con 0x7fae60100610 2023-07-18T03:26:41.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.354+0000 7fae6867b700 1 -- 172.21.15.40:0/2456836294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae6010a2e0 con 0x7fae60100610 2023-07-18T03:26:41.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.355+0000 7fae577fe700 1 -- 172.21.15.40:0/2456836294 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fae5800db20 con 0x7fae60100610 2023-07-18T03:26:41.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.356+0000 7fae577fe700 1 --2- 172.21.15.40:0/2456836294 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fae4c0a1150 0x7fae4c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:26:41.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.356+0000 7fae66417700 1 --2- 172.21.15.40:0/2456836294 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fae4c0a1150 0x7fae4c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:26:41.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.356+0000 7fae577fe700 1 -- 172.21.15.40:0/2456836294 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4566+0+0 (secure 0 0 0) 0x7fae580cb900 con 0x7fae60100610 2023-07-18T03:26:41.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.356+0000 7fae66417700 1 --2- 172.21.15.40:0/2456836294 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fae4c0a1150 0x7fae4c0a3600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fae600ffae0 tx=0x7fae5c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:26:41.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.359+0000 7fae577fe700 1 -- 172.21.15.40:0/2456836294 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fae58095e50 con 0x7fae60100610 2023-07-18T03:26:41.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:41.568+0000 7fae6867b700 1 -- 172.21.15.40:0/2456836294 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fae60060950 con 0x7fae4c0a1150 2023-07-18T03:26:41.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:41 smithi040 ceph-mon[96439]: purged_snaps scrub starts 2023-07-18T03:26:41.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:41 smithi040 ceph-mon[96439]: purged_snaps scrub ok 2023-07-18T03:26:41.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:41 smithi040 ceph-mon[96439]: osd.5 [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956] boot 2023-07-18T03:26:41.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:41 smithi040 ceph-mon[96439]: osdmap e37: 6 total, 6 up, 6 in 2023-07-18T03:26:41.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:41 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:26:41.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:41 smithi040 ceph-mon[96439]: osdmap e38: 6 total, 6 up, 6 in 2023-07-18T03:26:41.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:41 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:26:41.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:41 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:26:41.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:41 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:41.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:41 smithi064 ceph-mon[98059]: purged_snaps scrub starts 2023-07-18T03:26:41.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:41 smithi064 ceph-mon[98059]: purged_snaps scrub ok 2023-07-18T03:26:41.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:41 smithi064 ceph-mon[98059]: osd.5 [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956] boot 2023-07-18T03:26:41.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:41 smithi064 ceph-mon[98059]: osdmap e37: 6 total, 6 up, 6 in 2023-07-18T03:26:41.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:41 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:26:41.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:41 smithi064 ceph-mon[98059]: osdmap e38: 6 total, 6 up, 6 in 2023-07-18T03:26:41.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:41 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:26:41.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:41 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:26:41.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:41 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:42.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:41 smithi134 ceph-mon[98216]: purged_snaps scrub starts 2023-07-18T03:26:42.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:41 smithi134 ceph-mon[98216]: purged_snaps scrub ok 2023-07-18T03:26:42.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:41 smithi134 ceph-mon[98216]: osd.5 [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956] boot 2023-07-18T03:26:42.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:41 smithi134 ceph-mon[98216]: osdmap e37: 6 total, 6 up, 6 in 2023-07-18T03:26:42.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:41 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-18T03:26:42.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:41 smithi134 ceph-mon[98216]: osdmap e38: 6 total, 6 up, 6 in 2023-07-18T03:26:42.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:41 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:26:42.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:41 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:26:42.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:41 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:42.629 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:42 smithi040 ceph-mon[96439]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:26:42.630 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:42 smithi040 ceph-mon[96439]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:26:42.630 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:42 smithi040 ceph-mon[96439]: osdmap e39: 6 total, 6 up, 6 in 2023-07-18T03:26:42.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:42 smithi064 ceph-mon[98059]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:26:42.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:42 smithi064 ceph-mon[98059]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:26:42.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:42 smithi064 ceph-mon[98059]: osdmap e39: 6 total, 6 up, 6 in 2023-07-18T03:26:43.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:42 smithi134 ceph-mon[98216]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:26:43.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:42 smithi134 ceph-mon[98216]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:26:43.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:42 smithi134 ceph-mon[98216]: osdmap e39: 6 total, 6 up, 6 in 2023-07-18T03:26:45.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:44 smithi134 ceph-mon[98216]: osdmap e40: 6 total, 6 up, 6 in 2023-07-18T03:26:45.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:44 smithi134 ceph-mon[98216]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:26:45.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:44 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3178051828' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ea6ecd-85aa-42cb-ab41-006ab774af0e"}]: dispatch 2023-07-18T03:26:45.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:44 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3178051828' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36ea6ecd-85aa-42cb-ab41-006ab774af0e"}]': finished 2023-07-18T03:26:45.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:44 smithi134 ceph-mon[98216]: osdmap e41: 7 total, 6 up, 7 in 2023-07-18T03:26:45.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:26:45.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:44 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/590739221' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:26:45.083 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:44 smithi040 ceph-mon[96439]: osdmap e40: 6 total, 6 up, 6 in 2023-07-18T03:26:45.083 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:44 smithi040 ceph-mon[96439]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:26:45.083 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:44 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3178051828' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ea6ecd-85aa-42cb-ab41-006ab774af0e"}]: dispatch 2023-07-18T03:26:45.084 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:44 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3178051828' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36ea6ecd-85aa-42cb-ab41-006ab774af0e"}]': finished 2023-07-18T03:26:45.084 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:44 smithi040 ceph-mon[96439]: osdmap e41: 7 total, 6 up, 7 in 2023-07-18T03:26:45.084 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:26:45.084 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:44 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/590739221' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:26:45.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:44 smithi064 ceph-mon[98059]: osdmap e40: 6 total, 6 up, 6 in 2023-07-18T03:26:45.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:44 smithi064 ceph-mon[98059]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:26:45.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:44 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3178051828' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ea6ecd-85aa-42cb-ab41-006ab774af0e"}]: dispatch 2023-07-18T03:26:45.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:44 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3178051828' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36ea6ecd-85aa-42cb-ab41-006ab774af0e"}]': finished 2023-07-18T03:26:45.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:44 smithi064 ceph-mon[98059]: osdmap e41: 7 total, 6 up, 7 in 2023-07-18T03:26:45.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:26:45.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:44 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/590739221' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:26:46.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:45 smithi064 ceph-mon[98059]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:26:46.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:45 smithi134 ceph-mon[98216]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:26:46.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:45 smithi040 ceph-mon[96439]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:26:48.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:47 smithi040 ceph-mon[96439]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:26:48.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:47 smithi064 ceph-mon[98059]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:26:48.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:47 smithi134 ceph-mon[98216]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:26:50.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:49 smithi064 ceph-mon[98059]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-07-18T03:26:50.162 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:49 smithi040 ceph-mon[96439]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-07-18T03:26:50.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:49 smithi134 ceph-mon[98216]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-07-18T03:26:51.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:50 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-18T03:26:51.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:50 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:51.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:50 smithi064 ceph-mon[98059]: Deploying daemon osd.6 on smithi040 2023-07-18T03:26:51.136 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:50 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-18T03:26:51.137 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:50 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:51.137 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:50 smithi040 ceph-mon[96439]: Deploying daemon osd.6 on smithi040 2023-07-18T03:26:51.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:50 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-18T03:26:51.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:50 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:51.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:50 smithi134 ceph-mon[98216]: Deploying daemon osd.6 on smithi040 2023-07-18T03:26:52.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:52 smithi040 ceph-mon[96439]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-07-18T03:26:52.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:52 smithi134 ceph-mon[98216]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-07-18T03:26:52.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:52 smithi064 ceph-mon[98059]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-07-18T03:26:54.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:53 smithi064 ceph-mon[98059]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-18T03:26:54.170 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:53 smithi040 ceph-mon[96439]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-18T03:26:54.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:53 smithi134 ceph-mon[98216]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-18T03:26:55.502 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:55.503 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:55.503 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:55.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:55.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:55.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:55.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:55.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:55.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:26:56.666 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:56 smithi040 ceph-mon[96439]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-18T03:26:56.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:56 smithi134 ceph-mon[98216]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-18T03:26:56.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:56 smithi064 ceph-mon[98059]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-18T03:26:57.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:57.018+0000 7fae577fe700 1 -- 172.21.15.40:0/2456836294 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fae60060950 con 0x7fae4c0a1150 2023-07-18T03:26:57.015 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 6 on host 'smithi040' 2023-07-18T03:26:57.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:57.020+0000 7fae6867b700 1 -- 172.21.15.40:0/2456836294 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fae4c0a1150 msgr2=0x7fae4c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:57.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:57.020+0000 7fae6867b700 1 --2- 172.21.15.40:0/2456836294 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fae4c0a1150 0x7fae4c0a3600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fae600ffae0 tx=0x7fae5c009190 comp rx=0 tx=0).stop 2023-07-18T03:26:57.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:57.020+0000 7fae6867b700 1 -- 172.21.15.40:0/2456836294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae60100610 msgr2=0x7fae6019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:26:57.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:57.020+0000 7fae6867b700 1 --2- 172.21.15.40:0/2456836294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae60100610 0x7fae6019cca0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fae58007ee0 tx=0x7fae5800d3b0 comp rx=0 tx=0).stop 2023-07-18T03:26:57.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:57.020+0000 7fae6867b700 1 -- 172.21.15.40:0/2456836294 shutdown_connections 2023-07-18T03:26:57.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:57.020+0000 7fae6867b700 1 --2- 172.21.15.40:0/2456836294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae60100610 0x7fae6019cca0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:57.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:57.020+0000 7fae6867b700 1 --2- 172.21.15.40:0/2456836294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae600fea80 0x7fae60068b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:57.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:57.020+0000 7fae6867b700 1 --2- 172.21.15.40:0/2456836294 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fae4c0a1150 0x7fae4c0a3600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:57.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:57.020+0000 7fae6867b700 1 --2- 172.21.15.40:0/2456836294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae600ffc80 0x7fae6019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:26:57.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:57.020+0000 7fae6867b700 1 -- 172.21.15.40:0/2456836294 >> 172.21.15.40:0/2456836294 conn(0x7fae600fa0b0 msgr2=0x7fae600fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:26:57.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:57.021+0000 7fae6867b700 1 -- 172.21.15.40:0/2456836294 shutdown_connections 2023-07-18T03:26:57.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:26:57.021+0000 7fae6867b700 1 -- 172.21.15.40:0/2456836294 wait complete. 2023-07-18T03:26:57.384 DEBUG:teuthology.orchestra.run.smithi040:osd.6> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.6.service 2023-07-18T03:26:57.386 INFO:tasks.cephadm:Deploying osd.7 on smithi064 with /dev/vg_nvme/lv_2... 2023-07-18T03:26:57.386 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-18T03:26:57.429 INFO:journalctl@ceph.osd.6.smithi040.stdout:-- Logs begin at Tue 2023-07-18 03:07:07 UTC. -- 2023-07-18T03:26:57.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:57 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:57.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:57 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:57.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:57 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:57.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:57 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:57.976 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:26:57.984 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:57 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:57.984 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:57 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:57.984 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:57 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:57.984 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:57 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:58.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:57 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:58.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:57 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:58.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:57 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:58.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:57 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:58.579 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:26:58 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:26:58.396+0000 7f201b9bc640 -1 osd.6 0 log_to_monitors true 2023-07-18T03:26:58.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:58 smithi064 ceph-mon[98059]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-18T03:26:58.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:58 smithi064 ceph-mon[98059]: from='osd.6 [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-18T03:26:59.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:58 smithi134 ceph-mon[98216]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-18T03:26:59.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:58 smithi134 ceph-mon[98216]: from='osd.6 [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-18T03:26:59.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:58 smithi040 ceph-mon[96439]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-18T03:26:59.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:58 smithi040 ceph-mon[96439]: from='osd.6 [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-18T03:26:59.494 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-18T03:26:59.494 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-18T03:26:59.495 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2023-07-18T03:26:59.495 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2023-07-18T03:26:59.495 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113203 s, 926 MB/s 2023-07-18T03:26:59.495 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2023-07-18T03:26:59.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:59 smithi064 ceph-mon[98059]: from='osd.6 [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-18T03:26:59.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:59 smithi064 ceph-mon[98059]: osdmap e42: 7 total, 6 up, 7 in 2023-07-18T03:26:59.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:26:59.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:59 smithi064 ceph-mon[98059]: from='osd.6 [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2023-07-18T03:26:59.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:59.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:59.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:26:59.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:26:59.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:26:59.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:59.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:26:59.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:26:59.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:26:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:26:59.934 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi064:vg_nvme/lv_2 2023-07-18T03:27:00.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:59 smithi134 ceph-mon[98216]: from='osd.6 [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-18T03:27:00.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:59 smithi134 ceph-mon[98216]: osdmap e42: 7 total, 6 up, 7 in 2023-07-18T03:27:00.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:27:00.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:59 smithi134 ceph-mon[98216]: from='osd.6 [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2023-07-18T03:27:00.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:00.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:00.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:00.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:00.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:00.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:00.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:00.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:00.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:26:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:00.079 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:26:59 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:26:59.700+0000 7f2012036700 -1 osd.6 0 waiting for initial osdmap 2023-07-18T03:27:00.079 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:26:59 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:26:59.714+0000 7f200be27700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:27:00.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:59 smithi040 ceph-mon[96439]: from='osd.6 [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-18T03:27:00.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:59 smithi040 ceph-mon[96439]: osdmap e42: 7 total, 6 up, 7 in 2023-07-18T03:27:00.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:27:00.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:59 smithi040 ceph-mon[96439]: from='osd.6 [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2023-07-18T03:27:00.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:00.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:00.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:00.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:00.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:00.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:00.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:00.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:00.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:26:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:00.492 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:27:01.028 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:00 smithi064 ceph-mon[98059]: Detected new or changed devices on smithi040 2023-07-18T03:27:01.028 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:00 smithi064 ceph-mon[98059]: Adjusting osd_memory_target on smithi040 to 5724M 2023-07-18T03:27:01.028 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:00 smithi064 ceph-mon[98059]: from='osd.6 [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2023-07-18T03:27:01.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:00 smithi064 ceph-mon[98059]: osdmap e43: 7 total, 6 up, 7 in 2023-07-18T03:27:01.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:00 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:27:01.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:00 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:27:01.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:00 smithi064 ceph-mon[98059]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:27:01.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:00 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:01.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:00 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:01.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:00 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:01.030 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:00 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:01.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:00 smithi134 ceph-mon[98216]: Detected new or changed devices on smithi040 2023-07-18T03:27:01.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:00 smithi134 ceph-mon[98216]: Adjusting osd_memory_target on smithi040 to 5724M 2023-07-18T03:27:01.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:00 smithi134 ceph-mon[98216]: from='osd.6 [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2023-07-18T03:27:01.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:00 smithi134 ceph-mon[98216]: osdmap e43: 7 total, 6 up, 7 in 2023-07-18T03:27:01.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:00 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:27:01.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:00 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:27:01.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:00 smithi134 ceph-mon[98216]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:27:01.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:00 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:01.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:00 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:01.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:00 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:01.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:00 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:01.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:00 smithi040 ceph-mon[96439]: Detected new or changed devices on smithi040 2023-07-18T03:27:01.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:00 smithi040 ceph-mon[96439]: Adjusting osd_memory_target on smithi040 to 5724M 2023-07-18T03:27:01.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:00 smithi040 ceph-mon[96439]: from='osd.6 [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2023-07-18T03:27:01.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:00 smithi040 ceph-mon[96439]: osdmap e43: 7 total, 6 up, 7 in 2023-07-18T03:27:01.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:00 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:27:01.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:00 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:27:01.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:00 smithi040 ceph-mon[96439]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-18T03:27:01.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:00 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:01.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:00 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:01.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:00 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:01.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:00 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:01.678 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.681+0000 7fda6d823700 1 -- 172.21.15.64:0/2019328741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda680fea80 msgr2=0x7fda680fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:01.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.681+0000 7fda6d823700 1 --2- 172.21.15.64:0/2019328741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda680fea80 0x7fda680fee90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fda58009990 tx=0x7fda58009ca0 comp rx=0 tx=0).stop 2023-07-18T03:27:01.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.681+0000 7fda6d823700 1 -- 172.21.15.64:0/2019328741 shutdown_connections 2023-07-18T03:27:01.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.681+0000 7fda6d823700 1 --2- 172.21.15.64:0/2019328741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda68100610 0x7fda68106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:01.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.681+0000 7fda6d823700 1 --2- 172.21.15.64:0/2019328741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda680fea80 0x7fda680fee90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:01.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.681+0000 7fda6d823700 1 --2- 172.21.15.64:0/2019328741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda680ffc80 0x7fda681000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:01.679 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.681+0000 7fda6d823700 1 -- 172.21.15.64:0/2019328741 >> 172.21.15.64:0/2019328741 conn(0x7fda680fa0b0 msgr2=0x7fda680fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:27:01.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.681+0000 7fda6d823700 1 -- 172.21.15.64:0/2019328741 shutdown_connections 2023-07-18T03:27:01.680 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.682+0000 7fda6d823700 1 -- 172.21.15.64:0/2019328741 wait complete. 2023-07-18T03:27:01.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.682+0000 7fda6d823700 1 Processor -- start 2023-07-18T03:27:01.681 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.682+0000 7fda6d823700 1 -- start start 2023-07-18T03:27:01.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda6d823700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda680fea80 0x7fda681958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:01.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda6d823700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda680ffc80 0x7fda68195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:01.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda6d823700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda68100610 0x7fda6819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:01.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda6d823700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6819d470 con 0x7fda680ffc80 2023-07-18T03:27:01.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda6d823700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6819d5b0 con 0x7fda68100610 2023-07-18T03:27:01.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda6d823700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6819d740 con 0x7fda680fea80 2023-07-18T03:27:01.682 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda677fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda68100610 0x7fda6819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:01.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda677fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda68100610 0x7fda6819cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37564/0 (socket says 172.21.15.64:37564) 2023-07-18T03:27:01.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda677fe700 1 -- 172.21.15.64:0/1490739850 learned_addr learned my addr 172.21.15.64:0/1490739850 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-07-18T03:27:01.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda66ffd700 1 --2- 172.21.15.64:0/1490739850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda680fea80 0x7fda681958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:01.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda667fc700 1 --2- 172.21.15.64:0/1490739850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda680ffc80 0x7fda68195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:01.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda677fe700 1 -- 172.21.15.64:0/1490739850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda680fea80 msgr2=0x7fda681958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:01.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda677fe700 1 --2- 172.21.15.64:0/1490739850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda680fea80 0x7fda681958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:01.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda677fe700 1 -- 172.21.15.64:0/1490739850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda680ffc80 msgr2=0x7fda68195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:01.683 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda677fe700 1 --2- 172.21.15.64:0/1490739850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda680ffc80 0x7fda68195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:01.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda677fe700 1 -- 172.21.15.64:0/1490739850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda580095f0 con 0x7fda68100610 2023-07-18T03:27:01.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda667fc700 1 --2- 172.21.15.64:0/1490739850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda680ffc80 0x7fda68195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:27:01.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda677fe700 1 --2- 172.21.15.64:0/1490739850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda68100610 0x7fda6819cea0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fda60007ee0 tx=0x7fda6000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:01.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda6c821700 1 -- 172.21.15.64:0/1490739850 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda60017070 con 0x7fda68100610 2023-07-18T03:27:01.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda6d823700 1 -- 172.21.15.64:0/1490739850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6819da20 con 0x7fda68100610 2023-07-18T03:27:01.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda6c821700 1 -- 172.21.15.64:0/1490739850 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda6000f040 con 0x7fda68100610 2023-07-18T03:27:01.684 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda6c821700 1 -- 172.21.15.64:0/1490739850 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda60005740 con 0x7fda68100610 2023-07-18T03:27:01.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.683+0000 7fda6d823700 1 -- 172.21.15.64:0/1490739850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6819df20 con 0x7fda68100610 2023-07-18T03:27:01.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.684+0000 7fda6d823700 1 -- 172.21.15.64:0/1490739850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6810a290 con 0x7fda68100610 2023-07-18T03:27:01.685 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.686+0000 7fda6c821700 1 -- 172.21.15.64:0/1490739850 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fda6000dc60 con 0x7fda68100610 2023-07-18T03:27:01.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.687+0000 7fda6c821700 1 --2- 172.21.15.64:0/1490739850 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fda500a1150 0x7fda500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:01.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.687+0000 7fda6c821700 1 -- 172.21.15.64:0/1490739850 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fda600cba20 con 0x7fda68100610 2023-07-18T03:27:01.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.687+0000 7fda66ffd700 1 --2- 172.21.15.64:0/1490739850 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fda500a1150 0x7fda500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:01.686 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.687+0000 7fda66ffd700 1 --2- 172.21.15.64:0/1490739850 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fda500a1150 0x7fda500a3600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda5800b5c0 tx=0x7fda58005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:01.687 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.690+0000 7fda6c821700 1 -- 172.21.15.64:0/1490739850 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fda60095de0 con 0x7fda68100610 2023-07-18T03:27:01.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:01 smithi064 ceph-mon[98059]: purged_snaps scrub starts 2023-07-18T03:27:01.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:01 smithi064 ceph-mon[98059]: purged_snaps scrub ok 2023-07-18T03:27:01.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:01 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:27:01.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:01 smithi064 ceph-mon[98059]: osd.6 [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677] boot 2023-07-18T03:27:01.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:01 smithi064 ceph-mon[98059]: osdmap e44: 7 total, 7 up, 7 in 2023-07-18T03:27:01.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:01 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:27:01.897 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:01.901+0000 7fda6d823700 1 -- 172.21.15.64:0/1490739850 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fda68068760 con 0x7fda500a1150 2023-07-18T03:27:02.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:01 smithi134 ceph-mon[98216]: purged_snaps scrub starts 2023-07-18T03:27:02.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:01 smithi134 ceph-mon[98216]: purged_snaps scrub ok 2023-07-18T03:27:02.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:01 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:27:02.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:01 smithi134 ceph-mon[98216]: osd.6 [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677] boot 2023-07-18T03:27:02.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:01 smithi134 ceph-mon[98216]: osdmap e44: 7 total, 7 up, 7 in 2023-07-18T03:27:02.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:01 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:27:02.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:01 smithi040 ceph-mon[96439]: purged_snaps scrub starts 2023-07-18T03:27:02.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:01 smithi040 ceph-mon[96439]: purged_snaps scrub ok 2023-07-18T03:27:02.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:01 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:27:02.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:01 smithi040 ceph-mon[96439]: osd.6 [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677] boot 2023-07-18T03:27:02.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:01 smithi040 ceph-mon[96439]: osdmap e44: 7 total, 7 up, 7 in 2023-07-18T03:27:02.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:01 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-18T03:27:02.945 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:02 smithi064 ceph-mon[98059]: osdmap e45: 7 total, 7 up, 7 in 2023-07-18T03:27:02.945 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:02 smithi064 ceph-mon[98059]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:02.946 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:27:02.946 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:27:02.946 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:03.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:02 smithi134 ceph-mon[98216]: osdmap e45: 7 total, 7 up, 7 in 2023-07-18T03:27:03.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:02 smithi134 ceph-mon[98216]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:03.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:27:03.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:27:03.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:03.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:02 smithi040 ceph-mon[96439]: osdmap e45: 7 total, 7 up, 7 in 2023-07-18T03:27:03.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:02 smithi040 ceph-mon[96439]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:03.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:27:03.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:27:03.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:03.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:03 smithi064 ceph-mon[98059]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:27:04.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:03 smithi134 ceph-mon[98216]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:27:04.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:03 smithi040 ceph-mon[96439]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:27:05.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:04 smithi134 ceph-mon[98216]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:05.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:04 smithi134 ceph-mon[98216]: from='client.? 172.21.15.64:0/638948779' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80a76d08-32dd-429e-b6e9-4a70a0f2b7e5"}]: dispatch 2023-07-18T03:27:05.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:04 smithi134 ceph-mon[98216]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80a76d08-32dd-429e-b6e9-4a70a0f2b7e5"}]: dispatch 2023-07-18T03:27:05.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:04 smithi134 ceph-mon[98216]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80a76d08-32dd-429e-b6e9-4a70a0f2b7e5"}]': finished 2023-07-18T03:27:05.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:04 smithi134 ceph-mon[98216]: osdmap e46: 8 total, 7 up, 8 in 2023-07-18T03:27:05.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:04 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:27:05.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:04 smithi040 ceph-mon[96439]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:05.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:04 smithi040 ceph-mon[96439]: from='client.? 172.21.15.64:0/638948779' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80a76d08-32dd-429e-b6e9-4a70a0f2b7e5"}]: dispatch 2023-07-18T03:27:05.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:04 smithi040 ceph-mon[96439]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80a76d08-32dd-429e-b6e9-4a70a0f2b7e5"}]: dispatch 2023-07-18T03:27:05.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:04 smithi040 ceph-mon[96439]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80a76d08-32dd-429e-b6e9-4a70a0f2b7e5"}]': finished 2023-07-18T03:27:05.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:04 smithi040 ceph-mon[96439]: osdmap e46: 8 total, 7 up, 8 in 2023-07-18T03:27:05.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:04 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:27:05.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:04 smithi064 ceph-mon[98059]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:05.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:04 smithi064 ceph-mon[98059]: from='client.? 172.21.15.64:0/638948779' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80a76d08-32dd-429e-b6e9-4a70a0f2b7e5"}]: dispatch 2023-07-18T03:27:05.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:04 smithi064 ceph-mon[98059]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80a76d08-32dd-429e-b6e9-4a70a0f2b7e5"}]: dispatch 2023-07-18T03:27:05.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:04 smithi064 ceph-mon[98059]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80a76d08-32dd-429e-b6e9-4a70a0f2b7e5"}]': finished 2023-07-18T03:27:05.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:04 smithi064 ceph-mon[98059]: osdmap e46: 8 total, 7 up, 8 in 2023-07-18T03:27:05.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:04 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:27:06.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:05 smithi134 ceph-mon[98216]: from='client.? 172.21.15.64:0/1794921721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:27:06.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:05 smithi040 ceph-mon[96439]: from='client.? 172.21.15.64:0/1794921721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:27:06.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:05 smithi064 ceph-mon[98059]: from='client.? 172.21.15.64:0/1794921721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:27:07.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:06 smithi134 ceph-mon[98216]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:07.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:06 smithi040 ceph-mon[96439]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:07.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:06 smithi064 ceph-mon[98059]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:08.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:07 smithi064 ceph-mon[98059]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:08.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:07 smithi134 ceph-mon[98216]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:08.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:07 smithi040 ceph-mon[96439]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:10.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:09 smithi064 ceph-mon[98059]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:10.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:09 smithi134 ceph-mon[98216]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:10.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:09 smithi040 ceph-mon[96439]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:11.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:11 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-18T03:27:11.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:11 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:11.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:11 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-18T03:27:11.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:11 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:11.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:11 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-18T03:27:11.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:11 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:12.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:12 smithi134 ceph-mon[98216]: Deploying daemon osd.7 on smithi064 2023-07-18T03:27:12.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:12 smithi134 ceph-mon[98216]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:12.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:12 smithi040 ceph-mon[96439]: Deploying daemon osd.7 on smithi064 2023-07-18T03:27:12.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:12 smithi040 ceph-mon[96439]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:12.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:12 smithi064 ceph-mon[98059]: Deploying daemon osd.7 on smithi064 2023-07-18T03:27:12.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:12 smithi064 ceph-mon[98059]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:14.065 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:13 smithi064 ceph-mon[98059]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:14.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:13 smithi134 ceph-mon[98216]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:14.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:13 smithi040 ceph-mon[96439]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:16.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:16.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:16.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:16 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:16.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:16 smithi064 ceph-mon[98059]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:16.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:16 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:16.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:16 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:16.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:16 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:16.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:16 smithi134 ceph-mon[98216]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:16.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:16 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:16.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:16 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:16.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:16 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:16.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:16 smithi040 ceph-mon[96439]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:18.089 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:17 smithi064 ceph-mon[98059]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:18.213 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:18.216+0000 7fda6c821700 1 -- 172.21.15.64:0/1490739850 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fda68068760 con 0x7fda500a1150 2023-07-18T03:27:18.213 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 7 on host 'smithi064' 2023-07-18T03:27:18.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:18.218+0000 7fda6d823700 1 -- 172.21.15.64:0/1490739850 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fda500a1150 msgr2=0x7fda500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:18.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:18.218+0000 7fda6d823700 1 --2- 172.21.15.64:0/1490739850 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fda500a1150 0x7fda500a3600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda5800b5c0 tx=0x7fda58005fb0 comp rx=0 tx=0).stop 2023-07-18T03:27:18.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:18.218+0000 7fda6d823700 1 -- 172.21.15.64:0/1490739850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda68100610 msgr2=0x7fda6819cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:18.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:18.218+0000 7fda6d823700 1 --2- 172.21.15.64:0/1490739850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda68100610 0x7fda6819cea0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fda60007ee0 tx=0x7fda6000d3b0 comp rx=0 tx=0).stop 2023-07-18T03:27:18.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:18.218+0000 7fda6d823700 1 -- 172.21.15.64:0/1490739850 shutdown_connections 2023-07-18T03:27:18.215 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:18.218+0000 7fda6d823700 1 --2- 172.21.15.64:0/1490739850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda68100610 0x7fda6819cea0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:18.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:18.218+0000 7fda6d823700 1 --2- 172.21.15.64:0/1490739850 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fda500a1150 0x7fda500a3600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:18.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:18.218+0000 7fda6d823700 1 --2- 172.21.15.64:0/1490739850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda680fea80 0x7fda681958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:18.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:18.218+0000 7fda6d823700 1 --2- 172.21.15.64:0/1490739850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda680ffc80 0x7fda68195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:18.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:18.218+0000 7fda6d823700 1 -- 172.21.15.64:0/1490739850 >> 172.21.15.64:0/1490739850 conn(0x7fda680fa0b0 msgr2=0x7fda68068050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:27:18.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:18.218+0000 7fda6d823700 1 -- 172.21.15.64:0/1490739850 shutdown_connections 2023-07-18T03:27:18.216 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:27:18.218+0000 7fda6d823700 1 -- 172.21.15.64:0/1490739850 wait complete. 2023-07-18T03:27:18.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:17 smithi134 ceph-mon[98216]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:18.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:17 smithi040 ceph-mon[96439]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:18.676 DEBUG:teuthology.orchestra.run.smithi064:osd.7> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.7.service 2023-07-18T03:27:18.678 INFO:tasks.cephadm:Deploying osd.8 on smithi134 with /dev/vg_nvme/lv_2... 2023-07-18T03:27:18.679 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-18T03:27:18.713 INFO:journalctl@ceph.osd.7.smithi064.stdout:-- Logs begin at Tue 2023-07-18 03:07:19 UTC. -- 2023-07-18T03:27:19.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:18 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:19.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:18 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:19.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:18 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:19.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:18 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:19.232 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:18 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:19.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:18 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:19.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:18 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:19.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:18 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:19.253 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:27:19.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:18 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:19.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:18 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:19.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:18 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:19.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:18 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:19.526 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:27:19 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:27:19.439+0000 7f587aa1b640 -1 osd.7 0 log_to_monitors true 2023-07-18T03:27:20.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:19 smithi134 ceph-mon[98216]: from='osd.7 [v2:172.21.15.64:6816/589664358,v1:172.21.15.64:6817/589664358]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-18T03:27:20.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:19 smithi134 ceph-mon[98216]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-18T03:27:20.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:19 smithi134 ceph-mon[98216]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:20.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:19 smithi040 ceph-mon[96439]: from='osd.7 [v2:172.21.15.64:6816/589664358,v1:172.21.15.64:6817/589664358]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-18T03:27:20.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:19 smithi040 ceph-mon[96439]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-18T03:27:20.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:19 smithi040 ceph-mon[96439]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:20.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:19 smithi064 ceph-mon[98059]: from='osd.7 [v2:172.21.15.64:6816/589664358,v1:172.21.15.64:6817/589664358]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-18T03:27:20.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:19 smithi064 ceph-mon[98059]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-18T03:27:20.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:19 smithi064 ceph-mon[98059]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:20.705 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-18T03:27:20.705 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-18T03:27:20.706 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2023-07-18T03:27:20.706 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2023-07-18T03:27:20.706 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109192 s, 960 MB/s 2023-07-18T03:27:20.706 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2023-07-18T03:27:21.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:20 smithi134 ceph-mon[98216]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-18T03:27:21.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:20 smithi134 ceph-mon[98216]: osdmap e47: 8 total, 7 up, 8 in 2023-07-18T03:27:21.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:20 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:27:21.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:20 smithi134 ceph-mon[98216]: from='osd.7 [v2:172.21.15.64:6816/589664358,v1:172.21.15.64:6817/589664358]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:27:21.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:20 smithi134 ceph-mon[98216]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:27:21.111 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi134:vg_nvme/lv_2 2023-07-18T03:27:21.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:20 smithi040 ceph-mon[96439]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-18T03:27:21.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:20 smithi040 ceph-mon[96439]: osdmap e47: 8 total, 7 up, 8 in 2023-07-18T03:27:21.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:20 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:27:21.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:20 smithi040 ceph-mon[96439]: from='osd.7 [v2:172.21.15.64:6816/589664358,v1:172.21.15.64:6817/589664358]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:27:21.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:20 smithi040 ceph-mon[96439]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:27:21.350 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:27:20 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:27:20.901+0000 7f586f892700 -1 osd.7 0 waiting for initial osdmap 2023-07-18T03:27:21.351 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:27:20 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:27:20.911+0000 7f586ae86700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:27:21.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:20 smithi064 ceph-mon[98059]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-18T03:27:21.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:20 smithi064 ceph-mon[98059]: osdmap e47: 8 total, 7 up, 8 in 2023-07-18T03:27:21.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:20 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:27:21.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:20 smithi064 ceph-mon[98059]: from='osd.7 [v2:172.21.15.64:6816/589664358,v1:172.21.15.64:6817/589664358]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:27:21.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:20 smithi064 ceph-mon[98059]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:27:21.671 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:27:22.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-07-18T03:27:22.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: osdmap e48: 8 total, 7 up, 8 in 2023-07-18T03:27:22.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:27:22.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:27:22.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: Detected new or changed devices on smithi064 2023-07-18T03:27:22.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:22.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:22.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:22.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:22.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:22.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: Adjusting osd_memory_target on smithi064 to 5722M 2023-07-18T03:27:22.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:22.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:22.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:22.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:22.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:22.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:22.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:22.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:22.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: osd.7 [v2:172.21.15.64:6816/589664358,v1:172.21.15.64:6817/589664358] boot 2023-07-18T03:27:22.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: osdmap e49: 8 total, 8 up, 8 in 2023-07-18T03:27:22.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:27:22.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:21 smithi134 ceph-mon[98216]: pgmap v154: 1 pgs: 1 remapped; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:22.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-07-18T03:27:22.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: osdmap e48: 8 total, 7 up, 8 in 2023-07-18T03:27:22.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:27:22.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:27:22.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: Detected new or changed devices on smithi064 2023-07-18T03:27:22.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:22.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:22.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:22.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:22.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:22.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: Adjusting osd_memory_target on smithi064 to 5722M 2023-07-18T03:27:22.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:22.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:22.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:22.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:22.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:22.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:22.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:22.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:22.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: osd.7 [v2:172.21.15.64:6816/589664358,v1:172.21.15.64:6817/589664358] boot 2023-07-18T03:27:22.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: osdmap e49: 8 total, 8 up, 8 in 2023-07-18T03:27:22.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:27:22.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:21 smithi040 ceph-mon[96439]: pgmap v154: 1 pgs: 1 remapped; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:22.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-07-18T03:27:22.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: osdmap e48: 8 total, 7 up, 8 in 2023-07-18T03:27:22.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:27:22.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:27:22.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: Detected new or changed devices on smithi064 2023-07-18T03:27:22.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:22.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:22.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:22.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:22.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:22.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: Adjusting osd_memory_target on smithi064 to 5722M 2023-07-18T03:27:22.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:22.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:22.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:22.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:22.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:22.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:22.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:22.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:22.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: osd.7 [v2:172.21.15.64:6816/589664358,v1:172.21.15.64:6817/589664358] boot 2023-07-18T03:27:22.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: osdmap e49: 8 total, 8 up, 8 in 2023-07-18T03:27:22.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-18T03:27:22.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:21 smithi064 ceph-mon[98059]: pgmap v154: 1 pgs: 1 remapped; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-18T03:27:22.825 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.828+0000 7f8f74c94700 1 -- 172.21.15.134:0/3741362379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f70100610 msgr2=0x7f8f70106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:22.825 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.828+0000 7f8f74c94700 1 --2- 172.21.15.134:0/3741362379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f70100610 0x7f8f70106ed0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800b0a0 tx=0x7f8f6800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:27:22.825 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.828+0000 7f8f74c94700 1 -- 172.21.15.134:0/3741362379 shutdown_connections 2023-07-18T03:27:22.826 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.828+0000 7f8f74c94700 1 --2- 172.21.15.134:0/3741362379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f70100610 0x7f8f70106ed0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:22.826 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.828+0000 7f8f74c94700 1 --2- 172.21.15.134:0/3741362379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f700fea80 0x7f8f700fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:22.826 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.828+0000 7f8f74c94700 1 --2- 172.21.15.134:0/3741362379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f700ffc80 0x7f8f701000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:22.826 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.828+0000 7f8f74c94700 1 -- 172.21.15.134:0/3741362379 >> 172.21.15.134:0/3741362379 conn(0x7f8f700fa0b0 msgr2=0x7f8f700fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:27:22.826 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.828+0000 7f8f74c94700 1 -- 172.21.15.134:0/3741362379 shutdown_connections 2023-07-18T03:27:22.826 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.828+0000 7f8f74c94700 1 -- 172.21.15.134:0/3741362379 wait complete. 2023-07-18T03:27:22.828 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f74c94700 1 Processor -- start 2023-07-18T03:27:22.829 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f74c94700 1 -- start start 2023-07-18T03:27:22.829 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f74c94700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f700fea80 0x7f8f701948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:22.829 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f74c94700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f700ffc80 0x7f8f70194de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:22.829 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f74c94700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f70100610 0x7f8f7019be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:22.829 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f74c94700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7019c430 con 0x7f8f70100610 2023-07-18T03:27:22.829 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f74c94700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7019c570 con 0x7f8f700fea80 2023-07-18T03:27:22.829 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f74c94700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7019c700 con 0x7f8f700ffc80 2023-07-18T03:27:22.830 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f6dd9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f700ffc80 0x7f8f70194de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:22.830 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f6dd9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f700ffc80 0x7f8f70194de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:48232/0 (socket says 172.21.15.134:48232) 2023-07-18T03:27:22.830 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f6dd9b700 1 -- 172.21.15.134:0/1691848696 learned_addr learned my addr 172.21.15.134:0/1691848696 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-07-18T03:27:22.830 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f6e59c700 1 --2- 172.21.15.134:0/1691848696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f700fea80 0x7f8f701948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:22.830 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f6ed9d700 1 --2- 172.21.15.134:0/1691848696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f70100610 0x7f8f7019be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:22.830 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f6dd9b700 1 -- 172.21.15.134:0/1691848696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f700fea80 msgr2=0x7f8f701948a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:22.831 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f6dd9b700 1 --2- 172.21.15.134:0/1691848696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f700fea80 0x7f8f701948a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:22.831 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f6dd9b700 1 -- 172.21.15.134:0/1691848696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f70100610 msgr2=0x7f8f7019be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:22.831 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f6dd9b700 1 --2- 172.21.15.134:0/1691848696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f70100610 0x7f8f7019be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:22.831 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f6dd9b700 1 -- 172.21.15.134:0/1691848696 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f68009d00 con 0x7f8f700ffc80 2023-07-18T03:27:22.831 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f6ed9d700 1 --2- 172.21.15.134:0/1691848696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f70100610 0x7f8f7019be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:27:22.831 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.829+0000 7f8f6dd9b700 1 --2- 172.21.15.134:0/1691848696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f700ffc80 0x7f8f70194de0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000e9d0 tx=0x7f8f6000ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:22.831 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.830+0000 7f8f5f7fe700 1 -- 172.21.15.134:0/1691848696 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60009790 con 0x7f8f700ffc80 2023-07-18T03:27:22.832 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.830+0000 7f8f5f7fe700 1 -- 172.21.15.134:0/1691848696 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f6000cbd0 con 0x7f8f700ffc80 2023-07-18T03:27:22.832 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.830+0000 7f8f74c94700 1 -- 172.21.15.134:0/1691848696 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f7019c9e0 con 0x7f8f700ffc80 2023-07-18T03:27:22.832 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.830+0000 7f8f5f7fe700 1 -- 172.21.15.134:0/1691848696 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60018680 con 0x7f8f700ffc80 2023-07-18T03:27:22.832 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.830+0000 7f8f74c94700 1 -- 172.21.15.134:0/1691848696 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7019cf30 con 0x7f8f700ffc80 2023-07-18T03:27:22.832 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.831+0000 7f8f74c94700 1 -- 172.21.15.134:0/1691848696 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f70106030 con 0x7f8f700ffc80 2023-07-18T03:27:22.833 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.832+0000 7f8f5f7fe700 1 -- 172.21.15.134:0/1691848696 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8f6000cd40 con 0x7f8f700ffc80 2023-07-18T03:27:22.833 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.832+0000 7f8f5f7fe700 1 --2- 172.21.15.134:0/1691848696 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8f580a11d0 0x7f8f580a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:22.834 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.832+0000 7f8f5f7fe700 1 -- 172.21.15.134:0/1691848696 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f8f60014070 con 0x7f8f700ffc80 2023-07-18T03:27:22.834 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.832+0000 7f8f6e59c700 1 --2- 172.21.15.134:0/1691848696 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8f580a11d0 0x7f8f580a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:22.834 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.833+0000 7f8f6e59c700 1 --2- 172.21.15.134:0/1691848696 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8f580a11d0 0x7f8f580a3680 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8f700ffae0 tx=0x7f8f6400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:22.834 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:22.835+0000 7f8f5f7fe700 1 -- 172.21.15.134:0/1691848696 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8f600965a0 con 0x7f8f700ffc80 2023-07-18T03:27:23.027 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:23.030+0000 7f8f74c94700 1 -- 172.21.15.134:0/1691848696 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f70060950 con 0x7f8f580a11d0 2023-07-18T03:27:23.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:22 smithi134 ceph-mon[98216]: purged_snaps scrub starts 2023-07-18T03:27:23.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:22 smithi134 ceph-mon[98216]: purged_snaps scrub ok 2023-07-18T03:27:23.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:22 smithi134 ceph-mon[98216]: osdmap e50: 8 total, 8 up, 8 in 2023-07-18T03:27:23.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:22 smithi040 ceph-mon[96439]: purged_snaps scrub starts 2023-07-18T03:27:23.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:22 smithi040 ceph-mon[96439]: purged_snaps scrub ok 2023-07-18T03:27:23.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:22 smithi040 ceph-mon[96439]: osdmap e50: 8 total, 8 up, 8 in 2023-07-18T03:27:23.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:22 smithi064 ceph-mon[98059]: purged_snaps scrub starts 2023-07-18T03:27:23.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:22 smithi064 ceph-mon[98059]: purged_snaps scrub ok 2023-07-18T03:27:23.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:22 smithi064 ceph-mon[98059]: osdmap e50: 8 total, 8 up, 8 in 2023-07-18T03:27:24.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:23 smithi134 ceph-mon[98216]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:27:24.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:23 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:27:24.096 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:23 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:27:24.096 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:23 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:24.096 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:23 smithi134 ceph-mon[98216]: osdmap e51: 8 total, 8 up, 8 in 2023-07-18T03:27:24.096 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:23 smithi134 ceph-mon[98216]: pgmap v157: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:24.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:23 smithi040 ceph-mon[96439]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:27:24.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:23 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:27:24.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:23 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:27:24.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:23 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:24.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:23 smithi040 ceph-mon[96439]: osdmap e51: 8 total, 8 up, 8 in 2023-07-18T03:27:24.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:23 smithi040 ceph-mon[96439]: pgmap v157: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:24.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:23 smithi064 ceph-mon[98059]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:27:24.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:27:24.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:27:24.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:24.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:23 smithi064 ceph-mon[98059]: osdmap e51: 8 total, 8 up, 8 in 2023-07-18T03:27:24.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:23 smithi064 ceph-mon[98059]: pgmap v157: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:25.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:25 smithi040 ceph-mon[96439]: osdmap e52: 8 total, 8 up, 8 in 2023-07-18T03:27:25.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:25 smithi040 ceph-mon[96439]: from='client.? 172.21.15.134:0/1944205186' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0442a0d-4350-4f98-a6a1-346905e55e52"}]: dispatch 2023-07-18T03:27:25.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:25 smithi040 ceph-mon[96439]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0442a0d-4350-4f98-a6a1-346905e55e52"}]: dispatch 2023-07-18T03:27:25.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:25 smithi040 ceph-mon[96439]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0442a0d-4350-4f98-a6a1-346905e55e52"}]': finished 2023-07-18T03:27:25.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:25 smithi040 ceph-mon[96439]: osdmap e53: 9 total, 8 up, 9 in 2023-07-18T03:27:25.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:25 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:27:25.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:25 smithi064 ceph-mon[98059]: osdmap e52: 8 total, 8 up, 8 in 2023-07-18T03:27:25.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:25 smithi064 ceph-mon[98059]: from='client.? 172.21.15.134:0/1944205186' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0442a0d-4350-4f98-a6a1-346905e55e52"}]: dispatch 2023-07-18T03:27:25.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:25 smithi064 ceph-mon[98059]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0442a0d-4350-4f98-a6a1-346905e55e52"}]: dispatch 2023-07-18T03:27:25.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:25 smithi064 ceph-mon[98059]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0442a0d-4350-4f98-a6a1-346905e55e52"}]': finished 2023-07-18T03:27:25.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:25 smithi064 ceph-mon[98059]: osdmap e53: 9 total, 8 up, 9 in 2023-07-18T03:27:25.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:25 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:27:26.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:25 smithi134 ceph-mon[98216]: osdmap e52: 8 total, 8 up, 8 in 2023-07-18T03:27:26.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:25 smithi134 ceph-mon[98216]: from='client.? 172.21.15.134:0/1944205186' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0442a0d-4350-4f98-a6a1-346905e55e52"}]: dispatch 2023-07-18T03:27:26.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:25 smithi134 ceph-mon[98216]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0442a0d-4350-4f98-a6a1-346905e55e52"}]: dispatch 2023-07-18T03:27:26.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:25 smithi134 ceph-mon[98216]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0442a0d-4350-4f98-a6a1-346905e55e52"}]': finished 2023-07-18T03:27:26.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:25 smithi134 ceph-mon[98216]: osdmap e53: 9 total, 8 up, 9 in 2023-07-18T03:27:26.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:25 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:27:26.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:26 smithi040 ceph-mon[96439]: pgmap v160: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:26.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:26 smithi040 ceph-mon[96439]: from='client.? 172.21.15.134:0/192894832' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:27:26.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:26 smithi064 ceph-mon[98059]: pgmap v160: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:26.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:26 smithi064 ceph-mon[98059]: from='client.? 172.21.15.134:0/192894832' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:27:27.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:26 smithi134 ceph-mon[98216]: pgmap v160: 1 pgs: 1 remapped; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:27.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:26 smithi134 ceph-mon[98216]: from='client.? 172.21.15.134:0/192894832' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:27:28.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:27 smithi064 ceph-mon[98059]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:28.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:27 smithi134 ceph-mon[98216]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:28.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:27 smithi040 ceph-mon[96439]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:30.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:29 smithi134 ceph-mon[98216]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:30.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:29 smithi064 ceph-mon[98059]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:30.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:29 smithi040 ceph-mon[96439]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:32.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:32 smithi134 ceph-mon[98216]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:32.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-18T03:27:32.829 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:32.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:32 smithi040 ceph-mon[96439]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:32.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-18T03:27:32.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:32.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:32 smithi064 ceph-mon[98059]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:32.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-18T03:27:32.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:33.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:33 smithi134 ceph-mon[98216]: Deploying daemon osd.8 on smithi134 2023-07-18T03:27:33.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:33 smithi040 ceph-mon[96439]: Deploying daemon osd.8 on smithi134 2023-07-18T03:27:33.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:33 smithi064 ceph-mon[98059]: Deploying daemon osd.8 on smithi134 2023-07-18T03:27:34.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:34 smithi134 ceph-mon[98216]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:34.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:34 smithi040 ceph-mon[96439]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:34.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:34 smithi064 ceph-mon[98059]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:36.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:35 smithi064 ceph-mon[98059]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:36.144 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:35 smithi134 ceph-mon[98216]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:36.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:35 smithi040 ceph-mon[96439]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:37.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:37 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:37.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:37 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:37.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:37 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:37.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:37 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:37.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:37 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:37.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:37 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:37.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:37 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:37.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:37 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:37.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:37 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:38.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:38 smithi134 ceph-mon[98216]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:38.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:38 smithi040 ceph-mon[96439]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:38.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:38 smithi064 ceph-mon[98059]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:39.244 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:39.246+0000 7f8f5f7fe700 1 -- 172.21.15.134:0/1691848696 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8f70060950 con 0x7f8f580a11d0 2023-07-18T03:27:39.244 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 8 on host 'smithi134' 2023-07-18T03:27:39.246 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:39.249+0000 7f8f74c94700 1 -- 172.21.15.134:0/1691848696 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8f580a11d0 msgr2=0x7f8f580a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:39.246 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:39.249+0000 7f8f74c94700 1 --2- 172.21.15.134:0/1691848696 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8f580a11d0 0x7f8f580a3680 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8f700ffae0 tx=0x7f8f6400a380 comp rx=0 tx=0).stop 2023-07-18T03:27:39.246 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:39.249+0000 7f8f74c94700 1 -- 172.21.15.134:0/1691848696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f700ffc80 msgr2=0x7f8f70194de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:39.246 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:39.249+0000 7f8f74c94700 1 --2- 172.21.15.134:0/1691848696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f700ffc80 0x7f8f70194de0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000e9d0 tx=0x7f8f6000ece0 comp rx=0 tx=0).stop 2023-07-18T03:27:39.247 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:39.249+0000 7f8f74c94700 1 -- 172.21.15.134:0/1691848696 shutdown_connections 2023-07-18T03:27:39.247 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:39.249+0000 7f8f74c94700 1 --2- 172.21.15.134:0/1691848696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f70100610 0x7f8f7019be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:39.247 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:39.249+0000 7f8f74c94700 1 --2- 172.21.15.134:0/1691848696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f700fea80 0x7f8f701948a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:39.247 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:39.249+0000 7f8f74c94700 1 --2- 172.21.15.134:0/1691848696 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8f580a11d0 0x7f8f580a3680 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:39.247 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:39.249+0000 7f8f74c94700 1 --2- 172.21.15.134:0/1691848696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f700ffc80 0x7f8f70194de0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:39.247 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:39.249+0000 7f8f74c94700 1 -- 172.21.15.134:0/1691848696 >> 172.21.15.134:0/1691848696 conn(0x7f8f700fa0b0 msgr2=0x7f8f70103170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:27:39.247 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:39.249+0000 7f8f74c94700 1 -- 172.21.15.134:0/1691848696 shutdown_connections 2023-07-18T03:27:39.247 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:27:39.249+0000 7f8f74c94700 1 -- 172.21.15.134:0/1691848696 wait complete. 2023-07-18T03:27:39.677 DEBUG:teuthology.orchestra.run.smithi134:osd.8> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.8.service 2023-07-18T03:27:39.680 INFO:tasks.cephadm:Deploying osd.9 on smithi040 with /dev/vg_nvme/lv_1... 2023-07-18T03:27:39.681 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-18T03:27:39.715 INFO:journalctl@ceph.osd.8.smithi134.stdout:-- Logs begin at Tue 2023-07-18 03:07:14 UTC. -- 2023-07-18T03:27:39.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:39.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:39.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:39.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:39.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:39.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:39.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:39.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:39.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:39.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:39.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:39.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:39.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:39.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:39.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:39.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:39.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:39.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:39.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:39.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:39.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:39.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:39.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:39.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:40.258 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:27:40.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:40 smithi134 ceph-mon[98216]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:40.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:40 smithi134 ceph-mon[98216]: from='osd.8 [v2:172.21.15.134:6816/2602435562,v1:172.21.15.134:6817/2602435562]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-18T03:27:40.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:40 smithi134 ceph-mon[98216]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-18T03:27:40.567 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:27:40 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:27:40.378+0000 7f1109def640 -1 osd.8 0 log_to_monitors true 2023-07-18T03:27:40.763 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:40 smithi040 ceph-mon[96439]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:40.763 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:40 smithi040 ceph-mon[96439]: from='osd.8 [v2:172.21.15.134:6816/2602435562,v1:172.21.15.134:6817/2602435562]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-18T03:27:40.763 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:40 smithi040 ceph-mon[96439]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-18T03:27:40.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:40 smithi064 ceph-mon[98059]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:40.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:40 smithi064 ceph-mon[98059]: from='osd.8 [v2:172.21.15.134:6816/2602435562,v1:172.21.15.134:6817/2602435562]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-18T03:27:40.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:40 smithi064 ceph-mon[98059]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-18T03:27:41.726 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-18T03:27:41.726 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-18T03:27:41.726 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2023-07-18T03:27:41.726 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2023-07-18T03:27:41.726 INFO:teuthology.orchestra.run.smithi040.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0107329 s, 977 MB/s 2023-07-18T03:27:41.726 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 2023-07-18T03:27:41.727 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2023-07-18T03:27:41.742 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:41 smithi134 ceph-mon[98216]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-18T03:27:41.743 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:41 smithi134 ceph-mon[98216]: osdmap e54: 9 total, 8 up, 9 in 2023-07-18T03:27:41.743 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:41 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:27:41.743 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:41 smithi134 ceph-mon[98216]: from='osd.8 [v2:172.21.15.134:6816/2602435562,v1:172.21.15.134:6817/2602435562]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:27:41.743 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:41 smithi134 ceph-mon[98216]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:27:41.743 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:27:41 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:27:41.512+0000 7f10fec66700 -1 osd.8 0 waiting for initial osdmap 2023-07-18T03:27:41.744 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:27:41 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:27:41.522+0000 7f10f7254700 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:27:41.764 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:41 smithi040 ceph-mon[96439]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-18T03:27:41.764 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:41 smithi040 ceph-mon[96439]: osdmap e54: 9 total, 8 up, 9 in 2023-07-18T03:27:41.765 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:41 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:27:41.765 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:41 smithi040 ceph-mon[96439]: from='osd.8 [v2:172.21.15.134:6816/2602435562,v1:172.21.15.134:6817/2602435562]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:27:41.765 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:41 smithi040 ceph-mon[96439]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:27:41.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:41 smithi064 ceph-mon[98059]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-18T03:27:41.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:41 smithi064 ceph-mon[98059]: osdmap e54: 9 total, 8 up, 9 in 2023-07-18T03:27:41.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:41 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:27:41.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:41 smithi064 ceph-mon[98059]: from='osd.8 [v2:172.21.15.134:6816/2602435562,v1:172.21.15.134:6817/2602435562]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:27:41.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:41 smithi064 ceph-mon[98059]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:27:42.107 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi040:vg_nvme/lv_1 2023-07-18T03:27:42.666 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:27:42.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:42 smithi134 ceph-mon[98216]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-18T03:27:42.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:42 smithi134 ceph-mon[98216]: osdmap e55: 9 total, 8 up, 9 in 2023-07-18T03:27:42.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:42 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:27:42.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:42 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:27:42.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:42 smithi134 ceph-mon[98216]: osd.8 [v2:172.21.15.134:6816/2602435562,v1:172.21.15.134:6817/2602435562] boot 2023-07-18T03:27:42.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:42 smithi134 ceph-mon[98216]: osdmap e56: 9 total, 9 up, 9 in 2023-07-18T03:27:42.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:42 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:27:42.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:42 smithi134 ceph-mon[98216]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:42.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:42 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:42.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:42 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:42.820 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:42 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:42.820 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:42 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:42.820 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:42 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:42.821 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:42 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:42.821 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:42 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:42.821 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:42 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:42.821 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:42 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:42.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:42 smithi040 ceph-mon[96439]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-18T03:27:42.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:42 smithi040 ceph-mon[96439]: osdmap e55: 9 total, 8 up, 9 in 2023-07-18T03:27:42.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:27:42.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:27:42.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:42 smithi040 ceph-mon[96439]: osd.8 [v2:172.21.15.134:6816/2602435562,v1:172.21.15.134:6817/2602435562] boot 2023-07-18T03:27:42.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:42 smithi040 ceph-mon[96439]: osdmap e56: 9 total, 9 up, 9 in 2023-07-18T03:27:42.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:27:42.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:42 smithi040 ceph-mon[96439]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:42.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:42.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:42.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:42.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:42.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:42.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:42.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:42.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:42.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:42.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:42 smithi064 ceph-mon[98059]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-18T03:27:42.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:42 smithi064 ceph-mon[98059]: osdmap e55: 9 total, 8 up, 9 in 2023-07-18T03:27:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:42 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:27:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:42 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:27:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:42 smithi064 ceph-mon[98059]: osd.8 [v2:172.21.15.134:6816/2602435562,v1:172.21.15.134:6817/2602435562] boot 2023-07-18T03:27:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:42 smithi064 ceph-mon[98059]: osdmap e56: 9 total, 9 up, 9 in 2023-07-18T03:27:42.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:42 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-18T03:27:42.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:42 smithi064 ceph-mon[98059]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-18T03:27:42.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:42 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:42.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:42 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:42.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:42 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:42.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:42 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:42.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:42 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:27:42.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:42 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:42.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:42 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:42.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:42 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:27:42.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:42 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:43.737 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:43 smithi040 ceph-mon[96439]: purged_snaps scrub starts 2023-07-18T03:27:43.738 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:43 smithi040 ceph-mon[96439]: purged_snaps scrub ok 2023-07-18T03:27:43.738 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:43 smithi040 ceph-mon[96439]: Detected new or changed devices on smithi134 2023-07-18T03:27:43.738 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:43 smithi040 ceph-mon[96439]: Adjusting osd_memory_target on smithi134 to 5722M 2023-07-18T03:27:43.738 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:43 smithi040 ceph-mon[96439]: osdmap e57: 9 total, 9 up, 9 in 2023-07-18T03:27:43.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:43 smithi064 ceph-mon[98059]: purged_snaps scrub starts 2023-07-18T03:27:43.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:43 smithi064 ceph-mon[98059]: purged_snaps scrub ok 2023-07-18T03:27:43.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:43 smithi064 ceph-mon[98059]: Detected new or changed devices on smithi134 2023-07-18T03:27:43.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:43 smithi064 ceph-mon[98059]: Adjusting osd_memory_target on smithi134 to 5722M 2023-07-18T03:27:43.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:43 smithi064 ceph-mon[98059]: osdmap e57: 9 total, 9 up, 9 in 2023-07-18T03:27:43.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.927+0000 7fdc122e3700 1 -- 172.21.15.40:0/3676181522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc0c0ffc80 msgr2=0x7fdc0c1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:43.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.927+0000 7fdc122e3700 1 --2- 172.21.15.40:0/3676181522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc0c0ffc80 0x7fdc0c1000d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009990 tx=0x7fdc00009ca0 comp rx=0 tx=0).stop 2023-07-18T03:27:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.927+0000 7fdc122e3700 1 -- 172.21.15.40:0/3676181522 shutdown_connections 2023-07-18T03:27:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.927+0000 7fdc122e3700 1 --2- 172.21.15.40:0/3676181522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc0c100610 0x7fdc0c106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.927+0000 7fdc122e3700 1 --2- 172.21.15.40:0/3676181522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c0fea80 0x7fdc0c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.927+0000 7fdc122e3700 1 --2- 172.21.15.40:0/3676181522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc0c0ffc80 0x7fdc0c1000d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.927+0000 7fdc122e3700 1 -- 172.21.15.40:0/3676181522 >> 172.21.15.40:0/3676181522 conn(0x7fdc0c0fa0b0 msgr2=0x7fdc0c0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:27:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.927+0000 7fdc122e3700 1 -- 172.21.15.40:0/3676181522 shutdown_connections 2023-07-18T03:27:43.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.927+0000 7fdc122e3700 1 -- 172.21.15.40:0/3676181522 wait complete. 2023-07-18T03:27:43.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.928+0000 7fdc122e3700 1 Processor -- start 2023-07-18T03:27:43.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.928+0000 7fdc122e3700 1 -- start start 2023-07-18T03:27:43.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.928+0000 7fdc122e3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc0c0fea80 0x7fdc0c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:43.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.928+0000 7fdc122e3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c0ffc80 0x7fdc0c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:43.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.928+0000 7fdc122e3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc0c100610 0x7fdc0c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:43.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.928+0000 7fdc122e3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c19f650 con 0x7fdc0c0fea80 2023-07-18T03:27:43.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.928+0000 7fdc122e3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c19f790 con 0x7fdc0c100610 2023-07-18T03:27:43.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.928+0000 7fdc122e3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c19f920 con 0x7fdc0c0ffc80 2023-07-18T03:27:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.928+0000 7fdc0bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc0c0fea80 0x7fdc0c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.929+0000 7fdc0bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc0c0fea80 0x7fdc0c197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55542/0 (socket says 172.21.15.40:55542) 2023-07-18T03:27:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.929+0000 7fdc0bfff700 1 -- 172.21.15.40:0/239276211 learned_addr learned my addr 172.21.15.40:0/239276211 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:27:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.929+0000 7fdc10880700 1 --2- 172.21.15.40:0/239276211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc0c100610 0x7fdc0c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.929+0000 7fdc0b7fe700 1 --2- 172.21.15.40:0/239276211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c0ffc80 0x7fdc0c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.929+0000 7fdc0bfff700 1 -- 172.21.15.40:0/239276211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c0ffc80 msgr2=0x7fdc0c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:43.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.929+0000 7fdc0bfff700 1 --2- 172.21.15.40:0/239276211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c0ffc80 0x7fdc0c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:43.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.929+0000 7fdc0bfff700 1 -- 172.21.15.40:0/239276211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc0c100610 msgr2=0x7fdc0c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:27:43.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.929+0000 7fdc0bfff700 1 --2- 172.21.15.40:0/239276211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc0c100610 0x7fdc0c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:27:43.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.929+0000 7fdc0bfff700 1 -- 172.21.15.40:0/239276211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc04009c40 con 0x7fdc0c0fea80 2023-07-18T03:27:43.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.929+0000 7fdc0bfff700 1 --2- 172.21.15.40:0/239276211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc0c0fea80 0x7fdc0c197ac0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00e9a0 tx=0x7fdbfc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:43.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.929+0000 7fdc097fa700 1 -- 172.21.15.40:0/239276211 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc009790 con 0x7fdc0c0fea80 2023-07-18T03:27:43.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.929+0000 7fdc122e3700 1 -- 172.21.15.40:0/239276211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc000095f0 con 0x7fdc0c0fea80 2023-07-18T03:27:43.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.929+0000 7fdc097fa700 1 -- 172.21.15.40:0/239276211 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbfc00c7c0 con 0x7fdc0c0fea80 2023-07-18T03:27:43.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.929+0000 7fdc097fa700 1 -- 172.21.15.40:0/239276211 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc003c20 con 0x7fdc0c0fea80 2023-07-18T03:27:43.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.929+0000 7fdc122e3700 1 -- 172.21.15.40:0/239276211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0c19ff70 con 0x7fdc0c0fea80 2023-07-18T03:27:43.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.930+0000 7fdc122e3700 1 -- 172.21.15.40:0/239276211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0c10a240 con 0x7fdc0c0fea80 2023-07-18T03:27:43.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.931+0000 7fdc097fa700 1 -- 172.21.15.40:0/239276211 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdbfc00cc30 con 0x7fdc0c0fea80 2023-07-18T03:27:43.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.932+0000 7fdc097fa700 1 --2- 172.21.15.40:0/239276211 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdbf40a1150 0x7fdbf40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:27:43.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.932+0000 7fdc0b7fe700 1 --2- 172.21.15.40:0/239276211 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdbf40a1150 0x7fdbf40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:27:43.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.932+0000 7fdc097fa700 1 -- 172.21.15.40:0/239276211 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5858+0+0 (secure 0 0 0) 0x7fdbfc014070 con 0x7fdc0c0fea80 2023-07-18T03:27:43.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.932+0000 7fdc0b7fe700 1 --2- 172.21.15.40:0/239276211 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdbf40a1150 0x7fdbf40a3600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000b2e0 tx=0x7fdc00005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:27:43.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:43.937+0000 7fdc097fa700 1 -- 172.21.15.40:0/239276211 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdbfc0961d0 con 0x7fdc0c0fea80 2023-07-18T03:27:44.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:43 smithi134 ceph-mon[98216]: purged_snaps scrub starts 2023-07-18T03:27:44.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:43 smithi134 ceph-mon[98216]: purged_snaps scrub ok 2023-07-18T03:27:44.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:43 smithi134 ceph-mon[98216]: Detected new or changed devices on smithi134 2023-07-18T03:27:44.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:43 smithi134 ceph-mon[98216]: Adjusting osd_memory_target on smithi134 to 5722M 2023-07-18T03:27:44.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:43 smithi134 ceph-mon[98216]: osdmap e57: 9 total, 9 up, 9 in 2023-07-18T03:27:44.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:27:44.142+0000 7fdc122e3700 1 -- 172.21.15.40:0/239276211 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc0c069030 con 0x7fdbf40a1150 2023-07-18T03:27:44.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:44 smithi064 ceph-mon[98059]: osdmap e58: 9 total, 9 up, 9 in 2023-07-18T03:27:44.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:44 smithi064 ceph-mon[98059]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:27:44.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:27:44.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:27:44.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:44.936 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:44 smithi040 ceph-mon[96439]: osdmap e58: 9 total, 9 up, 9 in 2023-07-18T03:27:44.936 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:44 smithi040 ceph-mon[96439]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:27:44.937 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:27:44.937 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:27:44.937 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:45.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:44 smithi134 ceph-mon[98216]: osdmap e58: 9 total, 9 up, 9 in 2023-07-18T03:27:45.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:44 smithi134 ceph-mon[98216]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:27:45.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:27:45.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:27:45.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:45.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:45 smithi064 ceph-mon[98059]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:27:45.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:45 smithi064 ceph-mon[98059]: osdmap e59: 9 total, 9 up, 9 in 2023-07-18T03:27:46.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:45 smithi134 ceph-mon[98216]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:27:46.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:45 smithi134 ceph-mon[98216]: osdmap e59: 9 total, 9 up, 9 in 2023-07-18T03:27:46.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:45 smithi040 ceph-mon[96439]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:27:46.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:45 smithi040 ceph-mon[96439]: osdmap e59: 9 total, 9 up, 9 in 2023-07-18T03:27:47.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:46 smithi134 ceph-mon[98216]: osdmap e60: 9 total, 9 up, 9 in 2023-07-18T03:27:47.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:46 smithi134 ceph-mon[98216]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:27:47.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:46 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/765536909' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56920a5e-7f3e-4776-921c-682aa946a6b7"}]: dispatch 2023-07-18T03:27:47.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:46 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/765536909' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56920a5e-7f3e-4776-921c-682aa946a6b7"}]': finished 2023-07-18T03:27:47.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:46 smithi134 ceph-mon[98216]: osdmap e61: 10 total, 9 up, 10 in 2023-07-18T03:27:47.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:46 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:27:47.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:46 smithi040 ceph-mon[96439]: osdmap e60: 9 total, 9 up, 9 in 2023-07-18T03:27:47.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:46 smithi040 ceph-mon[96439]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:27:47.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:46 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/765536909' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56920a5e-7f3e-4776-921c-682aa946a6b7"}]: dispatch 2023-07-18T03:27:47.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:46 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/765536909' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56920a5e-7f3e-4776-921c-682aa946a6b7"}]': finished 2023-07-18T03:27:47.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:46 smithi040 ceph-mon[96439]: osdmap e61: 10 total, 9 up, 10 in 2023-07-18T03:27:47.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:46 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:27:47.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:46 smithi064 ceph-mon[98059]: osdmap e60: 9 total, 9 up, 9 in 2023-07-18T03:27:47.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:46 smithi064 ceph-mon[98059]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:27:47.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:46 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/765536909' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56920a5e-7f3e-4776-921c-682aa946a6b7"}]: dispatch 2023-07-18T03:27:47.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:46 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/765536909' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56920a5e-7f3e-4776-921c-682aa946a6b7"}]': finished 2023-07-18T03:27:47.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:46 smithi064 ceph-mon[98059]: osdmap e61: 10 total, 9 up, 10 in 2023-07-18T03:27:47.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:46 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:27:48.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:47 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3698929094' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:27:48.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:47 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3698929094' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:27:48.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:47 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3698929094' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:27:49.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:48 smithi134 ceph-mon[98216]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 137 KiB/s, 0 objects/s recovering 2023-07-18T03:27:49.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:48 smithi040 ceph-mon[96439]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 137 KiB/s, 0 objects/s recovering 2023-07-18T03:27:49.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:48 smithi064 ceph-mon[98059]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 137 KiB/s, 0 objects/s recovering 2023-07-18T03:27:50.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:49 smithi064 ceph-mon[98059]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:27:50.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:49 smithi134 ceph-mon[98216]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:27:50.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:49 smithi040 ceph-mon[96439]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:27:52.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:52 smithi040 ceph-mon[96439]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-07-18T03:27:52.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:52 smithi064 ceph-mon[98059]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-07-18T03:27:53.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:52 smithi134 ceph-mon[98216]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-07-18T03:27:53.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:53 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-18T03:27:53.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:53 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:53.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:53 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-18T03:27:53.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:53 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:54.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:53 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-18T03:27:54.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:53 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:27:54.668 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:54 smithi040 ceph-mon[96439]: Deploying daemon osd.9 on smithi040 2023-07-18T03:27:54.668 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:54 smithi040 ceph-mon[96439]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-07-18T03:27:54.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:54 smithi064 ceph-mon[98059]: Deploying daemon osd.9 on smithi040 2023-07-18T03:27:54.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:54 smithi064 ceph-mon[98059]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-07-18T03:27:55.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:54 smithi134 ceph-mon[98216]: Deploying daemon osd.9 on smithi040 2023-07-18T03:27:55.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:54 smithi134 ceph-mon[98216]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-07-18T03:27:56.074 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:55 smithi040 ceph-mon[96439]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-18T03:27:56.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:55 smithi064 ceph-mon[98059]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-18T03:27:56.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:55 smithi134 ceph-mon[98216]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-18T03:27:58.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:58.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:58.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:58 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:58.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:58 smithi040 ceph-mon[96439]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-07-18T03:27:58.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:58.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:58.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:58 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:58.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:58 smithi134 ceph-mon[98216]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-07-18T03:27:58.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:58.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:27:58.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:58 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:27:58.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:58 smithi064 ceph-mon[98059]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-07-18T03:27:59.951 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:59 smithi040 ceph-mon[96439]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:27:59.951 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:27:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:00.043 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 9 on host 'smithi040' 2023-07-18T03:28:00.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:00.046+0000 7fdc097fa700 1 -- 172.21.15.40:0/239276211 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdc0c069030 con 0x7fdbf40a1150 2023-07-18T03:28:00.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:00.049+0000 7fdc122e3700 1 -- 172.21.15.40:0/239276211 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdbf40a1150 msgr2=0x7fdbf40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:00.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:00.049+0000 7fdc122e3700 1 --2- 172.21.15.40:0/239276211 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdbf40a1150 0x7fdbf40a3600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000b2e0 tx=0x7fdc00005fb0 comp rx=0 tx=0).stop 2023-07-18T03:28:00.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:00.049+0000 7fdc122e3700 1 -- 172.21.15.40:0/239276211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc0c0fea80 msgr2=0x7fdc0c197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:00.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:00.049+0000 7fdc122e3700 1 --2- 172.21.15.40:0/239276211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc0c0fea80 0x7fdc0c197ac0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00e9a0 tx=0x7fdbfc00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:28:00.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:00.049+0000 7fdc122e3700 1 -- 172.21.15.40:0/239276211 shutdown_connections 2023-07-18T03:28:00.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:00.049+0000 7fdc122e3700 1 --2- 172.21.15.40:0/239276211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc0c100610 0x7fdc0c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:00.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:00.049+0000 7fdc122e3700 1 --2- 172.21.15.40:0/239276211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc0c0fea80 0x7fdc0c197ac0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:00.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:00.049+0000 7fdc122e3700 1 --2- 172.21.15.40:0/239276211 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdbf40a1150 0x7fdbf40a3600 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:00.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:00.049+0000 7fdc122e3700 1 --2- 172.21.15.40:0/239276211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc0c0ffc80 0x7fdc0c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:00.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:00.049+0000 7fdc122e3700 1 -- 172.21.15.40:0/239276211 >> 172.21.15.40:0/239276211 conn(0x7fdc0c0fa0b0 msgr2=0x7fdc0c068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:00.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:00.049+0000 7fdc122e3700 1 -- 172.21.15.40:0/239276211 shutdown_connections 2023-07-18T03:28:00.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:00.049+0000 7fdc122e3700 1 -- 172.21.15.40:0/239276211 wait complete. 2023-07-18T03:28:00.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:59 smithi064 ceph-mon[98059]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:28:00.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:27:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:00.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:59 smithi134 ceph-mon[98216]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:28:00.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:27:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:00.359 DEBUG:teuthology.orchestra.run.smithi040:osd.9> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.9.service 2023-07-18T03:28:00.361 INFO:tasks.cephadm:Deploying osd.10 on smithi064 with /dev/vg_nvme/lv_1... 2023-07-18T03:28:00.361 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-18T03:28:00.392 INFO:journalctl@ceph.osd.9.smithi040.stdout:-- Logs begin at Tue 2023-07-18 03:07:07 UTC. -- 2023-07-18T03:28:00.936 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:28:01.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:00 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:01.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:00 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:01.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:00 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:01.158 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:00 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:01.159 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:00 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:01.159 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:00 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:01.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:00 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:01.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:00 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:01.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:00 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:01.829 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:28:01 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:28:01.415+0000 7f7f3ebd8640 -1 osd.9 0 log_to_monitors true 2023-07-18T03:28:02.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:01 smithi064 ceph-mon[98059]: from='osd.9 [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-18T03:28:02.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:01 smithi064 ceph-mon[98059]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:28:02.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:01 smithi134 ceph-mon[98216]: from='osd.9 [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-18T03:28:02.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:01 smithi134 ceph-mon[98216]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:28:02.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:01 smithi040 ceph-mon[96439]: from='osd.9 [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-18T03:28:02.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:01 smithi040 ceph-mon[96439]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:28:02.435 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-18T03:28:02.435 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-18T03:28:02.435 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2023-07-18T03:28:02.435 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2023-07-18T03:28:02.435 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110752 s, 947 MB/s 2023-07-18T03:28:02.436 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2023-07-18T03:28:02.785 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi064:vg_nvme/lv_1 2023-07-18T03:28:03.079 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:28:02.866+0000 7f7f33a4f700 -1 osd.9 0 waiting for initial osdmap 2023-07-18T03:28:03.079 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:28:02.879+0000 7f7f2d840700 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:28:03.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-mon[96439]: from='osd.9 [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-18T03:28:03.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-mon[96439]: osdmap e62: 10 total, 9 up, 10 in 2023-07-18T03:28:03.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:28:03.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-mon[96439]: from='osd.9 [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2023-07-18T03:28:03.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-mon[96439]: Detected new or changed devices on smithi040 2023-07-18T03:28:03.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:03.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:03.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:03.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:03.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:03.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:03.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-mon[96439]: Adjusting osd_memory_target on smithi040 to 4293M 2023-07-18T03:28:03.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:03.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:03.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:03.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:03.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:02 smithi064 ceph-mon[98059]: from='osd.9 [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-18T03:28:03.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:02 smithi064 ceph-mon[98059]: osdmap e62: 10 total, 9 up, 10 in 2023-07-18T03:28:03.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:28:03.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:02 smithi064 ceph-mon[98059]: from='osd.9 [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2023-07-18T03:28:03.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:02 smithi064 ceph-mon[98059]: Detected new or changed devices on smithi040 2023-07-18T03:28:03.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:03.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:02 smithi064 ceph-mon[98059]: Adjusting osd_memory_target on smithi040 to 4293M 2023-07-18T03:28:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:03.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:03.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:03.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:03.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:02 smithi134 ceph-mon[98216]: from='osd.9 [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-18T03:28:03.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:02 smithi134 ceph-mon[98216]: osdmap e62: 10 total, 9 up, 10 in 2023-07-18T03:28:03.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:28:03.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:02 smithi134 ceph-mon[98216]: from='osd.9 [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2023-07-18T03:28:03.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:02 smithi134 ceph-mon[98216]: Detected new or changed devices on smithi040 2023-07-18T03:28:03.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:03.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:03.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:03.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:03.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:03.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:03.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:02 smithi134 ceph-mon[98216]: Adjusting osd_memory_target on smithi040 to 4293M 2023-07-18T03:28:03.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:03.319 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:03.319 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:03.319 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:03.349 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:28:04.132 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:03 smithi064 ceph-mon[98059]: from='osd.9 [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2023-07-18T03:28:04.132 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:03 smithi064 ceph-mon[98059]: osdmap e63: 10 total, 9 up, 10 in 2023-07-18T03:28:04.133 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:03 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:28:04.133 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:03 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:28:04.133 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:03 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:04.133 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:03 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:04.133 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:03 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:04.133 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:03 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:04.133 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:03 smithi064 ceph-mon[98059]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:28:04.134 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:03 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:28:04.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:03 smithi134 ceph-mon[98216]: from='osd.9 [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2023-07-18T03:28:04.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:03 smithi134 ceph-mon[98216]: osdmap e63: 10 total, 9 up, 10 in 2023-07-18T03:28:04.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:03 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:28:04.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:03 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:28:04.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:03 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:04.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:03 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:04.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:03 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:04.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:03 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:04.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:03 smithi134 ceph-mon[98216]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:28:04.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:03 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:28:04.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:03 smithi040 ceph-mon[96439]: from='osd.9 [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2023-07-18T03:28:04.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:03 smithi040 ceph-mon[96439]: osdmap e63: 10 total, 9 up, 10 in 2023-07-18T03:28:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:28:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:28:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:03 smithi040 ceph-mon[96439]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-18T03:28:04.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:28:04.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.543+0000 7f0468733700 1 -- 172.21.15.64:0/3325056413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04600ffc80 msgr2=0x7f04601000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:04.540 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.543+0000 7f0468733700 1 --2- 172.21.15.64:0/3325056413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04600ffc80 0x7f04601000d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0454009990 tx=0x7f0454009ca0 comp rx=0 tx=0).stop 2023-07-18T03:28:04.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.543+0000 7f0468733700 1 -- 172.21.15.64:0/3325056413 shutdown_connections 2023-07-18T03:28:04.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.543+0000 7f0468733700 1 --2- 172.21.15.64:0/3325056413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0460100610 0x7f0460106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:04.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.543+0000 7f0468733700 1 --2- 172.21.15.64:0/3325056413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04600fea80 0x7f04600fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:04.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.543+0000 7f0468733700 1 --2- 172.21.15.64:0/3325056413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04600ffc80 0x7f04601000d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:04.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.543+0000 7f0468733700 1 -- 172.21.15.64:0/3325056413 >> 172.21.15.64:0/3325056413 conn(0x7f04600fa0b0 msgr2=0x7f04600fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:04.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.543+0000 7f0468733700 1 -- 172.21.15.64:0/3325056413 shutdown_connections 2023-07-18T03:28:04.541 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.543+0000 7f0468733700 1 -- 172.21.15.64:0/3325056413 wait complete. 2023-07-18T03:28:04.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f0468733700 1 Processor -- start 2023-07-18T03:28:04.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f0468733700 1 -- start start 2023-07-18T03:28:04.543 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f0468733700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04600fea80 0x7f0460102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:04.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f0468733700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04600ffc80 0x7f0460101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:04.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f0468733700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0460100610 0x7f04601015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:04.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f0468733700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0460101d20 con 0x7f04600ffc80 2023-07-18T03:28:04.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f0468733700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0460101e90 con 0x7f04600fea80 2023-07-18T03:28:04.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f0468733700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046019f9c0 con 0x7f0460100610 2023-07-18T03:28:04.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f04664cf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04600fea80 0x7f0460102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:04.544 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f04664cf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04600fea80 0x7f0460102a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37702/0 (socket says 172.21.15.64:37702) 2023-07-18T03:28:04.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f04664cf700 1 -- 172.21.15.64:0/4139160139 learned_addr learned my addr 172.21.15.64:0/4139160139 (peer_addr_for_me v2:172.21.15.64:0/0) 2023-07-18T03:28:04.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f0465cce700 1 --2- 172.21.15.64:0/4139160139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04600ffc80 0x7f0460101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:04.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f0466cd0700 1 --2- 172.21.15.64:0/4139160139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0460100610 0x7f04601015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:04.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f04664cf700 1 -- 172.21.15.64:0/4139160139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0460100610 msgr2=0x7f04601015a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:04.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f04664cf700 1 --2- 172.21.15.64:0/4139160139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0460100610 0x7f04601015a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:04.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f04664cf700 1 -- 172.21.15.64:0/4139160139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04600ffc80 msgr2=0x7f0460101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:04.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f04664cf700 1 --2- 172.21.15.64:0/4139160139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04600ffc80 0x7f0460101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:04.545 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.544+0000 7f04664cf700 1 -- 172.21.15.64:0/4139160139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04540095f0 con 0x7f04600fea80 2023-07-18T03:28:04.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.545+0000 7f0466cd0700 1 --2- 172.21.15.64:0/4139160139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0460100610 0x7f04601015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:28:04.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.545+0000 7f04664cf700 1 --2- 172.21.15.64:0/4139160139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04600fea80 0x7f0460102a10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f045c00b500 tx=0x7f045c00b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:04.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.545+0000 7f04537fe700 1 -- 172.21.15.64:0/4139160139 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c011830 con 0x7f04600fea80 2023-07-18T03:28:04.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.545+0000 7f04537fe700 1 -- 172.21.15.64:0/4139160139 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f045c011e50 con 0x7f04600fea80 2023-07-18T03:28:04.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.545+0000 7f0468733700 1 -- 172.21.15.64:0/4139160139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f046019fca0 con 0x7f04600fea80 2023-07-18T03:28:04.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.545+0000 7f04537fe700 1 -- 172.21.15.64:0/4139160139 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045c00f2e0 con 0x7f04600fea80 2023-07-18T03:28:04.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.545+0000 7f0468733700 1 -- 172.21.15.64:0/4139160139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04601a01a0 con 0x7f04600fea80 2023-07-18T03:28:04.546 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.546+0000 7f0468733700 1 -- 172.21.15.64:0/4139160139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f046010a2e0 con 0x7f04600fea80 2023-07-18T03:28:04.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.547+0000 7f04537fe700 1 -- 172.21.15.64:0/4139160139 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f045c010580 con 0x7f04600fea80 2023-07-18T03:28:04.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.548+0000 7f04537fe700 1 --2- 172.21.15.64:0/4139160139 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f044c0a11d0 0x7f044c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:04.547 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.548+0000 7f04537fe700 1 -- 172.21.15.64:0/4139160139 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f045c0cbaf0 con 0x7f04600fea80 2023-07-18T03:28:04.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.548+0000 7f0465cce700 1 --2- 172.21.15.64:0/4139160139 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f044c0a11d0 0x7f044c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:04.548 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.549+0000 7f0465cce700 1 --2- 172.21.15.64:0/4139160139 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f044c0a11d0 0x7f044c0a3680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f045400b5c0 tx=0x7f0454005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:04.549 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.552+0000 7f04537fe700 1 -- 172.21.15.64:0/4139160139 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f045c095a40 con 0x7f04600fea80 2023-07-18T03:28:04.754 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:04.758+0000 7f0468733700 1 -- 172.21.15.64:0/4139160139 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0460068760 con 0x7f044c0a11d0 2023-07-18T03:28:05.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:04 smithi134 ceph-mon[98216]: purged_snaps scrub starts 2023-07-18T03:28:05.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:04 smithi134 ceph-mon[98216]: purged_snaps scrub ok 2023-07-18T03:28:05.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:04 smithi134 ceph-mon[98216]: osd.9 [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323] boot 2023-07-18T03:28:05.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:04 smithi134 ceph-mon[98216]: osdmap e64: 10 total, 10 up, 10 in 2023-07-18T03:28:05.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:04 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:28:05.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:04 smithi134 ceph-mon[98216]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:28:05.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:04 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:28:05.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:04 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:28:05.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:04 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:05.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:04 smithi040 ceph-mon[96439]: purged_snaps scrub starts 2023-07-18T03:28:05.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:04 smithi040 ceph-mon[96439]: purged_snaps scrub ok 2023-07-18T03:28:05.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:04 smithi040 ceph-mon[96439]: osd.9 [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323] boot 2023-07-18T03:28:05.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:04 smithi040 ceph-mon[96439]: osdmap e64: 10 total, 10 up, 10 in 2023-07-18T03:28:05.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:04 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:28:05.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:04 smithi040 ceph-mon[96439]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:28:05.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:04 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:28:05.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:04 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:28:05.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:04 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:05.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:04 smithi064 ceph-mon[98059]: purged_snaps scrub starts 2023-07-18T03:28:05.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:04 smithi064 ceph-mon[98059]: purged_snaps scrub ok 2023-07-18T03:28:05.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:04 smithi064 ceph-mon[98059]: osd.9 [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323] boot 2023-07-18T03:28:05.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:04 smithi064 ceph-mon[98059]: osdmap e64: 10 total, 10 up, 10 in 2023-07-18T03:28:05.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:04 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-18T03:28:05.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:04 smithi064 ceph-mon[98059]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:28:05.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:04 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:28:05.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:04 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:28:05.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:04 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:06.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:05 smithi134 ceph-mon[98216]: osdmap e65: 10 total, 10 up, 10 in 2023-07-18T03:28:06.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:05 smithi134 ceph-mon[98216]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:06.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:05 smithi040 ceph-mon[96439]: osdmap e65: 10 total, 10 up, 10 in 2023-07-18T03:28:06.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:05 smithi040 ceph-mon[96439]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:06.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:05 smithi064 ceph-mon[98059]: osdmap e65: 10 total, 10 up, 10 in 2023-07-18T03:28:06.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:05 smithi064 ceph-mon[98059]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:07.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:07 smithi040 ceph-mon[96439]: from='client.? 172.21.15.64:0/2121058941' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50f2740d-5e2a-43b5-9dd0-11d322238231"}]: dispatch 2023-07-18T03:28:07.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:07 smithi040 ceph-mon[96439]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50f2740d-5e2a-43b5-9dd0-11d322238231"}]: dispatch 2023-07-18T03:28:07.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:07 smithi040 ceph-mon[96439]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50f2740d-5e2a-43b5-9dd0-11d322238231"}]': finished 2023-07-18T03:28:07.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:07 smithi040 ceph-mon[96439]: osdmap e66: 11 total, 10 up, 11 in 2023-07-18T03:28:07.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:07 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:07.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:07 smithi064 ceph-mon[98059]: from='client.? 172.21.15.64:0/2121058941' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50f2740d-5e2a-43b5-9dd0-11d322238231"}]: dispatch 2023-07-18T03:28:07.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:07 smithi064 ceph-mon[98059]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50f2740d-5e2a-43b5-9dd0-11d322238231"}]: dispatch 2023-07-18T03:28:07.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:07 smithi064 ceph-mon[98059]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50f2740d-5e2a-43b5-9dd0-11d322238231"}]': finished 2023-07-18T03:28:07.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:07 smithi064 ceph-mon[98059]: osdmap e66: 11 total, 10 up, 11 in 2023-07-18T03:28:07.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:07 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:08.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:07 smithi134 ceph-mon[98216]: from='client.? 172.21.15.64:0/2121058941' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50f2740d-5e2a-43b5-9dd0-11d322238231"}]: dispatch 2023-07-18T03:28:08.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:07 smithi134 ceph-mon[98216]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50f2740d-5e2a-43b5-9dd0-11d322238231"}]: dispatch 2023-07-18T03:28:08.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:07 smithi134 ceph-mon[98216]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50f2740d-5e2a-43b5-9dd0-11d322238231"}]': finished 2023-07-18T03:28:08.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:07 smithi134 ceph-mon[98216]: osdmap e66: 11 total, 10 up, 11 in 2023-07-18T03:28:08.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:07 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:08.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:08 smithi040 ceph-mon[96439]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:08.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:08 smithi040 ceph-mon[96439]: from='client.? 172.21.15.64:0/2809110736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:28:08.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:08 smithi064 ceph-mon[98059]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:08.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:08 smithi064 ceph-mon[98059]: from='client.? 172.21.15.64:0/2809110736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:28:09.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:08 smithi134 ceph-mon[98216]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:09.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:08 smithi134 ceph-mon[98216]: from='client.? 172.21.15.64:0/2809110736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:28:10.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:09 smithi064 ceph-mon[98059]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:10.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:09 smithi134 ceph-mon[98216]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:10.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:09 smithi040 ceph-mon[96439]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:12.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:12 smithi064 ceph-mon[98059]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:12.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:12 smithi040 ceph-mon[96439]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:13.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:12 smithi134 ceph-mon[98216]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:15.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:14 smithi064 ceph-mon[98059]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:15.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:14 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-18T03:28:15.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:14 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:15.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:14 smithi064 ceph-mon[98059]: Deploying daemon osd.10 on smithi064 2023-07-18T03:28:15.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:14 smithi134 ceph-mon[98216]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:15.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:14 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-18T03:28:15.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:14 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:15.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:14 smithi134 ceph-mon[98216]: Deploying daemon osd.10 on smithi064 2023-07-18T03:28:15.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:14 smithi040 ceph-mon[96439]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:15.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:14 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-18T03:28:15.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:14 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:15.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:14 smithi040 ceph-mon[96439]: Deploying daemon osd.10 on smithi064 2023-07-18T03:28:17.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:17 smithi040 ceph-mon[96439]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:17.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:17 smithi064 ceph-mon[98059]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:18.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:17 smithi134 ceph-mon[98216]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:19.659 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:19 smithi064 ceph-mon[98059]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:19.660 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:19.660 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:19.660 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:19.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:19 smithi134 ceph-mon[98216]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:19.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:19.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:19.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:19.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:19 smithi040 ceph-mon[96439]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:19.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:19.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:19.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:21.174 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 10 on host 'smithi064' 2023-07-18T03:28:21.175 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:21.177+0000 7f04537fe700 1 -- 172.21.15.64:0/4139160139 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f0460068760 con 0x7f044c0a11d0 2023-07-18T03:28:21.176 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:21.179+0000 7f0468733700 1 -- 172.21.15.64:0/4139160139 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f044c0a11d0 msgr2=0x7f044c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:21.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:21.179+0000 7f0468733700 1 --2- 172.21.15.64:0/4139160139 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f044c0a11d0 0x7f044c0a3680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f045400b5c0 tx=0x7f0454005fd0 comp rx=0 tx=0).stop 2023-07-18T03:28:21.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:21.179+0000 7f0468733700 1 -- 172.21.15.64:0/4139160139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04600fea80 msgr2=0x7f0460102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:21.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:21.179+0000 7f0468733700 1 --2- 172.21.15.64:0/4139160139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04600fea80 0x7f0460102a10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f045c00b500 tx=0x7f045c00b810 comp rx=0 tx=0).stop 2023-07-18T03:28:21.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:21.179+0000 7f0468733700 1 -- 172.21.15.64:0/4139160139 shutdown_connections 2023-07-18T03:28:21.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:21.179+0000 7f0468733700 1 --2- 172.21.15.64:0/4139160139 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f044c0a11d0 0x7f044c0a3680 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:21.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:21.180+0000 7f0468733700 1 --2- 172.21.15.64:0/4139160139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0460100610 0x7f04601015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:21.177 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:21.180+0000 7f0468733700 1 --2- 172.21.15.64:0/4139160139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04600fea80 0x7f0460102a10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:21.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:21.180+0000 7f0468733700 1 --2- 172.21.15.64:0/4139160139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04600ffc80 0x7f0460101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:21.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:21.180+0000 7f0468733700 1 -- 172.21.15.64:0/4139160139 >> 172.21.15.64:0/4139160139 conn(0x7f04600fa0b0 msgr2=0x7f0460068050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:21.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:21.180+0000 7f0468733700 1 -- 172.21.15.64:0/4139160139 shutdown_connections 2023-07-18T03:28:21.178 INFO:teuthology.orchestra.run.smithi064.stderr:2023-07-18T03:28:21.180+0000 7f0468733700 1 -- 172.21.15.64:0/4139160139 wait complete. 2023-07-18T03:28:21.645 DEBUG:teuthology.orchestra.run.smithi064:osd.10> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.10.service 2023-07-18T03:28:21.647 INFO:tasks.cephadm:Deploying osd.11 on smithi134 with /dev/vg_nvme/lv_1... 2023-07-18T03:28:21.647 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-18T03:28:21.680 INFO:journalctl@ceph.osd.10.smithi064.stdout:-- Logs begin at Tue 2023-07-18 03:07:19 UTC. -- 2023-07-18T03:28:21.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:21 smithi134 ceph-mon[98216]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:21.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:21.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:21.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:21.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:21.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:21 smithi040 ceph-mon[96439]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:21.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:21.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:21.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:21.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:21.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:21 smithi064 ceph-mon[98059]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:21.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:21.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:21.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:21.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:22.215 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:28:22.687 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:22 smithi134 ceph-mon[98216]: from='osd.10 [v2:172.21.15.64:6824/2791256757,v1:172.21.15.64:6825/2791256757]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-18T03:28:22.687 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:22 smithi134 ceph-mon[98216]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-18T03:28:22.692 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:22 smithi064 ceph-mon[98059]: from='osd.10 [v2:172.21.15.64:6824/2791256757,v1:172.21.15.64:6825/2791256757]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-18T03:28:22.692 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:22 smithi064 ceph-mon[98059]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-18T03:28:22.693 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:28:22 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:28:22.431+0000 7fa5d458a640 -1 osd.10 0 log_to_monitors true 2023-07-18T03:28:23.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:22 smithi040 ceph-mon[96439]: from='osd.10 [v2:172.21.15.64:6824/2791256757,v1:172.21.15.64:6825/2791256757]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-18T03:28:23.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:22 smithi040 ceph-mon[96439]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-18T03:28:23.559 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-18T03:28:23.559 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-18T03:28:23.559 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2023-07-18T03:28:23.559 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2023-07-18T03:28:23.559 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110208 s, 951 MB/s 2023-07-18T03:28:23.560 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2023-07-18T03:28:23.851 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:28:23 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:28:23.609+0000 7fa5c9401700 -1 osd.10 0 waiting for initial osdmap 2023-07-18T03:28:23.851 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:28:23 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:28:23.622+0000 7fa5c49f5700 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:28:23.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:23 smithi064 ceph-mon[98059]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:23.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:23 smithi064 ceph-mon[98059]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-18T03:28:23.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:23 smithi064 ceph-mon[98059]: osdmap e67: 11 total, 10 up, 11 in 2023-07-18T03:28:23.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:23.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:23 smithi064 ceph-mon[98059]: from='osd.10 [v2:172.21.15.64:6824/2791256757,v1:172.21.15.64:6825/2791256757]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:28:23.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:23 smithi064 ceph-mon[98059]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:28:23.959 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:23 smithi134 ceph-mon[98216]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:23.959 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:23 smithi134 ceph-mon[98216]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-18T03:28:23.959 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:23 smithi134 ceph-mon[98216]: osdmap e67: 11 total, 10 up, 11 in 2023-07-18T03:28:23.959 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:23 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:23.959 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:23 smithi134 ceph-mon[98216]: from='osd.10 [v2:172.21.15.64:6824/2791256757,v1:172.21.15.64:6825/2791256757]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:28:23.960 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:23 smithi134 ceph-mon[98216]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:28:23.961 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi134:vg_nvme/lv_1 2023-07-18T03:28:24.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:23 smithi040 ceph-mon[96439]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:24.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:23 smithi040 ceph-mon[96439]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-18T03:28:24.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:23 smithi040 ceph-mon[96439]: osdmap e67: 11 total, 10 up, 11 in 2023-07-18T03:28:24.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:23 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:24.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:23 smithi040 ceph-mon[96439]: from='osd.10 [v2:172.21.15.64:6824/2791256757,v1:172.21.15.64:6825/2791256757]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:28:24.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:23 smithi040 ceph-mon[96439]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2023-07-18T03:28:24.515 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:28:24.873 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:24 smithi134 ceph-mon[98216]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-07-18T03:28:24.874 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:24 smithi134 ceph-mon[98216]: osdmap e68: 11 total, 10 up, 11 in 2023-07-18T03:28:24.874 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:24.874 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:24.874 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:24.874 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:24.874 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:24.875 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:24.875 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:24.875 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:24.875 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:24.875 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:24.875 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:24.875 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:25.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:24 smithi040 ceph-mon[96439]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-07-18T03:28:25.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:24 smithi040 ceph-mon[96439]: osdmap e68: 11 total, 10 up, 11 in 2023-07-18T03:28:25.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:25.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:25.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:25.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:25.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:25.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:24 smithi064 ceph-mon[98059]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2023-07-18T03:28:25.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:24 smithi064 ceph-mon[98059]: osdmap e68: 11 total, 10 up, 11 in 2023-07-18T03:28:25.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:25.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:25.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:25.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:25.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:25.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:25.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:25.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:25.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:25.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:25.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:25.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:25.600 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.602+0000 7fbaa43cf700 1 -- 172.21.15.134:0/2894915162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba9c0ff6c0 msgr2=0x7fba9c106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:25.600 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.602+0000 7fbaa43cf700 1 --2- 172.21.15.134:0/2894915162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba9c0ff6c0 0x7fba9c106f70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fba9400b0a0 tx=0x7fba9400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:28:25.601 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.602+0000 7fbaa43cf700 1 -- 172.21.15.134:0/2894915162 shutdown_connections 2023-07-18T03:28:25.601 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.602+0000 7fbaa43cf700 1 --2- 172.21.15.134:0/2894915162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba9c0ff6c0 0x7fba9c106f70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:25.601 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.602+0000 7fbaa43cf700 1 --2- 172.21.15.134:0/2894915162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba9c1000b0 0x7fba9c1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:25.601 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.602+0000 7fbaa43cf700 1 --2- 172.21.15.134:0/2894915162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c0fed10 0x7fba9c0ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:25.601 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.602+0000 7fbaa43cf700 1 -- 172.21.15.134:0/2894915162 >> 172.21.15.134:0/2894915162 conn(0x7fba9c0f9890 msgr2=0x7fba9c0fbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:25.601 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.602+0000 7fbaa43cf700 1 -- 172.21.15.134:0/2894915162 shutdown_connections 2023-07-18T03:28:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.602+0000 7fbaa43cf700 1 -- 172.21.15.134:0/2894915162 wait complete. 2023-07-18T03:28:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.602+0000 7fbaa43cf700 1 Processor -- start 2023-07-18T03:28:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.602+0000 7fbaa43cf700 1 -- start start 2023-07-18T03:28:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa43cf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba9c0fed10 0x7fba9c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa43cf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c0ff6c0 0x7fba9c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa43cf700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba9c1000b0 0x7fba9c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa43cf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c19f620 con 0x7fba9c0ff6c0 2023-07-18T03:28:25.602 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa43cf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c19f760 con 0x7fba9c0fed10 2023-07-18T03:28:25.603 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa43cf700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c19f8f0 con 0x7fba9c1000b0 2023-07-18T03:28:25.603 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa296c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba9c1000b0 0x7fba9c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:25.603 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa296c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba9c1000b0 0x7fba9c19f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:48390/0 (socket says 172.21.15.134:48390) 2023-07-18T03:28:25.603 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa296c700 1 -- 172.21.15.134:0/904218523 learned_addr learned my addr 172.21.15.134:0/904218523 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-07-18T03:28:25.603 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa216b700 1 --2- 172.21.15.134:0/904218523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba9c0fed10 0x7fba9c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:25.603 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa196a700 1 --2- 172.21.15.134:0/904218523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c0ff6c0 0x7fba9c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:25.603 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa296c700 1 -- 172.21.15.134:0/904218523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba9c0fed10 msgr2=0x7fba9c197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa296c700 1 --2- 172.21.15.134:0/904218523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba9c0fed10 0x7fba9c197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa296c700 1 -- 172.21.15.134:0/904218523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c0ff6c0 msgr2=0x7fba9c197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa296c700 1 --2- 172.21.15.134:0/904218523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c0ff6c0 0x7fba9c197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa296c700 1 -- 172.21.15.134:0/904218523 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba94009d00 con 0x7fba9c1000b0 2023-07-18T03:28:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa216b700 1 --2- 172.21.15.134:0/904218523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba9c0fed10 0x7fba9c197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:28:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa296c700 1 --2- 172.21.15.134:0/904218523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba9c1000b0 0x7fba9c19f050 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fba94004d00 tx=0x7fba94004de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:25.604 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fba937fe700 1 -- 172.21.15.134:0/904218523 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9400e040 con 0x7fba9c1000b0 2023-07-18T03:28:25.605 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fba937fe700 1 -- 172.21.15.134:0/904218523 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba9401bd20 con 0x7fba9c1000b0 2023-07-18T03:28:25.605 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa43cf700 1 -- 172.21.15.134:0/904218523 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9c19fb70 con 0x7fba9c1000b0 2023-07-18T03:28:25.605 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fba937fe700 1 -- 172.21.15.134:0/904218523 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba94005940 con 0x7fba9c1000b0 2023-07-18T03:28:25.605 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.603+0000 7fbaa43cf700 1 -- 172.21.15.134:0/904218523 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9c19ffe0 con 0x7fba9c1000b0 2023-07-18T03:28:25.606 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.604+0000 7fbaa43cf700 1 -- 172.21.15.134:0/904218523 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba9c10a2d0 con 0x7fba9c1000b0 2023-07-18T03:28:25.607 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.605+0000 7fba937fe700 1 -- 172.21.15.134:0/904218523 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fba94005040 con 0x7fba9c1000b0 2023-07-18T03:28:25.607 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.606+0000 7fba937fe700 1 --2- 172.21.15.134:0/904218523 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fba880a11d0 0x7fba880a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:25.607 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.606+0000 7fba937fe700 1 -- 172.21.15.134:0/904218523 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6727+0+0 (secure 0 0 0) 0x7fba940ce5a0 con 0x7fba9c1000b0 2023-07-18T03:28:25.607 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.606+0000 7fbaa216b700 1 --2- 172.21.15.134:0/904218523 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fba880a11d0 0x7fba880a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:25.607 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.607+0000 7fbaa216b700 1 --2- 172.21.15.134:0/904218523 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fba880a11d0 0x7fba880a3680 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fba9c068400 tx=0x7fba9800a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:25.607 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.609+0000 7fba937fe700 1 -- 172.21.15.134:0/904218523 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fba940983b0 con 0x7fba9c1000b0 2023-07-18T03:28:25.801 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:25.804+0000 7fbaa43cf700 1 -- 172.21.15.134:0/904218523 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fba9c060950 con 0x7fba880a11d0 2023-07-18T03:28:25.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:25 smithi134 ceph-mon[98216]: purged_snaps scrub starts 2023-07-18T03:28:25.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:25 smithi134 ceph-mon[98216]: purged_snaps scrub ok 2023-07-18T03:28:25.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:25 smithi134 ceph-mon[98216]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:25.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:25 smithi134 ceph-mon[98216]: Detected new or changed devices on smithi064 2023-07-18T03:28:25.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:25 smithi134 ceph-mon[98216]: Adjusting osd_memory_target on smithi064 to 4292M 2023-07-18T03:28:25.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:25 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:25.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:25 smithi134 ceph-mon[98216]: osd.10 [v2:172.21.15.64:6824/2791256757,v1:172.21.15.64:6825/2791256757] boot 2023-07-18T03:28:25.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:25 smithi134 ceph-mon[98216]: osdmap e69: 11 total, 11 up, 11 in 2023-07-18T03:28:25.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:25 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:25.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:25 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:25.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:25 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:25.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:25 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:25.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:25 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:26.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:25 smithi040 ceph-mon[96439]: purged_snaps scrub starts 2023-07-18T03:28:26.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:25 smithi040 ceph-mon[96439]: purged_snaps scrub ok 2023-07-18T03:28:26.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:25 smithi040 ceph-mon[96439]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:26.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:25 smithi040 ceph-mon[96439]: Detected new or changed devices on smithi064 2023-07-18T03:28:26.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:25 smithi040 ceph-mon[96439]: Adjusting osd_memory_target on smithi064 to 4292M 2023-07-18T03:28:26.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:25 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:26.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:25 smithi040 ceph-mon[96439]: osd.10 [v2:172.21.15.64:6824/2791256757,v1:172.21.15.64:6825/2791256757] boot 2023-07-18T03:28:26.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:25 smithi040 ceph-mon[96439]: osdmap e69: 11 total, 11 up, 11 in 2023-07-18T03:28:26.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:25 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:26.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:25 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:26.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:25 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:26.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:25 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:26.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:25 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:26.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:25 smithi064 ceph-mon[98059]: purged_snaps scrub starts 2023-07-18T03:28:26.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:25 smithi064 ceph-mon[98059]: purged_snaps scrub ok 2023-07-18T03:28:26.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:25 smithi064 ceph-mon[98059]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-18T03:28:26.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:25 smithi064 ceph-mon[98059]: Detected new or changed devices on smithi064 2023-07-18T03:28:26.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:25 smithi064 ceph-mon[98059]: Adjusting osd_memory_target on smithi064 to 4292M 2023-07-18T03:28:26.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:25 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:26.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:25 smithi064 ceph-mon[98059]: osd.10 [v2:172.21.15.64:6824/2791256757,v1:172.21.15.64:6825/2791256757] boot 2023-07-18T03:28:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:25 smithi064 ceph-mon[98059]: osdmap e69: 11 total, 11 up, 11 in 2023-07-18T03:28:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:25 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-18T03:28:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:25 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:25 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:25 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:26.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:25 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:26.902 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:26 smithi134 ceph-mon[98216]: osdmap e70: 11 total, 11 up, 11 in 2023-07-18T03:28:26.902 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:26 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:28:26.902 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:26 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:28:26.903 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:26 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:27.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:26 smithi040 ceph-mon[96439]: osdmap e70: 11 total, 11 up, 11 in 2023-07-18T03:28:27.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:26 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:28:27.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:26 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:28:27.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:26 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:27.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:26 smithi064 ceph-mon[98059]: osdmap e70: 11 total, 11 up, 11 in 2023-07-18T03:28:27.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:26 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-18T03:28:27.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:26 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-18T03:28:27.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:26 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:28.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:27 smithi134 ceph-mon[98216]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:28.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:27 smithi134 ceph-mon[98216]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:28:28.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:27 smithi134 ceph-mon[98216]: osdmap e71: 11 total, 11 up, 11 in 2023-07-18T03:28:28.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:27 smithi040 ceph-mon[96439]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:28.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:27 smithi040 ceph-mon[96439]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:28:28.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:27 smithi040 ceph-mon[96439]: osdmap e71: 11 total, 11 up, 11 in 2023-07-18T03:28:28.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:27 smithi064 ceph-mon[98059]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:28.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:27 smithi064 ceph-mon[98059]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:28:28.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:27 smithi064 ceph-mon[98059]: osdmap e71: 11 total, 11 up, 11 in 2023-07-18T03:28:29.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:28 smithi134 ceph-mon[98216]: osdmap e72: 11 total, 11 up, 11 in 2023-07-18T03:28:29.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:28 smithi134 ceph-mon[98216]: from='client.? 172.21.15.134:0/2477642103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "735e8994-4fce-4cab-a980-dbfe3259b2ae"}]: dispatch 2023-07-18T03:28:29.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:28 smithi134 ceph-mon[98216]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "735e8994-4fce-4cab-a980-dbfe3259b2ae"}]: dispatch 2023-07-18T03:28:29.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:28 smithi134 ceph-mon[98216]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "735e8994-4fce-4cab-a980-dbfe3259b2ae"}]': finished 2023-07-18T03:28:29.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:28 smithi134 ceph-mon[98216]: osdmap e73: 12 total, 11 up, 12 in 2023-07-18T03:28:29.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:28:29.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:28 smithi040 ceph-mon[96439]: osdmap e72: 11 total, 11 up, 11 in 2023-07-18T03:28:29.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:28 smithi040 ceph-mon[96439]: from='client.? 172.21.15.134:0/2477642103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "735e8994-4fce-4cab-a980-dbfe3259b2ae"}]: dispatch 2023-07-18T03:28:29.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:28 smithi040 ceph-mon[96439]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "735e8994-4fce-4cab-a980-dbfe3259b2ae"}]: dispatch 2023-07-18T03:28:29.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:28 smithi040 ceph-mon[96439]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "735e8994-4fce-4cab-a980-dbfe3259b2ae"}]': finished 2023-07-18T03:28:29.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:28 smithi040 ceph-mon[96439]: osdmap e73: 12 total, 11 up, 12 in 2023-07-18T03:28:29.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:28:29.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:28 smithi064 ceph-mon[98059]: osdmap e72: 11 total, 11 up, 11 in 2023-07-18T03:28:29.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:28 smithi064 ceph-mon[98059]: from='client.? 172.21.15.134:0/2477642103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "735e8994-4fce-4cab-a980-dbfe3259b2ae"}]: dispatch 2023-07-18T03:28:29.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:28 smithi064 ceph-mon[98059]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "735e8994-4fce-4cab-a980-dbfe3259b2ae"}]: dispatch 2023-07-18T03:28:29.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:28 smithi064 ceph-mon[98059]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "735e8994-4fce-4cab-a980-dbfe3259b2ae"}]': finished 2023-07-18T03:28:29.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:28 smithi064 ceph-mon[98059]: osdmap e73: 12 total, 11 up, 12 in 2023-07-18T03:28:29.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:28:30.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:29 smithi134 ceph-mon[98216]: pgmap v210: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-18T03:28:30.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:29 smithi134 ceph-mon[98216]: from='client.? 172.21.15.134:0/1850639477' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:28:30.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:29 smithi040 ceph-mon[96439]: pgmap v210: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-18T03:28:30.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:29 smithi040 ceph-mon[96439]: from='client.? 172.21.15.134:0/1850639477' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:28:30.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:29 smithi064 ceph-mon[98059]: pgmap v210: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-18T03:28:30.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:29 smithi064 ceph-mon[98059]: from='client.? 172.21.15.134:0/1850639477' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-18T03:28:32.049 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:31 smithi134 ceph-mon[98216]: pgmap v212: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 138 KiB/s, 0 objects/s recovering 2023-07-18T03:28:32.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:31 smithi040 ceph-mon[96439]: pgmap v212: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 138 KiB/s, 0 objects/s recovering 2023-07-18T03:28:32.079 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:28:31 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:28:31.768+0000 7fe0e1b80700 -1 calc_pg_upmaps abort due to max <= 0 2023-07-18T03:28:32.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:31 smithi064 ceph-mon[98059]: pgmap v212: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 138 KiB/s, 0 objects/s recovering 2023-07-18T03:28:34.041 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:33 smithi134 ceph-mon[98216]: pgmap v213: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:28:34.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:33 smithi040 ceph-mon[96439]: pgmap v213: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:28:34.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:33 smithi064 ceph-mon[98059]: pgmap v213: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-18T03:28:35.684 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:35 smithi134 ceph-mon[98216]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-07-18T03:28:35.684 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-18T03:28:35.684 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:36.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:35 smithi040 ceph-mon[96439]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-07-18T03:28:36.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-18T03:28:36.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:36.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:35 smithi064 ceph-mon[98059]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-07-18T03:28:36.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-18T03:28:36.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:36.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:36 smithi134 ceph-mon[98216]: Deploying daemon osd.11 on smithi134 2023-07-18T03:28:37.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:36 smithi040 ceph-mon[96439]: Deploying daemon osd.11 on smithi134 2023-07-18T03:28:37.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:36 smithi064 ceph-mon[98059]: Deploying daemon osd.11 on smithi134 2023-07-18T03:28:38.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:37 smithi134 ceph-mon[98216]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:38.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:37 smithi040 ceph-mon[96439]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:38.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:37 smithi064 ceph-mon[98059]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:39.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:39 smithi134 ceph-mon[98216]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:39.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:39.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:39.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:40.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:39 smithi040 ceph-mon[96439]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:40.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:40.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:40.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:40.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:39 smithi064 ceph-mon[98059]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:40.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:40.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:40.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:41.617 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:41.620+0000 7fba937fe700 1 -- 172.21.15.134:0/904218523 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fba9c060950 con 0x7fba880a11d0 2023-07-18T03:28:41.617 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 11 on host 'smithi134' 2023-07-18T03:28:41.619 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:41.623+0000 7fbaa43cf700 1 -- 172.21.15.134:0/904218523 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fba880a11d0 msgr2=0x7fba880a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:41.619 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:41.623+0000 7fbaa43cf700 1 --2- 172.21.15.134:0/904218523 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fba880a11d0 0x7fba880a3680 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fba9c068400 tx=0x7fba9800a680 comp rx=0 tx=0).stop 2023-07-18T03:28:41.619 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:41.623+0000 7fbaa43cf700 1 -- 172.21.15.134:0/904218523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba9c1000b0 msgr2=0x7fba9c19f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:41.619 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:41.623+0000 7fbaa43cf700 1 --2- 172.21.15.134:0/904218523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba9c1000b0 0x7fba9c19f050 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fba94004d00 tx=0x7fba94004de0 comp rx=0 tx=0).stop 2023-07-18T03:28:41.619 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:41.623+0000 7fbaa43cf700 1 -- 172.21.15.134:0/904218523 shutdown_connections 2023-07-18T03:28:41.620 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:41.623+0000 7fbaa43cf700 1 --2- 172.21.15.134:0/904218523 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fba880a11d0 0x7fba880a3680 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:41.620 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:41.623+0000 7fbaa43cf700 1 --2- 172.21.15.134:0/904218523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba9c1000b0 0x7fba9c19f050 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:41.620 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:41.623+0000 7fbaa43cf700 1 --2- 172.21.15.134:0/904218523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba9c0fed10 0x7fba9c197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:41.620 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:41.623+0000 7fbaa43cf700 1 --2- 172.21.15.134:0/904218523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba9c0ff6c0 0x7fba9c197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:41.620 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:41.623+0000 7fbaa43cf700 1 -- 172.21.15.134:0/904218523 >> 172.21.15.134:0/904218523 conn(0x7fba9c0f9890 msgr2=0x7fba9c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:41.620 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:41.623+0000 7fbaa43cf700 1 -- 172.21.15.134:0/904218523 shutdown_connections 2023-07-18T03:28:41.620 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:41.623+0000 7fbaa43cf700 1 -- 172.21.15.134:0/904218523 wait complete. 2023-07-18T03:28:41.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:41 smithi134 ceph-mon[98216]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:41.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:41 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:41.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:41 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:41.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:41 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:41.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:41 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:41.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:41 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:41.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:41 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:41.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:41 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:42.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:41 smithi040 ceph-mon[96439]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:42.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:41 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:42.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:41 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:42.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:41 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:42.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:41 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:42.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:41 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:42.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:41 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:42.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:41 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:42.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:41 smithi064 ceph-mon[98059]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:42.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:41 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:42.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:41 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:42.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:41 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:42.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:41 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:42.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:41 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:42.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:41 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:42.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:41 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:42.113 DEBUG:teuthology.orchestra.run.smithi134:osd.11> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.11.service 2023-07-18T03:28:42.115 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-07-18T03:28:42.115 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd stat -f json 2023-07-18T03:28:42.144 INFO:journalctl@ceph.osd.11.smithi134.stdout:-- Logs begin at Tue 2023-07-18 03:07:14 UTC. -- 2023-07-18T03:28:42.696 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:28:42.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:42 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:42.928 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:42 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:43.066 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:28:43 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:28:43.027+0000 7fca48a66640 -1 osd.11 0 log_to_monitors true 2023-07-18T03:28:43.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:42 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:43.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.798+0000 7f1f94acd700 1 -- 172.21.15.40:0/1885863179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f901058f0 msgr2=0x7f1f90107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:43.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.798+0000 7f1f94acd700 1 --2- 172.21.15.40:0/1885863179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f901058f0 0x7f1f90107cd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800b0a0 tx=0x7f1f8800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:28:43.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.798+0000 7f1f94acd700 1 -- 172.21.15.40:0/1885863179 shutdown_connections 2023-07-18T03:28:43.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.798+0000 7f1f94acd700 1 --2- 172.21.15.40:0/1885863179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f901058f0 0x7f1f90107cd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:43.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.798+0000 7f1f94acd700 1 --2- 172.21.15.40:0/1885863179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f901006b0 0x7f1f90102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:43.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.798+0000 7f1f94acd700 1 --2- 172.21.15.40:0/1885863179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90102fd0 0x7f1f901053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:43.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.798+0000 7f1f94acd700 1 -- 172.21.15.40:0/1885863179 >> 172.21.15.40:0/1885863179 conn(0x7f1f900fa0b0 msgr2=0x7f1f900fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:43.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.798+0000 7f1f94acd700 1 -- 172.21.15.40:0/1885863179 shutdown_connections 2023-07-18T03:28:43.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.798+0000 7f1f94acd700 1 -- 172.21.15.40:0/1885863179 wait complete. 2023-07-18T03:28:43.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.799+0000 7f1f94acd700 1 Processor -- start 2023-07-18T03:28:43.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.799+0000 7f1f94acd700 1 -- start start 2023-07-18T03:28:43.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f94acd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f901006b0 0x7f1f90195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:43.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f94acd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90102fd0 0x7f1f90195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:43.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f94acd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f901058f0 0x7f1f9019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:43.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f94acd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9019d510 con 0x7f1f901058f0 2023-07-18T03:28:43.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f94acd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9019d650 con 0x7f1f90102fd0 2023-07-18T03:28:43.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f94acd700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9019d7e0 con 0x7f1f901006b0 2023-07-18T03:28:43.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f8ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f901058f0 0x7f1f9019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:43.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f8e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f901006b0 0x7f1f90195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:43.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f8e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f901006b0 0x7f1f90195980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:44724/0 (socket says 172.21.15.40:44724) 2023-07-18T03:28:43.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f8ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f901058f0 0x7f1f9019cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55700/0 (socket says 172.21.15.40:55700) 2023-07-18T03:28:43.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f8e59c700 1 -- 172.21.15.40:0/46498645 learned_addr learned my addr 172.21.15.40:0/46498645 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:28:43.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f8dd9b700 1 --2- 172.21.15.40:0/46498645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90102fd0 0x7f1f90195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:43.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f8e59c700 1 -- 172.21.15.40:0/46498645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90102fd0 msgr2=0x7f1f90195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:43.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f8e59c700 1 --2- 172.21.15.40:0/46498645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90102fd0 0x7f1f90195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:43.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f8e59c700 1 -- 172.21.15.40:0/46498645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f901058f0 msgr2=0x7f1f9019cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:43.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f8e59c700 1 --2- 172.21.15.40:0/46498645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f901058f0 0x7f1f9019cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:43.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f8e59c700 1 -- 172.21.15.40:0/46498645 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f88009d00 con 0x7f1f901006b0 2023-07-18T03:28:43.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f8ed9d700 1 --2- 172.21.15.40:0/46498645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f901058f0 0x7f1f9019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:28:43.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.800+0000 7f1f8dd9b700 1 --2- 172.21.15.40:0/46498645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90102fd0 0x7f1f90195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:28:43.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.801+0000 7f1f8e59c700 1 --2- 172.21.15.40:0/46498645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f901006b0 0x7f1f90195980 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000e9d0 tx=0x7f1f8000ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:43.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.801+0000 7f1f7f7fe700 1 -- 172.21.15.40:0/46498645 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80009790 con 0x7f1f901006b0 2023-07-18T03:28:43.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.801+0000 7f1f7f7fe700 1 -- 172.21.15.40:0/46498645 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f8000cbd0 con 0x7f1f901006b0 2023-07-18T03:28:43.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.801+0000 7f1f94acd700 1 -- 172.21.15.40:0/46498645 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9019dac0 con 0x7f1f901006b0 2023-07-18T03:28:43.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.801+0000 7f1f7f7fe700 1 -- 172.21.15.40:0/46498645 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80018680 con 0x7f1f901006b0 2023-07-18T03:28:43.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.801+0000 7f1f94acd700 1 -- 172.21.15.40:0/46498645 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9019dfc0 con 0x7f1f901006b0 2023-07-18T03:28:43.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.802+0000 7f1f94acd700 1 -- 172.21.15.40:0/46498645 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f900fbc20 con 0x7f1f901006b0 2023-07-18T03:28:43.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.803+0000 7f1f7f7fe700 1 -- 172.21.15.40:0/46498645 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1f8000cd40 con 0x7f1f901006b0 2023-07-18T03:28:43.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.803+0000 7f1f7f7fe700 1 --2- 172.21.15.40:0/46498645 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1f780a1150 0x7f1f780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:43.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.804+0000 7f1f8dd9b700 1 --2- 172.21.15.40:0/46498645 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1f780a1150 0x7f1f780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:43.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.804+0000 7f1f7f7fe700 1 -- 172.21.15.40:0/46498645 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f1f80014070 con 0x7f1f901006b0 2023-07-18T03:28:43.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.804+0000 7f1f8dd9b700 1 --2- 172.21.15.40:0/46498645 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1f780a1150 0x7f1f780a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1f84005950 tx=0x7f1f840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:43.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:43.807+0000 7f1f7f7fe700 1 -- 172.21.15.40:0/46498645 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1f80096600 con 0x7f1f901006b0 2023-07-18T03:28:43.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:43 smithi134 ceph-mon[98216]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:43.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:43 smithi134 ceph-mon[98216]: from='osd.11 [v2:172.21.15.134:6824/1938250059,v1:172.21.15.134:6825/1938250059]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-18T03:28:43.925 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:43 smithi134 ceph-mon[98216]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-18T03:28:44.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:44.006+0000 7f1f94acd700 1 -- 172.21.15.40:0/46498645 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f1f9002c850 con 0x7f1f901006b0 2023-07-18T03:28:44.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:44.007+0000 7f1f7f7fe700 1 -- 172.21.15.40:0/46498645 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) v1 ==== 74+0+133 (secure 0 0 0) 0x7f1f80099c20 con 0x7f1f901006b0 2023-07-18T03:28:44.004 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:28:44.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:44.009+0000 7f1f94acd700 1 -- 172.21.15.40:0/46498645 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1f780a1150 msgr2=0x7f1f780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:44.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:44.009+0000 7f1f94acd700 1 --2- 172.21.15.40:0/46498645 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1f780a1150 0x7f1f780a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1f84005950 tx=0x7f1f840058e0 comp rx=0 tx=0).stop 2023-07-18T03:28:44.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:44.009+0000 7f1f94acd700 1 -- 172.21.15.40:0/46498645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f901006b0 msgr2=0x7f1f90195980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:44.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:44.009+0000 7f1f94acd700 1 --2- 172.21.15.40:0/46498645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f901006b0 0x7f1f90195980 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000e9d0 tx=0x7f1f8000ece0 comp rx=0 tx=0).stop 2023-07-18T03:28:44.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:44.009+0000 7f1f94acd700 1 -- 172.21.15.40:0/46498645 shutdown_connections 2023-07-18T03:28:44.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:44.009+0000 7f1f94acd700 1 --2- 172.21.15.40:0/46498645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f901058f0 0x7f1f9019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:44.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:44.009+0000 7f1f94acd700 1 --2- 172.21.15.40:0/46498645 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1f780a1150 0x7f1f780a3600 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:44.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:44.009+0000 7f1f94acd700 1 --2- 172.21.15.40:0/46498645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f901006b0 0x7f1f90195980 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:44.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:44.009+0000 7f1f94acd700 1 --2- 172.21.15.40:0/46498645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90102fd0 0x7f1f90195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:44.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:44.009+0000 7f1f94acd700 1 -- 172.21.15.40:0/46498645 >> 172.21.15.40:0/46498645 conn(0x7f1f900fa0b0 msgr2=0x7f1f90101310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:44.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:44.009+0000 7f1f94acd700 1 -- 172.21.15.40:0/46498645 shutdown_connections 2023-07-18T03:28:44.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:44.009+0000 7f1f94acd700 1 -- 172.21.15.40:0/46498645 wait complete. 2023-07-18T03:28:44.031 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:43 smithi040 ceph-mon[96439]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:44.031 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:43 smithi040 ceph-mon[96439]: from='osd.11 [v2:172.21.15.134:6824/1938250059,v1:172.21.15.134:6825/1938250059]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-18T03:28:44.032 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:43 smithi040 ceph-mon[96439]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-18T03:28:44.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:43 smithi064 ceph-mon[98059]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:44.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:43 smithi064 ceph-mon[98059]: from='osd.11 [v2:172.21.15.134:6824/1938250059,v1:172.21.15.134:6825/1938250059]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-18T03:28:44.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:43 smithi064 ceph-mon[98059]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-18T03:28:44.351 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1689650904,"num_in_osds":12,"osd_in_since":1689650908,"num_remapped_pgs":0} 2023-07-18T03:28:45.066 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:28:44.687+0000 7fca3f0e0700 -1 osd.11 0 waiting for initial osdmap 2023-07-18T03:28:45.067 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:28:44.702+0000 7fca37ecf700 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-18T03:28:45.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-mon[98216]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-18T03:28:45.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-mon[98216]: osdmap e74: 12 total, 11 up, 12 in 2023-07-18T03:28:45.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:28:45.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-mon[98216]: from='osd.11 [v2:172.21.15.134:6824/1938250059,v1:172.21.15.134:6825/1938250059]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:28:45.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-mon[98216]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:28:45.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/46498645' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-18T03:28:45.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:45.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:45.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:45.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:45.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:45.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:45.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:45.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:45.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:45.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:44 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:45.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:44 smithi040 ceph-mon[96439]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-18T03:28:45.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:44 smithi040 ceph-mon[96439]: osdmap e74: 12 total, 11 up, 12 in 2023-07-18T03:28:45.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:28:45.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:44 smithi040 ceph-mon[96439]: from='osd.11 [v2:172.21.15.134:6824/1938250059,v1:172.21.15.134:6825/1938250059]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:28:45.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:44 smithi040 ceph-mon[96439]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:28:45.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:44 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/46498645' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-18T03:28:45.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:45.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:45.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:45.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:45.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:45.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:45.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:45.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:45.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:45.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:44 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:45.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:44 smithi064 ceph-mon[98059]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-18T03:28:45.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:44 smithi064 ceph-mon[98059]: osdmap e74: 12 total, 11 up, 12 in 2023-07-18T03:28:45.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:28:45.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:44 smithi064 ceph-mon[98059]: from='osd.11 [v2:172.21.15.134:6824/1938250059,v1:172.21.15.134:6825/1938250059]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:28:45.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:44 smithi064 ceph-mon[98059]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2023-07-18T03:28:45.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:44 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/46498645' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-18T03:28:45.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:45.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:45.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:45.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:45.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:45.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:28:45.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:45.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:45.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:45.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:44 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:45.353 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd stat -f json 2023-07-18T03:28:45.927 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:28:46.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:45 smithi134 ceph-mon[98216]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:46.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:45 smithi134 ceph-mon[98216]: Detected new or changed devices on smithi134 2023-07-18T03:28:46.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:45 smithi134 ceph-mon[98216]: Adjusting osd_memory_target on smithi134 to 4292M 2023-07-18T03:28:46.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:45 smithi134 ceph-mon[98216]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-18T03:28:46.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:45 smithi134 ceph-mon[98216]: osdmap e75: 12 total, 11 up, 12 in 2023-07-18T03:28:46.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:45 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:28:46.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:45 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:28:46.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:45 smithi040 ceph-mon[96439]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:46.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:45 smithi040 ceph-mon[96439]: Detected new or changed devices on smithi134 2023-07-18T03:28:46.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:45 smithi040 ceph-mon[96439]: Adjusting osd_memory_target on smithi134 to 4292M 2023-07-18T03:28:46.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:45 smithi040 ceph-mon[96439]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-18T03:28:46.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:45 smithi040 ceph-mon[96439]: osdmap e75: 12 total, 11 up, 12 in 2023-07-18T03:28:46.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:45 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:28:46.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:45 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:28:46.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:45 smithi064 ceph-mon[98059]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:46.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:45 smithi064 ceph-mon[98059]: Detected new or changed devices on smithi134 2023-07-18T03:28:46.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:45 smithi064 ceph-mon[98059]: Adjusting osd_memory_target on smithi134 to 4292M 2023-07-18T03:28:46.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:45 smithi064 ceph-mon[98059]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2023-07-18T03:28:46.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:45 smithi064 ceph-mon[98059]: osdmap e75: 12 total, 11 up, 12 in 2023-07-18T03:28:46.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:45 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:28:46.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:45 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:28:46.964 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:46 smithi040 ceph-mon[96439]: purged_snaps scrub starts 2023-07-18T03:28:46.965 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:46 smithi040 ceph-mon[96439]: purged_snaps scrub ok 2023-07-18T03:28:46.965 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:46 smithi040 ceph-mon[96439]: osd.11 [v2:172.21.15.134:6824/1938250059,v1:172.21.15.134:6825/1938250059] boot 2023-07-18T03:28:46.965 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:46 smithi040 ceph-mon[96439]: osdmap e76: 12 total, 12 up, 12 in 2023-07-18T03:28:46.965 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:46 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:28:47.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:46 smithi134 ceph-mon[98216]: purged_snaps scrub starts 2023-07-18T03:28:47.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:46 smithi134 ceph-mon[98216]: purged_snaps scrub ok 2023-07-18T03:28:47.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:46 smithi134 ceph-mon[98216]: osd.11 [v2:172.21.15.134:6824/1938250059,v1:172.21.15.134:6825/1938250059] boot 2023-07-18T03:28:47.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:46 smithi134 ceph-mon[98216]: osdmap e76: 12 total, 12 up, 12 in 2023-07-18T03:28:47.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:46 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:28:47.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.096+0000 7f1f7435c700 1 -- 172.21.15.40:0/639927366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f6c101750 msgr2=0x7f1f6c108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:47.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.096+0000 7f1f7435c700 1 --2- 172.21.15.40:0/639927366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f6c101750 0x7f1f6c108010 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400b0a0 tx=0x7f1f6400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:28:47.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.097+0000 7f1f7435c700 1 -- 172.21.15.40:0/639927366 shutdown_connections 2023-07-18T03:28:47.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.097+0000 7f1f7435c700 1 --2- 172.21.15.40:0/639927366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f6c101750 0x7f1f6c108010 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:47.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.097+0000 7f1f7435c700 1 --2- 172.21.15.40:0/639927366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f6c0ffbc0 0x7f1f6c0fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:47.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.097+0000 7f1f7435c700 1 --2- 172.21.15.40:0/639927366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f6c100dc0 0x7f1f6c101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:47.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.097+0000 7f1f7435c700 1 -- 172.21.15.40:0/639927366 >> 172.21.15.40:0/639927366 conn(0x7f1f6c0fb230 msgr2=0x7f1f6c0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:47.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.097+0000 7f1f7435c700 1 -- 172.21.15.40:0/639927366 shutdown_connections 2023-07-18T03:28:47.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.097+0000 7f1f7435c700 1 -- 172.21.15.40:0/639927366 wait complete. 2023-07-18T03:28:47.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.097+0000 7f1f7435c700 1 Processor -- start 2023-07-18T03:28:47.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.097+0000 7f1f7435c700 1 -- start start 2023-07-18T03:28:47.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.097+0000 7f1f7435c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f6c0ffbc0 0x7f1f6c195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:47.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.097+0000 7f1f7435c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f6c100dc0 0x7f1f6c195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:47.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f7435c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f6c101750 0x7f1f6c19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:47.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f7435c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c19d4a0 con 0x7f1f6c100dc0 2023-07-18T03:28:47.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f7435c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c19d5e0 con 0x7f1f6c101750 2023-07-18T03:28:47.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f7435c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c19d770 con 0x7f1f6c0ffbc0 2023-07-18T03:28:47.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f718f7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f6c100dc0 0x7f1f6c195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:47.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f720f8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f6c0ffbc0 0x7f1f6c195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:47.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f718f7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f6c100dc0 0x7f1f6c195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55714/0 (socket says 172.21.15.40:55714) 2023-07-18T03:28:47.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f718f7700 1 -- 172.21.15.40:0/2220964137 learned_addr learned my addr 172.21.15.40:0/2220964137 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:28:47.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f728f9700 1 --2- 172.21.15.40:0/2220964137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f6c101750 0x7f1f6c19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:47.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f718f7700 1 -- 172.21.15.40:0/2220964137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f6c0ffbc0 msgr2=0x7f1f6c195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:47.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f718f7700 1 --2- 172.21.15.40:0/2220964137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f6c0ffbc0 0x7f1f6c195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:47.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f718f7700 1 -- 172.21.15.40:0/2220964137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f6c101750 msgr2=0x7f1f6c19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:47.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f718f7700 1 --2- 172.21.15.40:0/2220964137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f6c101750 0x7f1f6c19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:47.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f718f7700 1 -- 172.21.15.40:0/2220964137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f64009d00 con 0x7f1f6c100dc0 2023-07-18T03:28:47.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f728f9700 1 --2- 172.21.15.40:0/2220964137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f6c101750 0x7f1f6c19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:28:47.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f718f7700 1 --2- 172.21.15.40:0/2220964137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f6c100dc0 0x7f1f6c195e50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00c950 tx=0x7f1f5c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:47.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f637fe700 1 -- 172.21.15.40:0/2220964137 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c00cea0 con 0x7f1f6c100dc0 2023-07-18T03:28:47.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f7435c700 1 -- 172.21.15.40:0/2220964137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6c19da50 con 0x7f1f6c100dc0 2023-07-18T03:28:47.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f637fe700 1 -- 172.21.15.40:0/2220964137 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f5c004830 con 0x7f1f6c100dc0 2023-07-18T03:28:47.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.098+0000 7f1f637fe700 1 -- 172.21.15.40:0/2220964137 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0054e0 con 0x7f1f6c100dc0 2023-07-18T03:28:47.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.099+0000 7f1f7435c700 1 -- 172.21.15.40:0/2220964137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c19df50 con 0x7f1f6c100dc0 2023-07-18T03:28:47.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.099+0000 7f1f7435c700 1 -- 172.21.15.40:0/2220964137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6c10b390 con 0x7f1f6c100dc0 2023-07-18T03:28:47.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.101+0000 7f1f637fe700 1 -- 172.21.15.40:0/2220964137 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1f5c0049a0 con 0x7f1f6c100dc0 2023-07-18T03:28:47.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.101+0000 7f1f637fe700 1 --2- 172.21.15.40:0/2220964137 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1f580a1150 0x7f1f580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:47.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.101+0000 7f1f720f8700 1 --2- 172.21.15.40:0/2220964137 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1f580a1150 0x7f1f580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:47.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.101+0000 7f1f637fe700 1 -- 172.21.15.40:0/2220964137 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1f5c0cc4f0 con 0x7f1f6c100dc0 2023-07-18T03:28:47.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.102+0000 7f1f720f8700 1 --2- 172.21.15.40:0/2220964137 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1f580a1150 0x7f1f580a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c100c20 tx=0x7f1f6800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:47.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:46 smithi064 ceph-mon[98059]: purged_snaps scrub starts 2023-07-18T03:28:47.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:46 smithi064 ceph-mon[98059]: purged_snaps scrub ok 2023-07-18T03:28:47.110 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:46 smithi064 ceph-mon[98059]: osd.11 [v2:172.21.15.134:6824/1938250059,v1:172.21.15.134:6825/1938250059] boot 2023-07-18T03:28:47.110 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:46 smithi064 ceph-mon[98059]: osdmap e76: 12 total, 12 up, 12 in 2023-07-18T03:28:47.110 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:46 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-18T03:28:47.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.105+0000 7f1f637fe700 1 -- 172.21.15.40:0/2220964137 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1f5c096040 con 0x7f1f6c100dc0 2023-07-18T03:28:47.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.304+0000 7f1f7435c700 1 -- 172.21.15.40:0/2220964137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f1f6c1069d0 con 0x7f1f6c100dc0 2023-07-18T03:28:47.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.304+0000 7f1f637fe700 1 -- 172.21.15.40:0/2220964137 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7f1f5c01d0a0 con 0x7f1f6c100dc0 2023-07-18T03:28:47.303 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:28:47.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.306+0000 7f1f7435c700 1 -- 172.21.15.40:0/2220964137 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1f580a1150 msgr2=0x7f1f580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:47.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.306+0000 7f1f7435c700 1 --2- 172.21.15.40:0/2220964137 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1f580a1150 0x7f1f580a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c100c20 tx=0x7f1f6800a3b0 comp rx=0 tx=0).stop 2023-07-18T03:28:47.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.306+0000 7f1f7435c700 1 -- 172.21.15.40:0/2220964137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f6c100dc0 msgr2=0x7f1f6c195e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:47.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.306+0000 7f1f7435c700 1 --2- 172.21.15.40:0/2220964137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f6c100dc0 0x7f1f6c195e50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00c950 tx=0x7f1f5c00cc60 comp rx=0 tx=0).stop 2023-07-18T03:28:47.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.307+0000 7f1f7435c700 1 -- 172.21.15.40:0/2220964137 shutdown_connections 2023-07-18T03:28:47.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.307+0000 7f1f7435c700 1 --2- 172.21.15.40:0/2220964137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f6c101750 0x7f1f6c19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:47.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.307+0000 7f1f7435c700 1 --2- 172.21.15.40:0/2220964137 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1f580a1150 0x7f1f580a3600 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:47.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.307+0000 7f1f7435c700 1 --2- 172.21.15.40:0/2220964137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f6c0ffbc0 0x7f1f6c195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:47.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.307+0000 7f1f7435c700 1 --2- 172.21.15.40:0/2220964137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f6c100dc0 0x7f1f6c195e50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:47.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.307+0000 7f1f7435c700 1 -- 172.21.15.40:0/2220964137 >> 172.21.15.40:0/2220964137 conn(0x7f1f6c0fb230 msgr2=0x7f1f6c0fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:47.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.307+0000 7f1f7435c700 1 -- 172.21.15.40:0/2220964137 shutdown_connections 2023-07-18T03:28:47.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:47.307+0000 7f1f7435c700 1 -- 172.21.15.40:0/2220964137 wait complete. 2023-07-18T03:28:47.690 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1689650925,"num_in_osds":12,"osd_in_since":1689650908,"num_remapped_pgs":0} 2023-07-18T03:28:47.691 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-18T03:28:48.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:47 smithi134 ceph-mon[98216]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:48.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:47 smithi134 ceph-mon[98216]: osdmap e77: 12 total, 12 up, 12 in 2023-07-18T03:28:48.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:47 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2220964137' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-18T03:28:48.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:47 smithi040 ceph-mon[96439]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:48.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:47 smithi040 ceph-mon[96439]: osdmap e77: 12 total, 12 up, 12 in 2023-07-18T03:28:48.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:47 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2220964137' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-18T03:28:48.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:47 smithi064 ceph-mon[98059]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-18T03:28:48.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:47 smithi064 ceph-mon[98059]: osdmap e77: 12 total, 12 up, 12 in 2023-07-18T03:28:48.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:47 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2220964137' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-18T03:28:48.253 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:28:49.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.428+0000 7f28e4da7700 1 -- 172.21.15.40:0/2306757329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28e0101700 msgr2=0x7f28e0109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:49.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.428+0000 7f28e4da7700 1 --2- 172.21.15.40:0/2306757329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28e0101700 0x7f28e0109c00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009990 tx=0x7f28c8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:28:49.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.428+0000 7f28e4da7700 1 -- 172.21.15.40:0/2306757329 shutdown_connections 2023-07-18T03:28:49.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.428+0000 7f28e4da7700 1 --2- 172.21.15.40:0/2306757329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28e010a140 0x7f28e010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:49.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.428+0000 7f28e4da7700 1 --2- 172.21.15.40:0/2306757329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e0100df0 0x7f28e01011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:49.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.428+0000 7f28e4da7700 1 --2- 172.21.15.40:0/2306757329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28e0101700 0x7f28e0109c00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:49.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.428+0000 7f28e4da7700 1 -- 172.21.15.40:0/2306757329 >> 172.21.15.40:0/2306757329 conn(0x7f28e0075b70 msgr2=0x7f28e0075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:49.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.428+0000 7f28e4da7700 1 -- 172.21.15.40:0/2306757329 shutdown_connections 2023-07-18T03:28:49.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.428+0000 7f28e4da7700 1 -- 172.21.15.40:0/2306757329 wait complete. 2023-07-18T03:28:49.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.429+0000 7f28e4da7700 1 Processor -- start 2023-07-18T03:28:49.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.429+0000 7f28e4da7700 1 -- start start 2023-07-18T03:28:49.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.429+0000 7f28e4da7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e0100df0 0x7f28e019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:49.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.429+0000 7f28e4da7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28e0101700 0x7f28e019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:49.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.429+0000 7f28e4da7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28e010a140 0x7f28e0195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:49.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.429+0000 7f28e4da7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e01960e0 con 0x7f28e0101700 2023-07-18T03:28:49.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.429+0000 7f28e4da7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e0196250 con 0x7f28e0100df0 2023-07-18T03:28:49.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.429+0000 7f28e4da7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e0196410 con 0x7f28e010a140 2023-07-18T03:28:49.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.429+0000 7f28d7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28e0101700 0x7f28e019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:49.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.429+0000 7f28d7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28e0101700 0x7f28e019bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55736/0 (socket says 172.21.15.40:55736) 2023-07-18T03:28:49.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.430+0000 7f28d7fff700 1 -- 172.21.15.40:0/1107160989 learned_addr learned my addr 172.21.15.40:0/1107160989 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:28:49.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.430+0000 7f28de59c700 1 --2- 172.21.15.40:0/1107160989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e0100df0 0x7f28e019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:49.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.430+0000 7f28ded9d700 1 --2- 172.21.15.40:0/1107160989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28e010a140 0x7f28e0195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:49.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.430+0000 7f28d7fff700 1 -- 172.21.15.40:0/1107160989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28e010a140 msgr2=0x7f28e0195a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:49.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.430+0000 7f28d7fff700 1 --2- 172.21.15.40:0/1107160989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28e010a140 0x7f28e0195a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:49.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.430+0000 7f28d7fff700 1 -- 172.21.15.40:0/1107160989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e0100df0 msgr2=0x7f28e019b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:49.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.430+0000 7f28d7fff700 1 --2- 172.21.15.40:0/1107160989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e0100df0 0x7f28e019b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:49.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.430+0000 7f28d7fff700 1 -- 172.21.15.40:0/1107160989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c80095f0 con 0x7f28e0101700 2023-07-18T03:28:49.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.430+0000 7f28d7fff700 1 --2- 172.21.15.40:0/1107160989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28e0101700 0x7f28e019bf30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f28c80095c0 tx=0x7f28c800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:49.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.430+0000 7f28d77fe700 1 -- 172.21.15.40:0/1107160989 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c801d070 con 0x7f28e0101700 2023-07-18T03:28:49.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.430+0000 7f28d77fe700 1 -- 172.21.15.40:0/1107160989 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28c800fb50 con 0x7f28e0101700 2023-07-18T03:28:49.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.430+0000 7f28e4da7700 1 -- 172.21.15.40:0/1107160989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e0196690 con 0x7f28e0101700 2023-07-18T03:28:49.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.430+0000 7f28d77fe700 1 -- 172.21.15.40:0/1107160989 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c80173f0 con 0x7f28e0101700 2023-07-18T03:28:49.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.430+0000 7f28e4da7700 1 -- 172.21.15.40:0/1107160989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e01a29e0 con 0x7f28e0101700 2023-07-18T03:28:49.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.431+0000 7f28e4da7700 1 -- 172.21.15.40:0/1107160989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e004e210 con 0x7f28e0101700 2023-07-18T03:28:49.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.432+0000 7f28d77fe700 1 -- 172.21.15.40:0/1107160989 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f28c800fcc0 con 0x7f28e0101700 2023-07-18T03:28:49.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.433+0000 7f28d77fe700 1 --2- 172.21.15.40:0/1107160989 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f28cc0a11d0 0x7f28cc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:49.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.433+0000 7f28de59c700 1 --2- 172.21.15.40:0/1107160989 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f28cc0a11d0 0x7f28cc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:49.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.433+0000 7f28d77fe700 1 -- 172.21.15.40:0/1107160989 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f28c80cd750 con 0x7f28e0101700 2023-07-18T03:28:49.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.434+0000 7f28de59c700 1 --2- 172.21.15.40:0/1107160989 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f28cc0a11d0 0x7f28cc0a3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f28d0007be0 tx=0x7f28d00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:49.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.437+0000 7f28d77fe700 1 -- 172.21.15.40:0/1107160989 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f28c80973d0 con 0x7f28e0101700 2023-07-18T03:28:49.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.632+0000 7f28e4da7700 1 -- 172.21.15.40:0/1107160989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28e0196f80 con 0x7f28e0101700 2023-07-18T03:28:49.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.633+0000 7f28d77fe700 1 -- 172.21.15.40:0/1107160989 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f28c809a9f0 con 0x7f28e0101700 2023-07-18T03:28:49.631 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:28:49.631 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":77,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:28:46.697008+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:28:49.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.635+0000 7f28e4da7700 1 -- 172.21.15.40:0/1107160989 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f28cc0a11d0 msgr2=0x7f28cc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:49.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.635+0000 7f28e4da7700 1 --2- 172.21.15.40:0/1107160989 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f28cc0a11d0 0x7f28cc0a3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f28d0007be0 tx=0x7f28d00073d0 comp rx=0 tx=0).stop 2023-07-18T03:28:49.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.635+0000 7f28e4da7700 1 -- 172.21.15.40:0/1107160989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28e0101700 msgr2=0x7f28e019bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:49.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.635+0000 7f28e4da7700 1 --2- 172.21.15.40:0/1107160989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28e0101700 0x7f28e019bf30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f28c80095c0 tx=0x7f28c800f710 comp rx=0 tx=0).stop 2023-07-18T03:28:49.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.635+0000 7f28e4da7700 1 -- 172.21.15.40:0/1107160989 shutdown_connections 2023-07-18T03:28:49.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.635+0000 7f28e4da7700 1 --2- 172.21.15.40:0/1107160989 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f28cc0a11d0 0x7f28cc0a3680 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:49.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.635+0000 7f28e4da7700 1 --2- 172.21.15.40:0/1107160989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28e010a140 0x7f28e0195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:49.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.635+0000 7f28e4da7700 1 --2- 172.21.15.40:0/1107160989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e0100df0 0x7f28e019b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:49.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.635+0000 7f28e4da7700 1 --2- 172.21.15.40:0/1107160989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28e0101700 0x7f28e019bf30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:49.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.635+0000 7f28e4da7700 1 -- 172.21.15.40:0/1107160989 >> 172.21.15.40:0/1107160989 conn(0x7f28e0075b70 msgr2=0x7f28e00fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:49.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.635+0000 7f28e4da7700 1 -- 172.21.15.40:0/1107160989 shutdown_connections 2023-07-18T03:28:49.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:49.635+0000 7f28e4da7700 1 -- 172.21.15.40:0/1107160989 wait complete. 2023-07-18T03:28:49.981 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:49 smithi040 ceph-mon[96439]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:49.981 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:49 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1107160989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:28:50.010 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:25:38.267262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:28:50.011 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-07-18T03:28:50.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:49 smithi134 ceph-mon[98216]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:50.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:49 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1107160989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:28:50.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:49 smithi064 ceph-mon[98059]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:50.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:49 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1107160989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:28:50.566 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:28:51.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.840+0000 7f0c7deb6700 1 -- 172.21.15.40:0/4231519502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c7810a140 msgr2=0x7f0c7810c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:51.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.840+0000 7f0c7deb6700 1 --2- 172.21.15.40:0/4231519502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c7810a140 0x7f0c7810c580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000b0a0 tx=0x7f0c7000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:28:51.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.840+0000 7f0c7deb6700 1 -- 172.21.15.40:0/4231519502 shutdown_connections 2023-07-18T03:28:51.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.840+0000 7f0c7deb6700 1 --2- 172.21.15.40:0/4231519502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c7810a140 0x7f0c7810c580 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:51.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.840+0000 7f0c7deb6700 1 --2- 172.21.15.40:0/4231519502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c78100df0 0x7f0c781011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:51.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.840+0000 7f0c7deb6700 1 --2- 172.21.15.40:0/4231519502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c78101700 0x7f0c78109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:51.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.840+0000 7f0c7deb6700 1 -- 172.21.15.40:0/4231519502 >> 172.21.15.40:0/4231519502 conn(0x7f0c78075b70 msgr2=0x7f0c78075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:51.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.840+0000 7f0c7deb6700 1 -- 172.21.15.40:0/4231519502 shutdown_connections 2023-07-18T03:28:51.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.840+0000 7f0c7deb6700 1 -- 172.21.15.40:0/4231519502 wait complete. 2023-07-18T03:28:51.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.840+0000 7f0c7deb6700 1 Processor -- start 2023-07-18T03:28:51.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c7deb6700 1 -- start start 2023-07-18T03:28:51.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c7deb6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c78100df0 0x7f0c78195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:51.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c7deb6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c78101700 0x7f0c781962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:51.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c7deb6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c7810a140 0x7f0c7819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:51.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c7deb6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7819ad40 con 0x7f0c78101700 2023-07-18T03:28:51.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c7deb6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7819aeb0 con 0x7f0c78100df0 2023-07-18T03:28:51.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c7deb6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7819b070 con 0x7f0c7810a140 2023-07-18T03:28:51.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c76ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c78101700 0x7f0c781962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:51.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c76ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c78101700 0x7f0c781962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55748/0 (socket says 172.21.15.40:55748) 2023-07-18T03:28:51.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c76ffd700 1 -- 172.21.15.40:0/3838680704 learned_addr learned my addr 172.21.15.40:0/3838680704 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:28:51.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c777fe700 1 --2- 172.21.15.40:0/3838680704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c78100df0 0x7f0c78195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:51.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c77fff700 1 --2- 172.21.15.40:0/3838680704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c7810a140 0x7f0c7819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:51.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c76ffd700 1 -- 172.21.15.40:0/3838680704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c7810a140 msgr2=0x7f0c7819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:51.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c76ffd700 1 --2- 172.21.15.40:0/3838680704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c7810a140 0x7f0c7819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:51.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c76ffd700 1 -- 172.21.15.40:0/3838680704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c78100df0 msgr2=0x7f0c78195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:51.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c76ffd700 1 --2- 172.21.15.40:0/3838680704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c78100df0 0x7f0c78195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:51.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c76ffd700 1 -- 172.21.15.40:0/3838680704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c70009d00 con 0x7f0c78101700 2023-07-18T03:28:51.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.841+0000 7f0c76ffd700 1 --2- 172.21.15.40:0/3838680704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c78101700 0x7f0c781962c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00d950 tx=0x7f0c6c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:51.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.842+0000 7f0c74ff9700 1 -- 172.21.15.40:0/3838680704 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c00f890 con 0x7f0c78101700 2023-07-18T03:28:51.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.842+0000 7f0c7deb6700 1 -- 172.21.15.40:0/3838680704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7819b350 con 0x7f0c78101700 2023-07-18T03:28:51.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.842+0000 7f0c74ff9700 1 -- 172.21.15.40:0/3838680704 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c6c010460 con 0x7f0c78101700 2023-07-18T03:28:51.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.842+0000 7f0c74ff9700 1 -- 172.21.15.40:0/3838680704 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c00e610 con 0x7f0c78101700 2023-07-18T03:28:51.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.842+0000 7f0c7deb6700 1 -- 172.21.15.40:0/3838680704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c781a2bb0 con 0x7f0c78101700 2023-07-18T03:28:51.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.843+0000 7f0c7deb6700 1 -- 172.21.15.40:0/3838680704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7804e210 con 0x7f0c78101700 2023-07-18T03:28:51.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.844+0000 7f0c74ff9700 1 -- 172.21.15.40:0/3838680704 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0c6c0108d0 con 0x7f0c78101700 2023-07-18T03:28:51.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.844+0000 7f0c74ff9700 1 --2- 172.21.15.40:0/3838680704 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0c600a1150 0x7f0c600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:51.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.845+0000 7f0c777fe700 1 --2- 172.21.15.40:0/3838680704 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0c600a1150 0x7f0c600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:51.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.845+0000 7f0c74ff9700 1 -- 172.21.15.40:0/3838680704 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0c6c0cc250 con 0x7f0c78101700 2023-07-18T03:28:51.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.845+0000 7f0c777fe700 1 --2- 172.21.15.40:0/3838680704 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0c600a1150 0x7f0c600a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0c68005950 tx=0x7f0c6800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:51.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:51.849+0000 7f0c74ff9700 1 -- 172.21.15.40:0/3838680704 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0c6c095da0 con 0x7f0c78101700 2023-07-18T03:28:52.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:52.045+0000 7f0c7deb6700 1 -- 172.21.15.40:0/3838680704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0c781a2e40 con 0x7f0c78101700 2023-07-18T03:28:52.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:52.046+0000 7f0c74ff9700 1 -- 172.21.15.40:0/3838680704 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0c6c0993c0 con 0x7f0c78101700 2023-07-18T03:28:52.044 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-07-18T03:28:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:52.048+0000 7f0c7deb6700 1 -- 172.21.15.40:0/3838680704 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0c600a1150 msgr2=0x7f0c600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:52.048+0000 7f0c7deb6700 1 --2- 172.21.15.40:0/3838680704 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0c600a1150 0x7f0c600a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0c68005950 tx=0x7f0c6800b410 comp rx=0 tx=0).stop 2023-07-18T03:28:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:52.048+0000 7f0c7deb6700 1 -- 172.21.15.40:0/3838680704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c78101700 msgr2=0x7f0c781962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:52.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:52.048+0000 7f0c7deb6700 1 --2- 172.21.15.40:0/3838680704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c78101700 0x7f0c781962c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00d950 tx=0x7f0c6c00dc60 comp rx=0 tx=0).stop 2023-07-18T03:28:52.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:52.048+0000 7f0c7deb6700 1 -- 172.21.15.40:0/3838680704 shutdown_connections 2023-07-18T03:28:52.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:52.048+0000 7f0c7deb6700 1 --2- 172.21.15.40:0/3838680704 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0c600a1150 0x7f0c600a3600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:52.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:52.048+0000 7f0c7deb6700 1 --2- 172.21.15.40:0/3838680704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c7810a140 0x7f0c7819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:52.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:52.048+0000 7f0c7deb6700 1 --2- 172.21.15.40:0/3838680704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c78100df0 0x7f0c78195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:52.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:52.048+0000 7f0c7deb6700 1 --2- 172.21.15.40:0/3838680704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c78101700 0x7f0c781962c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:52.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:52.048+0000 7f0c7deb6700 1 -- 172.21.15.40:0/3838680704 >> 172.21.15.40:0/3838680704 conn(0x7f0c78075b70 msgr2=0x7f0c780fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:52.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:52.048+0000 7f0c7deb6700 1 -- 172.21.15.40:0/3838680704 shutdown_connections 2023-07-18T03:28:52.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:52.048+0000 7f0c7deb6700 1 -- 172.21.15.40:0/3838680704 wait complete. 2023-07-18T03:28:52.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:51 smithi134 ceph-mon[98216]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:52.067 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:51 smithi040 ceph-mon[96439]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:52.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:51 smithi064 ceph-mon[98059]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:52.414 INFO:tasks.cephadm:Adding mds.a on smithi040 2023-07-18T03:28:52.415 INFO:tasks.cephadm:Adding mds.d on smithi040 2023-07-18T03:28:52.415 INFO:tasks.cephadm:Adding mds.g on smithi040 2023-07-18T03:28:52.416 INFO:tasks.cephadm:Adding mds.j on smithi040 2023-07-18T03:28:52.416 INFO:tasks.cephadm:Adding mds.b on smithi064 2023-07-18T03:28:52.417 INFO:tasks.cephadm:Adding mds.e on smithi064 2023-07-18T03:28:52.417 INFO:tasks.cephadm:Adding mds.h on smithi064 2023-07-18T03:28:52.418 INFO:tasks.cephadm:Adding mds.k on smithi064 2023-07-18T03:28:52.418 INFO:tasks.cephadm:Adding mds.c on smithi134 2023-07-18T03:28:52.419 INFO:tasks.cephadm:Adding mds.f on smithi134 2023-07-18T03:28:52.419 INFO:tasks.cephadm:Adding mds.i on smithi134 2023-07-18T03:28:52.419 INFO:tasks.cephadm:Adding mds.l on smithi134 2023-07-18T03:28:52.419 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph orch apply mds all '12;smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l' 2023-07-18T03:28:52.990 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:28:53.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:52 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3838680704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:28:53.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:52 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3838680704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:28:53.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:52 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3838680704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:28:53.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:53 smithi134 ceph-mon[98216]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:54.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:53 smithi040 ceph-mon[96439]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:54.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:53 smithi064 ceph-mon[98059]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:54.198 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.202+0000 7f5f0afad700 1 -- 172.21.15.134:0/2678223050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f04102f30 msgr2=0x7f5f04105310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:54.199 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.202+0000 7f5f0afad700 1 --2- 172.21.15.134:0/2678223050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f04102f30 0x7f5f04105310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009990 tx=0x7f5ef0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:28:54.199 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.202+0000 7f5f0afad700 1 -- 172.21.15.134:0/2678223050 shutdown_connections 2023-07-18T03:28:54.199 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.202+0000 7f5f0afad700 1 --2- 172.21.15.134:0/2678223050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f04105850 0x7f5f04107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:54.199 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.202+0000 7f5f0afad700 1 --2- 172.21.15.134:0/2678223050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f04100610 0x7f5f041029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:54.199 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.202+0000 7f5f0afad700 1 --2- 172.21.15.134:0/2678223050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f04102f30 0x7f5f04105310 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:54.199 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.202+0000 7f5f0afad700 1 -- 172.21.15.134:0/2678223050 >> 172.21.15.134:0/2678223050 conn(0x7f5f040fa0b0 msgr2=0x7f5f040fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:54.199 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.202+0000 7f5f0afad700 1 -- 172.21.15.134:0/2678223050 shutdown_connections 2023-07-18T03:28:54.200 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.202+0000 7f5f0afad700 1 -- 172.21.15.134:0/2678223050 wait complete. 2023-07-18T03:28:54.201 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f0afad700 1 Processor -- start 2023-07-18T03:28:54.201 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f0afad700 1 -- start start 2023-07-18T03:28:54.201 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f0afad700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f04100610 0x7f5f04195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:54.201 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f0afad700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f04102f30 0x7f5f04195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:54.202 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f0afad700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f04105850 0x7f5f0419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:54.202 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f0afad700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0419d420 con 0x7f5f04102f30 2023-07-18T03:28:54.202 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f0afad700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0419d560 con 0x7f5f04100610 2023-07-18T03:28:54.202 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f0afad700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0419d6f0 con 0x7f5f04105850 2023-07-18T03:28:54.202 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f0954a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f04105850 0x7f5f0419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:54.202 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f0954a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f04105850 0x7f5f0419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:48512/0 (socket says 172.21.15.134:48512) 2023-07-18T03:28:54.202 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f0954a700 1 -- 172.21.15.134:0/3568375542 learned_addr learned my addr 172.21.15.134:0/3568375542 (peer_addr_for_me v2:172.21.15.134:0/0) 2023-07-18T03:28:54.203 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f03fff700 1 --2- 172.21.15.134:0/3568375542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f04102f30 0x7f5f04195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:54.203 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f08d49700 1 --2- 172.21.15.134:0/3568375542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f04100610 0x7f5f04195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:54.203 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f0954a700 1 -- 172.21.15.134:0/3568375542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f04100610 msgr2=0x7f5f04195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:54.203 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f0954a700 1 --2- 172.21.15.134:0/3568375542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f04100610 0x7f5f04195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:54.203 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f0954a700 1 -- 172.21.15.134:0/3568375542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f04102f30 msgr2=0x7f5f04195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:54.204 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f0954a700 1 --2- 172.21.15.134:0/3568375542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f04102f30 0x7f5f04195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:54.204 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f0954a700 1 -- 172.21.15.134:0/3568375542 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef00095f0 con 0x7f5f04105850 2023-07-18T03:28:54.204 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.203+0000 7f5f0954a700 1 --2- 172.21.15.134:0/3568375542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f04105850 0x7f5f0419ce50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5efc007ef0 tx=0x7f5efc00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:54.204 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.204+0000 7f5f01ffb700 1 -- 172.21.15.134:0/3568375542 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc017070 con 0x7f5f04105850 2023-07-18T03:28:54.204 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.204+0000 7f5f01ffb700 1 -- 172.21.15.134:0/3568375542 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5efc00f040 con 0x7f5f04105850 2023-07-18T03:28:54.204 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.204+0000 7f5f0afad700 1 -- 172.21.15.134:0/3568375542 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0419d9d0 con 0x7f5f04105850 2023-07-18T03:28:54.204 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.204+0000 7f5f01ffb700 1 -- 172.21.15.134:0/3568375542 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc004280 con 0x7f5f04105850 2023-07-18T03:28:54.205 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.204+0000 7f5f0afad700 1 -- 172.21.15.134:0/3568375542 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0419de80 con 0x7f5f04105850 2023-07-18T03:28:54.205 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.205+0000 7f5f0afad700 1 -- 172.21.15.134:0/3568375542 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f040fbc20 con 0x7f5f04105850 2023-07-18T03:28:54.205 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.206+0000 7f5f01ffb700 1 -- 172.21.15.134:0/3568375542 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5efc00db20 con 0x7f5f04105850 2023-07-18T03:28:54.206 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.206+0000 7f5f01ffb700 1 --2- 172.21.15.134:0/3568375542 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5ef40a1150 0x7f5ef40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:54.206 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.206+0000 7f5f01ffb700 1 -- 172.21.15.134:0/3568375542 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5efc0cbae0 con 0x7f5f04105850 2023-07-18T03:28:54.206 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.207+0000 7f5f08d49700 1 --2- 172.21.15.134:0/3568375542 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5ef40a1150 0x7f5ef40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:54.206 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.207+0000 7f5f08d49700 1 --2- 172.21.15.134:0/3568375542 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5ef40a1150 0x7f5ef40a3600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8005950 tx=0x7f5ef800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:54.207 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.211+0000 7f5f01ffb700 1 -- 172.21.15.134:0/3568375542 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5efc0956c0 con 0x7f5f04105850 2023-07-18T03:28:54.399 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.403+0000 7f5f0afad700 1 -- 172.21.15.134:0/3568375542 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f04060c10 con 0x7f5ef40a1150 2023-07-18T03:28:54.409 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.412+0000 7f5f01ffb700 1 -- 172.21.15.134:0/3568375542 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f5f04060c10 con 0x7f5ef40a1150 2023-07-18T03:28:54.409 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mds.all update... 2023-07-18T03:28:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.415+0000 7f5f0afad700 1 -- 172.21.15.134:0/3568375542 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5ef40a1150 msgr2=0x7f5ef40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.415+0000 7f5f0afad700 1 --2- 172.21.15.134:0/3568375542 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5ef40a1150 0x7f5ef40a3600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8005950 tx=0x7f5ef800a300 comp rx=0 tx=0).stop 2023-07-18T03:28:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.415+0000 7f5f0afad700 1 -- 172.21.15.134:0/3568375542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f04105850 msgr2=0x7f5f0419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.415+0000 7f5f0afad700 1 --2- 172.21.15.134:0/3568375542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f04105850 0x7f5f0419ce50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5efc007ef0 tx=0x7f5efc00d3b0 comp rx=0 tx=0).stop 2023-07-18T03:28:54.411 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.415+0000 7f5f0afad700 1 -- 172.21.15.134:0/3568375542 shutdown_connections 2023-07-18T03:28:54.412 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.415+0000 7f5f0afad700 1 --2- 172.21.15.134:0/3568375542 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5ef40a1150 0x7f5ef40a3600 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:54.412 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.415+0000 7f5f0afad700 1 --2- 172.21.15.134:0/3568375542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f04105850 0x7f5f0419ce50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:54.412 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.415+0000 7f5f0afad700 1 --2- 172.21.15.134:0/3568375542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f04100610 0x7f5f04195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:54.413 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.415+0000 7f5f0afad700 1 --2- 172.21.15.134:0/3568375542 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f04102f30 0x7f5f04195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:54.413 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.415+0000 7f5f0afad700 1 -- 172.21.15.134:0/3568375542 >> 172.21.15.134:0/3568375542 conn(0x7f5f040fa0b0 msgr2=0x7f5f04101270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:54.413 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.415+0000 7f5f0afad700 1 -- 172.21.15.134:0/3568375542 shutdown_connections 2023-07-18T03:28:54.414 INFO:teuthology.orchestra.run.smithi134.stderr:2023-07-18T03:28:54.415+0000 7f5f0afad700 1 -- 172.21.15.134:0/3568375542 wait complete. 2023-07-18T03:28:54.836 DEBUG:teuthology.orchestra.run.smithi040:mds.a> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.a.service 2023-07-18T03:28:54.839 DEBUG:teuthology.orchestra.run.smithi040:mds.d> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.d.service 2023-07-18T03:28:54.875 INFO:journalctl@ceph.mds.a.smithi040.stdout:-- Logs begin at Tue 2023-07-18 03:07:07 UTC. -- 2023-07-18T03:28:54.876 DEBUG:teuthology.orchestra.run.smithi040:mds.g> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.g.service 2023-07-18T03:28:54.905 INFO:journalctl@ceph.mds.d.smithi040.stdout:-- Logs begin at Tue 2023-07-18 03:07:07 UTC. -- 2023-07-18T03:28:54.906 DEBUG:teuthology.orchestra.run.smithi040:mds.j> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.j.service 2023-07-18T03:28:54.935 INFO:journalctl@ceph.mds.g.smithi040.stdout:-- Logs begin at Tue 2023-07-18 03:07:07 UTC. -- 2023-07-18T03:28:54.936 DEBUG:teuthology.orchestra.run.smithi064:mds.b> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.b.service 2023-07-18T03:28:54.938 DEBUG:teuthology.orchestra.run.smithi064:mds.e> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.e.service 2023-07-18T03:28:54.963 INFO:journalctl@ceph.mds.j.smithi040.stdout:-- Logs begin at Tue 2023-07-18 03:07:07 UTC. -- 2023-07-18T03:28:54.976 INFO:journalctl@ceph.mds.b.smithi064.stdout:-- Logs begin at Tue 2023-07-18 03:07:19 UTC. -- 2023-07-18T03:28:54.977 DEBUG:teuthology.orchestra.run.smithi064:mds.h> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.h.service 2023-07-18T03:28:55.008 INFO:journalctl@ceph.mds.e.smithi064.stdout:-- Logs begin at Tue 2023-07-18 03:07:19 UTC. -- 2023-07-18T03:28:55.009 DEBUG:teuthology.orchestra.run.smithi064:mds.k> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.k.service 2023-07-18T03:28:55.037 INFO:journalctl@ceph.mds.h.smithi064.stdout:-- Logs begin at Tue 2023-07-18 03:07:19 UTC. -- 2023-07-18T03:28:55.039 DEBUG:teuthology.orchestra.run.smithi134:mds.c> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.c.service 2023-07-18T03:28:55.042 DEBUG:teuthology.orchestra.run.smithi134:mds.f> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.f.service 2023-07-18T03:28:55.066 INFO:journalctl@ceph.mds.k.smithi064.stdout:-- Logs begin at Tue 2023-07-18 03:07:19 UTC. -- 2023-07-18T03:28:55.072 INFO:journalctl@ceph.mds.c.smithi134.stdout:-- Logs begin at Tue 2023-07-18 03:07:14 UTC. -- 2023-07-18T03:28:55.073 DEBUG:teuthology.orchestra.run.smithi134:mds.i> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.i.service 2023-07-18T03:28:55.101 INFO:journalctl@ceph.mds.f.smithi134.stdout:-- Logs begin at Tue 2023-07-18 03:07:14 UTC. -- 2023-07-18T03:28:55.102 DEBUG:teuthology.orchestra.run.smithi134:mds.l> sudo journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.l.service 2023-07-18T03:28:55.131 INFO:journalctl@ceph.mds.i.smithi134.stdout:-- Logs begin at Tue 2023-07-18 03:07:14 UTC. -- 2023-07-18T03:28:55.132 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-07-18T03:28:55.133 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:28:55.158 INFO:journalctl@ceph.mds.l.smithi134.stdout:-- Logs begin at Tue 2023-07-18 03:07:14 UTC. -- 2023-07-18T03:28:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.275+0000 7f74cd192700 1 -- 172.21.15.40:0/592630869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c8100cf0 msgr2=0x7f74c8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.275+0000 7f74cd192700 1 --2- 172.21.15.40:0/592630869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c8100cf0 0x7f74c8101140 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009990 tx=0x7f74bc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.275+0000 7f74cd192700 1 -- 172.21.15.40:0/592630869 shutdown_connections 2023-07-18T03:28:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.275+0000 7f74cd192700 1 --2- 172.21.15.40:0/592630869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74c8101680 0x7f74c8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.275+0000 7f74cd192700 1 --2- 172.21.15.40:0/592630869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74c80ffb00 0x7f74c80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.275+0000 7f74cd192700 1 --2- 172.21.15.40:0/592630869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c8100cf0 0x7f74c8101140 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.275+0000 7f74cd192700 1 -- 172.21.15.40:0/592630869 >> 172.21.15.40:0/592630869 conn(0x7f74c80fb100 msgr2=0x7f74c80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:55.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.275+0000 7f74cd192700 1 -- 172.21.15.40:0/592630869 shutdown_connections 2023-07-18T03:28:55.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.275+0000 7f74cd192700 1 -- 172.21.15.40:0/592630869 wait complete. 2023-07-18T03:28:55.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74cd192700 1 Processor -- start 2023-07-18T03:28:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74cd192700 1 -- start start 2023-07-18T03:28:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74cd192700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c80ffb00 0x7f74c8195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74cd192700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74c8100cf0 0x7f74c8195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74cd192700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74c8101680 0x7f74c819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:55.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74cd192700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c819d310 con 0x7f74c80ffb00 2023-07-18T03:28:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74cd192700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c819d450 con 0x7f74c8100cf0 2023-07-18T03:28:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74cd192700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c819d5e0 con 0x7f74c8101680 2023-07-18T03:28:55.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74c6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c80ffb00 0x7f74c8195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74c6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c80ffb00 0x7f74c8195790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55758/0 (socket says 172.21.15.40:55758) 2023-07-18T03:28:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74c6d9d700 1 -- 172.21.15.40:0/862878229 learned_addr learned my addr 172.21.15.40:0/862878229 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:28:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74c759e700 1 --2- 172.21.15.40:0/862878229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74c8101680 0x7f74c819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74c659c700 1 --2- 172.21.15.40:0/862878229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74c8100cf0 0x7f74c8195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74c6d9d700 1 -- 172.21.15.40:0/862878229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74c8101680 msgr2=0x7f74c819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74c6d9d700 1 --2- 172.21.15.40:0/862878229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74c8101680 0x7f74c819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74c6d9d700 1 -- 172.21.15.40:0/862878229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74c8100cf0 msgr2=0x7f74c8195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:55.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74c6d9d700 1 --2- 172.21.15.40:0/862878229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74c8100cf0 0x7f74c8195cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.276+0000 7f74c6d9d700 1 -- 172.21.15.40:0/862878229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc0095f0 con 0x7f74c80ffb00 2023-07-18T03:28:55.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.277+0000 7f74c6d9d700 1 --2- 172.21.15.40:0/862878229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c80ffb00 0x7f74c8195790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f74b800e970 tx=0x7f74b800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:55.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.277+0000 7f74affff700 1 -- 172.21.15.40:0/862878229 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8009790 con 0x7f74c80ffb00 2023-07-18T03:28:55.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.277+0000 7f74affff700 1 -- 172.21.15.40:0/862878229 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74b800c7e0 con 0x7f74c80ffb00 2023-07-18T03:28:55.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.277+0000 7f74cd192700 1 -- 172.21.15.40:0/862878229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c819d8c0 con 0x7f74c80ffb00 2023-07-18T03:28:55.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.277+0000 7f74affff700 1 -- 172.21.15.40:0/862878229 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8003c70 con 0x7f74c80ffb00 2023-07-18T03:28:55.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.277+0000 7f74cd192700 1 -- 172.21.15.40:0/862878229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c819ddd0 con 0x7f74c80ffb00 2023-07-18T03:28:55.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.278+0000 7f74cd192700 1 -- 172.21.15.40:0/862878229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c8107020 con 0x7f74c80ffb00 2023-07-18T03:28:55.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.279+0000 7f74affff700 1 -- 172.21.15.40:0/862878229 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f74b800cc50 con 0x7f74c80ffb00 2023-07-18T03:28:55.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.280+0000 7f74affff700 1 --2- 172.21.15.40:0/862878229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74b00a11a0 0x7f74b00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:55.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.280+0000 7f74c659c700 1 --2- 172.21.15.40:0/862878229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74b00a11a0 0x7f74b00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:55.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.280+0000 7f74affff700 1 -- 172.21.15.40:0/862878229 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f74b8014070 con 0x7f74c80ffb00 2023-07-18T03:28:55.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.280+0000 7f74c659c700 1 --2- 172.21.15.40:0/862878229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74b00a11a0 0x7f74b00a3650 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f74bc00b2e0 tx=0x7f74bc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:55.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.283+0000 7f74affff700 1 -- 172.21.15.40:0/862878229 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f74b8096370 con 0x7f74c80ffb00 2023-07-18T03:28:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.483+0000 7f74cd192700 1 -- 172.21.15.40:0/862878229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74c802c850 con 0x7f74c80ffb00 2023-07-18T03:28:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.484+0000 7f74affff700 1 -- 172.21.15.40:0/862878229 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f74b8099990 con 0x7f74c80ffb00 2023-07-18T03:28:55.482 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:28:55.482 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":77,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:28:46.697008+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:28:55.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.486+0000 7f74cd192700 1 -- 172.21.15.40:0/862878229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74b00a11a0 msgr2=0x7f74b00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:55.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.486+0000 7f74cd192700 1 --2- 172.21.15.40:0/862878229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74b00a11a0 0x7f74b00a3650 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f74bc00b2e0 tx=0x7f74bc005960 comp rx=0 tx=0).stop 2023-07-18T03:28:55.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.486+0000 7f74cd192700 1 -- 172.21.15.40:0/862878229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c80ffb00 msgr2=0x7f74c8195790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:55.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.486+0000 7f74cd192700 1 --2- 172.21.15.40:0/862878229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c80ffb00 0x7f74c8195790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f74b800e970 tx=0x7f74b800ec80 comp rx=0 tx=0).stop 2023-07-18T03:28:55.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.486+0000 7f74cd192700 1 -- 172.21.15.40:0/862878229 shutdown_connections 2023-07-18T03:28:55.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.486+0000 7f74cd192700 1 --2- 172.21.15.40:0/862878229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74b00a11a0 0x7f74b00a3650 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.486+0000 7f74cd192700 1 --2- 172.21.15.40:0/862878229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74c8101680 0x7f74c819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.486+0000 7f74cd192700 1 --2- 172.21.15.40:0/862878229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74c80ffb00 0x7f74c8195790 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.486+0000 7f74cd192700 1 --2- 172.21.15.40:0/862878229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74c8100cf0 0x7f74c8195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.486+0000 7f74cd192700 1 -- 172.21.15.40:0/862878229 >> 172.21.15.40:0/862878229 conn(0x7f74c80fb100 msgr2=0x7f74c8104160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:55.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.486+0000 7f74cd192700 1 -- 172.21.15.40:0/862878229 shutdown_connections 2023-07-18T03:28:55.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.486+0000 7f74cd192700 1 -- 172.21.15.40:0/862878229 wait complete. 2023-07-18T03:28:55.496 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:55 smithi040 ceph-mon[96439]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:55.497 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:55 smithi040 ceph-mon[96439]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:28:55.497 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:55 smithi040 ceph-mon[96439]: Saving service mds.all spec with placement smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2023-07-18T03:28:55.497 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:55.497 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:55.497 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:55.498 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:55.498 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:55.498 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:28:55.498 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:28:55.498 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:55.499 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:55 smithi040 ceph-mon[96439]: Deploying daemon mds.l on smithi134 2023-07-18T03:28:55.501 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:25:38.267262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:28:55.502 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:28:55.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.634+0000 7f3a513bd700 1 -- 172.21.15.40:0/4007648042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a4c101d10 msgr2=0x7f3a4c102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.634+0000 7f3a513bd700 1 --2- 172.21.15.40:0/4007648042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a4c101d10 0x7f3a4c102120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009990 tx=0x7f3a3c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.634+0000 7f3a513bd700 1 -- 172.21.15.40:0/4007648042 shutdown_connections 2023-07-18T03:28:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.634+0000 7f3a513bd700 1 --2- 172.21.15.40:0/4007648042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a4c103890 0x7f3a4c10a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.634+0000 7f3a513bd700 1 --2- 172.21.15.40:0/4007648042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a4c101d10 0x7f3a4c102120 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.634+0000 7f3a513bd700 1 --2- 172.21.15.40:0/4007648042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a4c102f00 0x7f3a4c103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.634+0000 7f3a513bd700 1 -- 172.21.15.40:0/4007648042 >> 172.21.15.40:0/4007648042 conn(0x7f3a4c0fd350 msgr2=0x7f3a4c0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.634+0000 7f3a513bd700 1 -- 172.21.15.40:0/4007648042 shutdown_connections 2023-07-18T03:28:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.634+0000 7f3a513bd700 1 -- 172.21.15.40:0/4007648042 wait complete. 2023-07-18T03:28:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.634+0000 7f3a513bd700 1 Processor -- start 2023-07-18T03:28:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a513bd700 1 -- start start 2023-07-18T03:28:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a513bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a4c101d10 0x7f3a4c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a513bd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a4c102f00 0x7f3a4c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a513bd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a4c103890 0x7f3a4c19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a513bd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c19f660 con 0x7f3a4c103890 2023-07-18T03:28:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a513bd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c19f7a0 con 0x7f3a4c101d10 2023-07-18T03:28:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a513bd700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c19f930 con 0x7f3a4c102f00 2023-07-18T03:28:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a4b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a4c103890 0x7f3a4c19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a4a7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a4c102f00 0x7f3a4c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a4b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a4c103890 0x7f3a4c19f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55776/0 (socket says 172.21.15.40:55776) 2023-07-18T03:28:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a4b7fe700 1 -- 172.21.15.40:0/1061053172 learned_addr learned my addr 172.21.15.40:0/1061053172 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:28:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a4affd700 1 --2- 172.21.15.40:0/1061053172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a4c101d10 0x7f3a4c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a4b7fe700 1 -- 172.21.15.40:0/1061053172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a4c102f00 msgr2=0x7f3a4c198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a4b7fe700 1 --2- 172.21.15.40:0/1061053172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a4c102f00 0x7f3a4c198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a4b7fe700 1 -- 172.21.15.40:0/1061053172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a4c101d10 msgr2=0x7f3a4c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a4b7fe700 1 --2- 172.21.15.40:0/1061053172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a4c101d10 0x7f3a4c197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a4b7fe700 1 -- 172.21.15.40:0/1061053172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c0095f0 con 0x7f3a4c103890 2023-07-18T03:28:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a4affd700 1 --2- 172.21.15.40:0/1061053172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a4c101d10 0x7f3a4c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:28:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.635+0000 7f3a4b7fe700 1 --2- 172.21.15.40:0/1061053172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a4c103890 0x7f3a4c19f090 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400b970 tx=0x7f3a4400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:55.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.636+0000 7f3a33fff700 1 -- 172.21.15.40:0/1061053172 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4400c930 con 0x7f3a4c103890 2023-07-18T03:28:55.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.636+0000 7f3a33fff700 1 -- 172.21.15.40:0/1061053172 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a44014440 con 0x7f3a4c103890 2023-07-18T03:28:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.636+0000 7f3a513bd700 1 -- 172.21.15.40:0/1061053172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a4c19fc10 con 0x7f3a4c103890 2023-07-18T03:28:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.636+0000 7f3a33fff700 1 -- 172.21.15.40:0/1061053172 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a440075d0 con 0x7f3a4c103890 2023-07-18T03:28:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.636+0000 7f3a513bd700 1 -- 172.21.15.40:0/1061053172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a4c1a00d0 con 0x7f3a4c103890 2023-07-18T03:28:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.637+0000 7f3a513bd700 1 -- 172.21.15.40:0/1061053172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a4c108c90 con 0x7f3a4c103890 2023-07-18T03:28:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.638+0000 7f3a33fff700 1 -- 172.21.15.40:0/1061053172 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3a440145b0 con 0x7f3a4c103890 2023-07-18T03:28:55.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.639+0000 7f3a33fff700 1 --2- 172.21.15.40:0/1061053172 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3a340a1150 0x7f3a340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:55.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.639+0000 7f3a4affd700 1 --2- 172.21.15.40:0/1061053172 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3a340a1150 0x7f3a340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:55.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.639+0000 7f3a33fff700 1 -- 172.21.15.40:0/1061053172 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3a44006e40 con 0x7f3a4c103890 2023-07-18T03:28:55.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.639+0000 7f3a4affd700 1 --2- 172.21.15.40:0/1061053172 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3a340a1150 0x7f3a340a3600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00b5c0 tx=0x7f3a3c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:55.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.642+0000 7f3a33fff700 1 -- 172.21.15.40:0/1061053172 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3a44095b70 con 0x7f3a4c103890 2023-07-18T03:28:55.731 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:55 smithi134 ceph-mon[98216]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:55.731 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:55 smithi134 ceph-mon[98216]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:28:55.731 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:55 smithi134 ceph-mon[98216]: Saving service mds.all spec with placement smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2023-07-18T03:28:55.732 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:55.732 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:55.732 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:55.732 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:55.732 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:55.732 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:28:55.732 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:28:55.733 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:55.733 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:55 smithi134 ceph-mon[98216]: Deploying daemon mds.l on smithi134 2023-07-18T03:28:55.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.840+0000 7f3a513bd700 1 -- 172.21.15.40:0/1061053172 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3a4c02c9e0 con 0x7f3a4c103890 2023-07-18T03:28:55.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.841+0000 7f3a33fff700 1 -- 172.21.15.40:0/1061053172 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a440b7e60 con 0x7f3a4c103890 2023-07-18T03:28:55.839 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-07-18T03:28:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.843+0000 7f3a513bd700 1 -- 172.21.15.40:0/1061053172 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3a340a1150 msgr2=0x7f3a340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.843+0000 7f3a513bd700 1 --2- 172.21.15.40:0/1061053172 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3a340a1150 0x7f3a340a3600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00b5c0 tx=0x7f3a3c005fd0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.843+0000 7f3a513bd700 1 -- 172.21.15.40:0/1061053172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a4c103890 msgr2=0x7f3a4c19f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.843+0000 7f3a513bd700 1 --2- 172.21.15.40:0/1061053172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a4c103890 0x7f3a4c19f090 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400b970 tx=0x7f3a4400bc80 comp rx=0 tx=0).stop 2023-07-18T03:28:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.843+0000 7f3a513bd700 1 -- 172.21.15.40:0/1061053172 shutdown_connections 2023-07-18T03:28:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.843+0000 7f3a513bd700 1 --2- 172.21.15.40:0/1061053172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a4c103890 0x7f3a4c19f090 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.843+0000 7f3a513bd700 1 --2- 172.21.15.40:0/1061053172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3a4c101d10 0x7f3a4c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.843+0000 7f3a513bd700 1 --2- 172.21.15.40:0/1061053172 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3a340a1150 0x7f3a340a3600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.843+0000 7f3a513bd700 1 --2- 172.21.15.40:0/1061053172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a4c102f00 0x7f3a4c198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.843+0000 7f3a513bd700 1 -- 172.21.15.40:0/1061053172 >> 172.21.15.40:0/1061053172 conn(0x7f3a4c0fd350 msgr2=0x7f3a4c0ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:55.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.843+0000 7f3a513bd700 1 -- 172.21.15.40:0/1061053172 shutdown_connections 2023-07-18T03:28:55.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.843+0000 7f3a513bd700 1 -- 172.21.15.40:0/1061053172 wait complete. 2023-07-18T03:28:55.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:55 smithi064 ceph-mon[98059]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:55.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:55 smithi064 ceph-mon[98059]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:28:55.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:55 smithi064 ceph-mon[98059]: Saving service mds.all spec with placement smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2023-07-18T03:28:55.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:55.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:28:55.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:55.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:28:55.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:55.853 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:28:55.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:28:55.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:55.854 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:55 smithi064 ceph-mon[98059]: Deploying daemon mds.l on smithi134 2023-07-18T03:28:55.856 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:28:55.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.985+0000 7ff6fc223700 1 -- 172.21.15.40:0/3058241661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f40ffbc0 msgr2=0x7ff6f4100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:55.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.985+0000 7ff6fc223700 1 --2- 172.21.15.40:0/3058241661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f40ffbc0 0x7ff6f4100010 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009990 tx=0x7ff6e8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.985+0000 7ff6fc223700 1 -- 172.21.15.40:0/3058241661 shutdown_connections 2023-07-18T03:28:55.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.985+0000 7ff6fc223700 1 --2- 172.21.15.40:0/3058241661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6f4100550 0x7ff6f4106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.985+0000 7ff6fc223700 1 --2- 172.21.15.40:0/3058241661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f40fe9d0 0x7ff6f40fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.985+0000 7ff6fc223700 1 --2- 172.21.15.40:0/3058241661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f40ffbc0 0x7ff6f4100010 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.985+0000 7ff6fc223700 1 -- 172.21.15.40:0/3058241661 >> 172.21.15.40:0/3058241661 conn(0x7ff6f40f9fd0 msgr2=0x7ff6f40fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:55.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.985+0000 7ff6fc223700 1 -- 172.21.15.40:0/3058241661 shutdown_connections 2023-07-18T03:28:55.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.985+0000 7ff6fc223700 1 -- 172.21.15.40:0/3058241661 wait complete. 2023-07-18T03:28:55.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6fc223700 1 Processor -- start 2023-07-18T03:28:55.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6fc223700 1 -- start start 2023-07-18T03:28:55.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6fc223700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6f40fe9d0 0x7ff6f4194710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:55.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6fc223700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f40ffbc0 0x7ff6f4194c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:55.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6fc223700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f4100550 0x7ff6f419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:55.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6fc223700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f419c290 con 0x7ff6f4100550 2023-07-18T03:28:55.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6fc223700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f419c3d0 con 0x7ff6f40fe9d0 2023-07-18T03:28:55.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6fc223700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f419c560 con 0x7ff6f40ffbc0 2023-07-18T03:28:55.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6fa7c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f4100550 0x7ff6f419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:55.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6f97be700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f40ffbc0 0x7ff6f4194c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:55.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6f9fbf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6f40fe9d0 0x7ff6f4194710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:55.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6fa7c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f4100550 0x7ff6f419bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55790/0 (socket says 172.21.15.40:55790) 2023-07-18T03:28:55.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6f97be700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f40ffbc0 0x7ff6f4194c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:44816/0 (socket says 172.21.15.40:44816) 2023-07-18T03:28:55.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6fa7c0700 1 -- 172.21.15.40:0/2788550550 learned_addr learned my addr 172.21.15.40:0/2788550550 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:28:55.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6fa7c0700 1 -- 172.21.15.40:0/2788550550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f40ffbc0 msgr2=0x7ff6f4194c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:55.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6fa7c0700 1 --2- 172.21.15.40:0/2788550550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f40ffbc0 0x7ff6f4194c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6fa7c0700 1 -- 172.21.15.40:0/2788550550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6f40fe9d0 msgr2=0x7ff6f4194710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:55.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6fa7c0700 1 --2- 172.21.15.40:0/2788550550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6f40fe9d0 0x7ff6f4194710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:55.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6fa7c0700 1 -- 172.21.15.40:0/2788550550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e80095f0 con 0x7ff6f4100550 2023-07-18T03:28:55.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6f97be700 1 --2- 172.21.15.40:0/2788550550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f40ffbc0 0x7ff6f4194c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:28:55.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.986+0000 7ff6fa7c0700 1 --2- 172.21.15.40:0/2788550550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f4100550 0x7ff6f419bcc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00ada0 tx=0x7ff6ec00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:55.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.987+0000 7ff6e6ffd700 1 -- 172.21.15.40:0/2788550550 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec00f070 con 0x7ff6f4100550 2023-07-18T03:28:55.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.987+0000 7ff6fc223700 1 -- 172.21.15.40:0/2788550550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f40676d0 con 0x7ff6f4100550 2023-07-18T03:28:55.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.987+0000 7ff6e6ffd700 1 -- 172.21.15.40:0/2788550550 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6ec007b30 con 0x7ff6f4100550 2023-07-18T03:28:55.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.987+0000 7ff6e6ffd700 1 -- 172.21.15.40:0/2788550550 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec0053e0 con 0x7ff6f4100550 2023-07-18T03:28:55.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.987+0000 7ff6fc223700 1 -- 172.21.15.40:0/2788550550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f4067b90 con 0x7ff6f4100550 2023-07-18T03:28:55.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.987+0000 7ff6fc223700 1 -- 172.21.15.40:0/2788550550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f4067300 con 0x7ff6f4100550 2023-07-18T03:28:55.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.989+0000 7ff6e6ffd700 1 -- 172.21.15.40:0/2788550550 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff6ec00c980 con 0x7ff6f4100550 2023-07-18T03:28:55.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.990+0000 7ff6e6ffd700 1 --2- 172.21.15.40:0/2788550550 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff6e00a11a0 0x7ff6e00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:55.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.990+0000 7ff6f9fbf700 1 --2- 172.21.15.40:0/2788550550 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff6e00a11a0 0x7ff6e00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:55.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.990+0000 7ff6e6ffd700 1 -- 172.21.15.40:0/2788550550 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff6ec0cc140 con 0x7ff6f4100550 2023-07-18T03:28:55.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.990+0000 7ff6f9fbf700 1 --2- 172.21.15.40:0/2788550550 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff6e00a11a0 0x7ff6e00a3650 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40ffa20 tx=0x7ff6f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:55.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:55.993+0000 7ff6e6ffd700 1 -- 172.21.15.40:0/2788550550 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff6ec095d20 con 0x7ff6f4100550 2023-07-18T03:28:56.067 INFO:journalctl@ceph.mds.l.smithi134.stdout:Jul 18 03:28:55 smithi134 systemd[1]: Starting Ceph mds.l for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:28:56.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:56.236+0000 7ff6fc223700 1 -- 172.21.15.40:0/2788550550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6f4105db0 con 0x7ff6f4100550 2023-07-18T03:28:56.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:56.237+0000 7ff6e6ffd700 1 -- 172.21.15.40:0/2788550550 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7ff6ec099340 con 0x7ff6f4100550 2023-07-18T03:28:56.235 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:28:56.235 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-07-18T03:28:56.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:56.239+0000 7ff6fc223700 1 -- 172.21.15.40:0/2788550550 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff6e00a11a0 msgr2=0x7ff6e00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:56.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:56.239+0000 7ff6fc223700 1 --2- 172.21.15.40:0/2788550550 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff6e00a11a0 0x7ff6e00a3650 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40ffa20 tx=0x7ff6f0009260 comp rx=0 tx=0).stop 2023-07-18T03:28:56.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:56.239+0000 7ff6fc223700 1 -- 172.21.15.40:0/2788550550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f4100550 msgr2=0x7ff6f419bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:56.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:56.239+0000 7ff6fc223700 1 --2- 172.21.15.40:0/2788550550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f4100550 0x7ff6f419bcc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00ada0 tx=0x7ff6ec00c330 comp rx=0 tx=0).stop 2023-07-18T03:28:56.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:56.239+0000 7ff6fc223700 1 -- 172.21.15.40:0/2788550550 shutdown_connections 2023-07-18T03:28:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:56.239+0000 7ff6fc223700 1 --2- 172.21.15.40:0/2788550550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f4100550 0x7ff6f419bcc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:56.239+0000 7ff6fc223700 1 --2- 172.21.15.40:0/2788550550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6f40fe9d0 0x7ff6f4194710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:56.239+0000 7ff6fc223700 1 --2- 172.21.15.40:0/2788550550 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff6e00a11a0 0x7ff6e00a3650 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:56.239+0000 7ff6fc223700 1 --2- 172.21.15.40:0/2788550550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6f40ffbc0 0x7ff6f4194c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:56.239+0000 7ff6fc223700 1 -- 172.21.15.40:0/2788550550 >> 172.21.15.40:0/2788550550 conn(0x7ff6f40f9fd0 msgr2=0x7ff6f4103080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:56.239+0000 7ff6fc223700 1 -- 172.21.15.40:0/2788550550 shutdown_connections 2023-07-18T03:28:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:56.239+0000 7ff6fc223700 1 -- 172.21.15.40:0/2788550550 wait complete. 2023-07-18T03:28:56.239 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 1 2023-07-18T03:28:56.520 INFO:journalctl@ceph.mds.l.smithi134.stdout:Jul 18 03:28:56 smithi134 podman[123512]: 2023-07-18T03:28:56.521 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:56 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/862878229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:28:56.521 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:56 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1061053172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:28:56.521 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:56 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2788550550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:28:56.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:56 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/862878229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:28:56.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:56 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1061053172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:28:56.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:56 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2788550550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:28:56.817 INFO:journalctl@ceph.mds.l.smithi134.stdout:Jul 18 03:28:56 smithi134 ceph-mds[123552]: -- 172.21.15.134:0/1724379222 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5581d20671e0 con 0x5581d203cc00 2023-07-18T03:28:56.817 INFO:journalctl@ceph.mds.l.smithi134.stdout:Jul 18 03:28:56 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-l[123528]: starting mds.l at 2023-07-18T03:28:56.817 INFO:journalctl@ceph.mds.l.smithi134.stdout:Jul 18 03:28:56 smithi134 bash[123512]: 7ae02476fcbae3348a478a606943d59e3cafe93ed5102056bb6b5f3a6e9cef91 2023-07-18T03:28:56.817 INFO:journalctl@ceph.mds.l.smithi134.stdout:Jul 18 03:28:56 smithi134 systemd[1]: Started Ceph mds.l for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:28:56.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:56 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/862878229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:28:56.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:56 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1061053172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:28:56.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:56 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2788550550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:28:57.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:57 smithi134 ceph-mon[98216]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:57.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:57 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] up:boot 2023-07-18T03:28:57.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:57 smithi134 ceph-mon[98216]: fsmap 1 up:standby 2023-07-18T03:28:57.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:57 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-18T03:28:57.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:57 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:57.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:57 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:57.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:57 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:57.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:57 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:28:57.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:57 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:28:57.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:57 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:57.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:57 smithi134 ceph-mon[98216]: Deploying daemon mds.k on smithi064 2023-07-18T03:28:57.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:57 smithi040 ceph-mon[96439]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:57 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] up:boot 2023-07-18T03:28:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:57 smithi040 ceph-mon[96439]: fsmap 1 up:standby 2023-07-18T03:28:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:57 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-18T03:28:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:57 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:57 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:57 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:57.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:57 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:28:57.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:57 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:28:57.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:57 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:57.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:57 smithi040 ceph-mon[96439]: Deploying daemon mds.k on smithi064 2023-07-18T03:28:57.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:57 smithi064 ceph-mon[98059]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:57.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:57 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] up:boot 2023-07-18T03:28:57.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:57 smithi064 ceph-mon[98059]: fsmap 1 up:standby 2023-07-18T03:28:57.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:57 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-18T03:28:57.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:57 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:57.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:57 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:57.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:57 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:57.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:57 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:28:57.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:57 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:28:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:57 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:57.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:57 smithi064 ceph-mon[98059]: Deploying daemon mds.k on smithi064 2023-07-18T03:28:58.249 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:28:58.387 INFO:journalctl@ceph.mds.k.smithi064.stdout:Jul 18 03:28:58 smithi064 systemd[1]: Starting Ceph mds.k for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:28:58.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.391+0000 7f35e151b700 1 -- 172.21.15.40:0/1204430612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35dc100db0 msgr2=0x7f35dc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:58.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.391+0000 7f35e151b700 1 --2- 172.21.15.40:0/1204430612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35dc100db0 0x7f35dc101200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0098f0 tx=0x7f35cc009c00 comp rx=0 tx=0).stop 2023-07-18T03:28:58.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.391+0000 7f35e151b700 1 -- 172.21.15.40:0/1204430612 shutdown_connections 2023-07-18T03:28:58.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.391+0000 7f35e151b700 1 --2- 172.21.15.40:0/1204430612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35dc101740 0x7f35dc108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:58.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.391+0000 7f35e151b700 1 --2- 172.21.15.40:0/1204430612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35dc0fdde0 0x7f35dc0fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:58.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.391+0000 7f35e151b700 1 --2- 172.21.15.40:0/1204430612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35dc100db0 0x7f35dc101200 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:58.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.391+0000 7f35e151b700 1 -- 172.21.15.40:0/1204430612 >> 172.21.15.40:0/1204430612 conn(0x7f35dc0f97c0 msgr2=0x7f35dc0fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.391+0000 7f35e151b700 1 -- 172.21.15.40:0/1204430612 shutdown_connections 2023-07-18T03:28:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.391+0000 7f35e151b700 1 -- 172.21.15.40:0/1204430612 wait complete. 2023-07-18T03:28:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35e151b700 1 Processor -- start 2023-07-18T03:28:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35e151b700 1 -- start start 2023-07-18T03:28:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35e151b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35dc0fdde0 0x7f35dc100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35e151b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35dc100db0 0x7f35dc0fed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35e151b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35dc101740 0x7f35dc0ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35e151b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc0ffa40 con 0x7f35dc0fdde0 2023-07-18T03:28:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35e151b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc0ffbb0 con 0x7f35dc101740 2023-07-18T03:28:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35e151b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc19f850 con 0x7f35dc100db0 2023-07-18T03:28:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35daffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35dc0fdde0 0x7f35dc100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35daffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35dc0fdde0 0x7f35dc100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55800/0 (socket says 172.21.15.40:55800) 2023-07-18T03:28:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35daffd700 1 -- 172.21.15.40:0/2283519325 learned_addr learned my addr 172.21.15.40:0/2283519325 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:28:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35db7fe700 1 --2- 172.21.15.40:0/2283519325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35dc101740 0x7f35dc0ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35da7fc700 1 --2- 172.21.15.40:0/2283519325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35dc100db0 0x7f35dc0fed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35daffd700 1 -- 172.21.15.40:0/2283519325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35dc100db0 msgr2=0x7f35dc0fed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35daffd700 1 --2- 172.21.15.40:0/2283519325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35dc100db0 0x7f35dc0fed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35daffd700 1 -- 172.21.15.40:0/2283519325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35dc101740 msgr2=0x7f35dc0ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35daffd700 1 --2- 172.21.15.40:0/2283519325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35dc101740 0x7f35dc0ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35daffd700 1 -- 172.21.15.40:0/2283519325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc009520 con 0x7f35dc0fdde0 2023-07-18T03:28:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.392+0000 7f35da7fc700 1 --2- 172.21.15.40:0/2283519325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35dc100db0 0x7f35dc0fed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:28:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.393+0000 7f35daffd700 1 --2- 172.21.15.40:0/2283519325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35dc0fdde0 0x7f35dc100720 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f35d000d980 tx=0x7f35d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.393+0000 7f35c3fff700 1 -- 172.21.15.40:0/2283519325 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0004020 con 0x7f35dc0fdde0 2023-07-18T03:28:58.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.393+0000 7f35e151b700 1 -- 172.21.15.40:0/2283519325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35dc19fb30 con 0x7f35dc0fdde0 2023-07-18T03:28:58.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.393+0000 7f35c3fff700 1 -- 172.21.15.40:0/2283519325 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35d0009730 con 0x7f35dc0fdde0 2023-07-18T03:28:58.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.393+0000 7f35c3fff700 1 -- 172.21.15.40:0/2283519325 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0003a30 con 0x7f35dc0fdde0 2023-07-18T03:28:58.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.393+0000 7f35e151b700 1 -- 172.21.15.40:0/2283519325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc1a0090 con 0x7f35dc0fdde0 2023-07-18T03:28:58.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.394+0000 7f35e151b700 1 -- 172.21.15.40:0/2283519325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35dc10b2f0 con 0x7f35dc0fdde0 2023-07-18T03:28:58.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.395+0000 7f35c3fff700 1 -- 172.21.15.40:0/2283519325 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f35d00098a0 con 0x7f35dc0fdde0 2023-07-18T03:28:58.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.396+0000 7f35c3fff700 1 --2- 172.21.15.40:0/2283519325 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35c40a11a0 0x7f35c40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:28:58.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.396+0000 7f35da7fc700 1 --2- 172.21.15.40:0/2283519325 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35c40a11a0 0x7f35c40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:28:58.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.396+0000 7f35c3fff700 1 -- 172.21.15.40:0/2283519325 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f35d00cc170 con 0x7f35dc0fdde0 2023-07-18T03:28:58.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.396+0000 7f35da7fc700 1 --2- 172.21.15.40:0/2283519325 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35c40a11a0 0x7f35c40a3650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00c010 tx=0x7f35cc005b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:28:58.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.400+0000 7f35c3fff700 1 -- 172.21.15.40:0/2283519325 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f35d0095d60 con 0x7f35dc0fdde0 2023-07-18T03:28:58.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.644+0000 7f35e151b700 1 -- 172.21.15.40:0/2283519325 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35dc10b4e0 con 0x7f35dc0fdde0 2023-07-18T03:28:58.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.644+0000 7f35c3fff700 1 -- 172.21.15.40:0/2283519325 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7f35d0099380 con 0x7f35dc0fdde0 2023-07-18T03:28:58.642 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:28:58.642 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-18T03:28:58.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.647+0000 7f35e151b700 1 -- 172.21.15.40:0/2283519325 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35c40a11a0 msgr2=0x7f35c40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:58.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.647+0000 7f35e151b700 1 --2- 172.21.15.40:0/2283519325 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35c40a11a0 0x7f35c40a3650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00c010 tx=0x7f35cc005b40 comp rx=0 tx=0).stop 2023-07-18T03:28:58.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.647+0000 7f35e151b700 1 -- 172.21.15.40:0/2283519325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35dc0fdde0 msgr2=0x7f35dc100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:28:58.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.647+0000 7f35e151b700 1 --2- 172.21.15.40:0/2283519325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35dc0fdde0 0x7f35dc100720 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f35d000d980 tx=0x7f35d000dc90 comp rx=0 tx=0).stop 2023-07-18T03:28:58.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.647+0000 7f35e151b700 1 -- 172.21.15.40:0/2283519325 shutdown_connections 2023-07-18T03:28:58.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.647+0000 7f35e151b700 1 --2- 172.21.15.40:0/2283519325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35dc101740 0x7f35dc0ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:58.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.647+0000 7f35e151b700 1 --2- 172.21.15.40:0/2283519325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35dc0fdde0 0x7f35dc100720 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:58.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.647+0000 7f35e151b700 1 --2- 172.21.15.40:0/2283519325 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35c40a11a0 0x7f35c40a3650 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:58.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.647+0000 7f35e151b700 1 --2- 172.21.15.40:0/2283519325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35dc100db0 0x7f35dc0fed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:28:58.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.647+0000 7f35e151b700 1 -- 172.21.15.40:0/2283519325 >> 172.21.15.40:0/2283519325 conn(0x7f35dc0f97c0 msgr2=0x7f35dc0fb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:28:58.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.647+0000 7f35e151b700 1 -- 172.21.15.40:0/2283519325 shutdown_connections 2023-07-18T03:28:58.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:28:58.647+0000 7f35e151b700 1 -- 172.21.15.40:0/2283519325 wait complete. 2023-07-18T03:28:58.646 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 2 2023-07-18T03:28:58.711 INFO:journalctl@ceph.mds.k.smithi064.stdout:Jul 18 03:28:58 smithi064 podman[122379]: 2023-07-18T03:28:58.712 INFO:journalctl@ceph.mds.k.smithi064.stdout:Jul 18 03:28:58 smithi064 bash[122379]: c856b53d03adce109f1f45785edc4ca381b02a5a7485491348b6acb903318f68 2023-07-18T03:28:58.712 INFO:journalctl@ceph.mds.k.smithi064.stdout:Jul 18 03:28:58 smithi064 ceph-mds[122420]: -- 172.21.15.64:0/2445067445 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5560db6a71e0 con 0x5560db67cc00 2023-07-18T03:28:58.712 INFO:journalctl@ceph.mds.k.smithi064.stdout:Jul 18 03:28:58 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-k[122395]: starting mds.k at 2023-07-18T03:28:59.100 INFO:journalctl@ceph.mds.k.smithi064.stdout:Jul 18 03:28:58 smithi064 systemd[1]: Started Ceph mds.k for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:28:59.678 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:59 smithi040 ceph-mon[96439]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:59.678 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:59 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2283519325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:28:59.678 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:59.678 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:59.679 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:59.679 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:28:59.679 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:28:59.679 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:28:59 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:59.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:59 smithi134 ceph-mon[98216]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:59.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:59 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2283519325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:28:59.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:59.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:59.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:59.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:28:59.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:28:59.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:28:59 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:28:59.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:59 smithi064 ceph-mon[98059]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:28:59.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:59 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2283519325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:28:59.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:59.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:59.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:28:59.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:28:59.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:28:59.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:28:59 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:00.276 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jul 18 03:29:00 smithi040 systemd[1]: Starting Ceph mds.j for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:29:00.657 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:29:00.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.795+0000 7f149e8bb700 1 -- 172.21.15.40:0/3937103764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1498101680 msgr2=0x7f1498107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:00.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.795+0000 7f149e8bb700 1 --2- 172.21.15.40:0/3937103764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1498101680 0x7f1498107f40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f149000b0a0 tx=0x7f149000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:00.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.795+0000 7f149e8bb700 1 -- 172.21.15.40:0/3937103764 shutdown_connections 2023-07-18T03:29:00.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.795+0000 7f149e8bb700 1 --2- 172.21.15.40:0/3937103764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1498101680 0x7f1498107f40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:00.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.795+0000 7f149e8bb700 1 --2- 172.21.15.40:0/3937103764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14980ffb00 0x7f14980fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:00.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.795+0000 7f149e8bb700 1 --2- 172.21.15.40:0/3937103764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1498100cf0 0x7f1498101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:00.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.795+0000 7f149e8bb700 1 -- 172.21.15.40:0/3937103764 >> 172.21.15.40:0/3937103764 conn(0x7f14980fb100 msgr2=0x7f14980fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:00.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.795+0000 7f149e8bb700 1 -- 172.21.15.40:0/3937103764 shutdown_connections 2023-07-18T03:29:00.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.795+0000 7f149e8bb700 1 -- 172.21.15.40:0/3937103764 wait complete. 2023-07-18T03:29:00.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.795+0000 7f149e8bb700 1 Processor -- start 2023-07-18T03:29:00.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.795+0000 7f149e8bb700 1 -- start start 2023-07-18T03:29:00.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f149e8bb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14980ffb00 0x7f14980742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:00.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f149e8bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1498100cf0 0x7f1498072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:00.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f149e8bb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1498101680 0x7f1498072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:00.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f149e8bb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14980735e0 con 0x7f1498100cf0 2023-07-18T03:29:00.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f149e8bb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1498073750 con 0x7f1498101680 2023-07-18T03:29:00.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f149e8bb700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149819f960 con 0x7f14980ffb00 2023-07-18T03:29:00.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f148ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1498100cf0 0x7f1498072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:00.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f148ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1498100cf0 0x7f1498072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55816/0 (socket says 172.21.15.40:55816) 2023-07-18T03:29:00.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f148ffff700 1 -- 172.21.15.40:0/3438738545 learned_addr learned my addr 172.21.15.40:0/3438738545 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:00.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f1497fff700 1 --2- 172.21.15.40:0/3438738545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14980ffb00 0x7f14980742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:00.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f149ce58700 1 --2- 172.21.15.40:0/3438738545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1498101680 0x7f1498072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:00.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f148ffff700 1 -- 172.21.15.40:0/3438738545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14980ffb00 msgr2=0x7f14980742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:00.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f148ffff700 1 --2- 172.21.15.40:0/3438738545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14980ffb00 0x7f14980742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:00.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f148ffff700 1 -- 172.21.15.40:0/3438738545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1498101680 msgr2=0x7f1498072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:00.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f148ffff700 1 --2- 172.21.15.40:0/3438738545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1498101680 0x7f1498072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:00.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f148ffff700 1 -- 172.21.15.40:0/3438738545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1490009d00 con 0x7f1498100cf0 2023-07-18T03:29:00.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f148ffff700 1 --2- 172.21.15.40:0/3438738545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1498100cf0 0x7f1498072900 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f148000d950 tx=0x7f148000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:00.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f1495ffb700 1 -- 172.21.15.40:0/3438738545 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480009940 con 0x7f1498100cf0 2023-07-18T03:29:00.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f1495ffb700 1 -- 172.21.15.40:0/3438738545 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1480010460 con 0x7f1498100cf0 2023-07-18T03:29:00.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f149e8bb700 1 -- 172.21.15.40:0/3438738545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149819fc40 con 0x7f1498100cf0 2023-07-18T03:29:00.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f1495ffb700 1 -- 172.21.15.40:0/3438738545 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148000f620 con 0x7f1498100cf0 2023-07-18T03:29:00.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.796+0000 7f149e8bb700 1 -- 172.21.15.40:0/3438738545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14981a0150 con 0x7f1498100cf0 2023-07-18T03:29:00.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.797+0000 7f149e8bb700 1 -- 172.21.15.40:0/3438738545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f149810b330 con 0x7f1498100cf0 2023-07-18T03:29:00.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.798+0000 7f1495ffb700 1 -- 172.21.15.40:0/3438738545 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f14800108d0 con 0x7f1498100cf0 2023-07-18T03:29:00.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.799+0000 7f1495ffb700 1 --2- 172.21.15.40:0/3438738545 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14780a11a0 0x7f14780a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:00.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.799+0000 7f1497fff700 1 --2- 172.21.15.40:0/3438738545 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14780a11a0 0x7f14780a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:00.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.799+0000 7f1495ffb700 1 -- 172.21.15.40:0/3438738545 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f14800cc2f0 con 0x7f1498100cf0 2023-07-18T03:29:00.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.799+0000 7f1497fff700 1 --2- 172.21.15.40:0/3438738545 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14780a11a0 0x7f14780a3650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1498100b50 tx=0x7f148800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:00.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:00.803+0000 7f1495ffb700 1 -- 172.21.15.40:0/3438738545 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1480095ee0 con 0x7f1498100cf0 2023-07-18T03:29:00.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:00 smithi134 ceph-mon[98216]: Deploying daemon mds.j on smithi040 2023-07-18T03:29:00.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:00 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.64:6832/646383396,v1:172.21.15.64:6833/646383396] up:boot 2023-07-18T03:29:00.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:00 smithi134 ceph-mon[98216]: fsmap 2 up:standby 2023-07-18T03:29:00.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:00 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-18T03:29:00.818 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:00 smithi040 ceph-mon[96439]: Deploying daemon mds.j on smithi040 2023-07-18T03:29:00.818 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:00 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.64:6832/646383396,v1:172.21.15.64:6833/646383396] up:boot 2023-07-18T03:29:00.818 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:00 smithi040 ceph-mon[96439]: fsmap 2 up:standby 2023-07-18T03:29:00.818 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:00 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-18T03:29:00.819 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jul 18 03:29:00 smithi040 podman[129843]: 2023-07-18T03:29:00.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:00 smithi064 ceph-mon[98059]: Deploying daemon mds.j on smithi040 2023-07-18T03:29:00.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:00 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.64:6832/646383396,v1:172.21.15.64:6833/646383396] up:boot 2023-07-18T03:29:00.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:00 smithi064 ceph-mon[98059]: fsmap 2 up:standby 2023-07-18T03:29:00.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:00 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-18T03:29:01.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:01.063+0000 7f149e8bb700 1 -- 172.21.15.40:0/3438738545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1498106930 con 0x7f1498100cf0 2023-07-18T03:29:01.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:01.064+0000 7f1495ffb700 1 -- 172.21.15.40:0/3438738545 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2014 (secure 0 0 0) 0x7f1480099500 con 0x7f1498100cf0 2023-07-18T03:29:01.062 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:01.062 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-18T03:29:01.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:01.066+0000 7f149e8bb700 1 -- 172.21.15.40:0/3438738545 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14780a11a0 msgr2=0x7f14780a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:01.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:01.066+0000 7f149e8bb700 1 --2- 172.21.15.40:0/3438738545 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14780a11a0 0x7f14780a3650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1498100b50 tx=0x7f148800b410 comp rx=0 tx=0).stop 2023-07-18T03:29:01.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:01.066+0000 7f149e8bb700 1 -- 172.21.15.40:0/3438738545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1498100cf0 msgr2=0x7f1498072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:01.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:01.066+0000 7f149e8bb700 1 --2- 172.21.15.40:0/3438738545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1498100cf0 0x7f1498072900 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f148000d950 tx=0x7f148000dc60 comp rx=0 tx=0).stop 2023-07-18T03:29:01.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:01.066+0000 7f149e8bb700 1 -- 172.21.15.40:0/3438738545 shutdown_connections 2023-07-18T03:29:01.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:01.066+0000 7f149e8bb700 1 --2- 172.21.15.40:0/3438738545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1498101680 0x7f1498072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:01.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:01.066+0000 7f149e8bb700 1 --2- 172.21.15.40:0/3438738545 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14780a11a0 0x7f14780a3650 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:01.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:01.066+0000 7f149e8bb700 1 --2- 172.21.15.40:0/3438738545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14980ffb00 0x7f14980742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:01.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:01.067+0000 7f149e8bb700 1 --2- 172.21.15.40:0/3438738545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1498100cf0 0x7f1498072900 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:01.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:01.067+0000 7f149e8bb700 1 -- 172.21.15.40:0/3438738545 >> 172.21.15.40:0/3438738545 conn(0x7f14980fb100 msgr2=0x7f14980fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:01.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:01.067+0000 7f149e8bb700 1 -- 172.21.15.40:0/3438738545 shutdown_connections 2023-07-18T03:29:01.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:01.067+0000 7f149e8bb700 1 -- 172.21.15.40:0/3438738545 wait complete. 2023-07-18T03:29:01.066 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 3 2023-07-18T03:29:01.076 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jul 18 03:29:00 smithi040 bash[129843]: f532e0aed9aaaa707a3e94752f99165f2e22f43edce81b1523f2cf47ef4fea0c 2023-07-18T03:29:01.076 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jul 18 03:29:00 smithi040 ceph-mds[129926]: -- 172.21.15.40:0/1771393245 <== mon.0 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b9cf21f1e0 con 0x55b9cf1f4c00 2023-07-18T03:29:01.076 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jul 18 03:29:00 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-j[129859]: starting mds.j at 2023-07-18T03:29:01.077 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jul 18 03:29:00 smithi040 systemd[1]: Started Ceph mds.j for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:29:01.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:01 smithi134 ceph-mon[98216]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:01.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:01 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:01.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:01 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:01.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:01 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:01.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:01 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:01.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:01 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:01.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:01 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:01.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:01 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3438738545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:01.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:01 smithi040 ceph-mon[96439]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:01.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:01 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:01.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:01 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:01.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:01 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:01.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:01 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:01.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:01 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:01.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:01 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:01.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:01 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3438738545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:01.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:01 smithi064 ceph-mon[98059]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:01.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:01 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:01.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:01 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:01.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:01 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:01.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:01 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:01.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:01 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:01.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:01 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:01.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:01 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3438738545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:02.565 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jul 18 03:29:02 smithi040 systemd[1]: Starting Ceph mds.d for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:29:02.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:02 smithi134 ceph-mon[98216]: Deploying daemon mds.d on smithi040 2023-07-18T03:29:02.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:02 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.40:6834/2289812880,v1:172.21.15.40:6835/2289812880] up:boot 2023-07-18T03:29:02.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:02 smithi134 ceph-mon[98216]: fsmap 3 up:standby 2023-07-18T03:29:02.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-18T03:29:02.829 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jul 18 03:29:02 smithi040 podman[130227]: 2023-07-18T03:29:02.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:02 smithi040 ceph-mon[96439]: Deploying daemon mds.d on smithi040 2023-07-18T03:29:02.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:02 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.40:6834/2289812880,v1:172.21.15.40:6835/2289812880] up:boot 2023-07-18T03:29:02.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:02 smithi040 ceph-mon[96439]: fsmap 3 up:standby 2023-07-18T03:29:02.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-18T03:29:02.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:02 smithi064 ceph-mon[98059]: Deploying daemon mds.d on smithi040 2023-07-18T03:29:02.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:02 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.40:6834/2289812880,v1:172.21.15.40:6835/2289812880] up:boot 2023-07-18T03:29:02.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:02 smithi064 ceph-mon[98059]: fsmap 3 up:standby 2023-07-18T03:29:02.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-18T03:29:03.079 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:29:03.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.212+0000 7f2c857b8700 1 -- 172.21.15.40:0/3241407786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c80100cd0 msgr2=0x7f2c80101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:03.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.212+0000 7f2c857b8700 1 --2- 172.21.15.40:0/3241407786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c80100cd0 0x7f2c80101120 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009990 tx=0x7f2c68009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:03.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.212+0000 7f2c857b8700 1 -- 172.21.15.40:0/3241407786 shutdown_connections 2023-07-18T03:29:03.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.212+0000 7f2c857b8700 1 --2- 172.21.15.40:0/3241407786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c80101660 0x7f2c80107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:03.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.212+0000 7f2c857b8700 1 --2- 172.21.15.40:0/3241407786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c800ffae0 0x7f2c800ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:03.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.212+0000 7f2c857b8700 1 --2- 172.21.15.40:0/3241407786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c80100cd0 0x7f2c80101120 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:03.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.212+0000 7f2c857b8700 1 -- 172.21.15.40:0/3241407786 >> 172.21.15.40:0/3241407786 conn(0x7f2c800fb100 msgr2=0x7f2c800fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:03.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.213+0000 7f2c857b8700 1 -- 172.21.15.40:0/3241407786 shutdown_connections 2023-07-18T03:29:03.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.213+0000 7f2c857b8700 1 -- 172.21.15.40:0/3241407786 wait complete. 2023-07-18T03:29:03.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.213+0000 7f2c857b8700 1 Processor -- start 2023-07-18T03:29:03.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.213+0000 7f2c857b8700 1 -- start start 2023-07-18T03:29:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.213+0000 7f2c857b8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c800ffae0 0x7f2c801957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.213+0000 7f2c857b8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c80100cd0 0x7f2c80195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.213+0000 7f2c857b8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c80101660 0x7f2c8019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.213+0000 7f2c857b8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8019d360 con 0x7f2c80100cd0 2023-07-18T03:29:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.213+0000 7f2c857b8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8019d4a0 con 0x7f2c80101660 2023-07-18T03:29:03.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.213+0000 7f2c857b8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8019d630 con 0x7f2c800ffae0 2023-07-18T03:29:03.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c7e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c80100cd0 0x7f2c80195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:03.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c7e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c80100cd0 0x7f2c80195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55852/0 (socket says 172.21.15.40:55852) 2023-07-18T03:29:03.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c7e7fc700 1 -- 172.21.15.40:0/990369501 learned_addr learned my addr 172.21.15.40:0/990369501 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:03.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c7f7fe700 1 --2- 172.21.15.40:0/990369501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c80101660 0x7f2c8019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:03.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c7effd700 1 --2- 172.21.15.40:0/990369501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c800ffae0 0x7f2c801957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:03.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c7e7fc700 1 -- 172.21.15.40:0/990369501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c800ffae0 msgr2=0x7f2c801957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:03.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c7e7fc700 1 --2- 172.21.15.40:0/990369501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c800ffae0 0x7f2c801957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:03.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c7e7fc700 1 -- 172.21.15.40:0/990369501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c80101660 msgr2=0x7f2c8019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c7e7fc700 1 --2- 172.21.15.40:0/990369501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c80101660 0x7f2c8019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c7e7fc700 1 -- 172.21.15.40:0/990369501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c680095f0 con 0x7f2c80100cd0 2023-07-18T03:29:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c7f7fe700 1 --2- 172.21.15.40:0/990369501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c80101660 0x7f2c8019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c7effd700 1 --2- 172.21.15.40:0/990369501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c800ffae0 0x7f2c801957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:29:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c7e7fc700 1 --2- 172.21.15.40:0/990369501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c80100cd0 0x7f2c80195d20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800c010 tx=0x7f2c68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c77fff700 1 -- 172.21.15.40:0/990369501 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6801d070 con 0x7f2c80100cd0 2023-07-18T03:29:03.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c857b8700 1 -- 172.21.15.40:0/990369501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8019d8b0 con 0x7f2c80100cd0 2023-07-18T03:29:03.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c77fff700 1 -- 172.21.15.40:0/990369501 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c680054f0 con 0x7f2c80100cd0 2023-07-18T03:29:03.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c77fff700 1 -- 172.21.15.40:0/990369501 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68004ca0 con 0x7f2c80100cd0 2023-07-18T03:29:03.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.214+0000 7f2c857b8700 1 -- 172.21.15.40:0/990369501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8019dcd0 con 0x7f2c80100cd0 2023-07-18T03:29:03.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.215+0000 7f2c857b8700 1 -- 172.21.15.40:0/990369501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8010b300 con 0x7f2c80100cd0 2023-07-18T03:29:03.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.216+0000 7f2c77fff700 1 -- 172.21.15.40:0/990369501 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2c6800fbc0 con 0x7f2c80100cd0 2023-07-18T03:29:03.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.217+0000 7f2c77fff700 1 --2- 172.21.15.40:0/990369501 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2c6c0a11a0 0x7f2c6c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:03.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.217+0000 7f2c77fff700 1 -- 172.21.15.40:0/990369501 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2c680cdc90 con 0x7f2c80100cd0 2023-07-18T03:29:03.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.217+0000 7f2c7effd700 1 --2- 172.21.15.40:0/990369501 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2c6c0a11a0 0x7f2c6c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:03.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.218+0000 7f2c7effd700 1 --2- 172.21.15.40:0/990369501 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2c6c0a11a0 0x7f2c6c0a3650 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2c80100b30 tx=0x7f2c70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:03.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.220+0000 7f2c77fff700 1 -- 172.21.15.40:0/990369501 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2c68097870 con 0x7f2c80100cd0 2023-07-18T03:29:03.329 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jul 18 03:29:02 smithi040 bash[130227]: fbf659c313e7f49feb0985a97bd980e8d9852ace475c12e81f5d491f86a06044 2023-07-18T03:29:03.329 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jul 18 03:29:02 smithi040 ceph-mds[130267]: -- 172.21.15.40:0/3902102959 <== mon.0 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a9c0a451e0 con 0x55a9c0a1ac00 2023-07-18T03:29:03.330 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jul 18 03:29:02 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-d[130243]: starting mds.d at 2023-07-18T03:29:03.330 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jul 18 03:29:02 smithi040 systemd[1]: Started Ceph mds.d for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:29:03.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.468+0000 7f2c857b8700 1 -- 172.21.15.40:0/990369501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c8019e3d0 con 0x7f2c80100cd0 2023-07-18T03:29:03.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.468+0000 7f2c77fff700 1 -- 172.21.15.40:0/990369501 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2771 (secure 0 0 0) 0x7f2c6809ae90 con 0x7f2c80100cd0 2023-07-18T03:29:03.466 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:03.466 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-18T03:29:03.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.470+0000 7f2c857b8700 1 -- 172.21.15.40:0/990369501 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2c6c0a11a0 msgr2=0x7f2c6c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:03.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.470+0000 7f2c857b8700 1 --2- 172.21.15.40:0/990369501 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2c6c0a11a0 0x7f2c6c0a3650 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2c80100b30 tx=0x7f2c70009260 comp rx=0 tx=0).stop 2023-07-18T03:29:03.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.470+0000 7f2c857b8700 1 -- 172.21.15.40:0/990369501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c80100cd0 msgr2=0x7f2c80195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:03.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.470+0000 7f2c857b8700 1 --2- 172.21.15.40:0/990369501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c80100cd0 0x7f2c80195d20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800c010 tx=0x7f2c68004730 comp rx=0 tx=0).stop 2023-07-18T03:29:03.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.470+0000 7f2c857b8700 1 -- 172.21.15.40:0/990369501 shutdown_connections 2023-07-18T03:29:03.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.470+0000 7f2c857b8700 1 --2- 172.21.15.40:0/990369501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c80101660 0x7f2c8019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:03.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.470+0000 7f2c857b8700 1 --2- 172.21.15.40:0/990369501 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2c6c0a11a0 0x7f2c6c0a3650 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:03.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.470+0000 7f2c857b8700 1 --2- 172.21.15.40:0/990369501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c800ffae0 0x7f2c801957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:03.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.470+0000 7f2c857b8700 1 --2- 172.21.15.40:0/990369501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c80100cd0 0x7f2c80195d20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:03.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.470+0000 7f2c857b8700 1 -- 172.21.15.40:0/990369501 >> 172.21.15.40:0/990369501 conn(0x7f2c800fb100 msgr2=0x7f2c800fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:03.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.470+0000 7f2c857b8700 1 -- 172.21.15.40:0/990369501 shutdown_connections 2023-07-18T03:29:03.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:03.470+0000 7f2c857b8700 1 -- 172.21.15.40:0/990369501 wait complete. 2023-07-18T03:29:03.470 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 4 2023-07-18T03:29:03.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:03 smithi040 ceph-mon[96439]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:03.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:03.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:03.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:03.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:03.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:03.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:03.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:03 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/990369501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:03.844 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:03 smithi064 ceph-mon[98059]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:03.844 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:03 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:03.844 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:03 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:03.844 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:03 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:03.844 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:03 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:03.844 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:03 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:03.845 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:03 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:03.845 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:03 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/990369501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:04.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:03 smithi134 ceph-mon[98216]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:04.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:03 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:04.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:03 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:04.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:03 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:04.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:03 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:04.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:03 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:04.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:03 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:04.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:03 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/990369501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:04.548 INFO:journalctl@ceph.mds.e.smithi064.stdout:Jul 18 03:29:04 smithi064 systemd[1]: Starting Ceph mds.e for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:29:04.819 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:04 smithi064 ceph-mon[98059]: Deploying daemon mds.e on smithi064 2023-07-18T03:29:04.819 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:04 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.40:6836/3377503144,v1:172.21.15.40:6837/3377503144] up:boot 2023-07-18T03:29:04.819 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:04 smithi064 ceph-mon[98059]: fsmap 4 up:standby 2023-07-18T03:29:04.820 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:04 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-18T03:29:05.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:04 smithi134 ceph-mon[98216]: Deploying daemon mds.e on smithi064 2023-07-18T03:29:05.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:04 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.40:6836/3377503144,v1:172.21.15.40:6837/3377503144] up:boot 2023-07-18T03:29:05.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:04 smithi134 ceph-mon[98216]: fsmap 4 up:standby 2023-07-18T03:29:05.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:04 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-18T03:29:05.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:04 smithi040 ceph-mon[96439]: Deploying daemon mds.e on smithi064 2023-07-18T03:29:05.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:04 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.40:6836/3377503144,v1:172.21.15.40:6837/3377503144] up:boot 2023-07-18T03:29:05.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:04 smithi040 ceph-mon[96439]: fsmap 4 up:standby 2023-07-18T03:29:05.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:04 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-18T03:29:05.101 INFO:journalctl@ceph.mds.e.smithi064.stdout:Jul 18 03:29:04 smithi064 podman[122722]: 2023-07-18T03:29:05.101 INFO:journalctl@ceph.mds.e.smithi064.stdout:Jul 18 03:29:05 smithi064 bash[122722]: d28733e6182f6f66b2b4ec1a8632a96bbbf0c3a4397ac3059eb22e64bd9d5d32 2023-07-18T03:29:05.101 INFO:journalctl@ceph.mds.e.smithi064.stdout:Jul 18 03:29:05 smithi064 ceph-mds[122762]: -- 172.21.15.64:0/3405867495 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560fcbec51e0 con 0x560fcbe9b000 2023-07-18T03:29:05.101 INFO:journalctl@ceph.mds.e.smithi064.stdout:Jul 18 03:29:05 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-e[122738]: starting mds.e at 2023-07-18T03:29:05.481 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:29:05.601 INFO:journalctl@ceph.mds.e.smithi064.stdout:Jul 18 03:29:05 smithi064 systemd[1]: Started Ceph mds.e for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:29:05.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.625+0000 7f25d8179700 1 -- 172.21.15.40:0/1364171244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d00fea00 msgr2=0x7f25d00fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:05.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.625+0000 7f25d8179700 1 --2- 172.21.15.40:0/1364171244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d00fea00 0x7f25d00fee10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009990 tx=0x7f25cc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:05.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.625+0000 7f25d8179700 1 -- 172.21.15.40:0/1364171244 shutdown_connections 2023-07-18T03:29:05.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.625+0000 7f25d8179700 1 --2- 172.21.15.40:0/1364171244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25d0100580 0x7f25d0106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:05.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.625+0000 7f25d8179700 1 --2- 172.21.15.40:0/1364171244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d00fea00 0x7f25d00fee10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.625+0000 7f25d8179700 1 --2- 172.21.15.40:0/1364171244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25d00ffbf0 0x7f25d0100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.625+0000 7f25d8179700 1 -- 172.21.15.40:0/1364171244 >> 172.21.15.40:0/1364171244 conn(0x7f25d00fa020 msgr2=0x7f25d00fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.625+0000 7f25d8179700 1 -- 172.21.15.40:0/1364171244 shutdown_connections 2023-07-18T03:29:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.625+0000 7f25d8179700 1 -- 172.21.15.40:0/1364171244 wait complete. 2023-07-18T03:29:05.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.626+0000 7f25d8179700 1 Processor -- start 2023-07-18T03:29:05.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.626+0000 7f25d8179700 1 -- start start 2023-07-18T03:29:05.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d8179700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d00fea00 0x7f25d0195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:05.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d8179700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25d00ffbf0 0x7f25d0195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:05.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d8179700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25d0100580 0x7f25d019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:05.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d8179700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d019d380 con 0x7f25d00fea00 2023-07-18T03:29:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d8179700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d019d4c0 con 0x7f25d00ffbf0 2023-07-18T03:29:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d8179700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d019d650 con 0x7f25d0100580 2023-07-18T03:29:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d5f15700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d00fea00 0x7f25d0195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d5f15700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d00fea00 0x7f25d0195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55866/0 (socket says 172.21.15.40:55866) 2023-07-18T03:29:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d5f15700 1 -- 172.21.15.40:0/1660745278 learned_addr learned my addr 172.21.15.40:0/1660745278 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d5714700 1 --2- 172.21.15.40:0/1660745278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25d00ffbf0 0x7f25d0195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:05.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d5f15700 1 -- 172.21.15.40:0/1660745278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25d0100580 msgr2=0x7f25d019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d5f15700 1 --2- 172.21.15.40:0/1660745278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25d0100580 0x7f25d019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d5f15700 1 -- 172.21.15.40:0/1660745278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25d00ffbf0 msgr2=0x7f25d0195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d5f15700 1 --2- 172.21.15.40:0/1660745278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25d00ffbf0 0x7f25d0195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d5f15700 1 -- 172.21.15.40:0/1660745278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc0095f0 con 0x7f25d00fea00 2023-07-18T03:29:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d5f15700 1 --2- 172.21.15.40:0/1660745278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d00fea00 0x7f25d0195800 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f25cc004760 tx=0x7f25cc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25c6ffd700 1 -- 172.21.15.40:0/1660745278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc01d070 con 0x7f25d00fea00 2023-07-18T03:29:05.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25c6ffd700 1 -- 172.21.15.40:0/1660745278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25cc004cd0 con 0x7f25d00fea00 2023-07-18T03:29:05.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d8179700 1 -- 172.21.15.40:0/1660745278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d019d8d0 con 0x7f25d00fea00 2023-07-18T03:29:05.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25c6ffd700 1 -- 172.21.15.40:0/1660745278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc005290 con 0x7f25d00fea00 2023-07-18T03:29:05.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.627+0000 7f25d8179700 1 -- 172.21.15.40:0/1660745278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d019dd50 con 0x7f25d00fea00 2023-07-18T03:29:05.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.628+0000 7f25d8179700 1 -- 172.21.15.40:0/1660745278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d0105ef0 con 0x7f25d00fea00 2023-07-18T03:29:05.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.630+0000 7f25c6ffd700 1 -- 172.21.15.40:0/1660745278 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f25cc003b60 con 0x7f25d00fea00 2023-07-18T03:29:05.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.631+0000 7f25c6ffd700 1 --2- 172.21.15.40:0/1660745278 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f25bc0a1220 0x7f25bc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:05.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.631+0000 7f25d5714700 1 --2- 172.21.15.40:0/1660745278 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f25bc0a1220 0x7f25bc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:05.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.631+0000 7f25c6ffd700 1 -- 172.21.15.40:0/1660745278 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f25cc0cd7d0 con 0x7f25d00fea00 2023-07-18T03:29:05.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.631+0000 7f25d5714700 1 --2- 172.21.15.40:0/1660745278 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f25bc0a1220 0x7f25bc0a36d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f25c0005950 tx=0x7f25c000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:05.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.634+0000 7f25c6ffd700 1 -- 172.21.15.40:0/1660745278 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f25cc097450 con 0x7f25d00fea00 2023-07-18T03:29:05.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:05 smithi040 ceph-mon[96439]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:05.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:05 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:05.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:05 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:05.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:05 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:05.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:05 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:05.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:05 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:05.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:05 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:05.869 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:05 smithi134 ceph-mon[98216]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:05.869 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:05 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:05.869 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:05 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:05.869 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:05 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:05.869 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:05 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:05.869 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:05 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:05.870 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:05 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:05.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.882+0000 7f25d8179700 1 -- 172.21.15.40:0/1660745278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25d002c850 con 0x7f25d00fea00 2023-07-18T03:29:05.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.882+0000 7f25c6ffd700 1 -- 172.21.15.40:0/1660745278 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4285 (secure 0 0 0) 0x7f25cc09aa70 con 0x7f25d00fea00 2023-07-18T03:29:05.880 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:05.880 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-18T03:29:05.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.884+0000 7f25d8179700 1 -- 172.21.15.40:0/1660745278 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f25bc0a1220 msgr2=0x7f25bc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:05.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.884+0000 7f25d8179700 1 --2- 172.21.15.40:0/1660745278 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f25bc0a1220 0x7f25bc0a36d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f25c0005950 tx=0x7f25c000b410 comp rx=0 tx=0).stop 2023-07-18T03:29:05.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.884+0000 7f25d8179700 1 -- 172.21.15.40:0/1660745278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d00fea00 msgr2=0x7f25d0195800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:05.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.884+0000 7f25d8179700 1 --2- 172.21.15.40:0/1660745278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d00fea00 0x7f25d0195800 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f25cc004760 tx=0x7f25cc009580 comp rx=0 tx=0).stop 2023-07-18T03:29:05.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.885+0000 7f25d8179700 1 -- 172.21.15.40:0/1660745278 shutdown_connections 2023-07-18T03:29:05.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.885+0000 7f25d8179700 1 --2- 172.21.15.40:0/1660745278 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f25bc0a1220 0x7f25bc0a36d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:05.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.885+0000 7f25d8179700 1 --2- 172.21.15.40:0/1660745278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25d0100580 0x7f25d019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:05.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.885+0000 7f25d8179700 1 --2- 172.21.15.40:0/1660745278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d00fea00 0x7f25d0195800 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:05.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.885+0000 7f25d8179700 1 --2- 172.21.15.40:0/1660745278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25d00ffbf0 0x7f25d0195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:05.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.885+0000 7f25d8179700 1 -- 172.21.15.40:0/1660745278 >> 172.21.15.40:0/1660745278 conn(0x7f25d00fa020 msgr2=0x7f25d0103030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:05.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.885+0000 7f25d8179700 1 -- 172.21.15.40:0/1660745278 shutdown_connections 2023-07-18T03:29:05.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:05.885+0000 7f25d8179700 1 -- 172.21.15.40:0/1660745278 wait complete. 2023-07-18T03:29:05.884 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 6 2023-07-18T03:29:06.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:05 smithi064 ceph-mon[98059]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:06.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:05 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:06.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:05 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:06.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:05 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:06.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:05 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:05 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:06.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:05 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:06.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:06 smithi134 ceph-mon[98216]: Deploying daemon mds.f on smithi134 2023-07-18T03:29:06.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:06 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] up:boot 2023-07-18T03:29:06.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:06 smithi134 ceph-mon[98216]: fsmap 5 up:standby 2023-07-18T03:29:06.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:06 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-18T03:29:06.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:06 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1660745278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:06.818 INFO:journalctl@ceph.mds.f.smithi134.stdout:Jul 18 03:29:06 smithi134 systemd[1]: Starting Ceph mds.f for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:29:07.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:06 smithi040 ceph-mon[96439]: Deploying daemon mds.f on smithi134 2023-07-18T03:29:07.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:06 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] up:boot 2023-07-18T03:29:07.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:06 smithi040 ceph-mon[96439]: fsmap 5 up:standby 2023-07-18T03:29:07.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:06 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-18T03:29:07.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:06 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1660745278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:07.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:06 smithi064 ceph-mon[98059]: Deploying daemon mds.f on smithi134 2023-07-18T03:29:07.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:06 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] up:boot 2023-07-18T03:29:07.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:06 smithi064 ceph-mon[98059]: fsmap 5 up:standby 2023-07-18T03:29:07.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:06 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-18T03:29:07.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:06 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1660745278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:07.310 INFO:journalctl@ceph.mds.f.smithi134.stdout:Jul 18 03:29:07 smithi134 podman[123853]: 2023-07-18T03:29:07.817 INFO:journalctl@ceph.mds.f.smithi134.stdout:Jul 18 03:29:07 smithi134 bash[123853]: 842d678a97b31d672caf3fc47bfd177148e480b191ae1694760d02feb755db1d 2023-07-18T03:29:07.817 INFO:journalctl@ceph.mds.f.smithi134.stdout:Jul 18 03:29:07 smithi134 ceph-mds[123893]: -- 172.21.15.134:0/886961308 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fd0145d1e0 con 0x55fd01433000 2023-07-18T03:29:07.817 INFO:journalctl@ceph.mds.f.smithi134.stdout:Jul 18 03:29:07 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-f[123869]: starting mds.f at 2023-07-18T03:29:07.817 INFO:journalctl@ceph.mds.f.smithi134.stdout:Jul 18 03:29:07 smithi134 systemd[1]: Started Ceph mds.f for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:29:07.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:07 smithi134 ceph-mon[98216]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:07.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:07 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:07.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:07 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:07.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:07 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:07.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:07 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:07.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:07 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:07.819 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:07 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:07.894 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:29:07.924 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:07 smithi040 ceph-mon[96439]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:07.924 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:07 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:07.924 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:07 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:07.925 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:07 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:07.925 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:07 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:07.925 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:07 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:07.925 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:07 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:08.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.037+0000 7f61a1e06700 1 -- 172.21.15.40:0/3826460389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619c0ff340 msgr2=0x7f619c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:08.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.037+0000 7f61a1e06700 1 --2- 172.21.15.40:0/3826460389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619c0ff340 0x7f619c0ff790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6184009990 tx=0x7f6184009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:08.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.037+0000 7f61a1e06700 1 -- 172.21.15.40:0/3826460389 shutdown_connections 2023-07-18T03:29:08.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.037+0000 7f61a1e06700 1 --2- 172.21.15.40:0/3826460389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f619c0ffcd0 0x7f619c106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:08.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.037+0000 7f61a1e06700 1 --2- 172.21.15.40:0/3826460389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f619c0fe150 0x7f619c0fe560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:08.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.037+0000 7f61a1e06700 1 --2- 172.21.15.40:0/3826460389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619c0ff340 0x7f619c0ff790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:08.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.037+0000 7f61a1e06700 1 -- 172.21.15.40:0/3826460389 >> 172.21.15.40:0/3826460389 conn(0x7f619c0f9770 msgr2=0x7f619c0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:08.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.037+0000 7f61a1e06700 1 -- 172.21.15.40:0/3826460389 shutdown_connections 2023-07-18T03:29:08.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.037+0000 7f61a1e06700 1 -- 172.21.15.40:0/3826460389 wait complete. 2023-07-18T03:29:08.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f61a1e06700 1 Processor -- start 2023-07-18T03:29:08.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f61a1e06700 1 -- start start 2023-07-18T03:29:08.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f61a1e06700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f619c0fe150 0x7f619c195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:08.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f61a1e06700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619c0ff340 0x7f619c195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:08.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f61a1e06700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f619c0ffcd0 0x7f619c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:08.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f61a1e06700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c19d3c0 con 0x7f619c0ff340 2023-07-18T03:29:08.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f61a1e06700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c19d500 con 0x7f619c0ffcd0 2023-07-18T03:29:08.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f61a1e06700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c19d690 con 0x7f619c0fe150 2023-07-18T03:29:08.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f619affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619c0ff340 0x7f619c195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:08.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f619affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619c0ff340 0x7f619c195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55882/0 (socket says 172.21.15.40:55882) 2023-07-18T03:29:08.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f619affd700 1 -- 172.21.15.40:0/159892806 learned_addr learned my addr 172.21.15.40:0/159892806 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:08.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f619b7fe700 1 --2- 172.21.15.40:0/159892806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f619c0fe150 0x7f619c195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:08.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f619bfff700 1 --2- 172.21.15.40:0/159892806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f619c0ffcd0 0x7f619c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f619affd700 1 -- 172.21.15.40:0/159892806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f619c0fe150 msgr2=0x7f619c195840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f619affd700 1 --2- 172.21.15.40:0/159892806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f619c0fe150 0x7f619c195840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f619affd700 1 -- 172.21.15.40:0/159892806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f619c0ffcd0 msgr2=0x7f619c19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f619affd700 1 --2- 172.21.15.40:0/159892806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f619c0ffcd0 0x7f619c19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.038+0000 7f619affd700 1 -- 172.21.15.40:0/159892806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61840095f0 con 0x7f619c0ff340 2023-07-18T03:29:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.039+0000 7f619affd700 1 --2- 172.21.15.40:0/159892806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619c0ff340 0x7f619c195d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f618400c010 tx=0x7f6184004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.039+0000 7f6198ff9700 1 -- 172.21.15.40:0/159892806 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618401d070 con 0x7f619c0ff340 2023-07-18T03:29:08.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.039+0000 7f61a1e06700 1 -- 172.21.15.40:0/159892806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619c19d910 con 0x7f619c0ff340 2023-07-18T03:29:08.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.039+0000 7f6198ff9700 1 -- 172.21.15.40:0/159892806 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6184005030 con 0x7f619c0ff340 2023-07-18T03:29:08.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.039+0000 7f6198ff9700 1 -- 172.21.15.40:0/159892806 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6184004b90 con 0x7f619c0ff340 2023-07-18T03:29:08.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.039+0000 7f61a1e06700 1 -- 172.21.15.40:0/159892806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619c19dce0 con 0x7f619c0ff340 2023-07-18T03:29:08.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.040+0000 7f61a1e06700 1 -- 172.21.15.40:0/159892806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619c106200 con 0x7f619c0ff340 2023-07-18T03:29:08.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.041+0000 7f6198ff9700 1 -- 172.21.15.40:0/159892806 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f61840051a0 con 0x7f619c0ff340 2023-07-18T03:29:08.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.041+0000 7f6198ff9700 1 --2- 172.21.15.40:0/159892806 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f61880a11a0 0x7f61880a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:08.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.042+0000 7f6198ff9700 1 -- 172.21.15.40:0/159892806 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f61840cdb70 con 0x7f619c0ff340 2023-07-18T03:29:08.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.042+0000 7f619b7fe700 1 --2- 172.21.15.40:0/159892806 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f61880a11a0 0x7f61880a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:08.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.042+0000 7f619b7fe700 1 --2- 172.21.15.40:0/159892806 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f61880a11a0 0x7f61880a3650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f619c0ff1a0 tx=0x7f618c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:08.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.045+0000 7f6198ff9700 1 -- 172.21.15.40:0/159892806 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f61840977f0 con 0x7f619c0ff340 2023-07-18T03:29:08.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:07 smithi064 ceph-mon[98059]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:08.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:07 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:08.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:07 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:08.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:07 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:08.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:07 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:08.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:07 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:08.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:07 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:08.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.295+0000 7f61a1e06700 1 -- 172.21.15.40:0/159892806 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f619c02c850 con 0x7f619c0ff340 2023-07-18T03:29:08.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.295+0000 7f6198ff9700 1 -- 172.21.15.40:0/159892806 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5045 (secure 0 0 0) 0x7f618409ae10 con 0x7f619c0ff340 2023-07-18T03:29:08.293 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:08.293 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2023-07-18T03:29:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.297+0000 7f61a1e06700 1 -- 172.21.15.40:0/159892806 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f61880a11a0 msgr2=0x7f61880a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.297+0000 7f61a1e06700 1 --2- 172.21.15.40:0/159892806 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f61880a11a0 0x7f61880a3650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f619c0ff1a0 tx=0x7f618c009260 comp rx=0 tx=0).stop 2023-07-18T03:29:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.298+0000 7f61a1e06700 1 -- 172.21.15.40:0/159892806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619c0ff340 msgr2=0x7f619c195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.298+0000 7f61a1e06700 1 --2- 172.21.15.40:0/159892806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619c0ff340 0x7f619c195d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f618400c010 tx=0x7f6184004730 comp rx=0 tx=0).stop 2023-07-18T03:29:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.298+0000 7f61a1e06700 1 -- 172.21.15.40:0/159892806 shutdown_connections 2023-07-18T03:29:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.298+0000 7f61a1e06700 1 --2- 172.21.15.40:0/159892806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f619c0ffcd0 0x7f619c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.298+0000 7f61a1e06700 1 --2- 172.21.15.40:0/159892806 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f61880a11a0 0x7f61880a3650 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:08.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.298+0000 7f61a1e06700 1 --2- 172.21.15.40:0/159892806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f619c0fe150 0x7f619c195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.298+0000 7f61a1e06700 1 --2- 172.21.15.40:0/159892806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f619c0ff340 0x7f619c195d80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.298+0000 7f61a1e06700 1 -- 172.21.15.40:0/159892806 >> 172.21.15.40:0/159892806 conn(0x7f619c0f9770 msgr2=0x7f619c0fb970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.298+0000 7f61a1e06700 1 -- 172.21.15.40:0/159892806 shutdown_connections 2023-07-18T03:29:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:08.298+0000 7f61a1e06700 1 -- 172.21.15.40:0/159892806 wait complete. 2023-07-18T03:29:08.297 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2023-07-18T03:29:08.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:08 smithi134 ceph-mon[98216]: Deploying daemon mds.c on smithi134 2023-07-18T03:29:08.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:08 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.134:6834/2946854745,v1:172.21.15.134:6835/2946854745] up:boot 2023-07-18T03:29:08.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:08 smithi134 ceph-mon[98216]: fsmap 6 up:standby 2023-07-18T03:29:08.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:08 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-18T03:29:08.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:08 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/159892806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:09.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:08 smithi040 ceph-mon[96439]: Deploying daemon mds.c on smithi134 2023-07-18T03:29:09.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:08 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.134:6834/2946854745,v1:172.21.15.134:6835/2946854745] up:boot 2023-07-18T03:29:09.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:08 smithi040 ceph-mon[96439]: fsmap 6 up:standby 2023-07-18T03:29:09.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:08 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-18T03:29:09.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:08 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/159892806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:09.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:08 smithi064 ceph-mon[98059]: Deploying daemon mds.c on smithi134 2023-07-18T03:29:09.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:08 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.134:6834/2946854745,v1:172.21.15.134:6835/2946854745] up:boot 2023-07-18T03:29:09.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:08 smithi064 ceph-mon[98059]: fsmap 6 up:standby 2023-07-18T03:29:09.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:08 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-18T03:29:09.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:08 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/159892806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:09.317 INFO:journalctl@ceph.mds.c.smithi134.stdout:Jul 18 03:29:09 smithi134 systemd[1]: Starting Ceph mds.c for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:29:09.662 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:09 smithi134 ceph-mon[98216]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:09.663 INFO:journalctl@ceph.mds.c.smithi134.stdout:Jul 18 03:29:09 smithi134 podman[124193]: 2023-07-18T03:29:09.663 INFO:journalctl@ceph.mds.c.smithi134.stdout:Jul 18 03:29:09 smithi134 ceph-mds[124233]: -- 172.21.15.134:0/3495800329 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555e097531e0 con 0x555e09728c00 2023-07-18T03:29:09.663 INFO:journalctl@ceph.mds.c.smithi134.stdout:Jul 18 03:29:09 smithi134 bash[124193]: e1f021a73de19475a09ef1de1eb2e79fe2950e1684b4319d1e656c86e41a51e6 2023-07-18T03:29:09.663 INFO:journalctl@ceph.mds.c.smithi134.stdout:Jul 18 03:29:09 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-c[124209]: starting mds.c at 2023-07-18T03:29:10.066 INFO:journalctl@ceph.mds.c.smithi134.stdout:Jul 18 03:29:09 smithi134 systemd[1]: Started Ceph mds.c for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:29:10.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:09 smithi040 ceph-mon[96439]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:10.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:09 smithi064 ceph-mon[98059]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:10.308 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:29:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.450+0000 7f26423f6700 1 -- 172.21.15.40:0/2312853439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f263c100580 msgr2=0x7f263c106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.450+0000 7f26423f6700 1 --2- 172.21.15.40:0/2312853439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f263c100580 0x7f263c106e40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f263400b0a0 tx=0x7f263400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.450+0000 7f26423f6700 1 -- 172.21.15.40:0/2312853439 shutdown_connections 2023-07-18T03:29:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.450+0000 7f26423f6700 1 --2- 172.21.15.40:0/2312853439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f263c100580 0x7f263c106e40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:10.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.450+0000 7f26423f6700 1 --2- 172.21.15.40:0/2312853439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f263c0fea00 0x7f263c0fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:10.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.450+0000 7f26423f6700 1 --2- 172.21.15.40:0/2312853439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c0ffbf0 0x7f263c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:10.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.450+0000 7f26423f6700 1 -- 172.21.15.40:0/2312853439 >> 172.21.15.40:0/2312853439 conn(0x7f263c0fa020 msgr2=0x7f263c0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:10.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.450+0000 7f26423f6700 1 -- 172.21.15.40:0/2312853439 shutdown_connections 2023-07-18T03:29:10.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.450+0000 7f26423f6700 1 -- 172.21.15.40:0/2312853439 wait complete. 2023-07-18T03:29:10.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.451+0000 7f26423f6700 1 Processor -- start 2023-07-18T03:29:10.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.451+0000 7f26423f6700 1 -- start start 2023-07-18T03:29:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.451+0000 7f26423f6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f263c0fea00 0x7f263c194780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.451+0000 7f26423f6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f263c0ffbf0 0x7f263c194cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:10.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.451+0000 7f26423f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c100580 0x7f263c19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.451+0000 7f26423f6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c19c300 con 0x7f263c0fea00 2023-07-18T03:29:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.451+0000 7f26423f6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c19c440 con 0x7f263c100580 2023-07-18T03:29:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.451+0000 7f26423f6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c19c5d0 con 0x7f263c0ffbf0 2023-07-18T03:29:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.451+0000 7f263bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f263c0fea00 0x7f263c194780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.451+0000 7f263bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f263c0fea00 0x7f263c194780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55894/0 (socket says 172.21.15.40:55894) 2023-07-18T03:29:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.451+0000 7f263bfff700 1 -- 172.21.15.40:0/3074882397 learned_addr learned my addr 172.21.15.40:0/3074882397 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.451+0000 7f2640993700 1 --2- 172.21.15.40:0/3074882397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c100580 0x7f263c19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:10.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.452+0000 7f263b7fe700 1 --2- 172.21.15.40:0/3074882397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f263c0ffbf0 0x7f263c194cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.452+0000 7f263bfff700 1 -- 172.21.15.40:0/3074882397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f263c0ffbf0 msgr2=0x7f263c194cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.452+0000 7f263bfff700 1 --2- 172.21.15.40:0/3074882397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f263c0ffbf0 0x7f263c194cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.452+0000 7f263bfff700 1 -- 172.21.15.40:0/3074882397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c100580 msgr2=0x7f263c19bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.452+0000 7f263bfff700 1 --2- 172.21.15.40:0/3074882397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c100580 0x7f263c19bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.452+0000 7f263bfff700 1 -- 172.21.15.40:0/3074882397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2634009d00 con 0x7f263c0fea00 2023-07-18T03:29:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.452+0000 7f2640993700 1 --2- 172.21.15.40:0/3074882397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c100580 0x7f263c19bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:10.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.452+0000 7f263bfff700 1 --2- 172.21.15.40:0/3074882397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f263c0fea00 0x7f263c194780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f262c0098f0 tx=0x7f262c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.452+0000 7f26397fa700 1 -- 172.21.15.40:0/3074882397 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c012070 con 0x7f263c0fea00 2023-07-18T03:29:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.452+0000 7f26397fa700 1 -- 172.21.15.40:0/3074882397 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f262c0044e0 con 0x7f263c0fea00 2023-07-18T03:29:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.452+0000 7f26423f6700 1 -- 172.21.15.40:0/3074882397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263c0688f0 con 0x7f263c0fea00 2023-07-18T03:29:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.452+0000 7f26397fa700 1 -- 172.21.15.40:0/3074882397 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c003df0 con 0x7f263c0fea00 2023-07-18T03:29:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.452+0000 7f26423f6700 1 -- 172.21.15.40:0/3074882397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263c068e50 con 0x7f263c0fea00 2023-07-18T03:29:10.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.453+0000 7f26423f6700 1 -- 172.21.15.40:0/3074882397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263c10a1e0 con 0x7f263c0fea00 2023-07-18T03:29:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.454+0000 7f26397fa700 1 -- 172.21.15.40:0/3074882397 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f262c016030 con 0x7f263c0fea00 2023-07-18T03:29:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.455+0000 7f26397fa700 1 --2- 172.21.15.40:0/3074882397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f26240a1220 0x7f26240a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.455+0000 7f26397fa700 1 -- 172.21.15.40:0/3074882397 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f262c0cd940 con 0x7f263c0fea00 2023-07-18T03:29:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.455+0000 7f263b7fe700 1 --2- 172.21.15.40:0/3074882397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f26240a1220 0x7f26240a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:10.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.455+0000 7f263b7fe700 1 --2- 172.21.15.40:0/3074882397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f26240a1220 0x7f26240a36d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2630009bf0 tx=0x7f2630009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:10.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.458+0000 7f26397fa700 1 -- 172.21.15.40:0/3074882397 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f262c097530 con 0x7f263c0fea00 2023-07-18T03:29:10.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.700+0000 7f26423f6700 1 -- 172.21.15.40:0/3074882397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f263c105ca0 con 0x7f263c0fea00 2023-07-18T03:29:10.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.701+0000 7f26397fa700 1 -- 172.21.15.40:0/3074882397 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5805 (secure 0 0 0) 0x7f262c09ab50 con 0x7f263c0fea00 2023-07-18T03:29:10.699 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:10.699 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3509655945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3509655945},{"type":"v1","addr":"172.21.15.134:6837","nonce":3509655945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-07-18T03:29:10.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.703+0000 7f26423f6700 1 -- 172.21.15.40:0/3074882397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f26240a1220 msgr2=0x7f26240a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:10.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.703+0000 7f26423f6700 1 --2- 172.21.15.40:0/3074882397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f26240a1220 0x7f26240a36d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2630009bf0 tx=0x7f2630009260 comp rx=0 tx=0).stop 2023-07-18T03:29:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.703+0000 7f26423f6700 1 -- 172.21.15.40:0/3074882397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f263c0fea00 msgr2=0x7f263c194780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.703+0000 7f26423f6700 1 --2- 172.21.15.40:0/3074882397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f263c0fea00 0x7f263c194780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f262c0098f0 tx=0x7f262c009c00 comp rx=0 tx=0).stop 2023-07-18T03:29:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.703+0000 7f26423f6700 1 -- 172.21.15.40:0/3074882397 shutdown_connections 2023-07-18T03:29:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.703+0000 7f26423f6700 1 --2- 172.21.15.40:0/3074882397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f263c100580 0x7f263c19bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.703+0000 7f26423f6700 1 --2- 172.21.15.40:0/3074882397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f263c0fea00 0x7f263c194780 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.703+0000 7f26423f6700 1 --2- 172.21.15.40:0/3074882397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f26240a1220 0x7f26240a36d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.703+0000 7f26423f6700 1 --2- 172.21.15.40:0/3074882397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f263c0ffbf0 0x7f263c194cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:10.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.703+0000 7f26423f6700 1 -- 172.21.15.40:0/3074882397 >> 172.21.15.40:0/3074882397 conn(0x7f263c0fa020 msgr2=0x7f263c102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:10.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.703+0000 7f26423f6700 1 -- 172.21.15.40:0/3074882397 shutdown_connections 2023-07-18T03:29:10.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:10.703+0000 7f26423f6700 1 -- 172.21.15.40:0/3074882397 wait complete. 2023-07-18T03:29:10.702 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 8 2023-07-18T03:29:11.005 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:10 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:11.005 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:10 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:11.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:10 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:11.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:10 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:11.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:10 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:11.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:10 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:11.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:10 smithi064 ceph-mon[98059]: Deploying daemon mds.h on smithi064 2023-07-18T03:29:11.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:10 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.134:6836/3509655945,v1:172.21.15.134:6837/3509655945] up:boot 2023-07-18T03:29:11.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:10 smithi064 ceph-mon[98059]: fsmap 7 up:standby 2023-07-18T03:29:11.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:10 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:29:11.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:10 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3074882397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:11.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:10 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:11.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:10 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:11.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:10 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:11.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:10 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:11.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:10 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:11.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:10 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:11.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:10 smithi134 ceph-mon[98216]: Deploying daemon mds.h on smithi064 2023-07-18T03:29:11.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:10 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.134:6836/3509655945,v1:172.21.15.134:6837/3509655945] up:boot 2023-07-18T03:29:11.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:10 smithi134 ceph-mon[98216]: fsmap 7 up:standby 2023-07-18T03:29:11.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:10 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:29:11.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:10 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3074882397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:11.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:10 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:11.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:10 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:11.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:10 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:11.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:10 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:11.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:10 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:11.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:10 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:11.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:10 smithi040 ceph-mon[96439]: Deploying daemon mds.h on smithi064 2023-07-18T03:29:11.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:10 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.134:6836/3509655945,v1:172.21.15.134:6837/3509655945] up:boot 2023-07-18T03:29:11.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:10 smithi040 ceph-mon[96439]: fsmap 7 up:standby 2023-07-18T03:29:11.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:10 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:29:11.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:10 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3074882397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:11.560 INFO:journalctl@ceph.mds.h.smithi064.stdout:Jul 18 03:29:11 smithi064 systemd[1]: Starting Ceph mds.h for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:29:11.813 INFO:journalctl@ceph.mds.h.smithi064.stdout:Jul 18 03:29:11 smithi064 podman[123061]: 2023-07-18T03:29:11.814 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:11 smithi064 ceph-mon[98059]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:12.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:11 smithi134 ceph-mon[98216]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:12.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:11 smithi040 ceph-mon[96439]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:12.100 INFO:journalctl@ceph.mds.h.smithi064.stdout:Jul 18 03:29:11 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-h[123077]: starting mds.h at 2023-07-18T03:29:12.101 INFO:journalctl@ceph.mds.h.smithi064.stdout:Jul 18 03:29:11 smithi064 bash[123061]: 23568a0fb5f7bc4f502d3688eda500858fec559b58e872ffcacd78b0d80613ef 2023-07-18T03:29:12.101 INFO:journalctl@ceph.mds.h.smithi064.stdout:Jul 18 03:29:11 smithi064 systemd[1]: Started Ceph mds.h for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:29:12.714 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:29:12.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.859+0000 7f7b33747700 1 -- 172.21.15.40:0/2151623225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b2c100cf0 msgr2=0x7f7b2c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:12.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.859+0000 7f7b33747700 1 --2- 172.21.15.40:0/2151623225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b2c100cf0 0x7f7b2c101140 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c009990 tx=0x7f7b1c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:12.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.859+0000 7f7b33747700 1 -- 172.21.15.40:0/2151623225 shutdown_connections 2023-07-18T03:29:12.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.859+0000 7f7b33747700 1 --2- 172.21.15.40:0/2151623225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b2c101680 0x7f7b2c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:12.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.859+0000 7f7b33747700 1 --2- 172.21.15.40:0/2151623225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b2c0ffb00 0x7f7b2c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:12.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.859+0000 7f7b33747700 1 --2- 172.21.15.40:0/2151623225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b2c100cf0 0x7f7b2c101140 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:12.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.859+0000 7f7b33747700 1 -- 172.21.15.40:0/2151623225 >> 172.21.15.40:0/2151623225 conn(0x7f7b2c0fb100 msgr2=0x7f7b2c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:12.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.859+0000 7f7b33747700 1 -- 172.21.15.40:0/2151623225 shutdown_connections 2023-07-18T03:29:12.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.859+0000 7f7b33747700 1 -- 172.21.15.40:0/2151623225 wait complete. 2023-07-18T03:29:12.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.859+0000 7f7b33747700 1 Processor -- start 2023-07-18T03:29:12.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.859+0000 7f7b33747700 1 -- start start 2023-07-18T03:29:12.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.859+0000 7f7b33747700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b2c0ffb00 0x7f7b2c1957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:12.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.859+0000 7f7b33747700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b2c100cf0 0x7f7b2c195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:12.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b33747700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b2c101680 0x7f7b2c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:12.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b33747700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c19d370 con 0x7f7b2c101680 2023-07-18T03:29:12.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b33747700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c19d4b0 con 0x7f7b2c0ffb00 2023-07-18T03:29:12.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b33747700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c19d640 con 0x7f7b2c100cf0 2023-07-18T03:29:12.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b31ce4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b2c101680 0x7f7b2c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:12.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b30ce2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b2c100cf0 0x7f7b2c195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:12.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b31ce4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b2c101680 0x7f7b2c19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55910/0 (socket says 172.21.15.40:55910) 2023-07-18T03:29:12.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b31ce4700 1 -- 172.21.15.40:0/3939261636 learned_addr learned my addr 172.21.15.40:0/3939261636 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:12.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b314e3700 1 --2- 172.21.15.40:0/3939261636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b2c0ffb00 0x7f7b2c1957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:12.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b31ce4700 1 -- 172.21.15.40:0/3939261636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b2c100cf0 msgr2=0x7f7b2c195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:12.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b31ce4700 1 --2- 172.21.15.40:0/3939261636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b2c100cf0 0x7f7b2c195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:12.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b31ce4700 1 -- 172.21.15.40:0/3939261636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b2c0ffb00 msgr2=0x7f7b2c1957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:12.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b31ce4700 1 --2- 172.21.15.40:0/3939261636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b2c0ffb00 0x7f7b2c1957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b31ce4700 1 -- 172.21.15.40:0/3939261636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c0095f0 con 0x7f7b2c101680 2023-07-18T03:29:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b30ce2700 1 --2- 172.21.15.40:0/3939261636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b2c100cf0 0x7f7b2c195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b314e3700 1 --2- 172.21.15.40:0/3939261636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b2c0ffb00 0x7f7b2c1957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:29:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b31ce4700 1 --2- 172.21.15.40:0/3939261636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b2c101680 0x7f7b2c19cda0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7b24007ee0 tx=0x7f7b2400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b227fc700 1 -- 172.21.15.40:0/3939261636 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24017070 con 0x7f7b2c101680 2023-07-18T03:29:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b227fc700 1 -- 172.21.15.40:0/3939261636 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b2400f040 con 0x7f7b2c101680 2023-07-18T03:29:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b33747700 1 -- 172.21.15.40:0/3939261636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2c19d920 con 0x7f7b2c101680 2023-07-18T03:29:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b227fc700 1 -- 172.21.15.40:0/3939261636 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24005740 con 0x7f7b2c101680 2023-07-18T03:29:12.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.860+0000 7f7b33747700 1 -- 172.21.15.40:0/3939261636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b2c19dde0 con 0x7f7b2c101680 2023-07-18T03:29:12.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.861+0000 7f7b33747700 1 -- 172.21.15.40:0/3939261636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b2c10b320 con 0x7f7b2c101680 2023-07-18T03:29:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.863+0000 7f7b227fc700 1 -- 172.21.15.40:0/3939261636 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7b2400dc60 con 0x7f7b2c101680 2023-07-18T03:29:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.863+0000 7f7b227fc700 1 --2- 172.21.15.40:0/3939261636 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7b180a11a0 0x7f7b180a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.863+0000 7f7b314e3700 1 --2- 172.21.15.40:0/3939261636 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7b180a11a0 0x7f7b180a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.863+0000 7f7b227fc700 1 -- 172.21.15.40:0/3939261636 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7b240cc2c0 con 0x7f7b2c101680 2023-07-18T03:29:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.864+0000 7f7b314e3700 1 --2- 172.21.15.40:0/3939261636 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7b180a11a0 0x7f7b180a3650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7b28005950 tx=0x7f7b280058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:12.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:12.867+0000 7f7b227fc700 1 -- 172.21.15.40:0/3939261636 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7b24095eb0 con 0x7f7b2c101680 2023-07-18T03:29:13.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:12 smithi040 ceph-mon[96439]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:13.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:12 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:13.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:12 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:13.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:12 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:13.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:12 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:13.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:12 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:13.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:12 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:13.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:12 smithi040 ceph-mon[96439]: Deploying daemon mds.i on smithi134 2023-07-18T03:29:13.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:12 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] up:boot 2023-07-18T03:29:13.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:12 smithi040 ceph-mon[96439]: fsmap 8 up:standby 2023-07-18T03:29:13.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:12 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-18T03:29:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:13.114+0000 7f7b33747700 1 -- 172.21.15.40:0/3939261636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b2c1069a0 con 0x7f7b2c101680 2023-07-18T03:29:13.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:13.115+0000 7f7b227fc700 1 -- 172.21.15.40:0/3939261636 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6562 (secure 0 0 0) 0x7f7b240994d0 con 0x7f7b2c101680 2023-07-18T03:29:13.111 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:13.112 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3509655945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3509655945},{"type":"v1","addr":"172.21.15.134:6837","nonce":3509655945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-07-18T03:29:13.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:13.117+0000 7f7b33747700 1 -- 172.21.15.40:0/3939261636 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7b180a11a0 msgr2=0x7f7b180a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:13.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:13.117+0000 7f7b33747700 1 --2- 172.21.15.40:0/3939261636 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7b180a11a0 0x7f7b180a3650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7b28005950 tx=0x7f7b280058e0 comp rx=0 tx=0).stop 2023-07-18T03:29:13.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:13.117+0000 7f7b33747700 1 -- 172.21.15.40:0/3939261636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b2c101680 msgr2=0x7f7b2c19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:13.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:13.117+0000 7f7b33747700 1 --2- 172.21.15.40:0/3939261636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b2c101680 0x7f7b2c19cda0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7b24007ee0 tx=0x7f7b2400d3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:13.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:13.117+0000 7f7b33747700 1 -- 172.21.15.40:0/3939261636 shutdown_connections 2023-07-18T03:29:13.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:13.117+0000 7f7b33747700 1 --2- 172.21.15.40:0/3939261636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b2c101680 0x7f7b2c19cda0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:13.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:13.117+0000 7f7b33747700 1 --2- 172.21.15.40:0/3939261636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b2c0ffb00 0x7f7b2c1957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:13.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:13.117+0000 7f7b33747700 1 --2- 172.21.15.40:0/3939261636 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7b180a11a0 0x7f7b180a3650 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:13.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:13.117+0000 7f7b33747700 1 --2- 172.21.15.40:0/3939261636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b2c100cf0 0x7f7b2c195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:13.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:13.117+0000 7f7b33747700 1 -- 172.21.15.40:0/3939261636 >> 172.21.15.40:0/3939261636 conn(0x7f7b2c0fb100 msgr2=0x7f7b2c0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:13.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:13.117+0000 7f7b33747700 1 -- 172.21.15.40:0/3939261636 shutdown_connections 2023-07-18T03:29:13.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:13.117+0000 7f7b33747700 1 -- 172.21.15.40:0/3939261636 wait complete. 2023-07-18T03:29:13.115 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 9 2023-07-18T03:29:13.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:12 smithi134 ceph-mon[98216]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:13.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:12 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:13.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:12 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:13.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:12 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:13.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:12 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:13.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:12 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:13.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:12 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:13.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:12 smithi134 ceph-mon[98216]: Deploying daemon mds.i on smithi134 2023-07-18T03:29:13.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:12 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] up:boot 2023-07-18T03:29:13.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:12 smithi134 ceph-mon[98216]: fsmap 8 up:standby 2023-07-18T03:29:13.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:12 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-18T03:29:13.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:12 smithi064 ceph-mon[98059]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:13.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:12 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:13.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:12 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:13.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:12 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:13.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:12 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:13.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:12 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:12 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:12 smithi064 ceph-mon[98059]: Deploying daemon mds.i on smithi134 2023-07-18T03:29:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:12 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] up:boot 2023-07-18T03:29:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:12 smithi064 ceph-mon[98059]: fsmap 8 up:standby 2023-07-18T03:29:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:12 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-18T03:29:13.567 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:29:13 smithi134 systemd[1]: Starting Ceph mds.i for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:29:14.033 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:13 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3939261636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:14.034 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:29:13 smithi134 podman[124533]: 2023-07-18T03:29:14.317 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:29:14 smithi134 ceph-mds[124573]: -- 172.21.15.134:0/1379034871 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bde5b4d1e0 con 0x55bde5b22c00 2023-07-18T03:29:14.317 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:29:14 smithi134 bash[124533]: 4ecaf9f3f7ec7d5603c750816f14d500ba85b47f6fe3321f0efba50f264c7908 2023-07-18T03:29:14.317 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:29:14 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: starting mds.i at 2023-07-18T03:29:14.317 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:29:14 smithi134 systemd[1]: Started Ceph mds.i for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:29:14.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:13 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3939261636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:14.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:13 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3939261636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:15.127 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:29:15.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.263+0000 7f7491904700 1 -- 172.21.15.40:0/3783614396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f748c101690 msgr2=0x7f748c107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.263+0000 7f7491904700 1 --2- 172.21.15.40:0/3783614396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f748c101690 0x7f748c107f50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f748400b0a0 tx=0x7f748400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.263+0000 7f7491904700 1 -- 172.21.15.40:0/3783614396 shutdown_connections 2023-07-18T03:29:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.263+0000 7f7491904700 1 --2- 172.21.15.40:0/3783614396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f748c101690 0x7f748c107f50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.263+0000 7f7491904700 1 --2- 172.21.15.40:0/3783614396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f748c0ffb10 0x7f748c0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.263+0000 7f7491904700 1 --2- 172.21.15.40:0/3783614396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f748c100d00 0x7f748c101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.263+0000 7f7491904700 1 -- 172.21.15.40:0/3783614396 >> 172.21.15.40:0/3783614396 conn(0x7f748c0fb110 msgr2=0x7f748c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:15.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.263+0000 7f7491904700 1 -- 172.21.15.40:0/3783614396 shutdown_connections 2023-07-18T03:29:15.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.263+0000 7f7491904700 1 -- 172.21.15.40:0/3783614396 wait complete. 2023-07-18T03:29:15.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.264+0000 7f7491904700 1 Processor -- start 2023-07-18T03:29:15.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.264+0000 7f7491904700 1 -- start start 2023-07-18T03:29:15.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.264+0000 7f7491904700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f748c0ffb10 0x7f748c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:15.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.264+0000 7f7491904700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f748c100d00 0x7f748c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:15.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.264+0000 7f7491904700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f748c101690 0x7f748c19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:15.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.264+0000 7f7491904700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c19f610 con 0x7f748c101690 2023-07-18T03:29:15.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.264+0000 7f7491904700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c19f750 con 0x7f748c100d00 2023-07-18T03:29:15.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.264+0000 7f7491904700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c19f8e0 con 0x7f748c0ffb10 2023-07-18T03:29:15.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.265+0000 7f748b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f748c101690 0x7f748c19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:15.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.265+0000 7f748b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f748c101690 0x7f748c19f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55922/0 (socket says 172.21.15.40:55922) 2023-07-18T03:29:15.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.265+0000 7f748b7fe700 1 -- 172.21.15.40:0/3616538940 learned_addr learned my addr 172.21.15.40:0/3616538940 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:15.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.265+0000 7f748affd700 1 --2- 172.21.15.40:0/3616538940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f748c0ffb10 0x7f748c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:15.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.265+0000 7f748a7fc700 1 --2- 172.21.15.40:0/3616538940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f748c100d00 0x7f748c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:15.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.265+0000 7f748b7fe700 1 -- 172.21.15.40:0/3616538940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f748c0ffb10 msgr2=0x7f748c197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:15.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.265+0000 7f748b7fe700 1 --2- 172.21.15.40:0/3616538940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f748c0ffb10 0x7f748c197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:15.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.265+0000 7f748b7fe700 1 -- 172.21.15.40:0/3616538940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f748c100d00 msgr2=0x7f748c197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:15.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.265+0000 7f748b7fe700 1 --2- 172.21.15.40:0/3616538940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f748c100d00 0x7f748c197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:15.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.265+0000 7f748b7fe700 1 -- 172.21.15.40:0/3616538940 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7484009d00 con 0x7f748c101690 2023-07-18T03:29:15.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.265+0000 7f748b7fe700 1 --2- 172.21.15.40:0/3616538940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f748c101690 0x7f748c19f040 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f748400b780 tx=0x7f748400bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:15.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.265+0000 7f7490902700 1 -- 172.21.15.40:0/3616538940 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748400e040 con 0x7f748c101690 2023-07-18T03:29:15.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.265+0000 7f7490902700 1 -- 172.21.15.40:0/3616538940 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7484004e60 con 0x7f748c101690 2023-07-18T03:29:15.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.265+0000 7f7491904700 1 -- 172.21.15.40:0/3616538940 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748c19fb60 con 0x7f748c101690 2023-07-18T03:29:15.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.265+0000 7f7490902700 1 -- 172.21.15.40:0/3616538940 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748401b740 con 0x7f748c101690 2023-07-18T03:29:15.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.265+0000 7f7491904700 1 -- 172.21.15.40:0/3616538940 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748c19ffe0 con 0x7f748c101690 2023-07-18T03:29:15.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.266+0000 7f7491904700 1 -- 172.21.15.40:0/3616538940 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748c10b2f0 con 0x7f748c101690 2023-07-18T03:29:15.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.267+0000 7f7490902700 1 -- 172.21.15.40:0/3616538940 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f748401da40 con 0x7f748c101690 2023-07-18T03:29:15.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.268+0000 7f7490902700 1 --2- 172.21.15.40:0/3616538940 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74740a11a0 0x7f74740a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.268+0000 7f748affd700 1 --2- 172.21.15.40:0/3616538940 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74740a11a0 0x7f74740a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.268+0000 7f7490902700 1 -- 172.21.15.40:0/3616538940 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f74840cdde0 con 0x7f748c101690 2023-07-18T03:29:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.268+0000 7f748affd700 1 --2- 172.21.15.40:0/3616538940 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74740a11a0 0x7f74740a3650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f748c100b60 tx=0x7f747c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:15.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.272+0000 7f7490902700 1 -- 172.21.15.40:0/3616538940 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7484097a60 con 0x7f748c101690 2023-07-18T03:29:15.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:15 smithi040 ceph-mon[96439]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:15.418 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:15.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:15.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:15.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:15.419 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:15.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:15.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:15 smithi040 ceph-mon[96439]: Deploying daemon mds.g on smithi040 2023-07-18T03:29:15.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:15 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] up:boot 2023-07-18T03:29:15.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:15 smithi040 ceph-mon[96439]: fsmap 9 up:standby 2023-07-18T03:29:15.420 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:15 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-18T03:29:15.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.544+0000 7f7491904700 1 -- 172.21.15.40:0/3616538940 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f748c1a05f0 con 0x7f748c101690 2023-07-18T03:29:15.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.545+0000 7f7490902700 1 -- 172.21.15.40:0/3616538940 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7324 (secure 0 0 0) 0x7f748409b080 con 0x7f748c101690 2023-07-18T03:29:15.542 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:15.542 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3509655945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3509655945},{"type":"v1","addr":"172.21.15.134:6837","nonce":3509655945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24523,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-18T03:29:15.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.547+0000 7f7491904700 1 -- 172.21.15.40:0/3616538940 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74740a11a0 msgr2=0x7f74740a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:15.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.547+0000 7f7491904700 1 --2- 172.21.15.40:0/3616538940 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74740a11a0 0x7f74740a3650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f748c100b60 tx=0x7f747c00b410 comp rx=0 tx=0).stop 2023-07-18T03:29:15.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.547+0000 7f7491904700 1 -- 172.21.15.40:0/3616538940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f748c101690 msgr2=0x7f748c19f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:15.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.547+0000 7f7491904700 1 --2- 172.21.15.40:0/3616538940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f748c101690 0x7f748c19f040 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f748400b780 tx=0x7f748400bd00 comp rx=0 tx=0).stop 2023-07-18T03:29:15.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.547+0000 7f7491904700 1 -- 172.21.15.40:0/3616538940 shutdown_connections 2023-07-18T03:29:15.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.547+0000 7f7491904700 1 --2- 172.21.15.40:0/3616538940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f748c101690 0x7f748c19f040 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:15.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.547+0000 7f7491904700 1 --2- 172.21.15.40:0/3616538940 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74740a11a0 0x7f74740a3650 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:15.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.547+0000 7f7491904700 1 --2- 172.21.15.40:0/3616538940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f748c0ffb10 0x7f748c197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:15.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.547+0000 7f7491904700 1 --2- 172.21.15.40:0/3616538940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f748c100d00 0x7f748c197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:15.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.548+0000 7f7491904700 1 -- 172.21.15.40:0/3616538940 >> 172.21.15.40:0/3616538940 conn(0x7f748c0fb110 msgr2=0x7f748c1064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:15.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.548+0000 7f7491904700 1 -- 172.21.15.40:0/3616538940 shutdown_connections 2023-07-18T03:29:15.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:15.548+0000 7f7491904700 1 -- 172.21.15.40:0/3616538940 wait complete. 2023-07-18T03:29:15.546 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 10 2023-07-18T03:29:15.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:15 smithi134 ceph-mon[98216]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:15.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:15.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:15.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:15.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:15.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:15.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:15.568 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:15 smithi134 ceph-mon[98216]: Deploying daemon mds.g on smithi040 2023-07-18T03:29:15.568 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:15 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] up:boot 2023-07-18T03:29:15.568 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:15 smithi134 ceph-mon[98216]: fsmap 9 up:standby 2023-07-18T03:29:15.568 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:15 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-18T03:29:15.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:15 smithi064 ceph-mon[98059]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:15.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:15.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:15.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:15.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:15.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:15.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:15.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:15 smithi064 ceph-mon[98059]: Deploying daemon mds.g on smithi040 2023-07-18T03:29:15.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:15 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] up:boot 2023-07-18T03:29:15.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:15 smithi064 ceph-mon[98059]: fsmap 9 up:standby 2023-07-18T03:29:15.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:15 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-18T03:29:15.691 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jul 18 03:29:15 smithi040 systemd[1]: Starting Ceph mds.g for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:29:16.286 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:16 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3616538940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:16.287 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jul 18 03:29:15 smithi040 podman[130847]: 2023-07-18T03:29:16.287 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jul 18 03:29:16 smithi040 bash[130847]: 80f792591c15300137517709daeb8e3b730a12d914cce604ca9069e5a07b2953 2023-07-18T03:29:16.287 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jul 18 03:29:16 smithi040 ceph-mds[130887]: -- 172.21.15.40:0/440796455 <== mon.0 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a53f6b11e0 con 0x55a53f687000 2023-07-18T03:29:16.288 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jul 18 03:29:16 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-g[130863]: starting mds.g at 2023-07-18T03:29:16.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:16 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3616538940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:16.579 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jul 18 03:29:16 smithi040 systemd[1]: Started Ceph mds.g for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:29:16.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:16 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3616538940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:17.558 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:29:17.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:17 smithi040 ceph-mon[96439]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:17.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:17 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:17 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:17 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:17 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:17 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:17.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:17 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:17.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:17 smithi040 ceph-mon[96439]: Deploying daemon mds.a on smithi040 2023-07-18T03:29:17.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:17 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.40:6838/1818167554,v1:172.21.15.40:6839/1818167554] up:boot 2023-07-18T03:29:17.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:17 smithi040 ceph-mon[96439]: fsmap 10 up:standby 2023-07-18T03:29:17.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:17 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-18T03:29:17.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:17 smithi064 ceph-mon[98059]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:17.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:17 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:17.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:17 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:17.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:17 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:17.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:17 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:17.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:17 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:17.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:17 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:17.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:17 smithi064 ceph-mon[98059]: Deploying daemon mds.a on smithi040 2023-07-18T03:29:17.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:17 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.40:6838/1818167554,v1:172.21.15.40:6839/1818167554] up:boot 2023-07-18T03:29:17.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:17 smithi064 ceph-mon[98059]: fsmap 10 up:standby 2023-07-18T03:29:17.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:17 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-18T03:29:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.732+0000 7f18c1d46700 1 -- 172.21.15.40:0/3490449222 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18bc0ffbc0 msgr2=0x7f18bc100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.732+0000 7f18c1d46700 1 --2- 172.21.15.40:0/3490449222 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18bc0ffbc0 0x7f18bc100010 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009990 tx=0x7f18b0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.732+0000 7f18c1d46700 1 -- 172.21.15.40:0/3490449222 shutdown_connections 2023-07-18T03:29:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.732+0000 7f18c1d46700 1 --2- 172.21.15.40:0/3490449222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18bc100550 0x7f18bc106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.732+0000 7f18c1d46700 1 --2- 172.21.15.40:0/3490449222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18bc0fe9d0 0x7f18bc0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.732+0000 7f18c1d46700 1 --2- 172.21.15.40:0/3490449222 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18bc0ffbc0 0x7f18bc100010 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.732+0000 7f18c1d46700 1 -- 172.21.15.40:0/3490449222 >> 172.21.15.40:0/3490449222 conn(0x7f18bc0f9fd0 msgr2=0x7f18bc0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.732+0000 7f18c1d46700 1 -- 172.21.15.40:0/3490449222 shutdown_connections 2023-07-18T03:29:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.732+0000 7f18c1d46700 1 -- 172.21.15.40:0/3490449222 wait complete. 2023-07-18T03:29:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.733+0000 7f18c1d46700 1 Processor -- start 2023-07-18T03:29:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.733+0000 7f18c1d46700 1 -- start start 2023-07-18T03:29:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.733+0000 7f18c1d46700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18bc0fe9d0 0x7f18bc194710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.733+0000 7f18c1d46700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18bc0ffbc0 0x7f18bc194c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.733+0000 7f18c1d46700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18bc100550 0x7f18bc19bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18c1d46700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc19c290 con 0x7f18bc100550 2023-07-18T03:29:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18c1d46700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc19c3d0 con 0x7f18bc0fe9d0 2023-07-18T03:29:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18c1d46700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc19c560 con 0x7f18bc0ffbc0 2023-07-18T03:29:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18bbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18bc100550 0x7f18bc19bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18bbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18bc100550 0x7f18bc19bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55948/0 (socket says 172.21.15.40:55948) 2023-07-18T03:29:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18bbfff700 1 -- 172.21.15.40:0/3723937735 learned_addr learned my addr 172.21.15.40:0/3723937735 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:17.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18bb7fe700 1 --2- 172.21.15.40:0/3723937735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18bc0fe9d0 0x7f18bc194710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:17.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18baffd700 1 --2- 172.21.15.40:0/3723937735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18bc0ffbc0 0x7f18bc194c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:17.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18bbfff700 1 -- 172.21.15.40:0/3723937735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18bc0ffbc0 msgr2=0x7f18bc194c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:17.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18bbfff700 1 --2- 172.21.15.40:0/3723937735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18bc0ffbc0 0x7f18bc194c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:17.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18bbfff700 1 -- 172.21.15.40:0/3723937735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18bc0fe9d0 msgr2=0x7f18bc194710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:17.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18bbfff700 1 --2- 172.21.15.40:0/3723937735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18bc0fe9d0 0x7f18bc194710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:17.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18bbfff700 1 -- 172.21.15.40:0/3723937735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b00095f0 con 0x7f18bc100550 2023-07-18T03:29:17.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18baffd700 1 --2- 172.21.15.40:0/3723937735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18bc0ffbc0 0x7f18bc194c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:29:17.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18bbfff700 1 --2- 172.21.15.40:0/3723937735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18bc100550 0x7f18bc19bcc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f18b400ada0 tx=0x7f18b400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:17.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18b8ff9700 1 -- 172.21.15.40:0/3723937735 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b400f070 con 0x7f18bc100550 2023-07-18T03:29:17.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18b8ff9700 1 -- 172.21.15.40:0/3723937735 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18b4007b30 con 0x7f18bc100550 2023-07-18T03:29:17.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18c1d46700 1 -- 172.21.15.40:0/3723937735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18bc19c840 con 0x7f18bc100550 2023-07-18T03:29:17.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18b8ff9700 1 -- 172.21.15.40:0/3723937735 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b40053e0 con 0x7f18bc100550 2023-07-18T03:29:17.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.734+0000 7f18c1d46700 1 -- 172.21.15.40:0/3723937735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18bc19cda0 con 0x7f18bc100550 2023-07-18T03:29:17.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.735+0000 7f18c1d46700 1 -- 172.21.15.40:0/3723937735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18bc105900 con 0x7f18bc100550 2023-07-18T03:29:17.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.736+0000 7f18b8ff9700 1 -- 172.21.15.40:0/3723937735 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f18b400c980 con 0x7f18bc100550 2023-07-18T03:29:17.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.737+0000 7f18b8ff9700 1 --2- 172.21.15.40:0/3723937735 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f18a40a1220 0x7f18a40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:17.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.737+0000 7f18b8ff9700 1 -- 172.21.15.40:0/3723937735 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f18b40cb9c0 con 0x7f18bc100550 2023-07-18T03:29:17.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.737+0000 7f18bb7fe700 1 --2- 172.21.15.40:0/3723937735 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f18a40a1220 0x7f18a40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:17.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.738+0000 7f18bb7fe700 1 --2- 172.21.15.40:0/3723937735 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f18a40a1220 0x7f18a40a36d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0ffa20 tx=0x7f18ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:17.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:17.741+0000 7f18b8ff9700 1 -- 172.21.15.40:0/3723937735 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f18b4095640 con 0x7f18bc100550 2023-07-18T03:29:17.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:17 smithi134 ceph-mon[98216]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:17.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:17 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:17.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:17 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:17.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:17 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:17.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:17 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:17.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:17 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:17.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:17 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:17.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:17 smithi134 ceph-mon[98216]: Deploying daemon mds.a on smithi040 2023-07-18T03:29:17.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:17 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.40:6838/1818167554,v1:172.21.15.40:6839/1818167554] up:boot 2023-07-18T03:29:17.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:17 smithi134 ceph-mon[98216]: fsmap 10 up:standby 2023-07-18T03:29:17.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:17 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-18T03:29:17.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:18.000+0000 7f18c1d46700 1 -- 172.21.15.40:0/3723937735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18bc02c850 con 0x7f18bc100550 2023-07-18T03:29:17.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:18.001+0000 7f18b8ff9700 1 -- 172.21.15.40:0/3723937735 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8082 (secure 0 0 0) 0x7f18b4098c60 con 0x7f18bc100550 2023-07-18T03:29:17.997 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:17.998 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/1818167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1818167554},{"type":"v1","addr":"172.21.15.40:6839","nonce":1818167554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3509655945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3509655945},{"type":"v1","addr":"172.21.15.134:6837","nonce":3509655945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24523,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-18T03:29:18.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:18.003+0000 7f18c1d46700 1 -- 172.21.15.40:0/3723937735 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f18a40a1220 msgr2=0x7f18a40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:18.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:18.003+0000 7f18c1d46700 1 --2- 172.21.15.40:0/3723937735 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f18a40a1220 0x7f18a40a36d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0ffa20 tx=0x7f18ac00b410 comp rx=0 tx=0).stop 2023-07-18T03:29:18.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:18.003+0000 7f18c1d46700 1 -- 172.21.15.40:0/3723937735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18bc100550 msgr2=0x7f18bc19bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:18.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:18.003+0000 7f18c1d46700 1 --2- 172.21.15.40:0/3723937735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18bc100550 0x7f18bc19bcc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f18b400ada0 tx=0x7f18b400c330 comp rx=0 tx=0).stop 2023-07-18T03:29:18.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:18.003+0000 7f18c1d46700 1 -- 172.21.15.40:0/3723937735 shutdown_connections 2023-07-18T03:29:18.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:18.003+0000 7f18c1d46700 1 --2- 172.21.15.40:0/3723937735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18bc100550 0x7f18bc19bcc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:18.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:18.003+0000 7f18c1d46700 1 --2- 172.21.15.40:0/3723937735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18bc0fe9d0 0x7f18bc194710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:18.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:18.003+0000 7f18c1d46700 1 --2- 172.21.15.40:0/3723937735 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f18a40a1220 0x7f18a40a36d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:18.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:18.003+0000 7f18c1d46700 1 --2- 172.21.15.40:0/3723937735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18bc0ffbc0 0x7f18bc194c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:18.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:18.003+0000 7f18c1d46700 1 -- 172.21.15.40:0/3723937735 >> 172.21.15.40:0/3723937735 conn(0x7f18bc0f9fd0 msgr2=0x7f18bc0fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:18.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:18.003+0000 7f18c1d46700 1 -- 172.21.15.40:0/3723937735 shutdown_connections 2023-07-18T03:29:18.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:18.003+0000 7f18c1d46700 1 -- 172.21.15.40:0/3723937735 wait complete. 2023-07-18T03:29:18.004 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 11 2023-07-18T03:29:18.011 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jul 18 03:29:17 smithi040 systemd[1]: Starting Ceph mds.a for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:29:18.281 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jul 18 03:29:18 smithi040 podman[131231]: 2023-07-18T03:29:18.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:18 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3723937735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:18.580 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jul 18 03:29:18 smithi040 bash[131231]: dd6806ce7cb5500091d9abddf57c8be18cce99604b008338abcdec5008c81b3d 2023-07-18T03:29:18.580 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jul 18 03:29:18 smithi040 ceph-mds[131273]: -- 172.21.15.40:0/341812131 <== mon.0 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d3179271e0 con 0x55d3178fcc00 2023-07-18T03:29:18.580 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jul 18 03:29:18 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-a[131249]: starting mds.a at 2023-07-18T03:29:18.580 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jul 18 03:29:18 smithi040 systemd[1]: Started Ceph mds.a for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:29:18.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:18 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3723937735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:18.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:18 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3723937735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:19.427 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:19 smithi064 ceph-mon[98059]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:19.427 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:19 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.40:6840/3953074960,v1:172.21.15.40:6841/3953074960] up:boot 2023-07-18T03:29:19.427 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:19 smithi064 ceph-mon[98059]: fsmap 11 up:standby 2023-07-18T03:29:19.427 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-18T03:29:19.427 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:19.427 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:19.428 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:19.428 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:19.428 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:19.428 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:19 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:19.428 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:19 smithi064 ceph-mon[98059]: Deploying daemon mds.b on smithi064 2023-07-18T03:29:19.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:19 smithi134 ceph-mon[98216]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:19.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:19 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.40:6840/3953074960,v1:172.21.15.40:6841/3953074960] up:boot 2023-07-18T03:29:19.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:19 smithi134 ceph-mon[98216]: fsmap 11 up:standby 2023-07-18T03:29:19.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-18T03:29:19.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:19.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:19.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:19.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:19.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:19.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:19 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:19.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:19 smithi134 ceph-mon[98216]: Deploying daemon mds.b on smithi064 2023-07-18T03:29:19.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:19 smithi040 ceph-mon[96439]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:19.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:19 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.40:6840/3953074960,v1:172.21.15.40:6841/3953074960] up:boot 2023-07-18T03:29:19.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:19 smithi040 ceph-mon[96439]: fsmap 11 up:standby 2023-07-18T03:29:19.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-18T03:29:19.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:19.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:19.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:19.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-18T03:29:19.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-18T03:29:19.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:19 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:19.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:19 smithi040 ceph-mon[96439]: Deploying daemon mds.b on smithi064 2023-07-18T03:29:20.013 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:29:20.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.161+0000 7fa88539e700 1 -- 172.21.15.40:0/3129108227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa880101690 msgr2=0x7fa880107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:20.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.161+0000 7fa88539e700 1 --2- 172.21.15.40:0/3129108227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa880101690 0x7fa880107f50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa87800b0a0 tx=0x7fa87800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:20.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.161+0000 7fa88539e700 1 -- 172.21.15.40:0/3129108227 shutdown_connections 2023-07-18T03:29:20.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.161+0000 7fa88539e700 1 --2- 172.21.15.40:0/3129108227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa880101690 0x7fa880107f50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:20.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.161+0000 7fa88539e700 1 --2- 172.21.15.40:0/3129108227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8800ffb10 0x7fa8800fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:20.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.161+0000 7fa88539e700 1 --2- 172.21.15.40:0/3129108227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa880100d00 0x7fa880101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:20.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.161+0000 7fa88539e700 1 -- 172.21.15.40:0/3129108227 >> 172.21.15.40:0/3129108227 conn(0x7fa8800fb110 msgr2=0x7fa8800fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:20.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.161+0000 7fa88539e700 1 -- 172.21.15.40:0/3129108227 shutdown_connections 2023-07-18T03:29:20.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.161+0000 7fa88539e700 1 -- 172.21.15.40:0/3129108227 wait complete. 2023-07-18T03:29:20.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.162+0000 7fa88539e700 1 Processor -- start 2023-07-18T03:29:20.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.162+0000 7fa88539e700 1 -- start start 2023-07-18T03:29:20.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.162+0000 7fa88539e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8800ffb10 0x7fa880197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:20.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.162+0000 7fa88539e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa880100d00 0x7fa880197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:20.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.162+0000 7fa88539e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa880101690 0x7fa88019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:20.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.162+0000 7fa88539e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88019f5f0 con 0x7fa880100d00 2023-07-18T03:29:20.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.162+0000 7fa88539e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88019f730 con 0x7fa880101690 2023-07-18T03:29:20.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.162+0000 7fa88539e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88019f8c0 con 0x7fa8800ffb10 2023-07-18T03:29:20.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.162+0000 7fa87e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa880100d00 0x7fa880197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:20.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.162+0000 7fa87e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa880100d00 0x7fa880197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55974/0 (socket says 172.21.15.40:55974) 2023-07-18T03:29:20.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.162+0000 7fa87e7fc700 1 -- 172.21.15.40:0/718995845 learned_addr learned my addr 172.21.15.40:0/718995845 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:20.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.162+0000 7fa87f7fe700 1 --2- 172.21.15.40:0/718995845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa880101690 0x7fa88019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:20.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.163+0000 7fa87effd700 1 --2- 172.21.15.40:0/718995845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8800ffb10 0x7fa880197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.163+0000 7fa87e7fc700 1 -- 172.21.15.40:0/718995845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8800ffb10 msgr2=0x7fa880197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.163+0000 7fa87e7fc700 1 --2- 172.21.15.40:0/718995845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8800ffb10 0x7fa880197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.163+0000 7fa87e7fc700 1 -- 172.21.15.40:0/718995845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa880101690 msgr2=0x7fa88019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.163+0000 7fa87e7fc700 1 --2- 172.21.15.40:0/718995845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa880101690 0x7fa88019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.163+0000 7fa87e7fc700 1 -- 172.21.15.40:0/718995845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa878009d00 con 0x7fa880100d00 2023-07-18T03:29:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.163+0000 7fa87f7fe700 1 --2- 172.21.15.40:0/718995845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa880101690 0x7fa88019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.163+0000 7fa87e7fc700 1 --2- 172.21.15.40:0/718995845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa880100d00 0x7fa880197fb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa87400e9a0 tx=0x7fa87400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.163+0000 7fa867fff700 1 -- 172.21.15.40:0/718995845 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874009790 con 0x7fa880100d00 2023-07-18T03:29:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.163+0000 7fa867fff700 1 -- 172.21.15.40:0/718995845 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa87400cbd0 con 0x7fa880100d00 2023-07-18T03:29:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.163+0000 7fa88539e700 1 -- 172.21.15.40:0/718995845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88019fba0 con 0x7fa880100d00 2023-07-18T03:29:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.163+0000 7fa867fff700 1 -- 172.21.15.40:0/718995845 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874018680 con 0x7fa880100d00 2023-07-18T03:29:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.163+0000 7fa88539e700 1 -- 172.21.15.40:0/718995845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8801a00b0 con 0x7fa880100d00 2023-07-18T03:29:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.164+0000 7fa88539e700 1 -- 172.21.15.40:0/718995845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88010b2f0 con 0x7fa880100d00 2023-07-18T03:29:20.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.165+0000 7fa867fff700 1 -- 172.21.15.40:0/718995845 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa87400cd40 con 0x7fa880100d00 2023-07-18T03:29:20.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.166+0000 7fa867fff700 1 --2- 172.21.15.40:0/718995845 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa8680a11a0 0x7fa8680a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:20.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.166+0000 7fa867fff700 1 -- 172.21.15.40:0/718995845 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa874014070 con 0x7fa880100d00 2023-07-18T03:29:20.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.166+0000 7fa87effd700 1 --2- 172.21.15.40:0/718995845 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa8680a11a0 0x7fa8680a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:20.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.166+0000 7fa87effd700 1 --2- 172.21.15.40:0/718995845 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa8680a11a0 0x7fa8680a3650 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa870005fd0 tx=0x7fa870005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:20.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.170+0000 7fa867fff700 1 -- 172.21.15.40:0/718995845 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa874051790 con 0x7fa880100d00 2023-07-18T03:29:20.351 INFO:journalctl@ceph.mds.b.smithi064.stdout:Jul 18 03:29:20 smithi064 systemd[1]: Starting Ceph mds.b for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:29:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.419+0000 7fa88539e700 1 -- 172.21.15.40:0/718995845 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8801a0660 con 0x7fa880100d00 2023-07-18T03:29:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.420+0000 7fa867fff700 1 -- 172.21.15.40:0/718995845 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8840 (secure 0 0 0) 0x7fa874096300 con 0x7fa880100d00 2023-07-18T03:29:20.416 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:20.416 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/1818167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1818167554},{"type":"v1","addr":"172.21.15.40:6839","nonce":1818167554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6841/3953074960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":3953074960},{"type":"v1","addr":"172.21.15.40:6841","nonce":3953074960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3509655945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3509655945},{"type":"v1","addr":"172.21.15.134:6837","nonce":3509655945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24523,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-07-18T03:29:20.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.422+0000 7fa88539e700 1 -- 172.21.15.40:0/718995845 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa8680a11a0 msgr2=0x7fa8680a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:20.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.422+0000 7fa88539e700 1 --2- 172.21.15.40:0/718995845 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa8680a11a0 0x7fa8680a3650 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa870005fd0 tx=0x7fa870005dd0 comp rx=0 tx=0).stop 2023-07-18T03:29:20.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.422+0000 7fa88539e700 1 -- 172.21.15.40:0/718995845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa880100d00 msgr2=0x7fa880197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:20.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.422+0000 7fa88539e700 1 --2- 172.21.15.40:0/718995845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa880100d00 0x7fa880197fb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa87400e9a0 tx=0x7fa87400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:29:20.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.422+0000 7fa88539e700 1 -- 172.21.15.40:0/718995845 shutdown_connections 2023-07-18T03:29:20.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.422+0000 7fa88539e700 1 --2- 172.21.15.40:0/718995845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa880101690 0x7fa88019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:20.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.422+0000 7fa88539e700 1 --2- 172.21.15.40:0/718995845 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa8680a11a0 0x7fa8680a3650 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:20.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.422+0000 7fa88539e700 1 --2- 172.21.15.40:0/718995845 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8800ffb10 0x7fa880197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:20.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.422+0000 7fa88539e700 1 --2- 172.21.15.40:0/718995845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa880100d00 0x7fa880197fb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:20.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.422+0000 7fa88539e700 1 -- 172.21.15.40:0/718995845 >> 172.21.15.40:0/718995845 conn(0x7fa8800fb110 msgr2=0x7fa8801064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:20.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.422+0000 7fa88539e700 1 -- 172.21.15.40:0/718995845 shutdown_connections 2023-07-18T03:29:20.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:20.422+0000 7fa88539e700 1 -- 172.21.15.40:0/718995845 wait complete. 2023-07-18T03:29:20.420 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 12 2023-07-18T03:29:20.617 INFO:journalctl@ceph.mds.b.smithi064.stdout:Jul 18 03:29:20 smithi064 podman[123401]: 2023-07-18T03:29:21.100 INFO:journalctl@ceph.mds.b.smithi064.stdout:Jul 18 03:29:20 smithi064 bash[123401]: 789ae0704aeb9f3d674019cfbddd0e4c77a2b477eb6e9f1f7c2a0f65d598c182 2023-07-18T03:29:21.101 INFO:journalctl@ceph.mds.b.smithi064.stdout:Jul 18 03:29:20 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-b[123417]: starting mds.b at 2023-07-18T03:29:21.101 INFO:journalctl@ceph.mds.b.smithi064.stdout:Jul 18 03:29:20 smithi064 systemd[1]: Started Ceph mds.b for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:29:21.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:21 smithi064 ceph-mon[98059]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:21.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:21 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/718995845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:21.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:21.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:21.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:21.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:21.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:21.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:21 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:29:21.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:21 smithi134 ceph-mon[98216]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:21.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:21 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/718995845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:21.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:21.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:21.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:21.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:21.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:21.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:21 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:29:21.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:21 smithi040 ceph-mon[96439]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:21.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:21 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/718995845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:21.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:21.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:21.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:21.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:21.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:21.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:21 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:29:22.431 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:29:22.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.567+0000 7f40d1f6e700 1 -- 172.21.15.40:0/499700830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40cc101690 msgr2=0x7f40cc107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:22.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.567+0000 7f40d1f6e700 1 --2- 172.21.15.40:0/499700830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40cc101690 0x7f40cc107f50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f40c400b0a0 tx=0x7f40c400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:22.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.567+0000 7f40d1f6e700 1 -- 172.21.15.40:0/499700830 shutdown_connections 2023-07-18T03:29:22.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.567+0000 7f40d1f6e700 1 --2- 172.21.15.40:0/499700830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40cc101690 0x7f40cc107f50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:22.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.567+0000 7f40d1f6e700 1 --2- 172.21.15.40:0/499700830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc0ffb10 0x7f40cc0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:22.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.567+0000 7f40d1f6e700 1 --2- 172.21.15.40:0/499700830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40cc100d00 0x7f40cc101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:22.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.567+0000 7f40d1f6e700 1 -- 172.21.15.40:0/499700830 >> 172.21.15.40:0/499700830 conn(0x7f40cc0fb130 msgr2=0x7f40cc0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:22.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.567+0000 7f40d1f6e700 1 -- 172.21.15.40:0/499700830 shutdown_connections 2023-07-18T03:29:22.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.567+0000 7f40d1f6e700 1 -- 172.21.15.40:0/499700830 wait complete. 2023-07-18T03:29:22.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.567+0000 7f40d1f6e700 1 Processor -- start 2023-07-18T03:29:22.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.567+0000 7f40d1f6e700 1 -- start start 2023-07-18T03:29:22.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40d1f6e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc0ffb10 0x7f40cc197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:22.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40d1f6e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40cc100d00 0x7f40cc197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:22.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40d1f6e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40cc101690 0x7f40cc19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:22.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40d1f6e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc19f610 con 0x7f40cc100d00 2023-07-18T03:29:22.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40d1f6e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc19f750 con 0x7f40cc0ffb10 2023-07-18T03:29:22.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40d1f6e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc19f8e0 con 0x7f40cc101690 2023-07-18T03:29:22.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40caffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40cc100d00 0x7f40cc197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:22.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40caffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40cc100d00 0x7f40cc197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55986/0 (socket says 172.21.15.40:55986) 2023-07-18T03:29:22.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40cb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc0ffb10 0x7f40cc197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:22.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40caffd700 1 -- 172.21.15.40:0/2452565659 learned_addr learned my addr 172.21.15.40:0/2452565659 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:22.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40cbfff700 1 --2- 172.21.15.40:0/2452565659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40cc101690 0x7f40cc19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:22.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40cb7fe700 1 -- 172.21.15.40:0/2452565659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40cc101690 msgr2=0x7f40cc19f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:22.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40cb7fe700 1 --2- 172.21.15.40:0/2452565659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40cc101690 0x7f40cc19f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:22.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40cb7fe700 1 -- 172.21.15.40:0/2452565659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40cc100d00 msgr2=0x7f40cc197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:22.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40cb7fe700 1 --2- 172.21.15.40:0/2452565659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40cc100d00 0x7f40cc197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:22.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40cb7fe700 1 -- 172.21.15.40:0/2452565659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c4009d00 con 0x7f40cc0ffb10 2023-07-18T03:29:22.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40cbfff700 1 --2- 172.21.15.40:0/2452565659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40cc101690 0x7f40cc19f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:22.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40caffd700 1 --2- 172.21.15.40:0/2452565659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40cc100d00 0x7f40cc197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:22.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40cb7fe700 1 --2- 172.21.15.40:0/2452565659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc0ffb10 0x7f40cc197a90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00e9c0 tx=0x7f40bc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:22.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.568+0000 7f40c8ff9700 1 -- 172.21.15.40:0/2452565659 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc004220 con 0x7f40cc0ffb10 2023-07-18T03:29:22.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.569+0000 7f40d1f6e700 1 -- 172.21.15.40:0/2452565659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40cc19fbc0 con 0x7f40cc0ffb10 2023-07-18T03:29:22.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.569+0000 7f40c8ff9700 1 -- 172.21.15.40:0/2452565659 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40bc004380 con 0x7f40cc0ffb10 2023-07-18T03:29:22.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.569+0000 7f40c8ff9700 1 -- 172.21.15.40:0/2452565659 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc003be0 con 0x7f40cc0ffb10 2023-07-18T03:29:22.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.569+0000 7f40d1f6e700 1 -- 172.21.15.40:0/2452565659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40cc1a00d0 con 0x7f40cc0ffb10 2023-07-18T03:29:22.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.569+0000 7f40d1f6e700 1 -- 172.21.15.40:0/2452565659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40cc10b2f0 con 0x7f40cc0ffb10 2023-07-18T03:29:22.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.571+0000 7f40c8ff9700 1 -- 172.21.15.40:0/2452565659 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f40bc00c7c0 con 0x7f40cc0ffb10 2023-07-18T03:29:22.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.571+0000 7f40c8ff9700 1 --2- 172.21.15.40:0/2452565659 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f40b80a11a0 0x7f40b80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:22.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.571+0000 7f40caffd700 1 --2- 172.21.15.40:0/2452565659 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f40b80a11a0 0x7f40b80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:22.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.571+0000 7f40c8ff9700 1 -- 172.21.15.40:0/2452565659 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f40bc014070 con 0x7f40cc0ffb10 2023-07-18T03:29:22.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.572+0000 7f40caffd700 1 --2- 172.21.15.40:0/2452565659 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f40b80a11a0 0x7f40b80a3650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f40b4011550 tx=0x7f40b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:22.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.575+0000 7f40c8ff9700 1 -- 172.21.15.40:0/2452565659 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f40bc096910 con 0x7f40cc0ffb10 2023-07-18T03:29:22.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:22 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] up:boot 2023-07-18T03:29:22.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:22 smithi064 ceph-mon[98059]: fsmap 12 up:standby 2023-07-18T03:29:22.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:22 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-18T03:29:22.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:22 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:22.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:22 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] up:boot 2023-07-18T03:29:22.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:22 smithi134 ceph-mon[98216]: fsmap 12 up:standby 2023-07-18T03:29:22.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:22 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-18T03:29:22.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:22 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:22.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.829+0000 7f40d1f6e700 1 -- 172.21.15.40:0/2452565659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40cc1a0880 con 0x7f40cc0ffb10 2023-07-18T03:29:22.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.830+0000 7f40c8ff9700 1 -- 172.21.15.40:0/2452565659 <== mon.2 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9598 (secure 0 0 0) 0x7f40bc099f30 con 0x7f40cc0ffb10 2023-07-18T03:29:22.827 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:22.827 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/1818167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1818167554},{"type":"v1","addr":"172.21.15.40:6839","nonce":1818167554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6841/3953074960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":3953074960},{"type":"v1","addr":"172.21.15.40:6841","nonce":3953074960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3509655945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3509655945},{"type":"v1","addr":"172.21.15.134:6837","nonce":3509655945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24523,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/3007039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3007039043},{"type":"v1","addr":"172.21.15.64:6839","nonce":3007039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-07-18T03:29:22.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.832+0000 7f40d1f6e700 1 -- 172.21.15.40:0/2452565659 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f40b80a11a0 msgr2=0x7f40b80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:22.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.832+0000 7f40d1f6e700 1 --2- 172.21.15.40:0/2452565659 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f40b80a11a0 0x7f40b80a3650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f40b4011550 tx=0x7f40b4009260 comp rx=0 tx=0).stop 2023-07-18T03:29:22.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.832+0000 7f40d1f6e700 1 -- 172.21.15.40:0/2452565659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc0ffb10 msgr2=0x7f40cc197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:22.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.832+0000 7f40d1f6e700 1 --2- 172.21.15.40:0/2452565659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc0ffb10 0x7f40cc197a90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00e9c0 tx=0x7f40bc00ecd0 comp rx=0 tx=0).stop 2023-07-18T03:29:22.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.832+0000 7f40d1f6e700 1 -- 172.21.15.40:0/2452565659 shutdown_connections 2023-07-18T03:29:22.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.832+0000 7f40d1f6e700 1 --2- 172.21.15.40:0/2452565659 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f40b80a11a0 0x7f40b80a3650 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:22.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.832+0000 7f40d1f6e700 1 --2- 172.21.15.40:0/2452565659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40cc101690 0x7f40cc19f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:22.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.832+0000 7f40d1f6e700 1 --2- 172.21.15.40:0/2452565659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40cc0ffb10 0x7f40cc197a90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:22.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.832+0000 7f40d1f6e700 1 --2- 172.21.15.40:0/2452565659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40cc100d00 0x7f40cc197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:22.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.832+0000 7f40d1f6e700 1 -- 172.21.15.40:0/2452565659 >> 172.21.15.40:0/2452565659 conn(0x7f40cc0fb130 msgr2=0x7f40cc1064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:22.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.832+0000 7f40d1f6e700 1 -- 172.21.15.40:0/2452565659 shutdown_connections 2023-07-18T03:29:22.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.832+0000 7f40d1f6e700 1 -- 172.21.15.40:0/2452565659 wait complete. 2023-07-18T03:29:22.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:22 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] up:boot 2023-07-18T03:29:22.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:22 smithi040 ceph-mon[96439]: fsmap 12 up:standby 2023-07-18T03:29:22.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:22 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-18T03:29:22.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:22 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:22.831 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 13 2023-07-18T03:29:22.841 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:29:22.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.977+0000 7ff876627700 1 -- 172.21.15.40:0/573676679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8701005b0 msgr2=0x7ff870102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:22.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.977+0000 7ff876627700 1 --2- 172.21.15.40:0/573676679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8701005b0 0x7ff870102990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff860009990 tx=0x7ff860009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:22.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.977+0000 7ff876627700 1 -- 172.21.15.40:0/573676679 shutdown_connections 2023-07-18T03:29:22.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.977+0000 7ff876627700 1 --2- 172.21.15.40:0/573676679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8701057f0 0x7ff870107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:22.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.977+0000 7ff876627700 1 --2- 172.21.15.40:0/573676679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8701005b0 0x7ff870102990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:22.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.977+0000 7ff876627700 1 --2- 172.21.15.40:0/573676679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff870102ed0 0x7ff8701052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:22.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.977+0000 7ff876627700 1 -- 172.21.15.40:0/573676679 >> 172.21.15.40:0/573676679 conn(0x7ff8700f9fd0 msgr2=0x7ff8700fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:22.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.977+0000 7ff876627700 1 -- 172.21.15.40:0/573676679 shutdown_connections 2023-07-18T03:29:22.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.978+0000 7ff876627700 1 -- 172.21.15.40:0/573676679 wait complete. 2023-07-18T03:29:22.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.978+0000 7ff876627700 1 Processor -- start 2023-07-18T03:29:22.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.978+0000 7ff876627700 1 -- start start 2023-07-18T03:29:22.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff876627700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8701005b0 0x7ff8701957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:22.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff876627700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff870102ed0 0x7ff870195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:22.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff876627700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8701057f0 0x7ff87019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:22.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff876627700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87019d330 con 0x7ff8701057f0 2023-07-18T03:29:22.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff876627700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87019d470 con 0x7ff8701005b0 2023-07-18T03:29:22.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff876627700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87019d600 con 0x7ff870102ed0 2023-07-18T03:29:22.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff874bc4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8701057f0 0x7ff87019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:22.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff874bc4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8701057f0 0x7ff87019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56002/0 (socket says 172.21.15.40:56002) 2023-07-18T03:29:22.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff874bc4700 1 -- 172.21.15.40:0/2348370107 learned_addr learned my addr 172.21.15.40:0/2348370107 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:22.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff86f7fe700 1 --2- 172.21.15.40:0/2348370107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff870102ed0 0x7ff870195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:22.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff86ffff700 1 --2- 172.21.15.40:0/2348370107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8701005b0 0x7ff8701957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:22.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff874bc4700 1 -- 172.21.15.40:0/2348370107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff870102ed0 msgr2=0x7ff870195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:22.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff874bc4700 1 --2- 172.21.15.40:0/2348370107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff870102ed0 0x7ff870195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:22.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff874bc4700 1 -- 172.21.15.40:0/2348370107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8701005b0 msgr2=0x7ff8701957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:22.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff874bc4700 1 --2- 172.21.15.40:0/2348370107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8701005b0 0x7ff8701957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:22.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff874bc4700 1 -- 172.21.15.40:0/2348370107 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8600095f0 con 0x7ff8701057f0 2023-07-18T03:29:22.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff874bc4700 1 --2- 172.21.15.40:0/2348370107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8701057f0 0x7ff87019cd60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff86800ef30 tx=0x7ff86800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.979+0000 7ff86d7fa700 1 -- 172.21.15.40:0/2348370107 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868018070 con 0x7ff8701057f0 2023-07-18T03:29:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.980+0000 7ff876627700 1 -- 172.21.15.40:0/2348370107 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87019d880 con 0x7ff8701057f0 2023-07-18T03:29:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.980+0000 7ff86d7fa700 1 -- 172.21.15.40:0/2348370107 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff86800f040 con 0x7ff8701057f0 2023-07-18T03:29:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.980+0000 7ff86d7fa700 1 -- 172.21.15.40:0/2348370107 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86800cd00 con 0x7ff8701057f0 2023-07-18T03:29:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.980+0000 7ff876627700 1 -- 172.21.15.40:0/2348370107 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87019dde0 con 0x7ff8701057f0 2023-07-18T03:29:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.980+0000 7ff876627700 1 -- 172.21.15.40:0/2348370107 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87010b280 con 0x7ff8701057f0 2023-07-18T03:29:22.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.981+0000 7ff86d7fa700 1 -- 172.21.15.40:0/2348370107 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff868003a00 con 0x7ff8701057f0 2023-07-18T03:29:22.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.982+0000 7ff86d7fa700 1 --2- 172.21.15.40:0/2348370107 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff8580a11a0 0x7ff8580a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:22.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.982+0000 7ff86ffff700 1 --2- 172.21.15.40:0/2348370107 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff8580a11a0 0x7ff8580a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:22.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.982+0000 7ff86d7fa700 1 -- 172.21.15.40:0/2348370107 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff8680cbf90 con 0x7ff8701057f0 2023-07-18T03:29:22.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.983+0000 7ff86ffff700 1 --2- 172.21.15.40:0/2348370107 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff8580a11a0 0x7ff8580a3650 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b2e0 tx=0x7ff860005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:22.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:22.987+0000 7ff86d7fa700 1 -- 172.21.15.40:0/2348370107 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff868095c10 con 0x7ff8701057f0 2023-07-18T03:29:23.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.193+0000 7ff876627700 1 -- 172.21.15.40:0/2348370107 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff87019e3c0 con 0x7ff8701057f0 2023-07-18T03:29:23.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.194+0000 7ff86d7fa700 1 -- 172.21.15.40:0/2348370107 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18374 (secure 0 0 0) 0x7ff868099230 con 0x7ff8701057f0 2023-07-18T03:29:23.191 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:23.191 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":77,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:28:46.697008+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:29:23.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.196+0000 7ff876627700 1 -- 172.21.15.40:0/2348370107 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff8580a11a0 msgr2=0x7ff8580a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:23.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.196+0000 7ff876627700 1 --2- 172.21.15.40:0/2348370107 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff8580a11a0 0x7ff8580a3650 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b2e0 tx=0x7ff860005fb0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.196+0000 7ff876627700 1 -- 172.21.15.40:0/2348370107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8701057f0 msgr2=0x7ff87019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:23.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.196+0000 7ff876627700 1 --2- 172.21.15.40:0/2348370107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8701057f0 0x7ff87019cd60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff86800ef30 tx=0x7ff86800c3a0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.196+0000 7ff876627700 1 -- 172.21.15.40:0/2348370107 shutdown_connections 2023-07-18T03:29:23.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.196+0000 7ff876627700 1 --2- 172.21.15.40:0/2348370107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8701057f0 0x7ff87019cd60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.196+0000 7ff876627700 1 --2- 172.21.15.40:0/2348370107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8701005b0 0x7ff8701957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.196+0000 7ff876627700 1 --2- 172.21.15.40:0/2348370107 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff8580a11a0 0x7ff8580a3650 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.196+0000 7ff876627700 1 --2- 172.21.15.40:0/2348370107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff870102ed0 0x7ff870195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.196+0000 7ff876627700 1 -- 172.21.15.40:0/2348370107 >> 172.21.15.40:0/2348370107 conn(0x7ff8700f9fd0 msgr2=0x7ff8701019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:23.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.196+0000 7ff876627700 1 -- 172.21.15.40:0/2348370107 shutdown_connections 2023-07-18T03:29:23.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.196+0000 7ff876627700 1 -- 172.21.15.40:0/2348370107 wait complete. 2023-07-18T03:29:23.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:25:38.267262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:29:23.205 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:29:23.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.341+0000 7fb831998700 1 -- 172.21.15.40:0/1984846996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb82c0fea40 msgr2=0x7fb82c0fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.341+0000 7fb831998700 1 --2- 172.21.15.40:0/1984846996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb82c0fea40 0x7fb82c0fee50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009990 tx=0x7fb81c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.341+0000 7fb831998700 1 -- 172.21.15.40:0/1984846996 shutdown_connections 2023-07-18T03:29:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.341+0000 7fb831998700 1 --2- 172.21.15.40:0/1984846996 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb82c1005c0 0x7fb82c106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.341+0000 7fb831998700 1 --2- 172.21.15.40:0/1984846996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb82c0fea40 0x7fb82c0fee50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.341+0000 7fb831998700 1 --2- 172.21.15.40:0/1984846996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb82c0ffc30 0x7fb82c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.341+0000 7fb831998700 1 -- 172.21.15.40:0/1984846996 >> 172.21.15.40:0/1984846996 conn(0x7fb82c0fa0a0 msgr2=0x7fb82c0fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.341+0000 7fb831998700 1 -- 172.21.15.40:0/1984846996 shutdown_connections 2023-07-18T03:29:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.341+0000 7fb831998700 1 -- 172.21.15.40:0/1984846996 wait complete. 2023-07-18T03:29:23.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.341+0000 7fb831998700 1 Processor -- start 2023-07-18T03:29:23.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.342+0000 7fb831998700 1 -- start start 2023-07-18T03:29:23.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.342+0000 7fb831998700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb82c0fea40 0x7fb82c194710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:23.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.342+0000 7fb831998700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb82c0ffc30 0x7fb82c194c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:23.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.342+0000 7fb831998700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb82c1005c0 0x7fb82c19bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:23.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.342+0000 7fb831998700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c19c290 con 0x7fb82c0fea40 2023-07-18T03:29:23.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.342+0000 7fb831998700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c19c3d0 con 0x7fb82c0ffc30 2023-07-18T03:29:23.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.342+0000 7fb831998700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c19c560 con 0x7fb82c1005c0 2023-07-18T03:29:23.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.342+0000 7fb82affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb82c0fea40 0x7fb82c194710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:23.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.342+0000 7fb82affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb82c0fea40 0x7fb82c194710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56012/0 (socket says 172.21.15.40:56012) 2023-07-18T03:29:23.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.342+0000 7fb82affd700 1 -- 172.21.15.40:0/4046553445 learned_addr learned my addr 172.21.15.40:0/4046553445 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:23.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.342+0000 7fb82a7fc700 1 --2- 172.21.15.40:0/4046553445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb82c0ffc30 0x7fb82c194c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:23.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.342+0000 7fb82b7fe700 1 --2- 172.21.15.40:0/4046553445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb82c1005c0 0x7fb82c19bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:23.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.342+0000 7fb82affd700 1 -- 172.21.15.40:0/4046553445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb82c1005c0 msgr2=0x7fb82c19bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:23.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.342+0000 7fb82affd700 1 --2- 172.21.15.40:0/4046553445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb82c1005c0 0x7fb82c19bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.342+0000 7fb82affd700 1 -- 172.21.15.40:0/4046553445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb82c0ffc30 msgr2=0x7fb82c194c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:23.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.342+0000 7fb82affd700 1 --2- 172.21.15.40:0/4046553445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb82c0ffc30 0x7fb82c194c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.342+0000 7fb82affd700 1 -- 172.21.15.40:0/4046553445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c0095f0 con 0x7fb82c0fea40 2023-07-18T03:29:23.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.343+0000 7fb82affd700 1 --2- 172.21.15.40:0/4046553445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb82c0fea40 0x7fb82c194710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb81c000c00 tx=0x7fb81c0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:23.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.343+0000 7fb830996700 1 -- 172.21.15.40:0/4046553445 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c01f070 con 0x7fb82c0fea40 2023-07-18T03:29:23.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.343+0000 7fb831998700 1 -- 172.21.15.40:0/4046553445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82c0688c0 con 0x7fb82c0fea40 2023-07-18T03:29:23.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.343+0000 7fb830996700 1 -- 172.21.15.40:0/4046553445 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb81c00f460 con 0x7fb82c0fea40 2023-07-18T03:29:23.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.343+0000 7fb830996700 1 -- 172.21.15.40:0/4046553445 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c025680 con 0x7fb82c0fea40 2023-07-18T03:29:23.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.343+0000 7fb831998700 1 -- 172.21.15.40:0/4046553445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82c068dc0 con 0x7fb82c0fea40 2023-07-18T03:29:23.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.344+0000 7fb831998700 1 -- 172.21.15.40:0/4046553445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82c10a1a0 con 0x7fb82c0fea40 2023-07-18T03:29:23.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.345+0000 7fb830996700 1 -- 172.21.15.40:0/4046553445 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb81c023070 con 0x7fb82c0fea40 2023-07-18T03:29:23.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.345+0000 7fb830996700 1 --2- 172.21.15.40:0/4046553445 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb8140a1150 0x7fb8140a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:23.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.346+0000 7fb830996700 1 -- 172.21.15.40:0/4046553445 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb81c0cfb30 con 0x7fb82c0fea40 2023-07-18T03:29:23.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.346+0000 7fb82a7fc700 1 --2- 172.21.15.40:0/4046553445 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb8140a1150 0x7fb8140a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:23.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.346+0000 7fb82a7fc700 1 --2- 172.21.15.40:0/4046553445 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb8140a1150 0x7fb8140a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb820009c60 tx=0x7fb820009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:23.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.349+0000 7fb830996700 1 -- 172.21.15.40:0/4046553445 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb81c0998d0 con 0x7fb82c0fea40 2023-07-18T03:29:23.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.565+0000 7fb831998700 1 -- 172.21.15.40:0/4046553445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb82c0696b0 con 0x7fb82c0fea40 2023-07-18T03:29:23.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.565+0000 7fb830996700 1 -- 172.21.15.40:0/4046553445 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb81c00f5d0 con 0x7fb82c0fea40 2023-07-18T03:29:23.562 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-07-18T03:29:23.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.567+0000 7fb831998700 1 -- 172.21.15.40:0/4046553445 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb8140a1150 msgr2=0x7fb8140a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:23.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.567+0000 7fb831998700 1 --2- 172.21.15.40:0/4046553445 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb8140a1150 0x7fb8140a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb820009c60 tx=0x7fb820009260 comp rx=0 tx=0).stop 2023-07-18T03:29:23.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.567+0000 7fb831998700 1 -- 172.21.15.40:0/4046553445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb82c0fea40 msgr2=0x7fb82c194710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:23.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.567+0000 7fb831998700 1 --2- 172.21.15.40:0/4046553445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb82c0fea40 0x7fb82c194710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb81c000c00 tx=0x7fb81c0047a0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.568+0000 7fb831998700 1 -- 172.21.15.40:0/4046553445 shutdown_connections 2023-07-18T03:29:23.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.568+0000 7fb831998700 1 --2- 172.21.15.40:0/4046553445 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb8140a1150 0x7fb8140a3600 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.568+0000 7fb831998700 1 --2- 172.21.15.40:0/4046553445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb82c1005c0 0x7fb82c19bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.568+0000 7fb831998700 1 --2- 172.21.15.40:0/4046553445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb82c0fea40 0x7fb82c194710 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.568+0000 7fb831998700 1 --2- 172.21.15.40:0/4046553445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb82c0ffc30 0x7fb82c194c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.568+0000 7fb831998700 1 -- 172.21.15.40:0/4046553445 >> 172.21.15.40:0/4046553445 conn(0x7fb82c0fa0a0 msgr2=0x7fb82c0fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:23.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.568+0000 7fb831998700 1 -- 172.21.15.40:0/4046553445 shutdown_connections 2023-07-18T03:29:23.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.568+0000 7fb831998700 1 -- 172.21.15.40:0/4046553445 wait complete. 2023-07-18T03:29:23.578 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-18T03:29:23.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.712+0000 7fd68ada6700 1 -- 172.21.15.40:0/1246370015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6840ffb00 msgr2=0x7fd6840fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:23.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.712+0000 7fd68ada6700 1 --2- 172.21.15.40:0/1246370015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6840ffb00 0x7fd6840fff10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd67400b0a0 tx=0x7fd67400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.712+0000 7fd68ada6700 1 -- 172.21.15.40:0/1246370015 shutdown_connections 2023-07-18T03:29:23.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.712+0000 7fd68ada6700 1 --2- 172.21.15.40:0/1246370015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd684101680 0x7fd684107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.712+0000 7fd68ada6700 1 --2- 172.21.15.40:0/1246370015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6840ffb00 0x7fd6840fff10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.712+0000 7fd68ada6700 1 --2- 172.21.15.40:0/1246370015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd684100cf0 0x7fd684101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.712+0000 7fd68ada6700 1 -- 172.21.15.40:0/1246370015 >> 172.21.15.40:0/1246370015 conn(0x7fd6840fb0e0 msgr2=0x7fd6840fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:23.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.712+0000 7fd68ada6700 1 -- 172.21.15.40:0/1246370015 shutdown_connections 2023-07-18T03:29:23.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.712+0000 7fd68ada6700 1 -- 172.21.15.40:0/1246370015 wait complete. 2023-07-18T03:29:23.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd68ada6700 1 Processor -- start 2023-07-18T03:29:23.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd68ada6700 1 -- start start 2023-07-18T03:29:23.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd68ada6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6840ffb00 0x7fd684195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:23.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd68ada6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd684100cf0 0x7fd684195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:23.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd68ada6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd684101680 0x7fd68419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:23.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd68ada6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68419d4a0 con 0x7fd684100cf0 2023-07-18T03:29:23.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd68ada6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68419d5e0 con 0x7fd684101680 2023-07-18T03:29:23.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd68ada6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68419d770 con 0x7fd6840ffb00 2023-07-18T03:29:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd683fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd684100cf0 0x7fd684195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd683fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd684100cf0 0x7fd684195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56028/0 (socket says 172.21.15.40:56028) 2023-07-18T03:29:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd683fff700 1 -- 172.21.15.40:0/3202944876 learned_addr learned my addr 172.21.15.40:0/3202944876 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd688b42700 1 --2- 172.21.15.40:0/3202944876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6840ffb00 0x7fd684195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd689343700 1 --2- 172.21.15.40:0/3202944876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd684101680 0x7fd68419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd683fff700 1 -- 172.21.15.40:0/3202944876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6840ffb00 msgr2=0x7fd684195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:23.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd683fff700 1 --2- 172.21.15.40:0/3202944876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6840ffb00 0x7fd684195920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd683fff700 1 -- 172.21.15.40:0/3202944876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd684101680 msgr2=0x7fd68419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd683fff700 1 --2- 172.21.15.40:0/3202944876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd684101680 0x7fd68419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd683fff700 1 -- 172.21.15.40:0/3202944876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd674009d00 con 0x7fd684100cf0 2023-07-18T03:29:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.713+0000 7fd683fff700 1 --2- 172.21.15.40:0/3202944876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd684100cf0 0x7fd684195e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00e9a0 tx=0x7fd66c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.714+0000 7fd681ffb700 1 -- 172.21.15.40:0/3202944876 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c009790 con 0x7fd684100cf0 2023-07-18T03:29:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.714+0000 7fd681ffb700 1 -- 172.21.15.40:0/3202944876 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd66c010430 con 0x7fd684100cf0 2023-07-18T03:29:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.714+0000 7fd68ada6700 1 -- 172.21.15.40:0/3202944876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6840727a0 con 0x7fd684100cf0 2023-07-18T03:29:23.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.714+0000 7fd681ffb700 1 -- 172.21.15.40:0/3202944876 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c018680 con 0x7fd684100cf0 2023-07-18T03:29:23.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.714+0000 7fd68ada6700 1 -- 172.21.15.40:0/3202944876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd684072d00 con 0x7fd684100cf0 2023-07-18T03:29:23.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.715+0000 7fd68ada6700 1 -- 172.21.15.40:0/3202944876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd68410b2d0 con 0x7fd684100cf0 2023-07-18T03:29:23.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.717+0000 7fd681ffb700 1 -- 172.21.15.40:0/3202944876 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd66c0105a0 con 0x7fd684100cf0 2023-07-18T03:29:23.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.717+0000 7fd681ffb700 1 --2- 172.21.15.40:0/3202944876 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd6700a1220 0x7fd6700a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:23.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.717+0000 7fd681ffb700 1 -- 172.21.15.40:0/3202944876 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd66c014070 con 0x7fd684100cf0 2023-07-18T03:29:23.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.717+0000 7fd688b42700 1 --2- 172.21.15.40:0/3202944876 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd6700a1220 0x7fd6700a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:23.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.718+0000 7fd688b42700 1 --2- 172.21.15.40:0/3202944876 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd6700a1220 0x7fd6700a36d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd6740079f0 tx=0x7fd67400b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:23.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.721+0000 7fd681ffb700 1 -- 172.21.15.40:0/3202944876 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd66c0963d0 con 0x7fd684100cf0 2023-07-18T03:29:23.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:23 smithi040 ceph-mon[96439]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:23.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:23 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2452565659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:23.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:23 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2348370107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:23.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:23 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/4046553445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:29:23.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:23 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:23.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:23 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:23.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:23 smithi064 ceph-mon[98059]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:23.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:23 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2452565659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:23.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:23 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2348370107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:23.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:23 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/4046553445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:29:23.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:23.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:23 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.930+0000 7fd68ada6700 1 -- 172.21.15.40:0/3202944876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd68410b4c0 con 0x7fd684100cf0 2023-07-18T03:29:23.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.930+0000 7fd681ffb700 1 -- 172.21.15.40:0/3202944876 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd66c0999f0 con 0x7fd684100cf0 2023-07-18T03:29:23.927 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:23.927 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-07-18T03:29:23.927 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:29:23.927 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2023-07-18T03:29:23.928 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2023-07-18T03:29:23.928 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:29:23.928 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-07-18T03:29:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.933+0000 7fd68ada6700 1 -- 172.21.15.40:0/3202944876 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd6700a1220 msgr2=0x7fd6700a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.933+0000 7fd68ada6700 1 --2- 172.21.15.40:0/3202944876 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd6700a1220 0x7fd6700a36d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd6740079f0 tx=0x7fd67400b700 comp rx=0 tx=0).stop 2023-07-18T03:29:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.933+0000 7fd68ada6700 1 -- 172.21.15.40:0/3202944876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd684100cf0 msgr2=0x7fd684195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.933+0000 7fd68ada6700 1 --2- 172.21.15.40:0/3202944876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd684100cf0 0x7fd684195e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00e9a0 tx=0x7fd66c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.933+0000 7fd68ada6700 1 -- 172.21.15.40:0/3202944876 shutdown_connections 2023-07-18T03:29:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.933+0000 7fd68ada6700 1 --2- 172.21.15.40:0/3202944876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd684101680 0x7fd68419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.933+0000 7fd68ada6700 1 --2- 172.21.15.40:0/3202944876 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd6700a1220 0x7fd6700a36d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.933+0000 7fd68ada6700 1 --2- 172.21.15.40:0/3202944876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6840ffb00 0x7fd684195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.933+0000 7fd68ada6700 1 --2- 172.21.15.40:0/3202944876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd684100cf0 0x7fd684195e60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.933+0000 7fd68ada6700 1 -- 172.21.15.40:0/3202944876 >> 172.21.15.40:0/3202944876 conn(0x7fd6840fb0e0 msgr2=0x7fd6840fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.933+0000 7fd68ada6700 1 -- 172.21.15.40:0/3202944876 shutdown_connections 2023-07-18T03:29:23.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:23.933+0000 7fd68ada6700 1 -- 172.21.15.40:0/3202944876 wait complete. 2023-07-18T03:29:23.946 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-18T03:29:23.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-18T03:29:24.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:23 smithi134 ceph-mon[98216]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:24.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:23 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2452565659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:24.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:23 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2348370107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:24.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:23 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/4046553445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:29:24.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:23 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:24.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:23 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:24.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.080+0000 7f412efa9700 1 -- 172.21.15.40:0/3051589134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41280fe9e0 msgr2=0x7f41280fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.080+0000 7f412efa9700 1 --2- 172.21.15.40:0/3051589134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41280fe9e0 0x7f41280fedf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f411c009990 tx=0x7f411c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.080+0000 7f412efa9700 1 -- 172.21.15.40:0/3051589134 shutdown_connections 2023-07-18T03:29:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.080+0000 7f412efa9700 1 --2- 172.21.15.40:0/3051589134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4128100560 0x7f4128106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.080+0000 7f412efa9700 1 --2- 172.21.15.40:0/3051589134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41280fe9e0 0x7f41280fedf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.080+0000 7f412efa9700 1 --2- 172.21.15.40:0/3051589134 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41280ffbd0 0x7f4128100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.080+0000 7f412efa9700 1 -- 172.21.15.40:0/3051589134 >> 172.21.15.40:0/3051589134 conn(0x7f41280fa000 msgr2=0x7f41280fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.080+0000 7f412efa9700 1 -- 172.21.15.40:0/3051589134 shutdown_connections 2023-07-18T03:29:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.080+0000 7f412efa9700 1 -- 172.21.15.40:0/3051589134 wait complete. 2023-07-18T03:29:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.081+0000 7f412efa9700 1 Processor -- start 2023-07-18T03:29:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.081+0000 7f412efa9700 1 -- start start 2023-07-18T03:29:24.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.081+0000 7f412efa9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41280fe9e0 0x7f41280681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:24.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.081+0000 7f412efa9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41280ffbd0 0x7f412819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:24.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.081+0000 7f412efa9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4128100560 0x7f412819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:24.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.081+0000 7f412efa9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412819d420 con 0x7f41280ffbd0 2023-07-18T03:29:24.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.081+0000 7f412efa9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412819d590 con 0x7f41280fe9e0 2023-07-18T03:29:24.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.081+0000 7f412efa9700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412819d750 con 0x7f4128100560 2023-07-18T03:29:24.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.082+0000 7f4127fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41280ffbd0 0x7f412819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:24.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.082+0000 7f4127fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41280ffbd0 0x7f412819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56042/0 (socket says 172.21.15.40:56042) 2023-07-18T03:29:24.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.082+0000 7f4127fff700 1 -- 172.21.15.40:0/1627985552 learned_addr learned my addr 172.21.15.40:0/1627985552 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:24.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.082+0000 7f412cd45700 1 --2- 172.21.15.40:0/1627985552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41280fe9e0 0x7f41280681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:24.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.082+0000 7f412d546700 1 --2- 172.21.15.40:0/1627985552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4128100560 0x7f412819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:24.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.082+0000 7f412cd45700 1 -- 172.21.15.40:0/1627985552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4128100560 msgr2=0x7f412819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:24.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.082+0000 7f412cd45700 1 --2- 172.21.15.40:0/1627985552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4128100560 0x7f412819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:24.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.082+0000 7f412cd45700 1 -- 172.21.15.40:0/1627985552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41280ffbd0 msgr2=0x7f412819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:24.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.082+0000 7f412cd45700 1 --2- 172.21.15.40:0/1627985552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41280ffbd0 0x7f412819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:24.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.082+0000 7f412cd45700 1 -- 172.21.15.40:0/1627985552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c0095f0 con 0x7f41280fe9e0 2023-07-18T03:29:24.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.082+0000 7f4127fff700 1 --2- 172.21.15.40:0/1627985552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41280ffbd0 0x7f412819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:24.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.082+0000 7f412cd45700 1 --2- 172.21.15.40:0/1627985552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41280fe9e0 0x7f41280681e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f411c0098e0 tx=0x7f411c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:24.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.082+0000 7f4125ffb700 1 -- 172.21.15.40:0/1627985552 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c01d070 con 0x7f41280fe9e0 2023-07-18T03:29:24.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.082+0000 7f412efa9700 1 -- 172.21.15.40:0/1627985552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412819d9d0 con 0x7f41280fe9e0 2023-07-18T03:29:24.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.082+0000 7f4125ffb700 1 -- 172.21.15.40:0/1627985552 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f411c0054e0 con 0x7f41280fe9e0 2023-07-18T03:29:24.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.082+0000 7f4125ffb700 1 -- 172.21.15.40:0/1627985552 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c004c40 con 0x7f41280fe9e0 2023-07-18T03:29:24.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.083+0000 7f412efa9700 1 -- 172.21.15.40:0/1627985552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412819de00 con 0x7f41280fe9e0 2023-07-18T03:29:24.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.084+0000 7f412efa9700 1 -- 172.21.15.40:0/1627985552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4128105990 con 0x7f41280fe9e0 2023-07-18T03:29:24.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.085+0000 7f4125ffb700 1 -- 172.21.15.40:0/1627985552 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f411c00fb90 con 0x7f41280fe9e0 2023-07-18T03:29:24.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.085+0000 7f4125ffb700 1 --2- 172.21.15.40:0/1627985552 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f41180a1150 0x7f41180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:24.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.085+0000 7f4127fff700 1 --2- 172.21.15.40:0/1627985552 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f41180a1150 0x7f41180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:24.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.085+0000 7f4125ffb700 1 -- 172.21.15.40:0/1627985552 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f411c0cdcc0 con 0x7f41280fe9e0 2023-07-18T03:29:24.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.086+0000 7f4127fff700 1 --2- 172.21.15.40:0/1627985552 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f41180a1150 0x7f41180a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4114005950 tx=0x7f411400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:24.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.089+0000 7f4125ffb700 1 -- 172.21.15.40:0/1627985552 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f411c097940 con 0x7f41280fe9e0 2023-07-18T03:29:24.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:24.295+0000 7f412efa9700 1 -- 172.21.15.40:0/1627985552 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f412802c850 con 0x7f41280fe9e0 2023-07-18T03:29:25.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:25.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:25.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:25.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:25.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:24 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3202944876' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-18T03:29:25.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:25.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:29:25.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:24 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:25.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:25.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:24 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3202944876' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-18T03:29:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:29:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:24 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:25.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:25.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:25.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:25.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:25.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:24 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3202944876' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-18T03:29:25.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:25.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:29:25.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:24 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:25.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.269+0000 7f4125ffb700 1 -- 172.21.15.40:0/1627985552 <== mon.2 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) v1 ==== 138+0+0 (secure 0 0 0) 0x7f411c09af60 con 0x7f41280fe9e0 2023-07-18T03:29:25.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.273+0000 7f412efa9700 1 -- 172.21.15.40:0/1627985552 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f41180a1150 msgr2=0x7f41180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:25.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.273+0000 7f412efa9700 1 --2- 172.21.15.40:0/1627985552 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f41180a1150 0x7f41180a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4114005950 tx=0x7f411400b500 comp rx=0 tx=0).stop 2023-07-18T03:29:25.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.273+0000 7f412efa9700 1 -- 172.21.15.40:0/1627985552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41280fe9e0 msgr2=0x7f41280681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:25.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.273+0000 7f412efa9700 1 --2- 172.21.15.40:0/1627985552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41280fe9e0 0x7f41280681e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f411c0098e0 tx=0x7f411c004730 comp rx=0 tx=0).stop 2023-07-18T03:29:25.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.273+0000 7f412efa9700 1 -- 172.21.15.40:0/1627985552 shutdown_connections 2023-07-18T03:29:25.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.273+0000 7f412efa9700 1 --2- 172.21.15.40:0/1627985552 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f41180a1150 0x7f41180a3600 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:25.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.273+0000 7f412efa9700 1 --2- 172.21.15.40:0/1627985552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4128100560 0x7f412819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:25.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.273+0000 7f412efa9700 1 --2- 172.21.15.40:0/1627985552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41280fe9e0 0x7f41280681e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:25.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.273+0000 7f412efa9700 1 --2- 172.21.15.40:0/1627985552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41280ffbd0 0x7f412819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:25.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.273+0000 7f412efa9700 1 -- 172.21.15.40:0/1627985552 >> 172.21.15.40:0/1627985552 conn(0x7f41280fa000 msgr2=0x7f41280fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.274+0000 7f412efa9700 1 -- 172.21.15.40:0/1627985552 shutdown_connections 2023-07-18T03:29:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.274+0000 7f412efa9700 1 -- 172.21.15.40:0/1627985552 wait complete. 2023-07-18T03:29:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2023-07-18T03:29:25.283 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-18T03:29:25.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.423+0000 7fae132a1700 1 -- 172.21.15.40:0/2662903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae0c0fed20 msgr2=0x7fae0c0ff190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:25.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.423+0000 7fae132a1700 1 --2- 172.21.15.40:0/2662903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae0c0fed20 0x7fae0c0ff190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fae00009990 tx=0x7fae00009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:25.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.423+0000 7fae132a1700 1 -- 172.21.15.40:0/2662903 shutdown_connections 2023-07-18T03:29:25.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.423+0000 7fae132a1700 1 --2- 172.21.15.40:0/2662903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae0c0ff6d0 0x7fae0c0ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:25.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.423+0000 7fae132a1700 1 --2- 172.21.15.40:0/2662903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae0c0fe3d0 0x7fae0c0fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:25.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.423+0000 7fae132a1700 1 --2- 172.21.15.40:0/2662903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae0c0fed20 0x7fae0c0ff190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:25.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.423+0000 7fae132a1700 1 -- 172.21.15.40:0/2662903 >> 172.21.15.40:0/2662903 conn(0x7fae0c0fa060 msgr2=0x7fae0c0fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:25.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.423+0000 7fae132a1700 1 -- 172.21.15.40:0/2662903 shutdown_connections 2023-07-18T03:29:25.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.423+0000 7fae132a1700 1 -- 172.21.15.40:0/2662903 wait complete. 2023-07-18T03:29:25.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae132a1700 1 Processor -- start 2023-07-18T03:29:25.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae132a1700 1 -- start start 2023-07-18T03:29:25.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae132a1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae0c0fe3d0 0x7fae0c197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:25.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae132a1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae0c0fed20 0x7fae0c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:25.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae132a1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae0c0ff6d0 0x7fae0c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:25.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae132a1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c19f590 con 0x7fae0c0fed20 2023-07-18T03:29:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae132a1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c19f6d0 con 0x7fae0c0fe3d0 2023-07-18T03:29:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae132a1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c19f860 con 0x7fae0c0ff6d0 2023-07-18T03:29:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae1083c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae0c0fed20 0x7fae0c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae1083c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae0c0fed20 0x7fae0c197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56056/0 (socket says 172.21.15.40:56056) 2023-07-18T03:29:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae1103d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae0c0fe3d0 0x7fae0c197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae1083c700 1 -- 172.21.15.40:0/1839147810 learned_addr learned my addr 172.21.15.40:0/1839147810 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:25.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae1183e700 1 --2- 172.21.15.40:0/1839147810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae0c0ff6d0 0x7fae0c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae1083c700 1 -- 172.21.15.40:0/1839147810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae0c0ff6d0 msgr2=0x7fae0c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae1083c700 1 --2- 172.21.15.40:0/1839147810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae0c0ff6d0 0x7fae0c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae1083c700 1 -- 172.21.15.40:0/1839147810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae0c0fe3d0 msgr2=0x7fae0c197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae1083c700 1 --2- 172.21.15.40:0/1839147810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae0c0fe3d0 0x7fae0c197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae1083c700 1 -- 172.21.15.40:0/1839147810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae000095f0 con 0x7fae0c0fed20 2023-07-18T03:29:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.424+0000 7fae1103d700 1 --2- 172.21.15.40:0/1839147810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae0c0fe3d0 0x7fae0c197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.425+0000 7fae1083c700 1 --2- 172.21.15.40:0/1839147810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae0c0fed20 0x7fae0c197f50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fae000059d0 tx=0x7fae000047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:25.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.425+0000 7fae0a7fc700 1 -- 172.21.15.40:0/1839147810 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0001d070 con 0x7fae0c0fed20 2023-07-18T03:29:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.425+0000 7fae0a7fc700 1 -- 172.21.15.40:0/1839147810 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae0000fb80 con 0x7fae0c0fed20 2023-07-18T03:29:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.425+0000 7fae132a1700 1 -- 172.21.15.40:0/1839147810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0c19fae0 con 0x7fae0c0fed20 2023-07-18T03:29:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.425+0000 7fae0a7fc700 1 -- 172.21.15.40:0/1839147810 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae00017650 con 0x7fae0c0fed20 2023-07-18T03:29:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.425+0000 7fae132a1700 1 -- 172.21.15.40:0/1839147810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0c19ff60 con 0x7fae0c0fed20 2023-07-18T03:29:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.426+0000 7fae132a1700 1 -- 172.21.15.40:0/1839147810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0c1091b0 con 0x7fae0c0fed20 2023-07-18T03:29:25.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.427+0000 7fae0a7fc700 1 -- 172.21.15.40:0/1839147810 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fae00022470 con 0x7fae0c0fed20 2023-07-18T03:29:25.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.428+0000 7fae0a7fc700 1 --2- 172.21.15.40:0/1839147810 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fadf40a11d0 0x7fadf40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:25.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.428+0000 7fae1103d700 1 --2- 172.21.15.40:0/1839147810 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fadf40a11d0 0x7fadf40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:25.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.428+0000 7fae0a7fc700 1 -- 172.21.15.40:0/1839147810 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fae000ce150 con 0x7fae0c0fed20 2023-07-18T03:29:25.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.428+0000 7fae1103d700 1 --2- 172.21.15.40:0/1839147810 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fadf40a11d0 0x7fadf40a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009480 tx=0x7fadfc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:25.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.431+0000 7fae0a7fc700 1 -- 172.21.15.40:0/1839147810 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fae00097ce0 con 0x7fae0c0fed20 2023-07-18T03:29:25.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:25.636+0000 7fae132a1700 1 -- 172.21.15.40:0/1839147810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fae0c02c850 con 0x7fae0c0fed20 2023-07-18T03:29:26.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.276+0000 7fae0a7fc700 1 -- 172.21.15.40:0/1839147810 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) v1 ==== 170+0+0 (secure 0 0 0) 0x7fae0009b300 con 0x7fae0c0fed20 2023-07-18T03:29:26.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.288+0000 7fae132a1700 1 -- 172.21.15.40:0/1839147810 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fadf40a11d0 msgr2=0x7fadf40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:26.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.288+0000 7fae132a1700 1 --2- 172.21.15.40:0/1839147810 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fadf40a11d0 0x7fadf40a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009480 tx=0x7fadfc00b040 comp rx=0 tx=0).stop 2023-07-18T03:29:26.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.288+0000 7fae132a1700 1 -- 172.21.15.40:0/1839147810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae0c0fed20 msgr2=0x7fae0c197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:26.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.288+0000 7fae132a1700 1 --2- 172.21.15.40:0/1839147810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae0c0fed20 0x7fae0c197f50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fae000059d0 tx=0x7fae000047a0 comp rx=0 tx=0).stop 2023-07-18T03:29:26.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.288+0000 7fae132a1700 1 -- 172.21.15.40:0/1839147810 shutdown_connections 2023-07-18T03:29:26.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.288+0000 7fae132a1700 1 --2- 172.21.15.40:0/1839147810 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fadf40a11d0 0x7fadf40a3680 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:26.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.288+0000 7fae132a1700 1 --2- 172.21.15.40:0/1839147810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae0c0ff6d0 0x7fae0c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:26.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.288+0000 7fae132a1700 1 --2- 172.21.15.40:0/1839147810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae0c0fe3d0 0x7fae0c197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:26.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.288+0000 7fae132a1700 1 --2- 172.21.15.40:0/1839147810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae0c0fed20 0x7fae0c197f50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:26.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.288+0000 7fae132a1700 1 -- 172.21.15.40:0/1839147810 >> 172.21.15.40:0/1839147810 conn(0x7fae0c0fa060 msgr2=0x7fae0c0fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:26.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.293+0000 7fae132a1700 1 -- 172.21.15.40:0/1839147810 shutdown_connections 2023-07-18T03:29:26.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.293+0000 7fae132a1700 1 -- 172.21.15.40:0/1839147810 wait complete. 2023-07-18T03:29:26.290 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2023-07-18T03:29:26.303 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-18T03:29:26.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:26 smithi040 ceph-mon[96439]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:26.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:26 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1627985552' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-18T03:29:26.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:26 smithi040 ceph-mon[96439]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-18T03:29:26.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:26 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:29:26.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:26 smithi040 ceph-mon[96439]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-18T03:29:26.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:26 smithi040 ceph-mon[96439]: osdmap e78: 12 total, 12 up, 12 in 2023-07-18T03:29:26.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:26 smithi064 ceph-mon[98059]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:26.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:26 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1627985552' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-18T03:29:26.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:26 smithi064 ceph-mon[98059]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-18T03:29:26.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:26 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:29:26.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:26 smithi064 ceph-mon[98059]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-18T03:29:26.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:26 smithi064 ceph-mon[98059]: osdmap e78: 12 total, 12 up, 12 in 2023-07-18T03:29:26.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.440+0000 7fb0852f6700 1 -- 172.21.15.40:0/2605712171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb080100560 msgr2=0x7fb080106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:26.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.440+0000 7fb0852f6700 1 --2- 172.21.15.40:0/2605712171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb080100560 0x7fb080106e20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb07800b0a0 tx=0x7fb07800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:26.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.440+0000 7fb0852f6700 1 -- 172.21.15.40:0/2605712171 shutdown_connections 2023-07-18T03:29:26.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.440+0000 7fb0852f6700 1 --2- 172.21.15.40:0/2605712171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb080100560 0x7fb080106e20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:26.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.440+0000 7fb0852f6700 1 --2- 172.21.15.40:0/2605712171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0800fe9e0 0x7fb0800fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:26.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.440+0000 7fb0852f6700 1 --2- 172.21.15.40:0/2605712171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0800ffbd0 0x7fb080100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:26.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.440+0000 7fb0852f6700 1 -- 172.21.15.40:0/2605712171 >> 172.21.15.40:0/2605712171 conn(0x7fb0800fa000 msgr2=0x7fb0800fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:26.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.440+0000 7fb0852f6700 1 -- 172.21.15.40:0/2605712171 shutdown_connections 2023-07-18T03:29:26.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.440+0000 7fb0852f6700 1 -- 172.21.15.40:0/2605712171 wait complete. 2023-07-18T03:29:26.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.441+0000 7fb0852f6700 1 Processor -- start 2023-07-18T03:29:26.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.441+0000 7fb0852f6700 1 -- start start 2023-07-18T03:29:26.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb0852f6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0800fe9e0 0x7fb0800681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:26.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb0852f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0800ffbd0 0x7fb08019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:26.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb0852f6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb080100560 0x7fb08019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:26.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb0852f6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08019f690 con 0x7fb080100560 2023-07-18T03:29:26.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb0852f6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08019f800 con 0x7fb0800ffbd0 2023-07-18T03:29:26.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb0852f6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08019f9c0 con 0x7fb0800fe9e0 2023-07-18T03:29:26.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb07f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb080100560 0x7fb08019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:26.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb07f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb080100560 0x7fb08019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56072/0 (socket says 172.21.15.40:56072) 2023-07-18T03:29:26.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb07f7fe700 1 -- 172.21.15.40:0/3799760594 learned_addr learned my addr 172.21.15.40:0/3799760594 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:26.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb07e7fc700 1 --2- 172.21.15.40:0/3799760594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0800ffbd0 0x7fb08019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:26.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb07effd700 1 --2- 172.21.15.40:0/3799760594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0800fe9e0 0x7fb0800681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:26.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb07f7fe700 1 -- 172.21.15.40:0/3799760594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0800fe9e0 msgr2=0x7fb0800681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:26.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb07f7fe700 1 --2- 172.21.15.40:0/3799760594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0800fe9e0 0x7fb0800681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:26.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb07f7fe700 1 -- 172.21.15.40:0/3799760594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0800ffbd0 msgr2=0x7fb08019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:26.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb07f7fe700 1 --2- 172.21.15.40:0/3799760594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0800ffbd0 0x7fb08019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:26.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb07f7fe700 1 -- 172.21.15.40:0/3799760594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb078009d00 con 0x7fb080100560 2023-07-18T03:29:26.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb07e7fc700 1 --2- 172.21.15.40:0/3799760594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0800ffbd0 0x7fb08019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:26.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb07f7fe700 1 --2- 172.21.15.40:0/3799760594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb080100560 0x7fb08019efe0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb07800b780 tx=0x7fb07800bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:26.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb067fff700 1 -- 172.21.15.40:0/3799760594 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07800e040 con 0x7fb080100560 2023-07-18T03:29:26.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb0852f6700 1 -- 172.21.15.40:0/3799760594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08019fc40 con 0x7fb080100560 2023-07-18T03:29:26.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb067fff700 1 -- 172.21.15.40:0/3799760594 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb078004c60 con 0x7fb080100560 2023-07-18T03:29:26.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.442+0000 7fb067fff700 1 -- 172.21.15.40:0/3799760594 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07801b6b0 con 0x7fb080100560 2023-07-18T03:29:26.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.443+0000 7fb0852f6700 1 -- 172.21.15.40:0/3799760594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0801a0110 con 0x7fb080100560 2023-07-18T03:29:26.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.443+0000 7fb0852f6700 1 -- 172.21.15.40:0/3799760594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08010a1c0 con 0x7fb080100560 2023-07-18T03:29:26.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.445+0000 7fb067fff700 1 -- 172.21.15.40:0/3799760594 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb07801d480 con 0x7fb080100560 2023-07-18T03:29:26.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.445+0000 7fb067fff700 1 --2- 172.21.15.40:0/3799760594 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb0680a1150 0x7fb0680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:26.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.445+0000 7fb067fff700 1 -- 172.21.15.40:0/3799760594 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fb0780ce2f0 con 0x7fb080100560 2023-07-18T03:29:26.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.445+0000 7fb07effd700 1 --2- 172.21.15.40:0/3799760594 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb0680a1150 0x7fb0680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:26.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.446+0000 7fb07effd700 1 --2- 172.21.15.40:0/3799760594 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb0680a1150 0x7fb0680a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb0800ffa30 tx=0x7fb074009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:26.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.449+0000 7fb067fff700 1 -- 172.21.15.40:0/3799760594 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb078097d00 con 0x7fb080100560 2023-07-18T03:29:26.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:26 smithi134 ceph-mon[98216]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:26.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:26 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1627985552' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-18T03:29:26.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:26 smithi134 ceph-mon[98216]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-18T03:29:26.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:26 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:29:26.568 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:26 smithi134 ceph-mon[98216]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-18T03:29:26.568 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:26 smithi134 ceph-mon[98216]: osdmap e78: 12 total, 12 up, 12 in 2023-07-18T03:29:26.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:26.717+0000 7fb0852f6700 1 -- 172.21.15.40:0/3799760594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb08010a3b0 con 0x7fb080100560 2023-07-18T03:29:27.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:27 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1839147810' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-18T03:29:27.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:27 smithi040 ceph-mon[96439]: pgmap v245: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:27.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:27 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1839147810' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-18T03:29:27.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:27 smithi040 ceph-mon[96439]: osdmap e79: 12 total, 12 up, 12 in 2023-07-18T03:29:27.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:27 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3799760594' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-18T03:29:27.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:27 smithi040 ceph-mon[96439]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-18T03:29:27.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:27 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1839147810' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-18T03:29:27.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:27 smithi064 ceph-mon[98059]: pgmap v245: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:27.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:27 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1839147810' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-18T03:29:27.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:27 smithi064 ceph-mon[98059]: osdmap e79: 12 total, 12 up, 12 in 2023-07-18T03:29:27.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:27 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3799760594' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-18T03:29:27.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:27 smithi064 ceph-mon[98059]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-18T03:29:27.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:27 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1839147810' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-18T03:29:27.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:27 smithi134 ceph-mon[98216]: pgmap v245: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:27.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:27 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1839147810' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-18T03:29:27.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:27 smithi134 ceph-mon[98216]: osdmap e79: 12 total, 12 up, 12 in 2023-07-18T03:29:27.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:27 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3799760594' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-18T03:29:27.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:27 smithi134 ceph-mon[98216]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-18T03:29:27.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.595+0000 7fb067fff700 1 -- 172.21.15.40:0/3799760594 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb0780210c0 con 0x7fb080100560 2023-07-18T03:29:27.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.598+0000 7fb0852f6700 1 -- 172.21.15.40:0/3799760594 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb0680a1150 msgr2=0x7fb0680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:27.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.598+0000 7fb0852f6700 1 --2- 172.21.15.40:0/3799760594 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb0680a1150 0x7fb0680a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb0800ffa30 tx=0x7fb074009260 comp rx=0 tx=0).stop 2023-07-18T03:29:27.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.598+0000 7fb0852f6700 1 -- 172.21.15.40:0/3799760594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb080100560 msgr2=0x7fb08019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:27.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.598+0000 7fb0852f6700 1 --2- 172.21.15.40:0/3799760594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb080100560 0x7fb08019efe0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb07800b780 tx=0x7fb07800bd00 comp rx=0 tx=0).stop 2023-07-18T03:29:27.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.598+0000 7fb0852f6700 1 -- 172.21.15.40:0/3799760594 shutdown_connections 2023-07-18T03:29:27.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.598+0000 7fb0852f6700 1 --2- 172.21.15.40:0/3799760594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb080100560 0x7fb08019efe0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:27.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.598+0000 7fb0852f6700 1 --2- 172.21.15.40:0/3799760594 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb0680a1150 0x7fb0680a3600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:27.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.598+0000 7fb0852f6700 1 --2- 172.21.15.40:0/3799760594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0800fe9e0 0x7fb0800681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:27.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.598+0000 7fb0852f6700 1 --2- 172.21.15.40:0/3799760594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0800ffbd0 0x7fb08019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:27.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.598+0000 7fb0852f6700 1 -- 172.21.15.40:0/3799760594 >> 172.21.15.40:0/3799760594 conn(0x7fb0800fa000 msgr2=0x7fb0800fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:27.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.599+0000 7fb0852f6700 1 -- 172.21.15.40:0/3799760594 shutdown_connections 2023-07-18T03:29:27.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.599+0000 7fb0852f6700 1 -- 172.21.15.40:0/3799760594 wait complete. 2023-07-18T03:29:27.597 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 2 and data pool 3 2023-07-18T03:29:27.609 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:29:27.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.780+0000 7f7f2ff51700 1 -- 172.21.15.40:0/3392359851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f28100550 msgr2=0x7f7f28102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:27.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.780+0000 7f7f2ff51700 1 --2- 172.21.15.40:0/3392359851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f28100550 0x7f7f28102930 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7f24009990 tx=0x7f7f24009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:27.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.780+0000 7f7f2ff51700 1 -- 172.21.15.40:0/3392359851 shutdown_connections 2023-07-18T03:29:27.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.780+0000 7f7f2ff51700 1 --2- 172.21.15.40:0/3392359851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f28105790 0x7f7f28107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.780+0000 7f7f2ff51700 1 --2- 172.21.15.40:0/3392359851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f28100550 0x7f7f28102930 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.780+0000 7f7f2ff51700 1 --2- 172.21.15.40:0/3392359851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f28102e70 0x7f7f28105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.780+0000 7f7f2ff51700 1 -- 172.21.15.40:0/3392359851 >> 172.21.15.40:0/3392359851 conn(0x7f7f280f9fd0 msgr2=0x7f7f280fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.780+0000 7f7f2ff51700 1 -- 172.21.15.40:0/3392359851 shutdown_connections 2023-07-18T03:29:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.780+0000 7f7f2ff51700 1 -- 172.21.15.40:0/3392359851 wait complete. 2023-07-18T03:29:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.780+0000 7f7f2ff51700 1 Processor -- start 2023-07-18T03:29:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2ff51700 1 -- start start 2023-07-18T03:29:27.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2ff51700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f28100550 0x7f7f281957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:27.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2ff51700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f28102e70 0x7f7f28195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:27.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2ff51700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f28105790 0x7f7f2819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:27.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2ff51700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2819d350 con 0x7f7f28105790 2023-07-18T03:29:27.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2ff51700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2819d490 con 0x7f7f28102e70 2023-07-18T03:29:27.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2ff51700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2819d620 con 0x7f7f28100550 2023-07-18T03:29:27.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2e4ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f28105790 0x7f7f2819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:27.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2e4ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f28105790 0x7f7f2819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56094/0 (socket says 172.21.15.40:56094) 2023-07-18T03:29:27.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2e4ee700 1 -- 172.21.15.40:0/1942809920 learned_addr learned my addr 172.21.15.40:0/1942809920 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:27.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2dced700 1 --2- 172.21.15.40:0/1942809920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f28100550 0x7f7f281957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:27.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2d4ec700 1 --2- 172.21.15.40:0/1942809920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f28102e70 0x7f7f28195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:27.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2e4ee700 1 -- 172.21.15.40:0/1942809920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f28100550 msgr2=0x7f7f281957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:27.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2e4ee700 1 --2- 172.21.15.40:0/1942809920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f28100550 0x7f7f281957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:27.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2e4ee700 1 -- 172.21.15.40:0/1942809920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f28102e70 msgr2=0x7f7f28195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:27.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2e4ee700 1 --2- 172.21.15.40:0/1942809920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f28102e70 0x7f7f28195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:27.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2e4ee700 1 -- 172.21.15.40:0/1942809920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f240095f0 con 0x7f7f28105790 2023-07-18T03:29:27.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2d4ec700 1 --2- 172.21.15.40:0/1942809920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f28102e70 0x7f7f28195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:29:27.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2e4ee700 1 --2- 172.21.15.40:0/1942809920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f28105790 0x7f7f2819cd80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7f20007ee0 tx=0x7f7f2000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:27.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f1effd700 1 -- 172.21.15.40:0/1942809920 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f20017070 con 0x7f7f28105790 2023-07-18T03:29:27.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f1effd700 1 -- 172.21.15.40:0/1942809920 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f2000f040 con 0x7f7f28105790 2023-07-18T03:29:27.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f2ff51700 1 -- 172.21.15.40:0/1942809920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f2819d900 con 0x7f7f28105790 2023-07-18T03:29:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.781+0000 7f7f1effd700 1 -- 172.21.15.40:0/1942809920 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f20005740 con 0x7f7f28105790 2023-07-18T03:29:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.782+0000 7f7f2ff51700 1 -- 172.21.15.40:0/1942809920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f2819ddc0 con 0x7f7f28105790 2023-07-18T03:29:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.782+0000 7f7f2ff51700 1 -- 172.21.15.40:0/1942809920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f2810b280 con 0x7f7f28105790 2023-07-18T03:29:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.784+0000 7f7f1effd700 1 -- 172.21.15.40:0/1942809920 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7f200058a0 con 0x7f7f28105790 2023-07-18T03:29:27.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.784+0000 7f7f1effd700 1 --2- 172.21.15.40:0/1942809920 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7f140a11a0 0x7f7f140a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:27.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.785+0000 7f7f2dced700 1 --2- 172.21.15.40:0/1942809920 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7f140a11a0 0x7f7f140a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:27.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.785+0000 7f7f1effd700 1 -- 172.21.15.40:0/1942809920 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7f200cc660 con 0x7f7f28105790 2023-07-18T03:29:27.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.785+0000 7f7f2dced700 1 --2- 172.21.15.40:0/1942809920 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7f140a11a0 0x7f7f140a3650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f24006010 tx=0x7f7f2400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:27.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.788+0000 7f7f1effd700 1 -- 172.21.15.40:0/1942809920 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7f20095f00 con 0x7f7f28105790 2023-07-18T03:29:27.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.990+0000 7f7f2ff51700 1 -- 172.21.15.40:0/1942809920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f2819e400 con 0x7f7f28105790 2023-07-18T03:29:27.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.991+0000 7f7f1effd700 1 -- 172.21.15.40:0/1942809920 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22084 (secure 0 0 0) 0x7f7f20099520 con 0x7f7f28105790 2023-07-18T03:29:27.988 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:27.988 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":80,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:29:26.718246+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:29:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.993+0000 7f7f2ff51700 1 -- 172.21.15.40:0/1942809920 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7f140a11a0 msgr2=0x7f7f140a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.993+0000 7f7f2ff51700 1 --2- 172.21.15.40:0/1942809920 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7f140a11a0 0x7f7f140a3650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f24006010 tx=0x7f7f2400b540 comp rx=0 tx=0).stop 2023-07-18T03:29:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.993+0000 7f7f2ff51700 1 -- 172.21.15.40:0/1942809920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f28105790 msgr2=0x7f7f2819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.993+0000 7f7f2ff51700 1 --2- 172.21.15.40:0/1942809920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f28105790 0x7f7f2819cd80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7f20007ee0 tx=0x7f7f2000d3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.993+0000 7f7f2ff51700 1 -- 172.21.15.40:0/1942809920 shutdown_connections 2023-07-18T03:29:27.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.993+0000 7f7f2ff51700 1 --2- 172.21.15.40:0/1942809920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f28105790 0x7f7f2819cd80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.993+0000 7f7f2ff51700 1 --2- 172.21.15.40:0/1942809920 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7f140a11a0 0x7f7f140a3650 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.993+0000 7f7f2ff51700 1 --2- 172.21.15.40:0/1942809920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f28100550 0x7f7f281957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.993+0000 7f7f2ff51700 1 --2- 172.21.15.40:0/1942809920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f28102e70 0x7f7f28195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.993+0000 7f7f2ff51700 1 -- 172.21.15.40:0/1942809920 >> 172.21.15.40:0/1942809920 conn(0x7f7f280f9fd0 msgr2=0x7f7f281019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.994+0000 7f7f2ff51700 1 -- 172.21.15.40:0/1942809920 shutdown_connections 2023-07-18T03:29:27.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:27.994+0000 7f7f2ff51700 1 -- 172.21.15.40:0/1942809920 wait complete. 2023-07-18T03:29:28.002 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:29:28.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.133+0000 7f2dc6c38700 1 -- 172.21.15.40:0/460902766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dc0100550 msgr2=0x7f2dc0106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.133+0000 7f2dc6c38700 1 --2- 172.21.15.40:0/460902766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dc0100550 0x7f2dc0106e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2db800b0a0 tx=0x7f2db800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.133+0000 7f2dc6c38700 1 -- 172.21.15.40:0/460902766 shutdown_connections 2023-07-18T03:29:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.133+0000 7f2dc6c38700 1 --2- 172.21.15.40:0/460902766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dc0100550 0x7f2dc0106e10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.133+0000 7f2dc6c38700 1 --2- 172.21.15.40:0/460902766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dc00fe9d0 0x7f2dc00fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:28.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.133+0000 7f2dc6c38700 1 --2- 172.21.15.40:0/460902766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dc00ffbc0 0x7f2dc0100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:28.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.133+0000 7f2dc6c38700 1 -- 172.21.15.40:0/460902766 >> 172.21.15.40:0/460902766 conn(0x7f2dc00f9fd0 msgr2=0x7f2dc00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:28.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.133+0000 7f2dc6c38700 1 -- 172.21.15.40:0/460902766 shutdown_connections 2023-07-18T03:29:28.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.133+0000 7f2dc6c38700 1 -- 172.21.15.40:0/460902766 wait complete. 2023-07-18T03:29:28.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.133+0000 7f2dc6c38700 1 Processor -- start 2023-07-18T03:29:28.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dc6c38700 1 -- start start 2023-07-18T03:29:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dc6c38700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dc00fe9d0 0x7f2dc0195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dc6c38700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dc00ffbc0 0x7f2dc0195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dc6c38700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dc0100550 0x7f2dc019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dc6c38700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc019d410 con 0x7f2dc00fe9d0 2023-07-18T03:29:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dc6c38700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc019d550 con 0x7f2dc00ffbc0 2023-07-18T03:29:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dc6c38700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc019d6e0 con 0x7f2dc0100550 2023-07-18T03:29:28.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dc49d4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dc00fe9d0 0x7f2dc0195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:28.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dc49d4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dc00fe9d0 0x7f2dc0195890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56104/0 (socket says 172.21.15.40:56104) 2023-07-18T03:29:28.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dc49d4700 1 -- 172.21.15.40:0/1929956771 learned_addr learned my addr 172.21.15.40:0/1929956771 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:28.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dbffff700 1 --2- 172.21.15.40:0/1929956771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dc00ffbc0 0x7f2dc0195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:28.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dc49d4700 1 -- 172.21.15.40:0/1929956771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dc0100550 msgr2=0x7f2dc019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:28.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dc51d5700 1 --2- 172.21.15.40:0/1929956771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dc0100550 0x7f2dc019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:28.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dc49d4700 1 --2- 172.21.15.40:0/1929956771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dc0100550 0x7f2dc019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:28.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dc49d4700 1 -- 172.21.15.40:0/1929956771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dc00ffbc0 msgr2=0x7f2dc0195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:28.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dc49d4700 1 --2- 172.21.15.40:0/1929956771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dc00ffbc0 0x7f2dc0195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:28.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dc49d4700 1 -- 172.21.15.40:0/1929956771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db8009d00 con 0x7f2dc00fe9d0 2023-07-18T03:29:28.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dc49d4700 1 --2- 172.21.15.40:0/1929956771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dc00fe9d0 0x7f2dc0195890 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2db400e970 tx=0x7f2db400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:28.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dbdffb700 1 -- 172.21.15.40:0/1929956771 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4009790 con 0x7f2dc00fe9d0 2023-07-18T03:29:28.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.134+0000 7f2dbdffb700 1 -- 172.21.15.40:0/1929956771 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2db400eec0 con 0x7f2dc00fe9d0 2023-07-18T03:29:28.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.135+0000 7f2dc6c38700 1 -- 172.21.15.40:0/1929956771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc019d9c0 con 0x7f2dc00fe9d0 2023-07-18T03:29:28.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.135+0000 7f2dbdffb700 1 -- 172.21.15.40:0/1929956771 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4003cf0 con 0x7f2dc00fe9d0 2023-07-18T03:29:28.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.135+0000 7f2dc6c38700 1 -- 172.21.15.40:0/1929956771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc019ded0 con 0x7f2dc00fe9d0 2023-07-18T03:29:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.136+0000 7f2dc6c38700 1 -- 172.21.15.40:0/1929956771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc010a200 con 0x7f2dc00fe9d0 2023-07-18T03:29:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.137+0000 7f2dbdffb700 1 -- 172.21.15.40:0/1929956771 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2db400ccd0 con 0x7f2dc00fe9d0 2023-07-18T03:29:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.138+0000 7f2dbdffb700 1 --2- 172.21.15.40:0/1929956771 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2db00a11a0 0x7f2db00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:28.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.138+0000 7f2dbffff700 1 --2- 172.21.15.40:0/1929956771 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2db00a11a0 0x7f2db00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:28.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.138+0000 7f2dbdffb700 1 -- 172.21.15.40:0/1929956771 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2db4014070 con 0x7f2dc00fe9d0 2023-07-18T03:29:28.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.138+0000 7f2dbffff700 1 --2- 172.21.15.40:0/1929956771 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2db00a11a0 0x7f2db00a3650 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2dac005950 tx=0x7f2dac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:28.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.142+0000 7f2dbdffb700 1 -- 172.21.15.40:0/1929956771 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2db40962b0 con 0x7f2dc00fe9d0 2023-07-18T03:29:28.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:28 smithi040 ceph-mon[96439]: osdmap e80: 12 total, 12 up, 12 in 2023-07-18T03:29:28.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:28.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:29:28.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:28 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:28 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3799760594' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-18T03:29:28.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:28 smithi040 ceph-mon[96439]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-18T03:29:28.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:28 smithi040 ceph-mon[96439]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-18T03:29:28.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:28 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1942809920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.346+0000 7f2dc6c38700 1 -- 172.21.15.40:0/1929956771 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2dc019e4e0 con 0x7f2dc00fe9d0 2023-07-18T03:29:28.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.347+0000 7f2dbdffb700 1 -- 172.21.15.40:0/1929956771 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22071 (secure 0 0 0) 0x7f2db40998d0 con 0x7f2dc00fe9d0 2023-07-18T03:29:28.344 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:28.344 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":81,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:29:28.081282+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:29:28.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.350+0000 7f2dc6c38700 1 -- 172.21.15.40:0/1929956771 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2db00a11a0 msgr2=0x7f2db00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:28.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.350+0000 7f2dc6c38700 1 --2- 172.21.15.40:0/1929956771 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2db00a11a0 0x7f2db00a3650 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2dac005950 tx=0x7f2dac00b410 comp rx=0 tx=0).stop 2023-07-18T03:29:28.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.350+0000 7f2dc6c38700 1 -- 172.21.15.40:0/1929956771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dc00fe9d0 msgr2=0x7f2dc0195890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:28.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.350+0000 7f2dc6c38700 1 --2- 172.21.15.40:0/1929956771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dc00fe9d0 0x7f2dc0195890 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2db400e970 tx=0x7f2db400ec80 comp rx=0 tx=0).stop 2023-07-18T03:29:28.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.350+0000 7f2dc6c38700 1 -- 172.21.15.40:0/1929956771 shutdown_connections 2023-07-18T03:29:28.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.350+0000 7f2dc6c38700 1 --2- 172.21.15.40:0/1929956771 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2db00a11a0 0x7f2db00a3650 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:28.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.350+0000 7f2dc6c38700 1 --2- 172.21.15.40:0/1929956771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2dc0100550 0x7f2dc019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:28.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.350+0000 7f2dc6c38700 1 --2- 172.21.15.40:0/1929956771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dc00fe9d0 0x7f2dc0195890 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:28.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.350+0000 7f2dc6c38700 1 --2- 172.21.15.40:0/1929956771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dc00ffbc0 0x7f2dc0195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:28.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.350+0000 7f2dc6c38700 1 -- 172.21.15.40:0/1929956771 >> 172.21.15.40:0/1929956771 conn(0x7f2dc00f9fd0 msgr2=0x7f2dc00676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:28.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.350+0000 7f2dc6c38700 1 -- 172.21.15.40:0/1929956771 shutdown_connections 2023-07-18T03:29:28.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.350+0000 7f2dc6c38700 1 -- 172.21.15.40:0/1929956771 wait complete. 2023-07-18T03:29:28.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:28 smithi064 ceph-mon[98059]: osdmap e80: 12 total, 12 up, 12 in 2023-07-18T03:29:28.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:28.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:29:28.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:28 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:28 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3799760594' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-18T03:29:28.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:28 smithi064 ceph-mon[98059]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-18T03:29:28.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:28 smithi064 ceph-mon[98059]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-18T03:29:28.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:28 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1942809920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:28.358 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-18T03:29:28.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.490+0000 7fbe95a85700 1 -- 172.21.15.40:0/1395040746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe90100590 msgr2=0x7fbe90102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:28.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.490+0000 7fbe95a85700 1 --2- 172.21.15.40:0/1395040746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe90100590 0x7fbe90102970 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009990 tx=0x7fbe80009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:28.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.490+0000 7fbe95a85700 1 -- 172.21.15.40:0/1395040746 shutdown_connections 2023-07-18T03:29:28.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.490+0000 7fbe95a85700 1 --2- 172.21.15.40:0/1395040746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe901057d0 0x7fbe90107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:28.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.490+0000 7fbe95a85700 1 --2- 172.21.15.40:0/1395040746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe90100590 0x7fbe90102970 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:28.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.490+0000 7fbe95a85700 1 --2- 172.21.15.40:0/1395040746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe90102eb0 0x7fbe90105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:28.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.490+0000 7fbe95a85700 1 -- 172.21.15.40:0/1395040746 >> 172.21.15.40:0/1395040746 conn(0x7fbe900fa030 msgr2=0x7fbe900fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:28.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.490+0000 7fbe95a85700 1 -- 172.21.15.40:0/1395040746 shutdown_connections 2023-07-18T03:29:28.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.490+0000 7fbe95a85700 1 -- 172.21.15.40:0/1395040746 wait complete. 2023-07-18T03:29:28.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.491+0000 7fbe95a85700 1 Processor -- start 2023-07-18T03:29:28.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.491+0000 7fbe95a85700 1 -- start start 2023-07-18T03:29:28.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.491+0000 7fbe95a85700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe90100590 0x7fbe90195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:28.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.491+0000 7fbe95a85700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe90102eb0 0x7fbe90195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:28.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.491+0000 7fbe95a85700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe901057d0 0x7fbe9019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:28.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.491+0000 7fbe95a85700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9019d3b0 con 0x7fbe901057d0 2023-07-18T03:29:28.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.491+0000 7fbe95a85700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9019d4f0 con 0x7fbe90100590 2023-07-18T03:29:28.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.491+0000 7fbe95a85700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9019d680 con 0x7fbe90102eb0 2023-07-18T03:29:28.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.492+0000 7fbe8ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe901057d0 0x7fbe9019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:28.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.492+0000 7fbe8ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe901057d0 0x7fbe9019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56122/0 (socket says 172.21.15.40:56122) 2023-07-18T03:29:28.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.492+0000 7fbe8ffff700 1 -- 172.21.15.40:0/192658370 learned_addr learned my addr 172.21.15.40:0/192658370 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:28.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.492+0000 7fbe8effd700 1 --2- 172.21.15.40:0/192658370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe90102eb0 0x7fbe90195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.492+0000 7fbe8f7fe700 1 --2- 172.21.15.40:0/192658370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe90100590 0x7fbe90195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.492+0000 7fbe8ffff700 1 -- 172.21.15.40:0/192658370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe90102eb0 msgr2=0x7fbe90195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.492+0000 7fbe8ffff700 1 --2- 172.21.15.40:0/192658370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe90102eb0 0x7fbe90195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.492+0000 7fbe8ffff700 1 -- 172.21.15.40:0/192658370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe90100590 msgr2=0x7fbe90195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.492+0000 7fbe8ffff700 1 --2- 172.21.15.40:0/192658370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe90100590 0x7fbe90195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.492+0000 7fbe8ffff700 1 -- 172.21.15.40:0/192658370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe800095f0 con 0x7fbe901057d0 2023-07-18T03:29:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.492+0000 7fbe8ffff700 1 --2- 172.21.15.40:0/192658370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe901057d0 0x7fbe9019cde0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800ef30 tx=0x7fbe8800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:28.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.492+0000 7fbe8cff9700 1 -- 172.21.15.40:0/192658370 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe88018070 con 0x7fbe901057d0 2023-07-18T03:29:28.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.492+0000 7fbe8cff9700 1 -- 172.21.15.40:0/192658370 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe8800f040 con 0x7fbe901057d0 2023-07-18T03:29:28.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.492+0000 7fbe95a85700 1 -- 172.21.15.40:0/192658370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9019d900 con 0x7fbe901057d0 2023-07-18T03:29:28.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.492+0000 7fbe8cff9700 1 -- 172.21.15.40:0/192658370 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8800cdb0 con 0x7fbe901057d0 2023-07-18T03:29:28.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.492+0000 7fbe95a85700 1 -- 172.21.15.40:0/192658370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9019de10 con 0x7fbe901057d0 2023-07-18T03:29:28.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.493+0000 7fbe95a85700 1 -- 172.21.15.40:0/192658370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe900fbba0 con 0x7fbe901057d0 2023-07-18T03:29:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.494+0000 7fbe8cff9700 1 -- 172.21.15.40:0/192658370 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbe88003a00 con 0x7fbe901057d0 2023-07-18T03:29:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.495+0000 7fbe8cff9700 1 --2- 172.21.15.40:0/192658370 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe780a1150 0x7fbe780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.495+0000 7fbe8f7fe700 1 --2- 172.21.15.40:0/192658370 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe780a1150 0x7fbe780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:28.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.495+0000 7fbe8cff9700 1 -- 172.21.15.40:0/192658370 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbe880ccd10 con 0x7fbe901057d0 2023-07-18T03:29:28.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.496+0000 7fbe8f7fe700 1 --2- 172.21.15.40:0/192658370 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe780a1150 0x7fbe780a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbe80006010 tx=0x7fbe8000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:28.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.498+0000 7fbe8cff9700 1 -- 172.21.15.40:0/192658370 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbe88096640 con 0x7fbe901057d0 2023-07-18T03:29:28.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:28 smithi134 ceph-mon[98216]: osdmap e80: 12 total, 12 up, 12 in 2023-07-18T03:29:28.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.568 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.568 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.568 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.569 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.569 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:29:28.569 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:29:28.569 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:28 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:29:28.570 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:28 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3799760594' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-18T03:29:28.570 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:28 smithi134 ceph-mon[98216]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-18T03:29:28.570 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:28 smithi134 ceph-mon[98216]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-18T03:29:28.571 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:28 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1942809920' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:28.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:28.745+0000 7fbe95a85700 1 -- 172.21.15.40:0/192658370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbe9002c850 con 0x7fbe901057d0 2023-07-18T03:29:29.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:29 smithi064 ceph-mon[98059]: pgmap v248: 97 pgs: 23 active+clean, 38 creating+peering, 36 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:29.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:29 smithi064 ceph-mon[98059]: osdmap e81: 12 total, 12 up, 12 in 2023-07-18T03:29:29.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:29 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1929956771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:29.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:29 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] up:active 2023-07-18T03:29:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:29 smithi064 ceph-mon[98059]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-18T03:29:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:29 smithi064 ceph-mon[98059]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-18T03:29:29.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:29 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/192658370' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-18T03:29:29.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:29 smithi134 ceph-mon[98216]: pgmap v248: 97 pgs: 23 active+clean, 38 creating+peering, 36 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:29.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:29 smithi134 ceph-mon[98216]: osdmap e81: 12 total, 12 up, 12 in 2023-07-18T03:29:29.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:29 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1929956771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:29.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:29 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] up:active 2023-07-18T03:29:29.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:29 smithi134 ceph-mon[98216]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-18T03:29:29.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:29 smithi134 ceph-mon[98216]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-18T03:29:29.568 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:29 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/192658370' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-18T03:29:29.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:29 smithi040 ceph-mon[96439]: pgmap v248: 97 pgs: 23 active+clean, 38 creating+peering, 36 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-18T03:29:29.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:29 smithi040 ceph-mon[96439]: osdmap e81: 12 total, 12 up, 12 in 2023-07-18T03:29:29.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:29 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1929956771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:29.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:29 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] up:active 2023-07-18T03:29:29.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:29 smithi040 ceph-mon[96439]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-18T03:29:29.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:29 smithi040 ceph-mon[96439]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-18T03:29:29.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:29 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/192658370' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-18T03:29:29.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.604+0000 7fbe8cff9700 1 -- 172.21.15.40:0/192658370 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbe88099c60 con 0x7fbe901057d0 2023-07-18T03:29:29.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.607+0000 7fbe95a85700 1 -- 172.21.15.40:0/192658370 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe780a1150 msgr2=0x7fbe780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:29.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.607+0000 7fbe95a85700 1 --2- 172.21.15.40:0/192658370 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe780a1150 0x7fbe780a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbe80006010 tx=0x7fbe8000b260 comp rx=0 tx=0).stop 2023-07-18T03:29:29.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.607+0000 7fbe95a85700 1 -- 172.21.15.40:0/192658370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe901057d0 msgr2=0x7fbe9019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:29.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.607+0000 7fbe95a85700 1 --2- 172.21.15.40:0/192658370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe901057d0 0x7fbe9019cde0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800ef30 tx=0x7fbe8800c3a0 comp rx=0 tx=0).stop 2023-07-18T03:29:29.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.607+0000 7fbe95a85700 1 -- 172.21.15.40:0/192658370 shutdown_connections 2023-07-18T03:29:29.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.607+0000 7fbe95a85700 1 --2- 172.21.15.40:0/192658370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe901057d0 0x7fbe9019cde0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:29.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.607+0000 7fbe95a85700 1 --2- 172.21.15.40:0/192658370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe90100590 0x7fbe90195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:29.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.607+0000 7fbe95a85700 1 --2- 172.21.15.40:0/192658370 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe780a1150 0x7fbe780a3600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:29.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.607+0000 7fbe95a85700 1 --2- 172.21.15.40:0/192658370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe90102eb0 0x7fbe90195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:29.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.607+0000 7fbe95a85700 1 -- 172.21.15.40:0/192658370 >> 172.21.15.40:0/192658370 conn(0x7fbe900fa030 msgr2=0x7fbe901011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:29.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.607+0000 7fbe95a85700 1 -- 172.21.15.40:0/192658370 shutdown_connections 2023-07-18T03:29:29.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.608+0000 7fbe95a85700 1 -- 172.21.15.40:0/192658370 wait complete. 2023-07-18T03:29:29.619 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-07-18T03:29:29.620 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-07-18T03:29:29.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.759+0000 7f3c26d19700 1 -- 172.21.15.40:0/2073493536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c20100570 msgr2=0x7f3c20106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:29.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.759+0000 7f3c26d19700 1 --2- 172.21.15.40:0/2073493536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c20100570 0x7f3c20106e30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800b0a0 tx=0x7f3c1800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:29.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.759+0000 7f3c26d19700 1 -- 172.21.15.40:0/2073493536 shutdown_connections 2023-07-18T03:29:29.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.759+0000 7f3c26d19700 1 --2- 172.21.15.40:0/2073493536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c20100570 0x7f3c20106e30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:29.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.759+0000 7f3c26d19700 1 --2- 172.21.15.40:0/2073493536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c200fe9f0 0x7f3c200fee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:29.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.759+0000 7f3c26d19700 1 --2- 172.21.15.40:0/2073493536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c200ffbe0 0x7f3c20100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:29.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.759+0000 7f3c26d19700 1 -- 172.21.15.40:0/2073493536 >> 172.21.15.40:0/2073493536 conn(0x7f3c200fa030 msgr2=0x7f3c200fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:29.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.759+0000 7f3c26d19700 1 -- 172.21.15.40:0/2073493536 shutdown_connections 2023-07-18T03:29:29.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.759+0000 7f3c26d19700 1 -- 172.21.15.40:0/2073493536 wait complete. 2023-07-18T03:29:29.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.759+0000 7f3c26d19700 1 Processor -- start 2023-07-18T03:29:29.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c26d19700 1 -- start start 2023-07-18T03:29:29.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c26d19700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c200fe9f0 0x7f3c20197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:29.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c26d19700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c200ffbe0 0x7f3c20198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:29.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c26d19700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c20100570 0x7f3c2019f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:29.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c26d19700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2019f770 con 0x7f3c200fe9f0 2023-07-18T03:29:29.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c26d19700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2019f8b0 con 0x7f3c20100570 2023-07-18T03:29:29.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c26d19700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2019fa40 con 0x7f3c200ffbe0 2023-07-18T03:29:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c24ab5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c200fe9f0 0x7f3c20197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c24ab5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c200fe9f0 0x7f3c20197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56130/0 (socket says 172.21.15.40:56130) 2023-07-18T03:29:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c24ab5700 1 -- 172.21.15.40:0/791151305 learned_addr learned my addr 172.21.15.40:0/791151305 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c252b6700 1 --2- 172.21.15.40:0/791151305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c20100570 0x7f3c2019f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c1ffff700 1 --2- 172.21.15.40:0/791151305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c200ffbe0 0x7f3c20198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c24ab5700 1 -- 172.21.15.40:0/791151305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c200ffbe0 msgr2=0x7f3c20198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c24ab5700 1 --2- 172.21.15.40:0/791151305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c200ffbe0 0x7f3c20198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:29.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c24ab5700 1 -- 172.21.15.40:0/791151305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c20100570 msgr2=0x7f3c2019f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:29.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c24ab5700 1 --2- 172.21.15.40:0/791151305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c20100570 0x7f3c2019f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:29.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c24ab5700 1 -- 172.21.15.40:0/791151305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c18009d00 con 0x7f3c200fe9f0 2023-07-18T03:29:29.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c24ab5700 1 --2- 172.21.15.40:0/791151305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c200fe9f0 0x7f3c20197bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400b500 tx=0x7f3c1400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:29.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c1dffb700 1 -- 172.21.15.40:0/791151305 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c14011830 con 0x7f3c200fe9f0 2023-07-18T03:29:29.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c1dffb700 1 -- 172.21.15.40:0/791151305 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c14011e50 con 0x7f3c200fe9f0 2023-07-18T03:29:29.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c1dffb700 1 -- 172.21.15.40:0/791151305 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1400f2e0 con 0x7f3c200fe9f0 2023-07-18T03:29:29.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.760+0000 7f3c26d19700 1 -- 172.21.15.40:0/791151305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2019fd20 con 0x7f3c200fe9f0 2023-07-18T03:29:29.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.761+0000 7f3c26d19700 1 -- 172.21.15.40:0/791151305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c201a0180 con 0x7f3c200fe9f0 2023-07-18T03:29:29.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.762+0000 7f3c26d19700 1 -- 172.21.15.40:0/791151305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2010a340 con 0x7f3c200fe9f0 2023-07-18T03:29:29.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.763+0000 7f3c1dffb700 1 -- 172.21.15.40:0/791151305 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3c14010580 con 0x7f3c200fe9f0 2023-07-18T03:29:29.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.763+0000 7f3c1dffb700 1 --2- 172.21.15.40:0/791151305 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c100a11d0 0x7f3c100a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:29.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.763+0000 7f3c1dffb700 1 -- 172.21.15.40:0/791151305 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3c140cc1f0 con 0x7f3c200fe9f0 2023-07-18T03:29:29.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.763+0000 7f3c1ffff700 1 --2- 172.21.15.40:0/791151305 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c100a11d0 0x7f3c100a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:29.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.764+0000 7f3c1ffff700 1 --2- 172.21.15.40:0/791151305 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c100a11d0 0x7f3c100a3680 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c005950 tx=0x7f3c0c005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:29.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:29.767+0000 7f3c1dffb700 1 -- 172.21.15.40:0/791151305 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3c14095a90 con 0x7f3c200fe9f0 2023-07-18T03:29:30.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.016+0000 7f3c26d19700 1 -- 172.21.15.40:0/791151305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f3c201a07f0 con 0x7f3c200fe9f0 2023-07-18T03:29:30.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.612+0000 7f3c1dffb700 1 -- 172.21.15.40:0/791151305 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f3c140990b0 con 0x7f3c200fe9f0 2023-07-18T03:29:30.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.615+0000 7f3c26d19700 1 -- 172.21.15.40:0/791151305 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c100a11d0 msgr2=0x7f3c100a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:30.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.615+0000 7f3c26d19700 1 --2- 172.21.15.40:0/791151305 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c100a11d0 0x7f3c100a3680 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c005950 tx=0x7f3c0c005eb0 comp rx=0 tx=0).stop 2023-07-18T03:29:30.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.615+0000 7f3c26d19700 1 -- 172.21.15.40:0/791151305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c200fe9f0 msgr2=0x7f3c20197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:30.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.615+0000 7f3c26d19700 1 --2- 172.21.15.40:0/791151305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c200fe9f0 0x7f3c20197bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400b500 tx=0x7f3c1400b8c0 comp rx=0 tx=0).stop 2023-07-18T03:29:30.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.615+0000 7f3c26d19700 1 -- 172.21.15.40:0/791151305 shutdown_connections 2023-07-18T03:29:30.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.615+0000 7f3c26d19700 1 --2- 172.21.15.40:0/791151305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c20100570 0x7f3c2019f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:30.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.615+0000 7f3c26d19700 1 --2- 172.21.15.40:0/791151305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c200fe9f0 0x7f3c20197bf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:30.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.615+0000 7f3c26d19700 1 --2- 172.21.15.40:0/791151305 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c100a11d0 0x7f3c100a3680 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:30.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.615+0000 7f3c26d19700 1 --2- 172.21.15.40:0/791151305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3c200ffbe0 0x7f3c20198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:30.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.615+0000 7f3c26d19700 1 -- 172.21.15.40:0/791151305 >> 172.21.15.40:0/791151305 conn(0x7f3c200fa030 msgr2=0x7f3c200686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:30.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.616+0000 7f3c26d19700 1 -- 172.21.15.40:0/791151305 shutdown_connections 2023-07-18T03:29:30.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.616+0000 7f3c26d19700 1 -- 172.21.15.40:0/791151305 wait complete. 2023-07-18T03:29:30.631 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-07-18T03:29:30.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.771+0000 7efd024e3700 1 -- 172.21.15.40:0/676049827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc100560 msgr2=0x7efcfc106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:30.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.771+0000 7efd024e3700 1 --2- 172.21.15.40:0/676049827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc100560 0x7efcfc106e20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efcf400b0a0 tx=0x7efcf400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:30.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.771+0000 7efd024e3700 1 -- 172.21.15.40:0/676049827 shutdown_connections 2023-07-18T03:29:30.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.771+0000 7efd024e3700 1 --2- 172.21.15.40:0/676049827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc100560 0x7efcfc106e20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:30.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.771+0000 7efd024e3700 1 --2- 172.21.15.40:0/676049827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcfc0fe9e0 0x7efcfc0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:30.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.771+0000 7efd024e3700 1 --2- 172.21.15.40:0/676049827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc0ffbd0 0x7efcfc100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:30.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.771+0000 7efd024e3700 1 -- 172.21.15.40:0/676049827 >> 172.21.15.40:0/676049827 conn(0x7efcfc0fa000 msgr2=0x7efcfc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:30.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.771+0000 7efd024e3700 1 -- 172.21.15.40:0/676049827 shutdown_connections 2023-07-18T03:29:30.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.771+0000 7efd024e3700 1 -- 172.21.15.40:0/676049827 wait complete. 2023-07-18T03:29:30.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efd024e3700 1 Processor -- start 2023-07-18T03:29:30.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efd024e3700 1 -- start start 2023-07-18T03:29:30.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efd024e3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcfc0fe9e0 0x7efcfc195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:30.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efd024e3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc0ffbd0 0x7efcfc195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:30.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efd024e3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc100560 0x7efcfc19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efd024e3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc19d4f0 con 0x7efcfc100560 2023-07-18T03:29:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efd024e3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc19d630 con 0x7efcfc0ffbd0 2023-07-18T03:29:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efd024e3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc19d7c0 con 0x7efcfc0fe9e0 2023-07-18T03:29:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efd00a80700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc100560 0x7efcfc19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efd00a80700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc100560 0x7efcfc19cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56148/0 (socket says 172.21.15.40:56148) 2023-07-18T03:29:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efd00a80700 1 -- 172.21.15.40:0/3774827101 learned_addr learned my addr 172.21.15.40:0/3774827101 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efcfb7fe700 1 --2- 172.21.15.40:0/3774827101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc0ffbd0 0x7efcfc195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efcfbfff700 1 --2- 172.21.15.40:0/3774827101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcfc0fe9e0 0x7efcfc195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efd00a80700 1 -- 172.21.15.40:0/3774827101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcfc0fe9e0 msgr2=0x7efcfc195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efd00a80700 1 --2- 172.21.15.40:0/3774827101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcfc0fe9e0 0x7efcfc195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efd00a80700 1 -- 172.21.15.40:0/3774827101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc0ffbd0 msgr2=0x7efcfc195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efd00a80700 1 --2- 172.21.15.40:0/3774827101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc0ffbd0 0x7efcfc195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.772+0000 7efd00a80700 1 -- 172.21.15.40:0/3774827101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf4009d00 con 0x7efcfc100560 2023-07-18T03:29:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.773+0000 7efd00a80700 1 --2- 172.21.15.40:0/3774827101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc100560 0x7efcfc19cf20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efcf400d010 tx=0x7efcf4004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.773+0000 7efcf97fa700 1 -- 172.21.15.40:0/3774827101 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf400e040 con 0x7efcfc100560 2023-07-18T03:29:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.773+0000 7efcf97fa700 1 -- 172.21.15.40:0/3774827101 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcf4009440 con 0x7efcfc100560 2023-07-18T03:29:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.773+0000 7efd024e3700 1 -- 172.21.15.40:0/3774827101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc19da40 con 0x7efcfc100560 2023-07-18T03:29:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.773+0000 7efcf97fa700 1 -- 172.21.15.40:0/3774827101 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4004020 con 0x7efcfc100560 2023-07-18T03:29:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.773+0000 7efd024e3700 1 -- 172.21.15.40:0/3774827101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc19dec0 con 0x7efcfc100560 2023-07-18T03:29:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.774+0000 7efd024e3700 1 -- 172.21.15.40:0/3774827101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcfc10a210 con 0x7efcfc100560 2023-07-18T03:29:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.775+0000 7efcf97fa700 1 -- 172.21.15.40:0/3774827101 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efcf401d690 con 0x7efcfc100560 2023-07-18T03:29:30.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.775+0000 7efcf97fa700 1 --2- 172.21.15.40:0/3774827101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efce40a1150 0x7efce40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:30.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.775+0000 7efcf97fa700 1 -- 172.21.15.40:0/3774827101 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efcf40cdd40 con 0x7efcfc100560 2023-07-18T03:29:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.776+0000 7efcfbfff700 1 --2- 172.21.15.40:0/3774827101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efce40a1150 0x7efce40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.776+0000 7efcfbfff700 1 --2- 172.21.15.40:0/3774827101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efce40a1150 0x7efce40a3600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0ffa30 tx=0x7efcec00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:30.780+0000 7efcf97fa700 1 -- 172.21.15.40:0/3774827101 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7efcf4097670 con 0x7efcfc100560 2023-07-18T03:29:31.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.023+0000 7efd024e3700 1 -- 172.21.15.40:0/3774827101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7efcfc19e4d0 con 0x7efcfc100560 2023-07-18T03:29:31.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:30 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/192658370' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-18T03:29:31.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:30 smithi134 ceph-mon[98216]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-18T03:29:31.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:30 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/791151305' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-18T03:29:31.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:30 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/192658370' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-18T03:29:31.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:30 smithi040 ceph-mon[96439]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-18T03:29:31.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:30 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/791151305' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-18T03:29:31.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:30 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/192658370' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-18T03:29:31.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:30 smithi064 ceph-mon[98059]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-18T03:29:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:30 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/791151305' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-18T03:29:31.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.594+0000 7efcf97fa700 1 -- 172.21.15.40:0/3774827101 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7efcf409ac90 con 0x7efcfc100560 2023-07-18T03:29:31.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.597+0000 7efd024e3700 1 -- 172.21.15.40:0/3774827101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efce40a1150 msgr2=0x7efce40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:31.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.597+0000 7efd024e3700 1 --2- 172.21.15.40:0/3774827101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efce40a1150 0x7efce40a3600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0ffa30 tx=0x7efcec00b5b0 comp rx=0 tx=0).stop 2023-07-18T03:29:31.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.597+0000 7efd024e3700 1 -- 172.21.15.40:0/3774827101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc100560 msgr2=0x7efcfc19cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:31.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.597+0000 7efd024e3700 1 --2- 172.21.15.40:0/3774827101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc100560 0x7efcfc19cf20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efcf400d010 tx=0x7efcf4004c80 comp rx=0 tx=0).stop 2023-07-18T03:29:31.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.598+0000 7efd024e3700 1 -- 172.21.15.40:0/3774827101 shutdown_connections 2023-07-18T03:29:31.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.598+0000 7efd024e3700 1 --2- 172.21.15.40:0/3774827101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcfc100560 0x7efcfc19cf20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:31.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.598+0000 7efd024e3700 1 --2- 172.21.15.40:0/3774827101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efce40a1150 0x7efce40a3600 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:31.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.598+0000 7efd024e3700 1 --2- 172.21.15.40:0/3774827101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcfc0fe9e0 0x7efcfc195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:31.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.598+0000 7efd024e3700 1 --2- 172.21.15.40:0/3774827101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcfc0ffbd0 0x7efcfc195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:31.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.598+0000 7efd024e3700 1 -- 172.21.15.40:0/3774827101 >> 172.21.15.40:0/3774827101 conn(0x7efcfc0fa000 msgr2=0x7efcfc067670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:31.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.598+0000 7efd024e3700 1 -- 172.21.15.40:0/3774827101 shutdown_connections 2023-07-18T03:29:31.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.598+0000 7efd024e3700 1 -- 172.21.15.40:0/3774827101 wait complete. 2023-07-18T03:29:31.607 INFO:journalctl@ceph.mds.b.smithi064.stdout:Jul 18 03:29:31 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-b[123417]: 2023-07-18T03:29:31.353+0000 7fbb06327700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-18T03:29:31.608 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:31 smithi064 ceph-mon[98059]: pgmap v250: 97 pgs: 40 active+clean, 38 creating+peering, 19 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 0 op/s 2023-07-18T03:29:31.608 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:31 smithi064 ceph-mon[98059]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-18T03:29:31.608 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:31 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/791151305' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-18T03:29:31.609 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:31 smithi064 ceph-mon[98059]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-18T03:29:31.609 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:31 smithi064 ceph-mon[98059]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-18T03:29:31.609 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:31 smithi064 ceph-mon[98059]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-18T03:29:31.609 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:31 smithi064 ceph-mon[98059]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-07-18T03:29:31.609 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:31 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3774827101' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-18T03:29:31.609 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:31 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] up:active 2023-07-18T03:29:31.609 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:31 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3774827101' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-18T03:29:31.610 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:31 smithi064 ceph-mon[98059]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-18T03:29:31.610 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:31 smithi064 ceph-mon[98059]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-07-18T03:29:31.610 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:31 smithi064 ceph-mon[98059]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 2 up:standby-replay 7 up:standby 2023-07-18T03:29:31.613 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-07-18T03:29:31.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.753+0000 7fafdb47e700 1 -- 172.21.15.40:0/3340779832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd40ffbd0 msgr2=0x7fafd4100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:31.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.753+0000 7fafdb47e700 1 --2- 172.21.15.40:0/3340779832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd40ffbd0 0x7fafd4100020 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009990 tx=0x7fafd0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:31.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.753+0000 7fafdb47e700 1 -- 172.21.15.40:0/3340779832 shutdown_connections 2023-07-18T03:29:31.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.753+0000 7fafdb47e700 1 --2- 172.21.15.40:0/3340779832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafd4100560 0x7fafd4106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:31.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.753+0000 7fafdb47e700 1 --2- 172.21.15.40:0/3340779832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafd40fe9e0 0x7fafd40fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:31.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.753+0000 7fafdb47e700 1 --2- 172.21.15.40:0/3340779832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd40ffbd0 0x7fafd4100020 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:31.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.753+0000 7fafdb47e700 1 -- 172.21.15.40:0/3340779832 >> 172.21.15.40:0/3340779832 conn(0x7fafd40fa000 msgr2=0x7fafd40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:31.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.753+0000 7fafdb47e700 1 -- 172.21.15.40:0/3340779832 shutdown_connections 2023-07-18T03:29:31.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.753+0000 7fafdb47e700 1 -- 172.21.15.40:0/3340779832 wait complete. 2023-07-18T03:29:31.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafdb47e700 1 Processor -- start 2023-07-18T03:29:31.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafdb47e700 1 -- start start 2023-07-18T03:29:31.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafdb47e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafd40fe9e0 0x7fafd4102970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:31.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafdb47e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd40ffbd0 0x7fafd4100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:31.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafdb47e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafd4100560 0x7fafd41014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:31.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafdb47e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd4101c90 con 0x7fafd40ffbd0 2023-07-18T03:29:31.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafdb47e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd4101e00 con 0x7fafd40fe9e0 2023-07-18T03:29:31.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafdb47e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd419f8d0 con 0x7fafd4100560 2023-07-18T03:29:31.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafd8a19700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd40ffbd0 0x7fafd4100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:31.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafd8a19700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd40ffbd0 0x7fafd4100fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56160/0 (socket says 172.21.15.40:56160) 2023-07-18T03:29:31.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafd8a19700 1 -- 172.21.15.40:0/2162406829 learned_addr learned my addr 172.21.15.40:0/2162406829 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:31.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafd9a1b700 1 --2- 172.21.15.40:0/2162406829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafd4100560 0x7fafd41014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:31.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafd8a19700 1 -- 172.21.15.40:0/2162406829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafd4100560 msgr2=0x7fafd41014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:31.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafd8a19700 1 --2- 172.21.15.40:0/2162406829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafd4100560 0x7fafd41014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:31.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafd921a700 1 --2- 172.21.15.40:0/2162406829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafd40fe9e0 0x7fafd4102970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:31.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafd8a19700 1 -- 172.21.15.40:0/2162406829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafd40fe9e0 msgr2=0x7fafd4102970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:31.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafd8a19700 1 --2- 172.21.15.40:0/2162406829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafd40fe9e0 0x7fafd4102970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:31.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafd8a19700 1 -- 172.21.15.40:0/2162406829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd00095f0 con 0x7fafd40ffbd0 2023-07-18T03:29:31.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.754+0000 7fafd8a19700 1 --2- 172.21.15.40:0/2162406829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd40ffbd0 0x7fafd4100fb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fafd0004760 tx=0x7fafd0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:31.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.755+0000 7fafca7fc700 1 -- 172.21.15.40:0/2162406829 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd001d070 con 0x7fafd40ffbd0 2023-07-18T03:29:31.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.755+0000 7fafdb47e700 1 -- 172.21.15.40:0/2162406829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd419fb50 con 0x7fafd40ffbd0 2023-07-18T03:29:31.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.755+0000 7fafca7fc700 1 -- 172.21.15.40:0/2162406829 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafd0004200 con 0x7fafd40ffbd0 2023-07-18T03:29:31.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.755+0000 7fafca7fc700 1 -- 172.21.15.40:0/2162406829 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd00049f0 con 0x7fafd40ffbd0 2023-07-18T03:29:31.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.755+0000 7fafdb47e700 1 -- 172.21.15.40:0/2162406829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd41a0050 con 0x7fafd40ffbd0 2023-07-18T03:29:31.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.755+0000 7fafdb47e700 1 -- 172.21.15.40:0/2162406829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd410a1d0 con 0x7fafd40ffbd0 2023-07-18T03:29:31.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.757+0000 7fafca7fc700 1 -- 172.21.15.40:0/2162406829 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fafd0003d40 con 0x7fafd40ffbd0 2023-07-18T03:29:31.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.757+0000 7fafca7fc700 1 --2- 172.21.15.40:0/2162406829 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fafc00a1150 0x7fafc00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:31.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.757+0000 7fafd921a700 1 --2- 172.21.15.40:0/2162406829 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fafc00a1150 0x7fafc00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:31.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.757+0000 7fafca7fc700 1 -- 172.21.15.40:0/2162406829 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fafd00cdb10 con 0x7fafd40ffbd0 2023-07-18T03:29:31.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.758+0000 7fafd921a700 1 --2- 172.21.15.40:0/2162406829 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fafc00a1150 0x7fafc00a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fafc4005950 tx=0x7fafc40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:31.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:31.761+0000 7fafca7fc700 1 -- 172.21.15.40:0/2162406829 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fafd0097440 con 0x7fafd40ffbd0 2023-07-18T03:29:32.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.009+0000 7fafdb47e700 1 -- 172.21.15.40:0/2162406829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fafd410a390 con 0x7fafd40ffbd0 2023-07-18T03:29:32.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:31 smithi134 ceph-mon[98216]: pgmap v250: 97 pgs: 40 active+clean, 38 creating+peering, 19 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 0 op/s 2023-07-18T03:29:32.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:31 smithi134 ceph-mon[98216]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-18T03:29:32.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:31 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/791151305' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-18T03:29:32.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:31 smithi134 ceph-mon[98216]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-18T03:29:32.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:31 smithi134 ceph-mon[98216]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-18T03:29:32.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:31 smithi134 ceph-mon[98216]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-18T03:29:32.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:31 smithi134 ceph-mon[98216]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-07-18T03:29:32.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:31 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3774827101' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-18T03:29:32.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:31 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] up:active 2023-07-18T03:29:32.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:31 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3774827101' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-18T03:29:32.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:31 smithi134 ceph-mon[98216]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-18T03:29:32.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:31 smithi134 ceph-mon[98216]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-07-18T03:29:32.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:31 smithi134 ceph-mon[98216]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 2 up:standby-replay 7 up:standby 2023-07-18T03:29:32.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:31 smithi040 ceph-mon[96439]: pgmap v250: 97 pgs: 40 active+clean, 38 creating+peering, 19 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 0 op/s 2023-07-18T03:29:32.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:31 smithi040 ceph-mon[96439]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-18T03:29:32.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:31 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/791151305' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-18T03:29:32.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:31 smithi040 ceph-mon[96439]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-18T03:29:32.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:31 smithi040 ceph-mon[96439]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-18T03:29:32.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:31 smithi040 ceph-mon[96439]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-07-18T03:29:32.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:31 smithi040 ceph-mon[96439]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-07-18T03:29:32.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:31 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3774827101' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-18T03:29:32.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:31 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] up:active 2023-07-18T03:29:32.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:31 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3774827101' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-18T03:29:32.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:31 smithi040 ceph-mon[96439]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-18T03:29:32.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:31 smithi040 ceph-mon[96439]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-07-18T03:29:32.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:31 smithi040 ceph-mon[96439]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 2 up:standby-replay 7 up:standby 2023-07-18T03:29:32.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.620+0000 7fafca7fc700 1 -- 172.21.15.40:0/2162406829 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7fafd009aa60 con 0x7fafd40ffbd0 2023-07-18T03:29:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.623+0000 7fafdb47e700 1 -- 172.21.15.40:0/2162406829 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fafc00a1150 msgr2=0x7fafc00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.623+0000 7fafdb47e700 1 --2- 172.21.15.40:0/2162406829 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fafc00a1150 0x7fafc00a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fafc4005950 tx=0x7fafc40058e0 comp rx=0 tx=0).stop 2023-07-18T03:29:32.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.623+0000 7fafdb47e700 1 -- 172.21.15.40:0/2162406829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd40ffbd0 msgr2=0x7fafd4100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:32.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.623+0000 7fafdb47e700 1 --2- 172.21.15.40:0/2162406829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd40ffbd0 0x7fafd4100fb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fafd0004760 tx=0x7fafd0009f90 comp rx=0 tx=0).stop 2023-07-18T03:29:32.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.623+0000 7fafdb47e700 1 -- 172.21.15.40:0/2162406829 shutdown_connections 2023-07-18T03:29:32.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.623+0000 7fafdb47e700 1 --2- 172.21.15.40:0/2162406829 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fafc00a1150 0x7fafc00a3600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:32.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.623+0000 7fafdb47e700 1 --2- 172.21.15.40:0/2162406829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fafd4100560 0x7fafd41014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:32.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.623+0000 7fafdb47e700 1 --2- 172.21.15.40:0/2162406829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fafd40fe9e0 0x7fafd4102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:32.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.623+0000 7fafdb47e700 1 --2- 172.21.15.40:0/2162406829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafd40ffbd0 0x7fafd4100fb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:32.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.623+0000 7fafdb47e700 1 -- 172.21.15.40:0/2162406829 >> 172.21.15.40:0/2162406829 conn(0x7fafd40fa000 msgr2=0x7fafd40676f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:32.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.623+0000 7fafdb47e700 1 -- 172.21.15.40:0/2162406829 shutdown_connections 2023-07-18T03:29:32.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.623+0000 7fafdb47e700 1 -- 172.21.15.40:0/2162406829 wait complete. 2023-07-18T03:29:32.638 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2023-07-18T03:29:32.638 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-07-18T03:29:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.778+0000 7f48d25d5700 1 -- 172.21.15.40:0/296920584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48cc0fe9e0 msgr2=0x7f48cc0fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.778+0000 7f48d25d5700 1 --2- 172.21.15.40:0/296920584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48cc0fe9e0 0x7f48cc0fedf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009990 tx=0x7f48bc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.778+0000 7f48d25d5700 1 -- 172.21.15.40:0/296920584 shutdown_connections 2023-07-18T03:29:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.778+0000 7f48d25d5700 1 --2- 172.21.15.40:0/296920584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48cc100560 0x7f48cc106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.778+0000 7f48d25d5700 1 --2- 172.21.15.40:0/296920584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48cc0fe9e0 0x7f48cc0fedf0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.778+0000 7f48d25d5700 1 --2- 172.21.15.40:0/296920584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48cc0ffbd0 0x7f48cc100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.778+0000 7f48d25d5700 1 -- 172.21.15.40:0/296920584 >> 172.21.15.40:0/296920584 conn(0x7f48cc0fa000 msgr2=0x7f48cc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.778+0000 7f48d25d5700 1 -- 172.21.15.40:0/296920584 shutdown_connections 2023-07-18T03:29:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.778+0000 7f48d25d5700 1 -- 172.21.15.40:0/296920584 wait complete. 2023-07-18T03:29:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.779+0000 7f48d25d5700 1 Processor -- start 2023-07-18T03:29:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.779+0000 7f48d25d5700 1 -- start start 2023-07-18T03:29:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.779+0000 7f48d25d5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48cc0fe9e0 0x7f48cc195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.779+0000 7f48d25d5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48cc0ffbd0 0x7f48cc195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.779+0000 7f48d25d5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48cc100560 0x7f48cc19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.779+0000 7f48d25d5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc19d4c0 con 0x7f48cc0fe9e0 2023-07-18T03:29:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.779+0000 7f48d25d5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc19d600 con 0x7f48cc100560 2023-07-18T03:29:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.779+0000 7f48d25d5700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc19d790 con 0x7f48cc0ffbd0 2023-07-18T03:29:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.779+0000 7f48cbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48cc0fe9e0 0x7f48cc195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.779+0000 7f48cbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48cc0fe9e0 0x7f48cc195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56172/0 (socket says 172.21.15.40:56172) 2023-07-18T03:29:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.779+0000 7f48cbfff700 1 -- 172.21.15.40:0/3023760761 learned_addr learned my addr 172.21.15.40:0/3023760761 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:32.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.779+0000 7f48d0b72700 1 --2- 172.21.15.40:0/3023760761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48cc100560 0x7f48cc19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:32.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.779+0000 7f48cb7fe700 1 --2- 172.21.15.40:0/3023760761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48cc0ffbd0 0x7f48cc195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:32.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.779+0000 7f48cbfff700 1 -- 172.21.15.40:0/3023760761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48cc0ffbd0 msgr2=0x7f48cc195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:32.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.780+0000 7f48cbfff700 1 --2- 172.21.15.40:0/3023760761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48cc0ffbd0 0x7f48cc195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:32.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.780+0000 7f48cbfff700 1 -- 172.21.15.40:0/3023760761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48cc100560 msgr2=0x7f48cc19cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:32.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.780+0000 7f48cbfff700 1 --2- 172.21.15.40:0/3023760761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48cc100560 0x7f48cc19cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:32.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.780+0000 7f48cbfff700 1 -- 172.21.15.40:0/3023760761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48bc0095f0 con 0x7f48cc0fe9e0 2023-07-18T03:29:32.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.780+0000 7f48cb7fe700 1 --2- 172.21.15.40:0/3023760761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48cc0ffbd0 0x7f48cc195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:29:32.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.780+0000 7f48d0b72700 1 --2- 172.21.15.40:0/3023760761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48cc100560 0x7f48cc19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:29:32.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.780+0000 7f48cbfff700 1 --2- 172.21.15.40:0/3023760761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48cc0fe9e0 0x7f48cc195940 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f48bc004730 tx=0x7f48bc009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:32.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.780+0000 7f48c97fa700 1 -- 172.21.15.40:0/3023760761 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc01d070 con 0x7f48cc0fe9e0 2023-07-18T03:29:32.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.780+0000 7f48d25d5700 1 -- 172.21.15.40:0/3023760761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48cc19da10 con 0x7f48cc0fe9e0 2023-07-18T03:29:32.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.780+0000 7f48c97fa700 1 -- 172.21.15.40:0/3023760761 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48bc004cd0 con 0x7f48cc0fe9e0 2023-07-18T03:29:32.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.780+0000 7f48c97fa700 1 -- 172.21.15.40:0/3023760761 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc005530 con 0x7f48cc0fe9e0 2023-07-18T03:29:32.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.780+0000 7f48d25d5700 1 -- 172.21.15.40:0/3023760761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48cc19de30 con 0x7f48cc0fe9e0 2023-07-18T03:29:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.781+0000 7f48d25d5700 1 -- 172.21.15.40:0/3023760761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48cc10a200 con 0x7f48cc0fe9e0 2023-07-18T03:29:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.782+0000 7f48c97fa700 1 -- 172.21.15.40:0/3023760761 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f48bc00fbc0 con 0x7f48cc0fe9e0 2023-07-18T03:29:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.783+0000 7f48c97fa700 1 --2- 172.21.15.40:0/3023760761 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f48b40a1150 0x7f48b40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.783+0000 7f48cb7fe700 1 --2- 172.21.15.40:0/3023760761 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f48b40a1150 0x7f48b40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.783+0000 7f48c97fa700 1 -- 172.21.15.40:0/3023760761 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f48bc0cdfd0 con 0x7f48cc0fe9e0 2023-07-18T03:29:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.783+0000 7f48cb7fe700 1 --2- 172.21.15.40:0/3023760761 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f48b40a1150 0x7f48b40a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009cb0 tx=0x7f48c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:32.786+0000 7f48c97fa700 1 -- 172.21.15.40:0/3023760761 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f48bc097860 con 0x7f48cc0fe9e0 2023-07-18T03:29:33.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.018+0000 7f48d25d5700 1 -- 172.21.15.40:0/3023760761 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f48cc067de0 con 0x7f48b40a1150 2023-07-18T03:29:33.022 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:32 smithi040 ceph-mon[96439]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-07-18T03:29:33.022 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:32 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2162406829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-18T03:29:33.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.065+0000 7f48c97fa700 1 -- 172.21.15.40:0/3023760761 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f48cc067de0 con 0x7f48b40a1150 2023-07-18T03:29:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.067+0000 7f48d25d5700 1 -- 172.21.15.40:0/3023760761 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f48b40a1150 msgr2=0x7f48b40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.067+0000 7f48d25d5700 1 --2- 172.21.15.40:0/3023760761 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f48b40a1150 0x7f48b40a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009cb0 tx=0x7f48c0009260 comp rx=0 tx=0).stop 2023-07-18T03:29:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.067+0000 7f48d25d5700 1 -- 172.21.15.40:0/3023760761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48cc0fe9e0 msgr2=0x7f48cc195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.067+0000 7f48d25d5700 1 --2- 172.21.15.40:0/3023760761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48cc0fe9e0 0x7f48cc195940 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f48bc004730 tx=0x7f48bc009510 comp rx=0 tx=0).stop 2023-07-18T03:29:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.067+0000 7f48d25d5700 1 -- 172.21.15.40:0/3023760761 shutdown_connections 2023-07-18T03:29:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.067+0000 7f48d25d5700 1 --2- 172.21.15.40:0/3023760761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48cc100560 0x7f48cc19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.067+0000 7f48d25d5700 1 --2- 172.21.15.40:0/3023760761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48cc0fe9e0 0x7f48cc195940 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.067+0000 7f48d25d5700 1 --2- 172.21.15.40:0/3023760761 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f48b40a1150 0x7f48b40a3600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.067+0000 7f48d25d5700 1 --2- 172.21.15.40:0/3023760761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48cc0ffbd0 0x7f48cc195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.067+0000 7f48d25d5700 1 -- 172.21.15.40:0/3023760761 >> 172.21.15.40:0/3023760761 conn(0x7f48cc0fa000 msgr2=0x7f48cc0676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.067+0000 7f48d25d5700 1 -- 172.21.15.40:0/3023760761 shutdown_connections 2023-07-18T03:29:33.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.067+0000 7f48d25d5700 1 -- 172.21.15.40:0/3023760761 wait complete. 2023-07-18T03:29:33.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:32 smithi134 ceph-mon[98216]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-07-18T03:29:33.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:32 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2162406829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-18T03:29:33.076 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-07-18T03:29:33.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:32 smithi064 ceph-mon[98059]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-07-18T03:29:33.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:32 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2162406829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-18T03:29:33.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.208+0000 7f97deb58700 1 -- 172.21.15.40:0/657210371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97d80ffbd0 msgr2=0x7f97d8100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:33.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.208+0000 7f97deb58700 1 --2- 172.21.15.40:0/657210371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97d80ffbd0 0x7f97d8100020 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009990 tx=0x7f97c4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.209+0000 7f97deb58700 1 -- 172.21.15.40:0/657210371 shutdown_connections 2023-07-18T03:29:33.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.209+0000 7f97deb58700 1 --2- 172.21.15.40:0/657210371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97d8100560 0x7f97d8106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.209+0000 7f97deb58700 1 --2- 172.21.15.40:0/657210371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97d80fe9e0 0x7f97d80fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.209+0000 7f97deb58700 1 --2- 172.21.15.40:0/657210371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97d80ffbd0 0x7f97d8100020 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.209+0000 7f97deb58700 1 -- 172.21.15.40:0/657210371 >> 172.21.15.40:0/657210371 conn(0x7f97d80fa000 msgr2=0x7f97d80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:33.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.209+0000 7f97deb58700 1 -- 172.21.15.40:0/657210371 shutdown_connections 2023-07-18T03:29:33.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.209+0000 7f97deb58700 1 -- 172.21.15.40:0/657210371 wait complete. 2023-07-18T03:29:33.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.209+0000 7f97deb58700 1 Processor -- start 2023-07-18T03:29:33.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.209+0000 7f97deb58700 1 -- start start 2023-07-18T03:29:33.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.209+0000 7f97deb58700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97d80fe9e0 0x7f97d8102970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:33.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.209+0000 7f97deb58700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97d80ffbd0 0x7f97d8100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:33.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.209+0000 7f97deb58700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97d8100560 0x7f97d81014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:33.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.209+0000 7f97deb58700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d8101c90 con 0x7f97d80ffbd0 2023-07-18T03:29:33.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.209+0000 7f97deb58700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d8101e00 con 0x7f97d8100560 2023-07-18T03:29:33.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.209+0000 7f97deb58700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d819f8b0 con 0x7f97d80fe9e0 2023-07-18T03:29:33.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.209+0000 7f97d7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97d80ffbd0 0x7f97d8100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:33.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.210+0000 7f97d7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97d80ffbd0 0x7f97d8100fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56206/0 (socket says 172.21.15.40:56206) 2023-07-18T03:29:33.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.210+0000 7f97d7fff700 1 -- 172.21.15.40:0/3951659975 learned_addr learned my addr 172.21.15.40:0/3951659975 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:33.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.210+0000 7f97dc8f4700 1 --2- 172.21.15.40:0/3951659975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97d80fe9e0 0x7f97d8102970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:33.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.210+0000 7f97dd0f5700 1 --2- 172.21.15.40:0/3951659975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97d8100560 0x7f97d81014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:33.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.210+0000 7f97d7fff700 1 -- 172.21.15.40:0/3951659975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97d80fe9e0 msgr2=0x7f97d8102970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:33.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.210+0000 7f97d7fff700 1 --2- 172.21.15.40:0/3951659975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97d80fe9e0 0x7f97d8102970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.210+0000 7f97d7fff700 1 -- 172.21.15.40:0/3951659975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97d8100560 msgr2=0x7f97d81014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:33.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.210+0000 7f97d7fff700 1 --2- 172.21.15.40:0/3951659975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97d8100560 0x7f97d81014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.210+0000 7f97d7fff700 1 -- 172.21.15.40:0/3951659975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c40095f0 con 0x7f97d80ffbd0 2023-07-18T03:29:33.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.210+0000 7f97d7fff700 1 --2- 172.21.15.40:0/3951659975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97d80ffbd0 0x7f97d8100fb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f97c4005b40 tx=0x7f97c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:33.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.210+0000 7f97d5ffb700 1 -- 172.21.15.40:0/3951659975 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c401d070 con 0x7f97d80ffbd0 2023-07-18T03:29:33.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.210+0000 7f97d5ffb700 1 -- 172.21.15.40:0/3951659975 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97c4004ca0 con 0x7f97d80ffbd0 2023-07-18T03:29:33.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.210+0000 7f97deb58700 1 -- 172.21.15.40:0/3951659975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d819fb30 con 0x7f97d80ffbd0 2023-07-18T03:29:33.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.210+0000 7f97d5ffb700 1 -- 172.21.15.40:0/3951659975 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4005290 con 0x7f97d80ffbd0 2023-07-18T03:29:33.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.210+0000 7f97deb58700 1 -- 172.21.15.40:0/3951659975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d819ff50 con 0x7f97d80ffbd0 2023-07-18T03:29:33.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.211+0000 7f97deb58700 1 -- 172.21.15.40:0/3951659975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d810a1b0 con 0x7f97d80ffbd0 2023-07-18T03:29:33.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.212+0000 7f97d5ffb700 1 -- 172.21.15.40:0/3951659975 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f97c4003be0 con 0x7f97d80ffbd0 2023-07-18T03:29:33.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.213+0000 7f97d5ffb700 1 --2- 172.21.15.40:0/3951659975 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f97c80a1150 0x7f97c80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:33.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.213+0000 7f97d5ffb700 1 -- 172.21.15.40:0/3951659975 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f97c40cdbb0 con 0x7f97d80ffbd0 2023-07-18T03:29:33.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.213+0000 7f97dc8f4700 1 --2- 172.21.15.40:0/3951659975 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f97c80a1150 0x7f97c80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:33.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.214+0000 7f97dc8f4700 1 --2- 172.21.15.40:0/3951659975 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f97c80a1150 0x7f97c80a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f97d80ffa30 tx=0x7f97cc007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:33.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.217+0000 7f97d5ffb700 1 -- 172.21.15.40:0/3951659975 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f97c4097440 con 0x7f97d80ffbd0 2023-07-18T03:29:33.329 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:29:33 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:29:33.024+0000 7fe0b4708700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:29:33.329 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:29:33 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:29:33.024+0000 7fe0b4708700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:29:33.329 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:29:33 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:29:33.024+0000 7fe0b4708700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:29:33.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:29:33 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:29:33.024+0000 7fe0b4708700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:29:33.330 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:29:33 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:29:33.024+0000 7fe0b4708700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-18T03:29:33.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.451+0000 7f97deb58700 1 -- 172.21.15.40:0/3951659975 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f97d8067da0 con 0x7f97c80a1150 2023-07-18T03:29:33.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.457+0000 7f97d5ffb700 1 -- 172.21.15.40:0/3951659975 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f97d8067da0 con 0x7f97c80a1150 2023-07-18T03:29:33.453 INFO:teuthology.orchestra.run.smithi040.stdout:/volumes/_nogroup/sv_0/7161fc2f-d334-451e-a60d-6b3f53f6f119 2023-07-18T03:29:33.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.459+0000 7f97deb58700 1 -- 172.21.15.40:0/3951659975 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f97c80a1150 msgr2=0x7f97c80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:33.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.459+0000 7f97deb58700 1 --2- 172.21.15.40:0/3951659975 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f97c80a1150 0x7f97c80a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f97d80ffa30 tx=0x7f97cc007400 comp rx=0 tx=0).stop 2023-07-18T03:29:33.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.459+0000 7f97deb58700 1 -- 172.21.15.40:0/3951659975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97d80ffbd0 msgr2=0x7f97d8100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:33.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.459+0000 7f97deb58700 1 --2- 172.21.15.40:0/3951659975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97d80ffbd0 0x7f97d8100fb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f97c4005b40 tx=0x7f97c4004730 comp rx=0 tx=0).stop 2023-07-18T03:29:33.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.459+0000 7f97deb58700 1 -- 172.21.15.40:0/3951659975 shutdown_connections 2023-07-18T03:29:33.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.459+0000 7f97deb58700 1 --2- 172.21.15.40:0/3951659975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97d8100560 0x7f97d81014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.459+0000 7f97deb58700 1 --2- 172.21.15.40:0/3951659975 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f97c80a1150 0x7f97c80a3600 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.459+0000 7f97deb58700 1 --2- 172.21.15.40:0/3951659975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97d80fe9e0 0x7f97d8102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.459+0000 7f97deb58700 1 --2- 172.21.15.40:0/3951659975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97d80ffbd0 0x7f97d8100fb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.459+0000 7f97deb58700 1 -- 172.21.15.40:0/3951659975 >> 172.21.15.40:0/3951659975 conn(0x7f97d80fa000 msgr2=0x7f97d8067690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:33.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.459+0000 7f97deb58700 1 -- 172.21.15.40:0/3951659975 shutdown_connections 2023-07-18T03:29:33.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.459+0000 7f97deb58700 1 -- 172.21.15.40:0/3951659975 wait complete. 2023-07-18T03:29:33.467 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-07-18T03:29:33.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.601+0000 7f67eecff700 1 -- 172.21.15.40:0/813785882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e81005c0 msgr2=0x7f67e81029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:33.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.601+0000 7f67eecff700 1 --2- 172.21.15.40:0/813785882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e81005c0 0x7f67e81029a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009990 tx=0x7f67dc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.601+0000 7f67eecff700 1 -- 172.21.15.40:0/813785882 shutdown_connections 2023-07-18T03:29:33.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.601+0000 7f67eecff700 1 --2- 172.21.15.40:0/813785882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67e8105800 0x7f67e8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.601+0000 7f67eecff700 1 --2- 172.21.15.40:0/813785882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e81005c0 0x7f67e81029a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.601+0000 7f67eecff700 1 --2- 172.21.15.40:0/813785882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e8102ee0 0x7f67e81052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.601+0000 7f67eecff700 1 -- 172.21.15.40:0/813785882 >> 172.21.15.40:0/813785882 conn(0x7f67e80fa000 msgr2=0x7f67e80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:33.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.601+0000 7f67eecff700 1 -- 172.21.15.40:0/813785882 shutdown_connections 2023-07-18T03:29:33.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.601+0000 7f67eecff700 1 -- 172.21.15.40:0/813785882 wait complete. 2023-07-18T03:29:33.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67eecff700 1 Processor -- start 2023-07-18T03:29:33.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67eecff700 1 -- start start 2023-07-18T03:29:33.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67eecff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67e81005c0 0x7f67e80fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:33.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67eecff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e8102ee0 0x7f67e80fe570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:33.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67eecff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e8105800 0x7f67e80feab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:33.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67eecff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e80ff250 con 0x7f67e8102ee0 2023-07-18T03:29:33.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67eecff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e80ff3c0 con 0x7f67e81005c0 2023-07-18T03:29:33.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67eecff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e819f8b0 con 0x7f67e8105800 2023-07-18T03:29:33.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67e7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e8102ee0 0x7f67e80fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:33.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67e7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e8102ee0 0x7f67e80fe570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56220/0 (socket says 172.21.15.40:56220) 2023-07-18T03:29:33.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67e7fff700 1 -- 172.21.15.40:0/2121509370 learned_addr learned my addr 172.21.15.40:0/2121509370 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:33.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67ed29c700 1 --2- 172.21.15.40:0/2121509370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e8105800 0x7f67e80feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:33.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67eca9b700 1 --2- 172.21.15.40:0/2121509370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67e81005c0 0x7f67e80fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:33.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67e7fff700 1 -- 172.21.15.40:0/2121509370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e8105800 msgr2=0x7f67e80feab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:33.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67e7fff700 1 --2- 172.21.15.40:0/2121509370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e8105800 0x7f67e80feab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67e7fff700 1 -- 172.21.15.40:0/2121509370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67e81005c0 msgr2=0x7f67e80fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:33.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67e7fff700 1 --2- 172.21.15.40:0/2121509370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67e81005c0 0x7f67e80fff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67e7fff700 1 -- 172.21.15.40:0/2121509370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc0095f0 con 0x7f67e8102ee0 2023-07-18T03:29:33.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.602+0000 7f67e7fff700 1 --2- 172.21.15.40:0/2121509370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e8102ee0 0x7f67e80fe570 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f67d400d950 tx=0x7f67d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:33.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.603+0000 7f67e5ffb700 1 -- 172.21.15.40:0/2121509370 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d400f890 con 0x7f67e8102ee0 2023-07-18T03:29:33.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.603+0000 7f67e5ffb700 1 -- 172.21.15.40:0/2121509370 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67d4010460 con 0x7f67e8102ee0 2023-07-18T03:29:33.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.603+0000 7f67eecff700 1 -- 172.21.15.40:0/2121509370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e819fb90 con 0x7f67e8102ee0 2023-07-18T03:29:33.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.603+0000 7f67e5ffb700 1 -- 172.21.15.40:0/2121509370 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d400e610 con 0x7f67e8102ee0 2023-07-18T03:29:33.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.603+0000 7f67eecff700 1 -- 172.21.15.40:0/2121509370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e81a00a0 con 0x7f67e8102ee0 2023-07-18T03:29:33.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.604+0000 7f67eecff700 1 -- 172.21.15.40:0/2121509370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e80fbb70 con 0x7f67e8102ee0 2023-07-18T03:29:33.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.605+0000 7f67e5ffb700 1 -- 172.21.15.40:0/2121509370 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f67d40108d0 con 0x7f67e8102ee0 2023-07-18T03:29:33.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.606+0000 7f67e5ffb700 1 --2- 172.21.15.40:0/2121509370 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67d80a1150 0x7f67d80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:33.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.606+0000 7f67eca9b700 1 --2- 172.21.15.40:0/2121509370 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67d80a1150 0x7f67d80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:33.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.606+0000 7f67e5ffb700 1 -- 172.21.15.40:0/2121509370 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f67d40cc610 con 0x7f67e8102ee0 2023-07-18T03:29:33.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.606+0000 7f67eca9b700 1 --2- 172.21.15.40:0/2121509370 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67d80a1150 0x7f67d80a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00b2e0 tx=0x7f67dc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:33.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.610+0000 7f67e5ffb700 1 -- 172.21.15.40:0/2121509370 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f67d4095e10 con 0x7f67e8102ee0 2023-07-18T03:29:33.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:33 smithi040 ceph-mon[96439]: pgmap v251: 97 pgs: 89 active+clean, 8 creating+peering; 580 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 7 op/s 2023-07-18T03:29:33.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:33 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] up:active 2023-07-18T03:29:33.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:33 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2162406829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-18T03:29:33.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:33 smithi040 ceph-mon[96439]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-18T03:29:33.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:33 smithi040 ceph-mon[96439]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-07-18T03:29:33.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:33 smithi040 ceph-mon[96439]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2023-07-18T03:29:33.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:33 smithi040 ceph-mon[96439]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-07-18T03:29:33.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.849+0000 7f67eecff700 1 -- 172.21.15.40:0/2121509370 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f67e80609e0 con 0x7f67d80a1150 2023-07-18T03:29:33.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.878+0000 7f67e5ffb700 1 -- 172.21.15.40:0/2121509370 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f67e80609e0 con 0x7f67d80a1150 2023-07-18T03:29:33.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.880+0000 7f67eecff700 1 -- 172.21.15.40:0/2121509370 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67d80a1150 msgr2=0x7f67d80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:33.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.880+0000 7f67eecff700 1 --2- 172.21.15.40:0/2121509370 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67d80a1150 0x7f67d80a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00b2e0 tx=0x7f67dc005fb0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.880+0000 7f67eecff700 1 -- 172.21.15.40:0/2121509370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e8102ee0 msgr2=0x7f67e80fe570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:33.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.880+0000 7f67eecff700 1 --2- 172.21.15.40:0/2121509370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e8102ee0 0x7f67e80fe570 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f67d400d950 tx=0x7f67d400dc60 comp rx=0 tx=0).stop 2023-07-18T03:29:33.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.880+0000 7f67eecff700 1 -- 172.21.15.40:0/2121509370 shutdown_connections 2023-07-18T03:29:33.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.880+0000 7f67eecff700 1 --2- 172.21.15.40:0/2121509370 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67d80a1150 0x7f67d80a3600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.880+0000 7f67eecff700 1 --2- 172.21.15.40:0/2121509370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67e8105800 0x7f67e80feab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.880+0000 7f67eecff700 1 --2- 172.21.15.40:0/2121509370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67e81005c0 0x7f67e80fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.880+0000 7f67eecff700 1 --2- 172.21.15.40:0/2121509370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e8102ee0 0x7f67e80fe570 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:33.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.880+0000 7f67eecff700 1 -- 172.21.15.40:0/2121509370 >> 172.21.15.40:0/2121509370 conn(0x7f67e80fa000 msgr2=0x7f67e8101220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:33.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.880+0000 7f67eecff700 1 -- 172.21.15.40:0/2121509370 shutdown_connections 2023-07-18T03:29:33.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:33.880+0000 7f67eecff700 1 -- 172.21.15.40:0/2121509370 wait complete. 2023-07-18T03:29:33.889 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-07-18T03:29:34.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.021+0000 7fec11dab700 1 -- 172.21.15.40:0/491524594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec0c100d40 msgr2=0x7fec0c101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:34.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.021+0000 7fec11dab700 1 --2- 172.21.15.40:0/491524594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec0c100d40 0x7fec0c101190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fec00009990 tx=0x7fec00009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.021+0000 7fec11dab700 1 -- 172.21.15.40:0/491524594 shutdown_connections 2023-07-18T03:29:34.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.021+0000 7fec11dab700 1 --2- 172.21.15.40:0/491524594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec0c1016d0 0x7fec0c107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.021+0000 7fec11dab700 1 --2- 172.21.15.40:0/491524594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec0c0ffb50 0x7fec0c0fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.021+0000 7fec11dab700 1 --2- 172.21.15.40:0/491524594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec0c100d40 0x7fec0c101190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.021+0000 7fec11dab700 1 -- 172.21.15.40:0/491524594 >> 172.21.15.40:0/491524594 conn(0x7fec0c0fb170 msgr2=0x7fec0c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:34.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.021+0000 7fec11dab700 1 -- 172.21.15.40:0/491524594 shutdown_connections 2023-07-18T03:29:34.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.021+0000 7fec11dab700 1 -- 172.21.15.40:0/491524594 wait complete. 2023-07-18T03:29:34.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec11dab700 1 Processor -- start 2023-07-18T03:29:34.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec11dab700 1 -- start start 2023-07-18T03:29:34.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec11dab700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec0c0ffb50 0x7fec0c1957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:34.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec11dab700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec0c100d40 0x7fec0c195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:34.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec11dab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec0c1016d0 0x7fec0c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:34.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec11dab700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c19d320 con 0x7fec0c0ffb50 2023-07-18T03:29:34.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec11dab700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c19d460 con 0x7fec0c1016d0 2023-07-18T03:29:34.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec11dab700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c19d5f0 con 0x7fec0c100d40 2023-07-18T03:29:34.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec0b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec0c0ffb50 0x7fec0c1957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:34.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec0b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec0c0ffb50 0x7fec0c1957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56234/0 (socket says 172.21.15.40:56234) 2023-07-18T03:29:34.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec0b7fe700 1 -- 172.21.15.40:0/1865980038 learned_addr learned my addr 172.21.15.40:0/1865980038 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:34.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec0bfff700 1 --2- 172.21.15.40:0/1865980038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec0c1016d0 0x7fec0c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:34.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec0affd700 1 --2- 172.21.15.40:0/1865980038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec0c100d40 0x7fec0c195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:34.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec0b7fe700 1 -- 172.21.15.40:0/1865980038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec0c100d40 msgr2=0x7fec0c195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:34.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec0b7fe700 1 --2- 172.21.15.40:0/1865980038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec0c100d40 0x7fec0c195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec0b7fe700 1 -- 172.21.15.40:0/1865980038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec0c1016d0 msgr2=0x7fec0c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:34.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec0b7fe700 1 --2- 172.21.15.40:0/1865980038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec0c1016d0 0x7fec0c19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.022+0000 7fec0b7fe700 1 -- 172.21.15.40:0/1865980038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec000095f0 con 0x7fec0c0ffb50 2023-07-18T03:29:34.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.023+0000 7fec0affd700 1 --2- 172.21.15.40:0/1865980038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec0c100d40 0x7fec0c195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:29:34.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.023+0000 7fec0bfff700 1 --2- 172.21.15.40:0/1865980038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec0c1016d0 0x7fec0c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:29:34.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.023+0000 7fec0b7fe700 1 --2- 172.21.15.40:0/1865980038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec0c0ffb50 0x7fec0c1957a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7febfc00d980 tx=0x7febfc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:34.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.023+0000 7fec08ff9700 1 -- 172.21.15.40:0/1865980038 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc00de60 con 0x7fec0c0ffb50 2023-07-18T03:29:34.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.023+0000 7fec08ff9700 1 -- 172.21.15.40:0/1865980038 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febfc0044e0 con 0x7fec0c0ffb50 2023-07-18T03:29:34.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.023+0000 7fec08ff9700 1 -- 172.21.15.40:0/1865980038 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc003be0 con 0x7fec0c0ffb50 2023-07-18T03:29:34.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.023+0000 7fec11dab700 1 -- 172.21.15.40:0/1865980038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0c19d8d0 con 0x7fec0c0ffb50 2023-07-18T03:29:34.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.023+0000 7fec11dab700 1 -- 172.21.15.40:0/1865980038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0c19dde0 con 0x7fec0c0ffb50 2023-07-18T03:29:34.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.024+0000 7fec11dab700 1 -- 172.21.15.40:0/1865980038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec0c10b2b0 con 0x7fec0c0ffb50 2023-07-18T03:29:34.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.025+0000 7fec08ff9700 1 -- 172.21.15.40:0/1865980038 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7febfc009bb0 con 0x7fec0c0ffb50 2023-07-18T03:29:34.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.026+0000 7fec08ff9700 1 --2- 172.21.15.40:0/1865980038 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7febf40a11d0 0x7febf40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:34.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.026+0000 7fec0affd700 1 --2- 172.21.15.40:0/1865980038 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7febf40a11d0 0x7febf40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:34.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.026+0000 7fec08ff9700 1 -- 172.21.15.40:0/1865980038 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7febfc0cc4f0 con 0x7fec0c0ffb50 2023-07-18T03:29:34.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.026+0000 7fec0affd700 1 --2- 172.21.15.40:0/1865980038 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7febf40a11d0 0x7febf40a3680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fec00000c00 tx=0x7fec00005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:34.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.030+0000 7fec08ff9700 1 -- 172.21.15.40:0/1865980038 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7febfc095d90 con 0x7fec0c0ffb50 2023-07-18T03:29:34.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:33 smithi134 ceph-mon[98216]: pgmap v251: 97 pgs: 89 active+clean, 8 creating+peering; 580 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 7 op/s 2023-07-18T03:29:34.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:33 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] up:active 2023-07-18T03:29:34.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:33 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2162406829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-18T03:29:34.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:33 smithi134 ceph-mon[98216]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-18T03:29:34.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:33 smithi134 ceph-mon[98216]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-07-18T03:29:34.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:33 smithi134 ceph-mon[98216]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2023-07-18T03:29:34.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:33 smithi134 ceph-mon[98216]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-07-18T03:29:34.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:33 smithi064 ceph-mon[98059]: pgmap v251: 97 pgs: 89 active+clean, 8 creating+peering; 580 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 7 op/s 2023-07-18T03:29:34.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:33 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] up:active 2023-07-18T03:29:34.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:33 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2162406829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-18T03:29:34.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:33 smithi064 ceph-mon[98059]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-18T03:29:34.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:33 smithi064 ceph-mon[98059]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-07-18T03:29:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:33 smithi064 ceph-mon[98059]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2023-07-18T03:29:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:33 smithi064 ceph-mon[98059]: daemon mds.e is now active in filesystem cephfs as rank 3 2023-07-18T03:29:34.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.267+0000 7fec11dab700 1 -- 172.21.15.40:0/1865980038 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fec0c060950 con 0x7febf40a11d0 2023-07-18T03:29:34.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.271+0000 7fec08ff9700 1 -- 172.21.15.40:0/1865980038 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fec0c060950 con 0x7febf40a11d0 2023-07-18T03:29:34.268 INFO:teuthology.orchestra.run.smithi040.stdout:/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c 2023-07-18T03:29:34.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.273+0000 7fec11dab700 1 -- 172.21.15.40:0/1865980038 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7febf40a11d0 msgr2=0x7febf40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:34.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.273+0000 7fec11dab700 1 --2- 172.21.15.40:0/1865980038 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7febf40a11d0 0x7febf40a3680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fec00000c00 tx=0x7fec00005fb0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.273+0000 7fec11dab700 1 -- 172.21.15.40:0/1865980038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec0c0ffb50 msgr2=0x7fec0c1957a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:34.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.273+0000 7fec11dab700 1 --2- 172.21.15.40:0/1865980038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec0c0ffb50 0x7fec0c1957a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7febfc00d980 tx=0x7febfc00dd40 comp rx=0 tx=0).stop 2023-07-18T03:29:34.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.273+0000 7fec11dab700 1 -- 172.21.15.40:0/1865980038 shutdown_connections 2023-07-18T03:29:34.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.273+0000 7fec11dab700 1 --2- 172.21.15.40:0/1865980038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec0c1016d0 0x7fec0c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.273+0000 7fec11dab700 1 --2- 172.21.15.40:0/1865980038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec0c0ffb50 0x7fec0c1957a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.273+0000 7fec11dab700 1 --2- 172.21.15.40:0/1865980038 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7febf40a11d0 0x7febf40a3680 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.273+0000 7fec11dab700 1 --2- 172.21.15.40:0/1865980038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec0c100d40 0x7fec0c195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.273+0000 7fec11dab700 1 -- 172.21.15.40:0/1865980038 >> 172.21.15.40:0/1865980038 conn(0x7fec0c0fb170 msgr2=0x7fec0c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.273+0000 7fec11dab700 1 -- 172.21.15.40:0/1865980038 shutdown_connections 2023-07-18T03:29:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.273+0000 7fec11dab700 1 -- 172.21.15.40:0/1865980038 wait complete. 2023-07-18T03:29:34.282 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:29:34.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.412+0000 7f7295d5d700 1 -- 172.21.15.40:0/3745678372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72900ffbc0 msgr2=0x7f7290100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:34.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.412+0000 7f7295d5d700 1 --2- 172.21.15.40:0/3745678372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72900ffbc0 0x7f7290100010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7278009990 tx=0x7f7278009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.412+0000 7f7295d5d700 1 -- 172.21.15.40:0/3745678372 shutdown_connections 2023-07-18T03:29:34.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.412+0000 7f7295d5d700 1 --2- 172.21.15.40:0/3745678372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7290100550 0x7f7290106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.412+0000 7f7295d5d700 1 --2- 172.21.15.40:0/3745678372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72900fe9d0 0x7f72900fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.412+0000 7f7295d5d700 1 --2- 172.21.15.40:0/3745678372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72900ffbc0 0x7f7290100010 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.412+0000 7f7295d5d700 1 -- 172.21.15.40:0/3745678372 >> 172.21.15.40:0/3745678372 conn(0x7f72900f9fd0 msgr2=0x7f72900fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:34.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.412+0000 7f7295d5d700 1 -- 172.21.15.40:0/3745678372 shutdown_connections 2023-07-18T03:29:34.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.412+0000 7f7295d5d700 1 -- 172.21.15.40:0/3745678372 wait complete. 2023-07-18T03:29:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.412+0000 7f7295d5d700 1 Processor -- start 2023-07-18T03:29:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f7295d5d700 1 -- start start 2023-07-18T03:29:34.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f7295d5d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72900fe9d0 0x7f7290195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:34.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f7295d5d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72900ffbc0 0x7f7290195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:34.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f7295d5d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7290100550 0x7f729019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:34.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f7295d5d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729019d4f0 con 0x7f72900ffbc0 2023-07-18T03:29:34.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f7295d5d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729019d630 con 0x7f7290100550 2023-07-18T03:29:34.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f7295d5d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729019d7c0 con 0x7f72900fe9d0 2023-07-18T03:29:34.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f728effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72900ffbc0 0x7f7290195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:34.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f728effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72900ffbc0 0x7f7290195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56250/0 (socket says 172.21.15.40:56250) 2023-07-18T03:29:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f728effd700 1 -- 172.21.15.40:0/2063035372 learned_addr learned my addr 172.21.15.40:0/2063035372 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f728f7fe700 1 --2- 172.21.15.40:0/2063035372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72900fe9d0 0x7f7290195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f728effd700 1 -- 172.21.15.40:0/2063035372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72900fe9d0 msgr2=0x7f7290195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f728effd700 1 --2- 172.21.15.40:0/2063035372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72900fe9d0 0x7f7290195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f728ffff700 1 --2- 172.21.15.40:0/2063035372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7290100550 0x7f729019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f728effd700 1 -- 172.21.15.40:0/2063035372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7290100550 msgr2=0x7f729019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f728effd700 1 --2- 172.21.15.40:0/2063035372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7290100550 0x7f729019cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f728effd700 1 -- 172.21.15.40:0/2063035372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72780095f0 con 0x7f72900ffbc0 2023-07-18T03:29:34.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.413+0000 7f728effd700 1 --2- 172.21.15.40:0/2063035372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72900ffbc0 0x7f7290195eb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7278005910 tx=0x7f7278009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:34.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.414+0000 7f728cff9700 1 -- 172.21.15.40:0/2063035372 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727801d070 con 0x7f72900ffbc0 2023-07-18T03:29:34.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.414+0000 7f728cff9700 1 -- 172.21.15.40:0/2063035372 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72780041d0 con 0x7f72900ffbc0 2023-07-18T03:29:34.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.414+0000 7f728cff9700 1 -- 172.21.15.40:0/2063035372 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727800f460 con 0x7f72900ffbc0 2023-07-18T03:29:34.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.414+0000 7f7295d5d700 1 -- 172.21.15.40:0/2063035372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729019da40 con 0x7f72900ffbc0 2023-07-18T03:29:34.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.414+0000 7f7295d5d700 1 -- 172.21.15.40:0/2063035372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729019dee0 con 0x7f72900ffbc0 2023-07-18T03:29:34.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.415+0000 7f7295d5d700 1 -- 172.21.15.40:0/2063035372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729010a310 con 0x7f72900ffbc0 2023-07-18T03:29:34.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.416+0000 7f728cff9700 1 -- 172.21.15.40:0/2063035372 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f72780226e0 con 0x7f72900ffbc0 2023-07-18T03:29:34.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.416+0000 7f728cff9700 1 --2- 172.21.15.40:0/2063035372 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f727c0a1220 0x7f727c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:34.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.417+0000 7f728f7fe700 1 --2- 172.21.15.40:0/2063035372 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f727c0a1220 0x7f727c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:34.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.417+0000 7f728cff9700 1 -- 172.21.15.40:0/2063035372 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f72780cdd50 con 0x7f72900ffbc0 2023-07-18T03:29:34.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.417+0000 7f728f7fe700 1 --2- 172.21.15.40:0/2063035372 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f727c0a1220 0x7f727c0a36d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7280006fd0 tx=0x7f7280008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:34.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.420+0000 7f728cff9700 1 -- 172.21.15.40:0/2063035372 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7278097680 con 0x7f72900ffbc0 2023-07-18T03:29:34.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.650+0000 7f728cff9700 1 -- 172.21.15.40:0/2063035372 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7278097a50 con 0x7f72900ffbc0 2023-07-18T03:29:34.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.676+0000 7f7295d5d700 1 -- 172.21.15.40:0/2063035372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f729019e4f0 con 0x7f72900ffbc0 2023-07-18T03:29:34.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.676+0000 7f728cff9700 1 -- 172.21.15.40:0/2063035372 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10888 (secure 0 0 0) 0x7f72780aeac0 con 0x7f72900ffbc0 2023-07-18T03:29:34.673 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:34.673 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:29:26.718789+0000","modified":"2023-07-18T03:29:34.650353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24523,"mds_2":24482,"mds_3":24452,"mds_4":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14601":{"gid":14601,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1818167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1818167554},{"type":"v1","addr":"172.21.15.40:6839","nonce":1818167554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14613":{"gid":14613,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/3953074960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":3953074960},{"type":"v1","addr":"172.21.15.40:6841","nonce":3953074960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3509655945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3509655945},{"type":"v1","addr":"172.21.15.134:6837","nonce":3509655945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/3007039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3007039043},{"type":"v1","addr":"172.21.15.64:6839","nonce":3007039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:29:34.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.678+0000 7f7295d5d700 1 -- 172.21.15.40:0/2063035372 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f727c0a1220 msgr2=0x7f727c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:34.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.678+0000 7f7295d5d700 1 --2- 172.21.15.40:0/2063035372 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f727c0a1220 0x7f727c0a36d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7280006fd0 tx=0x7f7280008040 comp rx=0 tx=0).stop 2023-07-18T03:29:34.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.678+0000 7f7295d5d700 1 -- 172.21.15.40:0/2063035372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72900ffbc0 msgr2=0x7f7290195eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:34.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.678+0000 7f7295d5d700 1 --2- 172.21.15.40:0/2063035372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72900ffbc0 0x7f7290195eb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7278005910 tx=0x7f7278009f90 comp rx=0 tx=0).stop 2023-07-18T03:29:34.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.678+0000 7f7295d5d700 1 -- 172.21.15.40:0/2063035372 shutdown_connections 2023-07-18T03:29:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.678+0000 7f7295d5d700 1 --2- 172.21.15.40:0/2063035372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7290100550 0x7f729019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.678+0000 7f7295d5d700 1 --2- 172.21.15.40:0/2063035372 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f727c0a1220 0x7f727c0a36d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.678+0000 7f7295d5d700 1 --2- 172.21.15.40:0/2063035372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72900fe9d0 0x7f7290195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.678+0000 7f7295d5d700 1 --2- 172.21.15.40:0/2063035372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72900ffbc0 0x7f7290195eb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.679+0000 7f7295d5d700 1 -- 172.21.15.40:0/2063035372 >> 172.21.15.40:0/2063035372 conn(0x7f72900f9fd0 msgr2=0x7f7290067670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.679+0000 7f7295d5d700 1 -- 172.21.15.40:0/2063035372 shutdown_connections 2023-07-18T03:29:34.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.679+0000 7f7295d5d700 1 -- 172.21.15.40:0/2063035372 wait complete. 2023-07-18T03:29:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 27 2023-07-18T03:29:34.687 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:29:34.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.817+0000 7f3ab6a95700 1 -- 172.21.15.40:0/4228478588 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab0102ed0 msgr2=0x7f3ab01052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:34.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.817+0000 7f3ab6a95700 1 --2- 172.21.15.40:0/4228478588 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab0102ed0 0x7f3ab01052b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009990 tx=0x7f3a9c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.817+0000 7f3ab6a95700 1 -- 172.21.15.40:0/4228478588 shutdown_connections 2023-07-18T03:29:34.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.817+0000 7f3ab6a95700 1 --2- 172.21.15.40:0/4228478588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab01057f0 0x7f3ab0107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.817+0000 7f3ab6a95700 1 --2- 172.21.15.40:0/4228478588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab01005b0 0x7f3ab0102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.817+0000 7f3ab6a95700 1 --2- 172.21.15.40:0/4228478588 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab0102ed0 0x7f3ab01052b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.817+0000 7f3ab6a95700 1 -- 172.21.15.40:0/4228478588 >> 172.21.15.40:0/4228478588 conn(0x7f3ab00f9fd0 msgr2=0x7f3ab00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:34.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.817+0000 7f3ab6a95700 1 -- 172.21.15.40:0/4228478588 shutdown_connections 2023-07-18T03:29:34.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.817+0000 7f3ab6a95700 1 -- 172.21.15.40:0/4228478588 wait complete. 2023-07-18T03:29:34.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3ab6a95700 1 Processor -- start 2023-07-18T03:29:34.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3ab6a95700 1 -- start start 2023-07-18T03:29:34.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3ab6a95700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab01005b0 0x7f3ab00fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:34.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3ab6a95700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab0102ed0 0x7f3ab00fe560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:34.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3ab6a95700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab01057f0 0x7f3ab00feaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:34.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3ab6a95700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab00ff240 con 0x7f3ab0102ed0 2023-07-18T03:29:34.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3ab6a95700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab00ff3b0 con 0x7f3ab01057f0 2023-07-18T03:29:34.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3ab6a95700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab019f7e0 con 0x7f3ab01005b0 2023-07-18T03:29:34.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3aaffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab0102ed0 0x7f3ab00fe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:34.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3aaffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab0102ed0 0x7f3ab00fe560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56294/0 (socket says 172.21.15.40:56294) 2023-07-18T03:29:34.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3aaffff700 1 -- 172.21.15.40:0/1173512477 learned_addr learned my addr 172.21.15.40:0/1173512477 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:34.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3ab4831700 1 --2- 172.21.15.40:0/1173512477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab01005b0 0x7f3ab00fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:34.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3ab5032700 1 --2- 172.21.15.40:0/1173512477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab01057f0 0x7f3ab00feaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:34.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3aaffff700 1 -- 172.21.15.40:0/1173512477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab01005b0 msgr2=0x7f3ab00fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3aaffff700 1 --2- 172.21.15.40:0/1173512477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab01005b0 0x7f3ab00fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3aaffff700 1 -- 172.21.15.40:0/1173512477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab01057f0 msgr2=0x7f3ab00feaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3aaffff700 1 --2- 172.21.15.40:0/1173512477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab01057f0 0x7f3ab00feaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3aaffff700 1 -- 172.21.15.40:0/1173512477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c0095f0 con 0x7f3ab0102ed0 2023-07-18T03:29:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.818+0000 7f3ab4831700 1 --2- 172.21.15.40:0/1173512477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab01005b0 0x7f3ab00fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:29:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.819+0000 7f3aaffff700 1 --2- 172.21.15.40:0/1173512477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab0102ed0 0x7f3ab00fe560 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c000c00 tx=0x7f3a9c00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:34.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.819+0000 7f3aadffb700 1 -- 172.21.15.40:0/1173512477 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c01d070 con 0x7f3ab0102ed0 2023-07-18T03:29:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.819+0000 7f3ab6a95700 1 -- 172.21.15.40:0/1173512477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab019fa60 con 0x7f3ab0102ed0 2023-07-18T03:29:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.819+0000 7f3aadffb700 1 -- 172.21.15.40:0/1173512477 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a9c00fc00 con 0x7f3ab0102ed0 2023-07-18T03:29:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.819+0000 7f3aadffb700 1 -- 172.21.15.40:0/1173512477 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c017650 con 0x7f3ab0102ed0 2023-07-18T03:29:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.819+0000 7f3ab6a95700 1 -- 172.21.15.40:0/1173512477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab019ff10 con 0x7f3ab0102ed0 2023-07-18T03:29:34.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.820+0000 7f3ab6a95700 1 -- 172.21.15.40:0/1173512477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab00fbb40 con 0x7f3ab0102ed0 2023-07-18T03:29:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.821+0000 7f3aadffb700 1 -- 172.21.15.40:0/1173512477 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3a9c022470 con 0x7f3ab0102ed0 2023-07-18T03:29:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.822+0000 7f3aadffb700 1 --2- 172.21.15.40:0/1173512477 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3aa00a1220 0x7f3aa00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:34.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.822+0000 7f3ab4831700 1 --2- 172.21.15.40:0/1173512477 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3aa00a1220 0x7f3aa00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:34.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.822+0000 7f3aadffb700 1 -- 172.21.15.40:0/1173512477 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3a9c0cdc30 con 0x7f3ab0102ed0 2023-07-18T03:29:34.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.822+0000 7f3ab4831700 1 --2- 172.21.15.40:0/1173512477 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3aa00a1220 0x7f3aa00a36d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4005fd0 tx=0x7f3aa4005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:34.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:34.826+0000 7f3aadffb700 1 -- 172.21.15.40:0/1173512477 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3a9c097360 con 0x7f3ab0102ed0 2023-07-18T03:29:35.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.025+0000 7f3ab6a95700 1 -- 172.21.15.40:0/1173512477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ab002c9e0 con 0x7f3ab0102ed0 2023-07-18T03:29:35.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.027+0000 7f3aadffb700 1 -- 172.21.15.40:0/1173512477 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22071 (secure 0 0 0) 0x7f3a9c09a980 con 0x7f3ab0102ed0 2023-07-18T03:29:35.024 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:35.024 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":81,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:29:28.081282+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:29:35.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.029+0000 7f3ab6a95700 1 -- 172.21.15.40:0/1173512477 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3aa00a1220 msgr2=0x7f3aa00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:35.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.029+0000 7f3ab6a95700 1 --2- 172.21.15.40:0/1173512477 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3aa00a1220 0x7f3aa00a36d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4005fd0 tx=0x7f3aa4005f60 comp rx=0 tx=0).stop 2023-07-18T03:29:35.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.029+0000 7f3ab6a95700 1 -- 172.21.15.40:0/1173512477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab0102ed0 msgr2=0x7f3ab00fe560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:35.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.029+0000 7f3ab6a95700 1 --2- 172.21.15.40:0/1173512477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab0102ed0 0x7f3ab00fe560 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c000c00 tx=0x7f3a9c00f710 comp rx=0 tx=0).stop 2023-07-18T03:29:35.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.029+0000 7f3ab6a95700 1 -- 172.21.15.40:0/1173512477 shutdown_connections 2023-07-18T03:29:35.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.029+0000 7f3ab6a95700 1 --2- 172.21.15.40:0/1173512477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ab01057f0 0x7f3ab00feaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.029+0000 7f3ab6a95700 1 --2- 172.21.15.40:0/1173512477 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3aa00a1220 0x7f3aa00a36d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.029+0000 7f3ab6a95700 1 --2- 172.21.15.40:0/1173512477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab01005b0 0x7f3ab00fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.029+0000 7f3ab6a95700 1 --2- 172.21.15.40:0/1173512477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab0102ed0 0x7f3ab00fe560 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.029+0000 7f3ab6a95700 1 -- 172.21.15.40:0/1173512477 >> 172.21.15.40:0/1173512477 conn(0x7f3ab00f9fd0 msgr2=0x7f3ab0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:35.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.029+0000 7f3ab6a95700 1 -- 172.21.15.40:0/1173512477 shutdown_connections 2023-07-18T03:29:35.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.029+0000 7f3ab6a95700 1 -- 172.21.15.40:0/1173512477 wait complete. 2023-07-18T03:29:35.036 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:34 smithi040 ceph-mon[96439]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:29:35.037 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:34 smithi040 ceph-mon[96439]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:29:35.037 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:34 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] up:active 2023-07-18T03:29:35.037 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:34 smithi040 ceph-mon[96439]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-18T03:29:35.037 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:34 smithi040 ceph-mon[96439]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-18T03:29:35.037 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:34 smithi040 ceph-mon[96439]: Cluster is now healthy 2023-07-18T03:29:35.037 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:34 smithi040 ceph-mon[96439]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2023-07-18T03:29:35.038 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:34 smithi040 ceph-mon[96439]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-07-18T03:29:35.038 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:34 smithi040 ceph-mon[96439]: daemon mds.l is now active in filesystem cephfs as rank 4 2023-07-18T03:29:35.039 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-18T03:29:35.040 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:29:35.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:34 smithi134 ceph-mon[98216]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:29:35.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:34 smithi134 ceph-mon[98216]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:29:35.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:34 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] up:active 2023-07-18T03:29:35.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:34 smithi134 ceph-mon[98216]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-18T03:29:35.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:34 smithi134 ceph-mon[98216]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-18T03:29:35.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:34 smithi134 ceph-mon[98216]: Cluster is now healthy 2023-07-18T03:29:35.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:34 smithi134 ceph-mon[98216]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2023-07-18T03:29:35.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:34 smithi134 ceph-mon[98216]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-07-18T03:29:35.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:34 smithi134 ceph-mon[98216]: daemon mds.l is now active in filesystem cephfs as rank 4 2023-07-18T03:29:35.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:34 smithi064 ceph-mon[98059]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:29:35.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:34 smithi064 ceph-mon[98059]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:29:35.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:34 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] up:active 2023-07-18T03:29:35.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:34 smithi064 ceph-mon[98059]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-18T03:29:35.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:34 smithi064 ceph-mon[98059]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-18T03:29:35.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:34 smithi064 ceph-mon[98059]: Cluster is now healthy 2023-07-18T03:29:35.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:34 smithi064 ceph-mon[98059]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2023-07-18T03:29:35.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:34 smithi064 ceph-mon[98059]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-07-18T03:29:35.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:34 smithi064 ceph-mon[98059]: daemon mds.l is now active in filesystem cephfs as rank 4 2023-07-18T03:29:35.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.171+0000 7f4af7c95700 1 -- 172.21.15.40:0/3739701157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4af00ffbc0 msgr2=0x7f4af0100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:35.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.171+0000 7f4af7c95700 1 --2- 172.21.15.40:0/3739701157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4af00ffbc0 0x7f4af0100010 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009990 tx=0x7f4ae0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.171+0000 7f4af7c95700 1 -- 172.21.15.40:0/3739701157 shutdown_connections 2023-07-18T03:29:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.171+0000 7f4af7c95700 1 --2- 172.21.15.40:0/3739701157 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4af0100550 0x7f4af0106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.171+0000 7f4af7c95700 1 --2- 172.21.15.40:0/3739701157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4af00fe9d0 0x7f4af00fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.171+0000 7f4af7c95700 1 --2- 172.21.15.40:0/3739701157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4af00ffbc0 0x7f4af0100010 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.171+0000 7f4af7c95700 1 -- 172.21.15.40:0/3739701157 >> 172.21.15.40:0/3739701157 conn(0x7f4af00f9fd0 msgr2=0x7f4af00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.172+0000 7f4af7c95700 1 -- 172.21.15.40:0/3739701157 shutdown_connections 2023-07-18T03:29:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.172+0000 7f4af7c95700 1 -- 172.21.15.40:0/3739701157 wait complete. 2023-07-18T03:29:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.172+0000 7f4af7c95700 1 Processor -- start 2023-07-18T03:29:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.172+0000 7f4af7c95700 1 -- start start 2023-07-18T03:29:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.172+0000 7f4af7c95700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4af00fe9d0 0x7f4af01946f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.172+0000 7f4af7c95700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4af00ffbc0 0x7f4af0194c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.172+0000 7f4af7c95700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4af0100550 0x7f4af019bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.172+0000 7f4af7c95700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af019c270 con 0x7f4af0100550 2023-07-18T03:29:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.172+0000 7f4af7c95700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af019c3b0 con 0x7f4af00ffbc0 2023-07-18T03:29:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.172+0000 7f4af7c95700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af019c540 con 0x7f4af00fe9d0 2023-07-18T03:29:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.172+0000 7f4af6232700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4af0100550 0x7f4af019bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.173+0000 7f4af6232700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4af0100550 0x7f4af019bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56310/0 (socket says 172.21.15.40:56310) 2023-07-18T03:29:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.173+0000 7f4af6232700 1 -- 172.21.15.40:0/770702436 learned_addr learned my addr 172.21.15.40:0/770702436 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.173+0000 7f4af5230700 1 --2- 172.21.15.40:0/770702436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4af00ffbc0 0x7f4af0194c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.173+0000 7f4af5a31700 1 --2- 172.21.15.40:0/770702436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4af00fe9d0 0x7f4af01946f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.173+0000 7f4af6232700 1 -- 172.21.15.40:0/770702436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4af00fe9d0 msgr2=0x7f4af01946f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.173+0000 7f4af6232700 1 --2- 172.21.15.40:0/770702436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4af00fe9d0 0x7f4af01946f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.173+0000 7f4af6232700 1 -- 172.21.15.40:0/770702436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4af00ffbc0 msgr2=0x7f4af0194c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.173+0000 7f4af6232700 1 --2- 172.21.15.40:0/770702436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4af00ffbc0 0x7f4af0194c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.173+0000 7f4af6232700 1 -- 172.21.15.40:0/770702436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae00095f0 con 0x7f4af0100550 2023-07-18T03:29:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.173+0000 7f4af6232700 1 --2- 172.21.15.40:0/770702436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4af0100550 0x7f4af019bca0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800ada0 tx=0x7f4ae800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.173+0000 7f4ae6ffd700 1 -- 172.21.15.40:0/770702436 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae800f070 con 0x7f4af0100550 2023-07-18T03:29:35.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.173+0000 7f4af7c95700 1 -- 172.21.15.40:0/770702436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af019c820 con 0x7f4af0100550 2023-07-18T03:29:35.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.173+0000 7f4ae6ffd700 1 -- 172.21.15.40:0/770702436 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ae8007b30 con 0x7f4af0100550 2023-07-18T03:29:35.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.173+0000 7f4ae6ffd700 1 -- 172.21.15.40:0/770702436 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae80053e0 con 0x7f4af0100550 2023-07-18T03:29:35.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.173+0000 7f4af7c95700 1 -- 172.21.15.40:0/770702436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af019cd80 con 0x7f4af0100550 2023-07-18T03:29:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.174+0000 7f4af7c95700 1 -- 172.21.15.40:0/770702436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af010a1f0 con 0x7f4af0100550 2023-07-18T03:29:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.175+0000 7f4ae6ffd700 1 -- 172.21.15.40:0/770702436 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4ae800c980 con 0x7f4af0100550 2023-07-18T03:29:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.176+0000 7f4ae6ffd700 1 --2- 172.21.15.40:0/770702436 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4adc0a1220 0x7f4adc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.176+0000 7f4ae6ffd700 1 -- 172.21.15.40:0/770702436 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4ae80cbe90 con 0x7f4af0100550 2023-07-18T03:29:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.176+0000 7f4af5a31700 1 --2- 172.21.15.40:0/770702436 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4adc0a1220 0x7f4adc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.176+0000 7f4af5a31700 1 --2- 172.21.15.40:0/770702436 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4adc0a1220 0x7f4adc0a36d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4aec005fd0 tx=0x7f4aec005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.180+0000 7f4ae6ffd700 1 -- 172.21.15.40:0/770702436 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4ae80956c0 con 0x7f4af0100550 2023-07-18T03:29:35.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.373+0000 7f4af7c95700 1 -- 172.21.15.40:0/770702436 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4af0060950 con 0x7f4adc0a1220 2023-07-18T03:29:35.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.386+0000 7f4ae6ffd700 1 -- 172.21.15.40:0/770702436 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335252 (secure 0 0 0) 0x7f4af0060950 con 0x7f4adc0a1220 2023-07-18T03:29:35.386 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:35.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.389+0000 7f4af7c95700 1 -- 172.21.15.40:0/770702436 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4adc0a1220 msgr2=0x7f4adc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:35.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.389+0000 7f4af7c95700 1 --2- 172.21.15.40:0/770702436 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4adc0a1220 0x7f4adc0a36d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4aec005fd0 tx=0x7f4aec005dd0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.389+0000 7f4af7c95700 1 -- 172.21.15.40:0/770702436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4af0100550 msgr2=0x7f4af019bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:35.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.389+0000 7f4af7c95700 1 --2- 172.21.15.40:0/770702436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4af0100550 0x7f4af019bca0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800ada0 tx=0x7f4ae800c330 comp rx=0 tx=0).stop 2023-07-18T03:29:35.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.389+0000 7f4af7c95700 1 -- 172.21.15.40:0/770702436 shutdown_connections 2023-07-18T03:29:35.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.389+0000 7f4af7c95700 1 --2- 172.21.15.40:0/770702436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4af0100550 0x7f4af019bca0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.389+0000 7f4af7c95700 1 --2- 172.21.15.40:0/770702436 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4adc0a1220 0x7f4adc0a36d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.389+0000 7f4af7c95700 1 --2- 172.21.15.40:0/770702436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4af00fe9d0 0x7f4af01946f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.389+0000 7f4af7c95700 1 --2- 172.21.15.40:0/770702436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4af00ffbc0 0x7f4af0194c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.389+0000 7f4af7c95700 1 -- 172.21.15.40:0/770702436 >> 172.21.15.40:0/770702436 conn(0x7f4af00f9fd0 msgr2=0x7f4af00fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:35.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.390+0000 7f4af7c95700 1 -- 172.21.15.40:0/770702436 shutdown_connections 2023-07-18T03:29:35.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.390+0000 7f4af7c95700 1 -- 172.21.15.40:0/770702436 wait complete. 2023-07-18T03:29:35.390 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-18T03:29:35.398 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2023-07-18T03:29:33.814993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595266,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":131,"num_read_kb":114,"num_write":279,"num_write_kb":2974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":283,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335492,"kb_used_data":12068,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606332,"statfs":{"total":1151940427776,"available":1151596883968,"internally_reserved":0,"allocated":12357632,"data_stored":6787763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4898,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":98,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.540621"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085961+0000","last_change":"2023-07-18T03:29:27.101623+0000","last_active":"2023-07-18T03:29:28.085961+0000","last_peered":"2023-07-18T03:29:28.085961+0000","last_clean":"2023-07-18T03:29:28.085961+0000","last_became_active":"2023-07-18T03:29:27.101297+0000","last_became_peered":"2023-07-18T03:29:27.101297+0000","last_unstale":"2023-07-18T03:29:28.085961+0000","last_undegraded":"2023-07-18T03:29:28.085961+0000","last_fullsized":"2023-07-18T03:29:28.085961+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:34:31.169755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248873+0000","last_change":"2023-07-18T03:29:27.104159+0000","last_active":"2023-07-18T03:29:28.248873+0000","last_peered":"2023-07-18T03:29:28.248873+0000","last_clean":"2023-07-18T03:29:28.248873+0000","last_became_active":"2023-07-18T03:29:27.103830+0000","last_became_peered":"2023-07-18T03:29:27.103830+0000","last_unstale":"2023-07-18T03:29:28.248873+0000","last_undegraded":"2023-07-18T03:29:28.248873+0000","last_fullsized":"2023-07-18T03:29:28.248873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:44:49.917899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249078+0000","last_change":"2023-07-18T03:29:27.098858+0000","last_active":"2023-07-18T03:29:28.249078+0000","last_peered":"2023-07-18T03:29:28.249078+0000","last_clean":"2023-07-18T03:29:28.249078+0000","last_became_active":"2023-07-18T03:29:27.098540+0000","last_became_peered":"2023-07-18T03:29:27.098540+0000","last_unstale":"2023-07-18T03:29:28.249078+0000","last_undegraded":"2023-07-18T03:29:28.249078+0000","last_fullsized":"2023-07-18T03:29:28.249078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:55:46.914785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248698+0000","last_change":"2023-07-18T03:29:27.103106+0000","last_active":"2023-07-18T03:29:28.248698+0000","last_peered":"2023-07-18T03:29:28.248698+0000","last_clean":"2023-07-18T03:29:28.248698+0000","last_became_active":"2023-07-18T03:29:27.102780+0000","last_became_peered":"2023-07-18T03:29:27.102780+0000","last_unstale":"2023-07-18T03:29:28.248698+0000","last_undegraded":"2023-07-18T03:29:28.248698+0000","last_fullsized":"2023-07-18T03:29:28.248698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:51:36.548110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248934+0000","last_change":"2023-07-18T03:29:27.095432+0000","last_active":"2023-07-18T03:29:28.248934+0000","last_peered":"2023-07-18T03:29:28.248934+0000","last_clean":"2023-07-18T03:29:28.248934+0000","last_became_active":"2023-07-18T03:29:27.095103+0000","last_became_peered":"2023-07-18T03:29:27.095103+0000","last_unstale":"2023-07-18T03:29:28.248934+0000","last_undegraded":"2023-07-18T03:29:28.248934+0000","last_fullsized":"2023-07-18T03:29:28.248934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:50:40.336206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085493+0000","last_change":"2023-07-18T03:29:27.091412+0000","last_active":"2023-07-18T03:29:28.085493+0000","last_peered":"2023-07-18T03:29:28.085493+0000","last_clean":"2023-07-18T03:29:28.085493+0000","last_became_active":"2023-07-18T03:29:27.091060+0000","last_became_peered":"2023-07-18T03:29:27.091060+0000","last_unstale":"2023-07-18T03:29:28.085493+0000","last_undegraded":"2023-07-18T03:29:28.085493+0000","last_fullsized":"2023-07-18T03:29:28.085493+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:31:43.393707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339267+0000","last_change":"2023-07-18T03:29:27.105223+0000","last_active":"2023-07-18T03:29:28.339267+0000","last_peered":"2023-07-18T03:29:28.339267+0000","last_clean":"2023-07-18T03:29:28.339267+0000","last_became_active":"2023-07-18T03:29:27.104926+0000","last_became_peered":"2023-07-18T03:29:27.104926+0000","last_unstale":"2023-07-18T03:29:28.339267+0000","last_undegraded":"2023-07-18T03:29:28.339267+0000","last_fullsized":"2023-07-18T03:29:28.339267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:43:23.908336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248819+0000","last_change":"2023-07-18T03:29:27.093722+0000","last_active":"2023-07-18T03:29:28.248819+0000","last_peered":"2023-07-18T03:29:28.248819+0000","last_clean":"2023-07-18T03:29:28.248819+0000","last_became_active":"2023-07-18T03:29:27.093103+0000","last_became_peered":"2023-07-18T03:29:27.093103+0000","last_unstale":"2023-07-18T03:29:28.248819+0000","last_undegraded":"2023-07-18T03:29:28.248819+0000","last_fullsized":"2023-07-18T03:29:28.248819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:03:07.122936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249009+0000","last_change":"2023-07-18T03:29:27.094681+0000","last_active":"2023-07-18T03:29:28.249009+0000","last_peered":"2023-07-18T03:29:28.249009+0000","last_clean":"2023-07-18T03:29:28.249009+0000","last_became_active":"2023-07-18T03:29:27.094284+0000","last_became_peered":"2023-07-18T03:29:27.094284+0000","last_unstale":"2023-07-18T03:29:28.249009+0000","last_undegraded":"2023-07-18T03:29:28.249009+0000","last_fullsized":"2023-07-18T03:29:28.249009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:25:08.820895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085737+0000","last_change":"2023-07-18T03:29:27.100759+0000","last_active":"2023-07-18T03:29:28.085737+0000","last_peered":"2023-07-18T03:29:28.085737+0000","last_clean":"2023-07-18T03:29:28.085737+0000","last_became_active":"2023-07-18T03:29:27.096660+0000","last_became_peered":"2023-07-18T03:29:27.096660+0000","last_unstale":"2023-07-18T03:29:28.085737+0000","last_undegraded":"2023-07-18T03:29:28.085737+0000","last_fullsized":"2023-07-18T03:29:28.085737+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:56:10.159967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339718+0000","last_change":"2023-07-18T03:29:27.107251+0000","last_active":"2023-07-18T03:29:28.339718+0000","last_peered":"2023-07-18T03:29:28.339718+0000","last_clean":"2023-07-18T03:29:28.339718+0000","last_became_active":"2023-07-18T03:29:27.106895+0000","last_became_peered":"2023-07-18T03:29:27.106895+0000","last_unstale":"2023-07-18T03:29:28.339718+0000","last_undegraded":"2023-07-18T03:29:28.339718+0000","last_fullsized":"2023-07-18T03:29:28.339718+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:12:15.805607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248978+0000","last_change":"2023-07-18T03:29:27.103905+0000","last_active":"2023-07-18T03:29:28.248978+0000","last_peered":"2023-07-18T03:29:28.248978+0000","last_clean":"2023-07-18T03:29:28.248978+0000","last_became_active":"2023-07-18T03:29:27.103021+0000","last_became_peered":"2023-07-18T03:29:27.103021+0000","last_unstale":"2023-07-18T03:29:28.248978+0000","last_undegraded":"2023-07-18T03:29:28.248978+0000","last_fullsized":"2023-07-18T03:29:28.248978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:48:20.771586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249855+0000","last_change":"2023-07-18T03:29:27.103529+0000","last_active":"2023-07-18T03:29:28.249855+0000","last_peered":"2023-07-18T03:29:28.249855+0000","last_clean":"2023-07-18T03:29:28.249855+0000","last_became_active":"2023-07-18T03:29:27.103217+0000","last_became_peered":"2023-07-18T03:29:27.103217+0000","last_unstale":"2023-07-18T03:29:28.249855+0000","last_undegraded":"2023-07-18T03:29:28.249855+0000","last_fullsized":"2023-07-18T03:29:28.249855+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:37:25.309228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248500+0000","last_change":"2023-07-18T03:29:27.096640+0000","last_active":"2023-07-18T03:29:28.248500+0000","last_peered":"2023-07-18T03:29:28.248500+0000","last_clean":"2023-07-18T03:29:28.248500+0000","last_became_active":"2023-07-18T03:29:27.096347+0000","last_became_peered":"2023-07-18T03:29:27.096347+0000","last_unstale":"2023-07-18T03:29:28.248500+0000","last_undegraded":"2023-07-18T03:29:28.248500+0000","last_fullsized":"2023-07-18T03:29:28.248500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:31:27.280331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248292+0000","last_change":"2023-07-18T03:29:27.095910+0000","last_active":"2023-07-18T03:29:28.248292+0000","last_peered":"2023-07-18T03:29:28.248292+0000","last_clean":"2023-07-18T03:29:28.248292+0000","last_became_active":"2023-07-18T03:29:27.095556+0000","last_became_peered":"2023-07-18T03:29:27.095556+0000","last_unstale":"2023-07-18T03:29:28.248292+0000","last_undegraded":"2023-07-18T03:29:28.248292+0000","last_fullsized":"2023-07-18T03:29:28.248292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:09:59.583920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248551+0000","last_change":"2023-07-18T03:29:27.108220+0000","last_active":"2023-07-18T03:29:28.248551+0000","last_peered":"2023-07-18T03:29:28.248551+0000","last_clean":"2023-07-18T03:29:28.248551+0000","last_became_active":"2023-07-18T03:29:27.107869+0000","last_became_peered":"2023-07-18T03:29:27.107869+0000","last_unstale":"2023-07-18T03:29:28.248551+0000","last_undegraded":"2023-07-18T03:29:28.248551+0000","last_fullsized":"2023-07-18T03:29:28.248551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:53:30.964666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.086002+0000","last_change":"2023-07-18T03:29:27.100634+0000","last_active":"2023-07-18T03:29:28.086002+0000","last_peered":"2023-07-18T03:29:28.086002+0000","last_clean":"2023-07-18T03:29:28.086002+0000","last_became_active":"2023-07-18T03:29:27.100316+0000","last_became_peered":"2023-07-18T03:29:27.100316+0000","last_unstale":"2023-07-18T03:29:28.086002+0000","last_undegraded":"2023-07-18T03:29:28.086002+0000","last_fullsized":"2023-07-18T03:29:28.086002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:35:30.023399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.235377+0000","last_change":"2023-07-18T03:29:27.100322+0000","last_active":"2023-07-18T03:29:28.235377+0000","last_peered":"2023-07-18T03:29:28.235377+0000","last_clean":"2023-07-18T03:29:28.235377+0000","last_became_active":"2023-07-18T03:29:27.099782+0000","last_became_peered":"2023-07-18T03:29:27.099782+0000","last_unstale":"2023-07-18T03:29:28.235377+0000","last_undegraded":"2023-07-18T03:29:28.235377+0000","last_fullsized":"2023-07-18T03:29:28.235377+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:28:12.377533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129942+0000","last_change":"2023-07-18T03:29:27.096639+0000","last_active":"2023-07-18T03:29:28.129942+0000","last_peered":"2023-07-18T03:29:28.129942+0000","last_clean":"2023-07-18T03:29:28.129942+0000","last_became_active":"2023-07-18T03:29:27.096348+0000","last_became_peered":"2023-07-18T03:29:27.096348+0000","last_unstale":"2023-07-18T03:29:28.129942+0000","last_undegraded":"2023-07-18T03:29:28.129942+0000","last_fullsized":"2023-07-18T03:29:28.129942+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:05:09.093995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.235481+0000","last_change":"2023-07-18T03:29:26.287722+0000","last_active":"2023-07-18T03:29:28.235481+0000","last_peered":"2023-07-18T03:29:28.235481+0000","last_clean":"2023-07-18T03:29:28.235481+0000","last_became_active":"2023-07-18T03:29:26.287327+0000","last_became_peered":"2023-07-18T03:29:26.287327+0000","last_unstale":"2023-07-18T03:29:28.235481+0000","last_undegraded":"2023-07-18T03:29:28.235481+0000","last_fullsized":"2023-07-18T03:29:28.235481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:57:15.630654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249206+0000","last_change":"2023-07-18T03:29:27.092771+0000","last_active":"2023-07-18T03:29:28.249206+0000","last_peered":"2023-07-18T03:29:28.249206+0000","last_clean":"2023-07-18T03:29:28.249206+0000","last_became_active":"2023-07-18T03:29:27.092456+0000","last_became_peered":"2023-07-18T03:29:27.092456+0000","last_unstale":"2023-07-18T03:29:28.249206+0000","last_undegraded":"2023-07-18T03:29:28.249206+0000","last_fullsized":"2023-07-18T03:29:28.249206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:02:05.004006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249555+0000","last_change":"2023-07-18T03:29:26.289683+0000","last_active":"2023-07-18T03:29:28.249555+0000","last_peered":"2023-07-18T03:29:28.249555+0000","last_clean":"2023-07-18T03:29:28.249555+0000","last_became_active":"2023-07-18T03:29:26.289356+0000","last_became_peered":"2023-07-18T03:29:26.289356+0000","last_unstale":"2023-07-18T03:29:28.249555+0000","last_undegraded":"2023-07-18T03:29:28.249555+0000","last_fullsized":"2023-07-18T03:29:28.249555+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:02:04.478209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249301+0000","last_change":"2023-07-18T03:29:27.098709+0000","last_active":"2023-07-18T03:29:28.249301+0000","last_peered":"2023-07-18T03:29:28.249301+0000","last_clean":"2023-07-18T03:29:28.249301+0000","last_became_active":"2023-07-18T03:29:27.098395+0000","last_became_peered":"2023-07-18T03:29:27.098395+0000","last_unstale":"2023-07-18T03:29:28.249301+0000","last_undegraded":"2023-07-18T03:29:28.249301+0000","last_fullsized":"2023-07-18T03:29:28.249301+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:26:19.426023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.608784+0000","last_change":"2023-07-18T03:29:26.297119+0000","last_active":"2023-07-18T03:29:31.608784+0000","last_peered":"2023-07-18T03:29:31.608784+0000","last_clean":"2023-07-18T03:29:31.608784+0000","last_became_active":"2023-07-18T03:29:26.296649+0000","last_became_peered":"2023-07-18T03:29:26.296649+0000","last_unstale":"2023-07-18T03:29:31.608784+0000","last_undegraded":"2023-07-18T03:29:31.608784+0000","last_fullsized":"2023-07-18T03:29:31.608784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:51:17.267059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248457+0000","last_change":"2023-07-18T03:29:27.105511+0000","last_active":"2023-07-18T03:29:28.248457+0000","last_peered":"2023-07-18T03:29:28.248457+0000","last_clean":"2023-07-18T03:29:28.248457+0000","last_became_active":"2023-07-18T03:29:27.104946+0000","last_became_peered":"2023-07-18T03:29:27.104946+0000","last_unstale":"2023-07-18T03:29:28.248457+0000","last_undegraded":"2023-07-18T03:29:28.248457+0000","last_fullsized":"2023-07-18T03:29:28.248457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:42:13.652404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249222+0000","last_change":"2023-07-18T03:29:26.290708+0000","last_active":"2023-07-18T03:29:28.249222+0000","last_peered":"2023-07-18T03:29:28.249222+0000","last_clean":"2023-07-18T03:29:28.249222+0000","last_became_active":"2023-07-18T03:29:26.290383+0000","last_became_peered":"2023-07-18T03:29:26.290383+0000","last_unstale":"2023-07-18T03:29:28.249222+0000","last_undegraded":"2023-07-18T03:29:28.249222+0000","last_fullsized":"2023-07-18T03:29:28.249222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:34:22.740683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.084957+0000","last_change":"2023-07-18T03:29:27.096630+0000","last_active":"2023-07-18T03:29:28.084957+0000","last_peered":"2023-07-18T03:29:28.084957+0000","last_clean":"2023-07-18T03:29:28.084957+0000","last_became_active":"2023-07-18T03:29:27.096337+0000","last_became_peered":"2023-07-18T03:29:27.096337+0000","last_unstale":"2023-07-18T03:29:28.084957+0000","last_undegraded":"2023-07-18T03:29:28.084957+0000","last_fullsized":"2023-07-18T03:29:28.084957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:23:46.241177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.628062+0000","last_change":"2023-07-18T03:29:26.291622+0000","last_active":"2023-07-18T03:29:30.628062+0000","last_peered":"2023-07-18T03:29:30.628062+0000","last_clean":"2023-07-18T03:29:30.628062+0000","last_became_active":"2023-07-18T03:29:26.291317+0000","last_became_peered":"2023-07-18T03:29:26.291317+0000","last_unstale":"2023-07-18T03:29:30.628062+0000","last_undegraded":"2023-07-18T03:29:30.628062+0000","last_fullsized":"2023-07-18T03:29:30.628062+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:37:53.996938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248914+0000","last_change":"2023-07-18T03:29:27.093107+0000","last_active":"2023-07-18T03:29:28.248914+0000","last_peered":"2023-07-18T03:29:28.248914+0000","last_clean":"2023-07-18T03:29:28.248914+0000","last_became_active":"2023-07-18T03:29:27.092785+0000","last_became_peered":"2023-07-18T03:29:27.092785+0000","last_unstale":"2023-07-18T03:29:28.248914+0000","last_undegraded":"2023-07-18T03:29:28.248914+0000","last_fullsized":"2023-07-18T03:29:28.248914+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:18:41.599570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085394+0000","last_change":"2023-07-18T03:29:26.290493+0000","last_active":"2023-07-18T03:29:28.085394+0000","last_peered":"2023-07-18T03:29:28.085394+0000","last_clean":"2023-07-18T03:29:28.085394+0000","last_became_active":"2023-07-18T03:29:26.290187+0000","last_became_peered":"2023-07-18T03:29:26.290187+0000","last_unstale":"2023-07-18T03:29:28.085394+0000","last_undegraded":"2023-07-18T03:29:28.085394+0000","last_fullsized":"2023-07-18T03:29:28.085394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:14:58.224602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.084989+0000","last_change":"2023-07-18T03:29:27.095129+0000","last_active":"2023-07-18T03:29:28.084989+0000","last_peered":"2023-07-18T03:29:28.084989+0000","last_clean":"2023-07-18T03:29:28.084989+0000","last_became_active":"2023-07-18T03:29:27.094821+0000","last_became_peered":"2023-07-18T03:29:27.094821+0000","last_unstale":"2023-07-18T03:29:28.084989+0000","last_undegraded":"2023-07-18T03:29:28.084989+0000","last_fullsized":"2023-07-18T03:29:28.084989+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:33:21.671061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.610770+0000","last_change":"2023-07-18T03:29:26.285860+0000","last_active":"2023-07-18T03:29:31.610770+0000","last_peered":"2023-07-18T03:29:31.610770+0000","last_clean":"2023-07-18T03:29:31.610770+0000","last_became_active":"2023-07-18T03:29:26.285491+0000","last_became_peered":"2023-07-18T03:29:26.285491+0000","last_unstale":"2023-07-18T03:29:31.610770+0000","last_undegraded":"2023-07-18T03:29:31.610770+0000","last_fullsized":"2023-07-18T03:29:31.610770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:14:20.414177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248682+0000","last_change":"2023-07-18T03:29:27.093861+0000","last_active":"2023-07-18T03:29:28.248682+0000","last_peered":"2023-07-18T03:29:28.248682+0000","last_clean":"2023-07-18T03:29:28.248682+0000","last_became_active":"2023-07-18T03:29:27.093445+0000","last_became_peered":"2023-07-18T03:29:27.093445+0000","last_unstale":"2023-07-18T03:29:28.248682+0000","last_undegraded":"2023-07-18T03:29:28.248682+0000","last_fullsized":"2023-07-18T03:29:28.248682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:05:28.911877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085209+0000","last_change":"2023-07-18T03:29:26.288108+0000","last_active":"2023-07-18T03:29:28.085209+0000","last_peered":"2023-07-18T03:29:28.085209+0000","last_clean":"2023-07-18T03:29:28.085209+0000","last_became_active":"2023-07-18T03:29:26.287531+0000","last_became_peered":"2023-07-18T03:29:26.287531+0000","last_unstale":"2023-07-18T03:29:28.085209+0000","last_undegraded":"2023-07-18T03:29:28.085209+0000","last_fullsized":"2023-07-18T03:29:28.085209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:35:36.416356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.235430+0000","last_change":"2023-07-18T03:29:27.092908+0000","last_active":"2023-07-18T03:29:28.235430+0000","last_peered":"2023-07-18T03:29:28.235430+0000","last_clean":"2023-07-18T03:29:28.235430+0000","last_became_active":"2023-07-18T03:29:27.092566+0000","last_became_peered":"2023-07-18T03:29:27.092566+0000","last_unstale":"2023-07-18T03:29:28.235430+0000","last_undegraded":"2023-07-18T03:29:28.235430+0000","last_fullsized":"2023-07-18T03:29:28.235430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:22:47.560376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085581+0000","last_change":"2023-07-18T03:29:26.294251+0000","last_active":"2023-07-18T03:29:28.085581+0000","last_peered":"2023-07-18T03:29:28.085581+0000","last_clean":"2023-07-18T03:29:28.085581+0000","last_became_active":"2023-07-18T03:29:26.293847+0000","last_became_peered":"2023-07-18T03:29:26.293847+0000","last_unstale":"2023-07-18T03:29:28.085581+0000","last_undegraded":"2023-07-18T03:29:28.085581+0000","last_fullsized":"2023-07-18T03:29:28.085581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:51:26.530234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248737+0000","last_change":"2023-07-18T03:29:27.106551+0000","last_active":"2023-07-18T03:29:28.248737+0000","last_peered":"2023-07-18T03:29:28.248737+0000","last_clean":"2023-07-18T03:29:28.248737+0000","last_became_active":"2023-07-18T03:29:27.105187+0000","last_became_peered":"2023-07-18T03:29:27.105187+0000","last_unstale":"2023-07-18T03:29:28.248737+0000","last_undegraded":"2023-07-18T03:29:28.248737+0000","last_fullsized":"2023-07-18T03:29:28.248737+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:45:05.223020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.625681+0000","last_change":"2023-07-18T03:29:26.293284+0000","last_active":"2023-07-18T03:29:30.625681+0000","last_peered":"2023-07-18T03:29:30.625681+0000","last_clean":"2023-07-18T03:29:30.625681+0000","last_became_active":"2023-07-18T03:29:26.292955+0000","last_became_peered":"2023-07-18T03:29:26.292955+0000","last_unstale":"2023-07-18T03:29:30.625681+0000","last_undegraded":"2023-07-18T03:29:30.625681+0000","last_fullsized":"2023-07-18T03:29:30.625681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:15:36.890771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249337+0000","last_change":"2023-07-18T03:29:27.097322+0000","last_active":"2023-07-18T03:29:28.249337+0000","last_peered":"2023-07-18T03:29:28.249337+0000","last_clean":"2023-07-18T03:29:28.249337+0000","last_became_active":"2023-07-18T03:29:27.097051+0000","last_became_peered":"2023-07-18T03:29:27.097051+0000","last_unstale":"2023-07-18T03:29:28.249337+0000","last_undegraded":"2023-07-18T03:29:28.249337+0000","last_fullsized":"2023-07-18T03:29:28.249337+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:36:33.129283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085166+0000","last_change":"2023-07-18T03:29:26.290810+0000","last_active":"2023-07-18T03:29:28.085166+0000","last_peered":"2023-07-18T03:29:28.085166+0000","last_clean":"2023-07-18T03:29:28.085166+0000","last_became_active":"2023-07-18T03:29:26.290448+0000","last_became_peered":"2023-07-18T03:29:26.290448+0000","last_unstale":"2023-07-18T03:29:28.085166+0000","last_undegraded":"2023-07-18T03:29:28.085166+0000","last_fullsized":"2023-07-18T03:29:28.085166+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:21:19.581142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249116+0000","last_change":"2023-07-18T03:29:27.105801+0000","last_active":"2023-07-18T03:29:28.249116+0000","last_peered":"2023-07-18T03:29:28.249116+0000","last_clean":"2023-07-18T03:29:28.249116+0000","last_became_active":"2023-07-18T03:29:27.104715+0000","last_became_peered":"2023-07-18T03:29:27.104715+0000","last_unstale":"2023-07-18T03:29:28.249116+0000","last_undegraded":"2023-07-18T03:29:28.249116+0000","last_fullsized":"2023-07-18T03:29:28.249116+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:49:34.118157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'6","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085739+0000","last_change":"2023-07-18T03:29:26.293306+0000","last_active":"2023-07-18T03:29:28.085739+0000","last_peered":"2023-07-18T03:29:28.085739+0000","last_clean":"2023-07-18T03:29:28.085739+0000","last_became_active":"2023-07-18T03:29:26.292946+0000","last_became_peered":"2023-07-18T03:29:26.292946+0000","last_unstale":"2023-07-18T03:29:28.085739+0000","last_undegraded":"2023-07-18T03:29:28.085739+0000","last_fullsized":"2023-07-18T03:29:28.085739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:36:19.196296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249042+0000","last_change":"2023-07-18T03:29:27.103941+0000","last_active":"2023-07-18T03:29:28.249042+0000","last_peered":"2023-07-18T03:29:28.249042+0000","last_clean":"2023-07-18T03:29:28.249042+0000","last_became_active":"2023-07-18T03:29:27.103506+0000","last_became_peered":"2023-07-18T03:29:27.103506+0000","last_unstale":"2023-07-18T03:29:28.249042+0000","last_undegraded":"2023-07-18T03:29:28.249042+0000","last_fullsized":"2023-07-18T03:29:28.249042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:13:27.299459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.608558+0000","last_change":"2023-07-18T03:29:26.293473+0000","last_active":"2023-07-18T03:29:31.608558+0000","last_peered":"2023-07-18T03:29:31.608558+0000","last_clean":"2023-07-18T03:29:31.608558+0000","last_became_active":"2023-07-18T03:29:26.293132+0000","last_became_peered":"2023-07-18T03:29:26.293132+0000","last_unstale":"2023-07-18T03:29:31.608558+0000","last_undegraded":"2023-07-18T03:29:31.608558+0000","last_fullsized":"2023-07-18T03:29:31.608558+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:32:07.174316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249663+0000","last_change":"2023-07-18T03:29:27.104032+0000","last_active":"2023-07-18T03:29:28.249663+0000","last_peered":"2023-07-18T03:29:28.249663+0000","last_clean":"2023-07-18T03:29:28.249663+0000","last_became_active":"2023-07-18T03:29:27.103736+0000","last_became_peered":"2023-07-18T03:29:27.103736+0000","last_unstale":"2023-07-18T03:29:28.249663+0000","last_undegraded":"2023-07-18T03:29:28.249663+0000","last_fullsized":"2023-07-18T03:29:28.249663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:29:13.489385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.626335+0000","last_change":"2023-07-18T03:29:26.290487+0000","last_active":"2023-07-18T03:29:30.626335+0000","last_peered":"2023-07-18T03:29:30.626335+0000","last_clean":"2023-07-18T03:29:30.626335+0000","last_became_active":"2023-07-18T03:29:26.290115+0000","last_became_peered":"2023-07-18T03:29:26.290115+0000","last_unstale":"2023-07-18T03:29:30.626335+0000","last_undegraded":"2023-07-18T03:29:30.626335+0000","last_fullsized":"2023-07-18T03:29:30.626335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:35:42.866787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339569+0000","last_change":"2023-07-18T03:29:27.108550+0000","last_active":"2023-07-18T03:29:28.339569+0000","last_peered":"2023-07-18T03:29:28.339569+0000","last_clean":"2023-07-18T03:29:28.339569+0000","last_became_active":"2023-07-18T03:29:27.108069+0000","last_became_peered":"2023-07-18T03:29:27.108069+0000","last_unstale":"2023-07-18T03:29:28.339569+0000","last_undegraded":"2023-07-18T03:29:28.339569+0000","last_fullsized":"2023-07-18T03:29:28.339569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:55:48.510403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248480+0000","last_change":"2023-07-18T03:29:26.290896+0000","last_active":"2023-07-18T03:29:28.248480+0000","last_peered":"2023-07-18T03:29:28.248480+0000","last_clean":"2023-07-18T03:29:28.248480+0000","last_became_active":"2023-07-18T03:29:26.290450+0000","last_became_peered":"2023-07-18T03:29:26.290450+0000","last_unstale":"2023-07-18T03:29:28.248480+0000","last_undegraded":"2023-07-18T03:29:28.248480+0000","last_fullsized":"2023-07-18T03:29:28.248480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:23:16.823095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339200+0000","last_change":"2023-07-18T03:29:27.109184+0000","last_active":"2023-07-18T03:29:28.339200+0000","last_peered":"2023-07-18T03:29:28.339200+0000","last_clean":"2023-07-18T03:29:28.339200+0000","last_became_active":"2023-07-18T03:29:27.107993+0000","last_became_peered":"2023-07-18T03:29:27.107993+0000","last_unstale":"2023-07-18T03:29:28.339200+0000","last_undegraded":"2023-07-18T03:29:28.339200+0000","last_fullsized":"2023-07-18T03:29:28.339200+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:38:53.665857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.612612+0000","last_change":"2023-07-18T03:29:26.293671+0000","last_active":"2023-07-18T03:29:31.612612+0000","last_peered":"2023-07-18T03:29:31.612612+0000","last_clean":"2023-07-18T03:29:31.612612+0000","last_became_active":"2023-07-18T03:29:26.293205+0000","last_became_peered":"2023-07-18T03:29:26.293205+0000","last_unstale":"2023-07-18T03:29:31.612612+0000","last_undegraded":"2023-07-18T03:29:31.612612+0000","last_fullsized":"2023-07-18T03:29:31.612612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:52:09.605399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.084887+0000","last_change":"2023-07-18T03:29:27.096711+0000","last_active":"2023-07-18T03:29:28.084887+0000","last_peered":"2023-07-18T03:29:28.084887+0000","last_clean":"2023-07-18T03:29:28.084887+0000","last_became_active":"2023-07-18T03:29:27.096423+0000","last_became_peered":"2023-07-18T03:29:27.096423+0000","last_unstale":"2023-07-18T03:29:28.084887+0000","last_undegraded":"2023-07-18T03:29:28.084887+0000","last_fullsized":"2023-07-18T03:29:28.084887+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:35:54.060369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249365+0000","last_change":"2023-07-18T03:29:26.291546+0000","last_active":"2023-07-18T03:29:28.249365+0000","last_peered":"2023-07-18T03:29:28.249365+0000","last_clean":"2023-07-18T03:29:28.249365+0000","last_became_active":"2023-07-18T03:29:26.291319+0000","last_became_peered":"2023-07-18T03:29:26.291319+0000","last_unstale":"2023-07-18T03:29:28.249365+0000","last_undegraded":"2023-07-18T03:29:28.249365+0000","last_fullsized":"2023-07-18T03:29:28.249365+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:23:25.182800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085688+0000","last_change":"2023-07-18T03:29:27.098479+0000","last_active":"2023-07-18T03:29:28.085688+0000","last_peered":"2023-07-18T03:29:28.085688+0000","last_clean":"2023-07-18T03:29:28.085688+0000","last_became_active":"2023-07-18T03:29:27.098153+0000","last_became_peered":"2023-07-18T03:29:27.098153+0000","last_unstale":"2023-07-18T03:29:28.085688+0000","last_undegraded":"2023-07-18T03:29:28.085688+0000","last_fullsized":"2023-07-18T03:29:28.085688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:53:01.674305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.235436+0000","last_change":"2023-07-18T03:29:26.290953+0000","last_active":"2023-07-18T03:29:28.235436+0000","last_peered":"2023-07-18T03:29:28.235436+0000","last_clean":"2023-07-18T03:29:28.235436+0000","last_became_active":"2023-07-18T03:29:26.290633+0000","last_became_peered":"2023-07-18T03:29:26.290633+0000","last_unstale":"2023-07-18T03:29:28.235436+0000","last_undegraded":"2023-07-18T03:29:28.235436+0000","last_fullsized":"2023-07-18T03:29:28.235436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:40:49.185595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085754+0000","last_change":"2023-07-18T03:29:27.093651+0000","last_active":"2023-07-18T03:29:28.085754+0000","last_peered":"2023-07-18T03:29:28.085754+0000","last_clean":"2023-07-18T03:29:28.085754+0000","last_became_active":"2023-07-18T03:29:27.093307+0000","last_became_peered":"2023-07-18T03:29:27.093307+0000","last_unstale":"2023-07-18T03:29:28.085754+0000","last_undegraded":"2023-07-18T03:29:28.085754+0000","last_fullsized":"2023-07-18T03:29:28.085754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:31:10.315148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339733+0000","last_change":"2023-07-18T03:29:26.287405+0000","last_active":"2023-07-18T03:29:28.339733+0000","last_peered":"2023-07-18T03:29:28.339733+0000","last_clean":"2023-07-18T03:29:28.339733+0000","last_became_active":"2023-07-18T03:29:26.287048+0000","last_became_peered":"2023-07-18T03:29:26.287048+0000","last_unstale":"2023-07-18T03:29:28.339733+0000","last_undegraded":"2023-07-18T03:29:28.339733+0000","last_fullsized":"2023-07-18T03:29:28.339733+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:50:53.303258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085803+0000","last_change":"2023-07-18T03:29:27.103218+0000","last_active":"2023-07-18T03:29:28.085803+0000","last_peered":"2023-07-18T03:29:28.085803+0000","last_clean":"2023-07-18T03:29:28.085803+0000","last_became_active":"2023-07-18T03:29:27.102917+0000","last_became_peered":"2023-07-18T03:29:27.102917+0000","last_unstale":"2023-07-18T03:29:28.085803+0000","last_undegraded":"2023-07-18T03:29:28.085803+0000","last_fullsized":"2023-07-18T03:29:28.085803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:23:49.743367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085272+0000","last_change":"2023-07-18T03:29:27.098696+0000","last_active":"2023-07-18T03:29:28.085272+0000","last_peered":"2023-07-18T03:29:28.085272+0000","last_clean":"2023-07-18T03:29:28.085272+0000","last_became_active":"2023-07-18T03:29:27.098253+0000","last_became_peered":"2023-07-18T03:29:27.098253+0000","last_unstale":"2023-07-18T03:29:28.085272+0000","last_undegraded":"2023-07-18T03:29:28.085272+0000","last_fullsized":"2023-07-18T03:29:28.085272+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:40:58.942045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.609724+0000","last_change":"2023-07-18T03:29:26.290308+0000","last_active":"2023-07-18T03:29:31.609724+0000","last_peered":"2023-07-18T03:29:31.609724+0000","last_clean":"2023-07-18T03:29:31.609724+0000","last_became_active":"2023-07-18T03:29:26.289937+0000","last_became_peered":"2023-07-18T03:29:26.289937+0000","last_unstale":"2023-07-18T03:29:31.609724+0000","last_undegraded":"2023-07-18T03:29:31.609724+0000","last_fullsized":"2023-07-18T03:29:31.609724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:21:51.452197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249450+0000","last_change":"2023-07-18T03:29:27.096469+0000","last_active":"2023-07-18T03:29:28.249450+0000","last_peered":"2023-07-18T03:29:28.249450+0000","last_clean":"2023-07-18T03:29:28.249450+0000","last_became_active":"2023-07-18T03:29:27.096171+0000","last_became_peered":"2023-07-18T03:29:27.096171+0000","last_unstale":"2023-07-18T03:29:28.249450+0000","last_undegraded":"2023-07-18T03:29:28.249450+0000","last_fullsized":"2023-07-18T03:29:28.249450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:14:47.599303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.626213+0000","last_change":"2023-07-18T03:29:26.290935+0000","last_active":"2023-07-18T03:29:30.626213+0000","last_peered":"2023-07-18T03:29:30.626213+0000","last_clean":"2023-07-18T03:29:30.626213+0000","last_became_active":"2023-07-18T03:29:26.290595+0000","last_became_peered":"2023-07-18T03:29:26.290595+0000","last_unstale":"2023-07-18T03:29:30.626213+0000","last_undegraded":"2023-07-18T03:29:30.626213+0000","last_fullsized":"2023-07-18T03:29:30.626213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:34:23.417106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":331,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339337+0000","last_change":"2023-07-18T03:28:28.122385+0000","last_active":"2023-07-18T03:29:28.339337+0000","last_peered":"2023-07-18T03:29:28.339337+0000","last_clean":"2023-07-18T03:29:28.339337+0000","last_became_active":"2023-07-18T03:28:28.121932+0000","last_became_peered":"2023-07-18T03:28:28.121932+0000","last_unstale":"2023-07-18T03:29:28.339337+0000","last_undegraded":"2023-07-18T03:29:28.339337+0000","last_fullsized":"2023-07-18T03:29:28.339337+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:25:38.267627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:25:38.267627+0000","last_clean_scrub_stamp":"2023-07-18T03:25:38.267627+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:46:37.243383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248934+0000","last_change":"2023-07-18T03:29:27.101582+0000","last_active":"2023-07-18T03:29:28.248934+0000","last_peered":"2023-07-18T03:29:28.248934+0000","last_clean":"2023-07-18T03:29:28.248934+0000","last_became_active":"2023-07-18T03:29:27.101226+0000","last_became_peered":"2023-07-18T03:29:27.101226+0000","last_unstale":"2023-07-18T03:29:28.248934+0000","last_undegraded":"2023-07-18T03:29:28.248934+0000","last_fullsized":"2023-07-18T03:29:28.248934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:42:14.571198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.607147+0000","last_change":"2023-07-18T03:29:26.291787+0000","last_active":"2023-07-18T03:29:31.607147+0000","last_peered":"2023-07-18T03:29:31.607147+0000","last_clean":"2023-07-18T03:29:31.607147+0000","last_became_active":"2023-07-18T03:29:26.291471+0000","last_became_peered":"2023-07-18T03:29:26.291471+0000","last_unstale":"2023-07-18T03:29:31.607147+0000","last_undegraded":"2023-07-18T03:29:31.607147+0000","last_fullsized":"2023-07-18T03:29:31.607147+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:49:20.527148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.626010+0000","last_change":"2023-07-18T03:29:26.296936+0000","last_active":"2023-07-18T03:29:30.626010+0000","last_peered":"2023-07-18T03:29:30.626010+0000","last_clean":"2023-07-18T03:29:30.626010+0000","last_became_active":"2023-07-18T03:29:26.296241+0000","last_became_peered":"2023-07-18T03:29:26.296241+0000","last_unstale":"2023-07-18T03:29:30.626010+0000","last_undegraded":"2023-07-18T03:29:30.626010+0000","last_fullsized":"2023-07-18T03:29:30.626010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:23:24.313635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249296+0000","last_change":"2023-07-18T03:29:27.102715+0000","last_active":"2023-07-18T03:29:28.249296+0000","last_peered":"2023-07-18T03:29:28.249296+0000","last_clean":"2023-07-18T03:29:28.249296+0000","last_became_active":"2023-07-18T03:29:27.100914+0000","last_became_peered":"2023-07-18T03:29:27.100914+0000","last_unstale":"2023-07-18T03:29:28.249296+0000","last_undegraded":"2023-07-18T03:29:28.249296+0000","last_fullsized":"2023-07-18T03:29:28.249296+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:43:38.965763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.235404+0000","last_change":"2023-07-18T03:29:26.290231+0000","last_active":"2023-07-18T03:29:28.235404+0000","last_peered":"2023-07-18T03:29:28.235404+0000","last_clean":"2023-07-18T03:29:28.235404+0000","last_became_active":"2023-07-18T03:29:26.289753+0000","last_became_peered":"2023-07-18T03:29:26.289753+0000","last_unstale":"2023-07-18T03:29:28.235404+0000","last_undegraded":"2023-07-18T03:29:28.235404+0000","last_fullsized":"2023-07-18T03:29:28.235404+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:33:36.792644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249097+0000","last_change":"2023-07-18T03:29:27.103964+0000","last_active":"2023-07-18T03:29:28.249097+0000","last_peered":"2023-07-18T03:29:28.249097+0000","last_clean":"2023-07-18T03:29:28.249097+0000","last_became_active":"2023-07-18T03:29:27.103660+0000","last_became_peered":"2023-07-18T03:29:27.103660+0000","last_unstale":"2023-07-18T03:29:28.249097+0000","last_undegraded":"2023-07-18T03:29:28.249097+0000","last_fullsized":"2023-07-18T03:29:28.249097+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:08:09.167239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.625316+0000","last_change":"2023-07-18T03:29:26.286436+0000","last_active":"2023-07-18T03:29:30.625316+0000","last_peered":"2023-07-18T03:29:30.625316+0000","last_clean":"2023-07-18T03:29:30.625316+0000","last_became_active":"2023-07-18T03:29:26.286070+0000","last_became_peered":"2023-07-18T03:29:26.286070+0000","last_unstale":"2023-07-18T03:29:30.625316+0000","last_undegraded":"2023-07-18T03:29:30.625316+0000","last_fullsized":"2023-07-18T03:29:30.625316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:14:53.466702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249329+0000","last_change":"2023-07-18T03:29:27.102303+0000","last_active":"2023-07-18T03:29:28.249329+0000","last_peered":"2023-07-18T03:29:28.249329+0000","last_clean":"2023-07-18T03:29:28.249329+0000","last_became_active":"2023-07-18T03:29:27.101980+0000","last_became_peered":"2023-07-18T03:29:27.101980+0000","last_unstale":"2023-07-18T03:29:28.249329+0000","last_undegraded":"2023-07-18T03:29:28.249329+0000","last_fullsized":"2023-07-18T03:29:28.249329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:46:48.299459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.624991+0000","last_change":"2023-07-18T03:29:26.289628+0000","last_active":"2023-07-18T03:29:30.624991+0000","last_peered":"2023-07-18T03:29:30.624991+0000","last_clean":"2023-07-18T03:29:30.624991+0000","last_became_active":"2023-07-18T03:29:26.289013+0000","last_became_peered":"2023-07-18T03:29:26.289013+0000","last_unstale":"2023-07-18T03:29:30.624991+0000","last_undegraded":"2023-07-18T03:29:30.624991+0000","last_fullsized":"2023-07-18T03:29:30.624991+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:58:20.665607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248709+0000","last_change":"2023-07-18T03:29:27.090943+0000","last_active":"2023-07-18T03:29:28.248709+0000","last_peered":"2023-07-18T03:29:28.248709+0000","last_clean":"2023-07-18T03:29:28.248709+0000","last_became_active":"2023-07-18T03:29:27.090465+0000","last_became_peered":"2023-07-18T03:29:27.090465+0000","last_unstale":"2023-07-18T03:29:28.248709+0000","last_undegraded":"2023-07-18T03:29:28.248709+0000","last_fullsized":"2023-07-18T03:29:28.248709+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:49:17.548432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249047+0000","last_change":"2023-07-18T03:29:26.288576+0000","last_active":"2023-07-18T03:29:28.249047+0000","last_peered":"2023-07-18T03:29:28.249047+0000","last_clean":"2023-07-18T03:29:28.249047+0000","last_became_active":"2023-07-18T03:29:26.287514+0000","last_became_peered":"2023-07-18T03:29:26.287514+0000","last_unstale":"2023-07-18T03:29:28.249047+0000","last_undegraded":"2023-07-18T03:29:28.249047+0000","last_fullsized":"2023-07-18T03:29:28.249047+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:37:47.805842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085745+0000","last_change":"2023-07-18T03:29:27.101111+0000","last_active":"2023-07-18T03:29:28.085745+0000","last_peered":"2023-07-18T03:29:28.085745+0000","last_clean":"2023-07-18T03:29:28.085745+0000","last_became_active":"2023-07-18T03:29:27.100737+0000","last_became_peered":"2023-07-18T03:29:27.100737+0000","last_unstale":"2023-07-18T03:29:28.085745+0000","last_undegraded":"2023-07-18T03:29:28.085745+0000","last_fullsized":"2023-07-18T03:29:28.085745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:00:58.906730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129626+0000","last_change":"2023-07-18T03:29:26.296974+0000","last_active":"2023-07-18T03:29:28.129626+0000","last_peered":"2023-07-18T03:29:28.129626+0000","last_clean":"2023-07-18T03:29:28.129626+0000","last_became_active":"2023-07-18T03:29:26.296451+0000","last_became_peered":"2023-07-18T03:29:26.296451+0000","last_unstale":"2023-07-18T03:29:28.129626+0000","last_undegraded":"2023-07-18T03:29:28.129626+0000","last_fullsized":"2023-07-18T03:29:28.129626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:35:42.821600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249813+0000","last_change":"2023-07-18T03:29:27.095857+0000","last_active":"2023-07-18T03:29:28.249813+0000","last_peered":"2023-07-18T03:29:28.249813+0000","last_clean":"2023-07-18T03:29:28.249813+0000","last_became_active":"2023-07-18T03:29:27.095553+0000","last_became_peered":"2023-07-18T03:29:27.095553+0000","last_unstale":"2023-07-18T03:29:28.249813+0000","last_undegraded":"2023-07-18T03:29:28.249813+0000","last_fullsized":"2023-07-18T03:29:28.249813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:30:11.142073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.627680+0000","last_change":"2023-07-18T03:29:26.295936+0000","last_active":"2023-07-18T03:29:30.627680+0000","last_peered":"2023-07-18T03:29:30.627680+0000","last_clean":"2023-07-18T03:29:30.627680+0000","last_became_active":"2023-07-18T03:29:26.295467+0000","last_became_peered":"2023-07-18T03:29:26.295467+0000","last_unstale":"2023-07-18T03:29:30.627680+0000","last_undegraded":"2023-07-18T03:29:30.627680+0000","last_fullsized":"2023-07-18T03:29:30.627680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:45:43.716125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339616+0000","last_change":"2023-07-18T03:29:27.108528+0000","last_active":"2023-07-18T03:29:28.339616+0000","last_peered":"2023-07-18T03:29:28.339616+0000","last_clean":"2023-07-18T03:29:28.339616+0000","last_became_active":"2023-07-18T03:29:27.108039+0000","last_became_peered":"2023-07-18T03:29:27.108039+0000","last_unstale":"2023-07-18T03:29:28.339616+0000","last_undegraded":"2023-07-18T03:29:28.339616+0000","last_fullsized":"2023-07-18T03:29:28.339616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:57:20.088291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:32.634752+0000","last_change":"2023-07-18T03:29:26.294006+0000","last_active":"2023-07-18T03:29:32.634752+0000","last_peered":"2023-07-18T03:29:32.634752+0000","last_clean":"2023-07-18T03:29:32.634752+0000","last_became_active":"2023-07-18T03:29:26.293700+0000","last_became_peered":"2023-07-18T03:29:26.293700+0000","last_unstale":"2023-07-18T03:29:32.634752+0000","last_undegraded":"2023-07-18T03:29:32.634752+0000","last_fullsized":"2023-07-18T03:29:32.634752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:44:37.451806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339715+0000","last_change":"2023-07-18T03:29:27.102764+0000","last_active":"2023-07-18T03:29:28.339715+0000","last_peered":"2023-07-18T03:29:28.339715+0000","last_clean":"2023-07-18T03:29:28.339715+0000","last_became_active":"2023-07-18T03:29:27.102200+0000","last_became_peered":"2023-07-18T03:29:27.102200+0000","last_unstale":"2023-07-18T03:29:28.339715+0000","last_undegraded":"2023-07-18T03:29:28.339715+0000","last_fullsized":"2023-07-18T03:29:28.339715+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:54:13.608760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.084954+0000","last_change":"2023-07-18T03:29:26.294271+0000","last_active":"2023-07-18T03:29:28.084954+0000","last_peered":"2023-07-18T03:29:28.084954+0000","last_clean":"2023-07-18T03:29:28.084954+0000","last_became_active":"2023-07-18T03:29:26.293938+0000","last_became_peered":"2023-07-18T03:29:26.293938+0000","last_unstale":"2023-07-18T03:29:28.084954+0000","last_undegraded":"2023-07-18T03:29:28.084954+0000","last_fullsized":"2023-07-18T03:29:28.084954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:10:38.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339388+0000","last_change":"2023-07-18T03:29:27.106510+0000","last_active":"2023-07-18T03:29:28.339388+0000","last_peered":"2023-07-18T03:29:28.339388+0000","last_clean":"2023-07-18T03:29:28.339388+0000","last_became_active":"2023-07-18T03:29:27.106222+0000","last_became_peered":"2023-07-18T03:29:27.106222+0000","last_unstale":"2023-07-18T03:29:28.339388+0000","last_undegraded":"2023-07-18T03:29:28.339388+0000","last_fullsized":"2023-07-18T03:29:28.339388+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:27:44.790790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248915+0000","last_change":"2023-07-18T03:29:26.293583+0000","last_active":"2023-07-18T03:29:28.248915+0000","last_peered":"2023-07-18T03:29:28.248915+0000","last_clean":"2023-07-18T03:29:28.248915+0000","last_became_active":"2023-07-18T03:29:26.293076+0000","last_became_peered":"2023-07-18T03:29:26.293076+0000","last_unstale":"2023-07-18T03:29:28.248915+0000","last_undegraded":"2023-07-18T03:29:28.248915+0000","last_fullsized":"2023-07-18T03:29:28.248915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:37:21.591425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339559+0000","last_change":"2023-07-18T03:29:27.109175+0000","last_active":"2023-07-18T03:29:28.339559+0000","last_peered":"2023-07-18T03:29:28.339559+0000","last_clean":"2023-07-18T03:29:28.339559+0000","last_became_active":"2023-07-18T03:29:27.108669+0000","last_became_peered":"2023-07-18T03:29:27.108669+0000","last_unstale":"2023-07-18T03:29:28.339559+0000","last_undegraded":"2023-07-18T03:29:28.339559+0000","last_fullsized":"2023-07-18T03:29:28.339559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:00:09.903589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085918+0000","last_change":"2023-07-18T03:29:27.099770+0000","last_active":"2023-07-18T03:29:28.085918+0000","last_peered":"2023-07-18T03:29:28.085918+0000","last_clean":"2023-07-18T03:29:28.085918+0000","last_became_active":"2023-07-18T03:29:27.099487+0000","last_became_peered":"2023-07-18T03:29:27.099487+0000","last_unstale":"2023-07-18T03:29:28.085918+0000","last_undegraded":"2023-07-18T03:29:28.085918+0000","last_fullsized":"2023-07-18T03:29:28.085918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:22:59.316119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129653+0000","last_change":"2023-07-18T03:29:27.100189+0000","last_active":"2023-07-18T03:29:28.129653+0000","last_peered":"2023-07-18T03:29:28.129653+0000","last_clean":"2023-07-18T03:29:28.129653+0000","last_became_active":"2023-07-18T03:29:27.099884+0000","last_became_peered":"2023-07-18T03:29:27.099884+0000","last_unstale":"2023-07-18T03:29:28.129653+0000","last_undegraded":"2023-07-18T03:29:28.129653+0000","last_fullsized":"2023-07-18T03:29:28.129653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:04:36.761236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129593+0000","last_change":"2023-07-18T03:29:27.099196+0000","last_active":"2023-07-18T03:29:28.129593+0000","last_peered":"2023-07-18T03:29:28.129593+0000","last_clean":"2023-07-18T03:29:28.129593+0000","last_became_active":"2023-07-18T03:29:27.098903+0000","last_became_peered":"2023-07-18T03:29:27.098903+0000","last_unstale":"2023-07-18T03:29:28.129593+0000","last_undegraded":"2023-07-18T03:29:28.129593+0000","last_fullsized":"2023-07-18T03:29:28.129593+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:46:05.998503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248469+0000","last_change":"2023-07-18T03:29:27.098138+0000","last_active":"2023-07-18T03:29:28.248469+0000","last_peered":"2023-07-18T03:29:28.248469+0000","last_clean":"2023-07-18T03:29:28.248469+0000","last_became_active":"2023-07-18T03:29:27.097793+0000","last_became_peered":"2023-07-18T03:29:27.097793+0000","last_unstale":"2023-07-18T03:29:28.248469+0000","last_undegraded":"2023-07-18T03:29:28.248469+0000","last_fullsized":"2023-07-18T03:29:28.248469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:07:31.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249184+0000","last_change":"2023-07-18T03:29:27.101610+0000","last_active":"2023-07-18T03:29:28.249184+0000","last_peered":"2023-07-18T03:29:28.249184+0000","last_clean":"2023-07-18T03:29:28.249184+0000","last_became_active":"2023-07-18T03:29:27.101332+0000","last_became_peered":"2023-07-18T03:29:27.101332+0000","last_unstale":"2023-07-18T03:29:28.249184+0000","last_undegraded":"2023-07-18T03:29:28.249184+0000","last_fullsized":"2023-07-18T03:29:28.249184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:43:01.583313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085879+0000","last_change":"2023-07-18T03:29:27.093345+0000","last_active":"2023-07-18T03:29:28.085879+0000","last_peered":"2023-07-18T03:29:28.085879+0000","last_clean":"2023-07-18T03:29:28.085879+0000","last_became_active":"2023-07-18T03:29:27.093048+0000","last_became_peered":"2023-07-18T03:29:27.093048+0000","last_unstale":"2023-07-18T03:29:28.085879+0000","last_undegraded":"2023-07-18T03:29:28.085879+0000","last_fullsized":"2023-07-18T03:29:28.085879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:00:19.659948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129830+0000","last_change":"2023-07-18T03:29:27.098311+0000","last_active":"2023-07-18T03:29:28.129830+0000","last_peered":"2023-07-18T03:29:28.129830+0000","last_clean":"2023-07-18T03:29:28.129830+0000","last_became_active":"2023-07-18T03:29:27.097980+0000","last_became_peered":"2023-07-18T03:29:27.097980+0000","last_unstale":"2023-07-18T03:29:28.129830+0000","last_undegraded":"2023-07-18T03:29:28.129830+0000","last_fullsized":"2023-07-18T03:29:28.129830+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:34:48.849264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248965+0000","last_change":"2023-07-18T03:29:27.093796+0000","last_active":"2023-07-18T03:29:28.248965+0000","last_peered":"2023-07-18T03:29:28.248965+0000","last_clean":"2023-07-18T03:29:28.248965+0000","last_became_active":"2023-07-18T03:29:27.093474+0000","last_became_peered":"2023-07-18T03:29:27.093474+0000","last_unstale":"2023-07-18T03:29:28.248965+0000","last_undegraded":"2023-07-18T03:29:28.248965+0000","last_fullsized":"2023-07-18T03:29:28.248965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:00:09.316029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085099+0000","last_change":"2023-07-18T03:29:27.098541+0000","last_active":"2023-07-18T03:29:28.085099+0000","last_peered":"2023-07-18T03:29:28.085099+0000","last_clean":"2023-07-18T03:29:28.085099+0000","last_became_active":"2023-07-18T03:29:27.097824+0000","last_became_peered":"2023-07-18T03:29:27.097824+0000","last_unstale":"2023-07-18T03:29:28.085099+0000","last_undegraded":"2023-07-18T03:29:28.085099+0000","last_fullsized":"2023-07-18T03:29:28.085099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:28:38.837863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248860+0000","last_change":"2023-07-18T03:29:27.096983+0000","last_active":"2023-07-18T03:29:28.248860+0000","last_peered":"2023-07-18T03:29:28.248860+0000","last_clean":"2023-07-18T03:29:28.248860+0000","last_became_active":"2023-07-18T03:29:27.096632+0000","last_became_peered":"2023-07-18T03:29:27.096632+0000","last_unstale":"2023-07-18T03:29:28.248860+0000","last_undegraded":"2023-07-18T03:29:28.248860+0000","last_fullsized":"2023-07-18T03:29:28.248860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:45:45.965235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249073+0000","last_change":"2023-07-18T03:29:27.105236+0000","last_active":"2023-07-18T03:29:28.249073+0000","last_peered":"2023-07-18T03:29:28.249073+0000","last_clean":"2023-07-18T03:29:28.249073+0000","last_became_active":"2023-07-18T03:29:27.104906+0000","last_became_peered":"2023-07-18T03:29:27.104906+0000","last_unstale":"2023-07-18T03:29:28.249073+0000","last_undegraded":"2023-07-18T03:29:28.249073+0000","last_fullsized":"2023-07-18T03:29:28.249073+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:53:05.499609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249970+0000","last_change":"2023-07-18T03:29:27.103461+0000","last_active":"2023-07-18T03:29:28.249970+0000","last_peered":"2023-07-18T03:29:28.249970+0000","last_clean":"2023-07-18T03:29:28.249970+0000","last_became_active":"2023-07-18T03:29:27.103162+0000","last_became_peered":"2023-07-18T03:29:27.103162+0000","last_unstale":"2023-07-18T03:29:28.249970+0000","last_undegraded":"2023-07-18T03:29:28.249970+0000","last_fullsized":"2023-07-18T03:29:28.249970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:26:14.600362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085635+0000","last_change":"2023-07-18T03:29:27.095300+0000","last_active":"2023-07-18T03:29:28.085635+0000","last_peered":"2023-07-18T03:29:28.085635+0000","last_clean":"2023-07-18T03:29:28.085635+0000","last_became_active":"2023-07-18T03:29:27.094985+0000","last_became_peered":"2023-07-18T03:29:27.094985+0000","last_unstale":"2023-07-18T03:29:28.085635+0000","last_undegraded":"2023-07-18T03:29:28.085635+0000","last_fullsized":"2023-07-18T03:29:28.085635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:09:05.373008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4898,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":98,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":12094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514508,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27800,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717352,"statfs":{"total":95995035648,"available":95966568448,"internally_reserved":0,"allocated":868352,"data_stored":417216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]}]},{"osd":10,"up_from":69,"seq":296352743440,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28404,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716748,"statfs":{"total":95995035648,"available":95965949952,"internally_reserved":0,"allocated":1486848,"data_stored":1009342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.42699999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1,"5min":1,"15min":1},"last":0.62}]},{"osd":3,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.59699999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.497},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.54600000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.86099999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":8,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.93100000000000005}]},{"osd":9,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.83699999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]}]},{"osd":9,"up_from":64,"seq":274877906964,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":872448,"data_stored":417798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.312},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.33100000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.54100000000000004}]},{"osd":2,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.40899999999999997}]},{"osd":4,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.69599999999999995}]},{"osd":5,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.50600000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.39000000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.629}]},{"osd":8,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.61399999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.81699999999999995}]}]},{"osd":8,"up_from":56,"seq":240518168600,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":417623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.44400000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.80100000000000005}]},{"osd":2,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.68600000000000005}]},{"osd":4,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.876}]},{"osd":5,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.65500000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.75600000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.81000000000000005}]},{"osd":9,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.82199999999999995}]},{"osd":10,"last update":"Tue Jul 18 03:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.022}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":7,"up_from":49,"seq":210453397532,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28392,"kb_used_data":1440,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716760,"statfs":{"total":95995035648,"available":95965962240,"internally_reserved":0,"allocated":1474560,"data_stored":1008740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46200000000000002,"15min":0.45900000000000002},"min":{"1min":0.30599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.753,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.48399999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46600000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.379,"15min":0.379},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.628},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.621,"15min":0.60899999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.68700000000000006}]},{"osd":3,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.64900000000000002,"15min":0.63900000000000001},"min":{"1min":0.40600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93400000000000005,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.745},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66400000000000003,"15min":0.65800000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.65200000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56000000000000005}]},{"osd":5,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.48299999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.70899999999999996,"15min":0.69599999999999995},"min":{"1min":0.46600000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.83299999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.74399999999999999,"15min":0.73399999999999999},"min":{"1min":0.499,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.04,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.52900000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97599999999999998,"5min":1.103,"15min":1.103},"last":0.89300000000000002}]},{"osd":8,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.81200000000000006}]},{"osd":9,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.79600000000000004}]},{"osd":10,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.91900000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]}]},{"osd":6,"up_from":44,"seq":188978561057,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":872448,"data_stored":417306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40200000000000002},"min":{"1min":0.309,"5min":0.254,"15min":0.254},"max":{"1min":0.60299999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.32100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.58099999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.59999999999999998}]},{"osd":1,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.45600000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.42099999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78800000000000003,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.47499999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83599999999999997,"5min":1.008,"15min":1.008},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.63500000000000001,"15min":0.61899999999999999},"min":{"1min":0.48499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.55600000000000005}]},{"osd":3,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.498,"15min":0.48899999999999999},"min":{"1min":0.36399999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.36699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71099999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.56999999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.623,"15min":0.61399999999999999},"min":{"1min":0.52000000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85299999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.68500000000000005,"15min":0.67100000000000004},"min":{"1min":0.55100000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.43099999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.68100000000000005,"15min":0.67100000000000004},"min":{"1min":0.54400000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.628},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.503,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.96399999999999997,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.79300000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73199999999999998,"15min":0.72799999999999998},"min":{"1min":0.47199999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56899999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0580000000000001,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.46500000000000002}]},{"osd":8,"last update":"Tue Jul 18 03:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.67100000000000004}]},{"osd":9,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.878},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.51200000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.64700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":418780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51500000000000001,"15min":0.47899999999999998},"min":{"1min":0.34999999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81100000000000005,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.52600000000000002,"15min":0.503},"min":{"1min":0.38300000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.89100000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.80300000000000005}]},{"osd":2,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.55400000000000005,"15min":0.53700000000000003},"min":{"1min":0.47199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.497,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78600000000000003,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.77000000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.59099999999999997,"15min":0.55500000000000005},"min":{"1min":0.441,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.60799999999999998,"15min":0.57799999999999996},"min":{"1min":0.54700000000000004,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.85299999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.44700000000000001,"15min":0.40699999999999997},"min":{"1min":0.33200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.45000000000000001,"15min":0.42199999999999999},"min":{"1min":0.30499999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.752,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59199999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.45700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.628},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.627,"15min":0.60999999999999999},"min":{"1min":0.60799999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.83499999999999996,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.82499999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.63200000000000001,"15min":0.61699999999999999},"min":{"1min":0.498,"5min":0.443,"15min":0.443},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.68500000000000005,"15min":0.66900000000000004},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.91500000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.874},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.69599999999999995,"15min":0.67700000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.84099999999999997}]},{"osd":8,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.94399999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":1.0209999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":872448,"data_stored":417306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53600000000000003,"15min":0.51400000000000001},"min":{"1min":0.35999999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.629},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.374,"5min":0.318,"15min":0.318},"max":{"1min":0.82299999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.61399999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.55600000000000005,"15min":0.52300000000000002},"min":{"1min":0.41799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.55300000000000005,"15min":0.52400000000000002},"min":{"1min":0.45600000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.45200000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.42299999999999999,"15min":0.41499999999999998},"min":{"1min":0.315,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.628,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.376}]},{"osd":4,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56899999999999995,"15min":0.55100000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.92800000000000005,"5min":1.018,"15min":1.018},"last":0.57999999999999996}]},{"osd":5,"last update":"Tue Jul 18 03:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63500000000000001,"15min":0.624},"min":{"1min":0.52900000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.503}]},{"osd":6,"last update":"Tue Jul 18 03:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56499999999999995,"15min":0.55200000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.47999999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.42999999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56200000000000006}]},{"osd":9,"last update":"Tue Jul 18 03:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.65800000000000003}]},{"osd":10,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.72099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":420082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51500000000000001,"15min":0.51200000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75800000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54100000000000004,"15min":0.55400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.69899999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55000000000000004,"15min":0.55500000000000005},"min":{"1min":0.41499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78200000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.53800000000000003,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.85799999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59099999999999997,"15min":0.57699999999999996},"min":{"1min":0.45300000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.41199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.58199999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55200000000000005,"15min":0.55500000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71999999999999997,"5min":0.872,"15min":0.872},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76500000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.749}]},{"osd":5,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.29699999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.503},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.42699999999999999,"15min":0.41999999999999998},"min":{"1min":0.314,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.47899999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55900000000000005,"15min":0.55200000000000005},"min":{"1min":0.40100000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.622},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58799999999999997,"15min":0.57499999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.79900000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.505,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.61699999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.68200000000000005}]},{"osd":9,"last update":"Tue Jul 18 03:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.81599999999999995}]},{"osd":10,"last update":"Tue Jul 18 03:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.83599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":876544,"data_stored":417320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.36399999999999999,"15min":0.34200000000000003},"min":{"1min":0.308,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.38600000000000001,"15min":0.36799999999999999},"min":{"1min":0.317,"5min":0.215,"15min":0.215},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.29699999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.56599999999999995,"15min":0.52700000000000002},"min":{"1min":0.42399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.53600000000000003,"15min":0.50600000000000001},"min":{"1min":0.42299999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78700000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.439}]},{"osd":2,"last update":"Tue Jul 18 03:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84699999999999998,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59499999999999997,"15min":0.56200000000000006},"min":{"1min":0.47599999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94299999999999995,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.60399999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.61199999999999999,"15min":0.57899999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.64900000000000002,"15min":0.60499999999999998},"min":{"1min":0.51500000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.45300000000000001}]},{"osd":5,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.53200000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0449999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66200000000000003,"15min":0.65900000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91100000000000003,"5min":1.827,"15min":1.827},"last":0.70899999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.58899999999999997,"15min":0.56699999999999995},"min":{"1min":0.39200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.41199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85599999999999998,"5min":0.88,"15min":0.88},"last":0.47599999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.79900000000000004,"15min":0.78700000000000003},"min":{"1min":0.55900000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.66100000000000003}]},{"osd":8,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.49399999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.51600000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.496}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]}]},{"osd":4,"up_from":32,"seq":137438953512,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":872448,"data_stored":417238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53200000000000003,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.54700000000000004,"15min":0.50700000000000001},"min":{"1min":0.48099999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89500000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.502}]},{"osd":1,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.39900000000000002,"15min":0.38600000000000001},"min":{"1min":0.32800000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.57299999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.40600000000000003,"15min":0.38100000000000001},"min":{"1min":0.32500000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65900000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.434,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.503},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.47199999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85599999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.48399999999999999}]},{"osd":3,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.58499999999999996,"15min":0.54900000000000004},"min":{"1min":0.39800000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.57599999999999996,"15min":0.54800000000000004},"min":{"1min":0.41599999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64500000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61099999999999999,"15min":0.60799999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.45200000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84499999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.50700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.72099999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.434,"5min":0.38,"15min":0.38},"max":{"1min":0.89900000000000002,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.622},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.40699999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88600000000000001,"5min":1.121,"15min":1.121},"last":0.68300000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.82299999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.83899999999999997}]},{"osd":10,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":5,"up_from":37,"seq":158913789989,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1490944,"data_stored":1009012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.38200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.88900000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.52100000000000002,"15min":0.50800000000000001},"min":{"1min":0.44400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.71299999999999997,"5min":0.745,"15min":0.745},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.38700000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.38900000000000001,"15min":0.378},"min":{"1min":0.32200000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.41899999999999998,"15min":0.41099999999999998},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.45600000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56299999999999994,"15min":0.55500000000000005},"min":{"1min":0.46400000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.44600000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.70299999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.47799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.57199999999999995,"15min":0.55700000000000005},"min":{"1min":0.45700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.59199999999999997}]},{"osd":6,"last update":"Tue Jul 18 03:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.621,"15min":0.61199999999999999},"min":{"1min":0.51700000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63200000000000001,"15min":0.622},"min":{"1min":0.52400000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.81200000000000006}]},{"osd":7,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70999999999999996,"15min":0.70799999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.77500000000000002}]},{"osd":8,"last update":"Tue Jul 18 03:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.53300000000000003}]},{"osd":9,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.90400000000000003}]},{"osd":10,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.85099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:29:35.408 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:29:35.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.567+0000 7f14ae925700 1 -- 172.21.15.40:0/802999318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a80ffb10 msgr2=0x7f14a80fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:35.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.567+0000 7f14ae925700 1 --2- 172.21.15.40:0/802999318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a80ffb10 0x7f14a80fff20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1498009990 tx=0x7f1498009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.567+0000 7f14ae925700 1 -- 172.21.15.40:0/802999318 shutdown_connections 2023-07-18T03:29:35.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.567+0000 7f14ae925700 1 --2- 172.21.15.40:0/802999318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14a8101690 0x7f14a8107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.567+0000 7f14ae925700 1 --2- 172.21.15.40:0/802999318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a80ffb10 0x7f14a80fff20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.567+0000 7f14ae925700 1 --2- 172.21.15.40:0/802999318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14a8100d00 0x7f14a8101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.567+0000 7f14ae925700 1 -- 172.21.15.40:0/802999318 >> 172.21.15.40:0/802999318 conn(0x7f14a80fb130 msgr2=0x7f14a80fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.567+0000 7f14ae925700 1 -- 172.21.15.40:0/802999318 shutdown_connections 2023-07-18T03:29:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.567+0000 7f14ae925700 1 -- 172.21.15.40:0/802999318 wait complete. 2023-07-18T03:29:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14ae925700 1 Processor -- start 2023-07-18T03:29:35.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14ae925700 1 -- start start 2023-07-18T03:29:35.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14ae925700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a80ffb10 0x7f14a81956a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:35.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14ae925700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14a8100d00 0x7f14a8195be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:35.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14ae925700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14a8101690 0x7f14a819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:35.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14ae925700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a819d220 con 0x7f14a80ffb10 2023-07-18T03:29:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14ae925700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a819d360 con 0x7f14a8101690 2023-07-18T03:29:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14ae925700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a819d4f0 con 0x7f14a8100d00 2023-07-18T03:29:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a80ffb10 0x7f14a81956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a80ffb10 0x7f14a81956a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56320/0 (socket says 172.21.15.40:56320) 2023-07-18T03:29:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14a7fff700 1 -- 172.21.15.40:0/1763744298 learned_addr learned my addr 172.21.15.40:0/1763744298 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14a77fe700 1 --2- 172.21.15.40:0/1763744298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14a8100d00 0x7f14a8195be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:35.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14acec2700 1 --2- 172.21.15.40:0/1763744298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14a8101690 0x7f14a819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:35.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14a7fff700 1 -- 172.21.15.40:0/1763744298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14a8100d00 msgr2=0x7f14a8195be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:35.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14a7fff700 1 --2- 172.21.15.40:0/1763744298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14a8100d00 0x7f14a8195be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14a7fff700 1 -- 172.21.15.40:0/1763744298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14a8101690 msgr2=0x7f14a819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:35.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14a7fff700 1 --2- 172.21.15.40:0/1763744298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14a8101690 0x7f14a819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14a7fff700 1 -- 172.21.15.40:0/1763744298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14980095f0 con 0x7f14a80ffb10 2023-07-18T03:29:35.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.568+0000 7f14a77fe700 1 --2- 172.21.15.40:0/1763744298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14a8100d00 0x7f14a8195be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:29:35.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.569+0000 7f14a7fff700 1 --2- 172.21.15.40:0/1763744298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a80ffb10 0x7f14a81956a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f14980059d0 tx=0x7f1498005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:35.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.569+0000 7f14a57fa700 1 -- 172.21.15.40:0/1763744298 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149801d070 con 0x7f14a80ffb10 2023-07-18T03:29:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.569+0000 7f14a57fa700 1 -- 172.21.15.40:0/1763744298 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14980041a0 con 0x7f14a80ffb10 2023-07-18T03:29:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.569+0000 7f14ae925700 1 -- 172.21.15.40:0/1763744298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a819d770 con 0x7f14a80ffb10 2023-07-18T03:29:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.569+0000 7f14a57fa700 1 -- 172.21.15.40:0/1763744298 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149800f460 con 0x7f14a80ffb10 2023-07-18T03:29:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.569+0000 7f14ae925700 1 -- 172.21.15.40:0/1763744298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a819dbf0 con 0x7f14a80ffb10 2023-07-18T03:29:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.570+0000 7f14ae925700 1 -- 172.21.15.40:0/1763744298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a8106860 con 0x7f14a80ffb10 2023-07-18T03:29:35.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.571+0000 7f14a57fa700 1 -- 172.21.15.40:0/1763744298 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1498022600 con 0x7f14a80ffb10 2023-07-18T03:29:35.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.571+0000 7f14a57fa700 1 --2- 172.21.15.40:0/1763744298 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14900a12a0 0x7f14900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:35.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.572+0000 7f14a57fa700 1 -- 172.21.15.40:0/1763744298 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f14980ce6b0 con 0x7f14a80ffb10 2023-07-18T03:29:35.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.572+0000 7f14a77fe700 1 --2- 172.21.15.40:0/1763744298 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14900a12a0 0x7f14900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:35.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.572+0000 7f14a77fe700 1 --2- 172.21.15.40:0/1763744298 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14900a12a0 0x7f14900a3750 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f149c005fd0 tx=0x7f149c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:35.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.575+0000 7f14a57fa700 1 -- 172.21.15.40:0/1763744298 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1498097de0 con 0x7f14a80ffb10 2023-07-18T03:29:35.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.773+0000 7f14ae925700 1 -- 172.21.15.40:0/1763744298 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f14a8060950 con 0x7f14900a12a0 2023-07-18T03:29:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.785+0000 7f14a57fa700 1 -- 172.21.15.40:0/1763744298 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335252 (secure 0 0 0) 0x7f14a8060950 con 0x7f14900a12a0 2023-07-18T03:29:35.785 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:35.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.788+0000 7f14ae925700 1 -- 172.21.15.40:0/1763744298 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14900a12a0 msgr2=0x7f14900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:35.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.788+0000 7f14ae925700 1 --2- 172.21.15.40:0/1763744298 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14900a12a0 0x7f14900a3750 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f149c005fd0 tx=0x7f149c005e30 comp rx=0 tx=0).stop 2023-07-18T03:29:35.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.788+0000 7f14ae925700 1 -- 172.21.15.40:0/1763744298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a80ffb10 msgr2=0x7f14a81956a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.788+0000 7f14ae925700 1 --2- 172.21.15.40:0/1763744298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a80ffb10 0x7f14a81956a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f14980059d0 tx=0x7f1498005740 comp rx=0 tx=0).stop 2023-07-18T03:29:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.788+0000 7f14ae925700 1 -- 172.21.15.40:0/1763744298 shutdown_connections 2023-07-18T03:29:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.788+0000 7f14ae925700 1 --2- 172.21.15.40:0/1763744298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14a8101690 0x7f14a819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.788+0000 7f14ae925700 1 --2- 172.21.15.40:0/1763744298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14a80ffb10 0x7f14a81956a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.788+0000 7f14ae925700 1 --2- 172.21.15.40:0/1763744298 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14900a12a0 0x7f14900a3750 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.788+0000 7f14ae925700 1 --2- 172.21.15.40:0/1763744298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14a8100d00 0x7f14a8195be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.788+0000 7f14ae925700 1 -- 172.21.15.40:0/1763744298 >> 172.21.15.40:0/1763744298 conn(0x7f14a80fb130 msgr2=0x7f14a80fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:35.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.788+0000 7f14ae925700 1 -- 172.21.15.40:0/1763744298 shutdown_connections 2023-07-18T03:29:35.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:35.789+0000 7f14ae925700 1 -- 172.21.15.40:0/1763744298 wait complete. 2023-07-18T03:29:35.789 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-18T03:29:35.796 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:35 smithi040 ceph-mon[96439]: pgmap v252: 97 pgs: 97 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 678 B/s rd, 4.8 KiB/s wr, 13 op/s 2023-07-18T03:29:35.796 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:35 smithi040 ceph-mon[96439]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:29:35.796 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:35 smithi040 ceph-mon[96439]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:29:35.797 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:35 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] up:active 2023-07-18T03:29:35.797 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:35 smithi040 ceph-mon[96439]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-07-18T03:29:35.797 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:35 smithi040 ceph-mon[96439]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-18T03:29:35.797 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:35 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:29:35.797 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:35 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2063035372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:35.797 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:35 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1173512477' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:35.798 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2023-07-18T03:29:33.814993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595266,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":131,"num_read_kb":114,"num_write":279,"num_write_kb":2974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":283,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335492,"kb_used_data":12068,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606332,"statfs":{"total":1151940427776,"available":1151596883968,"internally_reserved":0,"allocated":12357632,"data_stored":6787763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4898,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":98,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.540621"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085961+0000","last_change":"2023-07-18T03:29:27.101623+0000","last_active":"2023-07-18T03:29:28.085961+0000","last_peered":"2023-07-18T03:29:28.085961+0000","last_clean":"2023-07-18T03:29:28.085961+0000","last_became_active":"2023-07-18T03:29:27.101297+0000","last_became_peered":"2023-07-18T03:29:27.101297+0000","last_unstale":"2023-07-18T03:29:28.085961+0000","last_undegraded":"2023-07-18T03:29:28.085961+0000","last_fullsized":"2023-07-18T03:29:28.085961+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:34:31.169755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248873+0000","last_change":"2023-07-18T03:29:27.104159+0000","last_active":"2023-07-18T03:29:28.248873+0000","last_peered":"2023-07-18T03:29:28.248873+0000","last_clean":"2023-07-18T03:29:28.248873+0000","last_became_active":"2023-07-18T03:29:27.103830+0000","last_became_peered":"2023-07-18T03:29:27.103830+0000","last_unstale":"2023-07-18T03:29:28.248873+0000","last_undegraded":"2023-07-18T03:29:28.248873+0000","last_fullsized":"2023-07-18T03:29:28.248873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:44:49.917899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249078+0000","last_change":"2023-07-18T03:29:27.098858+0000","last_active":"2023-07-18T03:29:28.249078+0000","last_peered":"2023-07-18T03:29:28.249078+0000","last_clean":"2023-07-18T03:29:28.249078+0000","last_became_active":"2023-07-18T03:29:27.098540+0000","last_became_peered":"2023-07-18T03:29:27.098540+0000","last_unstale":"2023-07-18T03:29:28.249078+0000","last_undegraded":"2023-07-18T03:29:28.249078+0000","last_fullsized":"2023-07-18T03:29:28.249078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:55:46.914785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248698+0000","last_change":"2023-07-18T03:29:27.103106+0000","last_active":"2023-07-18T03:29:28.248698+0000","last_peered":"2023-07-18T03:29:28.248698+0000","last_clean":"2023-07-18T03:29:28.248698+0000","last_became_active":"2023-07-18T03:29:27.102780+0000","last_became_peered":"2023-07-18T03:29:27.102780+0000","last_unstale":"2023-07-18T03:29:28.248698+0000","last_undegraded":"2023-07-18T03:29:28.248698+0000","last_fullsized":"2023-07-18T03:29:28.248698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:51:36.548110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248934+0000","last_change":"2023-07-18T03:29:27.095432+0000","last_active":"2023-07-18T03:29:28.248934+0000","last_peered":"2023-07-18T03:29:28.248934+0000","last_clean":"2023-07-18T03:29:28.248934+0000","last_became_active":"2023-07-18T03:29:27.095103+0000","last_became_peered":"2023-07-18T03:29:27.095103+0000","last_unstale":"2023-07-18T03:29:28.248934+0000","last_undegraded":"2023-07-18T03:29:28.248934+0000","last_fullsized":"2023-07-18T03:29:28.248934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:50:40.336206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085493+0000","last_change":"2023-07-18T03:29:27.091412+0000","last_active":"2023-07-18T03:29:28.085493+0000","last_peered":"2023-07-18T03:29:28.085493+0000","last_clean":"2023-07-18T03:29:28.085493+0000","last_became_active":"2023-07-18T03:29:27.091060+0000","last_became_peered":"2023-07-18T03:29:27.091060+0000","last_unstale":"2023-07-18T03:29:28.085493+0000","last_undegraded":"2023-07-18T03:29:28.085493+0000","last_fullsized":"2023-07-18T03:29:28.085493+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:31:43.393707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339267+0000","last_change":"2023-07-18T03:29:27.105223+0000","last_active":"2023-07-18T03:29:28.339267+0000","last_peered":"2023-07-18T03:29:28.339267+0000","last_clean":"2023-07-18T03:29:28.339267+0000","last_became_active":"2023-07-18T03:29:27.104926+0000","last_became_peered":"2023-07-18T03:29:27.104926+0000","last_unstale":"2023-07-18T03:29:28.339267+0000","last_undegraded":"2023-07-18T03:29:28.339267+0000","last_fullsized":"2023-07-18T03:29:28.339267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:43:23.908336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248819+0000","last_change":"2023-07-18T03:29:27.093722+0000","last_active":"2023-07-18T03:29:28.248819+0000","last_peered":"2023-07-18T03:29:28.248819+0000","last_clean":"2023-07-18T03:29:28.248819+0000","last_became_active":"2023-07-18T03:29:27.093103+0000","last_became_peered":"2023-07-18T03:29:27.093103+0000","last_unstale":"2023-07-18T03:29:28.248819+0000","last_undegraded":"2023-07-18T03:29:28.248819+0000","last_fullsized":"2023-07-18T03:29:28.248819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:03:07.122936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249009+0000","last_change":"2023-07-18T03:29:27.094681+0000","last_active":"2023-07-18T03:29:28.249009+0000","last_peered":"2023-07-18T03:29:28.249009+0000","last_clean":"2023-07-18T03:29:28.249009+0000","last_became_active":"2023-07-18T03:29:27.094284+0000","last_became_peered":"2023-07-18T03:29:27.094284+0000","last_unstale":"2023-07-18T03:29:28.249009+0000","last_undegraded":"2023-07-18T03:29:28.249009+0000","last_fullsized":"2023-07-18T03:29:28.249009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:25:08.820895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085737+0000","last_change":"2023-07-18T03:29:27.100759+0000","last_active":"2023-07-18T03:29:28.085737+0000","last_peered":"2023-07-18T03:29:28.085737+0000","last_clean":"2023-07-18T03:29:28.085737+0000","last_became_active":"2023-07-18T03:29:27.096660+0000","last_became_peered":"2023-07-18T03:29:27.096660+0000","last_unstale":"2023-07-18T03:29:28.085737+0000","last_undegraded":"2023-07-18T03:29:28.085737+0000","last_fullsized":"2023-07-18T03:29:28.085737+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:56:10.159967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339718+0000","last_change":"2023-07-18T03:29:27.107251+0000","last_active":"2023-07-18T03:29:28.339718+0000","last_peered":"2023-07-18T03:29:28.339718+0000","last_clean":"2023-07-18T03:29:28.339718+0000","last_became_active":"2023-07-18T03:29:27.106895+0000","last_became_peered":"2023-07-18T03:29:27.106895+0000","last_unstale":"2023-07-18T03:29:28.339718+0000","last_undegraded":"2023-07-18T03:29:28.339718+0000","last_fullsized":"2023-07-18T03:29:28.339718+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:12:15.805607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248978+0000","last_change":"2023-07-18T03:29:27.103905+0000","last_active":"2023-07-18T03:29:28.248978+0000","last_peered":"2023-07-18T03:29:28.248978+0000","last_clean":"2023-07-18T03:29:28.248978+0000","last_became_active":"2023-07-18T03:29:27.103021+0000","last_became_peered":"2023-07-18T03:29:27.103021+0000","last_unstale":"2023-07-18T03:29:28.248978+0000","last_undegraded":"2023-07-18T03:29:28.248978+0000","last_fullsized":"2023-07-18T03:29:28.248978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:48:20.771586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249855+0000","last_change":"2023-07-18T03:29:27.103529+0000","last_active":"2023-07-18T03:29:28.249855+0000","last_peered":"2023-07-18T03:29:28.249855+0000","last_clean":"2023-07-18T03:29:28.249855+0000","last_became_active":"2023-07-18T03:29:27.103217+0000","last_became_peered":"2023-07-18T03:29:27.103217+0000","last_unstale":"2023-07-18T03:29:28.249855+0000","last_undegraded":"2023-07-18T03:29:28.249855+0000","last_fullsized":"2023-07-18T03:29:28.249855+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:37:25.309228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248500+0000","last_change":"2023-07-18T03:29:27.096640+0000","last_active":"2023-07-18T03:29:28.248500+0000","last_peered":"2023-07-18T03:29:28.248500+0000","last_clean":"2023-07-18T03:29:28.248500+0000","last_became_active":"2023-07-18T03:29:27.096347+0000","last_became_peered":"2023-07-18T03:29:27.096347+0000","last_unstale":"2023-07-18T03:29:28.248500+0000","last_undegraded":"2023-07-18T03:29:28.248500+0000","last_fullsized":"2023-07-18T03:29:28.248500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:31:27.280331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248292+0000","last_change":"2023-07-18T03:29:27.095910+0000","last_active":"2023-07-18T03:29:28.248292+0000","last_peered":"2023-07-18T03:29:28.248292+0000","last_clean":"2023-07-18T03:29:28.248292+0000","last_became_active":"2023-07-18T03:29:27.095556+0000","last_became_peered":"2023-07-18T03:29:27.095556+0000","last_unstale":"2023-07-18T03:29:28.248292+0000","last_undegraded":"2023-07-18T03:29:28.248292+0000","last_fullsized":"2023-07-18T03:29:28.248292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:09:59.583920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248551+0000","last_change":"2023-07-18T03:29:27.108220+0000","last_active":"2023-07-18T03:29:28.248551+0000","last_peered":"2023-07-18T03:29:28.248551+0000","last_clean":"2023-07-18T03:29:28.248551+0000","last_became_active":"2023-07-18T03:29:27.107869+0000","last_became_peered":"2023-07-18T03:29:27.107869+0000","last_unstale":"2023-07-18T03:29:28.248551+0000","last_undegraded":"2023-07-18T03:29:28.248551+0000","last_fullsized":"2023-07-18T03:29:28.248551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:53:30.964666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.086002+0000","last_change":"2023-07-18T03:29:27.100634+0000","last_active":"2023-07-18T03:29:28.086002+0000","last_peered":"2023-07-18T03:29:28.086002+0000","last_clean":"2023-07-18T03:29:28.086002+0000","last_became_active":"2023-07-18T03:29:27.100316+0000","last_became_peered":"2023-07-18T03:29:27.100316+0000","last_unstale":"2023-07-18T03:29:28.086002+0000","last_undegraded":"2023-07-18T03:29:28.086002+0000","last_fullsized":"2023-07-18T03:29:28.086002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:35:30.023399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.235377+0000","last_change":"2023-07-18T03:29:27.100322+0000","last_active":"2023-07-18T03:29:28.235377+0000","last_peered":"2023-07-18T03:29:28.235377+0000","last_clean":"2023-07-18T03:29:28.235377+0000","last_became_active":"2023-07-18T03:29:27.099782+0000","last_became_peered":"2023-07-18T03:29:27.099782+0000","last_unstale":"2023-07-18T03:29:28.235377+0000","last_undegraded":"2023-07-18T03:29:28.235377+0000","last_fullsized":"2023-07-18T03:29:28.235377+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:28:12.377533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129942+0000","last_change":"2023-07-18T03:29:27.096639+0000","last_active":"2023-07-18T03:29:28.129942+0000","last_peered":"2023-07-18T03:29:28.129942+0000","last_clean":"2023-07-18T03:29:28.129942+0000","last_became_active":"2023-07-18T03:29:27.096348+0000","last_became_peered":"2023-07-18T03:29:27.096348+0000","last_unstale":"2023-07-18T03:29:28.129942+0000","last_undegraded":"2023-07-18T03:29:28.129942+0000","last_fullsized":"2023-07-18T03:29:28.129942+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:05:09.093995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.235481+0000","last_change":"2023-07-18T03:29:26.287722+0000","last_active":"2023-07-18T03:29:28.235481+0000","last_peered":"2023-07-18T03:29:28.235481+0000","last_clean":"2023-07-18T03:29:28.235481+0000","last_became_active":"2023-07-18T03:29:26.287327+0000","last_became_peered":"2023-07-18T03:29:26.287327+0000","last_unstale":"2023-07-18T03:29:28.235481+0000","last_undegraded":"2023-07-18T03:29:28.235481+0000","last_fullsized":"2023-07-18T03:29:28.235481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:57:15.630654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249206+0000","last_change":"2023-07-18T03:29:27.092771+0000","last_active":"2023-07-18T03:29:28.249206+0000","last_peered":"2023-07-18T03:29:28.249206+0000","last_clean":"2023-07-18T03:29:28.249206+0000","last_became_active":"2023-07-18T03:29:27.092456+0000","last_became_peered":"2023-07-18T03:29:27.092456+0000","last_unstale":"2023-07-18T03:29:28.249206+0000","last_undegraded":"2023-07-18T03:29:28.249206+0000","last_fullsized":"2023-07-18T03:29:28.249206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:02:05.004006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249555+0000","last_change":"2023-07-18T03:29:26.289683+0000","last_active":"2023-07-18T03:29:28.249555+0000","last_peered":"2023-07-18T03:29:28.249555+0000","last_clean":"2023-07-18T03:29:28.249555+0000","last_became_active":"2023-07-18T03:29:26.289356+0000","last_became_peered":"2023-07-18T03:29:26.289356+0000","last_unstale":"2023-07-18T03:29:28.249555+0000","last_undegraded":"2023-07-18T03:29:28.249555+0000","last_fullsized":"2023-07-18T03:29:28.249555+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:02:04.478209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249301+0000","last_change":"2023-07-18T03:29:27.098709+0000","last_active":"2023-07-18T03:29:28.249301+0000","last_peered":"2023-07-18T03:29:28.249301+0000","last_clean":"2023-07-18T03:29:28.249301+0000","last_became_active":"2023-07-18T03:29:27.098395+0000","last_became_peered":"2023-07-18T03:29:27.098395+0000","last_unstale":"2023-07-18T03:29:28.249301+0000","last_undegraded":"2023-07-18T03:29:28.249301+0000","last_fullsized":"2023-07-18T03:29:28.249301+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:26:19.426023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.608784+0000","last_change":"2023-07-18T03:29:26.297119+0000","last_active":"2023-07-18T03:29:31.608784+0000","last_peered":"2023-07-18T03:29:31.608784+0000","last_clean":"2023-07-18T03:29:31.608784+0000","last_became_active":"2023-07-18T03:29:26.296649+0000","last_became_peered":"2023-07-18T03:29:26.296649+0000","last_unstale":"2023-07-18T03:29:31.608784+0000","last_undegraded":"2023-07-18T03:29:31.608784+0000","last_fullsized":"2023-07-18T03:29:31.608784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:51:17.267059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248457+0000","last_change":"2023-07-18T03:29:27.105511+0000","last_active":"2023-07-18T03:29:28.248457+0000","last_peered":"2023-07-18T03:29:28.248457+0000","last_clean":"2023-07-18T03:29:28.248457+0000","last_became_active":"2023-07-18T03:29:27.104946+0000","last_became_peered":"2023-07-18T03:29:27.104946+0000","last_unstale":"2023-07-18T03:29:28.248457+0000","last_undegraded":"2023-07-18T03:29:28.248457+0000","last_fullsized":"2023-07-18T03:29:28.248457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:42:13.652404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249222+0000","last_change":"2023-07-18T03:29:26.290708+0000","last_active":"2023-07-18T03:29:28.249222+0000","last_peered":"2023-07-18T03:29:28.249222+0000","last_clean":"2023-07-18T03:29:28.249222+0000","last_became_active":"2023-07-18T03:29:26.290383+0000","last_became_peered":"2023-07-18T03:29:26.290383+0000","last_unstale":"2023-07-18T03:29:28.249222+0000","last_undegraded":"2023-07-18T03:29:28.249222+0000","last_fullsized":"2023-07-18T03:29:28.249222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:34:22.740683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.084957+0000","last_change":"2023-07-18T03:29:27.096630+0000","last_active":"2023-07-18T03:29:28.084957+0000","last_peered":"2023-07-18T03:29:28.084957+0000","last_clean":"2023-07-18T03:29:28.084957+0000","last_became_active":"2023-07-18T03:29:27.096337+0000","last_became_peered":"2023-07-18T03:29:27.096337+0000","last_unstale":"2023-07-18T03:29:28.084957+0000","last_undegraded":"2023-07-18T03:29:28.084957+0000","last_fullsized":"2023-07-18T03:29:28.084957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:23:46.241177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.628062+0000","last_change":"2023-07-18T03:29:26.291622+0000","last_active":"2023-07-18T03:29:30.628062+0000","last_peered":"2023-07-18T03:29:30.628062+0000","last_clean":"2023-07-18T03:29:30.628062+0000","last_became_active":"2023-07-18T03:29:26.291317+0000","last_became_peered":"2023-07-18T03:29:26.291317+0000","last_unstale":"2023-07-18T03:29:30.628062+0000","last_undegraded":"2023-07-18T03:29:30.628062+0000","last_fullsized":"2023-07-18T03:29:30.628062+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:37:53.996938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248914+0000","last_change":"2023-07-18T03:29:27.093107+0000","last_active":"2023-07-18T03:29:28.248914+0000","last_peered":"2023-07-18T03:29:28.248914+0000","last_clean":"2023-07-18T03:29:28.248914+0000","last_became_active":"2023-07-18T03:29:27.092785+0000","last_became_peered":"2023-07-18T03:29:27.092785+0000","last_unstale":"2023-07-18T03:29:28.248914+0000","last_undegraded":"2023-07-18T03:29:28.248914+0000","last_fullsized":"2023-07-18T03:29:28.248914+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:18:41.599570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085394+0000","last_change":"2023-07-18T03:29:26.290493+0000","last_active":"2023-07-18T03:29:28.085394+0000","last_peered":"2023-07-18T03:29:28.085394+0000","last_clean":"2023-07-18T03:29:28.085394+0000","last_became_active":"2023-07-18T03:29:26.290187+0000","last_became_peered":"2023-07-18T03:29:26.290187+0000","last_unstale":"2023-07-18T03:29:28.085394+0000","last_undegraded":"2023-07-18T03:29:28.085394+0000","last_fullsized":"2023-07-18T03:29:28.085394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:14:58.224602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.084989+0000","last_change":"2023-07-18T03:29:27.095129+0000","last_active":"2023-07-18T03:29:28.084989+0000","last_peered":"2023-07-18T03:29:28.084989+0000","last_clean":"2023-07-18T03:29:28.084989+0000","last_became_active":"2023-07-18T03:29:27.094821+0000","last_became_peered":"2023-07-18T03:29:27.094821+0000","last_unstale":"2023-07-18T03:29:28.084989+0000","last_undegraded":"2023-07-18T03:29:28.084989+0000","last_fullsized":"2023-07-18T03:29:28.084989+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:33:21.671061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.610770+0000","last_change":"2023-07-18T03:29:26.285860+0000","last_active":"2023-07-18T03:29:31.610770+0000","last_peered":"2023-07-18T03:29:31.610770+0000","last_clean":"2023-07-18T03:29:31.610770+0000","last_became_active":"2023-07-18T03:29:26.285491+0000","last_became_peered":"2023-07-18T03:29:26.285491+0000","last_unstale":"2023-07-18T03:29:31.610770+0000","last_undegraded":"2023-07-18T03:29:31.610770+0000","last_fullsized":"2023-07-18T03:29:31.610770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:14:20.414177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248682+0000","last_change":"2023-07-18T03:29:27.093861+0000","last_active":"2023-07-18T03:29:28.248682+0000","last_peered":"2023-07-18T03:29:28.248682+0000","last_clean":"2023-07-18T03:29:28.248682+0000","last_became_active":"2023-07-18T03:29:27.093445+0000","last_became_peered":"2023-07-18T03:29:27.093445+0000","last_unstale":"2023-07-18T03:29:28.248682+0000","last_undegraded":"2023-07-18T03:29:28.248682+0000","last_fullsized":"2023-07-18T03:29:28.248682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:05:28.911877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085209+0000","last_change":"2023-07-18T03:29:26.288108+0000","last_active":"2023-07-18T03:29:28.085209+0000","last_peered":"2023-07-18T03:29:28.085209+0000","last_clean":"2023-07-18T03:29:28.085209+0000","last_became_active":"2023-07-18T03:29:26.287531+0000","last_became_peered":"2023-07-18T03:29:26.287531+0000","last_unstale":"2023-07-18T03:29:28.085209+0000","last_undegraded":"2023-07-18T03:29:28.085209+0000","last_fullsized":"2023-07-18T03:29:28.085209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:35:36.416356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.235430+0000","last_change":"2023-07-18T03:29:27.092908+0000","last_active":"2023-07-18T03:29:28.235430+0000","last_peered":"2023-07-18T03:29:28.235430+0000","last_clean":"2023-07-18T03:29:28.235430+0000","last_became_active":"2023-07-18T03:29:27.092566+0000","last_became_peered":"2023-07-18T03:29:27.092566+0000","last_unstale":"2023-07-18T03:29:28.235430+0000","last_undegraded":"2023-07-18T03:29:28.235430+0000","last_fullsized":"2023-07-18T03:29:28.235430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:22:47.560376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085581+0000","last_change":"2023-07-18T03:29:26.294251+0000","last_active":"2023-07-18T03:29:28.085581+0000","last_peered":"2023-07-18T03:29:28.085581+0000","last_clean":"2023-07-18T03:29:28.085581+0000","last_became_active":"2023-07-18T03:29:26.293847+0000","last_became_peered":"2023-07-18T03:29:26.293847+0000","last_unstale":"2023-07-18T03:29:28.085581+0000","last_undegraded":"2023-07-18T03:29:28.085581+0000","last_fullsized":"2023-07-18T03:29:28.085581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:51:26.530234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248737+0000","last_change":"2023-07-18T03:29:27.106551+0000","last_active":"2023-07-18T03:29:28.248737+0000","last_peered":"2023-07-18T03:29:28.248737+0000","last_clean":"2023-07-18T03:29:28.248737+0000","last_became_active":"2023-07-18T03:29:27.105187+0000","last_became_peered":"2023-07-18T03:29:27.105187+0000","last_unstale":"2023-07-18T03:29:28.248737+0000","last_undegraded":"2023-07-18T03:29:28.248737+0000","last_fullsized":"2023-07-18T03:29:28.248737+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:45:05.223020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.625681+0000","last_change":"2023-07-18T03:29:26.293284+0000","last_active":"2023-07-18T03:29:30.625681+0000","last_peered":"2023-07-18T03:29:30.625681+0000","last_clean":"2023-07-18T03:29:30.625681+0000","last_became_active":"2023-07-18T03:29:26.292955+0000","last_became_peered":"2023-07-18T03:29:26.292955+0000","last_unstale":"2023-07-18T03:29:30.625681+0000","last_undegraded":"2023-07-18T03:29:30.625681+0000","last_fullsized":"2023-07-18T03:29:30.625681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:15:36.890771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249337+0000","last_change":"2023-07-18T03:29:27.097322+0000","last_active":"2023-07-18T03:29:28.249337+0000","last_peered":"2023-07-18T03:29:28.249337+0000","last_clean":"2023-07-18T03:29:28.249337+0000","last_became_active":"2023-07-18T03:29:27.097051+0000","last_became_peered":"2023-07-18T03:29:27.097051+0000","last_unstale":"2023-07-18T03:29:28.249337+0000","last_undegraded":"2023-07-18T03:29:28.249337+0000","last_fullsized":"2023-07-18T03:29:28.249337+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:36:33.129283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085166+0000","last_change":"2023-07-18T03:29:26.290810+0000","last_active":"2023-07-18T03:29:28.085166+0000","last_peered":"2023-07-18T03:29:28.085166+0000","last_clean":"2023-07-18T03:29:28.085166+0000","last_became_active":"2023-07-18T03:29:26.290448+0000","last_became_peered":"2023-07-18T03:29:26.290448+0000","last_unstale":"2023-07-18T03:29:28.085166+0000","last_undegraded":"2023-07-18T03:29:28.085166+0000","last_fullsized":"2023-07-18T03:29:28.085166+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:21:19.581142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249116+0000","last_change":"2023-07-18T03:29:27.105801+0000","last_active":"2023-07-18T03:29:28.249116+0000","last_peered":"2023-07-18T03:29:28.249116+0000","last_clean":"2023-07-18T03:29:28.249116+0000","last_became_active":"2023-07-18T03:29:27.104715+0000","last_became_peered":"2023-07-18T03:29:27.104715+0000","last_unstale":"2023-07-18T03:29:28.249116+0000","last_undegraded":"2023-07-18T03:29:28.249116+0000","last_fullsized":"2023-07-18T03:29:28.249116+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:49:34.118157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'6","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085739+0000","last_change":"2023-07-18T03:29:26.293306+0000","last_active":"2023-07-18T03:29:28.085739+0000","last_peered":"2023-07-18T03:29:28.085739+0000","last_clean":"2023-07-18T03:29:28.085739+0000","last_became_active":"2023-07-18T03:29:26.292946+0000","last_became_peered":"2023-07-18T03:29:26.292946+0000","last_unstale":"2023-07-18T03:29:28.085739+0000","last_undegraded":"2023-07-18T03:29:28.085739+0000","last_fullsized":"2023-07-18T03:29:28.085739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:36:19.196296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249042+0000","last_change":"2023-07-18T03:29:27.103941+0000","last_active":"2023-07-18T03:29:28.249042+0000","last_peered":"2023-07-18T03:29:28.249042+0000","last_clean":"2023-07-18T03:29:28.249042+0000","last_became_active":"2023-07-18T03:29:27.103506+0000","last_became_peered":"2023-07-18T03:29:27.103506+0000","last_unstale":"2023-07-18T03:29:28.249042+0000","last_undegraded":"2023-07-18T03:29:28.249042+0000","last_fullsized":"2023-07-18T03:29:28.249042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:13:27.299459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.608558+0000","last_change":"2023-07-18T03:29:26.293473+0000","last_active":"2023-07-18T03:29:31.608558+0000","last_peered":"2023-07-18T03:29:31.608558+0000","last_clean":"2023-07-18T03:29:31.608558+0000","last_became_active":"2023-07-18T03:29:26.293132+0000","last_became_peered":"2023-07-18T03:29:26.293132+0000","last_unstale":"2023-07-18T03:29:31.608558+0000","last_undegraded":"2023-07-18T03:29:31.608558+0000","last_fullsized":"2023-07-18T03:29:31.608558+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:32:07.174316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249663+0000","last_change":"2023-07-18T03:29:27.104032+0000","last_active":"2023-07-18T03:29:28.249663+0000","last_peered":"2023-07-18T03:29:28.249663+0000","last_clean":"2023-07-18T03:29:28.249663+0000","last_became_active":"2023-07-18T03:29:27.103736+0000","last_became_peered":"2023-07-18T03:29:27.103736+0000","last_unstale":"2023-07-18T03:29:28.249663+0000","last_undegraded":"2023-07-18T03:29:28.249663+0000","last_fullsized":"2023-07-18T03:29:28.249663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:29:13.489385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.626335+0000","last_change":"2023-07-18T03:29:26.290487+0000","last_active":"2023-07-18T03:29:30.626335+0000","last_peered":"2023-07-18T03:29:30.626335+0000","last_clean":"2023-07-18T03:29:30.626335+0000","last_became_active":"2023-07-18T03:29:26.290115+0000","last_became_peered":"2023-07-18T03:29:26.290115+0000","last_unstale":"2023-07-18T03:29:30.626335+0000","last_undegraded":"2023-07-18T03:29:30.626335+0000","last_fullsized":"2023-07-18T03:29:30.626335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:35:42.866787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339569+0000","last_change":"2023-07-18T03:29:27.108550+0000","last_active":"2023-07-18T03:29:28.339569+0000","last_peered":"2023-07-18T03:29:28.339569+0000","last_clean":"2023-07-18T03:29:28.339569+0000","last_became_active":"2023-07-18T03:29:27.108069+0000","last_became_peered":"2023-07-18T03:29:27.108069+0000","last_unstale":"2023-07-18T03:29:28.339569+0000","last_undegraded":"2023-07-18T03:29:28.339569+0000","last_fullsized":"2023-07-18T03:29:28.339569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:55:48.510403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248480+0000","last_change":"2023-07-18T03:29:26.290896+0000","last_active":"2023-07-18T03:29:28.248480+0000","last_peered":"2023-07-18T03:29:28.248480+0000","last_clean":"2023-07-18T03:29:28.248480+0000","last_became_active":"2023-07-18T03:29:26.290450+0000","last_became_peered":"2023-07-18T03:29:26.290450+0000","last_unstale":"2023-07-18T03:29:28.248480+0000","last_undegraded":"2023-07-18T03:29:28.248480+0000","last_fullsized":"2023-07-18T03:29:28.248480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:23:16.823095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339200+0000","last_change":"2023-07-18T03:29:27.109184+0000","last_active":"2023-07-18T03:29:28.339200+0000","last_peered":"2023-07-18T03:29:28.339200+0000","last_clean":"2023-07-18T03:29:28.339200+0000","last_became_active":"2023-07-18T03:29:27.107993+0000","last_became_peered":"2023-07-18T03:29:27.107993+0000","last_unstale":"2023-07-18T03:29:28.339200+0000","last_undegraded":"2023-07-18T03:29:28.339200+0000","last_fullsized":"2023-07-18T03:29:28.339200+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:38:53.665857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.612612+0000","last_change":"2023-07-18T03:29:26.293671+0000","last_active":"2023-07-18T03:29:31.612612+0000","last_peered":"2023-07-18T03:29:31.612612+0000","last_clean":"2023-07-18T03:29:31.612612+0000","last_became_active":"2023-07-18T03:29:26.293205+0000","last_became_peered":"2023-07-18T03:29:26.293205+0000","last_unstale":"2023-07-18T03:29:31.612612+0000","last_undegraded":"2023-07-18T03:29:31.612612+0000","last_fullsized":"2023-07-18T03:29:31.612612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:52:09.605399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.084887+0000","last_change":"2023-07-18T03:29:27.096711+0000","last_active":"2023-07-18T03:29:28.084887+0000","last_peered":"2023-07-18T03:29:28.084887+0000","last_clean":"2023-07-18T03:29:28.084887+0000","last_became_active":"2023-07-18T03:29:27.096423+0000","last_became_peered":"2023-07-18T03:29:27.096423+0000","last_unstale":"2023-07-18T03:29:28.084887+0000","last_undegraded":"2023-07-18T03:29:28.084887+0000","last_fullsized":"2023-07-18T03:29:28.084887+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:35:54.060369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249365+0000","last_change":"2023-07-18T03:29:26.291546+0000","last_active":"2023-07-18T03:29:28.249365+0000","last_peered":"2023-07-18T03:29:28.249365+0000","last_clean":"2023-07-18T03:29:28.249365+0000","last_became_active":"2023-07-18T03:29:26.291319+0000","last_became_peered":"2023-07-18T03:29:26.291319+0000","last_unstale":"2023-07-18T03:29:28.249365+0000","last_undegraded":"2023-07-18T03:29:28.249365+0000","last_fullsized":"2023-07-18T03:29:28.249365+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:23:25.182800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085688+0000","last_change":"2023-07-18T03:29:27.098479+0000","last_active":"2023-07-18T03:29:28.085688+0000","last_peered":"2023-07-18T03:29:28.085688+0000","last_clean":"2023-07-18T03:29:28.085688+0000","last_became_active":"2023-07-18T03:29:27.098153+0000","last_became_peered":"2023-07-18T03:29:27.098153+0000","last_unstale":"2023-07-18T03:29:28.085688+0000","last_undegraded":"2023-07-18T03:29:28.085688+0000","last_fullsized":"2023-07-18T03:29:28.085688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:53:01.674305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.235436+0000","last_change":"2023-07-18T03:29:26.290953+0000","last_active":"2023-07-18T03:29:28.235436+0000","last_peered":"2023-07-18T03:29:28.235436+0000","last_clean":"2023-07-18T03:29:28.235436+0000","last_became_active":"2023-07-18T03:29:26.290633+0000","last_became_peered":"2023-07-18T03:29:26.290633+0000","last_unstale":"2023-07-18T03:29:28.235436+0000","last_undegraded":"2023-07-18T03:29:28.235436+0000","last_fullsized":"2023-07-18T03:29:28.235436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:40:49.185595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085754+0000","last_change":"2023-07-18T03:29:27.093651+0000","last_active":"2023-07-18T03:29:28.085754+0000","last_peered":"2023-07-18T03:29:28.085754+0000","last_clean":"2023-07-18T03:29:28.085754+0000","last_became_active":"2023-07-18T03:29:27.093307+0000","last_became_peered":"2023-07-18T03:29:27.093307+0000","last_unstale":"2023-07-18T03:29:28.085754+0000","last_undegraded":"2023-07-18T03:29:28.085754+0000","last_fullsized":"2023-07-18T03:29:28.085754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:31:10.315148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339733+0000","last_change":"2023-07-18T03:29:26.287405+0000","last_active":"2023-07-18T03:29:28.339733+0000","last_peered":"2023-07-18T03:29:28.339733+0000","last_clean":"2023-07-18T03:29:28.339733+0000","last_became_active":"2023-07-18T03:29:26.287048+0000","last_became_peered":"2023-07-18T03:29:26.287048+0000","last_unstale":"2023-07-18T03:29:28.339733+0000","last_undegraded":"2023-07-18T03:29:28.339733+0000","last_fullsized":"2023-07-18T03:29:28.339733+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:50:53.303258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085803+0000","last_change":"2023-07-18T03:29:27.103218+0000","last_active":"2023-07-18T03:29:28.085803+0000","last_peered":"2023-07-18T03:29:28.085803+0000","last_clean":"2023-07-18T03:29:28.085803+0000","last_became_active":"2023-07-18T03:29:27.102917+0000","last_became_peered":"2023-07-18T03:29:27.102917+0000","last_unstale":"2023-07-18T03:29:28.085803+0000","last_undegraded":"2023-07-18T03:29:28.085803+0000","last_fullsized":"2023-07-18T03:29:28.085803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:23:49.743367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085272+0000","last_change":"2023-07-18T03:29:27.098696+0000","last_active":"2023-07-18T03:29:28.085272+0000","last_peered":"2023-07-18T03:29:28.085272+0000","last_clean":"2023-07-18T03:29:28.085272+0000","last_became_active":"2023-07-18T03:29:27.098253+0000","last_became_peered":"2023-07-18T03:29:27.098253+0000","last_unstale":"2023-07-18T03:29:28.085272+0000","last_undegraded":"2023-07-18T03:29:28.085272+0000","last_fullsized":"2023-07-18T03:29:28.085272+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:40:58.942045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.609724+0000","last_change":"2023-07-18T03:29:26.290308+0000","last_active":"2023-07-18T03:29:31.609724+0000","last_peered":"2023-07-18T03:29:31.609724+0000","last_clean":"2023-07-18T03:29:31.609724+0000","last_became_active":"2023-07-18T03:29:26.289937+0000","last_became_peered":"2023-07-18T03:29:26.289937+0000","last_unstale":"2023-07-18T03:29:31.609724+0000","last_undegraded":"2023-07-18T03:29:31.609724+0000","last_fullsized":"2023-07-18T03:29:31.609724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:21:51.452197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249450+0000","last_change":"2023-07-18T03:29:27.096469+0000","last_active":"2023-07-18T03:29:28.249450+0000","last_peered":"2023-07-18T03:29:28.249450+0000","last_clean":"2023-07-18T03:29:28.249450+0000","last_became_active":"2023-07-18T03:29:27.096171+0000","last_became_peered":"2023-07-18T03:29:27.096171+0000","last_unstale":"2023-07-18T03:29:28.249450+0000","last_undegraded":"2023-07-18T03:29:28.249450+0000","last_fullsized":"2023-07-18T03:29:28.249450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:14:47.599303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.626213+0000","last_change":"2023-07-18T03:29:26.290935+0000","last_active":"2023-07-18T03:29:30.626213+0000","last_peered":"2023-07-18T03:29:30.626213+0000","last_clean":"2023-07-18T03:29:30.626213+0000","last_became_active":"2023-07-18T03:29:26.290595+0000","last_became_peered":"2023-07-18T03:29:26.290595+0000","last_unstale":"2023-07-18T03:29:30.626213+0000","last_undegraded":"2023-07-18T03:29:30.626213+0000","last_fullsized":"2023-07-18T03:29:30.626213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:34:23.417106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":331,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339337+0000","last_change":"2023-07-18T03:28:28.122385+0000","last_active":"2023-07-18T03:29:28.339337+0000","last_peered":"2023-07-18T03:29:28.339337+0000","last_clean":"2023-07-18T03:29:28.339337+0000","last_became_active":"2023-07-18T03:28:28.121932+0000","last_became_peered":"2023-07-18T03:28:28.121932+0000","last_unstale":"2023-07-18T03:29:28.339337+0000","last_undegraded":"2023-07-18T03:29:28.339337+0000","last_fullsized":"2023-07-18T03:29:28.339337+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:25:38.267627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:25:38.267627+0000","last_clean_scrub_stamp":"2023-07-18T03:25:38.267627+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:46:37.243383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248934+0000","last_change":"2023-07-18T03:29:27.101582+0000","last_active":"2023-07-18T03:29:28.248934+0000","last_peered":"2023-07-18T03:29:28.248934+0000","last_clean":"2023-07-18T03:29:28.248934+0000","last_became_active":"2023-07-18T03:29:27.101226+0000","last_became_peered":"2023-07-18T03:29:27.101226+0000","last_unstale":"2023-07-18T03:29:28.248934+0000","last_undegraded":"2023-07-18T03:29:28.248934+0000","last_fullsized":"2023-07-18T03:29:28.248934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:42:14.571198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.607147+0000","last_change":"2023-07-18T03:29:26.291787+0000","last_active":"2023-07-18T03:29:31.607147+0000","last_peered":"2023-07-18T03:29:31.607147+0000","last_clean":"2023-07-18T03:29:31.607147+0000","last_became_active":"2023-07-18T03:29:26.291471+0000","last_became_peered":"2023-07-18T03:29:26.291471+0000","last_unstale":"2023-07-18T03:29:31.607147+0000","last_undegraded":"2023-07-18T03:29:31.607147+0000","last_fullsized":"2023-07-18T03:29:31.607147+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:49:20.527148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.626010+0000","last_change":"2023-07-18T03:29:26.296936+0000","last_active":"2023-07-18T03:29:30.626010+0000","last_peered":"2023-07-18T03:29:30.626010+0000","last_clean":"2023-07-18T03:29:30.626010+0000","last_became_active":"2023-07-18T03:29:26.296241+0000","last_became_peered":"2023-07-18T03:29:26.296241+0000","last_unstale":"2023-07-18T03:29:30.626010+0000","last_undegraded":"2023-07-18T03:29:30.626010+0000","last_fullsized":"2023-07-18T03:29:30.626010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:23:24.313635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249296+0000","last_change":"2023-07-18T03:29:27.102715+0000","last_active":"2023-07-18T03:29:28.249296+0000","last_peered":"2023-07-18T03:29:28.249296+0000","last_clean":"2023-07-18T03:29:28.249296+0000","last_became_active":"2023-07-18T03:29:27.100914+0000","last_became_peered":"2023-07-18T03:29:27.100914+0000","last_unstale":"2023-07-18T03:29:28.249296+0000","last_undegraded":"2023-07-18T03:29:28.249296+0000","last_fullsized":"2023-07-18T03:29:28.249296+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:43:38.965763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.235404+0000","last_change":"2023-07-18T03:29:26.290231+0000","last_active":"2023-07-18T03:29:28.235404+0000","last_peered":"2023-07-18T03:29:28.235404+0000","last_clean":"2023-07-18T03:29:28.235404+0000","last_became_active":"2023-07-18T03:29:26.289753+0000","last_became_peered":"2023-07-18T03:29:26.289753+0000","last_unstale":"2023-07-18T03:29:28.235404+0000","last_undegraded":"2023-07-18T03:29:28.235404+0000","last_fullsized":"2023-07-18T03:29:28.235404+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:33:36.792644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249097+0000","last_change":"2023-07-18T03:29:27.103964+0000","last_active":"2023-07-18T03:29:28.249097+0000","last_peered":"2023-07-18T03:29:28.249097+0000","last_clean":"2023-07-18T03:29:28.249097+0000","last_became_active":"2023-07-18T03:29:27.103660+0000","last_became_peered":"2023-07-18T03:29:27.103660+0000","last_unstale":"2023-07-18T03:29:28.249097+0000","last_undegraded":"2023-07-18T03:29:28.249097+0000","last_fullsized":"2023-07-18T03:29:28.249097+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:08:09.167239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.625316+0000","last_change":"2023-07-18T03:29:26.286436+0000","last_active":"2023-07-18T03:29:30.625316+0000","last_peered":"2023-07-18T03:29:30.625316+0000","last_clean":"2023-07-18T03:29:30.625316+0000","last_became_active":"2023-07-18T03:29:26.286070+0000","last_became_peered":"2023-07-18T03:29:26.286070+0000","last_unstale":"2023-07-18T03:29:30.625316+0000","last_undegraded":"2023-07-18T03:29:30.625316+0000","last_fullsized":"2023-07-18T03:29:30.625316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:14:53.466702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249329+0000","last_change":"2023-07-18T03:29:27.102303+0000","last_active":"2023-07-18T03:29:28.249329+0000","last_peered":"2023-07-18T03:29:28.249329+0000","last_clean":"2023-07-18T03:29:28.249329+0000","last_became_active":"2023-07-18T03:29:27.101980+0000","last_became_peered":"2023-07-18T03:29:27.101980+0000","last_unstale":"2023-07-18T03:29:28.249329+0000","last_undegraded":"2023-07-18T03:29:28.249329+0000","last_fullsized":"2023-07-18T03:29:28.249329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:46:48.299459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.624991+0000","last_change":"2023-07-18T03:29:26.289628+0000","last_active":"2023-07-18T03:29:30.624991+0000","last_peered":"2023-07-18T03:29:30.624991+0000","last_clean":"2023-07-18T03:29:30.624991+0000","last_became_active":"2023-07-18T03:29:26.289013+0000","last_became_peered":"2023-07-18T03:29:26.289013+0000","last_unstale":"2023-07-18T03:29:30.624991+0000","last_undegraded":"2023-07-18T03:29:30.624991+0000","last_fullsized":"2023-07-18T03:29:30.624991+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:58:20.665607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248709+0000","last_change":"2023-07-18T03:29:27.090943+0000","last_active":"2023-07-18T03:29:28.248709+0000","last_peered":"2023-07-18T03:29:28.248709+0000","last_clean":"2023-07-18T03:29:28.248709+0000","last_became_active":"2023-07-18T03:29:27.090465+0000","last_became_peered":"2023-07-18T03:29:27.090465+0000","last_unstale":"2023-07-18T03:29:28.248709+0000","last_undegraded":"2023-07-18T03:29:28.248709+0000","last_fullsized":"2023-07-18T03:29:28.248709+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:49:17.548432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249047+0000","last_change":"2023-07-18T03:29:26.288576+0000","last_active":"2023-07-18T03:29:28.249047+0000","last_peered":"2023-07-18T03:29:28.249047+0000","last_clean":"2023-07-18T03:29:28.249047+0000","last_became_active":"2023-07-18T03:29:26.287514+0000","last_became_peered":"2023-07-18T03:29:26.287514+0000","last_unstale":"2023-07-18T03:29:28.249047+0000","last_undegraded":"2023-07-18T03:29:28.249047+0000","last_fullsized":"2023-07-18T03:29:28.249047+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:37:47.805842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085745+0000","last_change":"2023-07-18T03:29:27.101111+0000","last_active":"2023-07-18T03:29:28.085745+0000","last_peered":"2023-07-18T03:29:28.085745+0000","last_clean":"2023-07-18T03:29:28.085745+0000","last_became_active":"2023-07-18T03:29:27.100737+0000","last_became_peered":"2023-07-18T03:29:27.100737+0000","last_unstale":"2023-07-18T03:29:28.085745+0000","last_undegraded":"2023-07-18T03:29:28.085745+0000","last_fullsized":"2023-07-18T03:29:28.085745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:00:58.906730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129626+0000","last_change":"2023-07-18T03:29:26.296974+0000","last_active":"2023-07-18T03:29:28.129626+0000","last_peered":"2023-07-18T03:29:28.129626+0000","last_clean":"2023-07-18T03:29:28.129626+0000","last_became_active":"2023-07-18T03:29:26.296451+0000","last_became_peered":"2023-07-18T03:29:26.296451+0000","last_unstale":"2023-07-18T03:29:28.129626+0000","last_undegraded":"2023-07-18T03:29:28.129626+0000","last_fullsized":"2023-07-18T03:29:28.129626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:35:42.821600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249813+0000","last_change":"2023-07-18T03:29:27.095857+0000","last_active":"2023-07-18T03:29:28.249813+0000","last_peered":"2023-07-18T03:29:28.249813+0000","last_clean":"2023-07-18T03:29:28.249813+0000","last_became_active":"2023-07-18T03:29:27.095553+0000","last_became_peered":"2023-07-18T03:29:27.095553+0000","last_unstale":"2023-07-18T03:29:28.249813+0000","last_undegraded":"2023-07-18T03:29:28.249813+0000","last_fullsized":"2023-07-18T03:29:28.249813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:30:11.142073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.627680+0000","last_change":"2023-07-18T03:29:26.295936+0000","last_active":"2023-07-18T03:29:30.627680+0000","last_peered":"2023-07-18T03:29:30.627680+0000","last_clean":"2023-07-18T03:29:30.627680+0000","last_became_active":"2023-07-18T03:29:26.295467+0000","last_became_peered":"2023-07-18T03:29:26.295467+0000","last_unstale":"2023-07-18T03:29:30.627680+0000","last_undegraded":"2023-07-18T03:29:30.627680+0000","last_fullsized":"2023-07-18T03:29:30.627680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:45:43.716125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339616+0000","last_change":"2023-07-18T03:29:27.108528+0000","last_active":"2023-07-18T03:29:28.339616+0000","last_peered":"2023-07-18T03:29:28.339616+0000","last_clean":"2023-07-18T03:29:28.339616+0000","last_became_active":"2023-07-18T03:29:27.108039+0000","last_became_peered":"2023-07-18T03:29:27.108039+0000","last_unstale":"2023-07-18T03:29:28.339616+0000","last_undegraded":"2023-07-18T03:29:28.339616+0000","last_fullsized":"2023-07-18T03:29:28.339616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:57:20.088291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:32.634752+0000","last_change":"2023-07-18T03:29:26.294006+0000","last_active":"2023-07-18T03:29:32.634752+0000","last_peered":"2023-07-18T03:29:32.634752+0000","last_clean":"2023-07-18T03:29:32.634752+0000","last_became_active":"2023-07-18T03:29:26.293700+0000","last_became_peered":"2023-07-18T03:29:26.293700+0000","last_unstale":"2023-07-18T03:29:32.634752+0000","last_undegraded":"2023-07-18T03:29:32.634752+0000","last_fullsized":"2023-07-18T03:29:32.634752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:44:37.451806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339715+0000","last_change":"2023-07-18T03:29:27.102764+0000","last_active":"2023-07-18T03:29:28.339715+0000","last_peered":"2023-07-18T03:29:28.339715+0000","last_clean":"2023-07-18T03:29:28.339715+0000","last_became_active":"2023-07-18T03:29:27.102200+0000","last_became_peered":"2023-07-18T03:29:27.102200+0000","last_unstale":"2023-07-18T03:29:28.339715+0000","last_undegraded":"2023-07-18T03:29:28.339715+0000","last_fullsized":"2023-07-18T03:29:28.339715+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:54:13.608760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.084954+0000","last_change":"2023-07-18T03:29:26.294271+0000","last_active":"2023-07-18T03:29:28.084954+0000","last_peered":"2023-07-18T03:29:28.084954+0000","last_clean":"2023-07-18T03:29:28.084954+0000","last_became_active":"2023-07-18T03:29:26.293938+0000","last_became_peered":"2023-07-18T03:29:26.293938+0000","last_unstale":"2023-07-18T03:29:28.084954+0000","last_undegraded":"2023-07-18T03:29:28.084954+0000","last_fullsized":"2023-07-18T03:29:28.084954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:10:38.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339388+0000","last_change":"2023-07-18T03:29:27.106510+0000","last_active":"2023-07-18T03:29:28.339388+0000","last_peered":"2023-07-18T03:29:28.339388+0000","last_clean":"2023-07-18T03:29:28.339388+0000","last_became_active":"2023-07-18T03:29:27.106222+0000","last_became_peered":"2023-07-18T03:29:27.106222+0000","last_unstale":"2023-07-18T03:29:28.339388+0000","last_undegraded":"2023-07-18T03:29:28.339388+0000","last_fullsized":"2023-07-18T03:29:28.339388+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:27:44.790790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248915+0000","last_change":"2023-07-18T03:29:26.293583+0000","last_active":"2023-07-18T03:29:28.248915+0000","last_peered":"2023-07-18T03:29:28.248915+0000","last_clean":"2023-07-18T03:29:28.248915+0000","last_became_active":"2023-07-18T03:29:26.293076+0000","last_became_peered":"2023-07-18T03:29:26.293076+0000","last_unstale":"2023-07-18T03:29:28.248915+0000","last_undegraded":"2023-07-18T03:29:28.248915+0000","last_fullsized":"2023-07-18T03:29:28.248915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:37:21.591425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339559+0000","last_change":"2023-07-18T03:29:27.109175+0000","last_active":"2023-07-18T03:29:28.339559+0000","last_peered":"2023-07-18T03:29:28.339559+0000","last_clean":"2023-07-18T03:29:28.339559+0000","last_became_active":"2023-07-18T03:29:27.108669+0000","last_became_peered":"2023-07-18T03:29:27.108669+0000","last_unstale":"2023-07-18T03:29:28.339559+0000","last_undegraded":"2023-07-18T03:29:28.339559+0000","last_fullsized":"2023-07-18T03:29:28.339559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:00:09.903589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085918+0000","last_change":"2023-07-18T03:29:27.099770+0000","last_active":"2023-07-18T03:29:28.085918+0000","last_peered":"2023-07-18T03:29:28.085918+0000","last_clean":"2023-07-18T03:29:28.085918+0000","last_became_active":"2023-07-18T03:29:27.099487+0000","last_became_peered":"2023-07-18T03:29:27.099487+0000","last_unstale":"2023-07-18T03:29:28.085918+0000","last_undegraded":"2023-07-18T03:29:28.085918+0000","last_fullsized":"2023-07-18T03:29:28.085918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:22:59.316119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129653+0000","last_change":"2023-07-18T03:29:27.100189+0000","last_active":"2023-07-18T03:29:28.129653+0000","last_peered":"2023-07-18T03:29:28.129653+0000","last_clean":"2023-07-18T03:29:28.129653+0000","last_became_active":"2023-07-18T03:29:27.099884+0000","last_became_peered":"2023-07-18T03:29:27.099884+0000","last_unstale":"2023-07-18T03:29:28.129653+0000","last_undegraded":"2023-07-18T03:29:28.129653+0000","last_fullsized":"2023-07-18T03:29:28.129653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:04:36.761236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129593+0000","last_change":"2023-07-18T03:29:27.099196+0000","last_active":"2023-07-18T03:29:28.129593+0000","last_peered":"2023-07-18T03:29:28.129593+0000","last_clean":"2023-07-18T03:29:28.129593+0000","last_became_active":"2023-07-18T03:29:27.098903+0000","last_became_peered":"2023-07-18T03:29:27.098903+0000","last_unstale":"2023-07-18T03:29:28.129593+0000","last_undegraded":"2023-07-18T03:29:28.129593+0000","last_fullsized":"2023-07-18T03:29:28.129593+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:46:05.998503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248469+0000","last_change":"2023-07-18T03:29:27.098138+0000","last_active":"2023-07-18T03:29:28.248469+0000","last_peered":"2023-07-18T03:29:28.248469+0000","last_clean":"2023-07-18T03:29:28.248469+0000","last_became_active":"2023-07-18T03:29:27.097793+0000","last_became_peered":"2023-07-18T03:29:27.097793+0000","last_unstale":"2023-07-18T03:29:28.248469+0000","last_undegraded":"2023-07-18T03:29:28.248469+0000","last_fullsized":"2023-07-18T03:29:28.248469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:07:31.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249184+0000","last_change":"2023-07-18T03:29:27.101610+0000","last_active":"2023-07-18T03:29:28.249184+0000","last_peered":"2023-07-18T03:29:28.249184+0000","last_clean":"2023-07-18T03:29:28.249184+0000","last_became_active":"2023-07-18T03:29:27.101332+0000","last_became_peered":"2023-07-18T03:29:27.101332+0000","last_unstale":"2023-07-18T03:29:28.249184+0000","last_undegraded":"2023-07-18T03:29:28.249184+0000","last_fullsized":"2023-07-18T03:29:28.249184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:43:01.583313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085879+0000","last_change":"2023-07-18T03:29:27.093345+0000","last_active":"2023-07-18T03:29:28.085879+0000","last_peered":"2023-07-18T03:29:28.085879+0000","last_clean":"2023-07-18T03:29:28.085879+0000","last_became_active":"2023-07-18T03:29:27.093048+0000","last_became_peered":"2023-07-18T03:29:27.093048+0000","last_unstale":"2023-07-18T03:29:28.085879+0000","last_undegraded":"2023-07-18T03:29:28.085879+0000","last_fullsized":"2023-07-18T03:29:28.085879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:00:19.659948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129830+0000","last_change":"2023-07-18T03:29:27.098311+0000","last_active":"2023-07-18T03:29:28.129830+0000","last_peered":"2023-07-18T03:29:28.129830+0000","last_clean":"2023-07-18T03:29:28.129830+0000","last_became_active":"2023-07-18T03:29:27.097980+0000","last_became_peered":"2023-07-18T03:29:27.097980+0000","last_unstale":"2023-07-18T03:29:28.129830+0000","last_undegraded":"2023-07-18T03:29:28.129830+0000","last_fullsized":"2023-07-18T03:29:28.129830+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:34:48.849264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248965+0000","last_change":"2023-07-18T03:29:27.093796+0000","last_active":"2023-07-18T03:29:28.248965+0000","last_peered":"2023-07-18T03:29:28.248965+0000","last_clean":"2023-07-18T03:29:28.248965+0000","last_became_active":"2023-07-18T03:29:27.093474+0000","last_became_peered":"2023-07-18T03:29:27.093474+0000","last_unstale":"2023-07-18T03:29:28.248965+0000","last_undegraded":"2023-07-18T03:29:28.248965+0000","last_fullsized":"2023-07-18T03:29:28.248965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:00:09.316029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085099+0000","last_change":"2023-07-18T03:29:27.098541+0000","last_active":"2023-07-18T03:29:28.085099+0000","last_peered":"2023-07-18T03:29:28.085099+0000","last_clean":"2023-07-18T03:29:28.085099+0000","last_became_active":"2023-07-18T03:29:27.097824+0000","last_became_peered":"2023-07-18T03:29:27.097824+0000","last_unstale":"2023-07-18T03:29:28.085099+0000","last_undegraded":"2023-07-18T03:29:28.085099+0000","last_fullsized":"2023-07-18T03:29:28.085099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:28:38.837863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248860+0000","last_change":"2023-07-18T03:29:27.096983+0000","last_active":"2023-07-18T03:29:28.248860+0000","last_peered":"2023-07-18T03:29:28.248860+0000","last_clean":"2023-07-18T03:29:28.248860+0000","last_became_active":"2023-07-18T03:29:27.096632+0000","last_became_peered":"2023-07-18T03:29:27.096632+0000","last_unstale":"2023-07-18T03:29:28.248860+0000","last_undegraded":"2023-07-18T03:29:28.248860+0000","last_fullsized":"2023-07-18T03:29:28.248860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:45:45.965235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249073+0000","last_change":"2023-07-18T03:29:27.105236+0000","last_active":"2023-07-18T03:29:28.249073+0000","last_peered":"2023-07-18T03:29:28.249073+0000","last_clean":"2023-07-18T03:29:28.249073+0000","last_became_active":"2023-07-18T03:29:27.104906+0000","last_became_peered":"2023-07-18T03:29:27.104906+0000","last_unstale":"2023-07-18T03:29:28.249073+0000","last_undegraded":"2023-07-18T03:29:28.249073+0000","last_fullsized":"2023-07-18T03:29:28.249073+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:53:05.499609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249970+0000","last_change":"2023-07-18T03:29:27.103461+0000","last_active":"2023-07-18T03:29:28.249970+0000","last_peered":"2023-07-18T03:29:28.249970+0000","last_clean":"2023-07-18T03:29:28.249970+0000","last_became_active":"2023-07-18T03:29:27.103162+0000","last_became_peered":"2023-07-18T03:29:27.103162+0000","last_unstale":"2023-07-18T03:29:28.249970+0000","last_undegraded":"2023-07-18T03:29:28.249970+0000","last_fullsized":"2023-07-18T03:29:28.249970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:26:14.600362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085635+0000","last_change":"2023-07-18T03:29:27.095300+0000","last_active":"2023-07-18T03:29:28.085635+0000","last_peered":"2023-07-18T03:29:28.085635+0000","last_clean":"2023-07-18T03:29:28.085635+0000","last_became_active":"2023-07-18T03:29:27.094985+0000","last_became_peered":"2023-07-18T03:29:27.094985+0000","last_unstale":"2023-07-18T03:29:28.085635+0000","last_undegraded":"2023-07-18T03:29:28.085635+0000","last_fullsized":"2023-07-18T03:29:28.085635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:09:05.373008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4898,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":98,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":12094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514508,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27800,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717352,"statfs":{"total":95995035648,"available":95966568448,"internally_reserved":0,"allocated":868352,"data_stored":417216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]}]},{"osd":10,"up_from":69,"seq":296352743440,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28404,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716748,"statfs":{"total":95995035648,"available":95965949952,"internally_reserved":0,"allocated":1486848,"data_stored":1009342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.42699999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1,"5min":1,"15min":1},"last":0.62}]},{"osd":3,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.59699999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.497},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.54600000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.86099999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.68999999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":8,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.93100000000000005}]},{"osd":9,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.83699999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]}]},{"osd":9,"up_from":64,"seq":274877906964,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":872448,"data_stored":417798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.312},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.33100000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.54100000000000004}]},{"osd":2,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.40899999999999997}]},{"osd":4,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.69599999999999995}]},{"osd":5,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.50600000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.39000000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.629}]},{"osd":8,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.61399999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.81699999999999995}]}]},{"osd":8,"up_from":56,"seq":240518168600,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":417623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.44400000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.80100000000000005}]},{"osd":2,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.68600000000000005}]},{"osd":4,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.876}]},{"osd":5,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.65500000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.75600000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.81000000000000005}]},{"osd":9,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.82199999999999995}]},{"osd":10,"last update":"Tue Jul 18 03:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.022}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":7,"up_from":49,"seq":210453397532,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28392,"kb_used_data":1440,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716760,"statfs":{"total":95995035648,"available":95965962240,"internally_reserved":0,"allocated":1474560,"data_stored":1008740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46200000000000002,"15min":0.45900000000000002},"min":{"1min":0.30599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.753,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.48399999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46600000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.379,"15min":0.379},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.628},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.621,"15min":0.60899999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.68700000000000006}]},{"osd":3,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.64900000000000002,"15min":0.63900000000000001},"min":{"1min":0.40600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93400000000000005,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.745},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66400000000000003,"15min":0.65800000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.65200000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56000000000000005}]},{"osd":5,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.48299999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.70899999999999996,"15min":0.69599999999999995},"min":{"1min":0.46600000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.83299999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.74399999999999999,"15min":0.73399999999999999},"min":{"1min":0.499,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.04,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.52900000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97599999999999998,"5min":1.103,"15min":1.103},"last":0.89300000000000002}]},{"osd":8,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.81200000000000006}]},{"osd":9,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.79600000000000004}]},{"osd":10,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.91900000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]}]},{"osd":6,"up_from":44,"seq":188978561057,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":872448,"data_stored":417306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40200000000000002},"min":{"1min":0.309,"5min":0.254,"15min":0.254},"max":{"1min":0.60299999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.32100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.58099999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.59999999999999998}]},{"osd":1,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.45600000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.42099999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78800000000000003,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.47499999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83599999999999997,"5min":1.008,"15min":1.008},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.63500000000000001,"15min":0.61899999999999999},"min":{"1min":0.48499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.55600000000000005}]},{"osd":3,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.498,"15min":0.48899999999999999},"min":{"1min":0.36399999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.36699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71099999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.56999999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.623,"15min":0.61399999999999999},"min":{"1min":0.52000000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85299999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.68500000000000005,"15min":0.67100000000000004},"min":{"1min":0.55100000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.43099999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.68100000000000005,"15min":0.67100000000000004},"min":{"1min":0.54400000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.628},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.503,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.96399999999999997,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.79300000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73199999999999998,"15min":0.72799999999999998},"min":{"1min":0.47199999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56899999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0580000000000001,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.46500000000000002}]},{"osd":8,"last update":"Tue Jul 18 03:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.67100000000000004}]},{"osd":9,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.878},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.51200000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.64700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":418780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51500000000000001,"15min":0.47899999999999998},"min":{"1min":0.34999999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81100000000000005,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.52600000000000002,"15min":0.503},"min":{"1min":0.38300000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.89100000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.80300000000000005}]},{"osd":2,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.55400000000000005,"15min":0.53700000000000003},"min":{"1min":0.47199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.497,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78600000000000003,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.77000000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.59099999999999997,"15min":0.55500000000000005},"min":{"1min":0.441,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.60799999999999998,"15min":0.57799999999999996},"min":{"1min":0.54700000000000004,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.85299999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.44700000000000001,"15min":0.40699999999999997},"min":{"1min":0.33200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.45000000000000001,"15min":0.42199999999999999},"min":{"1min":0.30499999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.752,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59199999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.45700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.628},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.627,"15min":0.60999999999999999},"min":{"1min":0.60799999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.83499999999999996,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.82499999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.63200000000000001,"15min":0.61699999999999999},"min":{"1min":0.498,"5min":0.443,"15min":0.443},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.68500000000000005,"15min":0.66900000000000004},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.91500000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.874},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.69599999999999995,"15min":0.67700000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.84099999999999997}]},{"osd":8,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.94399999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":1.0209999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":872448,"data_stored":417306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53600000000000003,"15min":0.51400000000000001},"min":{"1min":0.35999999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.629},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.374,"5min":0.318,"15min":0.318},"max":{"1min":0.82299999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.61399999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.55600000000000005,"15min":0.52300000000000002},"min":{"1min":0.41799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.55300000000000005,"15min":0.52400000000000002},"min":{"1min":0.45600000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.45200000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.42299999999999999,"15min":0.41499999999999998},"min":{"1min":0.315,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.628,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.376}]},{"osd":4,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56899999999999995,"15min":0.55100000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.92800000000000005,"5min":1.018,"15min":1.018},"last":0.57999999999999996}]},{"osd":5,"last update":"Tue Jul 18 03:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63500000000000001,"15min":0.624},"min":{"1min":0.52900000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.503}]},{"osd":6,"last update":"Tue Jul 18 03:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56499999999999995,"15min":0.55200000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.47999999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.42999999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56200000000000006}]},{"osd":9,"last update":"Tue Jul 18 03:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.65800000000000003}]},{"osd":10,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.72099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":420082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51500000000000001,"15min":0.51200000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75800000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54100000000000004,"15min":0.55400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.69899999999999995}]},{"osd":1,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55000000000000004,"15min":0.55500000000000005},"min":{"1min":0.41499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78200000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.53800000000000003,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.85799999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59099999999999997,"15min":0.57699999999999996},"min":{"1min":0.45300000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.41199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.58199999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55200000000000005,"15min":0.55500000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71999999999999997,"5min":0.872,"15min":0.872},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76500000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.749}]},{"osd":5,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.29699999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.503},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.42699999999999999,"15min":0.41999999999999998},"min":{"1min":0.314,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.47899999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55900000000000005,"15min":0.55200000000000005},"min":{"1min":0.40100000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.622},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58799999999999997,"15min":0.57499999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.79900000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.505,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.61699999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.68200000000000005}]},{"osd":9,"last update":"Tue Jul 18 03:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.81599999999999995}]},{"osd":10,"last update":"Tue Jul 18 03:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.83599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":876544,"data_stored":417320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.36399999999999999,"15min":0.34200000000000003},"min":{"1min":0.308,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.38600000000000001,"15min":0.36799999999999999},"min":{"1min":0.317,"5min":0.215,"15min":0.215},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.29699999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.56599999999999995,"15min":0.52700000000000002},"min":{"1min":0.42399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.53600000000000003,"15min":0.50600000000000001},"min":{"1min":0.42299999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78700000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.439}]},{"osd":2,"last update":"Tue Jul 18 03:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84699999999999998,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59499999999999997,"15min":0.56200000000000006},"min":{"1min":0.47599999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94299999999999995,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.60399999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.61199999999999999,"15min":0.57899999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.64900000000000002,"15min":0.60499999999999998},"min":{"1min":0.51500000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.45300000000000001}]},{"osd":5,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.53200000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0449999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66200000000000003,"15min":0.65900000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91100000000000003,"5min":1.827,"15min":1.827},"last":0.70899999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.58899999999999997,"15min":0.56699999999999995},"min":{"1min":0.39200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.41199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85599999999999998,"5min":0.88,"15min":0.88},"last":0.47599999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.79900000000000004,"15min":0.78700000000000003},"min":{"1min":0.55900000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.66100000000000003}]},{"osd":8,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.49399999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.51600000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.496}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]}]},{"osd":4,"up_from":32,"seq":137438953512,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":872448,"data_stored":417238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53200000000000003,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.54700000000000004,"15min":0.50700000000000001},"min":{"1min":0.48099999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89500000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.502}]},{"osd":1,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.39900000000000002,"15min":0.38600000000000001},"min":{"1min":0.32800000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.57299999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.40600000000000003,"15min":0.38100000000000001},"min":{"1min":0.32500000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65900000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.52800000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.434,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.503},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.47199999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85599999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.48399999999999999}]},{"osd":3,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.58499999999999996,"15min":0.54900000000000004},"min":{"1min":0.39800000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.57599999999999996,"15min":0.54800000000000004},"min":{"1min":0.41599999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64500000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61099999999999999,"15min":0.60799999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.45200000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84499999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.50700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.72099999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.434,"5min":0.38,"15min":0.38},"max":{"1min":0.89900000000000002,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.622},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.40699999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88600000000000001,"5min":1.121,"15min":1.121},"last":0.68300000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.82299999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.83899999999999997}]},{"osd":10,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":5,"up_from":37,"seq":158913789989,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1456,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1490944,"data_stored":1009012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.38200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.88900000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.52100000000000002,"15min":0.50800000000000001},"min":{"1min":0.44400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.71299999999999997,"5min":0.745,"15min":0.745},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.53800000000000003,"15min":0.52200000000000002},"min":{"1min":0.38700000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.38900000000000001,"15min":0.378},"min":{"1min":0.32200000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.41899999999999998,"15min":0.41099999999999998},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.45600000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56299999999999994,"15min":0.55500000000000005},"min":{"1min":0.46400000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.44600000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.70299999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.47799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.57199999999999995,"15min":0.55700000000000005},"min":{"1min":0.45700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.59199999999999997}]},{"osd":6,"last update":"Tue Jul 18 03:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.621,"15min":0.61199999999999999},"min":{"1min":0.51700000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63200000000000001,"15min":0.622},"min":{"1min":0.52400000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.81200000000000006}]},{"osd":7,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70999999999999996,"15min":0.70799999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.77500000000000002}]},{"osd":8,"last update":"Tue Jul 18 03:28:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.53300000000000003}]},{"osd":9,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.90400000000000003}]},{"osd":10,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.85099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:29:35.806 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-18T03:29:35.807 INFO:tasks.cephadm:Setting up client nodes... 2023-07-18T03:29:35.808 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-07-18T03:29:36.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:35 smithi134 ceph-mon[98216]: pgmap v252: 97 pgs: 97 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 678 B/s rd, 4.8 KiB/s wr, 13 op/s 2023-07-18T03:29:36.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:35 smithi134 ceph-mon[98216]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:29:36.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:35 smithi134 ceph-mon[98216]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:29:36.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:35 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] up:active 2023-07-18T03:29:36.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:35 smithi134 ceph-mon[98216]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-07-18T03:29:36.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:35 smithi134 ceph-mon[98216]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-18T03:29:36.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:35 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:29:36.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:35 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2063035372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:36.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:35 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1173512477' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:36.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:35 smithi064 ceph-mon[98059]: pgmap v252: 97 pgs: 97 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 678 B/s rd, 4.8 KiB/s wr, 13 op/s 2023-07-18T03:29:36.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:35 smithi064 ceph-mon[98059]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:29:36.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:35 smithi064 ceph-mon[98059]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:29:36.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:35 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] up:active 2023-07-18T03:29:36.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:35 smithi064 ceph-mon[98059]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-07-18T03:29:36.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:35 smithi064 ceph-mon[98059]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-18T03:29:36.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:35 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:29:36.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:35 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2063035372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:29:36.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:35 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1173512477' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:36.394 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:36.600 INFO:journalctl@ceph.mds.b.smithi064.stdout:Jul 18 03:29:36 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-b[123417]: 2023-07-18T03:29:36.354+0000 7fbb06327700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2023-07-18T03:29:36.601 INFO:journalctl@ceph.mds.b.smithi064.stdout:Jul 18 03:29:36 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-b[123417]: 2023-07-18T03:29:36.354+0000 7fbb06327700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-07-18T03:29:36.889 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:36 smithi040 ceph-mon[96439]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:29:36.939 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:36 smithi040 ceph-mon[96439]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:29:37.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:36 smithi134 ceph-mon[98216]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:29:37.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:36 smithi134 ceph-mon[98216]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:29:37.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:36 smithi064 ceph-mon[98059]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:29:37.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:36 smithi064 ceph-mon[98059]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:29:37.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.540+0000 7f0a35d9d700 1 -- 172.21.15.40:0/3516876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a30100640 msgr2=0x7f0a30106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:37.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.540+0000 7f0a35d9d700 1 --2- 172.21.15.40:0/3516876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a30100640 0x7f0a30106f00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800b0a0 tx=0x7f0a2800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:37.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.540+0000 7f0a35d9d700 1 -- 172.21.15.40:0/3516876 shutdown_connections 2023-07-18T03:29:37.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.540+0000 7f0a35d9d700 1 --2- 172.21.15.40:0/3516876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a30100640 0x7f0a30106f00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:37.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.540+0000 7f0a35d9d700 1 --2- 172.21.15.40:0/3516876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a300feab0 0x7f0a300feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:37.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.540+0000 7f0a35d9d700 1 --2- 172.21.15.40:0/3516876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a300ffcb0 0x7f0a30100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:37.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.540+0000 7f0a35d9d700 1 -- 172.21.15.40:0/3516876 >> 172.21.15.40:0/3516876 conn(0x7f0a300fa0e0 msgr2=0x7f0a300fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:37.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.540+0000 7f0a35d9d700 1 -- 172.21.15.40:0/3516876 shutdown_connections 2023-07-18T03:29:37.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.540+0000 7f0a35d9d700 1 -- 172.21.15.40:0/3516876 wait complete. 2023-07-18T03:29:37.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.541+0000 7f0a35d9d700 1 Processor -- start 2023-07-18T03:29:37.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.541+0000 7f0a35d9d700 1 -- start start 2023-07-18T03:29:37.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.541+0000 7f0a35d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a300feab0 0x7f0a30068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:37.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.541+0000 7f0a35d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a300ffcb0 0x7f0a3019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:37.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.541+0000 7f0a35d9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a30100640 0x7f0a3019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:37.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.541+0000 7f0a35d9d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3019d5f0 con 0x7f0a300ffcb0 2023-07-18T03:29:37.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.541+0000 7f0a35d9d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3019d760 con 0x7f0a300feab0 2023-07-18T03:29:37.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.541+0000 7f0a35d9d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3019d920 con 0x7f0a30100640 2023-07-18T03:29:37.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.542+0000 7f0a2effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a300ffcb0 0x7f0a3019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:37.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.542+0000 7f0a2effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a300ffcb0 0x7f0a3019a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56334/0 (socket says 172.21.15.40:56334) 2023-07-18T03:29:37.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.542+0000 7f0a2effd700 1 -- 172.21.15.40:0/1384322662 learned_addr learned my addr 172.21.15.40:0/1384322662 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:37.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.542+0000 7f0a2ffff700 1 --2- 172.21.15.40:0/1384322662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a30100640 0x7f0a3019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:37.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.542+0000 7f0a2f7fe700 1 --2- 172.21.15.40:0/1384322662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a300feab0 0x7f0a30068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:37.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.542+0000 7f0a2effd700 1 -- 172.21.15.40:0/1384322662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a30100640 msgr2=0x7f0a3019ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:37.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.542+0000 7f0a2effd700 1 --2- 172.21.15.40:0/1384322662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a30100640 0x7f0a3019ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:37.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.542+0000 7f0a2effd700 1 -- 172.21.15.40:0/1384322662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a300feab0 msgr2=0x7f0a30068b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:37.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.542+0000 7f0a2effd700 1 --2- 172.21.15.40:0/1384322662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a300feab0 0x7f0a30068b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:37.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.542+0000 7f0a2effd700 1 -- 172.21.15.40:0/1384322662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a28009d00 con 0x7f0a300ffcb0 2023-07-18T03:29:37.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.542+0000 7f0a2f7fe700 1 --2- 172.21.15.40:0/1384322662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a300feab0 0x7f0a30068b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:29:37.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.542+0000 7f0a2effd700 1 --2- 172.21.15.40:0/1384322662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a300ffcb0 0x7f0a3019a800 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000b530 tx=0x7f0a2000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:37.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.542+0000 7f0a2cff9700 1 -- 172.21.15.40:0/1384322662 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a20011830 con 0x7f0a300ffcb0 2023-07-18T03:29:37.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.542+0000 7f0a2cff9700 1 -- 172.21.15.40:0/1384322662 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a20011e50 con 0x7f0a300ffcb0 2023-07-18T03:29:37.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.542+0000 7f0a35d9d700 1 -- 172.21.15.40:0/1384322662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3019dc00 con 0x7f0a300ffcb0 2023-07-18T03:29:37.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.542+0000 7f0a2cff9700 1 -- 172.21.15.40:0/1384322662 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2000f2e0 con 0x7f0a300ffcb0 2023-07-18T03:29:37.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.542+0000 7f0a35d9d700 1 -- 172.21.15.40:0/1384322662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3019e100 con 0x7f0a300ffcb0 2023-07-18T03:29:37.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.543+0000 7f0a35d9d700 1 -- 172.21.15.40:0/1384322662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3010a390 con 0x7f0a300ffcb0 2023-07-18T03:29:37.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.544+0000 7f0a2cff9700 1 -- 172.21.15.40:0/1384322662 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0a20010380 con 0x7f0a300ffcb0 2023-07-18T03:29:37.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.545+0000 7f0a2cff9700 1 --2- 172.21.15.40:0/1384322662 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0a180a11e0 0x7f0a180a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:37.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.545+0000 7f0a2f7fe700 1 --2- 172.21.15.40:0/1384322662 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0a180a11e0 0x7f0a180a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:37.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.545+0000 7f0a2cff9700 1 -- 172.21.15.40:0/1384322662 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0a200cc420 con 0x7f0a300ffcb0 2023-07-18T03:29:37.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.546+0000 7f0a2f7fe700 1 --2- 172.21.15.40:0/1384322662 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0a180a11e0 0x7f0a180a3690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0a300ffb10 tx=0x7f0a24008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:37.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.550+0000 7f0a2cff9700 1 -- 172.21.15.40:0/1384322662 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0a20095bd0 con 0x7f0a300ffcb0 2023-07-18T03:29:37.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.830+0000 7f0a35d9d700 1 -- 172.21.15.40:0/1384322662 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f0a301059d0 con 0x7f0a300ffcb0 2023-07-18T03:29:37.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.835+0000 7f0a2cff9700 1 -- 172.21.15.40:0/1384322662 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f0a200991f0 con 0x7f0a300ffcb0 2023-07-18T03:29:37.831 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2023-07-18T03:29:37.831 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAhB7ZkYc6RMRAAUL1vAfNR0XTOD2nMg81IHA== 2023-07-18T03:29:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.837+0000 7f0a35d9d700 1 -- 172.21.15.40:0/1384322662 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0a180a11e0 msgr2=0x7f0a180a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.837+0000 7f0a35d9d700 1 --2- 172.21.15.40:0/1384322662 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0a180a11e0 0x7f0a180a3690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0a300ffb10 tx=0x7f0a24008040 comp rx=0 tx=0).stop 2023-07-18T03:29:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.837+0000 7f0a35d9d700 1 -- 172.21.15.40:0/1384322662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a300ffcb0 msgr2=0x7f0a3019a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.837+0000 7f0a35d9d700 1 --2- 172.21.15.40:0/1384322662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a300ffcb0 0x7f0a3019a800 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000b530 tx=0x7f0a2000b840 comp rx=0 tx=0).stop 2023-07-18T03:29:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.837+0000 7f0a35d9d700 1 -- 172.21.15.40:0/1384322662 shutdown_connections 2023-07-18T03:29:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.837+0000 7f0a35d9d700 1 --2- 172.21.15.40:0/1384322662 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0a180a11e0 0x7f0a180a3690 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.837+0000 7f0a35d9d700 1 --2- 172.21.15.40:0/1384322662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a30100640 0x7f0a3019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.837+0000 7f0a35d9d700 1 --2- 172.21.15.40:0/1384322662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a300feab0 0x7f0a30068b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:37.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.837+0000 7f0a35d9d700 1 --2- 172.21.15.40:0/1384322662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a300ffcb0 0x7f0a3019a800 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.837+0000 7f0a35d9d700 1 -- 172.21.15.40:0/1384322662 >> 172.21.15.40:0/1384322662 conn(0x7f0a300fa0e0 msgr2=0x7f0a300fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.837+0000 7f0a35d9d700 1 -- 172.21.15.40:0/1384322662 shutdown_connections 2023-07-18T03:29:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:37.837+0000 7f0a35d9d700 1 -- 172.21.15.40:0/1384322662 wait complete. 2023-07-18T03:29:37.972 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:37 smithi040 ceph-mon[96439]: pgmap v253: 97 pgs: 97 active+clean; 616 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.7 KiB/s wr, 21 op/s 2023-07-18T03:29:38.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:37 smithi134 ceph-mon[98216]: pgmap v253: 97 pgs: 97 active+clean; 616 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.7 KiB/s wr, 21 op/s 2023-07-18T03:29:38.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:37 smithi064 ceph-mon[98059]: pgmap v253: 97 pgs: 97 active+clean; 616 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.7 KiB/s wr, 21 op/s 2023-07-18T03:29:38.156 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:29:38.157 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-07-18T03:29:38.157 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-18T03:29:38.199 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-18T03:29:38.200 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-07-18T03:29:38.200 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph mgr dump --format=json 2023-07-18T03:29:38.796 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:38.995 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:38 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1384322662' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-18T03:29:38.995 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:38 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1384322662' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-18T03:29:39.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:38 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1384322662' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-18T03:29:39.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:38 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1384322662' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-18T03:29:39.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:38 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1384322662' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-18T03:29:39.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:38 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1384322662' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-18T03:29:39.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.973+0000 7ff979eee700 1 -- 172.21.15.40:0/3671669622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9740ff460 msgr2=0x7ff97410b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:39.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.973+0000 7ff979eee700 1 --2- 172.21.15.40:0/3671669622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9740ff460 0x7ff97410b8f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00b0a0 tx=0x7ff96c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:39.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.973+0000 7ff979eee700 1 -- 172.21.15.40:0/3671669622 shutdown_connections 2023-07-18T03:29:39.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.973+0000 7ff979eee700 1 --2- 172.21.15.40:0/3671669622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9740ff460 0x7ff97410b8f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:39.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.973+0000 7ff979eee700 1 --2- 172.21.15.40:0/3671669622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff974104ab0 0x7ff974104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:39.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.973+0000 7ff979eee700 1 --2- 172.21.15.40:0/3671669622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9740feab0 0x7ff9740fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:39.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.973+0000 7ff979eee700 1 -- 172.21.15.40:0/3671669622 >> 172.21.15.40:0/3671669622 conn(0x7ff974075b70 msgr2=0x7ff974075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:39.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.973+0000 7ff979eee700 1 -- 172.21.15.40:0/3671669622 shutdown_connections 2023-07-18T03:29:39.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.973+0000 7ff979eee700 1 -- 172.21.15.40:0/3671669622 wait complete. 2023-07-18T03:29:39.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.974+0000 7ff979eee700 1 Processor -- start 2023-07-18T03:29:39.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.974+0000 7ff979eee700 1 -- start start 2023-07-18T03:29:39.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.974+0000 7ff979eee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9740feab0 0x7ff97419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:39.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.974+0000 7ff979eee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9740ff460 0x7ff97419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:39.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.974+0000 7ff979eee700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff974104ab0 0x7ff974195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.974+0000 7ff979eee700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff974196150 con 0x7ff9740ff460 2023-07-18T03:29:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.974+0000 7ff979eee700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9741962c0 con 0x7ff9740feab0 2023-07-18T03:29:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.974+0000 7ff979eee700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff974196480 con 0x7ff974104ab0 2023-07-18T03:29:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.974+0000 7ff972ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9740ff460 0x7ff97419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.974+0000 7ff972ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9740ff460 0x7ff97419bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56348/0 (socket says 172.21.15.40:56348) 2023-07-18T03:29:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.974+0000 7ff972ffd700 1 -- 172.21.15.40:0/3035183208 learned_addr learned my addr 172.21.15.40:0/3035183208 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.974+0000 7ff9737fe700 1 --2- 172.21.15.40:0/3035183208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9740feab0 0x7ff97419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.975+0000 7ff973fff700 1 --2- 172.21.15.40:0/3035183208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff974104ab0 0x7ff974195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.975+0000 7ff972ffd700 1 -- 172.21.15.40:0/3035183208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff974104ab0 msgr2=0x7ff974195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.975+0000 7ff972ffd700 1 --2- 172.21.15.40:0/3035183208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff974104ab0 0x7ff974195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.975+0000 7ff972ffd700 1 -- 172.21.15.40:0/3035183208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9740feab0 msgr2=0x7ff97419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.975+0000 7ff972ffd700 1 --2- 172.21.15.40:0/3035183208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9740feab0 0x7ff97419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.975+0000 7ff972ffd700 1 -- 172.21.15.40:0/3035183208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c009d00 con 0x7ff9740ff460 2023-07-18T03:29:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.975+0000 7ff972ffd700 1 --2- 172.21.15.40:0/3035183208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9740ff460 0x7ff97419bfa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff96800d950 tx=0x7ff96800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.975+0000 7ff970ff9700 1 -- 172.21.15.40:0/3035183208 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968009940 con 0x7ff9740ff460 2023-07-18T03:29:39.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.975+0000 7ff970ff9700 1 -- 172.21.15.40:0/3035183208 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff968010460 con 0x7ff9740ff460 2023-07-18T03:29:39.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.975+0000 7ff979eee700 1 -- 172.21.15.40:0/3035183208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff974196760 con 0x7ff9740ff460 2023-07-18T03:29:39.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.975+0000 7ff970ff9700 1 -- 172.21.15.40:0/3035183208 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96800f620 con 0x7ff9740ff460 2023-07-18T03:29:39.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.975+0000 7ff979eee700 1 -- 172.21.15.40:0/3035183208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9741a2a50 con 0x7ff9740ff460 2023-07-18T03:29:39.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.976+0000 7ff979eee700 1 -- 172.21.15.40:0/3035183208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff97404e210 con 0x7ff9740ff460 2023-07-18T03:29:39.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.977+0000 7ff970ff9700 1 -- 172.21.15.40:0/3035183208 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff9680108d0 con 0x7ff9740ff460 2023-07-18T03:29:39.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.978+0000 7ff970ff9700 1 --2- 172.21.15.40:0/3035183208 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff95c0a1260 0x7ff95c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:39.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.978+0000 7ff9737fe700 1 --2- 172.21.15.40:0/3035183208 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff95c0a1260 0x7ff95c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:39.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.978+0000 7ff970ff9700 1 -- 172.21.15.40:0/3035183208 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff9680cc7f0 con 0x7ff9740ff460 2023-07-18T03:29:39.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.978+0000 7ff9737fe700 1 --2- 172.21.15.40:0/3035183208 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff95c0a1260 0x7ff95c0a3710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff9740ffbf0 tx=0x7ff96400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:39.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:39.981+0000 7ff970ff9700 1 -- 172.21.15.40:0/3035183208 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff968095e90 con 0x7ff9740ff460 2023-07-18T03:29:40.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:39 smithi134 ceph-mon[98216]: pgmap v254: 97 pgs: 97 active+clean; 619 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 10 KiB/s wr, 27 op/s 2023-07-18T03:29:40.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:39 smithi040 ceph-mon[96439]: pgmap v254: 97 pgs: 97 active+clean; 619 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 10 KiB/s wr, 27 op/s 2023-07-18T03:29:40.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:39 smithi064 ceph-mon[98059]: pgmap v254: 97 pgs: 97 active+clean; 619 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 10 KiB/s wr, 27 op/s 2023-07-18T03:29:40.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:40.237+0000 7ff979eee700 1 -- 172.21.15.40:0/3035183208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff9741a2d70 con 0x7ff9740ff460 2023-07-18T03:29:40.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:40.242+0000 7ff970ff9700 1 -- 172.21.15.40:0/3035183208 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260125 (secure 0 0 0) 0x7ff968020090 con 0x7ff9740ff460 2023-07-18T03:29:40.241 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:40.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:40.245+0000 7ff979eee700 1 -- 172.21.15.40:0/3035183208 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff95c0a1260 msgr2=0x7ff95c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:40.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:40.245+0000 7ff979eee700 1 --2- 172.21.15.40:0/3035183208 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff95c0a1260 0x7ff95c0a3710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff9740ffbf0 tx=0x7ff96400a380 comp rx=0 tx=0).stop 2023-07-18T03:29:40.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:40.245+0000 7ff979eee700 1 -- 172.21.15.40:0/3035183208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9740ff460 msgr2=0x7ff97419bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:40.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:40.245+0000 7ff979eee700 1 --2- 172.21.15.40:0/3035183208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9740ff460 0x7ff97419bfa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff96800d950 tx=0x7ff96800dc60 comp rx=0 tx=0).stop 2023-07-18T03:29:40.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:40.245+0000 7ff979eee700 1 -- 172.21.15.40:0/3035183208 shutdown_connections 2023-07-18T03:29:40.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:40.245+0000 7ff979eee700 1 --2- 172.21.15.40:0/3035183208 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff95c0a1260 0x7ff95c0a3710 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:40.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:40.245+0000 7ff979eee700 1 --2- 172.21.15.40:0/3035183208 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff974104ab0 0x7ff974195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:40.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:40.245+0000 7ff979eee700 1 --2- 172.21.15.40:0/3035183208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9740feab0 0x7ff97419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:40.245+0000 7ff979eee700 1 --2- 172.21.15.40:0/3035183208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9740ff460 0x7ff97419bfa0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:40.245+0000 7ff979eee700 1 -- 172.21.15.40:0/3035183208 >> 172.21.15.40:0/3035183208 conn(0x7ff974075b70 msgr2=0x7ff974067fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:40.245+0000 7ff979eee700 1 -- 172.21.15.40:0/3035183208 shutdown_connections 2023-07-18T03:29:40.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:40.245+0000 7ff979eee700 1 -- 172.21.15.40:0/3035183208 wait complete. 2023-07-18T03:29:40.542 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3612011194},{"type":"v1","addr":"172.21.15.40:6801","nonce":3612011194}]},"active_addr":"172.21.15.40:6801/3612011194","active_change":"2023-07-18T03:23:31.734747+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.40:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":4139476041}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":755430971}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":3730969862}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":3309852071}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":2794802705}]}]} 2023-07-18T03:29:40.550 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-07-18T03:29:40.550 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-07-18T03:29:40.550 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-18T03:29:40.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:40 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3035183208' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-18T03:29:41.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:40 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3035183208' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-18T03:29:41.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:40 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3035183208' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-18T03:29:41.114 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:41.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:41 smithi040 ceph-mon[96439]: pgmap v255: 97 pgs: 97 active+clean; 619 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.9 KiB/s wr, 24 op/s 2023-07-18T03:29:42.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:41 smithi134 ceph-mon[98216]: pgmap v255: 97 pgs: 97 active+clean; 619 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.9 KiB/s wr, 24 op/s 2023-07-18T03:29:42.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:41 smithi064 ceph-mon[98059]: pgmap v255: 97 pgs: 97 active+clean; 619 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.9 KiB/s wr, 24 op/s 2023-07-18T03:29:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.235+0000 7f6beea63700 1 -- 172.21.15.40:0/404519743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6be80feab0 msgr2=0x7f6be80fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.235+0000 7f6beea63700 1 --2- 172.21.15.40:0/404519743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6be80feab0 0x7f6be80fef20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009990 tx=0x7f6bd0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.235+0000 7f6beea63700 1 -- 172.21.15.40:0/404519743 shutdown_connections 2023-07-18T03:29:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.235+0000 7f6beea63700 1 --2- 172.21.15.40:0/404519743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6be80ff460 0x7f6be810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.235+0000 7f6beea63700 1 --2- 172.21.15.40:0/404519743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6be8104ab0 0x7f6be8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.235+0000 7f6beea63700 1 --2- 172.21.15.40:0/404519743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6be80feab0 0x7f6be80fef20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.235+0000 7f6beea63700 1 -- 172.21.15.40:0/404519743 >> 172.21.15.40:0/404519743 conn(0x7f6be8075b70 msgr2=0x7f6be8075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.235+0000 7f6beea63700 1 -- 172.21.15.40:0/404519743 shutdown_connections 2023-07-18T03:29:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.235+0000 7f6beea63700 1 -- 172.21.15.40:0/404519743 wait complete. 2023-07-18T03:29:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.236+0000 7f6beea63700 1 Processor -- start 2023-07-18T03:29:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.236+0000 7f6beea63700 1 -- start start 2023-07-18T03:29:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.236+0000 7f6beea63700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6be80feab0 0x7f6be819b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.236+0000 7f6beea63700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6be80ff460 0x7f6be819bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:42.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.236+0000 7f6beea63700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6be8104ab0 0x7f6be8195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:42.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.236+0000 7f6beea63700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be81960a0 con 0x7f6be8104ab0 2023-07-18T03:29:42.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.236+0000 7f6beea63700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be8196210 con 0x7f6be80ff460 2023-07-18T03:29:42.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.236+0000 7f6beea63700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be81963d0 con 0x7f6be80feab0 2023-07-18T03:29:42.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.236+0000 7f6bed000700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6be8104ab0 0x7f6be8195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:42.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.236+0000 7f6bed000700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6be8104ab0 0x7f6be8195a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56364/0 (socket says 172.21.15.40:56364) 2023-07-18T03:29:42.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.236+0000 7f6bed000700 1 -- 172.21.15.40:0/1977613085 learned_addr learned my addr 172.21.15.40:0/1977613085 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:42.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.236+0000 7f6be77fe700 1 --2- 172.21.15.40:0/1977613085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6be80ff460 0x7f6be819bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:42.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.237+0000 7f6be7fff700 1 --2- 172.21.15.40:0/1977613085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6be80feab0 0x7f6be819b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:42.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.237+0000 7f6bed000700 1 -- 172.21.15.40:0/1977613085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6be80feab0 msgr2=0x7f6be819b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:42.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.237+0000 7f6bed000700 1 --2- 172.21.15.40:0/1977613085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6be80feab0 0x7f6be819b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:42.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.237+0000 7f6bed000700 1 -- 172.21.15.40:0/1977613085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6be80ff460 msgr2=0x7f6be819bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:42.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.237+0000 7f6bed000700 1 --2- 172.21.15.40:0/1977613085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6be80ff460 0x7f6be819bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:42.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.237+0000 7f6bed000700 1 -- 172.21.15.40:0/1977613085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd00095f0 con 0x7f6be8104ab0 2023-07-18T03:29:42.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.237+0000 7f6be7fff700 1 --2- 172.21.15.40:0/1977613085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6be80feab0 0x7f6be819b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:29:42.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.237+0000 7f6bed000700 1 --2- 172.21.15.40:0/1977613085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6be8104ab0 0x7f6be8195a30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6be0007f10 tx=0x7f6be000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.237+0000 7f6be57fa700 1 -- 172.21.15.40:0/1977613085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be0017070 con 0x7f6be8104ab0 2023-07-18T03:29:42.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.237+0000 7f6beea63700 1 -- 172.21.15.40:0/1977613085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be81966b0 con 0x7f6be8104ab0 2023-07-18T03:29:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.237+0000 7f6be57fa700 1 -- 172.21.15.40:0/1977613085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6be000f040 con 0x7f6be8104ab0 2023-07-18T03:29:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.237+0000 7f6be57fa700 1 -- 172.21.15.40:0/1977613085 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be0005740 con 0x7f6be8104ab0 2023-07-18T03:29:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.237+0000 7f6beea63700 1 -- 172.21.15.40:0/1977613085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be81a29a0 con 0x7f6be8104ab0 2023-07-18T03:29:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.238+0000 7f6beea63700 1 -- 172.21.15.40:0/1977613085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be804e210 con 0x7f6be8104ab0 2023-07-18T03:29:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.239+0000 7f6be57fa700 1 -- 172.21.15.40:0/1977613085 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6be000dc60 con 0x7f6be8104ab0 2023-07-18T03:29:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.240+0000 7f6be57fa700 1 --2- 172.21.15.40:0/1977613085 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6bd40a11e0 0x7f6bd40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:42.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.240+0000 7f6be7fff700 1 --2- 172.21.15.40:0/1977613085 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6bd40a11e0 0x7f6bd40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.240+0000 7f6be57fa700 1 -- 172.21.15.40:0/1977613085 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6be00cc150 con 0x7f6be8104ab0 2023-07-18T03:29:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.240+0000 7f6be7fff700 1 --2- 172.21.15.40:0/1977613085 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6bd40a11e0 0x7f6bd40a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8005950 tx=0x7f6bd80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:42.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.244+0000 7f6be57fa700 1 -- 172.21.15.40:0/1977613085 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6be00957f0 con 0x7f6be8104ab0 2023-07-18T03:29:42.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.446+0000 7f6beea63700 1 -- 172.21.15.40:0/1977613085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6be81a2b90 con 0x7f6be8104ab0 2023-07-18T03:29:42.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.447+0000 7f6be57fa700 1 -- 172.21.15.40:0/1977613085 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22071 (secure 0 0 0) 0x7f6be0098e10 con 0x7f6be8104ab0 2023-07-18T03:29:42.444 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:42.444 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":81,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:29:28.081282+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:29:42.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.449+0000 7f6beea63700 1 -- 172.21.15.40:0/1977613085 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6bd40a11e0 msgr2=0x7f6bd40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:42.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.449+0000 7f6beea63700 1 --2- 172.21.15.40:0/1977613085 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6bd40a11e0 0x7f6bd40a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8005950 tx=0x7f6bd80058e0 comp rx=0 tx=0).stop 2023-07-18T03:29:42.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.449+0000 7f6beea63700 1 -- 172.21.15.40:0/1977613085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6be8104ab0 msgr2=0x7f6be8195a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:42.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.449+0000 7f6beea63700 1 --2- 172.21.15.40:0/1977613085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6be8104ab0 0x7f6be8195a30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6be0007f10 tx=0x7f6be000d3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:42.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.450+0000 7f6beea63700 1 -- 172.21.15.40:0/1977613085 shutdown_connections 2023-07-18T03:29:42.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.450+0000 7f6beea63700 1 --2- 172.21.15.40:0/1977613085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6be8104ab0 0x7f6be8195a30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:42.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.450+0000 7f6beea63700 1 --2- 172.21.15.40:0/1977613085 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6bd40a11e0 0x7f6bd40a3690 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:42.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.450+0000 7f6beea63700 1 --2- 172.21.15.40:0/1977613085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6be80feab0 0x7f6be819b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:42.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.450+0000 7f6beea63700 1 --2- 172.21.15.40:0/1977613085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6be80ff460 0x7f6be819bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:42.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.450+0000 7f6beea63700 1 -- 172.21.15.40:0/1977613085 >> 172.21.15.40:0/1977613085 conn(0x7f6be8075b70 msgr2=0x7f6be80fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:42.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.450+0000 7f6beea63700 1 -- 172.21.15.40:0/1977613085 shutdown_connections 2023-07-18T03:29:42.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:42.450+0000 7f6beea63700 1 -- 172.21.15.40:0/1977613085 wait complete. 2023-07-18T03:29:42.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:42 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1977613085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:42.876 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-07-18T03:29:42.877 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-18T03:29:43.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:42 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1977613085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:43.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:42 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1977613085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:43.436 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:43.921 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:43 smithi040 ceph-mon[96439]: pgmap v256: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 KiB/s wr, 27 op/s 2023-07-18T03:29:44.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:43 smithi134 ceph-mon[98216]: pgmap v256: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 KiB/s wr, 27 op/s 2023-07-18T03:29:44.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:43 smithi064 ceph-mon[98059]: pgmap v256: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.2 KiB/s wr, 27 op/s 2023-07-18T03:29:44.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.521+0000 7f47771ec700 1 -- 172.21.15.40:0/121436160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47700feab0 msgr2=0x7f47700fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:44.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.521+0000 7f47771ec700 1 --2- 172.21.15.40:0/121436160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47700feab0 0x7f47700fef20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f475c009990 tx=0x7f475c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:44.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.521+0000 7f47771ec700 1 -- 172.21.15.40:0/121436160 shutdown_connections 2023-07-18T03:29:44.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.521+0000 7f47771ec700 1 --2- 172.21.15.40:0/121436160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47700ff460 0x7f477010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:44.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.521+0000 7f47771ec700 1 --2- 172.21.15.40:0/121436160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4770104ab0 0x7f4770104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:44.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.521+0000 7f47771ec700 1 --2- 172.21.15.40:0/121436160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47700feab0 0x7f47700fef20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:44.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.521+0000 7f47771ec700 1 -- 172.21.15.40:0/121436160 >> 172.21.15.40:0/121436160 conn(0x7f4770075b70 msgr2=0x7f4770075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:44.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.521+0000 7f47771ec700 1 -- 172.21.15.40:0/121436160 shutdown_connections 2023-07-18T03:29:44.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.521+0000 7f47771ec700 1 -- 172.21.15.40:0/121436160 wait complete. 2023-07-18T03:29:44.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.521+0000 7f47771ec700 1 Processor -- start 2023-07-18T03:29:44.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.521+0000 7f47771ec700 1 -- start start 2023-07-18T03:29:44.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f47771ec700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47700feab0 0x7f4770195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:44.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f47771ec700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47700ff460 0x7f4770196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:44.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f47771ec700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4770104ab0 0x7f47701995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:44.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f47771ec700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4770199bc0 con 0x7f4770104ab0 2023-07-18T03:29:44.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f47771ec700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4770199d30 con 0x7f47700feab0 2023-07-18T03:29:44.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f47771ec700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4770199ef0 con 0x7f47700ff460 2023-07-18T03:29:44.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f4775789700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4770104ab0 0x7f47701995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f4775789700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4770104ab0 0x7f47701995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56378/0 (socket says 172.21.15.40:56378) 2023-07-18T03:29:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f4775789700 1 -- 172.21.15.40:0/4192693243 learned_addr learned my addr 172.21.15.40:0/4192693243 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f4774f88700 1 --2- 172.21.15.40:0/4192693243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47700feab0 0x7f4770195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f476ffff700 1 --2- 172.21.15.40:0/4192693243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47700ff460 0x7f4770196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f4775789700 1 -- 172.21.15.40:0/4192693243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47700ff460 msgr2=0x7f4770196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f4775789700 1 --2- 172.21.15.40:0/4192693243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47700ff460 0x7f4770196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:44.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f4775789700 1 -- 172.21.15.40:0/4192693243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47700feab0 msgr2=0x7f4770195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:44.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f4775789700 1 --2- 172.21.15.40:0/4192693243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47700feab0 0x7f4770195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:44.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f4775789700 1 -- 172.21.15.40:0/4192693243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c0095f0 con 0x7f4770104ab0 2023-07-18T03:29:44.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f4774f88700 1 --2- 172.21.15.40:0/4192693243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47700feab0 0x7f4770195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:29:44.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f476ffff700 1 --2- 172.21.15.40:0/4192693243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47700ff460 0x7f4770196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:29:44.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f4775789700 1 --2- 172.21.15.40:0/4192693243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4770104ab0 0x7f47701995c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f476800b9e0 tx=0x7f476800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:44.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.522+0000 7f476dffb700 1 -- 172.21.15.40:0/4192693243 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476800c950 con 0x7f4770104ab0 2023-07-18T03:29:44.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.523+0000 7f476dffb700 1 -- 172.21.15.40:0/4192693243 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4768014440 con 0x7f4770104ab0 2023-07-18T03:29:44.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.523+0000 7f47771ec700 1 -- 172.21.15.40:0/4192693243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477019a1d0 con 0x7f4770104ab0 2023-07-18T03:29:44.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.523+0000 7f476dffb700 1 -- 172.21.15.40:0/4192693243 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47680075d0 con 0x7f4770104ab0 2023-07-18T03:29:44.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.523+0000 7f47771ec700 1 -- 172.21.15.40:0/4192693243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47701a2b20 con 0x7f4770104ab0 2023-07-18T03:29:44.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.524+0000 7f47771ec700 1 -- 172.21.15.40:0/4192693243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477004e210 con 0x7f4770104ab0 2023-07-18T03:29:44.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.525+0000 7f476dffb700 1 -- 172.21.15.40:0/4192693243 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f47680145b0 con 0x7f4770104ab0 2023-07-18T03:29:44.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.525+0000 7f476dffb700 1 --2- 172.21.15.40:0/4192693243 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f47600a11e0 0x7f47600a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:44.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.525+0000 7f4774f88700 1 --2- 172.21.15.40:0/4192693243 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f47600a11e0 0x7f47600a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:44.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.525+0000 7f476dffb700 1 -- 172.21.15.40:0/4192693243 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4768006e40 con 0x7f4770104ab0 2023-07-18T03:29:44.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.526+0000 7f4774f88700 1 --2- 172.21.15.40:0/4192693243 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f47600a11e0 0x7f47600a3690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f47700ffbf0 tx=0x7f4764008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:44.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.529+0000 7f476dffb700 1 -- 172.21.15.40:0/4192693243 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f476804e790 con 0x7f4770104ab0 2023-07-18T03:29:44.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.728+0000 7f47771ec700 1 -- 172.21.15.40:0/4192693243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47701a2d10 con 0x7f4770104ab0 2023-07-18T03:29:44.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.730+0000 7f476dffb700 1 -- 172.21.15.40:0/4192693243 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22071 (secure 0 0 0) 0x7f4768095780 con 0x7f4770104ab0 2023-07-18T03:29:44.727 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:29:44.727 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":81,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:29:28.081282+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:29:44.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.732+0000 7f47771ec700 1 -- 172.21.15.40:0/4192693243 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f47600a11e0 msgr2=0x7f47600a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:44.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.732+0000 7f47771ec700 1 --2- 172.21.15.40:0/4192693243 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f47600a11e0 0x7f47600a3690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f47700ffbf0 tx=0x7f4764008040 comp rx=0 tx=0).stop 2023-07-18T03:29:44.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.732+0000 7f47771ec700 1 -- 172.21.15.40:0/4192693243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4770104ab0 msgr2=0x7f47701995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:44.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.732+0000 7f47771ec700 1 --2- 172.21.15.40:0/4192693243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4770104ab0 0x7f47701995c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f476800b9e0 tx=0x7f476800bda0 comp rx=0 tx=0).stop 2023-07-18T03:29:44.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.733+0000 7f47771ec700 1 -- 172.21.15.40:0/4192693243 shutdown_connections 2023-07-18T03:29:44.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.733+0000 7f47771ec700 1 --2- 172.21.15.40:0/4192693243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4770104ab0 0x7f47701995c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:44.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.733+0000 7f47771ec700 1 --2- 172.21.15.40:0/4192693243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47700feab0 0x7f4770195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:44.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.733+0000 7f47771ec700 1 --2- 172.21.15.40:0/4192693243 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f47600a11e0 0x7f47600a3690 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:44.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.733+0000 7f47771ec700 1 --2- 172.21.15.40:0/4192693243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47700ff460 0x7f4770196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:44.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.733+0000 7f47771ec700 1 -- 172.21.15.40:0/4192693243 >> 172.21.15.40:0/4192693243 conn(0x7f4770075b70 msgr2=0x7f4770105930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:44.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.733+0000 7f47771ec700 1 -- 172.21.15.40:0/4192693243 shutdown_connections 2023-07-18T03:29:44.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:44.733+0000 7f47771ec700 1 -- 172.21.15.40:0/4192693243 wait complete. 2023-07-18T03:29:45.097 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-07-18T03:29:45.098 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-07-18T03:29:45.098 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-07-18T03:29:45.099 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-07-18T03:29:45.099 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-07-18T03:29:45.100 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-07-18T03:29:45.100 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-07-18T03:29:45.101 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-07-18T03:29:45.101 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-07-18T03:29:45.102 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-07-18T03:29:45.102 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-07-18T03:29:45.103 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-07-18T03:29:46.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:45 smithi134 ceph-mon[98216]: pgmap v257: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 KiB/s wr, 24 op/s 2023-07-18T03:29:46.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:45 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/4192693243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:46.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:45 smithi040 ceph-mon[96439]: pgmap v257: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 KiB/s wr, 24 op/s 2023-07-18T03:29:46.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:45 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/4192693243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:46.096 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:46.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:45 smithi064 ceph-mon[98059]: pgmap v257: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.9 KiB/s wr, 24 op/s 2023-07-18T03:29:46.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:45 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/4192693243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:29:46.122 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:46.381 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:46.518 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:46.519 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:46.551 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:46.709 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:46.710 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:48.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:47 smithi134 ceph-mon[98216]: pgmap v258: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 22 op/s 2023-07-18T03:29:48.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:47 smithi040 ceph-mon[96439]: pgmap v258: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 22 op/s 2023-07-18T03:29:48.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:47 smithi064 ceph-mon[98059]: pgmap v258: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 22 op/s 2023-07-18T03:29:49.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:49 smithi040 ceph-mon[96439]: pgmap v259: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.7 KiB/s wr, 16 op/s 2023-07-18T03:29:50.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:49 smithi134 ceph-mon[98216]: pgmap v259: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.7 KiB/s wr, 16 op/s 2023-07-18T03:29:50.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:49 smithi064 ceph-mon[98059]: pgmap v259: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.7 KiB/s wr, 16 op/s 2023-07-18T03:29:50.124 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:50.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.808+0000 7f58c31f0700 1 -- 172.21.15.40:0/723608593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc0fee80 msgr2=0x7f58bc0ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.808+0000 7f58c31f0700 1 --2- 172.21.15.40:0/723608593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc0fee80 0x7f58bc0ff250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f58b00099c0 tx=0x7f58b0009cd0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.808+0000 7f58c31f0700 1 -- 172.21.15.40:0/723608593 shutdown_connections 2023-07-18T03:29:50.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.808+0000 7f58c31f0700 1 --2- 172.21.15.40:0/723608593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58bc100f80 0x7f58bc10ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.808+0000 7f58c31f0700 1 --2- 172.21.15.40:0/723608593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc0fee80 0x7f58bc0ff250 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.808+0000 7f58c31f0700 1 --2- 172.21.15.40:0/723608593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58bc1005d0 0x7f58bc100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.808+0000 7f58c31f0700 1 -- 172.21.15.40:0/723608593 >> 172.21.15.40:0/723608593 conn(0x7f58bc075b60 msgr2=0x7f58bc075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:50.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.808+0000 7f58c31f0700 1 -- 172.21.15.40:0/723608593 shutdown_connections 2023-07-18T03:29:50.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.808+0000 7f58c31f0700 1 -- 172.21.15.40:0/723608593 wait complete. 2023-07-18T03:29:50.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.809+0000 7f58c31f0700 1 Processor -- start 2023-07-18T03:29:50.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.809+0000 7f58c31f0700 1 -- start start 2023-07-18T03:29:50.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.809+0000 7f58c31f0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58bc0fee80 0x7f58bc196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.809+0000 7f58c31f0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc1005d0 0x7f58bc197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.809+0000 7f58c31f0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58bc100f80 0x7f58bc1a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.809+0000 7f58c31f0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc1a20f0 con 0x7f58bc1005d0 2023-07-18T03:29:50.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.809+0000 7f58c31f0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc1a2230 con 0x7f58bc100f80 2023-07-18T03:29:50.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.809+0000 7f58c31f0700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc1a2370 con 0x7f58bc0fee80 2023-07-18T03:29:50.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.810+0000 7f58bbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc1005d0 0x7f58bc197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.810+0000 7f58bbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc1005d0 0x7f58bc197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56388/0 (socket says 172.21.15.40:56388) 2023-07-18T03:29:50.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.810+0000 7f58bbfff700 1 -- 172.21.15.40:0/1171772351 learned_addr learned my addr 172.21.15.40:0/1171772351 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:50.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.810+0000 7f58c178d700 1 --2- 172.21.15.40:0/1171772351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58bc100f80 0x7f58bc1a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.810+0000 7f58c0f8c700 1 --2- 172.21.15.40:0/1171772351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58bc0fee80 0x7f58bc196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.810+0000 7f58bbfff700 1 -- 172.21.15.40:0/1171772351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58bc0fee80 msgr2=0x7f58bc196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.810+0000 7f58bbfff700 1 --2- 172.21.15.40:0/1171772351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58bc0fee80 0x7f58bc196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.810+0000 7f58bbfff700 1 -- 172.21.15.40:0/1171772351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58bc100f80 msgr2=0x7f58bc1a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.810+0000 7f58bbfff700 1 --2- 172.21.15.40:0/1171772351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58bc100f80 0x7f58bc1a1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.810+0000 7f58bbfff700 1 -- 172.21.15.40:0/1171772351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b00095f0 con 0x7f58bc1005d0 2023-07-18T03:29:50.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.810+0000 7f58c0f8c700 1 --2- 172.21.15.40:0/1171772351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58bc0fee80 0x7f58bc196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:29:50.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.810+0000 7f58bbfff700 1 --2- 172.21.15.40:0/1171772351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc1005d0 0x7f58bc197070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f58a800e9a0 tx=0x7f58a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:50.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.810+0000 7f58b9ffb700 1 -- 172.21.15.40:0/1171772351 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8009790 con 0x7f58bc1005d0 2023-07-18T03:29:50.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.810+0000 7f58c31f0700 1 -- 172.21.15.40:0/1171772351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc1a25d0 con 0x7f58bc1005d0 2023-07-18T03:29:50.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.810+0000 7f58b9ffb700 1 -- 172.21.15.40:0/1171772351 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58a800cbd0 con 0x7f58bc1005d0 2023-07-18T03:29:50.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.810+0000 7f58b9ffb700 1 -- 172.21.15.40:0/1171772351 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8018680 con 0x7f58bc1005d0 2023-07-18T03:29:50.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.810+0000 7f58c31f0700 1 -- 172.21.15.40:0/1171772351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc1a2af0 con 0x7f58bc1005d0 2023-07-18T03:29:50.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.811+0000 7f58c31f0700 1 -- 172.21.15.40:0/1171772351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f58bc04e210 con 0x7f58bc1005d0 2023-07-18T03:29:50.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.812+0000 7f58b9ffb700 1 -- 172.21.15.40:0/1171772351 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f58a800cd40 con 0x7f58bc1005d0 2023-07-18T03:29:50.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.813+0000 7f58b9ffb700 1 --2- 172.21.15.40:0/1171772351 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f58ac0a11e0 0x7f58ac0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.813+0000 7f58b9ffb700 1 -- 172.21.15.40:0/1171772351 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f58a8014070 con 0x7f58bc1005d0 2023-07-18T03:29:50.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.813+0000 7f58c0f8c700 1 --2- 172.21.15.40:0/1171772351 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f58ac0a11e0 0x7f58ac0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.813+0000 7f58b9ffb700 1 --2- 172.21.15.40:0/1171772351 >> [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956] conn(0x7f58ac0aa9b0 0x7f58ac0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.813+0000 7f58b9ffb700 1 -- 172.21.15.40:0/1171772351 --> [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f58ac0ad470 con 0x7f58ac0aa9b0 2023-07-18T03:29:50.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.813+0000 7f58b9ffb700 1 -- 172.21.15.40:0/1171772351 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58a80cd3d0 con 0x7f58bc1005d0 2023-07-18T03:29:50.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.814+0000 7f58c178d700 1 --2- 172.21.15.40:0/1171772351 >> [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956] conn(0x7f58ac0aa9b0 0x7f58ac0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.814+0000 7f58c0f8c700 1 --2- 172.21.15.40:0/1171772351 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f58ac0a11e0 0x7f58ac0a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f58b0000c00 tx=0x7f58b0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:50.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.814+0000 7f58c178d700 1 --2- 172.21.15.40:0/1171772351 >> [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956] conn(0x7f58ac0aa9b0 0x7f58ac0acdc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:50.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.815+0000 7f58b9ffb700 1 -- 172.21.15.40:0/1171772351 <== osd.5 v2:172.21.15.134:6808/3544269956 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f58ac0ad470 con 0x7f58ac0aa9b0 2023-07-18T03:29:50.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.831+0000 7f58c31f0700 1 -- 172.21.15.40:0/1171772351 --> [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f58bc10eda0 con 0x7f58ac0aa9b0 2023-07-18T03:29:50.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.832+0000 7f58b9ffb700 1 -- 172.21.15.40:0/1171772351 <== osd.5 v2:172.21.15.134:6808/3544269956 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f58b400f040 con 0x7f58ac0aa9b0 2023-07-18T03:29:50.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.832+0000 7f58c31f0700 1 -- 172.21.15.40:0/1171772351 >> [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956] conn(0x7f58ac0aa9b0 msgr2=0x7f58ac0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.832+0000 7f58c31f0700 1 --2- 172.21.15.40:0/1171772351 >> [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956] conn(0x7f58ac0aa9b0 0x7f58ac0acdc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.833+0000 7f58c31f0700 1 -- 172.21.15.40:0/1171772351 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f58ac0a11e0 msgr2=0x7f58ac0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.833+0000 7f58c31f0700 1 --2- 172.21.15.40:0/1171772351 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f58ac0a11e0 0x7f58ac0a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f58b0000c00 tx=0x7f58b0019040 comp rx=0 tx=0).stop 2023-07-18T03:29:50.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.833+0000 7f58c31f0700 1 -- 172.21.15.40:0/1171772351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc1005d0 msgr2=0x7f58bc197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.833+0000 7f58c31f0700 1 --2- 172.21.15.40:0/1171772351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc1005d0 0x7f58bc197070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f58a800e9a0 tx=0x7f58a800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.833+0000 7f58c178d700 1 -- 172.21.15.40:0/1171772351 reap_dead start 2023-07-18T03:29:50.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.833+0000 7f58c31f0700 1 -- 172.21.15.40:0/1171772351 shutdown_connections 2023-07-18T03:29:50.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.833+0000 7f58c31f0700 1 -- 172.21.15.40:0/1171772351 >> 172.21.15.40:0/1171772351 conn(0x7f58bc075b60 msgr2=0x7f58bc0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:50.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.833+0000 7f58c31f0700 1 -- 172.21.15.40:0/1171772351 shutdown_connections 2023-07-18T03:29:50.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:49.833+0000 7f58c31f0700 1 -- 172.21.15.40:0/1171772351 wait complete. 2023-07-18T03:29:50.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.261+0000 7ff3ea33d700 1 -- 172.21.15.40:0/3381523042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e410a140 msgr2=0x7ff3e410c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.261+0000 7ff3ea33d700 1 --2- 172.21.15.40:0/3381523042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e410a140 0x7ff3e410c580 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00b0a0 tx=0x7ff3dc00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.262+0000 7ff3ea33d700 1 -- 172.21.15.40:0/3381523042 shutdown_connections 2023-07-18T03:29:50.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.262+0000 7ff3ea33d700 1 --2- 172.21.15.40:0/3381523042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e410a140 0x7ff3e410c580 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.262+0000 7ff3ea33d700 1 --2- 172.21.15.40:0/3381523042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3e4100df0 0x7ff3e41011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.262+0000 7ff3ea33d700 1 --2- 172.21.15.40:0/3381523042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3e4101700 0x7ff3e4109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.262+0000 7ff3ea33d700 1 -- 172.21.15.40:0/3381523042 >> 172.21.15.40:0/3381523042 conn(0x7ff3e4075b70 msgr2=0x7ff3e4075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:50.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.262+0000 7ff3ea33d700 1 -- 172.21.15.40:0/3381523042 shutdown_connections 2023-07-18T03:29:50.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.262+0000 7ff3ea33d700 1 -- 172.21.15.40:0/3381523042 wait complete. 2023-07-18T03:29:50.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.262+0000 7ff3ea33d700 1 Processor -- start 2023-07-18T03:29:50.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.265+0000 7ff3ea33d700 1 -- start start 2023-07-18T03:29:50.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.265+0000 7ff3ea33d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3e4100df0 0x7ff3e4108a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.265+0000 7ff3ea33d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e4101700 0x7ff3e4108fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.265+0000 7ff3ea33d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3e410a140 0x7ff3e41b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.265+0000 7ff3ea33d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e4109770 con 0x7ff3e4101700 2023-07-18T03:29:50.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.265+0000 7ff3ea33d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e41b0ef0 con 0x7ff3e4100df0 2023-07-18T03:29:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.265+0000 7ff3ea33d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e41b10b0 con 0x7ff3e410a140 2023-07-18T03:29:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.265+0000 7ff3e37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e4101700 0x7ff3e4108fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.265+0000 7ff3e37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e4101700 0x7ff3e4108fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56402/0 (socket says 172.21.15.40:56402) 2023-07-18T03:29:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.265+0000 7ff3e37fe700 1 -- 172.21.15.40:0/3372953183 learned_addr learned my addr 172.21.15.40:0/3372953183 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.265+0000 7ff3e3fff700 1 --2- 172.21.15.40:0/3372953183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3e4100df0 0x7ff3e4108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.265+0000 7ff3e88da700 1 --2- 172.21.15.40:0/3372953183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3e410a140 0x7ff3e41b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.266+0000 7ff3e37fe700 1 -- 172.21.15.40:0/3372953183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3e410a140 msgr2=0x7ff3e41b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.266+0000 7ff3e37fe700 1 --2- 172.21.15.40:0/3372953183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3e410a140 0x7ff3e41b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.266+0000 7ff3e37fe700 1 -- 172.21.15.40:0/3372953183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3e4100df0 msgr2=0x7ff3e4108a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.266+0000 7ff3e37fe700 1 --2- 172.21.15.40:0/3372953183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3e4100df0 0x7ff3e4108a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.266+0000 7ff3e37fe700 1 -- 172.21.15.40:0/3372953183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3dc009d00 con 0x7ff3e4101700 2023-07-18T03:29:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.266+0000 7ff3e37fe700 1 --2- 172.21.15.40:0/3372953183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e4101700 0x7ff3e4108fd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800c9b0 tx=0x7ff3d800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:50.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.266+0000 7ff3e17fa700 1 -- 172.21.15.40:0/3372953183 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8007b00 con 0x7ff3e4101700 2023-07-18T03:29:50.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.266+0000 7ff3ea33d700 1 -- 172.21.15.40:0/3372953183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e41b1390 con 0x7ff3e4101700 2023-07-18T03:29:50.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.266+0000 7ff3e17fa700 1 -- 172.21.15.40:0/3372953183 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3d800ce50 con 0x7ff3e4101700 2023-07-18T03:29:50.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.266+0000 7ff3e17fa700 1 -- 172.21.15.40:0/3372953183 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8018660 con 0x7ff3e4101700 2023-07-18T03:29:50.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.266+0000 7ff3ea33d700 1 -- 172.21.15.40:0/3372953183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e41b18e0 con 0x7ff3e4101700 2023-07-18T03:29:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.267+0000 7ff3ea33d700 1 -- 172.21.15.40:0/3372953183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff3e410fcb0 con 0x7ff3e4101700 2023-07-18T03:29:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.270+0000 7ff3e17fa700 1 -- 172.21.15.40:0/3372953183 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff3d800f450 con 0x7ff3e4101700 2023-07-18T03:29:50.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.270+0000 7ff3e17fa700 1 --2- 172.21.15.40:0/3372953183 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff3cc0a11e0 0x7ff3cc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.270+0000 7ff3e17fa700 1 -- 172.21.15.40:0/3372953183 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff3d80cc980 con 0x7ff3e4101700 2023-07-18T03:29:50.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.271+0000 7ff3e3fff700 1 --2- 172.21.15.40:0/3372953183 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff3cc0a11e0 0x7ff3cc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.271+0000 7ff3e17fa700 1 --2- 172.21.15.40:0/3372953183 >> [v2:172.21.15.64:6816/589664358,v1:172.21.15.64:6817/589664358] conn(0x7ff3cc0aa9b0 0x7ff3cc0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.271+0000 7ff3e17fa700 1 -- 172.21.15.40:0/3372953183 --> [v2:172.21.15.64:6816/589664358,v1:172.21.15.64:6817/589664358] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3cc0ad470 con 0x7ff3cc0aa9b0 2023-07-18T03:29:50.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.271+0000 7ff3e17fa700 1 -- 172.21.15.40:0/3372953183 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3d80cce20 con 0x7ff3e4101700 2023-07-18T03:29:50.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.271+0000 7ff3e88da700 1 --2- 172.21.15.40:0/3372953183 >> [v2:172.21.15.64:6816/589664358,v1:172.21.15.64:6817/589664358] conn(0x7ff3cc0aa9b0 0x7ff3cc0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.271+0000 7ff3e3fff700 1 --2- 172.21.15.40:0/3372953183 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff3cc0a11e0 0x7ff3cc0a3690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4005950 tx=0x7ff3d400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:50.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.271+0000 7ff3e88da700 1 --2- 172.21.15.40:0/3372953183 >> [v2:172.21.15.64:6816/589664358,v1:172.21.15.64:6817/589664358] conn(0x7ff3cc0aa9b0 0x7ff3cc0acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:50.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.272+0000 7ff3e17fa700 1 -- 172.21.15.40:0/3372953183 <== osd.7 v2:172.21.15.64:6816/589664358 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff3cc0ad470 con 0x7ff3cc0aa9b0 2023-07-18T03:29:50.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.289+0000 7ff3ea33d700 1 -- 172.21.15.40:0/3372953183 --> [v2:172.21.15.64:6816/589664358,v1:172.21.15.64:6817/589664358] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff3e404e210 con 0x7ff3cc0aa9b0 2023-07-18T03:29:50.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.289+0000 7ff3e17fa700 1 -- 172.21.15.40:0/3372953183 <== osd.7 v2:172.21.15.64:6816/589664358 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff3e404e210 con 0x7ff3cc0aa9b0 2023-07-18T03:29:50.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.290+0000 7ff3ea33d700 1 -- 172.21.15.40:0/3372953183 >> [v2:172.21.15.64:6816/589664358,v1:172.21.15.64:6817/589664358] conn(0x7ff3cc0aa9b0 msgr2=0x7ff3cc0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.290+0000 7ff3ea33d700 1 --2- 172.21.15.40:0/3372953183 >> [v2:172.21.15.64:6816/589664358,v1:172.21.15.64:6817/589664358] conn(0x7ff3cc0aa9b0 0x7ff3cc0acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.290+0000 7ff3ea33d700 1 -- 172.21.15.40:0/3372953183 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff3cc0a11e0 msgr2=0x7ff3cc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.290+0000 7ff3ea33d700 1 --2- 172.21.15.40:0/3372953183 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff3cc0a11e0 0x7ff3cc0a3690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4005950 tx=0x7ff3d400a300 comp rx=0 tx=0).stop 2023-07-18T03:29:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.290+0000 7ff3ea33d700 1 -- 172.21.15.40:0/3372953183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e4101700 msgr2=0x7ff3e4108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.290+0000 7ff3ea33d700 1 --2- 172.21.15.40:0/3372953183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3e4101700 0x7ff3e4108fd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800c9b0 tx=0x7ff3d800ccc0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.290+0000 7ff3e88da700 1 -- 172.21.15.40:0/3372953183 reap_dead start 2023-07-18T03:29:50.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.290+0000 7ff3ea33d700 1 -- 172.21.15.40:0/3372953183 shutdown_connections 2023-07-18T03:29:50.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.290+0000 7ff3ea33d700 1 -- 172.21.15.40:0/3372953183 >> 172.21.15.40:0/3372953183 conn(0x7ff3e4075b70 msgr2=0x7ff3e40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:50.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.290+0000 7ff3ea33d700 1 -- 172.21.15.40:0/3372953183 shutdown_connections 2023-07-18T03:29:50.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.290+0000 7ff3ea33d700 1 -- 172.21.15.40:0/3372953183 wait complete. 2023-07-18T03:29:50.407 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:50.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.423+0000 7f48456e2700 1 -- 172.21.15.40:0/2166280838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4840100f80 msgr2=0x7f484010ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.423+0000 7f48456e2700 1 --2- 172.21.15.40:0/2166280838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4840100f80 0x7f484010ba10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f483800b0d0 tx=0x7f483800b3e0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.424+0000 7f48456e2700 1 -- 172.21.15.40:0/2166280838 shutdown_connections 2023-07-18T03:29:50.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.424+0000 7f48456e2700 1 --2- 172.21.15.40:0/2166280838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4840100f80 0x7f484010ba10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.424+0000 7f48456e2700 1 --2- 172.21.15.40:0/2166280838 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48400fee80 0x7f48400ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.424+0000 7f48456e2700 1 --2- 172.21.15.40:0/2166280838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48401005d0 0x7f4840100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.424+0000 7f48456e2700 1 -- 172.21.15.40:0/2166280838 >> 172.21.15.40:0/2166280838 conn(0x7f4840075b60 msgr2=0x7f4840075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:50.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.424+0000 7f48456e2700 1 -- 172.21.15.40:0/2166280838 shutdown_connections 2023-07-18T03:29:50.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.424+0000 7f48456e2700 1 -- 172.21.15.40:0/2166280838 wait complete. 2023-07-18T03:29:50.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.424+0000 7f48456e2700 1 Processor -- start 2023-07-18T03:29:50.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.424+0000 7f48456e2700 1 -- start start 2023-07-18T03:29:50.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f48456e2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48400fee80 0x7f4840105a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f48456e2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48401005d0 0x7f4840105f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f48456e2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4840100f80 0x7f48401064e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f48456e2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48401a4250 con 0x7f48401005d0 2023-07-18T03:29:50.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f48456e2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48401a43c0 con 0x7f4840100f80 2023-07-18T03:29:50.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f48456e2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48401a4580 con 0x7f48400fee80 2023-07-18T03:29:50.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f48367fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48401005d0 0x7f4840105f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f48367fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48401005d0 0x7f4840105f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56418/0 (socket says 172.21.15.40:56418) 2023-07-18T03:29:50.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f48367fc700 1 -- 172.21.15.40:0/2208224468 learned_addr learned my addr 172.21.15.40:0/2208224468 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:50.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f483f7fe700 1 --2- 172.21.15.40:0/2208224468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4840100f80 0x7f48401064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f483effd700 1 --2- 172.21.15.40:0/2208224468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48400fee80 0x7f4840105a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f48367fc700 1 -- 172.21.15.40:0/2208224468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48400fee80 msgr2=0x7f4840105a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f48367fc700 1 --2- 172.21.15.40:0/2208224468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48400fee80 0x7f4840105a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f48367fc700 1 -- 172.21.15.40:0/2208224468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4840100f80 msgr2=0x7f48401064e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f48367fc700 1 --2- 172.21.15.40:0/2208224468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4840100f80 0x7f48401064e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f48367fc700 1 -- 172.21.15.40:0/2208224468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4838009d00 con 0x7f48401005d0 2023-07-18T03:29:50.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f483effd700 1 --2- 172.21.15.40:0/2208224468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48400fee80 0x7f4840105a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:29:50.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.425+0000 7f483f7fe700 1 --2- 172.21.15.40:0/2208224468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4840100f80 0x7f48401064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:29:50.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.426+0000 7f48367fc700 1 --2- 172.21.15.40:0/2208224468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48401005d0 0x7f4840105f60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f482800e9a0 tx=0x7f482800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:50.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.426+0000 7f483cff9700 1 -- 172.21.15.40:0/2208224468 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4828009790 con 0x7f48401005d0 2023-07-18T03:29:50.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.426+0000 7f48456e2700 1 -- 172.21.15.40:0/2208224468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48401a4860 con 0x7f48401005d0 2023-07-18T03:29:50.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.426+0000 7f483cff9700 1 -- 172.21.15.40:0/2208224468 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f482800cbd0 con 0x7f48401005d0 2023-07-18T03:29:50.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.426+0000 7f483cff9700 1 -- 172.21.15.40:0/2208224468 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4828018680 con 0x7f48401005d0 2023-07-18T03:29:50.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.426+0000 7f48456e2700 1 -- 172.21.15.40:0/2208224468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48401a4d60 con 0x7f48401005d0 2023-07-18T03:29:50.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.427+0000 7f48456e2700 1 -- 172.21.15.40:0/2208224468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f484004e210 con 0x7f48401005d0 2023-07-18T03:29:50.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.428+0000 7f483cff9700 1 -- 172.21.15.40:0/2208224468 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f482800cd40 con 0x7f48401005d0 2023-07-18T03:29:50.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.429+0000 7f483cff9700 1 --2- 172.21.15.40:0/2208224468 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f482c0a11e0 0x7f482c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.429+0000 7f483effd700 1 --2- 172.21.15.40:0/2208224468 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f482c0a11e0 0x7f482c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.429+0000 7f483cff9700 1 -- 172.21.15.40:0/2208224468 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4828014070 con 0x7f48401005d0 2023-07-18T03:29:50.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.429+0000 7f483cff9700 1 --2- 172.21.15.40:0/2208224468 >> [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323] conn(0x7f482c0aa9a0 0x7f482c0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.429+0000 7f483cff9700 1 -- 172.21.15.40:0/2208224468 --> [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f482c0ad460 con 0x7f482c0aa9a0 2023-07-18T03:29:50.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.429+0000 7f483cff9700 1 -- 172.21.15.40:0/2208224468 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48280cd730 con 0x7f48401005d0 2023-07-18T03:29:50.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.429+0000 7f483f7fe700 1 --2- 172.21.15.40:0/2208224468 >> [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323] conn(0x7f482c0aa9a0 0x7f482c0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.429+0000 7f483effd700 1 --2- 172.21.15.40:0/2208224468 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f482c0a11e0 0x7f482c0a3690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4830005950 tx=0x7f48300058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:50.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.429+0000 7f483f7fe700 1 --2- 172.21.15.40:0/2208224468 >> [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323] conn(0x7f482c0aa9a0 0x7f482c0acdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:50.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.430+0000 7f483cff9700 1 -- 172.21.15.40:0/2208224468 <== osd.9 v2:172.21.15.40:6826/2771612323 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f482c0ad460 con 0x7f482c0aa9a0 2023-07-18T03:29:50.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.457+0000 7f48456e2700 1 -- 172.21.15.40:0/2208224468 --> [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f484010ed60 con 0x7f482c0aa9a0 2023-07-18T03:29:50.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.457+0000 7f483cff9700 1 -- 172.21.15.40:0/2208224468 <== osd.9 v2:172.21.15.40:6826/2771612323 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4838009400 con 0x7f482c0aa9a0 2023-07-18T03:29:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.457+0000 7f48456e2700 1 -- 172.21.15.40:0/2208224468 >> [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323] conn(0x7f482c0aa9a0 msgr2=0x7f482c0acdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.457+0000 7f48456e2700 1 --2- 172.21.15.40:0/2208224468 >> [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323] conn(0x7f482c0aa9a0 0x7f482c0acdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.457+0000 7f48456e2700 1 -- 172.21.15.40:0/2208224468 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f482c0a11e0 msgr2=0x7f482c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.457+0000 7f48456e2700 1 --2- 172.21.15.40:0/2208224468 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f482c0a11e0 0x7f482c0a3690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4830005950 tx=0x7f48300058e0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.457+0000 7f48456e2700 1 -- 172.21.15.40:0/2208224468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48401005d0 msgr2=0x7f4840105f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.457+0000 7f48456e2700 1 --2- 172.21.15.40:0/2208224468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48401005d0 0x7f4840105f60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f482800e9a0 tx=0x7f482800ed60 comp rx=0 tx=0).stop 2023-07-18T03:29:50.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.457+0000 7f483f7fe700 1 -- 172.21.15.40:0/2208224468 reap_dead start 2023-07-18T03:29:50.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.458+0000 7f48456e2700 1 -- 172.21.15.40:0/2208224468 shutdown_connections 2023-07-18T03:29:50.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.458+0000 7f48456e2700 1 -- 172.21.15.40:0/2208224468 >> 172.21.15.40:0/2208224468 conn(0x7f4840075b60 msgr2=0x7f4840068860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:50.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.458+0000 7f48456e2700 1 -- 172.21.15.40:0/2208224468 shutdown_connections 2023-07-18T03:29:50.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.458+0000 7f48456e2700 1 -- 172.21.15.40:0/2208224468 wait complete. 2023-07-18T03:29:50.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.856+0000 7f1c66319700 1 -- 172.21.15.40:0/3625238082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c60104ab0 msgr2=0x7f1c60104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.856+0000 7f1c66319700 1 --2- 172.21.15.40:0/3625238082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c60104ab0 0x7f1c60104e80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009990 tx=0x7f1c50009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.856+0000 7f1c66319700 1 -- 172.21.15.40:0/3625238082 shutdown_connections 2023-07-18T03:29:50.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.856+0000 7f1c66319700 1 --2- 172.21.15.40:0/3625238082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c600ff460 0x7f1c6010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.856+0000 7f1c66319700 1 --2- 172.21.15.40:0/3625238082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c60104ab0 0x7f1c60104e80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.856+0000 7f1c66319700 1 --2- 172.21.15.40:0/3625238082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c600feab0 0x7f1c600fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.856+0000 7f1c66319700 1 -- 172.21.15.40:0/3625238082 >> 172.21.15.40:0/3625238082 conn(0x7f1c60075b70 msgr2=0x7f1c60075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:50.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.856+0000 7f1c66319700 1 -- 172.21.15.40:0/3625238082 shutdown_connections 2023-07-18T03:29:50.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.857+0000 7f1c66319700 1 -- 172.21.15.40:0/3625238082 wait complete. 2023-07-18T03:29:50.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.857+0000 7f1c66319700 1 Processor -- start 2023-07-18T03:29:50.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.857+0000 7f1c66319700 1 -- start start 2023-07-18T03:29:50.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c66319700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c600feab0 0x7f1c6019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c66319700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c600ff460 0x7f1c6019bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c66319700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c60104ab0 0x7f1c60195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c66319700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60196090 con 0x7f1c600feab0 2023-07-18T03:29:50.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c66319700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60196200 con 0x7f1c60104ab0 2023-07-18T03:29:50.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c66319700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c601963c0 con 0x7f1c600ff460 2023-07-18T03:29:50.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c5ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c600feab0 0x7f1c6019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c5ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c600feab0 0x7f1c6019b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56432/0 (socket says 172.21.15.40:56432) 2023-07-18T03:29:50.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c5ffff700 1 -- 172.21.15.40:0/4210985654 learned_addr learned my addr 172.21.15.40:0/4210985654 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:50.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c5f7fe700 1 --2- 172.21.15.40:0/4210985654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c600ff460 0x7f1c6019bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c648b6700 1 --2- 172.21.15.40:0/4210985654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c60104ab0 0x7f1c60195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c5ffff700 1 -- 172.21.15.40:0/4210985654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c600ff460 msgr2=0x7f1c6019bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c5ffff700 1 --2- 172.21.15.40:0/4210985654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c600ff460 0x7f1c6019bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c5ffff700 1 -- 172.21.15.40:0/4210985654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c60104ab0 msgr2=0x7f1c60195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c5ffff700 1 --2- 172.21.15.40:0/4210985654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c60104ab0 0x7f1c60195a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c5ffff700 1 -- 172.21.15.40:0/4210985654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c500095f0 con 0x7f1c600feab0 2023-07-18T03:29:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c5f7fe700 1 --2- 172.21.15.40:0/4210985654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c600ff460 0x7f1c6019bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:29:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c5ffff700 1 --2- 172.21.15.40:0/4210985654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c600feab0 0x7f1c6019b9a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1c50003730 tx=0x7f1c5000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c5d7fa700 1 -- 172.21.15.40:0/4210985654 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5001d070 con 0x7f1c600feab0 2023-07-18T03:29:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c5d7fa700 1 -- 172.21.15.40:0/4210985654 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c50004c90 con 0x7f1c600feab0 2023-07-18T03:29:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.858+0000 7f1c5d7fa700 1 -- 172.21.15.40:0/4210985654 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c500053f0 con 0x7f1c600feab0 2023-07-18T03:29:50.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.859+0000 7f1c66319700 1 -- 172.21.15.40:0/4210985654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c60196640 con 0x7f1c600feab0 2023-07-18T03:29:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.859+0000 7f1c66319700 1 -- 172.21.15.40:0/4210985654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c601a2990 con 0x7f1c600feab0 2023-07-18T03:29:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.859+0000 7f1c66319700 1 -- 172.21.15.40:0/4210985654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c6004e210 con 0x7f1c600feab0 2023-07-18T03:29:50.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.861+0000 7f1c5d7fa700 1 -- 172.21.15.40:0/4210985654 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1c50005550 con 0x7f1c600feab0 2023-07-18T03:29:50.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.861+0000 7f1c5d7fa700 1 --2- 172.21.15.40:0/4210985654 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1c480a1260 0x7f1c480a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.861+0000 7f1c5f7fe700 1 --2- 172.21.15.40:0/4210985654 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1c480a1260 0x7f1c480a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.861+0000 7f1c5d7fa700 1 -- 172.21.15.40:0/4210985654 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1c500cdf00 con 0x7f1c600feab0 2023-07-18T03:29:50.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.862+0000 7f1c5d7fa700 1 --2- 172.21.15.40:0/4210985654 >> [v2:172.21.15.134:6816/2602435562,v1:172.21.15.134:6817/2602435562] conn(0x7f1c480aaa30 0x7f1c480ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:50.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.862+0000 7f1c5d7fa700 1 -- 172.21.15.40:0/4210985654 --> [v2:172.21.15.134:6816/2602435562,v1:172.21.15.134:6817/2602435562] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c480ad4f0 con 0x7f1c480aaa30 2023-07-18T03:29:50.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.862+0000 7f1c5d7fa700 1 -- 172.21.15.40:0/4210985654 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c500ce3a0 con 0x7f1c600feab0 2023-07-18T03:29:50.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.862+0000 7f1c5f7fe700 1 --2- 172.21.15.40:0/4210985654 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1c480a1260 0x7f1c480a3710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1c54005fd0 tx=0x7f1c54005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:50.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.862+0000 7f1c648b6700 1 --2- 172.21.15.40:0/4210985654 >> [v2:172.21.15.134:6816/2602435562,v1:172.21.15.134:6817/2602435562] conn(0x7f1c480aaa30 0x7f1c480ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:50.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.862+0000 7f1c648b6700 1 --2- 172.21.15.40:0/4210985654 >> [v2:172.21.15.134:6816/2602435562,v1:172.21.15.134:6817/2602435562] conn(0x7f1c480aaa30 0x7f1c480ace40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:50.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.863+0000 7f1c5d7fa700 1 -- 172.21.15.40:0/4210985654 <== osd.8 v2:172.21.15.134:6816/2602435562 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1c480ad4f0 con 0x7f1c480aaa30 2023-07-18T03:29:50.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.879+0000 7f1c66319700 1 -- 172.21.15.40:0/4210985654 --> [v2:172.21.15.134:6816/2602435562,v1:172.21.15.134:6817/2602435562] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1c60102fc0 con 0x7f1c480aaa30 2023-07-18T03:29:50.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.880+0000 7f1c5d7fa700 1 -- 172.21.15.40:0/4210985654 <== osd.8 v2:172.21.15.134:6816/2602435562 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1c60102fc0 con 0x7f1c480aaa30 2023-07-18T03:29:50.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.880+0000 7f1c66319700 1 -- 172.21.15.40:0/4210985654 >> [v2:172.21.15.134:6816/2602435562,v1:172.21.15.134:6817/2602435562] conn(0x7f1c480aaa30 msgr2=0x7f1c480ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.880+0000 7f1c66319700 1 --2- 172.21.15.40:0/4210985654 >> [v2:172.21.15.134:6816/2602435562,v1:172.21.15.134:6817/2602435562] conn(0x7f1c480aaa30 0x7f1c480ace40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:50.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.880+0000 7f1c66319700 1 -- 172.21.15.40:0/4210985654 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1c480a1260 msgr2=0x7f1c480a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.880+0000 7f1c66319700 1 --2- 172.21.15.40:0/4210985654 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1c480a1260 0x7f1c480a3710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1c54005fd0 tx=0x7f1c54005e30 comp rx=0 tx=0).stop 2023-07-18T03:29:50.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.880+0000 7f1c66319700 1 -- 172.21.15.40:0/4210985654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c600feab0 msgr2=0x7f1c6019b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:50.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.880+0000 7f1c66319700 1 --2- 172.21.15.40:0/4210985654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c600feab0 0x7f1c6019b9a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1c50003730 tx=0x7f1c5000f740 comp rx=0 tx=0).stop 2023-07-18T03:29:50.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.880+0000 7f1c648b6700 1 -- 172.21.15.40:0/4210985654 reap_dead start 2023-07-18T03:29:50.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.880+0000 7f1c66319700 1 -- 172.21.15.40:0/4210985654 shutdown_connections 2023-07-18T03:29:50.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.880+0000 7f1c66319700 1 -- 172.21.15.40:0/4210985654 >> 172.21.15.40:0/4210985654 conn(0x7f1c60075b70 msgr2=0x7f1c60102670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:50.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.880+0000 7f1c66319700 1 -- 172.21.15.40:0/4210985654 shutdown_connections 2023-07-18T03:29:50.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:50.880+0000 7f1c66319700 1 -- 172.21.15.40:0/4210985654 wait complete. 2023-07-18T03:29:51.001 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:51.567 INFO:teuthology.orchestra.run.smithi040.stdout:274877906969 2023-07-18T03:29:51.568 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-07-18T03:29:51.637 INFO:teuthology.orchestra.run.smithi040.stdout:210453397536 2023-07-18T03:29:51.637 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-07-18T03:29:51.740 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:51 smithi040 ceph-mon[96439]: pgmap v260: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-07-18T03:29:52.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:51 smithi134 ceph-mon[98216]: pgmap v260: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-07-18T03:29:52.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:51 smithi064 ceph-mon[98059]: pgmap v260: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-07-18T03:29:52.185 INFO:teuthology.orchestra.run.smithi040.stdout:158913789993 2023-07-18T03:29:52.185 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-07-18T03:29:52.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.374+0000 7f5b98dc1700 1 -- 172.21.15.40:0/3360100989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b9410a270 msgr2=0x7f5b9410c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.374+0000 7f5b98dc1700 1 --2- 172.21.15.40:0/3360100989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b9410a270 0x7f5b9410c6c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00b0a0 tx=0x7f5b8c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.375+0000 7f5b98dc1700 1 -- 172.21.15.40:0/3360100989 shutdown_connections 2023-07-18T03:29:52.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.375+0000 7f5b98dc1700 1 --2- 172.21.15.40:0/3360100989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b9410a270 0x7f5b9410c6c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.375+0000 7f5b98dc1700 1 --2- 172.21.15.40:0/3360100989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b94100df0 0x7f5b941011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.375+0000 7f5b98dc1700 1 --2- 172.21.15.40:0/3360100989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b94101790 0x7f5b94109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.375+0000 7f5b98dc1700 1 -- 172.21.15.40:0/3360100989 >> 172.21.15.40:0/3360100989 conn(0x7f5b94075b60 msgr2=0x7f5b94075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:52.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.375+0000 7f5b98dc1700 1 -- 172.21.15.40:0/3360100989 shutdown_connections 2023-07-18T03:29:52.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.375+0000 7f5b98dc1700 1 -- 172.21.15.40:0/3360100989 wait complete. 2023-07-18T03:29:52.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.376+0000 7f5b98dc1700 1 Processor -- start 2023-07-18T03:29:52.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.376+0000 7f5b98dc1700 1 -- start start 2023-07-18T03:29:52.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.376+0000 7f5b98dc1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b94100df0 0x7f5b94195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:52.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.376+0000 7f5b98dc1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b94101790 0x7f5b941963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:52.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.376+0000 7f5b98dc1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b9410a270 0x7f5b9419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:52.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.376+0000 7f5b98dc1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9419ae50 con 0x7f5b94100df0 2023-07-18T03:29:52.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.376+0000 7f5b98dc1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9419afc0 con 0x7f5b9410a270 2023-07-18T03:29:52.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.376+0000 7f5b98dc1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9419b180 con 0x7f5b94101790 2023-07-18T03:29:52.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.376+0000 7f5b9259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b94100df0 0x7f5b94195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:52.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.377+0000 7f5b9259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b94100df0 0x7f5b94195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56446/0 (socket says 172.21.15.40:56446) 2023-07-18T03:29:52.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.377+0000 7f5b9259c700 1 -- 172.21.15.40:0/2387910648 learned_addr learned my addr 172.21.15.40:0/2387910648 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.377+0000 7f5b91d9b700 1 --2- 172.21.15.40:0/2387910648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b94101790 0x7f5b941963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.377+0000 7f5b92d9d700 1 --2- 172.21.15.40:0/2387910648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b9410a270 0x7f5b9419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.377+0000 7f5b9259c700 1 -- 172.21.15.40:0/2387910648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b94101790 msgr2=0x7f5b941963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.377+0000 7f5b9259c700 1 --2- 172.21.15.40:0/2387910648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b94101790 0x7f5b941963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.377+0000 7f5b9259c700 1 -- 172.21.15.40:0/2387910648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b9410a270 msgr2=0x7f5b9419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.377+0000 7f5b9259c700 1 --2- 172.21.15.40:0/2387910648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b9410a270 0x7f5b9419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.377+0000 7f5b9259c700 1 -- 172.21.15.40:0/2387910648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c009d00 con 0x7f5b94100df0 2023-07-18T03:29:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.377+0000 7f5b91d9b700 1 --2- 172.21.15.40:0/2387910648 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5b94101790 0x7f5b941963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.377+0000 7f5b92d9d700 1 --2- 172.21.15.40:0/2387910648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b9410a270 0x7f5b9419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.377+0000 7f5b9259c700 1 --2- 172.21.15.40:0/2387910648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b94100df0 0x7f5b94195e90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400e9a0 tx=0x7f5b8400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.377+0000 7f5b837fe700 1 -- 172.21.15.40:0/2387910648 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84009790 con 0x7f5b94100df0 2023-07-18T03:29:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.377+0000 7f5b98dc1700 1 -- 172.21.15.40:0/2387910648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9419b460 con 0x7f5b94100df0 2023-07-18T03:29:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.377+0000 7f5b837fe700 1 -- 172.21.15.40:0/2387910648 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b84010430 con 0x7f5b94100df0 2023-07-18T03:29:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.377+0000 7f5b837fe700 1 -- 172.21.15.40:0/2387910648 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84018680 con 0x7f5b94100df0 2023-07-18T03:29:52.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.377+0000 7f5b98dc1700 1 -- 172.21.15.40:0/2387910648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b940feb50 con 0x7f5b94100df0 2023-07-18T03:29:52.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.378+0000 7f5b98dc1700 1 -- 172.21.15.40:0/2387910648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5b9404e210 con 0x7f5b94100df0 2023-07-18T03:29:52.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.380+0000 7f5b837fe700 1 -- 172.21.15.40:0/2387910648 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5b840105a0 con 0x7f5b94100df0 2023-07-18T03:29:52.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.381+0000 7f5b837fe700 1 --2- 172.21.15.40:0/2387910648 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5b7c0a11e0 0x7f5b7c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:52.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.381+0000 7f5b91d9b700 1 --2- 172.21.15.40:0/2387910648 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5b7c0a11e0 0x7f5b7c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:52.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.381+0000 7f5b837fe700 1 -- 172.21.15.40:0/2387910648 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5b84014070 con 0x7f5b94100df0 2023-07-18T03:29:52.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.381+0000 7f5b837fe700 1 --2- 172.21.15.40:0/2387910648 >> [v2:172.21.15.64:6800/2992805245,v1:172.21.15.64:6801/2992805245] conn(0x7f5b7c0aa9b0 0x7f5b7c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:52.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.381+0000 7f5b837fe700 1 -- 172.21.15.40:0/2387910648 --> [v2:172.21.15.64:6800/2992805245,v1:172.21.15.64:6801/2992805245] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5b7c0ad470 con 0x7f5b7c0aa9b0 2023-07-18T03:29:52.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.381+0000 7f5b837fe700 1 -- 172.21.15.40:0/2387910648 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b840cd930 con 0x7f5b94100df0 2023-07-18T03:29:52.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.381+0000 7f5b92d9d700 1 --2- 172.21.15.40:0/2387910648 >> [v2:172.21.15.64:6800/2992805245,v1:172.21.15.64:6801/2992805245] conn(0x7f5b7c0aa9b0 0x7f5b7c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:52.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.382+0000 7f5b91d9b700 1 --2- 172.21.15.40:0/2387910648 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5b7c0a11e0 0x7f5b7c0a3690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b941974b0 tx=0x7f5b88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:52.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.382+0000 7f5b92d9d700 1 --2- 172.21.15.40:0/2387910648 >> [v2:172.21.15.64:6800/2992805245,v1:172.21.15.64:6801/2992805245] conn(0x7f5b7c0aa9b0 0x7f5b7c0acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:52.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.383+0000 7f5b837fe700 1 -- 172.21.15.40:0/2387910648 <== osd.1 v2:172.21.15.64:6800/2992805245 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5b7c0ad470 con 0x7f5b7c0aa9b0 2023-07-18T03:29:52.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.403+0000 7f5b98dc1700 1 -- 172.21.15.40:0/2387910648 --> [v2:172.21.15.64:6800/2992805245,v1:172.21.15.64:6801/2992805245] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5b9410fee0 con 0x7f5b7c0aa9b0 2023-07-18T03:29:52.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.403+0000 7f5b837fe700 1 -- 172.21.15.40:0/2387910648 <== osd.1 v2:172.21.15.64:6800/2992805245 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5b8c009b80 con 0x7f5b7c0aa9b0 2023-07-18T03:29:52.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.404+0000 7f5b98dc1700 1 -- 172.21.15.40:0/2387910648 >> [v2:172.21.15.64:6800/2992805245,v1:172.21.15.64:6801/2992805245] conn(0x7f5b7c0aa9b0 msgr2=0x7f5b7c0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.404+0000 7f5b98dc1700 1 --2- 172.21.15.40:0/2387910648 >> [v2:172.21.15.64:6800/2992805245,v1:172.21.15.64:6801/2992805245] conn(0x7f5b7c0aa9b0 0x7f5b7c0acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.404+0000 7f5b98dc1700 1 -- 172.21.15.40:0/2387910648 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5b7c0a11e0 msgr2=0x7f5b7c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.404+0000 7f5b98dc1700 1 --2- 172.21.15.40:0/2387910648 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5b7c0a11e0 0x7f5b7c0a3690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b941974b0 tx=0x7f5b88009260 comp rx=0 tx=0).stop 2023-07-18T03:29:52.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.404+0000 7f5b98dc1700 1 -- 172.21.15.40:0/2387910648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b94100df0 msgr2=0x7f5b94195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.404+0000 7f5b98dc1700 1 --2- 172.21.15.40:0/2387910648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b94100df0 0x7f5b94195e90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400e9a0 tx=0x7f5b8400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.404+0000 7f5b92d9d700 1 -- 172.21.15.40:0/2387910648 reap_dead start 2023-07-18T03:29:52.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.404+0000 7f5b98dc1700 1 -- 172.21.15.40:0/2387910648 shutdown_connections 2023-07-18T03:29:52.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.404+0000 7f5b98dc1700 1 -- 172.21.15.40:0/2387910648 >> 172.21.15.40:0/2387910648 conn(0x7f5b94075b60 msgr2=0x7f5b94103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:52.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.404+0000 7f5b98dc1700 1 -- 172.21.15.40:0/2387910648 shutdown_connections 2023-07-18T03:29:52.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.404+0000 7f5b98dc1700 1 -- 172.21.15.40:0/2387910648 wait complete. 2023-07-18T03:29:52.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.456+0000 7fd35f960700 1 -- 172.21.15.40:0/1667599736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3580ffe00 msgr2=0x7fd3581001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.456+0000 7fd35f960700 1 --2- 172.21.15.40:0/1667599736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3580ffe00 0x7fd3581001d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd3540099c0 tx=0x7fd354009cd0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.456+0000 7fd35f960700 1 -- 172.21.15.40:0/1667599736 shutdown_connections 2023-07-18T03:29:52.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.456+0000 7fd35f960700 1 --2- 172.21.15.40:0/1667599736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd358104ea0 0x7fd35810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.456+0000 7fd35f960700 1 --2- 172.21.15.40:0/1667599736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3580ffe00 0x7fd3581001d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.456+0000 7fd35f960700 1 --2- 172.21.15.40:0/1667599736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3581007a0 0x7fd358104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.456+0000 7fd35f960700 1 -- 172.21.15.40:0/1667599736 >> 172.21.15.40:0/1667599736 conn(0x7fd3580fb660 msgr2=0x7fd3580fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:52.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.456+0000 7fd35f960700 1 -- 172.21.15.40:0/1667599736 shutdown_connections 2023-07-18T03:29:52.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.456+0000 7fd35f960700 1 -- 172.21.15.40:0/1667599736 wait complete. 2023-07-18T03:29:52.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.457+0000 7fd35f960700 1 Processor -- start 2023-07-18T03:29:52.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.457+0000 7fd35f960700 1 -- start start 2023-07-18T03:29:52.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.457+0000 7fd35f960700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3580ffe00 0x7fd358072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:52.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.457+0000 7fd35f960700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3581007a0 0x7fd358072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:52.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.457+0000 7fd35f960700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd358104ea0 0x7fd3580732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:52.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.457+0000 7fd35f960700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3581a4040 con 0x7fd358104ea0 2023-07-18T03:29:52.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.457+0000 7fd35f960700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3581a41b0 con 0x7fd3580ffe00 2023-07-18T03:29:52.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.457+0000 7fd35f960700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3581a4370 con 0x7fd3581007a0 2023-07-18T03:29:52.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.457+0000 7fd35defd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd358104ea0 0x7fd3580732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:52.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.457+0000 7fd35d6fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3580ffe00 0x7fd358072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:52.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.457+0000 7fd35defd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd358104ea0 0x7fd3580732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56466/0 (socket says 172.21.15.40:56466) 2023-07-18T03:29:52.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.457+0000 7fd35d6fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3580ffe00 0x7fd358072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:39788/0 (socket says 172.21.15.40:39788) 2023-07-18T03:29:52.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.457+0000 7fd35defd700 1 -- 172.21.15.40:0/2078588891 learned_addr learned my addr 172.21.15.40:0/2078588891 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:52.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.458+0000 7fd35cefb700 1 --2- 172.21.15.40:0/2078588891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3581007a0 0x7fd358072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:52.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.458+0000 7fd35defd700 1 -- 172.21.15.40:0/2078588891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3581007a0 msgr2=0x7fd358072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.458+0000 7fd35defd700 1 --2- 172.21.15.40:0/2078588891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3581007a0 0x7fd358072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.458+0000 7fd35defd700 1 -- 172.21.15.40:0/2078588891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3580ffe00 msgr2=0x7fd358072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.458+0000 7fd35defd700 1 --2- 172.21.15.40:0/2078588891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3580ffe00 0x7fd358072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.458+0000 7fd35defd700 1 -- 172.21.15.40:0/2078588891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3540095f0 con 0x7fd358104ea0 2023-07-18T03:29:52.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.458+0000 7fd35d6fc700 1 --2- 172.21.15.40:0/2078588891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3580ffe00 0x7fd358072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:29:52.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.458+0000 7fd35defd700 1 --2- 172.21.15.40:0/2078588891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd358104ea0 0x7fd3580732d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd350007ee0 tx=0x7fd35000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:52.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.458+0000 7fd34a7fc700 1 -- 172.21.15.40:0/2078588891 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35000ddd0 con 0x7fd358104ea0 2023-07-18T03:29:52.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.458+0000 7fd34a7fc700 1 -- 172.21.15.40:0/2078588891 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd35000f040 con 0x7fd358104ea0 2023-07-18T03:29:52.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.458+0000 7fd35f960700 1 -- 172.21.15.40:0/2078588891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3581a4650 con 0x7fd358104ea0 2023-07-18T03:29:52.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.458+0000 7fd34a7fc700 1 -- 172.21.15.40:0/2078588891 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3500048e0 con 0x7fd358104ea0 2023-07-18T03:29:52.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.458+0000 7fd35f960700 1 -- 172.21.15.40:0/2078588891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3581a4b00 con 0x7fd358104ea0 2023-07-18T03:29:52.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.459+0000 7fd35f960700 1 -- 172.21.15.40:0/2078588891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd35810fd50 con 0x7fd358104ea0 2023-07-18T03:29:52.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.460+0000 7fd34a7fc700 1 -- 172.21.15.40:0/2078588891 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd350003a00 con 0x7fd358104ea0 2023-07-18T03:29:52.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.461+0000 7fd34a7fc700 1 --2- 172.21.15.40:0/2078588891 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd3440a11e0 0x7fd3440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:52.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.461+0000 7fd35d6fc700 1 --2- 172.21.15.40:0/2078588891 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd3440a11e0 0x7fd3440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:52.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.461+0000 7fd34a7fc700 1 -- 172.21.15.40:0/2078588891 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd3500ccd20 con 0x7fd358104ea0 2023-07-18T03:29:52.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.461+0000 7fd34a7fc700 1 --2- 172.21.15.40:0/2078588891 >> [v2:172.21.15.134:6800/1877339886,v1:172.21.15.134:6801/1877339886] conn(0x7fd3440aa840 0x7fd3440acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:52.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.461+0000 7fd34a7fc700 1 -- 172.21.15.40:0/2078588891 --> [v2:172.21.15.134:6800/1877339886,v1:172.21.15.134:6801/1877339886] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd3440ad300 con 0x7fd3440aa840 2023-07-18T03:29:52.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.461+0000 7fd34a7fc700 1 -- 172.21.15.40:0/2078588891 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3500cd1c0 con 0x7fd358104ea0 2023-07-18T03:29:52.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.461+0000 7fd35d6fc700 1 --2- 172.21.15.40:0/2078588891 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd3440a11e0 0x7fd3440a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd354009990 tx=0x7fd354000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:52.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.461+0000 7fd35cefb700 1 --2- 172.21.15.40:0/2078588891 >> [v2:172.21.15.134:6800/1877339886,v1:172.21.15.134:6801/1877339886] conn(0x7fd3440aa840 0x7fd3440acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:52.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.462+0000 7fd35cefb700 1 --2- 172.21.15.40:0/2078588891 >> [v2:172.21.15.134:6800/1877339886,v1:172.21.15.134:6801/1877339886] conn(0x7fd3440aa840 0x7fd3440acc50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:52.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.463+0000 7fd34a7fc700 1 -- 172.21.15.40:0/2078588891 <== osd.2 v2:172.21.15.134:6800/1877339886 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd3440ad300 con 0x7fd3440aa840 2023-07-18T03:29:52.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.479+0000 7fd35f960700 1 -- 172.21.15.40:0/2078588891 --> [v2:172.21.15.134:6800/1877339886,v1:172.21.15.134:6801/1877339886] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd35804e210 con 0x7fd3440aa840 2023-07-18T03:29:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.480+0000 7fd34a7fc700 1 -- 172.21.15.40:0/2078588891 <== osd.2 v2:172.21.15.134:6800/1877339886 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd35804e210 con 0x7fd3440aa840 2023-07-18T03:29:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.480+0000 7fd35f960700 1 -- 172.21.15.40:0/2078588891 >> [v2:172.21.15.134:6800/1877339886,v1:172.21.15.134:6801/1877339886] conn(0x7fd3440aa840 msgr2=0x7fd3440acc50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.480+0000 7fd35f960700 1 --2- 172.21.15.40:0/2078588891 >> [v2:172.21.15.134:6800/1877339886,v1:172.21.15.134:6801/1877339886] conn(0x7fd3440aa840 0x7fd3440acc50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.480+0000 7fd35f960700 1 -- 172.21.15.40:0/2078588891 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd3440a11e0 msgr2=0x7fd3440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.480+0000 7fd35f960700 1 --2- 172.21.15.40:0/2078588891 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd3440a11e0 0x7fd3440a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd354009990 tx=0x7fd354000bc0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.480+0000 7fd35f960700 1 -- 172.21.15.40:0/2078588891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd358104ea0 msgr2=0x7fd3580732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.480+0000 7fd35f960700 1 --2- 172.21.15.40:0/2078588891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd358104ea0 0x7fd3580732d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd350007ee0 tx=0x7fd35000d3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.480+0000 7fd35defd700 1 -- 172.21.15.40:0/2078588891 reap_dead start 2023-07-18T03:29:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.480+0000 7fd35f960700 1 -- 172.21.15.40:0/2078588891 shutdown_connections 2023-07-18T03:29:52.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.480+0000 7fd35f960700 1 -- 172.21.15.40:0/2078588891 >> 172.21.15.40:0/2078588891 conn(0x7fd3580fb660 msgr2=0x7fd358103750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:52.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.480+0000 7fd35f960700 1 -- 172.21.15.40:0/2078588891 shutdown_connections 2023-07-18T03:29:52.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.480+0000 7fd35f960700 1 -- 172.21.15.40:0/2078588891 wait complete. 2023-07-18T03:29:52.726 INFO:teuthology.orchestra.run.smithi040.stdout:240518168604 2023-07-18T03:29:52.727 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-07-18T03:29:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.969+0000 7faa21a96700 1 -- 172.21.15.40:0/440276660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa1c101700 msgr2=0x7faa1c109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.969+0000 7faa21a96700 1 --2- 172.21.15.40:0/440276660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa1c101700 0x7faa1c109c00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faa10009990 tx=0x7faa10009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.969+0000 7faa21a96700 1 -- 172.21.15.40:0/440276660 shutdown_connections 2023-07-18T03:29:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.969+0000 7faa21a96700 1 --2- 172.21.15.40:0/440276660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa1c10a140 0x7faa1c10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.969+0000 7faa21a96700 1 --2- 172.21.15.40:0/440276660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c100df0 0x7faa1c1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.969+0000 7faa21a96700 1 --2- 172.21.15.40:0/440276660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa1c101700 0x7faa1c109c00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.969+0000 7faa21a96700 1 -- 172.21.15.40:0/440276660 >> 172.21.15.40:0/440276660 conn(0x7faa1c075b70 msgr2=0x7faa1c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.969+0000 7faa21a96700 1 -- 172.21.15.40:0/440276660 shutdown_connections 2023-07-18T03:29:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.969+0000 7faa21a96700 1 -- 172.21.15.40:0/440276660 wait complete. 2023-07-18T03:29:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.970+0000 7faa21a96700 1 Processor -- start 2023-07-18T03:29:52.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.970+0000 7faa21a96700 1 -- start start 2023-07-18T03:29:52.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.970+0000 7faa21a96700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa1c100df0 0x7faa1c197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:52.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.970+0000 7faa21a96700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c101700 0x7faa1c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:52.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.970+0000 7faa21a96700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa1c10a140 0x7faa1c19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:52.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.970+0000 7faa21a96700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c19cda0 con 0x7faa1c10a140 2023-07-18T03:29:52.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.970+0000 7faa21a96700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c19cf10 con 0x7faa1c101700 2023-07-18T03:29:52.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.970+0000 7faa21a96700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c19d0d0 con 0x7faa1c100df0 2023-07-18T03:29:52.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.970+0000 7faa1bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa1c10a140 0x7faa1c19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:52.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.970+0000 7faa1bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa1c10a140 0x7faa1c19c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56482/0 (socket says 172.21.15.40:56482) 2023-07-18T03:29:52.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.970+0000 7faa1bfff700 1 -- 172.21.15.40:0/49483305 learned_addr learned my addr 172.21.15.40:0/49483305 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:52.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.970+0000 7faa1affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c101700 0x7faa1c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:52.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.970+0000 7faa1b7fe700 1 --2- 172.21.15.40:0/49483305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa1c100df0 0x7faa1c197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:52.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.971+0000 7faa1bfff700 1 -- 172.21.15.40:0/49483305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa1c100df0 msgr2=0x7faa1c197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.971+0000 7faa1bfff700 1 --2- 172.21.15.40:0/49483305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa1c100df0 0x7faa1c197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.971+0000 7faa1bfff700 1 -- 172.21.15.40:0/49483305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c101700 msgr2=0x7faa1c198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.971+0000 7faa1bfff700 1 --2- 172.21.15.40:0/49483305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa1c101700 0x7faa1c198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.971+0000 7faa1bfff700 1 -- 172.21.15.40:0/49483305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa100095f0 con 0x7faa1c10a140 2023-07-18T03:29:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.971+0000 7faa1bfff700 1 --2- 172.21.15.40:0/49483305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa1c10a140 0x7faa1c19c790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7faa1400cca0 tx=0x7faa1400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:52.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.971+0000 7faa18ff9700 1 -- 172.21.15.40:0/49483305 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1400aaf0 con 0x7faa1c10a140 2023-07-18T03:29:52.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.971+0000 7faa21a96700 1 -- 172.21.15.40:0/49483305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c19d3b0 con 0x7faa1c10a140 2023-07-18T03:29:52.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.971+0000 7faa18ff9700 1 -- 172.21.15.40:0/49483305 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa1400ac50 con 0x7faa1c10a140 2023-07-18T03:29:52.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.971+0000 7faa18ff9700 1 -- 172.21.15.40:0/49483305 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140115e0 con 0x7faa1c10a140 2023-07-18T03:29:52.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.971+0000 7faa21a96700 1 -- 172.21.15.40:0/49483305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c1a4c10 con 0x7faa1c10a140 2023-07-18T03:29:52.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.972+0000 7faa21a96700 1 -- 172.21.15.40:0/49483305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faa1c10fd40 con 0x7faa1c10a140 2023-07-18T03:29:52.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.973+0000 7faa18ff9700 1 -- 172.21.15.40:0/49483305 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7faa14012420 con 0x7faa1c10a140 2023-07-18T03:29:52.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.974+0000 7faa18ff9700 1 --2- 172.21.15.40:0/49483305 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faa040a11e0 0x7faa040a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:52.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.974+0000 7faa1b7fe700 1 --2- 172.21.15.40:0/49483305 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faa040a11e0 0x7faa040a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:52.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.974+0000 7faa18ff9700 1 -- 172.21.15.40:0/49483305 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faa140ccf60 con 0x7faa1c10a140 2023-07-18T03:29:52.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.974+0000 7faa18ff9700 1 --2- 172.21.15.40:0/49483305 >> [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677] conn(0x7faa040aa9b0 0x7faa040acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:52.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.974+0000 7faa18ff9700 1 -- 172.21.15.40:0/49483305 --> [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa040ad470 con 0x7faa040aa9b0 2023-07-18T03:29:52.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.974+0000 7faa18ff9700 1 -- 172.21.15.40:0/49483305 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa140cd2d0 con 0x7faa1c10a140 2023-07-18T03:29:52.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.974+0000 7faa1affd700 1 --2- 172.21.15.40:0/49483305 >> [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677] conn(0x7faa040aa9b0 0x7faa040acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:52.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.974+0000 7faa1b7fe700 1 --2- 172.21.15.40:0/49483305 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faa040a11e0 0x7faa040a3690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faa0c007be0 tx=0x7faa0c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:52.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.975+0000 7faa1affd700 1 --2- 172.21.15.40:0/49483305 >> [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677] conn(0x7faa040aa9b0 0x7faa040acdc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:52.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.975+0000 7faa18ff9700 1 -- 172.21.15.40:0/49483305 <== osd.6 v2:172.21.15.40:6818/380513677 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7faa040ad470 con 0x7faa040aa9b0 2023-07-18T03:29:52.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.998+0000 7faa21a96700 1 -- 172.21.15.40:0/49483305 --> [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faa1c04e210 con 0x7faa040aa9b0 2023-07-18T03:29:52.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.998+0000 7faa18ff9700 1 -- 172.21.15.40:0/49483305 <== osd.6 v2:172.21.15.40:6818/380513677 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7faa1c04e210 con 0x7faa040aa9b0 2023-07-18T03:29:52.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.998+0000 7faa21a96700 1 -- 172.21.15.40:0/49483305 >> [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677] conn(0x7faa040aa9b0 msgr2=0x7faa040acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.998+0000 7faa21a96700 1 --2- 172.21.15.40:0/49483305 >> [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677] conn(0x7faa040aa9b0 0x7faa040acdc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.999+0000 7faa21a96700 1 -- 172.21.15.40:0/49483305 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faa040a11e0 msgr2=0x7faa040a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.999+0000 7faa21a96700 1 --2- 172.21.15.40:0/49483305 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faa040a11e0 0x7faa040a3690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faa0c007be0 tx=0x7faa0c0073d0 comp rx=0 tx=0).stop 2023-07-18T03:29:52.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.999+0000 7faa21a96700 1 -- 172.21.15.40:0/49483305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa1c10a140 msgr2=0x7faa1c19c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:52.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.999+0000 7faa21a96700 1 --2- 172.21.15.40:0/49483305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa1c10a140 0x7faa1c19c790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7faa1400cca0 tx=0x7faa1400a060 comp rx=0 tx=0).stop 2023-07-18T03:29:52.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.999+0000 7faa1bfff700 1 -- 172.21.15.40:0/49483305 reap_dead start 2023-07-18T03:29:52.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.999+0000 7faa21a96700 1 -- 172.21.15.40:0/49483305 shutdown_connections 2023-07-18T03:29:52.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.999+0000 7faa21a96700 1 -- 172.21.15.40:0/49483305 >> 172.21.15.40:0/49483305 conn(0x7faa1c075b70 msgr2=0x7faa1c0fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:52.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.999+0000 7faa21a96700 1 -- 172.21.15.40:0/49483305 shutdown_connections 2023-07-18T03:29:52.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:52.999+0000 7faa21a96700 1 -- 172.21.15.40:0/49483305 wait complete. 2023-07-18T03:29:53.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.031+0000 7fb48928e700 1 -- 172.21.15.40:0/2409019261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb484100df0 msgr2=0x7fb4841011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:53.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.031+0000 7fb48928e700 1 --2- 172.21.15.40:0/2409019261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb484100df0 0x7fb4841011c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb474009990 tx=0x7fb474009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:53.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.031+0000 7fb48928e700 1 -- 172.21.15.40:0/2409019261 shutdown_connections 2023-07-18T03:29:53.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.031+0000 7fb48928e700 1 --2- 172.21.15.40:0/2409019261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb48410a270 0x7fb48410c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:53.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.031+0000 7fb48928e700 1 --2- 172.21.15.40:0/2409019261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb484100df0 0x7fb4841011c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:53.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.031+0000 7fb48928e700 1 --2- 172.21.15.40:0/2409019261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb484101790 0x7fb484109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:53.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.031+0000 7fb48928e700 1 -- 172.21.15.40:0/2409019261 >> 172.21.15.40:0/2409019261 conn(0x7fb484075b60 msgr2=0x7fb484075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:53.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.031+0000 7fb48928e700 1 -- 172.21.15.40:0/2409019261 shutdown_connections 2023-07-18T03:29:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.031+0000 7fb48928e700 1 -- 172.21.15.40:0/2409019261 wait complete. 2023-07-18T03:29:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb48928e700 1 Processor -- start 2023-07-18T03:29:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb48928e700 1 -- start start 2023-07-18T03:29:53.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb48928e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb484100df0 0x7fb484195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:53.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb48928e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb484101790 0x7fb484196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:53.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb48928e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb48410a270 0x7fb48419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:53.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb48928e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48419ad90 con 0x7fb48410a270 2023-07-18T03:29:53.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb48928e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48419af00 con 0x7fb484101790 2023-07-18T03:29:53.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb48928e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48419b0c0 con 0x7fb484100df0 2023-07-18T03:29:53.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb4837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb48410a270 0x7fb48419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:53.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb4837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb48410a270 0x7fb48419a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56498/0 (socket says 172.21.15.40:56498) 2023-07-18T03:29:53.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb4837fe700 1 -- 172.21.15.40:0/651699725 learned_addr learned my addr 172.21.15.40:0/651699725 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:53.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb4837fe700 1 -- 172.21.15.40:0/651699725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb484100df0 msgr2=0x7fb484195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:53.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb4827fc700 1 --2- 172.21.15.40:0/651699725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb484101790 0x7fb484196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:53.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb482ffd700 1 --2- 172.21.15.40:0/651699725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb484100df0 0x7fb484195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:53.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb4837fe700 1 --2- 172.21.15.40:0/651699725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb484100df0 0x7fb484195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:53.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb4837fe700 1 -- 172.21.15.40:0/651699725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb484101790 msgr2=0x7fb484196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:53.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb4837fe700 1 --2- 172.21.15.40:0/651699725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb484101790 0x7fb484196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:53.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb4837fe700 1 -- 172.21.15.40:0/651699725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4740095f0 con 0x7fb48410a270 2023-07-18T03:29:53.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.032+0000 7fb482ffd700 1 --2- 172.21.15.40:0/651699725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb484100df0 0x7fb484195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:29:53.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.033+0000 7fb4837fe700 1 --2- 172.21.15.40:0/651699725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb48410a270 0x7fb48419a780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb47c007ee0 tx=0x7fb47c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:53.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.033+0000 7fb46bfff700 1 -- 172.21.15.40:0/651699725 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c017070 con 0x7fb48410a270 2023-07-18T03:29:53.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.033+0000 7fb46bfff700 1 -- 172.21.15.40:0/651699725 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb47c00f040 con 0x7fb48410a270 2023-07-18T03:29:53.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.033+0000 7fb48928e700 1 -- 172.21.15.40:0/651699725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48419b3a0 con 0x7fb48410a270 2023-07-18T03:29:53.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.033+0000 7fb46bfff700 1 -- 172.21.15.40:0/651699725 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c005740 con 0x7fb48410a270 2023-07-18T03:29:53.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.033+0000 7fb48928e700 1 -- 172.21.15.40:0/651699725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4841a2c00 con 0x7fb48410a270 2023-07-18T03:29:53.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.034+0000 7fb48928e700 1 -- 172.21.15.40:0/651699725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb48404e210 con 0x7fb48410a270 2023-07-18T03:29:53.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.035+0000 7fb46bfff700 1 -- 172.21.15.40:0/651699725 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb47c0058a0 con 0x7fb48410a270 2023-07-18T03:29:53.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.036+0000 7fb46bfff700 1 --2- 172.21.15.40:0/651699725 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb46c0a11e0 0x7fb46c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:53.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.036+0000 7fb482ffd700 1 --2- 172.21.15.40:0/651699725 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb46c0a11e0 0x7fb46c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:53.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.036+0000 7fb46bfff700 1 -- 172.21.15.40:0/651699725 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb47c0cc840 con 0x7fb48410a270 2023-07-18T03:29:53.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.036+0000 7fb46bfff700 1 --2- 172.21.15.40:0/651699725 >> [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213] conn(0x7fb46c0aa9b0 0x7fb46c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:53.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.036+0000 7fb46bfff700 1 -- 172.21.15.40:0/651699725 --> [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb46c0ad470 con 0x7fb46c0aa9b0 2023-07-18T03:29:53.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.036+0000 7fb46bfff700 1 -- 172.21.15.40:0/651699725 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb47c0ccce0 con 0x7fb48410a270 2023-07-18T03:29:53.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.036+0000 7fb4827fc700 1 --2- 172.21.15.40:0/651699725 >> [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213] conn(0x7fb46c0aa9b0 0x7fb46c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:53.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.036+0000 7fb482ffd700 1 --2- 172.21.15.40:0/651699725 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb46c0a11e0 0x7fb46c0a3690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb47400b5c0 tx=0x7fb474005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:53.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.037+0000 7fb4827fc700 1 --2- 172.21.15.40:0/651699725 >> [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213] conn(0x7fb46c0aa9b0 0x7fb46c0acdc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:53.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.037+0000 7fb46bfff700 1 -- 172.21.15.40:0/651699725 <== osd.3 v2:172.21.15.40:6810/3916741213 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb46c0ad470 con 0x7fb46c0aa9b0 2023-07-18T03:29:53.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.053+0000 7fb48928e700 1 -- 172.21.15.40:0/651699725 --> [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb48410fef0 con 0x7fb46c0aa9b0 2023-07-18T03:29:53.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.053+0000 7fb46bfff700 1 -- 172.21.15.40:0/651699725 <== osd.3 v2:172.21.15.40:6810/3916741213 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb47800a9a0 con 0x7fb46c0aa9b0 2023-07-18T03:29:53.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.054+0000 7fb48928e700 1 -- 172.21.15.40:0/651699725 >> [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213] conn(0x7fb46c0aa9b0 msgr2=0x7fb46c0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:53.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.054+0000 7fb48928e700 1 --2- 172.21.15.40:0/651699725 >> [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213] conn(0x7fb46c0aa9b0 0x7fb46c0acdc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:53.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.054+0000 7fb48928e700 1 -- 172.21.15.40:0/651699725 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb46c0a11e0 msgr2=0x7fb46c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:53.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.054+0000 7fb48928e700 1 --2- 172.21.15.40:0/651699725 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb46c0a11e0 0x7fb46c0a3690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb47400b5c0 tx=0x7fb474005fb0 comp rx=0 tx=0).stop 2023-07-18T03:29:53.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.054+0000 7fb48928e700 1 -- 172.21.15.40:0/651699725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb48410a270 msgr2=0x7fb48419a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:53.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.054+0000 7fb48928e700 1 --2- 172.21.15.40:0/651699725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb48410a270 0x7fb48419a780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb47c007ee0 tx=0x7fb47c00d3b0 comp rx=0 tx=0).stop 2023-07-18T03:29:53.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.054+0000 7fb4837fe700 1 -- 172.21.15.40:0/651699725 reap_dead start 2023-07-18T03:29:53.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.054+0000 7fb48928e700 1 -- 172.21.15.40:0/651699725 shutdown_connections 2023-07-18T03:29:53.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.054+0000 7fb48928e700 1 -- 172.21.15.40:0/651699725 >> 172.21.15.40:0/651699725 conn(0x7fb484075b60 msgr2=0x7fb4840fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:53.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.054+0000 7fb48928e700 1 -- 172.21.15.40:0/651699725 shutdown_connections 2023-07-18T03:29:53.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.054+0000 7fb48928e700 1 -- 172.21.15.40:0/651699725 wait complete. 2023-07-18T03:29:53.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.357+0000 7fca50cb0700 1 -- 172.21.15.40:0/3707433814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca4c0ffe30 msgr2=0x7fca4c100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:53.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.357+0000 7fca50cb0700 1 --2- 172.21.15.40:0/3707433814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca4c0ffe30 0x7fca4c100200 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0099c0 tx=0x7fca3c009cd0 comp rx=0 tx=0).stop 2023-07-18T03:29:53.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.358+0000 7fca50cb0700 1 -- 172.21.15.40:0/3707433814 shutdown_connections 2023-07-18T03:29:53.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.358+0000 7fca50cb0700 1 --2- 172.21.15.40:0/3707433814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca4c104ed0 0x7fca4c10ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:53.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.358+0000 7fca50cb0700 1 --2- 172.21.15.40:0/3707433814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca4c0ffe30 0x7fca4c100200 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:53.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.358+0000 7fca50cb0700 1 --2- 172.21.15.40:0/3707433814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca4c1007d0 0x7fca4c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:53.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.358+0000 7fca50cb0700 1 -- 172.21.15.40:0/3707433814 >> 172.21.15.40:0/3707433814 conn(0x7fca4c0fb6d0 msgr2=0x7fca4c0fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:53.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.358+0000 7fca50cb0700 1 -- 172.21.15.40:0/3707433814 shutdown_connections 2023-07-18T03:29:53.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.358+0000 7fca50cb0700 1 -- 172.21.15.40:0/3707433814 wait complete. 2023-07-18T03:29:53.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.359+0000 7fca50cb0700 1 Processor -- start 2023-07-18T03:29:53.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.359+0000 7fca50cb0700 1 -- start start 2023-07-18T03:29:53.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.359+0000 7fca50cb0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca4c0ffe30 0x7fca4c199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:53.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.359+0000 7fca50cb0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca4c1007d0 0x7fca4c199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:53.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.359+0000 7fca50cb0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca4c104ed0 0x7fca4c193980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:53.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.359+0000 7fca50cb0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c193ff0 con 0x7fca4c0ffe30 2023-07-18T03:29:53.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.359+0000 7fca50cb0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c194160 con 0x7fca4c104ed0 2023-07-18T03:29:53.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.359+0000 7fca50cb0700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c194320 con 0x7fca4c1007d0 2023-07-18T03:29:53.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.359+0000 7fca4a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca4c0ffe30 0x7fca4c199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:53.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.359+0000 7fca4a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca4c0ffe30 0x7fca4c199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56510/0 (socket says 172.21.15.40:56510) 2023-07-18T03:29:53.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.359+0000 7fca4a59c700 1 -- 172.21.15.40:0/3171762544 learned_addr learned my addr 172.21.15.40:0/3171762544 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:53.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.359+0000 7fca49d9b700 1 --2- 172.21.15.40:0/3171762544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca4c1007d0 0x7fca4c199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:53.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.359+0000 7fca4ad9d700 1 --2- 172.21.15.40:0/3171762544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca4c104ed0 0x7fca4c193980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:53.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.360+0000 7fca4a59c700 1 -- 172.21.15.40:0/3171762544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca4c1007d0 msgr2=0x7fca4c199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:53.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.360+0000 7fca4a59c700 1 --2- 172.21.15.40:0/3171762544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca4c1007d0 0x7fca4c199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:53.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.360+0000 7fca4a59c700 1 -- 172.21.15.40:0/3171762544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca4c104ed0 msgr2=0x7fca4c193980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:53.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.360+0000 7fca4a59c700 1 --2- 172.21.15.40:0/3171762544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca4c104ed0 0x7fca4c193980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:53.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.360+0000 7fca4a59c700 1 -- 172.21.15.40:0/3171762544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c0095f0 con 0x7fca4c0ffe30 2023-07-18T03:29:53.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.360+0000 7fca4ad9d700 1 --2- 172.21.15.40:0/3171762544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca4c104ed0 0x7fca4c193980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:29:53.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.360+0000 7fca4a59c700 1 --2- 172.21.15.40:0/3171762544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca4c0ffe30 0x7fca4c199900 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fca3c01a040 tx=0x7fca3c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:53.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.360+0000 7fca3b7fe700 1 -- 172.21.15.40:0/3171762544 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c004810 con 0x7fca4c0ffe30 2023-07-18T03:29:53.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.360+0000 7fca50cb0700 1 -- 172.21.15.40:0/3171762544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4c1945a0 con 0x7fca4c0ffe30 2023-07-18T03:29:53.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.360+0000 7fca3b7fe700 1 -- 172.21.15.40:0/3171762544 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca3c004e30 con 0x7fca4c0ffe30 2023-07-18T03:29:53.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.360+0000 7fca3b7fe700 1 -- 172.21.15.40:0/3171762544 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c022660 con 0x7fca4c0ffe30 2023-07-18T03:29:53.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.360+0000 7fca50cb0700 1 -- 172.21.15.40:0/3171762544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4c1a08f0 con 0x7fca4c0ffe30 2023-07-18T03:29:53.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.361+0000 7fca50cb0700 1 -- 172.21.15.40:0/3171762544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fca4c10dc40 con 0x7fca4c0ffe30 2023-07-18T03:29:53.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.363+0000 7fca3b7fe700 1 -- 172.21.15.40:0/3171762544 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fca3c005220 con 0x7fca4c0ffe30 2023-07-18T03:29:53.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.363+0000 7fca3b7fe700 1 --2- 172.21.15.40:0/3171762544 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fca340a11e0 0x7fca340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:53.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.363+0000 7fca3b7fe700 1 -- 172.21.15.40:0/3171762544 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fca3c0d2c80 con 0x7fca4c0ffe30 2023-07-18T03:29:53.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.363+0000 7fca49d9b700 1 --2- 172.21.15.40:0/3171762544 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fca340a11e0 0x7fca340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:53.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.363+0000 7fca3b7fe700 1 --2- 172.21.15.40:0/3171762544 >> [v2:172.21.15.64:6808/788347693,v1:172.21.15.64:6809/788347693] conn(0x7fca340aa9b0 0x7fca340acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:53.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.363+0000 7fca3b7fe700 1 -- 172.21.15.40:0/3171762544 --> [v2:172.21.15.64:6808/788347693,v1:172.21.15.64:6809/788347693] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca340ad470 con 0x7fca340aa9b0 2023-07-18T03:29:53.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.363+0000 7fca3b7fe700 1 -- 172.21.15.40:0/3171762544 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca3c0d3120 con 0x7fca4c0ffe30 2023-07-18T03:29:53.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.364+0000 7fca49d9b700 1 --2- 172.21.15.40:0/3171762544 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fca340a11e0 0x7fca340a3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fca40005fd0 tx=0x7fca40005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.364+0000 7fca4ad9d700 1 --2- 172.21.15.40:0/3171762544 >> [v2:172.21.15.64:6808/788347693,v1:172.21.15.64:6809/788347693] conn(0x7fca340aa9b0 0x7fca340acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.364+0000 7fca4ad9d700 1 --2- 172.21.15.40:0/3171762544 >> [v2:172.21.15.64:6808/788347693,v1:172.21.15.64:6809/788347693] conn(0x7fca340aa9b0 0x7fca340acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.365+0000 7fca3b7fe700 1 -- 172.21.15.40:0/3171762544 <== osd.4 v2:172.21.15.64:6808/788347693 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fca340ad470 con 0x7fca340aa9b0 2023-07-18T03:29:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.391+0000 7fca50cb0700 1 -- 172.21.15.40:0/3171762544 --> [v2:172.21.15.64:6808/788347693,v1:172.21.15.64:6809/788347693] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fca4c0fca80 con 0x7fca340aa9b0 2023-07-18T03:29:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.392+0000 7fca3b7fe700 1 -- 172.21.15.40:0/3171762544 <== osd.4 v2:172.21.15.64:6808/788347693 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fca4c0fca80 con 0x7fca340aa9b0 2023-07-18T03:29:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.392+0000 7fca50cb0700 1 -- 172.21.15.40:0/3171762544 >> [v2:172.21.15.64:6808/788347693,v1:172.21.15.64:6809/788347693] conn(0x7fca340aa9b0 msgr2=0x7fca340acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.392+0000 7fca50cb0700 1 --2- 172.21.15.40:0/3171762544 >> [v2:172.21.15.64:6808/788347693,v1:172.21.15.64:6809/788347693] conn(0x7fca340aa9b0 0x7fca340acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:53.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.392+0000 7fca50cb0700 1 -- 172.21.15.40:0/3171762544 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fca340a11e0 msgr2=0x7fca340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:53.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.392+0000 7fca50cb0700 1 --2- 172.21.15.40:0/3171762544 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fca340a11e0 0x7fca340a3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fca40005fd0 tx=0x7fca40005f60 comp rx=0 tx=0).stop 2023-07-18T03:29:53.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.392+0000 7fca50cb0700 1 -- 172.21.15.40:0/3171762544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca4c0ffe30 msgr2=0x7fca4c199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:53.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.392+0000 7fca50cb0700 1 --2- 172.21.15.40:0/3171762544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca4c0ffe30 0x7fca4c199900 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fca3c01a040 tx=0x7fca3c00be60 comp rx=0 tx=0).stop 2023-07-18T03:29:53.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.392+0000 7fca4ad9d700 1 -- 172.21.15.40:0/3171762544 reap_dead start 2023-07-18T03:29:53.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.392+0000 7fca50cb0700 1 -- 172.21.15.40:0/3171762544 shutdown_connections 2023-07-18T03:29:53.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.392+0000 7fca50cb0700 1 -- 172.21.15.40:0/3171762544 >> 172.21.15.40:0/3171762544 conn(0x7fca4c0fb6d0 msgr2=0x7fca4c0fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:53.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.392+0000 7fca50cb0700 1 -- 172.21.15.40:0/3171762544 shutdown_connections 2023-07-18T03:29:53.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:53.392+0000 7fca50cb0700 1 -- 172.21.15.40:0/3171762544 wait complete. 2023-07-18T03:29:53.641 INFO:teuthology.orchestra.run.smithi040.stdout:55834574907 2023-07-18T03:29:53.641 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-07-18T03:29:53.974 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:53 smithi040 ceph-mon[96439]: pgmap v261: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-07-18T03:29:54.008 INFO:teuthology.orchestra.run.smithi040.stdout:107374182450 2023-07-18T03:29:54.009 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-07-18T03:29:54.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:53 smithi134 ceph-mon[98216]: pgmap v261: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-07-18T03:29:54.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:53 smithi064 ceph-mon[98059]: pgmap v261: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-07-18T03:29:54.210 INFO:teuthology.orchestra.run.smithi040.stdout:77309411381 2023-07-18T03:29:54.210 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-07-18T03:29:54.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.512+0000 7f252c7b4700 1 -- 172.21.15.40:0/2026905863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25240ffe00 msgr2=0x7f25241001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:54.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.512+0000 7f252c7b4700 1 --2- 172.21.15.40:0/2026905863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25240ffe00 0x7f25241001d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f251400b0d0 tx=0x7f251400b3e0 comp rx=0 tx=0).stop 2023-07-18T03:29:54.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.512+0000 7f252c7b4700 1 -- 172.21.15.40:0/2026905863 shutdown_connections 2023-07-18T03:29:54.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.512+0000 7f252c7b4700 1 --2- 172.21.15.40:0/2026905863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2524104ea0 0x7f252410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:54.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.512+0000 7f252c7b4700 1 --2- 172.21.15.40:0/2026905863 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25240ffe00 0x7f25241001d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:54.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.512+0000 7f252c7b4700 1 --2- 172.21.15.40:0/2026905863 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25241007a0 0x7f2524104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:54.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.512+0000 7f252c7b4700 1 -- 172.21.15.40:0/2026905863 >> 172.21.15.40:0/2026905863 conn(0x7f25240fb660 msgr2=0x7f25240fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:54.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.513+0000 7f252c7b4700 1 -- 172.21.15.40:0/2026905863 shutdown_connections 2023-07-18T03:29:54.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.513+0000 7f252c7b4700 1 -- 172.21.15.40:0/2026905863 wait complete. 2023-07-18T03:29:54.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.513+0000 7f252c7b4700 1 Processor -- start 2023-07-18T03:29:54.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.513+0000 7f252c7b4700 1 -- start start 2023-07-18T03:29:54.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.513+0000 7f252c7b4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25240ffe00 0x7f2524195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:54.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.513+0000 7f252c7b4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25241007a0 0x7f25241962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:54.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f252c7b4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2524104ea0 0x7f252419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:54.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f252c7b4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252419ad60 con 0x7f2524104ea0 2023-07-18T03:29:54.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f252c7b4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252419aed0 con 0x7f25241007a0 2023-07-18T03:29:54.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f252c7b4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252419b090 con 0x7f25240ffe00 2023-07-18T03:29:54.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f252ad51700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2524104ea0 0x7f252419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:54.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f252ad51700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2524104ea0 0x7f252419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56530/0 (socket says 172.21.15.40:56530) 2023-07-18T03:29:54.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f252ad51700 1 -- 172.21.15.40:0/568222312 learned_addr learned my addr 172.21.15.40:0/568222312 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:54.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f2529d4f700 1 --2- 172.21.15.40:0/568222312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25241007a0 0x7f25241962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:54.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f252a550700 1 --2- 172.21.15.40:0/568222312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25240ffe00 0x7f2524195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:54.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f252ad51700 1 -- 172.21.15.40:0/568222312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25240ffe00 msgr2=0x7f2524195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:54.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f252ad51700 1 --2- 172.21.15.40:0/568222312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25240ffe00 0x7f2524195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:54.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f252ad51700 1 -- 172.21.15.40:0/568222312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25241007a0 msgr2=0x7f25241962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:54.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f252ad51700 1 --2- 172.21.15.40:0/568222312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25241007a0 0x7f25241962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:54.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f252ad51700 1 -- 172.21.15.40:0/568222312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2514009d00 con 0x7f2524104ea0 2023-07-18T03:29:54.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f252a550700 1 --2- 172.21.15.40:0/568222312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25240ffe00 0x7f2524195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:29:54.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f252ad51700 1 --2- 172.21.15.40:0/568222312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2524104ea0 0x7f252419a750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f251c00d980 tx=0x7f251c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:54.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f25237fe700 1 -- 172.21.15.40:0/568222312 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c004a30 con 0x7f2524104ea0 2023-07-18T03:29:54.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f25237fe700 1 -- 172.21.15.40:0/568222312 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f251c00b5c0 con 0x7f2524104ea0 2023-07-18T03:29:54.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f252c7b4700 1 -- 172.21.15.40:0/568222312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252419b370 con 0x7f2524104ea0 2023-07-18T03:29:54.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f25237fe700 1 -- 172.21.15.40:0/568222312 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c009b50 con 0x7f2524104ea0 2023-07-18T03:29:54.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.514+0000 7f252c7b4700 1 -- 172.21.15.40:0/568222312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25241a2bd0 con 0x7f2524104ea0 2023-07-18T03:29:54.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.515+0000 7f252c7b4700 1 -- 172.21.15.40:0/568222312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f252410fcd0 con 0x7f2524104ea0 2023-07-18T03:29:54.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.517+0000 7f25237fe700 1 -- 172.21.15.40:0/568222312 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f251c009cb0 con 0x7f2524104ea0 2023-07-18T03:29:54.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.517+0000 7f25237fe700 1 --2- 172.21.15.40:0/568222312 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f250c0a11e0 0x7f250c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:54.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.518+0000 7f252a550700 1 --2- 172.21.15.40:0/568222312 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f250c0a11e0 0x7f250c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:54.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.518+0000 7f25237fe700 1 -- 172.21.15.40:0/568222312 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f251c0ccc80 con 0x7f2524104ea0 2023-07-18T03:29:54.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.518+0000 7f25237fe700 1 --2- 172.21.15.40:0/568222312 >> [v2:172.21.15.64:6824/2791256757,v1:172.21.15.64:6825/2791256757] conn(0x7f250c0aa9b0 0x7f250c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:54.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.518+0000 7f25237fe700 1 -- 172.21.15.40:0/568222312 --> [v2:172.21.15.64:6824/2791256757,v1:172.21.15.64:6825/2791256757] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f250c0ad470 con 0x7f250c0aa9b0 2023-07-18T03:29:54.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.518+0000 7f25237fe700 1 -- 172.21.15.40:0/568222312 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f251c0cd120 con 0x7f2524104ea0 2023-07-18T03:29:54.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.518+0000 7f252a550700 1 --2- 172.21.15.40:0/568222312 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f250c0a11e0 0x7f250c0a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2514000f80 tx=0x7f251400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:54.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.518+0000 7f2529d4f700 1 --2- 172.21.15.40:0/568222312 >> [v2:172.21.15.64:6824/2791256757,v1:172.21.15.64:6825/2791256757] conn(0x7f250c0aa9b0 0x7f250c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:54.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.519+0000 7f2529d4f700 1 --2- 172.21.15.40:0/568222312 >> [v2:172.21.15.64:6824/2791256757,v1:172.21.15.64:6825/2791256757] conn(0x7f250c0aa9b0 0x7f250c0acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:54.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.519+0000 7f25237fe700 1 -- 172.21.15.40:0/568222312 <== osd.10 v2:172.21.15.64:6824/2791256757 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f250c0ad470 con 0x7f250c0aa9b0 2023-07-18T03:29:54.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.540+0000 7f252c7b4700 1 -- 172.21.15.40:0/568222312 --> [v2:172.21.15.64:6824/2791256757,v1:172.21.15.64:6825/2791256757] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f252404e210 con 0x7f250c0aa9b0 2023-07-18T03:29:54.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.541+0000 7f25237fe700 1 -- 172.21.15.40:0/568222312 <== osd.10 v2:172.21.15.64:6824/2791256757 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f252404e210 con 0x7f250c0aa9b0 2023-07-18T03:29:54.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.542+0000 7f252c7b4700 1 -- 172.21.15.40:0/568222312 >> [v2:172.21.15.64:6824/2791256757,v1:172.21.15.64:6825/2791256757] conn(0x7f250c0aa9b0 msgr2=0x7f250c0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:54.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.542+0000 7f252c7b4700 1 --2- 172.21.15.40:0/568222312 >> [v2:172.21.15.64:6824/2791256757,v1:172.21.15.64:6825/2791256757] conn(0x7f250c0aa9b0 0x7f250c0acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:54.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.542+0000 7f252c7b4700 1 -- 172.21.15.40:0/568222312 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f250c0a11e0 msgr2=0x7f250c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:54.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.542+0000 7f252c7b4700 1 --2- 172.21.15.40:0/568222312 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f250c0a11e0 0x7f250c0a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2514000f80 tx=0x7f251400b040 comp rx=0 tx=0).stop 2023-07-18T03:29:54.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.542+0000 7f252c7b4700 1 -- 172.21.15.40:0/568222312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2524104ea0 msgr2=0x7f252419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:54.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.542+0000 7f252c7b4700 1 --2- 172.21.15.40:0/568222312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2524104ea0 0x7f252419a750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f251c00d980 tx=0x7f251c00dc90 comp rx=0 tx=0).stop 2023-07-18T03:29:54.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.542+0000 7f252ad51700 1 -- 172.21.15.40:0/568222312 reap_dead start 2023-07-18T03:29:54.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.543+0000 7f252c7b4700 1 -- 172.21.15.40:0/568222312 shutdown_connections 2023-07-18T03:29:54.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.543+0000 7f252c7b4700 1 -- 172.21.15.40:0/568222312 >> 172.21.15.40:0/568222312 conn(0x7f25240fb660 msgr2=0x7f2524103750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:54.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.543+0000 7f252c7b4700 1 -- 172.21.15.40:0/568222312 shutdown_connections 2023-07-18T03:29:54.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:54.543+0000 7f252c7b4700 1 -- 172.21.15.40:0/568222312 wait complete. 2023-07-18T03:29:55.000 INFO:teuthology.orchestra.run.smithi040.stdout:137438953517 2023-07-18T03:29:55.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-07-18T03:29:55.002 INFO:teuthology.orchestra.run.smithi040.stdout:188978561062 2023-07-18T03:29:55.003 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-07-18T03:29:55.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.318+0000 7f0afb1ca700 1 -- 172.21.15.40:0/967784088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af4104ab0 msgr2=0x7f0af4104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:55.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.318+0000 7f0afb1ca700 1 --2- 172.21.15.40:0/967784088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af4104ab0 0x7f0af4104e80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009990 tx=0x7f0ae8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:55.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.318+0000 7f0afb1ca700 1 -- 172.21.15.40:0/967784088 shutdown_connections 2023-07-18T03:29:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.318+0000 7f0afb1ca700 1 --2- 172.21.15.40:0/967784088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0af40ff460 0x7f0af410b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.318+0000 7f0afb1ca700 1 --2- 172.21.15.40:0/967784088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af4104ab0 0x7f0af4104e80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.318+0000 7f0afb1ca700 1 --2- 172.21.15.40:0/967784088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0af40feab0 0x7f0af40fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.318+0000 7f0afb1ca700 1 -- 172.21.15.40:0/967784088 >> 172.21.15.40:0/967784088 conn(0x7f0af4075b70 msgr2=0x7f0af4075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.318+0000 7f0afb1ca700 1 -- 172.21.15.40:0/967784088 shutdown_connections 2023-07-18T03:29:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.318+0000 7f0afb1ca700 1 -- 172.21.15.40:0/967784088 wait complete. 2023-07-18T03:29:55.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.319+0000 7f0afb1ca700 1 Processor -- start 2023-07-18T03:29:55.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.319+0000 7f0afb1ca700 1 -- start start 2023-07-18T03:29:55.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.319+0000 7f0afb1ca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af40feab0 0x7f0af419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:55.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.319+0000 7f0afb1ca700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0af40ff460 0x7f0af419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.319+0000 7f0afb1ca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0af4104ab0 0x7f0af4194960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.319+0000 7f0afb1ca700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af4194fd0 con 0x7f0af40feab0 2023-07-18T03:29:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.319+0000 7f0afb1ca700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af4195140 con 0x7f0af4104ab0 2023-07-18T03:29:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.319+0000 7f0afb1ca700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af4195300 con 0x7f0af40ff460 2023-07-18T03:29:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.319+0000 7f0af8f66700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af40feab0 0x7f0af419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.320+0000 7f0af8f66700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af40feab0 0x7f0af419a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56540/0 (socket says 172.21.15.40:56540) 2023-07-18T03:29:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.320+0000 7f0af8f66700 1 -- 172.21.15.40:0/680339678 learned_addr learned my addr 172.21.15.40:0/680339678 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.320+0000 7f0af9767700 1 --2- 172.21.15.40:0/680339678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0af4104ab0 0x7f0af4194960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.320+0000 7f0af3fff700 1 --2- 172.21.15.40:0/680339678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0af40ff460 0x7f0af419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.320+0000 7f0af8f66700 1 -- 172.21.15.40:0/680339678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0af40ff460 msgr2=0x7f0af419ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.320+0000 7f0af8f66700 1 --2- 172.21.15.40:0/680339678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0af40ff460 0x7f0af419ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.320+0000 7f0af8f66700 1 -- 172.21.15.40:0/680339678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0af4104ab0 msgr2=0x7f0af4194960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:55.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.320+0000 7f0af8f66700 1 --2- 172.21.15.40:0/680339678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0af4104ab0 0x7f0af4194960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.320+0000 7f0af8f66700 1 -- 172.21.15.40:0/680339678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae80095f0 con 0x7f0af40feab0 2023-07-18T03:29:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.320+0000 7f0af9767700 1 --2- 172.21.15.40:0/680339678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0af4104ab0 0x7f0af4194960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:29:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.320+0000 7f0af8f66700 1 --2- 172.21.15.40:0/680339678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af40feab0 0x7f0af419a8e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80098e0 tx=0x7f0ae800f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.320+0000 7f0af1ffb700 1 -- 172.21.15.40:0/680339678 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae801d070 con 0x7f0af40feab0 2023-07-18T03:29:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.320+0000 7f0af1ffb700 1 -- 172.21.15.40:0/680339678 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ae8005510 con 0x7f0af40feab0 2023-07-18T03:29:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.320+0000 7f0afb1ca700 1 -- 172.21.15.40:0/680339678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af4195580 con 0x7f0af40feab0 2023-07-18T03:29:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.320+0000 7f0af1ffb700 1 -- 172.21.15.40:0/680339678 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8004c90 con 0x7f0af40feab0 2023-07-18T03:29:55.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.320+0000 7f0afb1ca700 1 -- 172.21.15.40:0/680339678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af40681a0 con 0x7f0af40feab0 2023-07-18T03:29:55.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.321+0000 7f0afb1ca700 1 -- 172.21.15.40:0/680339678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0af404e210 con 0x7f0af40feab0 2023-07-18T03:29:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.322+0000 7f0af1ffb700 1 -- 172.21.15.40:0/680339678 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0ae800fc10 con 0x7f0af40feab0 2023-07-18T03:29:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.323+0000 7f0af1ffb700 1 --2- 172.21.15.40:0/680339678 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0ae40a11e0 0x7f0ae40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.323+0000 7f0af3fff700 1 --2- 172.21.15.40:0/680339678 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0ae40a11e0 0x7f0ae40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.323+0000 7f0af1ffb700 1 -- 172.21.15.40:0/680339678 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0ae80cdcb0 con 0x7f0af40feab0 2023-07-18T03:29:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.323+0000 7f0af1ffb700 1 --2- 172.21.15.40:0/680339678 >> [v2:172.21.15.134:6824/1938250059,v1:172.21.15.134:6825/1938250059] conn(0x7f0ae40aa9b0 0x7f0ae40acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.323+0000 7f0af1ffb700 1 -- 172.21.15.40:0/680339678 --> [v2:172.21.15.134:6824/1938250059,v1:172.21.15.134:6825/1938250059] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ae40ad470 con 0x7f0ae40aa9b0 2023-07-18T03:29:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.323+0000 7f0af1ffb700 1 -- 172.21.15.40:0/680339678 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ae80ce020 con 0x7f0af40feab0 2023-07-18T03:29:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.323+0000 7f0af3fff700 1 --2- 172.21.15.40:0/680339678 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0ae40a11e0 0x7f0ae40a3690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0005950 tx=0x7f0ae00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.324+0000 7f0af9767700 1 --2- 172.21.15.40:0/680339678 >> [v2:172.21.15.134:6824/1938250059,v1:172.21.15.134:6825/1938250059] conn(0x7f0ae40aa9b0 0x7f0ae40acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.324+0000 7f0af9767700 1 --2- 172.21.15.40:0/680339678 >> [v2:172.21.15.134:6824/1938250059,v1:172.21.15.134:6825/1938250059] conn(0x7f0ae40aa9b0 0x7f0ae40acdc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.325+0000 7f0af1ffb700 1 -- 172.21.15.40:0/680339678 <== osd.11 v2:172.21.15.134:6824/1938250059 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0ae40ad470 con 0x7f0ae40aa9b0 2023-07-18T03:29:55.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.340+0000 7f0afb1ca700 1 -- 172.21.15.40:0/680339678 --> [v2:172.21.15.134:6824/1938250059,v1:172.21.15.134:6825/1938250059] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0af40fd7d0 con 0x7f0ae40aa9b0 2023-07-18T03:29:55.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.341+0000 7f0af1ffb700 1 -- 172.21.15.40:0/680339678 <== osd.11 v2:172.21.15.134:6824/1938250059 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0af40fd7d0 con 0x7f0ae40aa9b0 2023-07-18T03:29:55.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.341+0000 7f0afb1ca700 1 -- 172.21.15.40:0/680339678 >> [v2:172.21.15.134:6824/1938250059,v1:172.21.15.134:6825/1938250059] conn(0x7f0ae40aa9b0 msgr2=0x7f0ae40acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:55.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.341+0000 7f0afb1ca700 1 --2- 172.21.15.40:0/680339678 >> [v2:172.21.15.134:6824/1938250059,v1:172.21.15.134:6825/1938250059] conn(0x7f0ae40aa9b0 0x7f0ae40acdc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:55.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.341+0000 7f0afb1ca700 1 -- 172.21.15.40:0/680339678 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0ae40a11e0 msgr2=0x7f0ae40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:55.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.341+0000 7f0afb1ca700 1 --2- 172.21.15.40:0/680339678 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0ae40a11e0 0x7f0ae40a3690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0005950 tx=0x7f0ae00058e0 comp rx=0 tx=0).stop 2023-07-18T03:29:55.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.341+0000 7f0afb1ca700 1 -- 172.21.15.40:0/680339678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af40feab0 msgr2=0x7f0af419a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:55.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.341+0000 7f0afb1ca700 1 --2- 172.21.15.40:0/680339678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af40feab0 0x7f0af419a8e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80098e0 tx=0x7f0ae800f720 comp rx=0 tx=0).stop 2023-07-18T03:29:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.341+0000 7f0af9767700 1 -- 172.21.15.40:0/680339678 reap_dead start 2023-07-18T03:29:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.341+0000 7f0afb1ca700 1 -- 172.21.15.40:0/680339678 shutdown_connections 2023-07-18T03:29:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.341+0000 7f0afb1ca700 1 -- 172.21.15.40:0/680339678 >> 172.21.15.40:0/680339678 conn(0x7f0af4075b70 msgr2=0x7f0af40fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.341+0000 7f0afb1ca700 1 -- 172.21.15.40:0/680339678 shutdown_connections 2023-07-18T03:29:55.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:55.341+0000 7f0afb1ca700 1 -- 172.21.15.40:0/680339678 wait complete. 2023-07-18T03:29:55.417 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:55.417 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:55.752 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:55.870 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:55 smithi040 ceph-mon[96439]: pgmap v262: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-07-18T03:29:55.871 INFO:teuthology.orchestra.run.smithi040.stdout:296352743445 2023-07-18T03:29:55.871 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-07-18T03:29:55.876 INFO:teuthology.orchestra.run.smithi040.stdout:326417514513 2023-07-18T03:29:55.876 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-07-18T03:29:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:55.995 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:56.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:55 smithi134 ceph-mon[98216]: pgmap v262: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-07-18T03:29:56.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:55 smithi064 ceph-mon[98059]: pgmap v262: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-07-18T03:29:56.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.273+0000 7f8f6a965700 1 -- 172.21.15.40:0/3545584667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f640feae0 msgr2=0x7f8f640fef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:56.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.273+0000 7f8f6a965700 1 --2- 172.21.15.40:0/3545584667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f640feae0 0x7f8f640fef50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009990 tx=0x7f8f58009ca0 comp rx=0 tx=0).stop 2023-07-18T03:29:56.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.274+0000 7f8f6a965700 1 -- 172.21.15.40:0/3545584667 shutdown_connections 2023-07-18T03:29:56.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.274+0000 7f8f6a965700 1 --2- 172.21.15.40:0/3545584667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f640ff490 0x7f8f6410b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:56.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.274+0000 7f8f6a965700 1 --2- 172.21.15.40:0/3545584667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f64104ae0 0x7f8f64104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:56.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.274+0000 7f8f6a965700 1 --2- 172.21.15.40:0/3545584667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f640feae0 0x7f8f640fef50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:56.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.274+0000 7f8f6a965700 1 -- 172.21.15.40:0/3545584667 >> 172.21.15.40:0/3545584667 conn(0x7f8f640759b0 msgr2=0x7f8f64075db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:56.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.274+0000 7f8f6a965700 1 -- 172.21.15.40:0/3545584667 shutdown_connections 2023-07-18T03:29:56.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.274+0000 7f8f6a965700 1 -- 172.21.15.40:0/3545584667 wait complete. 2023-07-18T03:29:56.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.274+0000 7f8f6a965700 1 Processor -- start 2023-07-18T03:29:56.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.274+0000 7f8f6a965700 1 -- start start 2023-07-18T03:29:56.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f6a965700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f640feae0 0x7f8f64197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:56.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f6a965700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f640ff490 0x7f8f641982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:56.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f6a965700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f64104ae0 0x7f8f6419c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:56.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f6a965700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6419cd50 con 0x7f8f640ff490 2023-07-18T03:29:56.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f6a965700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6419cec0 con 0x7f8f64104ae0 2023-07-18T03:29:56.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f6a965700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6419d080 con 0x7f8f640feae0 2023-07-18T03:29:56.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f637fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f640ff490 0x7f8f641982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:56.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f637fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f640ff490 0x7f8f641982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56558/0 (socket says 172.21.15.40:56558) 2023-07-18T03:29:56.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f637fe700 1 -- 172.21.15.40:0/3555982322 learned_addr learned my addr 172.21.15.40:0/3555982322 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:29:56.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f68f02700 1 --2- 172.21.15.40:0/3555982322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f64104ae0 0x7f8f6419c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:56.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f63fff700 1 --2- 172.21.15.40:0/3555982322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f640feae0 0x7f8f64197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:56.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f637fe700 1 -- 172.21.15.40:0/3555982322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f640feae0 msgr2=0x7f8f64197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:56.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f637fe700 1 --2- 172.21.15.40:0/3555982322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f640feae0 0x7f8f64197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:56.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f637fe700 1 -- 172.21.15.40:0/3555982322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f64104ae0 msgr2=0x7f8f6419c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:56.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f637fe700 1 --2- 172.21.15.40:0/3555982322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f64104ae0 0x7f8f6419c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:56.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f637fe700 1 -- 172.21.15.40:0/3555982322 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f580095f0 con 0x7f8f640ff490 2023-07-18T03:29:56.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f68f02700 1 --2- 172.21.15.40:0/3555982322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8f64104ae0 0x7f8f6419c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:29:56.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.275+0000 7f8f637fe700 1 --2- 172.21.15.40:0/3555982322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f640ff490 0x7f8f641982d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8f58000c00 tx=0x7f8f5800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:56.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.276+0000 7f8f617fa700 1 -- 172.21.15.40:0/3555982322 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5801d070 con 0x7f8f640ff490 2023-07-18T03:29:56.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.276+0000 7f8f6a965700 1 -- 172.21.15.40:0/3555982322 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6419d300 con 0x7f8f640ff490 2023-07-18T03:29:56.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.276+0000 7f8f6a965700 1 -- 172.21.15.40:0/3555982322 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f641a4bc0 con 0x7f8f640ff490 2023-07-18T03:29:56.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.276+0000 7f8f617fa700 1 -- 172.21.15.40:0/3555982322 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f5800fc30 con 0x7f8f640ff490 2023-07-18T03:29:56.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.276+0000 7f8f617fa700 1 -- 172.21.15.40:0/3555982322 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f58017770 con 0x7f8f640ff490 2023-07-18T03:29:56.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.277+0000 7f8f6a965700 1 -- 172.21.15.40:0/3555982322 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8f50000fc0 con 0x7f8f640ff490 2023-07-18T03:29:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.278+0000 7f8f617fa700 1 -- 172.21.15.40:0/3555982322 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8f580050d0 con 0x7f8f640ff490 2023-07-18T03:29:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.278+0000 7f8f617fa700 1 --2- 172.21.15.40:0/3555982322 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8f4c0a1260 0x7f8f4c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.279+0000 7f8f617fa700 1 -- 172.21.15.40:0/3555982322 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8f580cdd40 con 0x7f8f640ff490 2023-07-18T03:29:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.279+0000 7f8f63fff700 1 --2- 172.21.15.40:0/3555982322 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8f4c0a1260 0x7f8f4c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.279+0000 7f8f617fa700 1 --2- 172.21.15.40:0/3555982322 >> [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254] conn(0x7f8f4c0aaa30 0x7f8f4c0ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:29:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.279+0000 7f8f617fa700 1 -- 172.21.15.40:0/3555982322 --> [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f4c0ad4f0 con 0x7f8f4c0aaa30 2023-07-18T03:29:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.279+0000 7f8f617fa700 1 -- 172.21.15.40:0/3555982322 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f580ce1e0 con 0x7f8f640ff490 2023-07-18T03:29:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.279+0000 7f8f68f02700 1 --2- 172.21.15.40:0/3555982322 >> [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254] conn(0x7f8f4c0aaa30 0x7f8f4c0ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:29:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.279+0000 7f8f63fff700 1 --2- 172.21.15.40:0/3555982322 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8f4c0a1260 0x7f8f4c0a3710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8f54005fd0 tx=0x7f8f5400af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.279+0000 7f8f68f02700 1 --2- 172.21.15.40:0/3555982322 >> [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254] conn(0x7f8f4c0aaa30 0x7f8f4c0ace40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:29:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.280+0000 7f8f617fa700 1 -- 172.21.15.40:0/3555982322 <== osd.0 v2:172.21.15.40:6802/3043492254 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8f4c0ad4f0 con 0x7f8f4c0aaa30 2023-07-18T03:29:56.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.298+0000 7f8f6a965700 1 -- 172.21.15.40:0/3555982322 --> [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8f50002d40 con 0x7f8f4c0aaa30 2023-07-18T03:29:56.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.299+0000 7f8f617fa700 1 -- 172.21.15.40:0/3555982322 <== osd.0 v2:172.21.15.40:6802/3043492254 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8f50002d40 con 0x7f8f4c0aaa30 2023-07-18T03:29:56.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.299+0000 7f8f6a965700 1 -- 172.21.15.40:0/3555982322 >> [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254] conn(0x7f8f4c0aaa30 msgr2=0x7f8f4c0ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:56.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.299+0000 7f8f6a965700 1 --2- 172.21.15.40:0/3555982322 >> [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254] conn(0x7f8f4c0aaa30 0x7f8f4c0ace40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:29:56.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.299+0000 7f8f6a965700 1 -- 172.21.15.40:0/3555982322 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8f4c0a1260 msgr2=0x7f8f4c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:56.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.299+0000 7f8f6a965700 1 --2- 172.21.15.40:0/3555982322 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8f4c0a1260 0x7f8f4c0a3710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8f54005fd0 tx=0x7f8f5400af20 comp rx=0 tx=0).stop 2023-07-18T03:29:56.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.299+0000 7f8f6a965700 1 -- 172.21.15.40:0/3555982322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f640ff490 msgr2=0x7f8f641982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:29:56.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.299+0000 7f8f6a965700 1 --2- 172.21.15.40:0/3555982322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f640ff490 0x7f8f641982d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8f58000c00 tx=0x7f8f5800f790 comp rx=0 tx=0).stop 2023-07-18T03:29:56.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.299+0000 7f8f68f02700 1 -- 172.21.15.40:0/3555982322 reap_dead start 2023-07-18T03:29:56.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.299+0000 7f8f6a965700 1 -- 172.21.15.40:0/3555982322 shutdown_connections 2023-07-18T03:29:56.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.299+0000 7f8f6a965700 1 -- 172.21.15.40:0/3555982322 >> 172.21.15.40:0/3555982322 conn(0x7f8f640759b0 msgr2=0x7f8f64109ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:29:56.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.299+0000 7f8f6a965700 1 -- 172.21.15.40:0/3555982322 shutdown_connections 2023-07-18T03:29:56.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:29:56.299+0000 7f8f6a965700 1 -- 172.21.15.40:0/3555982322 wait complete. 2023-07-18T03:29:56.325 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:56.786 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:57.068 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:29:57.200 INFO:teuthology.orchestra.run.smithi040.stdout:34359738432 2023-07-18T03:29:57.200 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-07-18T03:29:58.051 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:57 smithi040 ceph-mon[96439]: pgmap v263: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-07-18T03:29:58.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:57 smithi134 ceph-mon[98216]: pgmap v263: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-07-18T03:29:58.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:57 smithi064 ceph-mon[98059]: pgmap v263: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-07-18T03:29:59.963 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:29:59 smithi040 ceph-mon[96439]: pgmap v264: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2023-07-18T03:30:00.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:29:59 smithi134 ceph-mon[98216]: pgmap v264: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2023-07-18T03:30:00.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:29:59 smithi064 ceph-mon[98059]: pgmap v264: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2023-07-18T03:30:00.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.152+0000 7f21eba4f700 1 -- 172.21.15.40:0/2614191696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e4101700 msgr2=0x7f21e4109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.152+0000 7f21eba4f700 1 --2- 172.21.15.40:0/2614191696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e4101700 0x7f21e4109c00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009990 tx=0x7f21d4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.152+0000 7f21eba4f700 1 -- 172.21.15.40:0/2614191696 shutdown_connections 2023-07-18T03:30:00.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.152+0000 7f21eba4f700 1 --2- 172.21.15.40:0/2614191696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21e410a140 0x7f21e410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.152+0000 7f21eba4f700 1 --2- 172.21.15.40:0/2614191696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e4100df0 0x7f21e41011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.152+0000 7f21eba4f700 1 --2- 172.21.15.40:0/2614191696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e4101700 0x7f21e4109c00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.152+0000 7f21eba4f700 1 -- 172.21.15.40:0/2614191696 >> 172.21.15.40:0/2614191696 conn(0x7f21e4075b70 msgr2=0x7f21e4075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:00.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.152+0000 7f21eba4f700 1 -- 172.21.15.40:0/2614191696 shutdown_connections 2023-07-18T03:30:00.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.153+0000 7f21eba4f700 1 -- 172.21.15.40:0/2614191696 wait complete. 2023-07-18T03:30:00.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.153+0000 7f21eba4f700 1 Processor -- start 2023-07-18T03:30:00.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.153+0000 7f21eba4f700 1 -- start start 2023-07-18T03:30:00.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.154+0000 7f21eba4f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e4100df0 0x7f21e419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:00.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.154+0000 7f21eba4f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e4101700 0x7f21e419bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:00.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.154+0000 7f21eba4f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21e410a140 0x7f21e4195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:00.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.154+0000 7f21eba4f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e41960d0 con 0x7f21e4100df0 2023-07-18T03:30:00.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.154+0000 7f21eba4f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e4196240 con 0x7f21e410a140 2023-07-18T03:30:00.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.154+0000 7f21eba4f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e4196400 con 0x7f21e4101700 2023-07-18T03:30:00.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.154+0000 7f21e97eb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e4100df0 0x7f21e419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:00.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.154+0000 7f21e97eb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e4100df0 0x7f21e419b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56572/0 (socket says 172.21.15.40:56572) 2023-07-18T03:30:00.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.154+0000 7f21e97eb700 1 -- 172.21.15.40:0/958686309 learned_addr learned my addr 172.21.15.40:0/958686309 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:00.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.154+0000 7f21e8fea700 1 --2- 172.21.15.40:0/958686309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e4101700 0x7f21e419bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:00.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.154+0000 7f21e9fec700 1 --2- 172.21.15.40:0/958686309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21e410a140 0x7f21e4195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:00.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.154+0000 7f21e97eb700 1 -- 172.21.15.40:0/958686309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e4101700 msgr2=0x7f21e419bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.154+0000 7f21e97eb700 1 --2- 172.21.15.40:0/958686309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e4101700 0x7f21e419bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.154+0000 7f21e97eb700 1 -- 172.21.15.40:0/958686309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21e410a140 msgr2=0x7f21e4195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.154+0000 7f21e97eb700 1 --2- 172.21.15.40:0/958686309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21e410a140 0x7f21e4195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.154+0000 7f21e97eb700 1 -- 172.21.15.40:0/958686309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d40095f0 con 0x7f21e4100df0 2023-07-18T03:30:00.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.154+0000 7f21e97eb700 1 --2- 172.21.15.40:0/958686309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e4100df0 0x7f21e419b9e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f21e000d950 tx=0x7f21e000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:00.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.155+0000 7f21da7fc700 1 -- 172.21.15.40:0/958686309 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e0009940 con 0x7f21e4100df0 2023-07-18T03:30:00.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.155+0000 7f21da7fc700 1 -- 172.21.15.40:0/958686309 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21e0010460 con 0x7f21e4100df0 2023-07-18T03:30:00.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.155+0000 7f21da7fc700 1 -- 172.21.15.40:0/958686309 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e000f620 con 0x7f21e4100df0 2023-07-18T03:30:00.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.155+0000 7f21eba4f700 1 -- 172.21.15.40:0/958686309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e41966e0 con 0x7f21e4100df0 2023-07-18T03:30:00.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.155+0000 7f21eba4f700 1 -- 172.21.15.40:0/958686309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e41a29d0 con 0x7f21e4100df0 2023-07-18T03:30:00.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.156+0000 7f21eba4f700 1 -- 172.21.15.40:0/958686309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e404e210 con 0x7f21e4100df0 2023-07-18T03:30:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.157+0000 7f21da7fc700 1 -- 172.21.15.40:0/958686309 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f21e00108d0 con 0x7f21e4100df0 2023-07-18T03:30:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.157+0000 7f21da7fc700 1 --2- 172.21.15.40:0/958686309 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f21d00a11e0 0x7f21d00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.157+0000 7f21e8fea700 1 --2- 172.21.15.40:0/958686309 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f21d00a11e0 0x7f21d00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.157+0000 7f21da7fc700 1 -- 172.21.15.40:0/958686309 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f21e00cc760 con 0x7f21e4100df0 2023-07-18T03:30:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.158+0000 7f21e8fea700 1 --2- 172.21.15.40:0/958686309 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f21d00a11e0 0x7f21d00a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f21e40fcea0 tx=0x7f21d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.162+0000 7f21da7fc700 1 -- 172.21.15.40:0/958686309 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f21e0095f00 con 0x7f21e4100df0 2023-07-18T03:30:00.243 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:30:00.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.288+0000 7f9899bdc700 1 -- 172.21.15.40:0/1292149809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98940fea00 msgr2=0x7f98940fedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.288+0000 7f9899bdc700 1 --2- 172.21.15.40:0/1292149809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98940fea00 0x7f98940fedd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9884009990 tx=0x7f9884009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.288+0000 7f9899bdc700 1 -- 172.21.15.40:0/1292149809 shutdown_connections 2023-07-18T03:30:00.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.288+0000 7f9899bdc700 1 --2- 172.21.15.40:0/1292149809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f989410c5b0 0x7f989410e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.288+0000 7f9899bdc700 1 --2- 172.21.15.40:0/1292149809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98940fea00 0x7f98940fedd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.288+0000 7f9899bdc700 1 --2- 172.21.15.40:0/1292149809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98940ff310 0x7f989410c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.288+0000 7f9899bdc700 1 -- 172.21.15.40:0/1292149809 >> 172.21.15.40:0/1292149809 conn(0x7f98940fa5f0 msgr2=0x7f98940fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:00.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.288+0000 7f9899bdc700 1 -- 172.21.15.40:0/1292149809 shutdown_connections 2023-07-18T03:30:00.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.288+0000 7f9899bdc700 1 -- 172.21.15.40:0/1292149809 wait complete. 2023-07-18T03:30:00.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.289+0000 7f9899bdc700 1 Processor -- start 2023-07-18T03:30:00.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.289+0000 7f9899bdc700 1 -- start start 2023-07-18T03:30:00.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.289+0000 7f9899bdc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98940fea00 0x7f9894197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:00.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.289+0000 7f9899bdc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98940ff310 0x7f98941984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.289+0000 7f9899bdc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f989410c5b0 0x7f989419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.289+0000 7f9899bdc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989419cf30 con 0x7f98940fea00 2023-07-18T03:30:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.289+0000 7f9899bdc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989419d0a0 con 0x7f98940ff310 2023-07-18T03:30:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.289+0000 7f9899bdc700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989419d260 con 0x7f989410c5b0 2023-07-18T03:30:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.289+0000 7f98937fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98940fea00 0x7f9894197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.289+0000 7f98937fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98940fea00 0x7f9894197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56586/0 (socket says 172.21.15.40:56586) 2023-07-18T03:30:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.289+0000 7f98937fe700 1 -- 172.21.15.40:0/949267448 learned_addr learned my addr 172.21.15.40:0/949267448 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:00.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.289+0000 7f9893fff700 1 --2- 172.21.15.40:0/949267448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f989410c5b0 0x7f989419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:00.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.290+0000 7f9892ffd700 1 --2- 172.21.15.40:0/949267448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98940ff310 0x7f98941984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:00.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.290+0000 7f9893fff700 1 -- 172.21.15.40:0/949267448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98940ff310 msgr2=0x7f98941984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.290+0000 7f9893fff700 1 --2- 172.21.15.40:0/949267448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98940ff310 0x7f98941984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.290+0000 7f9893fff700 1 -- 172.21.15.40:0/949267448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98940fea00 msgr2=0x7f9894197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.290+0000 7f9893fff700 1 --2- 172.21.15.40:0/949267448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98940fea00 0x7f9894197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.290+0000 7f9893fff700 1 -- 172.21.15.40:0/949267448 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98840095f0 con 0x7f989410c5b0 2023-07-18T03:30:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.290+0000 7f98937fe700 1 --2- 172.21.15.40:0/949267448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98940fea00 0x7f9894197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.290+0000 7f9893fff700 1 --2- 172.21.15.40:0/949267448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f989410c5b0 0x7f989419c920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9894100be0 tx=0x7f988c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.290+0000 7f9890ff9700 1 -- 172.21.15.40:0/949267448 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c00ddd0 con 0x7f989410c5b0 2023-07-18T03:30:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.290+0000 7f9899bdc700 1 -- 172.21.15.40:0/949267448 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f989419d540 con 0x7f989410c5b0 2023-07-18T03:30:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.290+0000 7f9890ff9700 1 -- 172.21.15.40:0/949267448 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f988c00f040 con 0x7f989410c5b0 2023-07-18T03:30:00.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.290+0000 7f9890ff9700 1 -- 172.21.15.40:0/949267448 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c0059f0 con 0x7f989410c5b0 2023-07-18T03:30:00.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.290+0000 7f9899bdc700 1 -- 172.21.15.40:0/949267448 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989406a100 con 0x7f989410c5b0 2023-07-18T03:30:00.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.292+0000 7f9899bdc700 1 -- 172.21.15.40:0/949267448 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9880005320 con 0x7f989410c5b0 2023-07-18T03:30:00.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.293+0000 7f9890ff9700 1 -- 172.21.15.40:0/949267448 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f988c005020 con 0x7f989410c5b0 2023-07-18T03:30:00.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.294+0000 7f5df264c700 1 -- 172.21.15.40:0/1926145404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dec06ccb0 msgr2=0x7f5dec10cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.294+0000 7f5df264c700 1 --2- 172.21.15.40:0/1926145404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dec06ccb0 0x7f5dec10cad0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009990 tx=0x7f5ddc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.294+0000 7f5df264c700 1 -- 172.21.15.40:0/1926145404 shutdown_connections 2023-07-18T03:30:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.294+0000 7f5df264c700 1 --2- 172.21.15.40:0/1926145404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dec107510 0x7f5dec107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.294+0000 7f5df264c700 1 --2- 172.21.15.40:0/1926145404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dec06ccb0 0x7f5dec10cad0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.294+0000 7f5df264c700 1 --2- 172.21.15.40:0/1926145404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dec06d080 0x7f5dec10d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.294+0000 7f5df264c700 1 -- 172.21.15.40:0/1926145404 >> 172.21.15.40:0/1926145404 conn(0x7f5dec06be40 msgr2=0x7f5dec071020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.294+0000 7f5df264c700 1 -- 172.21.15.40:0/1926145404 shutdown_connections 2023-07-18T03:30:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.294+0000 7f5df264c700 1 -- 172.21.15.40:0/1926145404 wait complete. 2023-07-18T03:30:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.294+0000 7f9890ff9700 1 --2- 172.21.15.40:0/949267448 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f987c0a11e0 0x7f987c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.294+0000 7f98937fe700 1 --2- 172.21.15.40:0/949267448 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f987c0a11e0 0x7f987c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.294+0000 7f9890ff9700 1 -- 172.21.15.40:0/949267448 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f988c0cc100 con 0x7f989410c5b0 2023-07-18T03:30:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.295+0000 7f5df264c700 1 Processor -- start 2023-07-18T03:30:00.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.295+0000 7f5df264c700 1 -- start start 2023-07-18T03:30:00.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.295+0000 7f98937fe700 1 --2- 172.21.15.40:0/949267448 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f987c0a11e0 0x7f987c0a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f988400bc60 tx=0x7f9884005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:00.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.295+0000 7f5df264c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dec06ccb0 0x7f5dec11a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:00.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.295+0000 7f5df264c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dec06d080 0x7f5dec11a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:00.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.295+0000 7f5df264c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dec107510 0x7f5dec117420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:00.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.295+0000 7f5df264c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec117bf0 con 0x7f5dec06ccb0 2023-07-18T03:30:00.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.295+0000 7f5df264c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec117d60 con 0x7f5dec107510 2023-07-18T03:30:00.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.295+0000 7f5df264c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec117f20 con 0x7f5dec06d080 2023-07-18T03:30:00.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.295+0000 7f5debfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dec06ccb0 0x7f5dec11a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:00.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.295+0000 7f5debfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dec06ccb0 0x7f5dec11a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56600/0 (socket says 172.21.15.40:56600) 2023-07-18T03:30:00.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.295+0000 7f5debfff700 1 -- 172.21.15.40:0/2683973622 learned_addr learned my addr 172.21.15.40:0/2683973622 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:00.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.295+0000 7f5de3fff700 1 --2- 172.21.15.40:0/2683973622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dec06d080 0x7f5dec11a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:00.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.295+0000 7f5df0be9700 1 --2- 172.21.15.40:0/2683973622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dec107510 0x7f5dec117420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:00.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.295+0000 7f5debfff700 1 -- 172.21.15.40:0/2683973622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dec06d080 msgr2=0x7f5dec11a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.295+0000 7f5debfff700 1 --2- 172.21.15.40:0/2683973622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dec06d080 0x7f5dec11a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.295+0000 7f5debfff700 1 -- 172.21.15.40:0/2683973622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dec107510 msgr2=0x7f5dec117420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.296+0000 7f5debfff700 1 --2- 172.21.15.40:0/2683973622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dec107510 0x7f5dec117420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.296+0000 7f5debfff700 1 -- 172.21.15.40:0/2683973622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc0095f0 con 0x7f5dec06ccb0 2023-07-18T03:30:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.296+0000 7f5df0be9700 1 --2- 172.21.15.40:0/2683973622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dec107510 0x7f5dec117420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.296+0000 7f5debfff700 1 --2- 172.21.15.40:0/2683973622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dec06ccb0 0x7f5dec11a330 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0095c0 tx=0x7f5ddc00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.296+0000 7f5de9ffb700 1 -- 172.21.15.40:0/2683973622 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc01d070 con 0x7f5dec06ccb0 2023-07-18T03:30:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.296+0000 7f5df264c700 1 -- 172.21.15.40:0/2683973622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dec1181a0 con 0x7f5dec06ccb0 2023-07-18T03:30:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.296+0000 7f5de9ffb700 1 -- 172.21.15.40:0/2683973622 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ddc00fb50 con 0x7f5dec06ccb0 2023-07-18T03:30:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.296+0000 7f5de9ffb700 1 -- 172.21.15.40:0/2683973622 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc0173f0 con 0x7f5dec06ccb0 2023-07-18T03:30:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.296+0000 7f5df264c700 1 -- 172.21.15.40:0/2683973622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dec1a6fe0 con 0x7f5dec06ccb0 2023-07-18T03:30:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.297+0000 7f5df264c700 1 -- 172.21.15.40:0/2683973622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dec04e210 con 0x7f5dec06ccb0 2023-07-18T03:30:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.298+0000 7f5de9ffb700 1 -- 172.21.15.40:0/2683973622 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5ddc00fcc0 con 0x7f5dec06ccb0 2023-07-18T03:30:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.299+0000 7f9890ff9700 1 -- 172.21.15.40:0/949267448 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f988c04e790 con 0x7f989410c5b0 2023-07-18T03:30:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.299+0000 7f5de9ffb700 1 --2- 172.21.15.40:0/2683973622 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5dd80a11e0 0x7f5dd80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.299+0000 7f5de3fff700 1 --2- 172.21.15.40:0/2683973622 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5dd80a11e0 0x7f5dd80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.299+0000 7f5de9ffb700 1 -- 172.21.15.40:0/2683973622 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5ddc0cdad0 con 0x7f5dec06ccb0 2023-07-18T03:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.300+0000 7f5de3fff700 1 --2- 172.21.15.40:0/2683973622 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5dd80a11e0 0x7f5dd80a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5dec118cc0 tx=0x7f5dd400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.304+0000 7f5de9ffb700 1 -- 172.21.15.40:0/2683973622 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5ddc05c790 con 0x7f5dec06ccb0 2023-07-18T03:30:00.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.409+0000 7f33d00df700 1 -- 172.21.15.40:0/1591272755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33c80febe0 msgr2=0x7f33c80fefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.409+0000 7f33d00df700 1 --2- 172.21.15.40:0/1591272755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33c80febe0 0x7f33c80fefb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009990 tx=0x7f33c4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.409+0000 7f33d00df700 1 -- 172.21.15.40:0/1591272755 shutdown_connections 2023-07-18T03:30:00.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.409+0000 7f33d00df700 1 --2- 172.21.15.40:0/1591272755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33c8103f10 0x7f33c8104340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.409+0000 7f33d00df700 1 --2- 172.21.15.40:0/1591272755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33c80febe0 0x7f33c80fefb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.409+0000 7f33d00df700 1 --2- 172.21.15.40:0/1591272755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33c80ff4f0 0x7f33c81039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.409+0000 7f33d00df700 1 -- 172.21.15.40:0/1591272755 >> 172.21.15.40:0/1591272755 conn(0x7f33c8075b70 msgr2=0x7f33c8075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:00.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.410+0000 7f33d00df700 1 -- 172.21.15.40:0/1591272755 shutdown_connections 2023-07-18T03:30:00.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.410+0000 7f33d00df700 1 -- 172.21.15.40:0/1591272755 wait complete. 2023-07-18T03:30:00.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.410+0000 7f33d00df700 1 Processor -- start 2023-07-18T03:30:00.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.410+0000 7f33d00df700 1 -- start start 2023-07-18T03:30:00.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.410+0000 7f33d00df700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33c80febe0 0x7f33c819c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:00.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.410+0000 7f33d00df700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33c80ff4f0 0x7f33c819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:00.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.410+0000 7f33d00df700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33c8103f10 0x7f33c81a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:00.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.411+0000 7f33d00df700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c81a1260 con 0x7f33c80ff4f0 2023-07-18T03:30:00.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.411+0000 7f33d00df700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c81a13d0 con 0x7f33c80febe0 2023-07-18T03:30:00.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.411+0000 7f33d00df700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c81a1590 con 0x7f33c8103f10 2023-07-18T03:30:00.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.411+0000 7f33cd67a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33c80ff4f0 0x7f33c819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:00.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.411+0000 7f33cd67a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33c80ff4f0 0x7f33c819c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56618/0 (socket says 172.21.15.40:56618) 2023-07-18T03:30:00.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.411+0000 7f33cd67a700 1 -- 172.21.15.40:0/924050675 learned_addr learned my addr 172.21.15.40:0/924050675 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:00.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.411+0000 7f33cd67a700 1 -- 172.21.15.40:0/924050675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33c8103f10 msgr2=0x7f33c81a0c50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:30:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.411+0000 7f33cd67a700 1 --2- 172.21.15.40:0/924050675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33c8103f10 0x7f33c81a0c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.411+0000 7f33cd67a700 1 -- 172.21.15.40:0/924050675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33c80febe0 msgr2=0x7f33c819c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.411+0000 7f33cd67a700 1 --2- 172.21.15.40:0/924050675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33c80febe0 0x7f33c819c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.411+0000 7f33cd67a700 1 -- 172.21.15.40:0/924050675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c40095f0 con 0x7f33c80ff4f0 2023-07-18T03:30:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.411+0000 7f33cd67a700 1 --2- 172.21.15.40:0/924050675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33c80ff4f0 0x7f33c819c7e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f33b800e9a0 tx=0x7f33b800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.411+0000 7f33beffd700 1 -- 172.21.15.40:0/924050675 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b800cbd0 con 0x7f33c80ff4f0 2023-07-18T03:30:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.412+0000 7f33beffd700 1 -- 172.21.15.40:0/924050675 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33b800cd30 con 0x7f33c80ff4f0 2023-07-18T03:30:00.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.412+0000 7f33beffd700 1 -- 172.21.15.40:0/924050675 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b8018680 con 0x7f33c80ff4f0 2023-07-18T03:30:00.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.412+0000 7f33d00df700 1 -- 172.21.15.40:0/924050675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c81a1870 con 0x7f33c80ff4f0 2023-07-18T03:30:00.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.412+0000 7f33d00df700 1 -- 172.21.15.40:0/924050675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c81a90d0 con 0x7f33c80ff4f0 2023-07-18T03:30:00.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.413+0000 7f33d00df700 1 -- 172.21.15.40:0/924050675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c804e210 con 0x7f33c80ff4f0 2023-07-18T03:30:00.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.414+0000 7f33beffd700 1 -- 172.21.15.40:0/924050675 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f33b80187e0 con 0x7f33c80ff4f0 2023-07-18T03:30:00.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.415+0000 7f33beffd700 1 --2- 172.21.15.40:0/924050675 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33b40a1330 0x7f33b40a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:00.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.415+0000 7f33cde7b700 1 --2- 172.21.15.40:0/924050675 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33b40a1330 0x7f33b40a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:00.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.415+0000 7f33beffd700 1 -- 172.21.15.40:0/924050675 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f33b8014070 con 0x7f33c80ff4f0 2023-07-18T03:30:00.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.415+0000 7f33cde7b700 1 --2- 172.21.15.40:0/924050675 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33b40a1330 0x7f33b40a37e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f33c4006010 tx=0x7f33c4005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:00.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.423+0000 7f33beffd700 1 -- 172.21.15.40:0/924050675 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f33b8051790 con 0x7f33c80ff4f0 2023-07-18T03:30:00.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.454+0000 7f21eba4f700 1 -- 172.21.15.40:0/958686309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f21e41a2c00 con 0x7f21e4100df0 2023-07-18T03:30:00.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.454+0000 7f21da7fc700 1 -- 172.21.15.40:0/958686309 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f21e0099520 con 0x7f21e4100df0 2023-07-18T03:30:00.451 INFO:teuthology.orchestra.run.smithi040.stdout:107374182451 2023-07-18T03:30:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.457+0000 7f21eba4f700 1 -- 172.21.15.40:0/958686309 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f21d00a11e0 msgr2=0x7f21d00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.457+0000 7f21eba4f700 1 --2- 172.21.15.40:0/958686309 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f21d00a11e0 0x7f21d00a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f21e40fcea0 tx=0x7f21d4005fd0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.457+0000 7f21eba4f700 1 -- 172.21.15.40:0/958686309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e4100df0 msgr2=0x7f21e419b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.457+0000 7f21eba4f700 1 --2- 172.21.15.40:0/958686309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e4100df0 0x7f21e419b9e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f21e000d950 tx=0x7f21e000dc60 comp rx=0 tx=0).stop 2023-07-18T03:30:00.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.457+0000 7f21eba4f700 1 -- 172.21.15.40:0/958686309 shutdown_connections 2023-07-18T03:30:00.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.457+0000 7f21eba4f700 1 --2- 172.21.15.40:0/958686309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21e410a140 0x7f21e4195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.457+0000 7f21eba4f700 1 --2- 172.21.15.40:0/958686309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e4100df0 0x7f21e419b9e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.457+0000 7f21eba4f700 1 --2- 172.21.15.40:0/958686309 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f21d00a11e0 0x7f21d00a3690 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.457+0000 7f21eba4f700 1 --2- 172.21.15.40:0/958686309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21e4101700 0x7f21e419bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.457+0000 7f21eba4f700 1 -- 172.21.15.40:0/958686309 >> 172.21.15.40:0/958686309 conn(0x7f21e4075b70 msgr2=0x7f21e40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:00.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.457+0000 7f21eba4f700 1 -- 172.21.15.40:0/958686309 shutdown_connections 2023-07-18T03:30:00.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.457+0000 7f21eba4f700 1 -- 172.21.15.40:0/958686309 wait complete. 2023-07-18T03:30:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.521+0000 7f9899bdc700 1 -- 172.21.15.40:0/949267448 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9880005cc0 con 0x7f989410c5b0 2023-07-18T03:30:00.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.521+0000 7f9890ff9700 1 -- 172.21.15.40:0/949267448 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f988c095020 con 0x7f989410c5b0 2023-07-18T03:30:00.518 INFO:teuthology.orchestra.run.smithi040.stdout:55834574908 2023-07-18T03:30:00.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.523+0000 7f9899bdc700 1 -- 172.21.15.40:0/949267448 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f987c0a11e0 msgr2=0x7f987c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.523+0000 7f9899bdc700 1 --2- 172.21.15.40:0/949267448 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f987c0a11e0 0x7f987c0a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f988400bc60 tx=0x7f9884005960 comp rx=0 tx=0).stop 2023-07-18T03:30:00.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.523+0000 7f9899bdc700 1 -- 172.21.15.40:0/949267448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f989410c5b0 msgr2=0x7f989419c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.523+0000 7f9899bdc700 1 --2- 172.21.15.40:0/949267448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f989410c5b0 0x7f989419c920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9894100be0 tx=0x7f988c00d3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.524+0000 7f9899bdc700 1 -- 172.21.15.40:0/949267448 shutdown_connections 2023-07-18T03:30:00.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.524+0000 7f9899bdc700 1 --2- 172.21.15.40:0/949267448 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f987c0a11e0 0x7f987c0a3690 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.524+0000 7f9899bdc700 1 --2- 172.21.15.40:0/949267448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f989410c5b0 0x7f989419c920 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.524+0000 7f9899bdc700 1 --2- 172.21.15.40:0/949267448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98940fea00 0x7f9894197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.524+0000 7f9899bdc700 1 --2- 172.21.15.40:0/949267448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98940ff310 0x7f98941984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.524+0000 7f9899bdc700 1 -- 172.21.15.40:0/949267448 >> 172.21.15.40:0/949267448 conn(0x7f98940fa5f0 msgr2=0x7f989410d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:00.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.524+0000 7f9899bdc700 1 -- 172.21.15.40:0/949267448 shutdown_connections 2023-07-18T03:30:00.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.524+0000 7f9899bdc700 1 -- 172.21.15.40:0/949267448 wait complete. 2023-07-18T03:30:00.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.527+0000 7f5df264c700 1 -- 172.21.15.40:0/2683973622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5dec1111e0 con 0x7f5dec06ccb0 2023-07-18T03:30:00.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.527+0000 7f5de9ffb700 1 -- 172.21.15.40:0/2683973622 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5ddc027090 con 0x7f5dec06ccb0 2023-07-18T03:30:00.524 INFO:teuthology.orchestra.run.smithi040.stdout:77309411382 2023-07-18T03:30:00.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.530+0000 7f5df264c700 1 -- 172.21.15.40:0/2683973622 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5dd80a11e0 msgr2=0x7f5dd80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.530+0000 7f5df264c700 1 --2- 172.21.15.40:0/2683973622 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5dd80a11e0 0x7f5dd80a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5dec118cc0 tx=0x7f5dd400b410 comp rx=0 tx=0).stop 2023-07-18T03:30:00.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.530+0000 7f5df264c700 1 -- 172.21.15.40:0/2683973622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dec06ccb0 msgr2=0x7f5dec11a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.530+0000 7f5df264c700 1 --2- 172.21.15.40:0/2683973622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dec06ccb0 0x7f5dec11a330 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0095c0 tx=0x7f5ddc00f710 comp rx=0 tx=0).stop 2023-07-18T03:30:00.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.530+0000 7f5df264c700 1 -- 172.21.15.40:0/2683973622 shutdown_connections 2023-07-18T03:30:00.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.530+0000 7f5df264c700 1 --2- 172.21.15.40:0/2683973622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5dec107510 0x7f5dec117420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.530+0000 7f5df264c700 1 --2- 172.21.15.40:0/2683973622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dec06ccb0 0x7f5dec11a330 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.530+0000 7f5df264c700 1 --2- 172.21.15.40:0/2683973622 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5dd80a11e0 0x7f5dd80a3690 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.530+0000 7f5df264c700 1 --2- 172.21.15.40:0/2683973622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5dec06d080 0x7f5dec11a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.530+0000 7f5df264c700 1 -- 172.21.15.40:0/2683973622 >> 172.21.15.40:0/2683973622 conn(0x7f5dec06be40 msgr2=0x7f5dec10e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:00.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.530+0000 7f5df264c700 1 -- 172.21.15.40:0/2683973622 shutdown_connections 2023-07-18T03:30:00.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.530+0000 7f5df264c700 1 -- 172.21.15.40:0/2683973622 wait complete. 2023-07-18T03:30:00.556 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:30:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:30:00.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.658+0000 7f33d00df700 1 -- 172.21.15.40:0/924050675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f33c81a9300 con 0x7f33c80ff4f0 2023-07-18T03:30:00.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.659+0000 7f33beffd700 1 -- 172.21.15.40:0/924050675 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f33b8099d60 con 0x7f33c80ff4f0 2023-07-18T03:30:00.656 INFO:teuthology.orchestra.run.smithi040.stdout:188978561063 2023-07-18T03:30:00.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.664+0000 7f33d00df700 1 -- 172.21.15.40:0/924050675 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33b40a1330 msgr2=0x7f33b40a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.664+0000 7f33d00df700 1 --2- 172.21.15.40:0/924050675 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33b40a1330 0x7f33b40a37e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f33c4006010 tx=0x7f33c4005e80 comp rx=0 tx=0).stop 2023-07-18T03:30:00.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.664+0000 7f33d00df700 1 -- 172.21.15.40:0/924050675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33c80ff4f0 msgr2=0x7f33c819c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:00.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.664+0000 7f33d00df700 1 --2- 172.21.15.40:0/924050675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33c80ff4f0 0x7f33c819c7e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f33b800e9a0 tx=0x7f33b800ed60 comp rx=0 tx=0).stop 2023-07-18T03:30:00.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.664+0000 7f33d00df700 1 -- 172.21.15.40:0/924050675 shutdown_connections 2023-07-18T03:30:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.664+0000 7f33d00df700 1 --2- 172.21.15.40:0/924050675 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33b40a1330 0x7f33b40a37e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.664+0000 7f33d00df700 1 --2- 172.21.15.40:0/924050675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33c8103f10 0x7f33c81a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.664+0000 7f33d00df700 1 --2- 172.21.15.40:0/924050675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33c80febe0 0x7f33c819c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.664+0000 7f33d00df700 1 --2- 172.21.15.40:0/924050675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33c80ff4f0 0x7f33c819c7e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.664+0000 7f33d00df700 1 -- 172.21.15.40:0/924050675 >> 172.21.15.40:0/924050675 conn(0x7f33c8075b70 msgr2=0x7f33c8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.664+0000 7f33d00df700 1 -- 172.21.15.40:0/924050675 shutdown_connections 2023-07-18T03:30:00.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:00.664+0000 7f33d00df700 1 -- 172.21.15.40:0/924050675 wait complete. 2023-07-18T03:30:00.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:00 smithi040 ceph-mon[96439]: overall HEALTH_OK 2023-07-18T03:30:00.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:00 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/958686309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-18T03:30:00.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:00 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/949267448' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-18T03:30:00.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:00 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2683973622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-18T03:30:00.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:00 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/924050675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-18T03:30:01.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:00 smithi134 ceph-mon[98216]: overall HEALTH_OK 2023-07-18T03:30:01.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:00 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/958686309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-18T03:30:01.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:00 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/949267448' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-18T03:30:01.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:00 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2683973622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-18T03:30:01.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:00 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/924050675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-18T03:30:01.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:00 smithi064 ceph-mon[98059]: overall HEALTH_OK 2023-07-18T03:30:01.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:00 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/958686309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-18T03:30:01.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:00 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/949267448' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-18T03:30:01.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:00 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2683973622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-18T03:30:01.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:00 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/924050675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-18T03:30:01.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.089+0000 7f68054ee700 1 -- 172.21.15.40:0/3580341919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68000ff440 msgr2=0x7f680010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:01.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.089+0000 7f68054ee700 1 --2- 172.21.15.40:0/3580341919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68000ff440 0x7f680010b8d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f67f800b0a0 tx=0x7f67f800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:01.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.090+0000 7f68054ee700 1 -- 172.21.15.40:0/3580341919 shutdown_connections 2023-07-18T03:30:01.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.090+0000 7f68054ee700 1 --2- 172.21.15.40:0/3580341919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68000ff440 0x7f680010b8d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:01.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.090+0000 7f68054ee700 1 --2- 172.21.15.40:0/3580341919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6800104a90 0x7f6800104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:01.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.090+0000 7f68054ee700 1 --2- 172.21.15.40:0/3580341919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68000fea90 0x7f68000fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:01.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.090+0000 7f68054ee700 1 -- 172.21.15.40:0/3580341919 >> 172.21.15.40:0/3580341919 conn(0x7f68000fa570 msgr2=0x7f68000fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:01.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.090+0000 7f68054ee700 1 -- 172.21.15.40:0/3580341919 shutdown_connections 2023-07-18T03:30:01.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.090+0000 7f68054ee700 1 -- 172.21.15.40:0/3580341919 wait complete. 2023-07-18T03:30:01.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.090+0000 7f68054ee700 1 Processor -- start 2023-07-18T03:30:01.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.090+0000 7f68054ee700 1 -- start start 2023-07-18T03:30:01.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f68054ee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68000fea90 0x7f6800196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:01.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f68054ee700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68000ff440 0x7f6800196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:01.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f68054ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6800104a90 0x7f68001a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:01.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f68054ee700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68001978d0 con 0x7f6800104a90 2023-07-18T03:30:01.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f68054ee700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68001a2040 con 0x7f68000fea90 2023-07-18T03:30:01.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f68054ee700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68001a21b0 con 0x7f68000ff440 2023-07-18T03:30:01.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f67ff7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6800104a90 0x7f68001a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:01.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f67feffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68000fea90 0x7f6800196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:01.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f67ff7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6800104a90 0x7f68001a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56632/0 (socket says 172.21.15.40:56632) 2023-07-18T03:30:01.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f67ff7fe700 1 -- 172.21.15.40:0/3689104652 learned_addr learned my addr 172.21.15.40:0/3689104652 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:01.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f67fe7fc700 1 --2- 172.21.15.40:0/3689104652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68000ff440 0x7f6800196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:01.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f67ff7fe700 1 -- 172.21.15.40:0/3689104652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68000ff440 msgr2=0x7f6800196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:01.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f67ff7fe700 1 --2- 172.21.15.40:0/3689104652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68000ff440 0x7f6800196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:01.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f67ff7fe700 1 -- 172.21.15.40:0/3689104652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68000fea90 msgr2=0x7f6800196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:01.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f67ff7fe700 1 --2- 172.21.15.40:0/3689104652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68000fea90 0x7f6800196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:01.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f67ff7fe700 1 -- 172.21.15.40:0/3689104652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f8009d00 con 0x7f6800104a90 2023-07-18T03:30:01.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f67ff7fe700 1 --2- 172.21.15.40:0/3689104652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6800104a90 0x7f68001a1b00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f67f80062a0 tx=0x7f67f800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:01.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f67e7fff700 1 -- 172.21.15.40:0/3689104652 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f800e040 con 0x7f6800104a90 2023-07-18T03:30:01.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f67e7fff700 1 -- 172.21.15.40:0/3689104652 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67f8004020 con 0x7f6800104a90 2023-07-18T03:30:01.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f68054ee700 1 -- 172.21.15.40:0/3689104652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68001a2430 con 0x7f6800104a90 2023-07-18T03:30:01.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.091+0000 7f67e7fff700 1 -- 172.21.15.40:0/3689104652 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f8004ea0 con 0x7f6800104a90 2023-07-18T03:30:01.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.092+0000 7f68054ee700 1 -- 172.21.15.40:0/3689104652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68001a28a0 con 0x7f6800104a90 2023-07-18T03:30:01.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.092+0000 7f68054ee700 1 -- 172.21.15.40:0/3689104652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680004e210 con 0x7f6800104a90 2023-07-18T03:30:01.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.094+0000 7f67e7fff700 1 -- 172.21.15.40:0/3689104652 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f67f8004470 con 0x7f6800104a90 2023-07-18T03:30:01.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.095+0000 7f67e7fff700 1 --2- 172.21.15.40:0/3689104652 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67e80a1260 0x7f67e80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:01.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.095+0000 7f67feffd700 1 --2- 172.21.15.40:0/3689104652 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67e80a1260 0x7f67e80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:01.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.095+0000 7f67e7fff700 1 -- 172.21.15.40:0/3689104652 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f67f80ce9e0 con 0x7f6800104a90 2023-07-18T03:30:01.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.095+0000 7f67feffd700 1 --2- 172.21.15.40:0/3689104652 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67e80a1260 0x7f67e80a3710 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f68000ffbd0 tx=0x7f67f000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:01.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.098+0000 7f67e7fff700 1 -- 172.21.15.40:0/3689104652 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f67f8098110 con 0x7f6800104a90 2023-07-18T03:30:01.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.321+0000 7f68054ee700 1 -- 172.21.15.40:0/3689104652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f68001a2c20 con 0x7f6800104a90 2023-07-18T03:30:01.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.321+0000 7f67e7fff700 1 -- 172.21.15.40:0/3689104652 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67f809b730 con 0x7f6800104a90 2023-07-18T03:30:01.319 INFO:teuthology.orchestra.run.smithi040.stdout:158913789995 2023-07-18T03:30:01.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.323+0000 7f68054ee700 1 -- 172.21.15.40:0/3689104652 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67e80a1260 msgr2=0x7f67e80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:01.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.323+0000 7f68054ee700 1 --2- 172.21.15.40:0/3689104652 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67e80a1260 0x7f67e80a3710 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f68000ffbd0 tx=0x7f67f000a3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:01.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.323+0000 7f68054ee700 1 -- 172.21.15.40:0/3689104652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6800104a90 msgr2=0x7f68001a1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:01.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.323+0000 7f68054ee700 1 --2- 172.21.15.40:0/3689104652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6800104a90 0x7f68001a1b00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f67f80062a0 tx=0x7f67f800cf70 comp rx=0 tx=0).stop 2023-07-18T03:30:01.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.324+0000 7f68054ee700 1 -- 172.21.15.40:0/3689104652 shutdown_connections 2023-07-18T03:30:01.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.324+0000 7f68054ee700 1 --2- 172.21.15.40:0/3689104652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6800104a90 0x7f68001a1b00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:01.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.324+0000 7f68054ee700 1 --2- 172.21.15.40:0/3689104652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68000fea90 0x7f6800196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:01.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.324+0000 7f68054ee700 1 --2- 172.21.15.40:0/3689104652 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67e80a1260 0x7f67e80a3710 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:01.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.324+0000 7f68054ee700 1 --2- 172.21.15.40:0/3689104652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68000ff440 0x7f6800196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:01.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.324+0000 7f68054ee700 1 -- 172.21.15.40:0/3689104652 >> 172.21.15.40:0/3689104652 conn(0x7f68000fa570 msgr2=0x7f6800105910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:01.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.324+0000 7f68054ee700 1 -- 172.21.15.40:0/3689104652 shutdown_connections 2023-07-18T03:30:01.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:01.324+0000 7f68054ee700 1 -- 172.21.15.40:0/3689104652 wait complete. 2023-07-18T03:30:01.721 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:01 smithi040 ceph-mon[96439]: pgmap v265: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-18T03:30:01.722 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:01 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3689104652' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-18T03:30:02.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:01 smithi134 ceph-mon[98216]: pgmap v265: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-18T03:30:02.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:01 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3689104652' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-18T03:30:02.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:01 smithi064 ceph-mon[98059]: pgmap v265: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-18T03:30:02.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:01 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3689104652' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-18T03:30:02.126 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411381 got 77309411382 for osd.2 2023-07-18T03:30:02.126 DEBUG:teuthology.parallel:result is None 2023-07-18T03:30:02.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.165+0000 7ff578ed5700 1 -- 172.21.15.40:0/3670157383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff574104ab0 msgr2=0x7ff574104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:02.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.165+0000 7ff578ed5700 1 --2- 172.21.15.40:0/3670157383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff574104ab0 0x7ff574104e80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff564009990 tx=0x7ff564009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.165+0000 7ff578ed5700 1 -- 172.21.15.40:0/3670157383 shutdown_connections 2023-07-18T03:30:02.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.165+0000 7ff578ed5700 1 --2- 172.21.15.40:0/3670157383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5740ff460 0x7ff57410b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.165+0000 7ff578ed5700 1 --2- 172.21.15.40:0/3670157383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff574104ab0 0x7ff574104e80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.165+0000 7ff578ed5700 1 --2- 172.21.15.40:0/3670157383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5740feab0 0x7ff5740fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.165+0000 7ff578ed5700 1 -- 172.21.15.40:0/3670157383 >> 172.21.15.40:0/3670157383 conn(0x7ff574075b70 msgr2=0x7ff574075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:02.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.165+0000 7ff578ed5700 1 -- 172.21.15.40:0/3670157383 shutdown_connections 2023-07-18T03:30:02.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.165+0000 7ff578ed5700 1 -- 172.21.15.40:0/3670157383 wait complete. 2023-07-18T03:30:02.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.166+0000 7ff578ed5700 1 Processor -- start 2023-07-18T03:30:02.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.166+0000 7ff578ed5700 1 -- start start 2023-07-18T03:30:02.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.166+0000 7ff578ed5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5740feab0 0x7ff574197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:02.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.166+0000 7ff578ed5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5740ff460 0x7ff5741983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:02.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.166+0000 7ff578ed5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff574104ab0 0x7ff57419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:02.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.166+0000 7ff578ed5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57419ce60 con 0x7ff5740ff460 2023-07-18T03:30:02.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.166+0000 7ff578ed5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57419cfd0 con 0x7ff5740feab0 2023-07-18T03:30:02.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.166+0000 7ff578ed5700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57419d190 con 0x7ff574104ab0 2023-07-18T03:30:02.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.166+0000 7ff56bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5740ff460 0x7ff5741983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:02.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.166+0000 7ff56bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5740ff460 0x7ff5741983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56644/0 (socket says 172.21.15.40:56644) 2023-07-18T03:30:02.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.166+0000 7ff56bfff700 1 -- 172.21.15.40:0/1189953186 learned_addr learned my addr 172.21.15.40:0/1189953186 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:02.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.166+0000 7ff572d9d700 1 --2- 172.21.15.40:0/1189953186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff574104ab0 0x7ff57419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:02.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.166+0000 7ff57259c700 1 --2- 172.21.15.40:0/1189953186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5740feab0 0x7ff574197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:02.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.167+0000 7ff56bfff700 1 -- 172.21.15.40:0/1189953186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff574104ab0 msgr2=0x7ff57419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:02.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.167+0000 7ff56bfff700 1 --2- 172.21.15.40:0/1189953186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff574104ab0 0x7ff57419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.167+0000 7ff56bfff700 1 -- 172.21.15.40:0/1189953186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5740feab0 msgr2=0x7ff574197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:02.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.167+0000 7ff56bfff700 1 --2- 172.21.15.40:0/1189953186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5740feab0 0x7ff574197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.167+0000 7ff56bfff700 1 -- 172.21.15.40:0/1189953186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5640095f0 con 0x7ff5740ff460 2023-07-18T03:30:02.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.167+0000 7ff57259c700 1 --2- 172.21.15.40:0/1189953186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5740feab0 0x7ff574197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:02.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.167+0000 7ff56bfff700 1 --2- 172.21.15.40:0/1189953186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5740ff460 0x7ff5741983e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00e9a0 tx=0x7ff55c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:02.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.167+0000 7ff56b7fe700 1 -- 172.21.15.40:0/1189953186 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c009790 con 0x7ff5740ff460 2023-07-18T03:30:02.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.167+0000 7ff578ed5700 1 -- 172.21.15.40:0/1189953186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff57419d470 con 0x7ff5740ff460 2023-07-18T03:30:02.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.167+0000 7ff56b7fe700 1 -- 172.21.15.40:0/1189953186 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff55c00cbd0 con 0x7ff5740ff460 2023-07-18T03:30:02.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.167+0000 7ff56b7fe700 1 -- 172.21.15.40:0/1189953186 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c018680 con 0x7ff5740ff460 2023-07-18T03:30:02.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.167+0000 7ff578ed5700 1 -- 172.21.15.40:0/1189953186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5741a4cd0 con 0x7ff5740ff460 2023-07-18T03:30:02.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.168+0000 7ff578ed5700 1 -- 172.21.15.40:0/1189953186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57404e210 con 0x7ff5740ff460 2023-07-18T03:30:02.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.169+0000 7ff56b7fe700 1 -- 172.21.15.40:0/1189953186 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff55c00cd40 con 0x7ff5740ff460 2023-07-18T03:30:02.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.170+0000 7ff56b7fe700 1 --2- 172.21.15.40:0/1189953186 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff5600a1260 0x7ff5600a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:02.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.170+0000 7ff57259c700 1 --2- 172.21.15.40:0/1189953186 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff5600a1260 0x7ff5600a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:02.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.170+0000 7ff56b7fe700 1 -- 172.21.15.40:0/1189953186 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff55c014070 con 0x7ff5740ff460 2023-07-18T03:30:02.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.170+0000 7ff57259c700 1 --2- 172.21.15.40:0/1189953186 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff5600a1260 0x7ff5600a3710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff564000c00 tx=0x7ff564005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:02.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.174+0000 7ff56b7fe700 1 -- 172.21.15.40:0/1189953186 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff55c096520 con 0x7ff5740ff460 2023-07-18T03:30:02.270 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182450 got 107374182451 for osd.3 2023-07-18T03:30:02.271 DEBUG:teuthology.parallel:result is None 2023-07-18T03:30:02.298 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561062 got 188978561063 for osd.6 2023-07-18T03:30:02.299 DEBUG:teuthology.parallel:result is None 2023-07-18T03:30:02.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.402+0000 7ff578ed5700 1 -- 172.21.15.40:0/1189953186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7ff5741a4f90 con 0x7ff5740ff460 2023-07-18T03:30:02.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.402+0000 7ff56b7fe700 1 -- 172.21.15.40:0/1189953186 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff55c026090 con 0x7ff5740ff460 2023-07-18T03:30:02.399 INFO:teuthology.orchestra.run.smithi040.stdout:240518168606 2023-07-18T03:30:02.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.404+0000 7ff578ed5700 1 -- 172.21.15.40:0/1189953186 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff5600a1260 msgr2=0x7ff5600a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:02.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.404+0000 7ff578ed5700 1 --2- 172.21.15.40:0/1189953186 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff5600a1260 0x7ff5600a3710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff564000c00 tx=0x7ff564005fb0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.404+0000 7ff578ed5700 1 -- 172.21.15.40:0/1189953186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5740ff460 msgr2=0x7ff5741983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:02.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.404+0000 7ff578ed5700 1 --2- 172.21.15.40:0/1189953186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5740ff460 0x7ff5741983e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00e9a0 tx=0x7ff55c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.404+0000 7ff578ed5700 1 -- 172.21.15.40:0/1189953186 shutdown_connections 2023-07-18T03:30:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.404+0000 7ff578ed5700 1 --2- 172.21.15.40:0/1189953186 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff5600a1260 0x7ff5600a3710 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.404+0000 7ff578ed5700 1 --2- 172.21.15.40:0/1189953186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff574104ab0 0x7ff57419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.404+0000 7ff578ed5700 1 --2- 172.21.15.40:0/1189953186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5740feab0 0x7ff574197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.404+0000 7ff578ed5700 1 --2- 172.21.15.40:0/1189953186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5740ff460 0x7ff5741983e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.405+0000 7ff578ed5700 1 -- 172.21.15.40:0/1189953186 >> 172.21.15.40:0/1189953186 conn(0x7ff574075b70 msgr2=0x7ff5740fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:02.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.405+0000 7ff578ed5700 1 -- 172.21.15.40:0/1189953186 shutdown_connections 2023-07-18T03:30:02.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.406+0000 7ff578ed5700 1 -- 172.21.15.40:0/1189953186 wait complete. 2023-07-18T03:30:02.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.313+0000 7fe060d2c700 1 -- 172.21.15.40:0/3529385194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe05c104a60 msgr2=0x7fe05c104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:02.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.313+0000 7fe060d2c700 1 --2- 172.21.15.40:0/3529385194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe05c104a60 0x7fe05c104e30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009990 tx=0x7fe04c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.313+0000 7fe060d2c700 1 -- 172.21.15.40:0/3529385194 shutdown_connections 2023-07-18T03:30:02.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.313+0000 7fe060d2c700 1 --2- 172.21.15.40:0/3529385194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe05c0ff410 0x7fe05c10b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.313+0000 7fe060d2c700 1 --2- 172.21.15.40:0/3529385194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe05c104a60 0x7fe05c104e30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.313+0000 7fe060d2c700 1 --2- 172.21.15.40:0/3529385194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe05c0fea60 0x7fe05c0feed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.313+0000 7fe060d2c700 1 -- 172.21.15.40:0/3529385194 >> 172.21.15.40:0/3529385194 conn(0x7fe05c075bc0 msgr2=0x7fe05c075fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:02.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.313+0000 7fe060d2c700 1 -- 172.21.15.40:0/3529385194 shutdown_connections 2023-07-18T03:30:02.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.314+0000 7fe060d2c700 1 -- 172.21.15.40:0/3529385194 wait complete. 2023-07-18T03:30:02.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.314+0000 7fe060d2c700 1 Processor -- start 2023-07-18T03:30:02.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.314+0000 7fe060d2c700 1 -- start start 2023-07-18T03:30:02.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.314+0000 7fe060d2c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe05c0fea60 0x7fe05c19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:02.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.314+0000 7fe060d2c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe05c0ff410 0x7fe05c19bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:02.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.314+0000 7fe060d2c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe05c104a60 0x7fe05c195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:02.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe060d2c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c196080 con 0x7fe05c0ff410 2023-07-18T03:30:02.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe060d2c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c1961f0 con 0x7fe05c104a60 2023-07-18T03:30:02.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe060d2c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c1963b0 con 0x7fe05c0fea60 2023-07-18T03:30:02.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe059d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe05c0ff410 0x7fe05c19bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:02.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe059d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe05c0ff410 0x7fe05c19bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56658/0 (socket says 172.21.15.40:56658) 2023-07-18T03:30:02.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe059d9b700 1 -- 172.21.15.40:0/2376612920 learned_addr learned my addr 172.21.15.40:0/2376612920 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:02.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe05ad9d700 1 --2- 172.21.15.40:0/2376612920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe05c104a60 0x7fe05c195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:02.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe05a59c700 1 --2- 172.21.15.40:0/2376612920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe05c0fea60 0x7fe05c19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:02.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe059d9b700 1 -- 172.21.15.40:0/2376612920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe05c0fea60 msgr2=0x7fe05c19b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe059d9b700 1 --2- 172.21.15.40:0/2376612920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe05c0fea60 0x7fe05c19b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe059d9b700 1 -- 172.21.15.40:0/2376612920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe05c104a60 msgr2=0x7fe05c195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe059d9b700 1 --2- 172.21.15.40:0/2376612920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe05c104a60 0x7fe05c195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe059d9b700 1 -- 172.21.15.40:0/2376612920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c0095f0 con 0x7fe05c0ff410 2023-07-18T03:30:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe059d9b700 1 --2- 172.21.15.40:0/2376612920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe05c0ff410 0x7fe05c19bed0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe05000d950 tx=0x7fe05000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe04b7fe700 1 -- 172.21.15.40:0/2376612920 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05000f890 con 0x7fe05c0ff410 2023-07-18T03:30:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe060d2c700 1 -- 172.21.15.40:0/2376612920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05c196690 con 0x7fe05c0ff410 2023-07-18T03:30:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe04b7fe700 1 -- 172.21.15.40:0/2376612920 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe050010460 con 0x7fe05c0ff410 2023-07-18T03:30:02.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe04b7fe700 1 -- 172.21.15.40:0/2376612920 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05000e610 con 0x7fe05c0ff410 2023-07-18T03:30:02.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.315+0000 7fe060d2c700 1 -- 172.21.15.40:0/2376612920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05c1a2980 con 0x7fe05c0ff410 2023-07-18T03:30:02.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.317+0000 7fe060d2c700 1 -- 172.21.15.40:0/2376612920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05c04e210 con 0x7fe05c0ff410 2023-07-18T03:30:02.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.318+0000 7fe04b7fe700 1 -- 172.21.15.40:0/2376612920 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe0500108d0 con 0x7fe05c0ff410 2023-07-18T03:30:02.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.318+0000 7fe04b7fe700 1 --2- 172.21.15.40:0/2376612920 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0440a11e0 0x7fe0440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:02.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.318+0000 7fe05a59c700 1 --2- 172.21.15.40:0/2376612920 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0440a11e0 0x7fe0440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:02.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.319+0000 7fe04b7fe700 1 -- 172.21.15.40:0/2376612920 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe0500cc7d0 con 0x7fe05c0ff410 2023-07-18T03:30:02.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.319+0000 7fe05a59c700 1 --2- 172.21.15.40:0/2376612920 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0440a11e0 0x7fe0440a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00b5c0 tx=0x7fe04c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:02.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.323+0000 7fe04b7fe700 1 -- 172.21.15.40:0/2376612920 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe050095e70 con 0x7fe05c0ff410 2023-07-18T03:30:02.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.550+0000 7fe060d2c700 1 -- 172.21.15.40:0/2376612920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe05c1a2b70 con 0x7fe05c0ff410 2023-07-18T03:30:02.609 INFO:teuthology.orchestra.run.smithi040.stdout:210453397538 2023-07-18T03:30:02.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.550+0000 7fe04b7fe700 1 -- 172.21.15.40:0/2376612920 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe050099490 con 0x7fe05c0ff410 2023-07-18T03:30:02.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.552+0000 7fe060d2c700 1 -- 172.21.15.40:0/2376612920 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0440a11e0 msgr2=0x7fe0440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:02.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.552+0000 7fe060d2c700 1 --2- 172.21.15.40:0/2376612920 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0440a11e0 0x7fe0440a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00b5c0 tx=0x7fe04c005fd0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.552+0000 7fe060d2c700 1 -- 172.21.15.40:0/2376612920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe05c0ff410 msgr2=0x7fe05c19bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:02.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.552+0000 7fe060d2c700 1 --2- 172.21.15.40:0/2376612920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe05c0ff410 0x7fe05c19bed0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe05000d950 tx=0x7fe05000dc60 comp rx=0 tx=0).stop 2023-07-18T03:30:02.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.552+0000 7fe060d2c700 1 -- 172.21.15.40:0/2376612920 shutdown_connections 2023-07-18T03:30:02.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.552+0000 7fe060d2c700 1 --2- 172.21.15.40:0/2376612920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe05c104a60 0x7fe05c195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.552+0000 7fe060d2c700 1 --2- 172.21.15.40:0/2376612920 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0440a11e0 0x7fe0440a3690 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.552+0000 7fe060d2c700 1 --2- 172.21.15.40:0/2376612920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe05c0fea60 0x7fe05c19b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.552+0000 7fe060d2c700 1 --2- 172.21.15.40:0/2376612920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe05c0ff410 0x7fe05c19bed0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:02.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.552+0000 7fe060d2c700 1 -- 172.21.15.40:0/2376612920 >> 172.21.15.40:0/2376612920 conn(0x7fe05c075bc0 msgr2=0x7fe05c0fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:02.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.552+0000 7fe060d2c700 1 -- 172.21.15.40:0/2376612920 shutdown_connections 2023-07-18T03:30:02.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.552+0000 7fe060d2c700 1 -- 172.21.15.40:0/2376612920 wait complete. 2023-07-18T03:30:02.715 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:02 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1189953186' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-18T03:30:02.715 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:02 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2376612920' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-18T03:30:03.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:02 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1189953186' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-18T03:30:03.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:02 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2376612920' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-18T03:30:03.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:02 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1189953186' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-18T03:30:03.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:02 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2376612920' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-18T03:30:03.203 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:30:03.261 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574907 got 55834574908 for osd.1 2023-07-18T03:30:03.261 DEBUG:teuthology.parallel:result is None 2023-07-18T03:30:03.359 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789993 got 158913789995 for osd.5 2023-07-18T03:30:03.359 DEBUG:teuthology.parallel:result is None 2023-07-18T03:30:03.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.755+0000 7fc88781e700 1 -- 172.21.15.40:0/3948354191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8800feab0 msgr2=0x7fc8800fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:03.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.755+0000 7fc88781e700 1 --2- 172.21.15.40:0/3948354191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8800feab0 0x7fc8800fef20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc874009990 tx=0x7fc874009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:03.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.755+0000 7fc88781e700 1 -- 172.21.15.40:0/3948354191 shutdown_connections 2023-07-18T03:30:03.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.755+0000 7fc88781e700 1 --2- 172.21.15.40:0/3948354191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8800ff460 0x7fc88010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:03.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.755+0000 7fc88781e700 1 --2- 172.21.15.40:0/3948354191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc880104ab0 0x7fc880104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:03.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.755+0000 7fc88781e700 1 --2- 172.21.15.40:0/3948354191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8800feab0 0x7fc8800fef20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:03.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.755+0000 7fc88781e700 1 -- 172.21.15.40:0/3948354191 >> 172.21.15.40:0/3948354191 conn(0x7fc880075b70 msgr2=0x7fc880075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:03.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.756+0000 7fc88781e700 1 -- 172.21.15.40:0/3948354191 shutdown_connections 2023-07-18T03:30:03.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.756+0000 7fc88781e700 1 -- 172.21.15.40:0/3948354191 wait complete. 2023-07-18T03:30:03.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.756+0000 7fc88781e700 1 Processor -- start 2023-07-18T03:30:03.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.756+0000 7fc88781e700 1 -- start start 2023-07-18T03:30:03.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc88781e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8800feab0 0x7fc88019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:03.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc88781e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8800ff460 0x7fc88019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:03.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc88781e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc880104ab0 0x7fc880194980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:03.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc88781e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc880194ff0 con 0x7fc880104ab0 2023-07-18T03:30:03.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc88781e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc880195160 con 0x7fc8800ff460 2023-07-18T03:30:03.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc88781e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc880195320 con 0x7fc8800feab0 2023-07-18T03:30:03.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc885dbb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc880104ab0 0x7fc880194980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:03.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc885dbb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc880104ab0 0x7fc880194980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56674/0 (socket says 172.21.15.40:56674) 2023-07-18T03:30:03.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc885dbb700 1 -- 172.21.15.40:0/64247249 learned_addr learned my addr 172.21.15.40:0/64247249 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:03.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc884db9700 1 --2- 172.21.15.40:0/64247249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8800ff460 0x7fc88019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:03.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc885dbb700 1 -- 172.21.15.40:0/64247249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8800feab0 msgr2=0x7fc88019a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:03.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc8855ba700 1 --2- 172.21.15.40:0/64247249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8800feab0 0x7fc88019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:03.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc885dbb700 1 --2- 172.21.15.40:0/64247249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8800feab0 0x7fc88019a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:03.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc885dbb700 1 -- 172.21.15.40:0/64247249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8800ff460 msgr2=0x7fc88019ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:03.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc885dbb700 1 --2- 172.21.15.40:0/64247249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8800ff460 0x7fc88019ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:03.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc885dbb700 1 -- 172.21.15.40:0/64247249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8740095f0 con 0x7fc880104ab0 2023-07-18T03:30:03.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc8855ba700 1 --2- 172.21.15.40:0/64247249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8800feab0 0x7fc88019a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:03.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.757+0000 7fc885dbb700 1 --2- 172.21.15.40:0/64247249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc880104ab0 0x7fc880194980 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc878007f10 tx=0x7fc87800f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:03.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.758+0000 7fc8727fc700 1 -- 172.21.15.40:0/64247249 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87800fd80 con 0x7fc880104ab0 2023-07-18T03:30:03.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.758+0000 7fc8727fc700 1 -- 172.21.15.40:0/64247249 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc878009360 con 0x7fc880104ab0 2023-07-18T03:30:03.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.758+0000 7fc88781e700 1 -- 172.21.15.40:0/64247249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc880195600 con 0x7fc880104ab0 2023-07-18T03:30:03.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.758+0000 7fc8727fc700 1 -- 172.21.15.40:0/64247249 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878005020 con 0x7fc880104ab0 2023-07-18T03:30:03.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.758+0000 7fc88781e700 1 -- 172.21.15.40:0/64247249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8800681a0 con 0x7fc880104ab0 2023-07-18T03:30:03.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.759+0000 7fc88781e700 1 -- 172.21.15.40:0/64247249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88004e210 con 0x7fc880104ab0 2023-07-18T03:30:03.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.760+0000 7fc8727fc700 1 -- 172.21.15.40:0/64247249 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc878012460 con 0x7fc880104ab0 2023-07-18T03:30:03.382 INFO:teuthology.orchestra.run.smithi040.stdout:274877906971 2023-07-18T03:30:03.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.760+0000 7fc8727fc700 1 --2- 172.21.15.40:0/64247249 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc86c0a1260 0x7fc86c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:03.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.760+0000 7fc8855ba700 1 --2- 172.21.15.40:0/64247249 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc86c0a1260 0x7fc86c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:03.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.761+0000 7fc8727fc700 1 -- 172.21.15.40:0/64247249 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc8780cf040 con 0x7fc880104ab0 2023-07-18T03:30:03.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.761+0000 7fc8855ba700 1 --2- 172.21.15.40:0/64247249 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc86c0a1260 0x7fc86c0a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc87c005e10 tx=0x7fc87c005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:03.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.764+0000 7fc8727fc700 1 -- 172.21.15.40:0/64247249 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc8780986e0 con 0x7fc880104ab0 2023-07-18T03:30:03.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.972+0000 7fc88781e700 1 -- 172.21.15.40:0/64247249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fc880195e90 con 0x7fc880104ab0 2023-07-18T03:30:03.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.972+0000 7fc8727fc700 1 -- 172.21.15.40:0/64247249 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc878016090 con 0x7fc880104ab0 2023-07-18T03:30:03.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.974+0000 7fc88781e700 1 -- 172.21.15.40:0/64247249 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc86c0a1260 msgr2=0x7fc86c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:03.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.974+0000 7fc88781e700 1 --2- 172.21.15.40:0/64247249 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc86c0a1260 0x7fc86c0a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc87c005e10 tx=0x7fc87c005d20 comp rx=0 tx=0).stop 2023-07-18T03:30:03.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.974+0000 7fc88781e700 1 -- 172.21.15.40:0/64247249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc880104ab0 msgr2=0x7fc880194980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:03.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.974+0000 7fc88781e700 1 --2- 172.21.15.40:0/64247249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc880104ab0 0x7fc880194980 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc878007f10 tx=0x7fc87800f5c0 comp rx=0 tx=0).stop 2023-07-18T03:30:03.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.974+0000 7fc88781e700 1 -- 172.21.15.40:0/64247249 shutdown_connections 2023-07-18T03:30:03.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.974+0000 7fc88781e700 1 --2- 172.21.15.40:0/64247249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc880104ab0 0x7fc880194980 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:03.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.974+0000 7fc88781e700 1 --2- 172.21.15.40:0/64247249 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc86c0a1260 0x7fc86c0a3710 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:03.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.974+0000 7fc88781e700 1 --2- 172.21.15.40:0/64247249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8800feab0 0x7fc88019a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:03.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.974+0000 7fc88781e700 1 --2- 172.21.15.40:0/64247249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8800ff460 0x7fc88019ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:03.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.974+0000 7fc88781e700 1 -- 172.21.15.40:0/64247249 >> 172.21.15.40:0/64247249 conn(0x7fc880075b70 msgr2=0x7fc880102670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:03.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.974+0000 7fc88781e700 1 -- 172.21.15.40:0/64247249 shutdown_connections 2023-07-18T03:30:03.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:02.974+0000 7fc88781e700 1 -- 172.21.15.40:0/64247249 wait complete. 2023-07-18T03:30:03.561 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397536 got 210453397538 for osd.7 2023-07-18T03:30:03.562 DEBUG:teuthology.parallel:result is None 2023-07-18T03:30:03.779 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168604 got 240518168606 for osd.8 2023-07-18T03:30:03.780 DEBUG:teuthology.parallel:result is None 2023-07-18T03:30:03.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:03 smithi040 ceph-mon[96439]: pgmap v266: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 11 op/s 2023-07-18T03:30:03.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:03 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/64247249' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-18T03:30:04.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:03 smithi134 ceph-mon[98216]: pgmap v266: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 11 op/s 2023-07-18T03:30:04.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:03 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/64247249' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-18T03:30:04.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:03 smithi064 ceph-mon[98059]: pgmap v266: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 11 op/s 2023-07-18T03:30:04.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:03 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/64247249' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-18T03:30:04.594 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906969 got 274877906971 for osd.9 2023-07-18T03:30:04.595 DEBUG:teuthology.parallel:result is None 2023-07-18T03:30:04.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.982+0000 7f89d2906700 1 -- 172.21.15.40:0/1526959502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89cc0feab0 msgr2=0x7f89cc0fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:04.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.982+0000 7f89d2906700 1 --2- 172.21.15.40:0/1526959502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89cc0feab0 0x7f89cc0fef20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009990 tx=0x7f89c0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:04.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.983+0000 7f89d2906700 1 -- 172.21.15.40:0/1526959502 shutdown_connections 2023-07-18T03:30:04.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.983+0000 7f89d2906700 1 --2- 172.21.15.40:0/1526959502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89cc0ff460 0x7f89cc10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:04.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.983+0000 7f89d2906700 1 --2- 172.21.15.40:0/1526959502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89cc104ab0 0x7f89cc104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:04.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.983+0000 7f89d2906700 1 --2- 172.21.15.40:0/1526959502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89cc0feab0 0x7f89cc0fef20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:04.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.983+0000 7f89d2906700 1 -- 172.21.15.40:0/1526959502 >> 172.21.15.40:0/1526959502 conn(0x7f89cc075b70 msgr2=0x7f89cc075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:04.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.983+0000 7f89d2906700 1 -- 172.21.15.40:0/1526959502 shutdown_connections 2023-07-18T03:30:04.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.983+0000 7f89d2906700 1 -- 172.21.15.40:0/1526959502 wait complete. 2023-07-18T03:30:04.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.983+0000 7f89d2906700 1 Processor -- start 2023-07-18T03:30:04.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.983+0000 7f89d2906700 1 -- start start 2023-07-18T03:30:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.984+0000 7f89d2906700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89cc0feab0 0x7f89cc19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.984+0000 7f89d2906700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89cc0ff460 0x7f89cc19bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.984+0000 7f89d2906700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89cc104ab0 0x7f89cc195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.984+0000 7f89d2906700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc196080 con 0x7f89cc0feab0 2023-07-18T03:30:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.984+0000 7f89d2906700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc1961f0 con 0x7f89cc104ab0 2023-07-18T03:30:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.984+0000 7f89d2906700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc1963b0 con 0x7f89cc0ff460 2023-07-18T03:30:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.984+0000 7f89cbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89cc0feab0 0x7f89cc19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:04.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.984+0000 7f89cbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89cc0feab0 0x7f89cc19b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56684/0 (socket says 172.21.15.40:56684) 2023-07-18T03:30:04.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.984+0000 7f89cbfff700 1 -- 172.21.15.40:0/4099552682 learned_addr learned my addr 172.21.15.40:0/4099552682 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:04.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.984+0000 7f89d0ea3700 1 --2- 172.21.15.40:0/4099552682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89cc104ab0 0x7f89cc195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:04.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.984+0000 7f89cb7fe700 1 --2- 172.21.15.40:0/4099552682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89cc0ff460 0x7f89cc19bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:04.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.984+0000 7f89cbfff700 1 -- 172.21.15.40:0/4099552682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89cc0ff460 msgr2=0x7f89cc19bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:04.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.984+0000 7f89cbfff700 1 --2- 172.21.15.40:0/4099552682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89cc0ff460 0x7f89cc19bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.984+0000 7f89cbfff700 1 -- 172.21.15.40:0/4099552682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89cc104ab0 msgr2=0x7f89cc195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.984+0000 7f89cbfff700 1 --2- 172.21.15.40:0/4099552682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89cc104ab0 0x7f89cc195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.984+0000 7f89cbfff700 1 -- 172.21.15.40:0/4099552682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c00095f0 con 0x7f89cc0feab0 2023-07-18T03:30:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.984+0000 7f89cbfff700 1 --2- 172.21.15.40:0/4099552682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89cc0feab0 0x7f89cc19b990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00d980 tx=0x7f89bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.985+0000 7f89c97fa700 1 -- 172.21.15.40:0/4099552682 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc00b950 con 0x7f89cc0feab0 2023-07-18T03:30:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.985+0000 7f89c97fa700 1 -- 172.21.15.40:0/4099552682 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89bc007620 con 0x7f89cc0feab0 2023-07-18T03:30:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.985+0000 7f89d2906700 1 -- 172.21.15.40:0/4099552682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89cc196690 con 0x7f89cc0feab0 2023-07-18T03:30:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.985+0000 7f89c97fa700 1 -- 172.21.15.40:0/4099552682 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc00f3c0 con 0x7f89cc0feab0 2023-07-18T03:30:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.985+0000 7f89d2906700 1 -- 172.21.15.40:0/4099552682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89cc1a2980 con 0x7f89cc0feab0 2023-07-18T03:30:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.986+0000 7f89d2906700 1 -- 172.21.15.40:0/4099552682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89cc04e210 con 0x7f89cc0feab0 2023-07-18T03:30:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.988+0000 7f89c97fa700 1 -- 172.21.15.40:0/4099552682 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f89bc010640 con 0x7f89cc0feab0 2023-07-18T03:30:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.988+0000 7f89c97fa700 1 --2- 172.21.15.40:0/4099552682 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89b40a11e0 0x7f89b40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.989+0000 7f89c97fa700 1 -- 172.21.15.40:0/4099552682 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f89bc0cc830 con 0x7f89cc0feab0 2023-07-18T03:30:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.989+0000 7f89cb7fe700 1 --2- 172.21.15.40:0/4099552682 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89b40a11e0 0x7f89b40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.989+0000 7f89cb7fe700 1 --2- 172.21.15.40:0/4099552682 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89b40a11e0 0x7f89b40a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f89c0006010 tx=0x7f89c000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:04.997+0000 7f89c97fa700 1 -- 172.21.15.40:0/4099552682 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f89bc095ed0 con 0x7f89cc0feab0 2023-07-18T03:30:05.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.046+0000 7fd640ffc700 1 -- 172.21.15.40:0/166090548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c1005d0 msgr2=0x7fd63c100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:05.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.046+0000 7fd640ffc700 1 --2- 172.21.15.40:0/166090548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c1005d0 0x7fd63c100a40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd6300099c0 tx=0x7fd630009cd0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.048+0000 7fd640ffc700 1 -- 172.21.15.40:0/166090548 shutdown_connections 2023-07-18T03:30:05.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.048+0000 7fd640ffc700 1 --2- 172.21.15.40:0/166090548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd63c100f80 0x7fd63c10ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.048+0000 7fd640ffc700 1 --2- 172.21.15.40:0/166090548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd63c0fee80 0x7fd63c0ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.048+0000 7fd640ffc700 1 --2- 172.21.15.40:0/166090548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c1005d0 0x7fd63c100a40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.048+0000 7fd640ffc700 1 -- 172.21.15.40:0/166090548 >> 172.21.15.40:0/166090548 conn(0x7fd63c075b60 msgr2=0x7fd63c075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:05.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.052+0000 7fd640ffc700 1 -- 172.21.15.40:0/166090548 shutdown_connections 2023-07-18T03:30:05.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.052+0000 7fd640ffc700 1 -- 172.21.15.40:0/166090548 wait complete. 2023-07-18T03:30:05.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.053+0000 7fd640ffc700 1 Processor -- start 2023-07-18T03:30:05.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.053+0000 7fd640ffc700 1 -- start start 2023-07-18T03:30:05.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.053+0000 7fd640ffc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd63c0fee80 0x7fd63c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:05.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.053+0000 7fd640ffc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd63c1005d0 0x7fd63c1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:05.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.053+0000 7fd640ffc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c100f80 0x7fd63c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:05.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.053+0000 7fd640ffc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c19ad20 con 0x7fd63c100f80 2023-07-18T03:30:05.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.053+0000 7fd640ffc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c19ae90 con 0x7fd63c0fee80 2023-07-18T03:30:05.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.053+0000 7fd640ffc700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c19b050 con 0x7fd63c1005d0 2023-07-18T03:30:05.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.054+0000 7fd63a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd63c0fee80 0x7fd63c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:05.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.054+0000 7fd63a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd63c0fee80 0x7fd63c195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:40024/0 (socket says 172.21.15.40:40024) 2023-07-18T03:30:05.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.054+0000 7fd63a59c700 1 -- 172.21.15.40:0/552038646 learned_addr learned my addr 172.21.15.40:0/552038646 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:05.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.054+0000 7fd63ad9d700 1 --2- 172.21.15.40:0/552038646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c100f80 0x7fd63c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:05.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.054+0000 7fd639d9b700 1 --2- 172.21.15.40:0/552038646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd63c1005d0 0x7fd63c1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:05.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.054+0000 7fd63ad9d700 1 -- 172.21.15.40:0/552038646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd63c1005d0 msgr2=0x7fd63c1962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:05.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.054+0000 7fd63ad9d700 1 --2- 172.21.15.40:0/552038646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd63c1005d0 0x7fd63c1962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.054+0000 7fd63ad9d700 1 -- 172.21.15.40:0/552038646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd63c0fee80 msgr2=0x7fd63c195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:05.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.054+0000 7fd63ad9d700 1 --2- 172.21.15.40:0/552038646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd63c0fee80 0x7fd63c195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.054+0000 7fd63ad9d700 1 -- 172.21.15.40:0/552038646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6300095f0 con 0x7fd63c100f80 2023-07-18T03:30:05.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.054+0000 7fd63ad9d700 1 --2- 172.21.15.40:0/552038646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c100f80 0x7fd63c19a710 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd63400bb00 tx=0x7fd63400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:05.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.055+0000 7fd62b7fe700 1 -- 172.21.15.40:0/552038646 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63400ca30 con 0x7fd63c100f80 2023-07-18T03:30:05.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.055+0000 7fd640ffc700 1 -- 172.21.15.40:0/552038646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c19b330 con 0x7fd63c100f80 2023-07-18T03:30:05.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.055+0000 7fd640ffc700 1 -- 172.21.15.40:0/552038646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c1a2b90 con 0x7fd63c100f80 2023-07-18T03:30:05.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.055+0000 7fd62b7fe700 1 -- 172.21.15.40:0/552038646 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd63400cb90 con 0x7fd63c100f80 2023-07-18T03:30:05.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.055+0000 7fd62b7fe700 1 -- 172.21.15.40:0/552038646 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63401d610 con 0x7fd63c100f80 2023-07-18T03:30:05.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.056+0000 7fd640ffc700 1 -- 172.21.15.40:0/552038646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61c005320 con 0x7fd63c100f80 2023-07-18T03:30:05.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.058+0000 7fd62b7fe700 1 -- 172.21.15.40:0/552038646 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd63400cd00 con 0x7fd63c100f80 2023-07-18T03:30:05.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.059+0000 7fd62b7fe700 1 --2- 172.21.15.40:0/552038646 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd6240a11e0 0x7fd6240a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:05.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.059+0000 7fd62b7fe700 1 -- 172.21.15.40:0/552038646 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd6340cc180 con 0x7fd63c100f80 2023-07-18T03:30:05.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.059+0000 7fd63a59c700 1 --2- 172.21.15.40:0/552038646 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd6240a11e0 0x7fd6240a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:05.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.060+0000 7fd63a59c700 1 --2- 172.21.15.40:0/552038646 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd6240a11e0 0x7fd6240a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd62c005950 tx=0x7fd62c004080 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:05.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.063+0000 7fd62b7fe700 1 -- 172.21.15.40:0/552038646 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd6340959c0 con 0x7fd63c100f80 2023-07-18T03:30:05.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.111+0000 7f4019412700 1 -- 172.21.15.40:0/389408469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40141053b0 msgr2=0x7f40140fe0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:05.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.111+0000 7f4019412700 1 --2- 172.21.15.40:0/389408469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40141053b0 0x7f40140fe0e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4004009990 tx=0x7f4004009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.111+0000 7f4019412700 1 -- 172.21.15.40:0/389408469 shutdown_connections 2023-07-18T03:30:05.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.111+0000 7f4019412700 1 --2- 172.21.15.40:0/389408469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40140fe7c0 0x7f40141037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.111+0000 7f4019412700 1 --2- 172.21.15.40:0/389408469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4014104aa0 0x7f4014104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.111+0000 7f4019412700 1 --2- 172.21.15.40:0/389408469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40141053b0 0x7f40140fe0e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.111+0000 7f4019412700 1 -- 172.21.15.40:0/389408469 >> 172.21.15.40:0/389408469 conn(0x7f4014075150 msgr2=0x7f4014075550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:05.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.112+0000 7f4019412700 1 -- 172.21.15.40:0/389408469 shutdown_connections 2023-07-18T03:30:05.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.112+0000 7f4019412700 1 -- 172.21.15.40:0/389408469 wait complete. 2023-07-18T03:30:05.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.113+0000 7f4019412700 1 Processor -- start 2023-07-18T03:30:05.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.113+0000 7f4019412700 1 -- start start 2023-07-18T03:30:05.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.113+0000 7f4019412700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40140fe7c0 0x7f401419c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:05.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.113+0000 7f4019412700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4014104aa0 0x7f401419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:05.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.113+0000 7f4019412700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40141053b0 0x7f40141a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:05.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.113+0000 7f4019412700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40141a12c0 con 0x7f40140fe7c0 2023-07-18T03:30:05.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.113+0000 7f4019412700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40141a1430 con 0x7f4014104aa0 2023-07-18T03:30:05.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.113+0000 7f4019412700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40141a15f0 con 0x7f40141053b0 2023-07-18T03:30:05.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.113+0000 7f4012ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40140fe7c0 0x7f401419c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:05.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.113+0000 7f4012ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40140fe7c0 0x7f401419c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56712/0 (socket says 172.21.15.40:56712) 2023-07-18T03:30:05.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.113+0000 7f4012ffd700 1 -- 172.21.15.40:0/2470868224 learned_addr learned my addr 172.21.15.40:0/2470868224 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:05.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.113+0000 7f40137fe700 1 --2- 172.21.15.40:0/2470868224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40141053b0 0x7f40141a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:05.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.114+0000 7f40127fc700 1 --2- 172.21.15.40:0/2470868224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4014104aa0 0x7f401419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:05.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.114+0000 7f4012ffd700 1 -- 172.21.15.40:0/2470868224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40141053b0 msgr2=0x7f40141a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:05.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.114+0000 7f4012ffd700 1 --2- 172.21.15.40:0/2470868224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40141053b0 0x7f40141a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.114+0000 7f4012ffd700 1 -- 172.21.15.40:0/2470868224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4014104aa0 msgr2=0x7f401419c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:05.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.114+0000 7f4012ffd700 1 --2- 172.21.15.40:0/2470868224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4014104aa0 0x7f401419c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.114+0000 7f4012ffd700 1 -- 172.21.15.40:0/2470868224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40040095f0 con 0x7f40140fe7c0 2023-07-18T03:30:05.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.114+0000 7f4012ffd700 1 --2- 172.21.15.40:0/2470868224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40140fe7c0 0x7f401419c300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00b530 tx=0x7f3ffc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:05.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.114+0000 7f400bfff700 1 -- 172.21.15.40:0/2470868224 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc011830 con 0x7f40140fe7c0 2023-07-18T03:30:05.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.114+0000 7f4019412700 1 -- 172.21.15.40:0/2470868224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40141a18d0 con 0x7f40140fe7c0 2023-07-18T03:30:05.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.114+0000 7f400bfff700 1 -- 172.21.15.40:0/2470868224 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ffc011e50 con 0x7f40140fe7c0 2023-07-18T03:30:05.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.114+0000 7f400bfff700 1 -- 172.21.15.40:0/2470868224 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc00f2e0 con 0x7f40140fe7c0 2023-07-18T03:30:05.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.114+0000 7f4019412700 1 -- 172.21.15.40:0/2470868224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40141a9130 con 0x7f40140fe7c0 2023-07-18T03:30:05.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.116+0000 7f4019412700 1 -- 172.21.15.40:0/2470868224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401404e210 con 0x7f40140fe7c0 2023-07-18T03:30:05.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.116+0000 7f400bfff700 1 -- 172.21.15.40:0/2470868224 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3ffc011990 con 0x7f40140fe7c0 2023-07-18T03:30:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.117+0000 7f400bfff700 1 --2- 172.21.15.40:0/2470868224 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f40000a11e0 0x7f40000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.117+0000 7f40127fc700 1 --2- 172.21.15.40:0/2470868224 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f40000a11e0 0x7f40000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.117+0000 7f400bfff700 1 -- 172.21.15.40:0/2470868224 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3ffc0cc1f0 con 0x7f40140fe7c0 2023-07-18T03:30:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.118+0000 7f40127fc700 1 --2- 172.21.15.40:0/2470868224 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f40000a11e0 0x7f40000a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f401419d920 tx=0x7f4004005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:05.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.121+0000 7f400bfff700 1 -- 172.21.15.40:0/2470868224 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3ffc095aa0 con 0x7f40140fe7c0 2023-07-18T03:30:05.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.215+0000 7f89d2906700 1 -- 172.21.15.40:0/4099552682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f89cc1a2c70 con 0x7f89cc0feab0 2023-07-18T03:30:05.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.216+0000 7f89c97fa700 1 -- 172.21.15.40:0/4099552682 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f89bc0994f0 con 0x7f89cc0feab0 2023-07-18T03:30:05.214 INFO:teuthology.orchestra.run.smithi040.stdout:326417514515 2023-07-18T03:30:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.219+0000 7f89d2906700 1 -- 172.21.15.40:0/4099552682 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89b40a11e0 msgr2=0x7f89b40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.219+0000 7f89d2906700 1 --2- 172.21.15.40:0/4099552682 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89b40a11e0 0x7f89b40a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f89c0006010 tx=0x7f89c000b260 comp rx=0 tx=0).stop 2023-07-18T03:30:05.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.219+0000 7f89d2906700 1 -- 172.21.15.40:0/4099552682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89cc0feab0 msgr2=0x7f89cc19b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.219+0000 7f89d2906700 1 --2- 172.21.15.40:0/4099552682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89cc0feab0 0x7f89cc19b990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00d980 tx=0x7f89bc00dc90 comp rx=0 tx=0).stop 2023-07-18T03:30:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.220+0000 7f89d2906700 1 -- 172.21.15.40:0/4099552682 shutdown_connections 2023-07-18T03:30:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.220+0000 7f89d2906700 1 --2- 172.21.15.40:0/4099552682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89cc104ab0 0x7f89cc195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.220+0000 7f89d2906700 1 --2- 172.21.15.40:0/4099552682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89cc0feab0 0x7f89cc19b990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.220+0000 7f89d2906700 1 --2- 172.21.15.40:0/4099552682 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89b40a11e0 0x7f89b40a3690 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.220+0000 7f89d2906700 1 --2- 172.21.15.40:0/4099552682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89cc0ff460 0x7f89cc19bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.220+0000 7f89d2906700 1 -- 172.21.15.40:0/4099552682 >> 172.21.15.40:0/4099552682 conn(0x7f89cc075b70 msgr2=0x7f89cc102800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:05.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.220+0000 7f89d2906700 1 -- 172.21.15.40:0/4099552682 shutdown_connections 2023-07-18T03:30:05.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.220+0000 7f89d2906700 1 -- 172.21.15.40:0/4099552682 wait complete. 2023-07-18T03:30:05.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.289+0000 7fd640ffc700 1 -- 172.21.15.40:0/552038646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fd61c005cc0 con 0x7fd63c100f80 2023-07-18T03:30:05.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.289+0000 7fd62b7fe700 1 -- 172.21.15.40:0/552038646 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fd634098fe0 con 0x7fd63c100f80 2023-07-18T03:30:05.287 INFO:teuthology.orchestra.run.smithi040.stdout:296352743447 2023-07-18T03:30:05.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.292+0000 7fd640ffc700 1 -- 172.21.15.40:0/552038646 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd6240a11e0 msgr2=0x7fd6240a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:05.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.292+0000 7fd640ffc700 1 --2- 172.21.15.40:0/552038646 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd6240a11e0 0x7fd6240a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd62c005950 tx=0x7fd62c004080 comp rx=0 tx=0).stop 2023-07-18T03:30:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.292+0000 7fd640ffc700 1 -- 172.21.15.40:0/552038646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c100f80 msgr2=0x7fd63c19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.292+0000 7fd640ffc700 1 --2- 172.21.15.40:0/552038646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c100f80 0x7fd63c19a710 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd63400bb00 tx=0x7fd63400be10 comp rx=0 tx=0).stop 2023-07-18T03:30:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.292+0000 7fd640ffc700 1 -- 172.21.15.40:0/552038646 shutdown_connections 2023-07-18T03:30:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.292+0000 7fd640ffc700 1 --2- 172.21.15.40:0/552038646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c100f80 0x7fd63c19a710 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.292+0000 7fd640ffc700 1 --2- 172.21.15.40:0/552038646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd63c0fee80 0x7fd63c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.292+0000 7fd640ffc700 1 --2- 172.21.15.40:0/552038646 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd6240a11e0 0x7fd6240a3690 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.292+0000 7fd640ffc700 1 --2- 172.21.15.40:0/552038646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd63c1005d0 0x7fd63c1962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.292+0000 7fd640ffc700 1 -- 172.21.15.40:0/552038646 >> 172.21.15.40:0/552038646 conn(0x7fd63c075b60 msgr2=0x7fd63c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:05.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.292+0000 7fd640ffc700 1 -- 172.21.15.40:0/552038646 shutdown_connections 2023-07-18T03:30:05.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.292+0000 7fd640ffc700 1 -- 172.21.15.40:0/552038646 wait complete. 2023-07-18T03:30:05.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.382+0000 7f4019412700 1 -- 172.21.15.40:0/2470868224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f40141a92c0 con 0x7f40140fe7c0 2023-07-18T03:30:05.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.383+0000 7f400bfff700 1 -- 172.21.15.40:0/2470868224 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ffc0990c0 con 0x7f40140fe7c0 2023-07-18T03:30:05.381 INFO:teuthology.orchestra.run.smithi040.stdout:137438953519 2023-07-18T03:30:05.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.385+0000 7f4019412700 1 -- 172.21.15.40:0/2470868224 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f40000a11e0 msgr2=0x7f40000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:05.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.385+0000 7f4019412700 1 --2- 172.21.15.40:0/2470868224 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f40000a11e0 0x7f40000a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f401419d920 tx=0x7f4004005fd0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.385+0000 7f4019412700 1 -- 172.21.15.40:0/2470868224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40140fe7c0 msgr2=0x7f401419c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.385+0000 7f4019412700 1 --2- 172.21.15.40:0/2470868224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40140fe7c0 0x7f401419c300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00b530 tx=0x7f3ffc00b840 comp rx=0 tx=0).stop 2023-07-18T03:30:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.385+0000 7f4019412700 1 -- 172.21.15.40:0/2470868224 shutdown_connections 2023-07-18T03:30:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.385+0000 7f4019412700 1 --2- 172.21.15.40:0/2470868224 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f40000a11e0 0x7f40000a3690 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.385+0000 7f4019412700 1 --2- 172.21.15.40:0/2470868224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40141053b0 0x7f40141a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.385+0000 7f4019412700 1 --2- 172.21.15.40:0/2470868224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40140fe7c0 0x7f401419c300 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.385+0000 7f4019412700 1 --2- 172.21.15.40:0/2470868224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4014104aa0 0x7f401419c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.385+0000 7f4019412700 1 -- 172.21.15.40:0/2470868224 >> 172.21.15.40:0/2470868224 conn(0x7f4014075150 msgr2=0x7f4014101850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.385+0000 7f4019412700 1 -- 172.21.15.40:0/2470868224 shutdown_connections 2023-07-18T03:30:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.385+0000 7f4019412700 1 -- 172.21.15.40:0/2470868224 wait complete. 2023-07-18T03:30:05.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.426+0000 7f4fd0f38700 1 -- 172.21.15.40:0/465840999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fcc1007a0 msgr2=0x7f4fcc104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:05.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.426+0000 7f4fd0f38700 1 --2- 172.21.15.40:0/465840999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fcc1007a0 0x7f4fcc104770 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00099c0 tx=0x7f4fc0009cd0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.426+0000 7f4fd0f38700 1 -- 172.21.15.40:0/465840999 shutdown_connections 2023-07-18T03:30:05.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.426+0000 7f4fd0f38700 1 --2- 172.21.15.40:0/465840999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fcc104ea0 0x7f4fcc10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.426+0000 7f4fd0f38700 1 --2- 172.21.15.40:0/465840999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fcc0ffe00 0x7f4fcc1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.426+0000 7f4fd0f38700 1 --2- 172.21.15.40:0/465840999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fcc1007a0 0x7f4fcc104770 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.426+0000 7f4fd0f38700 1 -- 172.21.15.40:0/465840999 >> 172.21.15.40:0/465840999 conn(0x7f4fcc0fb6a0 msgr2=0x7f4fcc0fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:05.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.426+0000 7f4fd0f38700 1 -- 172.21.15.40:0/465840999 shutdown_connections 2023-07-18T03:30:05.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.426+0000 7f4fd0f38700 1 -- 172.21.15.40:0/465840999 wait complete. 2023-07-18T03:30:05.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fd0f38700 1 Processor -- start 2023-07-18T03:30:05.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fd0f38700 1 -- start start 2023-07-18T03:30:05.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fd0f38700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fcc0ffe00 0x7f4fcc072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fd0f38700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fcc1007a0 0x7f4fcc072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fd0f38700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fcc104ea0 0x7f4fcc0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fd0f38700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc1a4120 con 0x7f4fcc0ffe00 2023-07-18T03:30:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fd0f38700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc1a4290 con 0x7f4fcc104ea0 2023-07-18T03:30:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fd0f38700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc1a4450 con 0x7f4fcc1007a0 2023-07-18T03:30:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fca59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fcc0ffe00 0x7f4fcc072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fca59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fcc0ffe00 0x7f4fcc072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56726/0 (socket says 172.21.15.40:56726) 2023-07-18T03:30:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fca59c700 1 -- 172.21.15.40:0/1352572196 learned_addr learned my addr 172.21.15.40:0/1352572196 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fc9d9b700 1 --2- 172.21.15.40:0/1352572196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fcc1007a0 0x7f4fcc072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fcad9d700 1 --2- 172.21.15.40:0/1352572196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fcc104ea0 0x7f4fcc0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fca59c700 1 -- 172.21.15.40:0/1352572196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fcc1007a0 msgr2=0x7f4fcc072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fca59c700 1 --2- 172.21.15.40:0/1352572196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fcc1007a0 0x7f4fcc072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fca59c700 1 -- 172.21.15.40:0/1352572196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fcc104ea0 msgr2=0x7f4fcc0732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fca59c700 1 --2- 172.21.15.40:0/1352572196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fcc104ea0 0x7f4fcc0732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fca59c700 1 -- 172.21.15.40:0/1352572196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc00095f0 con 0x7f4fcc0ffe00 2023-07-18T03:30:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.427+0000 7f4fcad9d700 1 --2- 172.21.15.40:0/1352572196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fcc104ea0 0x7f4fcc0732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.428+0000 7f4fca59c700 1 --2- 172.21.15.40:0/1352572196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fcc0ffe00 0x7f4fcc072810 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00d980 tx=0x7f4fbc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.428+0000 7f4fbb7fe700 1 -- 172.21.15.40:0/1352572196 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc004220 con 0x7f4fcc0ffe00 2023-07-18T03:30:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.428+0000 7f4fd0f38700 1 -- 172.21.15.40:0/1352572196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fcc1a4730 con 0x7f4fcc0ffe00 2023-07-18T03:30:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.428+0000 7f4fbb7fe700 1 -- 172.21.15.40:0/1352572196 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fbc007640 con 0x7f4fcc0ffe00 2023-07-18T03:30:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.428+0000 7f4fbb7fe700 1 -- 172.21.15.40:0/1352572196 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc005610 con 0x7f4fcc0ffe00 2023-07-18T03:30:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.428+0000 7f4fd0f38700 1 -- 172.21.15.40:0/1352572196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc1a4c50 con 0x7f4fcc0ffe00 2023-07-18T03:30:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.429+0000 7f4fd0f38700 1 -- 172.21.15.40:0/1352572196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fcc04e210 con 0x7f4fcc0ffe00 2023-07-18T03:30:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.430+0000 7f4fbb7fe700 1 -- 172.21.15.40:0/1352572196 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4fbc004750 con 0x7f4fcc0ffe00 2023-07-18T03:30:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.431+0000 7f4fbb7fe700 1 --2- 172.21.15.40:0/1352572196 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4fb40a11e0 0x7f4fb40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.431+0000 7f4fc9d9b700 1 --2- 172.21.15.40:0/1352572196 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4fb40a11e0 0x7f4fb40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.431+0000 7f4fbb7fe700 1 -- 172.21.15.40:0/1352572196 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4fbc0ccd50 con 0x7f4fcc0ffe00 2023-07-18T03:30:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.431+0000 7f4fc9d9b700 1 --2- 172.21.15.40:0/1352572196 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4fb40a11e0 0x7f4fb40a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0006010 tx=0x7f4fc0003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:05.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.434+0000 7f4fbb7fe700 1 -- 172.21.15.40:0/1352572196 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4fbc0963f0 con 0x7f4fcc0ffe00 2023-07-18T03:30:05.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.650+0000 7f4fd0f38700 1 -- 172.21.15.40:0/1352572196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4fcc1a4e70 con 0x7f4fcc0ffe00 2023-07-18T03:30:05.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.651+0000 7f4fbb7fe700 1 -- 172.21.15.40:0/1352572196 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4fbc016090 con 0x7f4fcc0ffe00 2023-07-18T03:30:05.649 INFO:teuthology.orchestra.run.smithi040.stdout:34359738433 2023-07-18T03:30:05.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.653+0000 7f4fd0f38700 1 -- 172.21.15.40:0/1352572196 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4fb40a11e0 msgr2=0x7f4fb40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:05.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.653+0000 7f4fd0f38700 1 --2- 172.21.15.40:0/1352572196 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4fb40a11e0 0x7f4fb40a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0006010 tx=0x7f4fc0003680 comp rx=0 tx=0).stop 2023-07-18T03:30:05.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.653+0000 7f4fd0f38700 1 -- 172.21.15.40:0/1352572196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fcc0ffe00 msgr2=0x7f4fcc072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:05.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.653+0000 7f4fd0f38700 1 --2- 172.21.15.40:0/1352572196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fcc0ffe00 0x7f4fcc072810 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00d980 tx=0x7f4fbc00dc90 comp rx=0 tx=0).stop 2023-07-18T03:30:05.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.653+0000 7f4fd0f38700 1 -- 172.21.15.40:0/1352572196 shutdown_connections 2023-07-18T03:30:05.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.653+0000 7f4fd0f38700 1 --2- 172.21.15.40:0/1352572196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fcc104ea0 0x7f4fcc0732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.653+0000 7f4fd0f38700 1 --2- 172.21.15.40:0/1352572196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fcc0ffe00 0x7f4fcc072810 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.653+0000 7f4fd0f38700 1 --2- 172.21.15.40:0/1352572196 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4fb40a11e0 0x7f4fb40a3690 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.653+0000 7f4fd0f38700 1 --2- 172.21.15.40:0/1352572196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fcc1007a0 0x7f4fcc072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:05.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.653+0000 7f4fd0f38700 1 -- 172.21.15.40:0/1352572196 >> 172.21.15.40:0/1352572196 conn(0x7f4fcc0fb6a0 msgr2=0x7f4fcc0fc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:05.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.653+0000 7f4fd0f38700 1 -- 172.21.15.40:0/1352572196 shutdown_connections 2023-07-18T03:30:05.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:05.653+0000 7f4fd0f38700 1 -- 172.21.15.40:0/1352572196 wait complete. 2023-07-18T03:30:05.765 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514513 got 326417514515 for osd.11 2023-07-18T03:30:05.766 DEBUG:teuthology.parallel:result is None 2023-07-18T03:30:05.948 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:05 smithi040 ceph-mon[96439]: pgmap v267: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-18T03:30:05.949 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:05 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/4099552682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-18T03:30:05.949 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:05 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/552038646' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-18T03:30:05.949 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:05 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2470868224' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-18T03:30:05.949 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:05 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1352572196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-18T03:30:05.973 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738432 got 34359738433 for osd.0 2023-07-18T03:30:05.973 DEBUG:teuthology.parallel:result is None 2023-07-18T03:30:06.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:05 smithi134 ceph-mon[98216]: pgmap v267: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-18T03:30:06.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:05 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/4099552682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-18T03:30:06.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:05 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/552038646' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-18T03:30:06.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:05 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2470868224' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-18T03:30:06.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:05 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1352572196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-18T03:30:06.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:05 smithi064 ceph-mon[98059]: pgmap v267: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-18T03:30:06.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:05 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/4099552682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-18T03:30:06.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:05 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/552038646' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-18T03:30:06.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:05 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2470868224' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-18T03:30:06.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:05 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1352572196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-18T03:30:06.274 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953517 got 137438953519 for osd.4 2023-07-18T03:30:06.275 DEBUG:teuthology.parallel:result is None 2023-07-18T03:30:06.376 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743445 got 296352743447 for osd.10 2023-07-18T03:30:06.376 DEBUG:teuthology.parallel:result is None 2023-07-18T03:30:06.376 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-07-18T03:30:06.377 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph pg dump --format=json 2023-07-18T03:30:06.937 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:30:07.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:07 smithi040 ceph-mon[96439]: pgmap v268: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 10 op/s 2023-07-18T03:30:08.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:07 smithi134 ceph-mon[98216]: pgmap v268: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 10 op/s 2023-07-18T03:30:08.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:07 smithi064 ceph-mon[98059]: pgmap v268: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 10 op/s 2023-07-18T03:30:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.137+0000 7f89488ff700 1 -- 172.21.15.40:0/3929663753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440feab0 msgr2=0x7f89440fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.137+0000 7f89488ff700 1 --2- 172.21.15.40:0/3929663753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440feab0 0x7f89440fef20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8938009990 tx=0x7f8938009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.137+0000 7f89488ff700 1 -- 172.21.15.40:0/3929663753 shutdown_connections 2023-07-18T03:30:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.137+0000 7f89488ff700 1 --2- 172.21.15.40:0/3929663753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89440ff460 0x7f894410b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.137+0000 7f89488ff700 1 --2- 172.21.15.40:0/3929663753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8944104ab0 0x7f8944104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:08.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.137+0000 7f89488ff700 1 --2- 172.21.15.40:0/3929663753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440feab0 0x7f89440fef20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:08.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.137+0000 7f89488ff700 1 -- 172.21.15.40:0/3929663753 >> 172.21.15.40:0/3929663753 conn(0x7f8944075b70 msgr2=0x7f8944075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:08.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.137+0000 7f89488ff700 1 -- 172.21.15.40:0/3929663753 shutdown_connections 2023-07-18T03:30:08.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.137+0000 7f89488ff700 1 -- 172.21.15.40:0/3929663753 wait complete. 2023-07-18T03:30:08.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f89488ff700 1 Processor -- start 2023-07-18T03:30:08.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f89488ff700 1 -- start start 2023-07-18T03:30:08.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f89488ff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440feab0 0x7f89441006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:08.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f89488ff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89440ff460 0x7f8944100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:08.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f89488ff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8944104ab0 0x7f8944101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:08.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f89488ff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89441a40e0 con 0x7f89440feab0 2023-07-18T03:30:08.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f89488ff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89441a4250 con 0x7f89440ff460 2023-07-18T03:30:08.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f89488ff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89441a4410 con 0x7f8944104ab0 2023-07-18T03:30:08.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f894259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440feab0 0x7f89441006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:08.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f894259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440feab0 0x7f89441006c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56738/0 (socket says 172.21.15.40:56738) 2023-07-18T03:30:08.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f894259c700 1 -- 172.21.15.40:0/2536794907 learned_addr learned my addr 172.21.15.40:0/2536794907 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:08.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f8941d9b700 1 --2- 172.21.15.40:0/2536794907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89440ff460 0x7f8944100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:08.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f894259c700 1 -- 172.21.15.40:0/2536794907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8944104ab0 msgr2=0x7f8944101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:08.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f894259c700 1 --2- 172.21.15.40:0/2536794907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8944104ab0 0x7f8944101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:08.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f894259c700 1 -- 172.21.15.40:0/2536794907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89440ff460 msgr2=0x7f8944100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:08.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f894259c700 1 --2- 172.21.15.40:0/2536794907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89440ff460 0x7f8944100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:08.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f894259c700 1 -- 172.21.15.40:0/2536794907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89380095f0 con 0x7f89440feab0 2023-07-18T03:30:08.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.138+0000 7f8941d9b700 1 --2- 172.21.15.40:0/2536794907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89440ff460 0x7f8944100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:08.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.139+0000 7f894259c700 1 --2- 172.21.15.40:0/2536794907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440feab0 0x7f89441006c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f893400b500 tx=0x7f893400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:08.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.139+0000 7f89337fe700 1 -- 172.21.15.40:0/2536794907 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934011830 con 0x7f89440feab0 2023-07-18T03:30:08.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.139+0000 7f89337fe700 1 -- 172.21.15.40:0/2536794907 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8934011e50 con 0x7f89440feab0 2023-07-18T03:30:08.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.139+0000 7f89337fe700 1 -- 172.21.15.40:0/2536794907 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893400f2e0 con 0x7f89440feab0 2023-07-18T03:30:08.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.139+0000 7f89488ff700 1 -- 172.21.15.40:0/2536794907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89441a46f0 con 0x7f89440feab0 2023-07-18T03:30:08.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.139+0000 7f89488ff700 1 -- 172.21.15.40:0/2536794907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89441a4c20 con 0x7f89440feab0 2023-07-18T03:30:08.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.140+0000 7f89488ff700 1 -- 172.21.15.40:0/2536794907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f894404e210 con 0x7f89440feab0 2023-07-18T03:30:08.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.141+0000 7f89337fe700 1 -- 172.21.15.40:0/2536794907 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8934010580 con 0x7f89440feab0 2023-07-18T03:30:08.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.141+0000 7f89337fe700 1 --2- 172.21.15.40:0/2536794907 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f892c0a11e0 0x7f892c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:08.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.142+0000 7f8941d9b700 1 --2- 172.21.15.40:0/2536794907 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f892c0a11e0 0x7f892c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:08.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.142+0000 7f89337fe700 1 -- 172.21.15.40:0/2536794907 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f89340cc3b0 con 0x7f89440feab0 2023-07-18T03:30:08.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.142+0000 7f8941d9b700 1 --2- 172.21.15.40:0/2536794907 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f892c0a11e0 0x7f892c0a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f893800b5c0 tx=0x7f8938005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:08.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.146+0000 7f89337fe700 1 -- 172.21.15.40:0/2536794907 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8934095b60 con 0x7f89440feab0 2023-07-18T03:30:08.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.339+0000 7f89488ff700 1 -- 172.21.15.40:0/2536794907 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f89440fd990 con 0x7f892c0a11e0 2023-07-18T03:30:08.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.351+0000 7f89337fe700 1 -- 172.21.15.40:0/2536794907 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340200 (secure 0 0 0) 0x7f89440fd990 con 0x7f892c0a11e0 2023-07-18T03:30:08.352 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:08.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.355+0000 7f89488ff700 1 -- 172.21.15.40:0/2536794907 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f892c0a11e0 msgr2=0x7f892c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:08.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.355+0000 7f89488ff700 1 --2- 172.21.15.40:0/2536794907 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f892c0a11e0 0x7f892c0a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f893800b5c0 tx=0x7f8938005fb0 comp rx=0 tx=0).stop 2023-07-18T03:30:08.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.355+0000 7f89488ff700 1 -- 172.21.15.40:0/2536794907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440feab0 msgr2=0x7f89441006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:08.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.355+0000 7f89488ff700 1 --2- 172.21.15.40:0/2536794907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440feab0 0x7f89441006c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f893400b500 tx=0x7f893400b8c0 comp rx=0 tx=0).stop 2023-07-18T03:30:08.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.355+0000 7f89488ff700 1 -- 172.21.15.40:0/2536794907 shutdown_connections 2023-07-18T03:30:08.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.355+0000 7f89488ff700 1 --2- 172.21.15.40:0/2536794907 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f892c0a11e0 0x7f892c0a3690 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:08.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.355+0000 7f89488ff700 1 --2- 172.21.15.40:0/2536794907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8944104ab0 0x7f8944101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:08.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.355+0000 7f89488ff700 1 --2- 172.21.15.40:0/2536794907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89440feab0 0x7f89441006c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:08.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.355+0000 7f89488ff700 1 --2- 172.21.15.40:0/2536794907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89440ff460 0x7f8944100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:08.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.355+0000 7f89488ff700 1 -- 172.21.15.40:0/2536794907 >> 172.21.15.40:0/2536794907 conn(0x7f8944075b70 msgr2=0x7f89440fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:08.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.355+0000 7f89488ff700 1 -- 172.21.15.40:0/2536794907 shutdown_connections 2023-07-18T03:30:08.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:08.355+0000 7f89488ff700 1 -- 172.21.15.40:0/2536794907 wait complete. 2023-07-18T03:30:08.355 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-18T03:30:08.697 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2023-07-18T03:30:07.824418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655761,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":491,"num_read_kb":363,"num_write":412,"num_write_kb":3066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335912,"kb_used_data":12488,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605912,"statfs":{"total":1151940427776,"available":1151596453888,"internally_reserved":0,"allocated":12787712,"data_stored":6973544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":56,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003156"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085961+0000","last_change":"2023-07-18T03:29:27.101623+0000","last_active":"2023-07-18T03:29:28.085961+0000","last_peered":"2023-07-18T03:29:28.085961+0000","last_clean":"2023-07-18T03:29:28.085961+0000","last_became_active":"2023-07-18T03:29:27.101297+0000","last_became_peered":"2023-07-18T03:29:27.101297+0000","last_unstale":"2023-07-18T03:29:28.085961+0000","last_undegraded":"2023-07-18T03:29:28.085961+0000","last_fullsized":"2023-07-18T03:29:28.085961+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:34:31.169755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248873+0000","last_change":"2023-07-18T03:29:27.104159+0000","last_active":"2023-07-18T03:29:28.248873+0000","last_peered":"2023-07-18T03:29:28.248873+0000","last_clean":"2023-07-18T03:29:28.248873+0000","last_became_active":"2023-07-18T03:29:27.103830+0000","last_became_peered":"2023-07-18T03:29:27.103830+0000","last_unstale":"2023-07-18T03:29:28.248873+0000","last_undegraded":"2023-07-18T03:29:28.248873+0000","last_fullsized":"2023-07-18T03:29:28.248873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:44:49.917899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249078+0000","last_change":"2023-07-18T03:29:27.098858+0000","last_active":"2023-07-18T03:29:28.249078+0000","last_peered":"2023-07-18T03:29:28.249078+0000","last_clean":"2023-07-18T03:29:28.249078+0000","last_became_active":"2023-07-18T03:29:27.098540+0000","last_became_peered":"2023-07-18T03:29:27.098540+0000","last_unstale":"2023-07-18T03:29:28.249078+0000","last_undegraded":"2023-07-18T03:29:28.249078+0000","last_fullsized":"2023-07-18T03:29:28.249078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:55:46.914785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248698+0000","last_change":"2023-07-18T03:29:27.103106+0000","last_active":"2023-07-18T03:29:28.248698+0000","last_peered":"2023-07-18T03:29:28.248698+0000","last_clean":"2023-07-18T03:29:28.248698+0000","last_became_active":"2023-07-18T03:29:27.102780+0000","last_became_peered":"2023-07-18T03:29:27.102780+0000","last_unstale":"2023-07-18T03:29:28.248698+0000","last_undegraded":"2023-07-18T03:29:28.248698+0000","last_fullsized":"2023-07-18T03:29:28.248698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:51:36.548110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248934+0000","last_change":"2023-07-18T03:29:27.095432+0000","last_active":"2023-07-18T03:29:28.248934+0000","last_peered":"2023-07-18T03:29:28.248934+0000","last_clean":"2023-07-18T03:29:28.248934+0000","last_became_active":"2023-07-18T03:29:27.095103+0000","last_became_peered":"2023-07-18T03:29:27.095103+0000","last_unstale":"2023-07-18T03:29:28.248934+0000","last_undegraded":"2023-07-18T03:29:28.248934+0000","last_fullsized":"2023-07-18T03:29:28.248934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:50:40.336206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085493+0000","last_change":"2023-07-18T03:29:27.091412+0000","last_active":"2023-07-18T03:29:28.085493+0000","last_peered":"2023-07-18T03:29:28.085493+0000","last_clean":"2023-07-18T03:29:28.085493+0000","last_became_active":"2023-07-18T03:29:27.091060+0000","last_became_peered":"2023-07-18T03:29:27.091060+0000","last_unstale":"2023-07-18T03:29:28.085493+0000","last_undegraded":"2023-07-18T03:29:28.085493+0000","last_fullsized":"2023-07-18T03:29:28.085493+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:31:43.393707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339267+0000","last_change":"2023-07-18T03:29:27.105223+0000","last_active":"2023-07-18T03:29:28.339267+0000","last_peered":"2023-07-18T03:29:28.339267+0000","last_clean":"2023-07-18T03:29:28.339267+0000","last_became_active":"2023-07-18T03:29:27.104926+0000","last_became_peered":"2023-07-18T03:29:27.104926+0000","last_unstale":"2023-07-18T03:29:28.339267+0000","last_undegraded":"2023-07-18T03:29:28.339267+0000","last_fullsized":"2023-07-18T03:29:28.339267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:43:23.908336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248819+0000","last_change":"2023-07-18T03:29:27.093722+0000","last_active":"2023-07-18T03:29:28.248819+0000","last_peered":"2023-07-18T03:29:28.248819+0000","last_clean":"2023-07-18T03:29:28.248819+0000","last_became_active":"2023-07-18T03:29:27.093103+0000","last_became_peered":"2023-07-18T03:29:27.093103+0000","last_unstale":"2023-07-18T03:29:28.248819+0000","last_undegraded":"2023-07-18T03:29:28.248819+0000","last_fullsized":"2023-07-18T03:29:28.248819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:03:07.122936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249009+0000","last_change":"2023-07-18T03:29:27.094681+0000","last_active":"2023-07-18T03:29:28.249009+0000","last_peered":"2023-07-18T03:29:28.249009+0000","last_clean":"2023-07-18T03:29:28.249009+0000","last_became_active":"2023-07-18T03:29:27.094284+0000","last_became_peered":"2023-07-18T03:29:27.094284+0000","last_unstale":"2023-07-18T03:29:28.249009+0000","last_undegraded":"2023-07-18T03:29:28.249009+0000","last_fullsized":"2023-07-18T03:29:28.249009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:25:08.820895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085737+0000","last_change":"2023-07-18T03:29:27.100759+0000","last_active":"2023-07-18T03:29:28.085737+0000","last_peered":"2023-07-18T03:29:28.085737+0000","last_clean":"2023-07-18T03:29:28.085737+0000","last_became_active":"2023-07-18T03:29:27.096660+0000","last_became_peered":"2023-07-18T03:29:27.096660+0000","last_unstale":"2023-07-18T03:29:28.085737+0000","last_undegraded":"2023-07-18T03:29:28.085737+0000","last_fullsized":"2023-07-18T03:29:28.085737+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:56:10.159967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339718+0000","last_change":"2023-07-18T03:29:27.107251+0000","last_active":"2023-07-18T03:29:28.339718+0000","last_peered":"2023-07-18T03:29:28.339718+0000","last_clean":"2023-07-18T03:29:28.339718+0000","last_became_active":"2023-07-18T03:29:27.106895+0000","last_became_peered":"2023-07-18T03:29:27.106895+0000","last_unstale":"2023-07-18T03:29:28.339718+0000","last_undegraded":"2023-07-18T03:29:28.339718+0000","last_fullsized":"2023-07-18T03:29:28.339718+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:12:15.805607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248978+0000","last_change":"2023-07-18T03:29:27.103905+0000","last_active":"2023-07-18T03:29:28.248978+0000","last_peered":"2023-07-18T03:29:28.248978+0000","last_clean":"2023-07-18T03:29:28.248978+0000","last_became_active":"2023-07-18T03:29:27.103021+0000","last_became_peered":"2023-07-18T03:29:27.103021+0000","last_unstale":"2023-07-18T03:29:28.248978+0000","last_undegraded":"2023-07-18T03:29:28.248978+0000","last_fullsized":"2023-07-18T03:29:28.248978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:48:20.771586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249855+0000","last_change":"2023-07-18T03:29:27.103529+0000","last_active":"2023-07-18T03:29:28.249855+0000","last_peered":"2023-07-18T03:29:28.249855+0000","last_clean":"2023-07-18T03:29:28.249855+0000","last_became_active":"2023-07-18T03:29:27.103217+0000","last_became_peered":"2023-07-18T03:29:27.103217+0000","last_unstale":"2023-07-18T03:29:28.249855+0000","last_undegraded":"2023-07-18T03:29:28.249855+0000","last_fullsized":"2023-07-18T03:29:28.249855+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:37:25.309228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248500+0000","last_change":"2023-07-18T03:29:27.096640+0000","last_active":"2023-07-18T03:29:28.248500+0000","last_peered":"2023-07-18T03:29:28.248500+0000","last_clean":"2023-07-18T03:29:28.248500+0000","last_became_active":"2023-07-18T03:29:27.096347+0000","last_became_peered":"2023-07-18T03:29:27.096347+0000","last_unstale":"2023-07-18T03:29:28.248500+0000","last_undegraded":"2023-07-18T03:29:28.248500+0000","last_fullsized":"2023-07-18T03:29:28.248500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:31:27.280331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248292+0000","last_change":"2023-07-18T03:29:27.095910+0000","last_active":"2023-07-18T03:29:28.248292+0000","last_peered":"2023-07-18T03:29:28.248292+0000","last_clean":"2023-07-18T03:29:28.248292+0000","last_became_active":"2023-07-18T03:29:27.095556+0000","last_became_peered":"2023-07-18T03:29:27.095556+0000","last_unstale":"2023-07-18T03:29:28.248292+0000","last_undegraded":"2023-07-18T03:29:28.248292+0000","last_fullsized":"2023-07-18T03:29:28.248292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:09:59.583920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248551+0000","last_change":"2023-07-18T03:29:27.108220+0000","last_active":"2023-07-18T03:29:28.248551+0000","last_peered":"2023-07-18T03:29:28.248551+0000","last_clean":"2023-07-18T03:29:28.248551+0000","last_became_active":"2023-07-18T03:29:27.107869+0000","last_became_peered":"2023-07-18T03:29:27.107869+0000","last_unstale":"2023-07-18T03:29:28.248551+0000","last_undegraded":"2023-07-18T03:29:28.248551+0000","last_fullsized":"2023-07-18T03:29:28.248551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:53:30.964666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.086002+0000","last_change":"2023-07-18T03:29:27.100634+0000","last_active":"2023-07-18T03:29:28.086002+0000","last_peered":"2023-07-18T03:29:28.086002+0000","last_clean":"2023-07-18T03:29:28.086002+0000","last_became_active":"2023-07-18T03:29:27.100316+0000","last_became_peered":"2023-07-18T03:29:27.100316+0000","last_unstale":"2023-07-18T03:29:28.086002+0000","last_undegraded":"2023-07-18T03:29:28.086002+0000","last_fullsized":"2023-07-18T03:29:28.086002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:35:30.023399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.235377+0000","last_change":"2023-07-18T03:29:27.100322+0000","last_active":"2023-07-18T03:29:28.235377+0000","last_peered":"2023-07-18T03:29:28.235377+0000","last_clean":"2023-07-18T03:29:28.235377+0000","last_became_active":"2023-07-18T03:29:27.099782+0000","last_became_peered":"2023-07-18T03:29:27.099782+0000","last_unstale":"2023-07-18T03:29:28.235377+0000","last_undegraded":"2023-07-18T03:29:28.235377+0000","last_fullsized":"2023-07-18T03:29:28.235377+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:28:12.377533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129942+0000","last_change":"2023-07-18T03:29:27.096639+0000","last_active":"2023-07-18T03:29:28.129942+0000","last_peered":"2023-07-18T03:29:28.129942+0000","last_clean":"2023-07-18T03:29:28.129942+0000","last_became_active":"2023-07-18T03:29:27.096348+0000","last_became_peered":"2023-07-18T03:29:27.096348+0000","last_unstale":"2023-07-18T03:29:28.129942+0000","last_undegraded":"2023-07-18T03:29:28.129942+0000","last_fullsized":"2023-07-18T03:29:28.129942+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:05:09.093995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":59,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:30:03.708335+0000","last_change":"2023-07-18T03:29:26.287722+0000","last_active":"2023-07-18T03:30:03.708335+0000","last_peered":"2023-07-18T03:30:03.708335+0000","last_clean":"2023-07-18T03:30:03.708335+0000","last_became_active":"2023-07-18T03:29:26.287327+0000","last_became_peered":"2023-07-18T03:29:26.287327+0000","last_unstale":"2023-07-18T03:30:03.708335+0000","last_undegraded":"2023-07-18T03:30:03.708335+0000","last_fullsized":"2023-07-18T03:30:03.708335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:57:15.630654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249206+0000","last_change":"2023-07-18T03:29:27.092771+0000","last_active":"2023-07-18T03:29:28.249206+0000","last_peered":"2023-07-18T03:29:28.249206+0000","last_clean":"2023-07-18T03:29:28.249206+0000","last_became_active":"2023-07-18T03:29:27.092456+0000","last_became_peered":"2023-07-18T03:29:27.092456+0000","last_unstale":"2023-07-18T03:29:28.249206+0000","last_undegraded":"2023-07-18T03:29:28.249206+0000","last_fullsized":"2023-07-18T03:29:28.249206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:02:05.004006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:29.697155+0000","last_change":"2023-07-18T03:29:26.289683+0000","last_active":"2023-07-18T03:29:29.697155+0000","last_peered":"2023-07-18T03:29:29.697155+0000","last_clean":"2023-07-18T03:29:29.697155+0000","last_became_active":"2023-07-18T03:29:26.289356+0000","last_became_peered":"2023-07-18T03:29:26.289356+0000","last_unstale":"2023-07-18T03:29:29.697155+0000","last_undegraded":"2023-07-18T03:29:29.697155+0000","last_fullsized":"2023-07-18T03:29:29.697155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:02:04.478209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249301+0000","last_change":"2023-07-18T03:29:27.098709+0000","last_active":"2023-07-18T03:29:28.249301+0000","last_peered":"2023-07-18T03:29:28.249301+0000","last_clean":"2023-07-18T03:29:28.249301+0000","last_became_active":"2023-07-18T03:29:27.098395+0000","last_became_peered":"2023-07-18T03:29:27.098395+0000","last_unstale":"2023-07-18T03:29:28.249301+0000","last_undegraded":"2023-07-18T03:29:28.249301+0000","last_fullsized":"2023-07-18T03:29:28.249301+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:26:19.426023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:34.657360+0000","last_change":"2023-07-18T03:29:26.297119+0000","last_active":"2023-07-18T03:29:34.657360+0000","last_peered":"2023-07-18T03:29:34.657360+0000","last_clean":"2023-07-18T03:29:34.657360+0000","last_became_active":"2023-07-18T03:29:26.296649+0000","last_became_peered":"2023-07-18T03:29:26.296649+0000","last_unstale":"2023-07-18T03:29:34.657360+0000","last_undegraded":"2023-07-18T03:29:34.657360+0000","last_fullsized":"2023-07-18T03:29:34.657360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:51:17.267059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248457+0000","last_change":"2023-07-18T03:29:27.105511+0000","last_active":"2023-07-18T03:29:28.248457+0000","last_peered":"2023-07-18T03:29:28.248457+0000","last_clean":"2023-07-18T03:29:28.248457+0000","last_became_active":"2023-07-18T03:29:27.104946+0000","last_became_peered":"2023-07-18T03:29:27.104946+0000","last_unstale":"2023-07-18T03:29:28.248457+0000","last_undegraded":"2023-07-18T03:29:28.248457+0000","last_fullsized":"2023-07-18T03:29:28.248457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:42:13.652404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.642412+0000","last_change":"2023-07-18T03:29:26.290708+0000","last_active":"2023-07-18T03:29:33.642412+0000","last_peered":"2023-07-18T03:29:33.642412+0000","last_clean":"2023-07-18T03:29:33.642412+0000","last_became_active":"2023-07-18T03:29:26.290383+0000","last_became_peered":"2023-07-18T03:29:26.290383+0000","last_unstale":"2023-07-18T03:29:33.642412+0000","last_undegraded":"2023-07-18T03:29:33.642412+0000","last_fullsized":"2023-07-18T03:29:33.642412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:34:22.740683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.084957+0000","last_change":"2023-07-18T03:29:27.096630+0000","last_active":"2023-07-18T03:29:28.084957+0000","last_peered":"2023-07-18T03:29:28.084957+0000","last_clean":"2023-07-18T03:29:28.084957+0000","last_became_active":"2023-07-18T03:29:27.096337+0000","last_became_peered":"2023-07-18T03:29:27.096337+0000","last_unstale":"2023-07-18T03:29:28.084957+0000","last_undegraded":"2023-07-18T03:29:28.084957+0000","last_fullsized":"2023-07-18T03:29:28.084957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:23:46.241177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":58,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:30:04.654580+0000","last_change":"2023-07-18T03:29:26.291622+0000","last_active":"2023-07-18T03:30:04.654580+0000","last_peered":"2023-07-18T03:30:04.654580+0000","last_clean":"2023-07-18T03:30:04.654580+0000","last_became_active":"2023-07-18T03:29:26.291317+0000","last_became_peered":"2023-07-18T03:29:26.291317+0000","last_unstale":"2023-07-18T03:30:04.654580+0000","last_undegraded":"2023-07-18T03:30:04.654580+0000","last_fullsized":"2023-07-18T03:30:04.654580+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:37:53.996938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248914+0000","last_change":"2023-07-18T03:29:27.093107+0000","last_active":"2023-07-18T03:29:28.248914+0000","last_peered":"2023-07-18T03:29:28.248914+0000","last_clean":"2023-07-18T03:29:28.248914+0000","last_became_active":"2023-07-18T03:29:27.092785+0000","last_became_peered":"2023-07-18T03:29:27.092785+0000","last_unstale":"2023-07-18T03:29:28.248914+0000","last_undegraded":"2023-07-18T03:29:28.248914+0000","last_fullsized":"2023-07-18T03:29:28.248914+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:18:41.599570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:32.636567+0000","last_change":"2023-07-18T03:29:26.290493+0000","last_active":"2023-07-18T03:29:32.636567+0000","last_peered":"2023-07-18T03:29:32.636567+0000","last_clean":"2023-07-18T03:29:32.636567+0000","last_became_active":"2023-07-18T03:29:26.290187+0000","last_became_peered":"2023-07-18T03:29:26.290187+0000","last_unstale":"2023-07-18T03:29:32.636567+0000","last_undegraded":"2023-07-18T03:29:32.636567+0000","last_fullsized":"2023-07-18T03:29:32.636567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:14:58.224602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.084989+0000","last_change":"2023-07-18T03:29:27.095129+0000","last_active":"2023-07-18T03:29:28.084989+0000","last_peered":"2023-07-18T03:29:28.084989+0000","last_clean":"2023-07-18T03:29:28.084989+0000","last_became_active":"2023-07-18T03:29:27.094821+0000","last_became_peered":"2023-07-18T03:29:27.094821+0000","last_unstale":"2023-07-18T03:29:28.084989+0000","last_undegraded":"2023-07-18T03:29:28.084989+0000","last_fullsized":"2023-07-18T03:29:28.084989+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:33:21.671061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.612409+0000","last_change":"2023-07-18T03:29:26.285860+0000","last_active":"2023-07-18T03:29:31.612409+0000","last_peered":"2023-07-18T03:29:31.612409+0000","last_clean":"2023-07-18T03:29:31.612409+0000","last_became_active":"2023-07-18T03:29:26.285491+0000","last_became_peered":"2023-07-18T03:29:26.285491+0000","last_unstale":"2023-07-18T03:29:31.612409+0000","last_undegraded":"2023-07-18T03:29:31.612409+0000","last_fullsized":"2023-07-18T03:29:31.612409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:14:20.414177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248682+0000","last_change":"2023-07-18T03:29:27.093861+0000","last_active":"2023-07-18T03:29:28.248682+0000","last_peered":"2023-07-18T03:29:28.248682+0000","last_clean":"2023-07-18T03:29:28.248682+0000","last_became_active":"2023-07-18T03:29:27.093445+0000","last_became_peered":"2023-07-18T03:29:27.093445+0000","last_unstale":"2023-07-18T03:29:28.248682+0000","last_undegraded":"2023-07-18T03:29:28.248682+0000","last_fullsized":"2023-07-18T03:29:28.248682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:05:28.911877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.642632+0000","last_change":"2023-07-18T03:29:26.288108+0000","last_active":"2023-07-18T03:29:33.642632+0000","last_peered":"2023-07-18T03:29:33.642632+0000","last_clean":"2023-07-18T03:29:33.642632+0000","last_became_active":"2023-07-18T03:29:26.287531+0000","last_became_peered":"2023-07-18T03:29:26.287531+0000","last_unstale":"2023-07-18T03:29:33.642632+0000","last_undegraded":"2023-07-18T03:29:33.642632+0000","last_fullsized":"2023-07-18T03:29:33.642632+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:35:36.416356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.235430+0000","last_change":"2023-07-18T03:29:27.092908+0000","last_active":"2023-07-18T03:29:28.235430+0000","last_peered":"2023-07-18T03:29:28.235430+0000","last_clean":"2023-07-18T03:29:28.235430+0000","last_became_active":"2023-07-18T03:29:27.092566+0000","last_became_peered":"2023-07-18T03:29:27.092566+0000","last_unstale":"2023-07-18T03:29:28.235430+0000","last_undegraded":"2023-07-18T03:29:28.235430+0000","last_fullsized":"2023-07-18T03:29:28.235430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:22:47.560376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":83,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:30:04.653808+0000","last_change":"2023-07-18T03:29:26.294251+0000","last_active":"2023-07-18T03:30:04.653808+0000","last_peered":"2023-07-18T03:30:04.653808+0000","last_clean":"2023-07-18T03:30:04.653808+0000","last_became_active":"2023-07-18T03:29:26.293847+0000","last_became_peered":"2023-07-18T03:29:26.293847+0000","last_unstale":"2023-07-18T03:30:04.653808+0000","last_undegraded":"2023-07-18T03:30:04.653808+0000","last_fullsized":"2023-07-18T03:30:04.653808+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:51:26.530234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248737+0000","last_change":"2023-07-18T03:29:27.106551+0000","last_active":"2023-07-18T03:29:28.248737+0000","last_peered":"2023-07-18T03:29:28.248737+0000","last_clean":"2023-07-18T03:29:28.248737+0000","last_became_active":"2023-07-18T03:29:27.105187+0000","last_became_peered":"2023-07-18T03:29:27.105187+0000","last_unstale":"2023-07-18T03:29:28.248737+0000","last_undegraded":"2023-07-18T03:29:28.248737+0000","last_fullsized":"2023-07-18T03:29:28.248737+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:45:05.223020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.640865+0000","last_change":"2023-07-18T03:29:26.293284+0000","last_active":"2023-07-18T03:29:33.640865+0000","last_peered":"2023-07-18T03:29:33.640865+0000","last_clean":"2023-07-18T03:29:33.640865+0000","last_became_active":"2023-07-18T03:29:26.292955+0000","last_became_peered":"2023-07-18T03:29:26.292955+0000","last_unstale":"2023-07-18T03:29:33.640865+0000","last_undegraded":"2023-07-18T03:29:33.640865+0000","last_fullsized":"2023-07-18T03:29:33.640865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:15:36.890771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249337+0000","last_change":"2023-07-18T03:29:27.097322+0000","last_active":"2023-07-18T03:29:28.249337+0000","last_peered":"2023-07-18T03:29:28.249337+0000","last_clean":"2023-07-18T03:29:28.249337+0000","last_became_active":"2023-07-18T03:29:27.097051+0000","last_became_peered":"2023-07-18T03:29:27.097051+0000","last_unstale":"2023-07-18T03:29:28.249337+0000","last_undegraded":"2023-07-18T03:29:28.249337+0000","last_fullsized":"2023-07-18T03:29:28.249337+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:36:33.129283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.601626+0000","last_change":"2023-07-18T03:29:26.290810+0000","last_active":"2023-07-18T03:29:31.601626+0000","last_peered":"2023-07-18T03:29:31.601626+0000","last_clean":"2023-07-18T03:29:31.601626+0000","last_became_active":"2023-07-18T03:29:26.290448+0000","last_became_peered":"2023-07-18T03:29:26.290448+0000","last_unstale":"2023-07-18T03:29:31.601626+0000","last_undegraded":"2023-07-18T03:29:31.601626+0000","last_fullsized":"2023-07-18T03:29:31.601626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:21:19.581142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249116+0000","last_change":"2023-07-18T03:29:27.105801+0000","last_active":"2023-07-18T03:29:28.249116+0000","last_peered":"2023-07-18T03:29:28.249116+0000","last_clean":"2023-07-18T03:29:28.249116+0000","last_became_active":"2023-07-18T03:29:27.104715+0000","last_became_peered":"2023-07-18T03:29:27.104715+0000","last_unstale":"2023-07-18T03:29:28.249116+0000","last_undegraded":"2023-07-18T03:29:28.249116+0000","last_fullsized":"2023-07-18T03:29:28.249116+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:49:34.118157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'29","reported_seq":119,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:30:04.653807+0000","last_change":"2023-07-18T03:29:26.293306+0000","last_active":"2023-07-18T03:30:04.653807+0000","last_peered":"2023-07-18T03:30:04.653807+0000","last_clean":"2023-07-18T03:30:04.653807+0000","last_became_active":"2023-07-18T03:29:26.292946+0000","last_became_peered":"2023-07-18T03:29:26.292946+0000","last_unstale":"2023-07-18T03:30:04.653807+0000","last_undegraded":"2023-07-18T03:30:04.653807+0000","last_fullsized":"2023-07-18T03:30:04.653807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:36:19.196296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58527,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":94,"num_write":31,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249042+0000","last_change":"2023-07-18T03:29:27.103941+0000","last_active":"2023-07-18T03:29:28.249042+0000","last_peered":"2023-07-18T03:29:28.249042+0000","last_clean":"2023-07-18T03:29:28.249042+0000","last_became_active":"2023-07-18T03:29:27.103506+0000","last_became_peered":"2023-07-18T03:29:27.103506+0000","last_unstale":"2023-07-18T03:29:28.249042+0000","last_undegraded":"2023-07-18T03:29:28.249042+0000","last_fullsized":"2023-07-18T03:29:28.249042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:13:27.299459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.642234+0000","last_change":"2023-07-18T03:29:26.293473+0000","last_active":"2023-07-18T03:29:33.642234+0000","last_peered":"2023-07-18T03:29:33.642234+0000","last_clean":"2023-07-18T03:29:33.642234+0000","last_became_active":"2023-07-18T03:29:26.293132+0000","last_became_peered":"2023-07-18T03:29:26.293132+0000","last_unstale":"2023-07-18T03:29:33.642234+0000","last_undegraded":"2023-07-18T03:29:33.642234+0000","last_fullsized":"2023-07-18T03:29:33.642234+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:32:07.174316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249663+0000","last_change":"2023-07-18T03:29:27.104032+0000","last_active":"2023-07-18T03:29:28.249663+0000","last_peered":"2023-07-18T03:29:28.249663+0000","last_clean":"2023-07-18T03:29:28.249663+0000","last_became_active":"2023-07-18T03:29:27.103736+0000","last_became_peered":"2023-07-18T03:29:27.103736+0000","last_unstale":"2023-07-18T03:29:28.249663+0000","last_undegraded":"2023-07-18T03:29:28.249663+0000","last_fullsized":"2023-07-18T03:29:28.249663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:29:13.489385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.642720+0000","last_change":"2023-07-18T03:29:26.290487+0000","last_active":"2023-07-18T03:29:33.642720+0000","last_peered":"2023-07-18T03:29:33.642720+0000","last_clean":"2023-07-18T03:29:33.642720+0000","last_became_active":"2023-07-18T03:29:26.290115+0000","last_became_peered":"2023-07-18T03:29:26.290115+0000","last_unstale":"2023-07-18T03:29:33.642720+0000","last_undegraded":"2023-07-18T03:29:33.642720+0000","last_fullsized":"2023-07-18T03:29:33.642720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:35:42.866787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339569+0000","last_change":"2023-07-18T03:29:27.108550+0000","last_active":"2023-07-18T03:29:28.339569+0000","last_peered":"2023-07-18T03:29:28.339569+0000","last_clean":"2023-07-18T03:29:28.339569+0000","last_became_active":"2023-07-18T03:29:27.108069+0000","last_became_peered":"2023-07-18T03:29:27.108069+0000","last_unstale":"2023-07-18T03:29:28.339569+0000","last_undegraded":"2023-07-18T03:29:28.339569+0000","last_fullsized":"2023-07-18T03:29:28.339569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:55:48.510403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:30:04.708509+0000","last_change":"2023-07-18T03:29:26.290896+0000","last_active":"2023-07-18T03:30:04.708509+0000","last_peered":"2023-07-18T03:30:04.708509+0000","last_clean":"2023-07-18T03:30:04.708509+0000","last_became_active":"2023-07-18T03:29:26.290450+0000","last_became_peered":"2023-07-18T03:29:26.290450+0000","last_unstale":"2023-07-18T03:30:04.708509+0000","last_undegraded":"2023-07-18T03:30:04.708509+0000","last_fullsized":"2023-07-18T03:30:04.708509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:23:16.823095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339200+0000","last_change":"2023-07-18T03:29:27.109184+0000","last_active":"2023-07-18T03:29:28.339200+0000","last_peered":"2023-07-18T03:29:28.339200+0000","last_clean":"2023-07-18T03:29:28.339200+0000","last_became_active":"2023-07-18T03:29:27.107993+0000","last_became_peered":"2023-07-18T03:29:27.107993+0000","last_unstale":"2023-07-18T03:29:28.339200+0000","last_undegraded":"2023-07-18T03:29:28.339200+0000","last_fullsized":"2023-07-18T03:29:28.339200+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:38:53.665857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.641695+0000","last_change":"2023-07-18T03:29:26.293671+0000","last_active":"2023-07-18T03:29:33.641695+0000","last_peered":"2023-07-18T03:29:33.641695+0000","last_clean":"2023-07-18T03:29:33.641695+0000","last_became_active":"2023-07-18T03:29:26.293205+0000","last_became_peered":"2023-07-18T03:29:26.293205+0000","last_unstale":"2023-07-18T03:29:33.641695+0000","last_undegraded":"2023-07-18T03:29:33.641695+0000","last_fullsized":"2023-07-18T03:29:33.641695+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:52:09.605399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.084887+0000","last_change":"2023-07-18T03:29:27.096711+0000","last_active":"2023-07-18T03:29:28.084887+0000","last_peered":"2023-07-18T03:29:28.084887+0000","last_clean":"2023-07-18T03:29:28.084887+0000","last_became_active":"2023-07-18T03:29:27.096423+0000","last_became_peered":"2023-07-18T03:29:27.096423+0000","last_unstale":"2023-07-18T03:29:28.084887+0000","last_undegraded":"2023-07-18T03:29:28.084887+0000","last_fullsized":"2023-07-18T03:29:28.084887+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:35:54.060369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.598395+0000","last_change":"2023-07-18T03:29:26.291546+0000","last_active":"2023-07-18T03:29:31.598395+0000","last_peered":"2023-07-18T03:29:31.598395+0000","last_clean":"2023-07-18T03:29:31.598395+0000","last_became_active":"2023-07-18T03:29:26.291319+0000","last_became_peered":"2023-07-18T03:29:26.291319+0000","last_unstale":"2023-07-18T03:29:31.598395+0000","last_undegraded":"2023-07-18T03:29:31.598395+0000","last_fullsized":"2023-07-18T03:29:31.598395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:23:25.182800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085688+0000","last_change":"2023-07-18T03:29:27.098479+0000","last_active":"2023-07-18T03:29:28.085688+0000","last_peered":"2023-07-18T03:29:28.085688+0000","last_clean":"2023-07-18T03:29:28.085688+0000","last_became_active":"2023-07-18T03:29:27.098153+0000","last_became_peered":"2023-07-18T03:29:27.098153+0000","last_unstale":"2023-07-18T03:29:28.085688+0000","last_undegraded":"2023-07-18T03:29:28.085688+0000","last_fullsized":"2023-07-18T03:29:28.085688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:53:01.674305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:29.873882+0000","last_change":"2023-07-18T03:29:26.290953+0000","last_active":"2023-07-18T03:29:29.873882+0000","last_peered":"2023-07-18T03:29:29.873882+0000","last_clean":"2023-07-18T03:29:29.873882+0000","last_became_active":"2023-07-18T03:29:26.290633+0000","last_became_peered":"2023-07-18T03:29:26.290633+0000","last_unstale":"2023-07-18T03:29:29.873882+0000","last_undegraded":"2023-07-18T03:29:29.873882+0000","last_fullsized":"2023-07-18T03:29:29.873882+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:40:49.185595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.611541+0000","last_change":"2023-07-18T03:29:27.093651+0000","last_active":"2023-07-18T03:29:30.611541+0000","last_peered":"2023-07-18T03:29:30.611541+0000","last_clean":"2023-07-18T03:29:30.611541+0000","last_became_active":"2023-07-18T03:29:27.093307+0000","last_became_peered":"2023-07-18T03:29:27.093307+0000","last_unstale":"2023-07-18T03:29:30.611541+0000","last_undegraded":"2023-07-18T03:29:30.611541+0000","last_fullsized":"2023-07-18T03:29:30.611541+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:31:10.315148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:30:05.710526+0000","last_change":"2023-07-18T03:29:26.287405+0000","last_active":"2023-07-18T03:30:05.710526+0000","last_peered":"2023-07-18T03:30:05.710526+0000","last_clean":"2023-07-18T03:30:05.710526+0000","last_became_active":"2023-07-18T03:29:26.287048+0000","last_became_peered":"2023-07-18T03:29:26.287048+0000","last_unstale":"2023-07-18T03:30:05.710526+0000","last_undegraded":"2023-07-18T03:30:05.710526+0000","last_fullsized":"2023-07-18T03:30:05.710526+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:50:53.303258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085803+0000","last_change":"2023-07-18T03:29:27.103218+0000","last_active":"2023-07-18T03:29:28.085803+0000","last_peered":"2023-07-18T03:29:28.085803+0000","last_clean":"2023-07-18T03:29:28.085803+0000","last_became_active":"2023-07-18T03:29:27.102917+0000","last_became_peered":"2023-07-18T03:29:27.102917+0000","last_unstale":"2023-07-18T03:29:28.085803+0000","last_undegraded":"2023-07-18T03:29:28.085803+0000","last_fullsized":"2023-07-18T03:29:28.085803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:23:49.743367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085272+0000","last_change":"2023-07-18T03:29:27.098696+0000","last_active":"2023-07-18T03:29:28.085272+0000","last_peered":"2023-07-18T03:29:28.085272+0000","last_clean":"2023-07-18T03:29:28.085272+0000","last_became_active":"2023-07-18T03:29:27.098253+0000","last_became_peered":"2023-07-18T03:29:27.098253+0000","last_unstale":"2023-07-18T03:29:28.085272+0000","last_undegraded":"2023-07-18T03:29:28.085272+0000","last_fullsized":"2023-07-18T03:29:28.085272+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:40:58.942045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:30:05.709200+0000","last_change":"2023-07-18T03:29:26.290308+0000","last_active":"2023-07-18T03:30:05.709200+0000","last_peered":"2023-07-18T03:30:05.709200+0000","last_clean":"2023-07-18T03:30:05.709200+0000","last_became_active":"2023-07-18T03:29:26.289937+0000","last_became_peered":"2023-07-18T03:29:26.289937+0000","last_unstale":"2023-07-18T03:30:05.709200+0000","last_undegraded":"2023-07-18T03:30:05.709200+0000","last_fullsized":"2023-07-18T03:30:05.709200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:21:51.452197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249450+0000","last_change":"2023-07-18T03:29:27.096469+0000","last_active":"2023-07-18T03:29:28.249450+0000","last_peered":"2023-07-18T03:29:28.249450+0000","last_clean":"2023-07-18T03:29:28.249450+0000","last_became_active":"2023-07-18T03:29:27.096171+0000","last_became_peered":"2023-07-18T03:29:27.096171+0000","last_unstale":"2023-07-18T03:29:28.249450+0000","last_undegraded":"2023-07-18T03:29:28.249450+0000","last_fullsized":"2023-07-18T03:29:28.249450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:14:47.599303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:34.656948+0000","last_change":"2023-07-18T03:29:26.290935+0000","last_active":"2023-07-18T03:29:34.656948+0000","last_peered":"2023-07-18T03:29:34.656948+0000","last_clean":"2023-07-18T03:29:34.656948+0000","last_became_active":"2023-07-18T03:29:26.290595+0000","last_became_peered":"2023-07-18T03:29:26.290595+0000","last_unstale":"2023-07-18T03:29:34.656948+0000","last_undegraded":"2023-07-18T03:29:34.656948+0000","last_fullsized":"2023-07-18T03:29:34.656948+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:34:23.417106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":331,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339337+0000","last_change":"2023-07-18T03:28:28.122385+0000","last_active":"2023-07-18T03:29:28.339337+0000","last_peered":"2023-07-18T03:29:28.339337+0000","last_clean":"2023-07-18T03:29:28.339337+0000","last_became_active":"2023-07-18T03:28:28.121932+0000","last_became_peered":"2023-07-18T03:28:28.121932+0000","last_unstale":"2023-07-18T03:29:28.339337+0000","last_undegraded":"2023-07-18T03:29:28.339337+0000","last_fullsized":"2023-07-18T03:29:28.339337+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:25:38.267627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:25:38.267627+0000","last_clean_scrub_stamp":"2023-07-18T03:25:38.267627+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:46:37.243383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248934+0000","last_change":"2023-07-18T03:29:27.101582+0000","last_active":"2023-07-18T03:29:28.248934+0000","last_peered":"2023-07-18T03:29:28.248934+0000","last_clean":"2023-07-18T03:29:28.248934+0000","last_became_active":"2023-07-18T03:29:27.101226+0000","last_became_peered":"2023-07-18T03:29:27.101226+0000","last_unstale":"2023-07-18T03:29:28.248934+0000","last_undegraded":"2023-07-18T03:29:28.248934+0000","last_fullsized":"2023-07-18T03:29:28.248934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:42:14.571198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.608398+0000","last_change":"2023-07-18T03:29:26.291787+0000","last_active":"2023-07-18T03:29:31.608398+0000","last_peered":"2023-07-18T03:29:31.608398+0000","last_clean":"2023-07-18T03:29:31.608398+0000","last_became_active":"2023-07-18T03:29:26.291471+0000","last_became_peered":"2023-07-18T03:29:26.291471+0000","last_unstale":"2023-07-18T03:29:31.608398+0000","last_undegraded":"2023-07-18T03:29:31.608398+0000","last_fullsized":"2023-07-18T03:29:31.608398+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:49:20.527148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:32.635875+0000","last_change":"2023-07-18T03:29:26.296936+0000","last_active":"2023-07-18T03:29:32.635875+0000","last_peered":"2023-07-18T03:29:32.635875+0000","last_clean":"2023-07-18T03:29:32.635875+0000","last_became_active":"2023-07-18T03:29:26.296241+0000","last_became_peered":"2023-07-18T03:29:26.296241+0000","last_unstale":"2023-07-18T03:29:32.635875+0000","last_undegraded":"2023-07-18T03:29:32.635875+0000","last_fullsized":"2023-07-18T03:29:32.635875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:23:24.313635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249296+0000","last_change":"2023-07-18T03:29:27.102715+0000","last_active":"2023-07-18T03:29:28.249296+0000","last_peered":"2023-07-18T03:29:28.249296+0000","last_clean":"2023-07-18T03:29:28.249296+0000","last_became_active":"2023-07-18T03:29:27.100914+0000","last_became_peered":"2023-07-18T03:29:27.100914+0000","last_unstale":"2023-07-18T03:29:28.249296+0000","last_undegraded":"2023-07-18T03:29:28.249296+0000","last_fullsized":"2023-07-18T03:29:28.249296+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:43:38.965763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'9","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:35.629147+0000","last_change":"2023-07-18T03:29:26.290231+0000","last_active":"2023-07-18T03:29:35.629147+0000","last_peered":"2023-07-18T03:29:35.629147+0000","last_clean":"2023-07-18T03:29:35.629147+0000","last_became_active":"2023-07-18T03:29:26.289753+0000","last_became_peered":"2023-07-18T03:29:26.289753+0000","last_unstale":"2023-07-18T03:29:35.629147+0000","last_undegraded":"2023-07-18T03:29:35.629147+0000","last_fullsized":"2023-07-18T03:29:35.629147+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:33:36.792644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249097+0000","last_change":"2023-07-18T03:29:27.103964+0000","last_active":"2023-07-18T03:29:28.249097+0000","last_peered":"2023-07-18T03:29:28.249097+0000","last_clean":"2023-07-18T03:29:28.249097+0000","last_became_active":"2023-07-18T03:29:27.103660+0000","last_became_peered":"2023-07-18T03:29:27.103660+0000","last_unstale":"2023-07-18T03:29:28.249097+0000","last_undegraded":"2023-07-18T03:29:28.249097+0000","last_fullsized":"2023-07-18T03:29:28.249097+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:08:09.167239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.641324+0000","last_change":"2023-07-18T03:29:26.286436+0000","last_active":"2023-07-18T03:29:33.641324+0000","last_peered":"2023-07-18T03:29:33.641324+0000","last_clean":"2023-07-18T03:29:33.641324+0000","last_became_active":"2023-07-18T03:29:26.286070+0000","last_became_peered":"2023-07-18T03:29:26.286070+0000","last_unstale":"2023-07-18T03:29:33.641324+0000","last_undegraded":"2023-07-18T03:29:33.641324+0000","last_fullsized":"2023-07-18T03:29:33.641324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:14:53.466702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249329+0000","last_change":"2023-07-18T03:29:27.102303+0000","last_active":"2023-07-18T03:29:28.249329+0000","last_peered":"2023-07-18T03:29:28.249329+0000","last_clean":"2023-07-18T03:29:28.249329+0000","last_became_active":"2023-07-18T03:29:27.101980+0000","last_became_peered":"2023-07-18T03:29:27.101980+0000","last_unstale":"2023-07-18T03:29:28.249329+0000","last_undegraded":"2023-07-18T03:29:28.249329+0000","last_fullsized":"2023-07-18T03:29:28.249329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:46:48.299459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.624991+0000","last_change":"2023-07-18T03:29:26.289628+0000","last_active":"2023-07-18T03:29:30.624991+0000","last_peered":"2023-07-18T03:29:30.624991+0000","last_clean":"2023-07-18T03:29:30.624991+0000","last_became_active":"2023-07-18T03:29:26.289013+0000","last_became_peered":"2023-07-18T03:29:26.289013+0000","last_unstale":"2023-07-18T03:29:30.624991+0000","last_undegraded":"2023-07-18T03:29:30.624991+0000","last_fullsized":"2023-07-18T03:29:30.624991+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:58:20.665607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248709+0000","last_change":"2023-07-18T03:29:27.090943+0000","last_active":"2023-07-18T03:29:28.248709+0000","last_peered":"2023-07-18T03:29:28.248709+0000","last_clean":"2023-07-18T03:29:28.248709+0000","last_became_active":"2023-07-18T03:29:27.090465+0000","last_became_peered":"2023-07-18T03:29:27.090465+0000","last_unstale":"2023-07-18T03:29:28.248709+0000","last_undegraded":"2023-07-18T03:29:28.248709+0000","last_fullsized":"2023-07-18T03:29:28.248709+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:49:17.548432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.642141+0000","last_change":"2023-07-18T03:29:26.288576+0000","last_active":"2023-07-18T03:29:33.642141+0000","last_peered":"2023-07-18T03:29:33.642141+0000","last_clean":"2023-07-18T03:29:33.642141+0000","last_became_active":"2023-07-18T03:29:26.287514+0000","last_became_peered":"2023-07-18T03:29:26.287514+0000","last_unstale":"2023-07-18T03:29:33.642141+0000","last_undegraded":"2023-07-18T03:29:33.642141+0000","last_fullsized":"2023-07-18T03:29:33.642141+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:37:47.805842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085745+0000","last_change":"2023-07-18T03:29:27.101111+0000","last_active":"2023-07-18T03:29:28.085745+0000","last_peered":"2023-07-18T03:29:28.085745+0000","last_clean":"2023-07-18T03:29:28.085745+0000","last_became_active":"2023-07-18T03:29:27.100737+0000","last_became_peered":"2023-07-18T03:29:27.100737+0000","last_unstale":"2023-07-18T03:29:28.085745+0000","last_undegraded":"2023-07-18T03:29:28.085745+0000","last_fullsized":"2023-07-18T03:29:28.085745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:00:58.906730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:34.044421+0000","last_change":"2023-07-18T03:29:26.296974+0000","last_active":"2023-07-18T03:29:34.044421+0000","last_peered":"2023-07-18T03:29:34.044421+0000","last_clean":"2023-07-18T03:29:34.044421+0000","last_became_active":"2023-07-18T03:29:26.296451+0000","last_became_peered":"2023-07-18T03:29:26.296451+0000","last_unstale":"2023-07-18T03:29:34.044421+0000","last_undegraded":"2023-07-18T03:29:34.044421+0000","last_fullsized":"2023-07-18T03:29:34.044421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:35:42.821600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249813+0000","last_change":"2023-07-18T03:29:27.095857+0000","last_active":"2023-07-18T03:29:28.249813+0000","last_peered":"2023-07-18T03:29:28.249813+0000","last_clean":"2023-07-18T03:29:28.249813+0000","last_became_active":"2023-07-18T03:29:27.095553+0000","last_became_peered":"2023-07-18T03:29:27.095553+0000","last_unstale":"2023-07-18T03:29:28.249813+0000","last_undegraded":"2023-07-18T03:29:28.249813+0000","last_fullsized":"2023-07-18T03:29:28.249813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:30:11.142073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.627680+0000","last_change":"2023-07-18T03:29:26.295936+0000","last_active":"2023-07-18T03:29:30.627680+0000","last_peered":"2023-07-18T03:29:30.627680+0000","last_clean":"2023-07-18T03:29:30.627680+0000","last_became_active":"2023-07-18T03:29:26.295467+0000","last_became_peered":"2023-07-18T03:29:26.295467+0000","last_unstale":"2023-07-18T03:29:30.627680+0000","last_undegraded":"2023-07-18T03:29:30.627680+0000","last_fullsized":"2023-07-18T03:29:30.627680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:45:43.716125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339616+0000","last_change":"2023-07-18T03:29:27.108528+0000","last_active":"2023-07-18T03:29:28.339616+0000","last_peered":"2023-07-18T03:29:28.339616+0000","last_clean":"2023-07-18T03:29:28.339616+0000","last_became_active":"2023-07-18T03:29:27.108039+0000","last_became_peered":"2023-07-18T03:29:27.108039+0000","last_unstale":"2023-07-18T03:29:28.339616+0000","last_undegraded":"2023-07-18T03:29:28.339616+0000","last_fullsized":"2023-07-18T03:29:28.339616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:57:20.088291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'5","reported_seq":50,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:30:01.707015+0000","last_change":"2023-07-18T03:29:26.294006+0000","last_active":"2023-07-18T03:30:01.707015+0000","last_peered":"2023-07-18T03:30:01.707015+0000","last_clean":"2023-07-18T03:30:01.707015+0000","last_became_active":"2023-07-18T03:29:26.293700+0000","last_became_peered":"2023-07-18T03:29:26.293700+0000","last_unstale":"2023-07-18T03:30:01.707015+0000","last_undegraded":"2023-07-18T03:30:01.707015+0000","last_fullsized":"2023-07-18T03:30:01.707015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:44:37.451806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339715+0000","last_change":"2023-07-18T03:29:27.102764+0000","last_active":"2023-07-18T03:29:28.339715+0000","last_peered":"2023-07-18T03:29:28.339715+0000","last_clean":"2023-07-18T03:29:28.339715+0000","last_became_active":"2023-07-18T03:29:27.102200+0000","last_became_peered":"2023-07-18T03:29:27.102200+0000","last_unstale":"2023-07-18T03:29:28.339715+0000","last_undegraded":"2023-07-18T03:29:28.339715+0000","last_fullsized":"2023-07-18T03:29:28.339715+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:54:13.608760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:32.635822+0000","last_change":"2023-07-18T03:29:26.294271+0000","last_active":"2023-07-18T03:29:32.635822+0000","last_peered":"2023-07-18T03:29:32.635822+0000","last_clean":"2023-07-18T03:29:32.635822+0000","last_became_active":"2023-07-18T03:29:26.293938+0000","last_became_peered":"2023-07-18T03:29:26.293938+0000","last_unstale":"2023-07-18T03:29:32.635822+0000","last_undegraded":"2023-07-18T03:29:32.635822+0000","last_fullsized":"2023-07-18T03:29:32.635822+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:10:38.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339451+0000","last_change":"2023-07-18T03:29:27.106510+0000","last_active":"2023-07-18T03:29:28.339451+0000","last_peered":"2023-07-18T03:29:28.339451+0000","last_clean":"2023-07-18T03:29:28.339451+0000","last_became_active":"2023-07-18T03:29:27.106222+0000","last_became_peered":"2023-07-18T03:29:27.106222+0000","last_unstale":"2023-07-18T03:29:28.339451+0000","last_undegraded":"2023-07-18T03:29:28.339451+0000","last_fullsized":"2023-07-18T03:29:28.339451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:27:44.790790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.640406+0000","last_change":"2023-07-18T03:29:26.293583+0000","last_active":"2023-07-18T03:29:33.640406+0000","last_peered":"2023-07-18T03:29:33.640406+0000","last_clean":"2023-07-18T03:29:33.640406+0000","last_became_active":"2023-07-18T03:29:26.293076+0000","last_became_peered":"2023-07-18T03:29:26.293076+0000","last_unstale":"2023-07-18T03:29:33.640406+0000","last_undegraded":"2023-07-18T03:29:33.640406+0000","last_fullsized":"2023-07-18T03:29:33.640406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:37:21.591425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339559+0000","last_change":"2023-07-18T03:29:27.109175+0000","last_active":"2023-07-18T03:29:28.339559+0000","last_peered":"2023-07-18T03:29:28.339559+0000","last_clean":"2023-07-18T03:29:28.339559+0000","last_became_active":"2023-07-18T03:29:27.108669+0000","last_became_peered":"2023-07-18T03:29:27.108669+0000","last_unstale":"2023-07-18T03:29:28.339559+0000","last_undegraded":"2023-07-18T03:29:28.339559+0000","last_fullsized":"2023-07-18T03:29:28.339559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:00:09.903589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085918+0000","last_change":"2023-07-18T03:29:27.099770+0000","last_active":"2023-07-18T03:29:28.085918+0000","last_peered":"2023-07-18T03:29:28.085918+0000","last_clean":"2023-07-18T03:29:28.085918+0000","last_became_active":"2023-07-18T03:29:27.099487+0000","last_became_peered":"2023-07-18T03:29:27.099487+0000","last_unstale":"2023-07-18T03:29:28.085918+0000","last_undegraded":"2023-07-18T03:29:28.085918+0000","last_fullsized":"2023-07-18T03:29:28.085918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:22:59.316119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129653+0000","last_change":"2023-07-18T03:29:27.100189+0000","last_active":"2023-07-18T03:29:28.129653+0000","last_peered":"2023-07-18T03:29:28.129653+0000","last_clean":"2023-07-18T03:29:28.129653+0000","last_became_active":"2023-07-18T03:29:27.099884+0000","last_became_peered":"2023-07-18T03:29:27.099884+0000","last_unstale":"2023-07-18T03:29:28.129653+0000","last_undegraded":"2023-07-18T03:29:28.129653+0000","last_fullsized":"2023-07-18T03:29:28.129653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:04:36.761236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129593+0000","last_change":"2023-07-18T03:29:27.099196+0000","last_active":"2023-07-18T03:29:28.129593+0000","last_peered":"2023-07-18T03:29:28.129593+0000","last_clean":"2023-07-18T03:29:28.129593+0000","last_became_active":"2023-07-18T03:29:27.098903+0000","last_became_peered":"2023-07-18T03:29:27.098903+0000","last_unstale":"2023-07-18T03:29:28.129593+0000","last_undegraded":"2023-07-18T03:29:28.129593+0000","last_fullsized":"2023-07-18T03:29:28.129593+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:46:05.998503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248469+0000","last_change":"2023-07-18T03:29:27.098138+0000","last_active":"2023-07-18T03:29:28.248469+0000","last_peered":"2023-07-18T03:29:28.248469+0000","last_clean":"2023-07-18T03:29:28.248469+0000","last_became_active":"2023-07-18T03:29:27.097793+0000","last_became_peered":"2023-07-18T03:29:27.097793+0000","last_unstale":"2023-07-18T03:29:28.248469+0000","last_undegraded":"2023-07-18T03:29:28.248469+0000","last_fullsized":"2023-07-18T03:29:28.248469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:07:31.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249184+0000","last_change":"2023-07-18T03:29:27.101610+0000","last_active":"2023-07-18T03:29:28.249184+0000","last_peered":"2023-07-18T03:29:28.249184+0000","last_clean":"2023-07-18T03:29:28.249184+0000","last_became_active":"2023-07-18T03:29:27.101332+0000","last_became_peered":"2023-07-18T03:29:27.101332+0000","last_unstale":"2023-07-18T03:29:28.249184+0000","last_undegraded":"2023-07-18T03:29:28.249184+0000","last_fullsized":"2023-07-18T03:29:28.249184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:43:01.583313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085879+0000","last_change":"2023-07-18T03:29:27.093345+0000","last_active":"2023-07-18T03:29:28.085879+0000","last_peered":"2023-07-18T03:29:28.085879+0000","last_clean":"2023-07-18T03:29:28.085879+0000","last_became_active":"2023-07-18T03:29:27.093048+0000","last_became_peered":"2023-07-18T03:29:27.093048+0000","last_unstale":"2023-07-18T03:29:28.085879+0000","last_undegraded":"2023-07-18T03:29:28.085879+0000","last_fullsized":"2023-07-18T03:29:28.085879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:00:19.659948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129830+0000","last_change":"2023-07-18T03:29:27.098311+0000","last_active":"2023-07-18T03:29:28.129830+0000","last_peered":"2023-07-18T03:29:28.129830+0000","last_clean":"2023-07-18T03:29:28.129830+0000","last_became_active":"2023-07-18T03:29:27.097980+0000","last_became_peered":"2023-07-18T03:29:27.097980+0000","last_unstale":"2023-07-18T03:29:28.129830+0000","last_undegraded":"2023-07-18T03:29:28.129830+0000","last_fullsized":"2023-07-18T03:29:28.129830+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:34:48.849264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248965+0000","last_change":"2023-07-18T03:29:27.093796+0000","last_active":"2023-07-18T03:29:28.248965+0000","last_peered":"2023-07-18T03:29:28.248965+0000","last_clean":"2023-07-18T03:29:28.248965+0000","last_became_active":"2023-07-18T03:29:27.093474+0000","last_became_peered":"2023-07-18T03:29:27.093474+0000","last_unstale":"2023-07-18T03:29:28.248965+0000","last_undegraded":"2023-07-18T03:29:28.248965+0000","last_fullsized":"2023-07-18T03:29:28.248965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:00:09.316029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085099+0000","last_change":"2023-07-18T03:29:27.098541+0000","last_active":"2023-07-18T03:29:28.085099+0000","last_peered":"2023-07-18T03:29:28.085099+0000","last_clean":"2023-07-18T03:29:28.085099+0000","last_became_active":"2023-07-18T03:29:27.097824+0000","last_became_peered":"2023-07-18T03:29:27.097824+0000","last_unstale":"2023-07-18T03:29:28.085099+0000","last_undegraded":"2023-07-18T03:29:28.085099+0000","last_fullsized":"2023-07-18T03:29:28.085099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:28:38.837863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248860+0000","last_change":"2023-07-18T03:29:27.096983+0000","last_active":"2023-07-18T03:29:28.248860+0000","last_peered":"2023-07-18T03:29:28.248860+0000","last_clean":"2023-07-18T03:29:28.248860+0000","last_became_active":"2023-07-18T03:29:27.096632+0000","last_became_peered":"2023-07-18T03:29:27.096632+0000","last_unstale":"2023-07-18T03:29:28.248860+0000","last_undegraded":"2023-07-18T03:29:28.248860+0000","last_fullsized":"2023-07-18T03:29:28.248860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:45:45.965235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249073+0000","last_change":"2023-07-18T03:29:27.105236+0000","last_active":"2023-07-18T03:29:28.249073+0000","last_peered":"2023-07-18T03:29:28.249073+0000","last_clean":"2023-07-18T03:29:28.249073+0000","last_became_active":"2023-07-18T03:29:27.104906+0000","last_became_peered":"2023-07-18T03:29:27.104906+0000","last_unstale":"2023-07-18T03:29:28.249073+0000","last_undegraded":"2023-07-18T03:29:28.249073+0000","last_fullsized":"2023-07-18T03:29:28.249073+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:53:05.499609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249970+0000","last_change":"2023-07-18T03:29:27.103461+0000","last_active":"2023-07-18T03:29:28.249970+0000","last_peered":"2023-07-18T03:29:28.249970+0000","last_clean":"2023-07-18T03:29:28.249970+0000","last_became_active":"2023-07-18T03:29:27.103162+0000","last_became_peered":"2023-07-18T03:29:27.103162+0000","last_unstale":"2023-07-18T03:29:28.249970+0000","last_undegraded":"2023-07-18T03:29:28.249970+0000","last_fullsized":"2023-07-18T03:29:28.249970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:26:14.600362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085635+0000","last_change":"2023-07-18T03:29:27.095300+0000","last_active":"2023-07-18T03:29:28.085635+0000","last_peered":"2023-07-18T03:29:28.085635+0000","last_clean":"2023-07-18T03:29:28.085635+0000","last_became_active":"2023-07-18T03:29:27.094985+0000","last_became_peered":"2023-07-18T03:29:27.094985+0000","last_unstale":"2023-07-18T03:29:28.085635+0000","last_undegraded":"2023-07-18T03:29:28.085635+0000","last_fullsized":"2023-07-18T03:29:28.085635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:09:05.373008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65147,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":365,"num_read_kb":254,"num_write":229,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":195441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":418360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":0.42599999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.72699999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.495},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":4,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60099999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.79600000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.77500000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":9,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.627}]},{"osd":10,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.86499999999999999}]}]},{"osd":10,"up_from":69,"seq":296352743448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28496,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716656,"statfs":{"total":95995035648,"available":95965855744,"internally_reserved":0,"allocated":1581056,"data_stored":1067922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.628},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.751}]},{"osd":2,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1,"5min":1,"15min":1},"last":0.78400000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.57799999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.64500000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.88600000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.81899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":8,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.87},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.82799999999999996}]},{"osd":9,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.80500000000000005}]},{"osd":11,"last update":"Tue Jul 18 03:29:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.84999999999999998}]}]},{"osd":9,"up_from":64,"seq":274877906972,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":418338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.34599999999999997}]},{"osd":1,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.501}]},{"osd":2,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.44500000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.40999999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53200000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.70799999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.51800000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.63200000000000001}]},{"osd":8,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.754},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.73399999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.71599999999999997}]}]},{"osd":8,"up_from":56,"seq":240518168608,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":418385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.39500000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.747,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57499999999999996}]},{"osd":1,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.49399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.41199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86599999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65700000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.439,"15min":0.433},"min":{"1min":0.35099999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.45700000000000002,"15min":0.44700000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.45700000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57899999999999996,"15min":0.57299999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.63400000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.749},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.63600000000000001,"15min":0.62},"min":{"1min":0.42999999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.76800000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.373,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54500000000000004,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.68100000000000005}]},{"osd":6,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.52000000000000002,"5min":0.437,"15min":0.437},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69599999999999995,"15min":0.69399999999999995},"min":{"1min":0.49399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89200000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.78300000000000003}]},{"osd":7,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.5,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.47099999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.81399999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:30:05 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.752,"15min":0.72999999999999998},"min":{"1min":0.56499999999999995,"5min":0.505,"15min":0.505},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.78300000000000003,"15min":0.77300000000000002},"min":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.89400000000000002}]},{"osd":10,"last update":"Tue Jul 18 03:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.85199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":7,"up_from":49,"seq":210453397540,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28416,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716736,"statfs":{"total":95995035648,"available":95965937664,"internally_reserved":0,"allocated":1499136,"data_stored":1011095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46200000000000002,"15min":0.45900000000000002},"min":{"1min":0.30599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.753,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.48399999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.61299999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.379,"15min":0.379},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.621,"15min":0.60899999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.81399999999999995}]},{"osd":3,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.64900000000000002,"15min":0.63900000000000001},"min":{"1min":0.40600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93400000000000005,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66400000000000003,"15min":0.65800000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.79100000000000004}]},{"osd":4,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.63800000000000001}]},{"osd":5,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.48299999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.70899999999999996,"15min":0.69599999999999995},"min":{"1min":0.46600000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":1.0609999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.74399999999999999,"15min":0.73399999999999999},"min":{"1min":0.499,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.04,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.52900000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97599999999999998,"5min":1.103,"15min":1.103},"last":0.92200000000000004}]},{"osd":8,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.80400000000000005,"15min":0.77600000000000002},"min":{"1min":0.51300000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":4.5880000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.76900000000000002,"15min":0.73299999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.6050000000000004,"5min":4.6050000000000004,"15min":4.6050000000000004},"last":1.0860000000000001}]},{"osd":9,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":1.024},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":1.0069999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.94399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]}]},{"osd":6,"up_from":44,"seq":188978561065,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":418013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40200000000000002},"min":{"1min":0.309,"5min":0.254,"15min":0.254},"max":{"1min":0.60299999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.32100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.58099999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.39600000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.45600000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.42099999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78800000000000003,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.48899999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83599999999999997,"5min":1.008,"15min":1.008},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.63500000000000001,"15min":0.61899999999999999},"min":{"1min":0.48499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.52400000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.498,"15min":0.48899999999999999},"min":{"1min":0.36399999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.373},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.36699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71099999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.42599999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.623,"15min":0.61399999999999999},"min":{"1min":0.52000000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85299999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.68500000000000005,"15min":0.67100000000000004},"min":{"1min":0.55100000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.60099999999999998}]},{"osd":5,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.68100000000000005,"15min":0.67100000000000004},"min":{"1min":0.54400000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.503,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.96399999999999997,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.61699999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73199999999999998,"15min":0.72799999999999998},"min":{"1min":0.47199999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56899999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0580000000000001,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.56499999999999995}]},{"osd":8,"last update":"Tue Jul 18 03:29:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.86399999999999999,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":5.1760000000000002,"5min":5.1760000000000002,"15min":5.1760000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.77400000000000002,"15min":0.78000000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.59299999999999997}]},{"osd":9,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.63900000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.71999999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":477447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51500000000000001,"15min":0.47899999999999998},"min":{"1min":0.34999999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81100000000000005,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.52600000000000002,"15min":0.503},"min":{"1min":0.38300000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.89100000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58699999999999997,"15min":0.54800000000000004},"min":{"1min":0.50600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88400000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56699999999999995,"15min":0.52500000000000002},"min":{"1min":0.51200000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.68799999999999994}]},{"osd":3,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.59099999999999997,"15min":0.55500000000000005},"min":{"1min":0.441,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.60799999999999998,"15min":0.57799999999999996},"min":{"1min":0.54700000000000004,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.81999999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.44700000000000001,"15min":0.40699999999999997},"min":{"1min":0.33200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.45000000000000001,"15min":0.42199999999999999},"min":{"1min":0.30499999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.752,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.63600000000000001,"15min":0.58899999999999997},"min":{"1min":0.53000000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.66400000000000003,"15min":0.622},"min":{"1min":0.56799999999999995,"5min":0.433,"15min":0.433},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.89400000000000002}]},{"osd":6,"last update":"Tue Jul 18 03:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.63200000000000001,"15min":0.61699999999999999},"min":{"1min":0.498,"5min":0.443,"15min":0.443},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.68500000000000005,"15min":0.66900000000000004},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.69599999999999995,"15min":0.67700000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.97699999999999998}]},{"osd":8,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.81299999999999994,"15min":0.80600000000000005},"min":{"1min":0.51900000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.01},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.78900000000000003,"15min":0.77900000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.96399999999999997}]},{"osd":9,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.88400000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.038}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":419346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53600000000000003,"15min":0.51400000000000001},"min":{"1min":0.35999999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.374,"5min":0.318,"15min":0.318},"max":{"1min":0.82299999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.38900000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57899999999999996,"15min":0.53100000000000003},"min":{"1min":0.46200000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.879,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56299999999999994,"15min":0.52700000000000002},"min":{"1min":0.39300000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80400000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.60199999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.42299999999999999,"15min":0.41499999999999998},"min":{"1min":0.315,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.628,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.32700000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56899999999999995,"15min":0.55100000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.92800000000000005,"5min":1.018,"15min":1.018},"last":0.40799999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.67500000000000004,"15min":0.63},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":7.1079999999999997,"5min":7.1079999999999997,"15min":7.1079999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.63200000000000001,"15min":0.623},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.57799999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56499999999999995,"15min":0.55200000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.437}]},{"osd":7,"last update":"Tue Jul 18 03:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.42999999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.81499999999999995,"15min":0.79200000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.1449999999999996,"5min":7.1449999999999996,"15min":7.1449999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.77000000000000002,"15min":0.79000000000000004},"min":{"1min":0.47999999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96799999999999997,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.54700000000000004}]},{"osd":9,"last update":"Tue Jul 18 03:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.63700000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.65500000000000003}]},{"osd":11,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":7.1790000000000003,"5min":7.1790000000000003,"15min":7.1790000000000003},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.58699999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411384,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":478009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.39800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.55500000000000005},"min":{"1min":0.36099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82199999999999995,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.70699999999999996}]},{"osd":1,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78400000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.80300000000000005}]},{"osd":3,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59099999999999997,"15min":0.57699999999999996},"min":{"1min":0.45300000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.755},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.41199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.76300000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55200000000000005,"15min":0.55500000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71999999999999997,"5min":0.872,"15min":0.872},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76500000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.74099999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.41699999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.45100000000000001,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.38900000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55900000000000005,"15min":0.55200000000000005},"min":{"1min":0.40100000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58799999999999997,"15min":0.57499999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.81799999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.505,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.61699999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.378,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.50800000000000001}]},{"osd":9,"last update":"Tue Jul 18 03:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.72499999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.77600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":417892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.36399999999999999,"15min":0.34200000000000003},"min":{"1min":0.308,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.38600000000000001,"15min":0.36799999999999999},"min":{"1min":0.317,"5min":0.215,"15min":0.215},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.317}]},{"osd":1,"last update":"Tue Jul 18 03:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.56599999999999995,"15min":0.52700000000000002},"min":{"1min":0.42399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.75},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.53600000000000003,"15min":0.50600000000000001},"min":{"1min":0.42299999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78700000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.52300000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84699999999999998,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.502},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59499999999999997,"15min":0.56200000000000006},"min":{"1min":0.47599999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94299999999999995,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.66300000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.61199999999999999,"15min":0.57899999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.64900000000000002,"15min":0.60499999999999998},"min":{"1min":0.51500000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.84199999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.50600000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.357,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.436,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3460000000000001,"5min":1.827,"15min":1.827},"last":0.73999999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.58899999999999997,"15min":0.56699999999999995},"min":{"1min":0.39200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.41199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85599999999999998,"5min":0.88,"15min":0.88},"last":0.435}]},{"osd":7,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.871,"5min":0.79900000000000004,"15min":0.78700000000000003},"min":{"1min":0.55900000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.55900000000000005}]},{"osd":8,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.49399999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.48699999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.63800000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]}]},{"osd":4,"up_from":32,"seq":137438953520,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":417868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53200000000000003,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.54700000000000004,"15min":0.50700000000000001},"min":{"1min":0.48099999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89500000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.502}]},{"osd":1,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.39900000000000002,"15min":0.38600000000000001},"min":{"1min":0.32800000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.57299999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.40600000000000003,"15min":0.38100000000000001},"min":{"1min":0.32500000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65900000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.443}]},{"osd":2,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.434,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.745},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.47199999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85599999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.54100000000000004}]},{"osd":3,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.58499999999999996,"15min":0.54900000000000004},"min":{"1min":0.39800000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.57599999999999996,"15min":0.54800000000000004},"min":{"1min":0.41599999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.48599999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63,"15min":0.61399999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89200000000000002,"5min":1.038,"15min":1.038},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.64300000000000002,"15min":0.61599999999999999},"min":{"1min":0.58599999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.70199999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.50700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51000000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.434,"5min":0.38,"15min":0.38},"max":{"1min":0.89900000000000002,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.40699999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88600000000000001,"5min":1.121,"15min":1.121},"last":0.78300000000000003}]},{"osd":8,"last update":"Tue Jul 18 03:29:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71399999999999997,"15min":0.71499999999999997},"min":{"1min":0.53000000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91100000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.78200000000000003,"15min":0.77500000000000002},"min":{"1min":0.59999999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0920000000000001,"5min":1.208,"15min":1.208},"last":0.80400000000000005}]},{"osd":9,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.82499999999999996}]},{"osd":10,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.872}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]}]},{"osd":5,"up_from":37,"seq":158913789997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28428,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716724,"statfs":{"total":95995035648,"available":95965925376,"internally_reserved":0,"allocated":1511424,"data_stored":1010869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.78900000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.31900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.50800000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.54800000000000004,"15min":0.51700000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.55800000000000005,"15min":0.52900000000000003},"min":{"1min":0.41399999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.60799999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.40799999999999997,"15min":0.38400000000000001},"min":{"1min":0.29099999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.43099999999999999,"15min":0.41499999999999998},"min":{"1min":0.29899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71299999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.56999999999999995}]},{"osd":3,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.55900000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75800000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.59699999999999998,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.59899999999999998,"15min":0.56599999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.79500000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:30:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.623},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82599999999999996,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.72499999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70999999999999996,"15min":0.70799999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.871},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.81799999999999995}]},{"osd":8,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85199999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94099999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.83699999999999997}]},{"osd":9,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.85199999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.88900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:30:08.706 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph pg dump --format=json 2023-07-18T03:30:09.269 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:30:10.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:09 smithi134 ceph-mon[98216]: pgmap v269: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-07-18T03:30:10.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:09 smithi134 ceph-mon[98216]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:30:10.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:09 smithi040 ceph-mon[96439]: pgmap v269: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-07-18T03:30:10.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:09 smithi040 ceph-mon[96439]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:30:10.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:09 smithi064 ceph-mon[98059]: pgmap v269: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-07-18T03:30:10.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:09 smithi064 ceph-mon[98059]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:30:10.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.342+0000 7fdb22b32700 1 -- 172.21.15.40:0/3896113689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb1c0feab0 msgr2=0x7fdb1c0fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:10.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.342+0000 7fdb22b32700 1 --2- 172.21.15.40:0/3896113689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb1c0feab0 0x7fdb1c0fef20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009990 tx=0x7fdb08009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:10.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.342+0000 7fdb22b32700 1 -- 172.21.15.40:0/3896113689 shutdown_connections 2023-07-18T03:30:10.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.342+0000 7fdb22b32700 1 --2- 172.21.15.40:0/3896113689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb1c0ff460 0x7fdb1c10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:10.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.342+0000 7fdb22b32700 1 --2- 172.21.15.40:0/3896113689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb1c104ab0 0x7fdb1c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:10.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.342+0000 7fdb22b32700 1 --2- 172.21.15.40:0/3896113689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb1c0feab0 0x7fdb1c0fef20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:10.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.342+0000 7fdb22b32700 1 -- 172.21.15.40:0/3896113689 >> 172.21.15.40:0/3896113689 conn(0x7fdb1c075b70 msgr2=0x7fdb1c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:10.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.342+0000 7fdb22b32700 1 -- 172.21.15.40:0/3896113689 shutdown_connections 2023-07-18T03:30:10.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.342+0000 7fdb22b32700 1 -- 172.21.15.40:0/3896113689 wait complete. 2023-07-18T03:30:10.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.343+0000 7fdb22b32700 1 Processor -- start 2023-07-18T03:30:10.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.343+0000 7fdb22b32700 1 -- start start 2023-07-18T03:30:10.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.343+0000 7fdb22b32700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb1c0feab0 0x7fdb1c195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:10.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.343+0000 7fdb22b32700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb1c0ff460 0x7fdb1c1961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:10.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.343+0000 7fdb22b32700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb1c104ab0 0x7fdb1c1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:10.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.343+0000 7fdb22b32700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c199bb0 con 0x7fdb1c0ff460 2023-07-18T03:30:10.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.343+0000 7fdb22b32700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c199d20 con 0x7fdb1c104ab0 2023-07-18T03:30:10.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.343+0000 7fdb22b32700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c199ee0 con 0x7fdb1c0feab0 2023-07-18T03:30:10.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.343+0000 7fdb1bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb1c0ff460 0x7fdb1c1961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:10.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.343+0000 7fdb1bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb1c0ff460 0x7fdb1c1961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56754/0 (socket says 172.21.15.40:56754) 2023-07-18T03:30:10.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.343+0000 7fdb1bfff700 1 -- 172.21.15.40:0/2187230397 learned_addr learned my addr 172.21.15.40:0/2187230397 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:10.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.343+0000 7fdb210cf700 1 --2- 172.21.15.40:0/2187230397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb1c104ab0 0x7fdb1c1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:10.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.344+0000 7fdb208ce700 1 --2- 172.21.15.40:0/2187230397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb1c0feab0 0x7fdb1c195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:10.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.344+0000 7fdb1bfff700 1 -- 172.21.15.40:0/2187230397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb1c0feab0 msgr2=0x7fdb1c195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:10.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.344+0000 7fdb1bfff700 1 --2- 172.21.15.40:0/2187230397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb1c0feab0 0x7fdb1c195cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:10.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.344+0000 7fdb1bfff700 1 -- 172.21.15.40:0/2187230397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb1c104ab0 msgr2=0x7fdb1c1995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:10.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.344+0000 7fdb1bfff700 1 --2- 172.21.15.40:0/2187230397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb1c104ab0 0x7fdb1c1995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:10.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.344+0000 7fdb1bfff700 1 -- 172.21.15.40:0/2187230397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb080095f0 con 0x7fdb1c0ff460 2023-07-18T03:30:10.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.344+0000 7fdb1bfff700 1 --2- 172.21.15.40:0/2187230397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb1c0ff460 0x7fdb1c1961f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdb08000c00 tx=0x7fdb0800f6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:10.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.344+0000 7fdb19ffb700 1 -- 172.21.15.40:0/2187230397 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0801d070 con 0x7fdb1c0ff460 2023-07-18T03:30:10.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.344+0000 7fdb19ffb700 1 -- 172.21.15.40:0/2187230397 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb08022470 con 0x7fdb1c0ff460 2023-07-18T03:30:10.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.344+0000 7fdb19ffb700 1 -- 172.21.15.40:0/2187230397 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08017650 con 0x7fdb1c0ff460 2023-07-18T03:30:10.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.344+0000 7fdb22b32700 1 -- 172.21.15.40:0/2187230397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb1c19a160 con 0x7fdb1c0ff460 2023-07-18T03:30:10.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.344+0000 7fdb22b32700 1 -- 172.21.15.40:0/2187230397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1c19a430 con 0x7fdb1c0ff460 2023-07-18T03:30:10.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.345+0000 7fdb22b32700 1 -- 172.21.15.40:0/2187230397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb1c04e210 con 0x7fdb1c0ff460 2023-07-18T03:30:10.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.346+0000 7fdb19ffb700 1 -- 172.21.15.40:0/2187230397 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fdb08022870 con 0x7fdb1c0ff460 2023-07-18T03:30:10.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.347+0000 7fdb19ffb700 1 --2- 172.21.15.40:0/2187230397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdb0c0a1260 0x7fdb0c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:10.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.347+0000 7fdb208ce700 1 --2- 172.21.15.40:0/2187230397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdb0c0a1260 0x7fdb0c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:10.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.347+0000 7fdb19ffb700 1 -- 172.21.15.40:0/2187230397 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdb080cdde0 con 0x7fdb1c0ff460 2023-07-18T03:30:10.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.348+0000 7fdb208ce700 1 --2- 172.21.15.40:0/2187230397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdb0c0a1260 0x7fdb0c0a3710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0ffbf0 tx=0x7fdb10009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:10.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.352+0000 7fdb19ffb700 1 -- 172.21.15.40:0/2187230397 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdb08097510 con 0x7fdb1c0ff460 2023-07-18T03:30:10.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.544+0000 7fdb22b32700 1 -- 172.21.15.40:0/2187230397 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdb1c0fd9d0 con 0x7fdb0c0a1260 2023-07-18T03:30:10.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.556+0000 7fdb19ffb700 1 -- 172.21.15.40:0/2187230397 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340254 (secure 0 0 0) 0x7fdb1c0fd9d0 con 0x7fdb0c0a1260 2023-07-18T03:30:10.558 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:10.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.559+0000 7fdb22b32700 1 -- 172.21.15.40:0/2187230397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdb0c0a1260 msgr2=0x7fdb0c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:10.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.559+0000 7fdb22b32700 1 --2- 172.21.15.40:0/2187230397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdb0c0a1260 0x7fdb0c0a3710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0ffbf0 tx=0x7fdb10009190 comp rx=0 tx=0).stop 2023-07-18T03:30:10.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.559+0000 7fdb22b32700 1 -- 172.21.15.40:0/2187230397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb1c0ff460 msgr2=0x7fdb1c1961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:10.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.559+0000 7fdb22b32700 1 --2- 172.21.15.40:0/2187230397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb1c0ff460 0x7fdb1c1961f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdb08000c00 tx=0x7fdb0800f6e0 comp rx=0 tx=0).stop 2023-07-18T03:30:10.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.559+0000 7fdb22b32700 1 -- 172.21.15.40:0/2187230397 shutdown_connections 2023-07-18T03:30:10.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.559+0000 7fdb22b32700 1 --2- 172.21.15.40:0/2187230397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb1c104ab0 0x7fdb1c1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.559+0000 7fdb22b32700 1 --2- 172.21.15.40:0/2187230397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdb0c0a1260 0x7fdb0c0a3710 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.559+0000 7fdb22b32700 1 --2- 172.21.15.40:0/2187230397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb1c0feab0 0x7fdb1c195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.559+0000 7fdb22b32700 1 --2- 172.21.15.40:0/2187230397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb1c0ff460 0x7fdb1c1961f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.559+0000 7fdb22b32700 1 -- 172.21.15.40:0/2187230397 >> 172.21.15.40:0/2187230397 conn(0x7fdb1c075b70 msgr2=0x7fdb1c0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.559+0000 7fdb22b32700 1 -- 172.21.15.40:0/2187230397 shutdown_connections 2023-07-18T03:30:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:10.560+0000 7fdb22b32700 1 -- 172.21.15.40:0/2187230397 wait complete. 2023-07-18T03:30:10.561 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-18T03:30:10.926 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2023-07-18T03:30:09.824963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655761,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":496,"num_read_kb":363,"num_write":412,"num_write_kb":3066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335912,"kb_used_data":12488,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605912,"statfs":{"total":1151940427776,"available":1151596453888,"internally_reserved":0,"allocated":12787712,"data_stored":6973544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":106,"num_read_kb":52,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003233"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085961+0000","last_change":"2023-07-18T03:29:27.101623+0000","last_active":"2023-07-18T03:29:28.085961+0000","last_peered":"2023-07-18T03:29:28.085961+0000","last_clean":"2023-07-18T03:29:28.085961+0000","last_became_active":"2023-07-18T03:29:27.101297+0000","last_became_peered":"2023-07-18T03:29:27.101297+0000","last_unstale":"2023-07-18T03:29:28.085961+0000","last_undegraded":"2023-07-18T03:29:28.085961+0000","last_fullsized":"2023-07-18T03:29:28.085961+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:34:31.169755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248873+0000","last_change":"2023-07-18T03:29:27.104159+0000","last_active":"2023-07-18T03:29:28.248873+0000","last_peered":"2023-07-18T03:29:28.248873+0000","last_clean":"2023-07-18T03:29:28.248873+0000","last_became_active":"2023-07-18T03:29:27.103830+0000","last_became_peered":"2023-07-18T03:29:27.103830+0000","last_unstale":"2023-07-18T03:29:28.248873+0000","last_undegraded":"2023-07-18T03:29:28.248873+0000","last_fullsized":"2023-07-18T03:29:28.248873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:44:49.917899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249078+0000","last_change":"2023-07-18T03:29:27.098858+0000","last_active":"2023-07-18T03:29:28.249078+0000","last_peered":"2023-07-18T03:29:28.249078+0000","last_clean":"2023-07-18T03:29:28.249078+0000","last_became_active":"2023-07-18T03:29:27.098540+0000","last_became_peered":"2023-07-18T03:29:27.098540+0000","last_unstale":"2023-07-18T03:29:28.249078+0000","last_undegraded":"2023-07-18T03:29:28.249078+0000","last_fullsized":"2023-07-18T03:29:28.249078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:55:46.914785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248698+0000","last_change":"2023-07-18T03:29:27.103106+0000","last_active":"2023-07-18T03:29:28.248698+0000","last_peered":"2023-07-18T03:29:28.248698+0000","last_clean":"2023-07-18T03:29:28.248698+0000","last_became_active":"2023-07-18T03:29:27.102780+0000","last_became_peered":"2023-07-18T03:29:27.102780+0000","last_unstale":"2023-07-18T03:29:28.248698+0000","last_undegraded":"2023-07-18T03:29:28.248698+0000","last_fullsized":"2023-07-18T03:29:28.248698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:51:36.548110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248934+0000","last_change":"2023-07-18T03:29:27.095432+0000","last_active":"2023-07-18T03:29:28.248934+0000","last_peered":"2023-07-18T03:29:28.248934+0000","last_clean":"2023-07-18T03:29:28.248934+0000","last_became_active":"2023-07-18T03:29:27.095103+0000","last_became_peered":"2023-07-18T03:29:27.095103+0000","last_unstale":"2023-07-18T03:29:28.248934+0000","last_undegraded":"2023-07-18T03:29:28.248934+0000","last_fullsized":"2023-07-18T03:29:28.248934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:50:40.336206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085493+0000","last_change":"2023-07-18T03:29:27.091412+0000","last_active":"2023-07-18T03:29:28.085493+0000","last_peered":"2023-07-18T03:29:28.085493+0000","last_clean":"2023-07-18T03:29:28.085493+0000","last_became_active":"2023-07-18T03:29:27.091060+0000","last_became_peered":"2023-07-18T03:29:27.091060+0000","last_unstale":"2023-07-18T03:29:28.085493+0000","last_undegraded":"2023-07-18T03:29:28.085493+0000","last_fullsized":"2023-07-18T03:29:28.085493+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:31:43.393707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339267+0000","last_change":"2023-07-18T03:29:27.105223+0000","last_active":"2023-07-18T03:29:28.339267+0000","last_peered":"2023-07-18T03:29:28.339267+0000","last_clean":"2023-07-18T03:29:28.339267+0000","last_became_active":"2023-07-18T03:29:27.104926+0000","last_became_peered":"2023-07-18T03:29:27.104926+0000","last_unstale":"2023-07-18T03:29:28.339267+0000","last_undegraded":"2023-07-18T03:29:28.339267+0000","last_fullsized":"2023-07-18T03:29:28.339267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:43:23.908336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248819+0000","last_change":"2023-07-18T03:29:27.093722+0000","last_active":"2023-07-18T03:29:28.248819+0000","last_peered":"2023-07-18T03:29:28.248819+0000","last_clean":"2023-07-18T03:29:28.248819+0000","last_became_active":"2023-07-18T03:29:27.093103+0000","last_became_peered":"2023-07-18T03:29:27.093103+0000","last_unstale":"2023-07-18T03:29:28.248819+0000","last_undegraded":"2023-07-18T03:29:28.248819+0000","last_fullsized":"2023-07-18T03:29:28.248819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:03:07.122936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249009+0000","last_change":"2023-07-18T03:29:27.094681+0000","last_active":"2023-07-18T03:29:28.249009+0000","last_peered":"2023-07-18T03:29:28.249009+0000","last_clean":"2023-07-18T03:29:28.249009+0000","last_became_active":"2023-07-18T03:29:27.094284+0000","last_became_peered":"2023-07-18T03:29:27.094284+0000","last_unstale":"2023-07-18T03:29:28.249009+0000","last_undegraded":"2023-07-18T03:29:28.249009+0000","last_fullsized":"2023-07-18T03:29:28.249009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:25:08.820895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085737+0000","last_change":"2023-07-18T03:29:27.100759+0000","last_active":"2023-07-18T03:29:28.085737+0000","last_peered":"2023-07-18T03:29:28.085737+0000","last_clean":"2023-07-18T03:29:28.085737+0000","last_became_active":"2023-07-18T03:29:27.096660+0000","last_became_peered":"2023-07-18T03:29:27.096660+0000","last_unstale":"2023-07-18T03:29:28.085737+0000","last_undegraded":"2023-07-18T03:29:28.085737+0000","last_fullsized":"2023-07-18T03:29:28.085737+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:56:10.159967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339718+0000","last_change":"2023-07-18T03:29:27.107251+0000","last_active":"2023-07-18T03:29:28.339718+0000","last_peered":"2023-07-18T03:29:28.339718+0000","last_clean":"2023-07-18T03:29:28.339718+0000","last_became_active":"2023-07-18T03:29:27.106895+0000","last_became_peered":"2023-07-18T03:29:27.106895+0000","last_unstale":"2023-07-18T03:29:28.339718+0000","last_undegraded":"2023-07-18T03:29:28.339718+0000","last_fullsized":"2023-07-18T03:29:28.339718+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:12:15.805607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248978+0000","last_change":"2023-07-18T03:29:27.103905+0000","last_active":"2023-07-18T03:29:28.248978+0000","last_peered":"2023-07-18T03:29:28.248978+0000","last_clean":"2023-07-18T03:29:28.248978+0000","last_became_active":"2023-07-18T03:29:27.103021+0000","last_became_peered":"2023-07-18T03:29:27.103021+0000","last_unstale":"2023-07-18T03:29:28.248978+0000","last_undegraded":"2023-07-18T03:29:28.248978+0000","last_fullsized":"2023-07-18T03:29:28.248978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:48:20.771586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249855+0000","last_change":"2023-07-18T03:29:27.103529+0000","last_active":"2023-07-18T03:29:28.249855+0000","last_peered":"2023-07-18T03:29:28.249855+0000","last_clean":"2023-07-18T03:29:28.249855+0000","last_became_active":"2023-07-18T03:29:27.103217+0000","last_became_peered":"2023-07-18T03:29:27.103217+0000","last_unstale":"2023-07-18T03:29:28.249855+0000","last_undegraded":"2023-07-18T03:29:28.249855+0000","last_fullsized":"2023-07-18T03:29:28.249855+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:37:25.309228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248500+0000","last_change":"2023-07-18T03:29:27.096640+0000","last_active":"2023-07-18T03:29:28.248500+0000","last_peered":"2023-07-18T03:29:28.248500+0000","last_clean":"2023-07-18T03:29:28.248500+0000","last_became_active":"2023-07-18T03:29:27.096347+0000","last_became_peered":"2023-07-18T03:29:27.096347+0000","last_unstale":"2023-07-18T03:29:28.248500+0000","last_undegraded":"2023-07-18T03:29:28.248500+0000","last_fullsized":"2023-07-18T03:29:28.248500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:31:27.280331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248292+0000","last_change":"2023-07-18T03:29:27.095910+0000","last_active":"2023-07-18T03:29:28.248292+0000","last_peered":"2023-07-18T03:29:28.248292+0000","last_clean":"2023-07-18T03:29:28.248292+0000","last_became_active":"2023-07-18T03:29:27.095556+0000","last_became_peered":"2023-07-18T03:29:27.095556+0000","last_unstale":"2023-07-18T03:29:28.248292+0000","last_undegraded":"2023-07-18T03:29:28.248292+0000","last_fullsized":"2023-07-18T03:29:28.248292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:09:59.583920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248551+0000","last_change":"2023-07-18T03:29:27.108220+0000","last_active":"2023-07-18T03:29:28.248551+0000","last_peered":"2023-07-18T03:29:28.248551+0000","last_clean":"2023-07-18T03:29:28.248551+0000","last_became_active":"2023-07-18T03:29:27.107869+0000","last_became_peered":"2023-07-18T03:29:27.107869+0000","last_unstale":"2023-07-18T03:29:28.248551+0000","last_undegraded":"2023-07-18T03:29:28.248551+0000","last_fullsized":"2023-07-18T03:29:28.248551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:53:30.964666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.086002+0000","last_change":"2023-07-18T03:29:27.100634+0000","last_active":"2023-07-18T03:29:28.086002+0000","last_peered":"2023-07-18T03:29:28.086002+0000","last_clean":"2023-07-18T03:29:28.086002+0000","last_became_active":"2023-07-18T03:29:27.100316+0000","last_became_peered":"2023-07-18T03:29:27.100316+0000","last_unstale":"2023-07-18T03:29:28.086002+0000","last_undegraded":"2023-07-18T03:29:28.086002+0000","last_fullsized":"2023-07-18T03:29:28.086002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:35:30.023399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.235377+0000","last_change":"2023-07-18T03:29:27.100322+0000","last_active":"2023-07-18T03:29:28.235377+0000","last_peered":"2023-07-18T03:29:28.235377+0000","last_clean":"2023-07-18T03:29:28.235377+0000","last_became_active":"2023-07-18T03:29:27.099782+0000","last_became_peered":"2023-07-18T03:29:27.099782+0000","last_unstale":"2023-07-18T03:29:28.235377+0000","last_undegraded":"2023-07-18T03:29:28.235377+0000","last_fullsized":"2023-07-18T03:29:28.235377+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:28:12.377533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129942+0000","last_change":"2023-07-18T03:29:27.096639+0000","last_active":"2023-07-18T03:29:28.129942+0000","last_peered":"2023-07-18T03:29:28.129942+0000","last_clean":"2023-07-18T03:29:28.129942+0000","last_became_active":"2023-07-18T03:29:27.096348+0000","last_became_peered":"2023-07-18T03:29:27.096348+0000","last_unstale":"2023-07-18T03:29:28.129942+0000","last_undegraded":"2023-07-18T03:29:28.129942+0000","last_fullsized":"2023-07-18T03:29:28.129942+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:05:09.093995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":59,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:30:03.708335+0000","last_change":"2023-07-18T03:29:26.287722+0000","last_active":"2023-07-18T03:30:03.708335+0000","last_peered":"2023-07-18T03:30:03.708335+0000","last_clean":"2023-07-18T03:30:03.708335+0000","last_became_active":"2023-07-18T03:29:26.287327+0000","last_became_peered":"2023-07-18T03:29:26.287327+0000","last_unstale":"2023-07-18T03:30:03.708335+0000","last_undegraded":"2023-07-18T03:30:03.708335+0000","last_fullsized":"2023-07-18T03:30:03.708335+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:57:15.630654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249206+0000","last_change":"2023-07-18T03:29:27.092771+0000","last_active":"2023-07-18T03:29:28.249206+0000","last_peered":"2023-07-18T03:29:28.249206+0000","last_clean":"2023-07-18T03:29:28.249206+0000","last_became_active":"2023-07-18T03:29:27.092456+0000","last_became_peered":"2023-07-18T03:29:27.092456+0000","last_unstale":"2023-07-18T03:29:28.249206+0000","last_undegraded":"2023-07-18T03:29:28.249206+0000","last_fullsized":"2023-07-18T03:29:28.249206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:02:05.004006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:29.697155+0000","last_change":"2023-07-18T03:29:26.289683+0000","last_active":"2023-07-18T03:29:29.697155+0000","last_peered":"2023-07-18T03:29:29.697155+0000","last_clean":"2023-07-18T03:29:29.697155+0000","last_became_active":"2023-07-18T03:29:26.289356+0000","last_became_peered":"2023-07-18T03:29:26.289356+0000","last_unstale":"2023-07-18T03:29:29.697155+0000","last_undegraded":"2023-07-18T03:29:29.697155+0000","last_fullsized":"2023-07-18T03:29:29.697155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:02:04.478209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249301+0000","last_change":"2023-07-18T03:29:27.098709+0000","last_active":"2023-07-18T03:29:28.249301+0000","last_peered":"2023-07-18T03:29:28.249301+0000","last_clean":"2023-07-18T03:29:28.249301+0000","last_became_active":"2023-07-18T03:29:27.098395+0000","last_became_peered":"2023-07-18T03:29:27.098395+0000","last_unstale":"2023-07-18T03:29:28.249301+0000","last_undegraded":"2023-07-18T03:29:28.249301+0000","last_fullsized":"2023-07-18T03:29:28.249301+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:26:19.426023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:34.657360+0000","last_change":"2023-07-18T03:29:26.297119+0000","last_active":"2023-07-18T03:29:34.657360+0000","last_peered":"2023-07-18T03:29:34.657360+0000","last_clean":"2023-07-18T03:29:34.657360+0000","last_became_active":"2023-07-18T03:29:26.296649+0000","last_became_peered":"2023-07-18T03:29:26.296649+0000","last_unstale":"2023-07-18T03:29:34.657360+0000","last_undegraded":"2023-07-18T03:29:34.657360+0000","last_fullsized":"2023-07-18T03:29:34.657360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:51:17.267059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248457+0000","last_change":"2023-07-18T03:29:27.105511+0000","last_active":"2023-07-18T03:29:28.248457+0000","last_peered":"2023-07-18T03:29:28.248457+0000","last_clean":"2023-07-18T03:29:28.248457+0000","last_became_active":"2023-07-18T03:29:27.104946+0000","last_became_peered":"2023-07-18T03:29:27.104946+0000","last_unstale":"2023-07-18T03:29:28.248457+0000","last_undegraded":"2023-07-18T03:29:28.248457+0000","last_fullsized":"2023-07-18T03:29:28.248457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:42:13.652404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.642412+0000","last_change":"2023-07-18T03:29:26.290708+0000","last_active":"2023-07-18T03:29:33.642412+0000","last_peered":"2023-07-18T03:29:33.642412+0000","last_clean":"2023-07-18T03:29:33.642412+0000","last_became_active":"2023-07-18T03:29:26.290383+0000","last_became_peered":"2023-07-18T03:29:26.290383+0000","last_unstale":"2023-07-18T03:29:33.642412+0000","last_undegraded":"2023-07-18T03:29:33.642412+0000","last_fullsized":"2023-07-18T03:29:33.642412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:34:22.740683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.084957+0000","last_change":"2023-07-18T03:29:27.096630+0000","last_active":"2023-07-18T03:29:28.084957+0000","last_peered":"2023-07-18T03:29:28.084957+0000","last_clean":"2023-07-18T03:29:28.084957+0000","last_became_active":"2023-07-18T03:29:27.096337+0000","last_became_peered":"2023-07-18T03:29:27.096337+0000","last_unstale":"2023-07-18T03:29:28.084957+0000","last_undegraded":"2023-07-18T03:29:28.084957+0000","last_fullsized":"2023-07-18T03:29:28.084957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:23:46.241177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":58,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:30:04.654580+0000","last_change":"2023-07-18T03:29:26.291622+0000","last_active":"2023-07-18T03:30:04.654580+0000","last_peered":"2023-07-18T03:30:04.654580+0000","last_clean":"2023-07-18T03:30:04.654580+0000","last_became_active":"2023-07-18T03:29:26.291317+0000","last_became_peered":"2023-07-18T03:29:26.291317+0000","last_unstale":"2023-07-18T03:30:04.654580+0000","last_undegraded":"2023-07-18T03:30:04.654580+0000","last_fullsized":"2023-07-18T03:30:04.654580+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:37:53.996938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248914+0000","last_change":"2023-07-18T03:29:27.093107+0000","last_active":"2023-07-18T03:29:28.248914+0000","last_peered":"2023-07-18T03:29:28.248914+0000","last_clean":"2023-07-18T03:29:28.248914+0000","last_became_active":"2023-07-18T03:29:27.092785+0000","last_became_peered":"2023-07-18T03:29:27.092785+0000","last_unstale":"2023-07-18T03:29:28.248914+0000","last_undegraded":"2023-07-18T03:29:28.248914+0000","last_fullsized":"2023-07-18T03:29:28.248914+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:18:41.599570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:32.636567+0000","last_change":"2023-07-18T03:29:26.290493+0000","last_active":"2023-07-18T03:29:32.636567+0000","last_peered":"2023-07-18T03:29:32.636567+0000","last_clean":"2023-07-18T03:29:32.636567+0000","last_became_active":"2023-07-18T03:29:26.290187+0000","last_became_peered":"2023-07-18T03:29:26.290187+0000","last_unstale":"2023-07-18T03:29:32.636567+0000","last_undegraded":"2023-07-18T03:29:32.636567+0000","last_fullsized":"2023-07-18T03:29:32.636567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:14:58.224602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.084989+0000","last_change":"2023-07-18T03:29:27.095129+0000","last_active":"2023-07-18T03:29:28.084989+0000","last_peered":"2023-07-18T03:29:28.084989+0000","last_clean":"2023-07-18T03:29:28.084989+0000","last_became_active":"2023-07-18T03:29:27.094821+0000","last_became_peered":"2023-07-18T03:29:27.094821+0000","last_unstale":"2023-07-18T03:29:28.084989+0000","last_undegraded":"2023-07-18T03:29:28.084989+0000","last_fullsized":"2023-07-18T03:29:28.084989+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:33:21.671061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.612409+0000","last_change":"2023-07-18T03:29:26.285860+0000","last_active":"2023-07-18T03:29:31.612409+0000","last_peered":"2023-07-18T03:29:31.612409+0000","last_clean":"2023-07-18T03:29:31.612409+0000","last_became_active":"2023-07-18T03:29:26.285491+0000","last_became_peered":"2023-07-18T03:29:26.285491+0000","last_unstale":"2023-07-18T03:29:31.612409+0000","last_undegraded":"2023-07-18T03:29:31.612409+0000","last_fullsized":"2023-07-18T03:29:31.612409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:14:20.414177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248682+0000","last_change":"2023-07-18T03:29:27.093861+0000","last_active":"2023-07-18T03:29:28.248682+0000","last_peered":"2023-07-18T03:29:28.248682+0000","last_clean":"2023-07-18T03:29:28.248682+0000","last_became_active":"2023-07-18T03:29:27.093445+0000","last_became_peered":"2023-07-18T03:29:27.093445+0000","last_unstale":"2023-07-18T03:29:28.248682+0000","last_undegraded":"2023-07-18T03:29:28.248682+0000","last_fullsized":"2023-07-18T03:29:28.248682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:05:28.911877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.642632+0000","last_change":"2023-07-18T03:29:26.288108+0000","last_active":"2023-07-18T03:29:33.642632+0000","last_peered":"2023-07-18T03:29:33.642632+0000","last_clean":"2023-07-18T03:29:33.642632+0000","last_became_active":"2023-07-18T03:29:26.287531+0000","last_became_peered":"2023-07-18T03:29:26.287531+0000","last_unstale":"2023-07-18T03:29:33.642632+0000","last_undegraded":"2023-07-18T03:29:33.642632+0000","last_fullsized":"2023-07-18T03:29:33.642632+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:35:36.416356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.235430+0000","last_change":"2023-07-18T03:29:27.092908+0000","last_active":"2023-07-18T03:29:28.235430+0000","last_peered":"2023-07-18T03:29:28.235430+0000","last_clean":"2023-07-18T03:29:28.235430+0000","last_became_active":"2023-07-18T03:29:27.092566+0000","last_became_peered":"2023-07-18T03:29:27.092566+0000","last_unstale":"2023-07-18T03:29:28.235430+0000","last_undegraded":"2023-07-18T03:29:28.235430+0000","last_fullsized":"2023-07-18T03:29:28.235430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:22:47.560376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":83,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:30:04.653808+0000","last_change":"2023-07-18T03:29:26.294251+0000","last_active":"2023-07-18T03:30:04.653808+0000","last_peered":"2023-07-18T03:30:04.653808+0000","last_clean":"2023-07-18T03:30:04.653808+0000","last_became_active":"2023-07-18T03:29:26.293847+0000","last_became_peered":"2023-07-18T03:29:26.293847+0000","last_unstale":"2023-07-18T03:30:04.653808+0000","last_undegraded":"2023-07-18T03:30:04.653808+0000","last_fullsized":"2023-07-18T03:30:04.653808+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:51:26.530234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248737+0000","last_change":"2023-07-18T03:29:27.106551+0000","last_active":"2023-07-18T03:29:28.248737+0000","last_peered":"2023-07-18T03:29:28.248737+0000","last_clean":"2023-07-18T03:29:28.248737+0000","last_became_active":"2023-07-18T03:29:27.105187+0000","last_became_peered":"2023-07-18T03:29:27.105187+0000","last_unstale":"2023-07-18T03:29:28.248737+0000","last_undegraded":"2023-07-18T03:29:28.248737+0000","last_fullsized":"2023-07-18T03:29:28.248737+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:45:05.223020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.640865+0000","last_change":"2023-07-18T03:29:26.293284+0000","last_active":"2023-07-18T03:29:33.640865+0000","last_peered":"2023-07-18T03:29:33.640865+0000","last_clean":"2023-07-18T03:29:33.640865+0000","last_became_active":"2023-07-18T03:29:26.292955+0000","last_became_peered":"2023-07-18T03:29:26.292955+0000","last_unstale":"2023-07-18T03:29:33.640865+0000","last_undegraded":"2023-07-18T03:29:33.640865+0000","last_fullsized":"2023-07-18T03:29:33.640865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:15:36.890771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249337+0000","last_change":"2023-07-18T03:29:27.097322+0000","last_active":"2023-07-18T03:29:28.249337+0000","last_peered":"2023-07-18T03:29:28.249337+0000","last_clean":"2023-07-18T03:29:28.249337+0000","last_became_active":"2023-07-18T03:29:27.097051+0000","last_became_peered":"2023-07-18T03:29:27.097051+0000","last_unstale":"2023-07-18T03:29:28.249337+0000","last_undegraded":"2023-07-18T03:29:28.249337+0000","last_fullsized":"2023-07-18T03:29:28.249337+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:36:33.129283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.601626+0000","last_change":"2023-07-18T03:29:26.290810+0000","last_active":"2023-07-18T03:29:31.601626+0000","last_peered":"2023-07-18T03:29:31.601626+0000","last_clean":"2023-07-18T03:29:31.601626+0000","last_became_active":"2023-07-18T03:29:26.290448+0000","last_became_peered":"2023-07-18T03:29:26.290448+0000","last_unstale":"2023-07-18T03:29:31.601626+0000","last_undegraded":"2023-07-18T03:29:31.601626+0000","last_fullsized":"2023-07-18T03:29:31.601626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:21:19.581142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249116+0000","last_change":"2023-07-18T03:29:27.105801+0000","last_active":"2023-07-18T03:29:28.249116+0000","last_peered":"2023-07-18T03:29:28.249116+0000","last_clean":"2023-07-18T03:29:28.249116+0000","last_became_active":"2023-07-18T03:29:27.104715+0000","last_became_peered":"2023-07-18T03:29:27.104715+0000","last_unstale":"2023-07-18T03:29:28.249116+0000","last_undegraded":"2023-07-18T03:29:28.249116+0000","last_fullsized":"2023-07-18T03:29:28.249116+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:49:34.118157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'29","reported_seq":119,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:30:04.653807+0000","last_change":"2023-07-18T03:29:26.293306+0000","last_active":"2023-07-18T03:30:04.653807+0000","last_peered":"2023-07-18T03:30:04.653807+0000","last_clean":"2023-07-18T03:30:04.653807+0000","last_became_active":"2023-07-18T03:29:26.292946+0000","last_became_peered":"2023-07-18T03:29:26.292946+0000","last_unstale":"2023-07-18T03:30:04.653807+0000","last_undegraded":"2023-07-18T03:30:04.653807+0000","last_fullsized":"2023-07-18T03:30:04.653807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:36:19.196296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58527,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":94,"num_write":31,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249042+0000","last_change":"2023-07-18T03:29:27.103941+0000","last_active":"2023-07-18T03:29:28.249042+0000","last_peered":"2023-07-18T03:29:28.249042+0000","last_clean":"2023-07-18T03:29:28.249042+0000","last_became_active":"2023-07-18T03:29:27.103506+0000","last_became_peered":"2023-07-18T03:29:27.103506+0000","last_unstale":"2023-07-18T03:29:28.249042+0000","last_undegraded":"2023-07-18T03:29:28.249042+0000","last_fullsized":"2023-07-18T03:29:28.249042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:13:27.299459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.642234+0000","last_change":"2023-07-18T03:29:26.293473+0000","last_active":"2023-07-18T03:29:33.642234+0000","last_peered":"2023-07-18T03:29:33.642234+0000","last_clean":"2023-07-18T03:29:33.642234+0000","last_became_active":"2023-07-18T03:29:26.293132+0000","last_became_peered":"2023-07-18T03:29:26.293132+0000","last_unstale":"2023-07-18T03:29:33.642234+0000","last_undegraded":"2023-07-18T03:29:33.642234+0000","last_fullsized":"2023-07-18T03:29:33.642234+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:32:07.174316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249663+0000","last_change":"2023-07-18T03:29:27.104032+0000","last_active":"2023-07-18T03:29:28.249663+0000","last_peered":"2023-07-18T03:29:28.249663+0000","last_clean":"2023-07-18T03:29:28.249663+0000","last_became_active":"2023-07-18T03:29:27.103736+0000","last_became_peered":"2023-07-18T03:29:27.103736+0000","last_unstale":"2023-07-18T03:29:28.249663+0000","last_undegraded":"2023-07-18T03:29:28.249663+0000","last_fullsized":"2023-07-18T03:29:28.249663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:29:13.489385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.642720+0000","last_change":"2023-07-18T03:29:26.290487+0000","last_active":"2023-07-18T03:29:33.642720+0000","last_peered":"2023-07-18T03:29:33.642720+0000","last_clean":"2023-07-18T03:29:33.642720+0000","last_became_active":"2023-07-18T03:29:26.290115+0000","last_became_peered":"2023-07-18T03:29:26.290115+0000","last_unstale":"2023-07-18T03:29:33.642720+0000","last_undegraded":"2023-07-18T03:29:33.642720+0000","last_fullsized":"2023-07-18T03:29:33.642720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:35:42.866787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339569+0000","last_change":"2023-07-18T03:29:27.108550+0000","last_active":"2023-07-18T03:29:28.339569+0000","last_peered":"2023-07-18T03:29:28.339569+0000","last_clean":"2023-07-18T03:29:28.339569+0000","last_became_active":"2023-07-18T03:29:27.108069+0000","last_became_peered":"2023-07-18T03:29:27.108069+0000","last_unstale":"2023-07-18T03:29:28.339569+0000","last_undegraded":"2023-07-18T03:29:28.339569+0000","last_fullsized":"2023-07-18T03:29:28.339569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:55:48.510403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:30:04.708509+0000","last_change":"2023-07-18T03:29:26.290896+0000","last_active":"2023-07-18T03:30:04.708509+0000","last_peered":"2023-07-18T03:30:04.708509+0000","last_clean":"2023-07-18T03:30:04.708509+0000","last_became_active":"2023-07-18T03:29:26.290450+0000","last_became_peered":"2023-07-18T03:29:26.290450+0000","last_unstale":"2023-07-18T03:30:04.708509+0000","last_undegraded":"2023-07-18T03:30:04.708509+0000","last_fullsized":"2023-07-18T03:30:04.708509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:23:16.823095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339200+0000","last_change":"2023-07-18T03:29:27.109184+0000","last_active":"2023-07-18T03:29:28.339200+0000","last_peered":"2023-07-18T03:29:28.339200+0000","last_clean":"2023-07-18T03:29:28.339200+0000","last_became_active":"2023-07-18T03:29:27.107993+0000","last_became_peered":"2023-07-18T03:29:27.107993+0000","last_unstale":"2023-07-18T03:29:28.339200+0000","last_undegraded":"2023-07-18T03:29:28.339200+0000","last_fullsized":"2023-07-18T03:29:28.339200+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:38:53.665857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.641695+0000","last_change":"2023-07-18T03:29:26.293671+0000","last_active":"2023-07-18T03:29:33.641695+0000","last_peered":"2023-07-18T03:29:33.641695+0000","last_clean":"2023-07-18T03:29:33.641695+0000","last_became_active":"2023-07-18T03:29:26.293205+0000","last_became_peered":"2023-07-18T03:29:26.293205+0000","last_unstale":"2023-07-18T03:29:33.641695+0000","last_undegraded":"2023-07-18T03:29:33.641695+0000","last_fullsized":"2023-07-18T03:29:33.641695+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:52:09.605399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.084887+0000","last_change":"2023-07-18T03:29:27.096711+0000","last_active":"2023-07-18T03:29:28.084887+0000","last_peered":"2023-07-18T03:29:28.084887+0000","last_clean":"2023-07-18T03:29:28.084887+0000","last_became_active":"2023-07-18T03:29:27.096423+0000","last_became_peered":"2023-07-18T03:29:27.096423+0000","last_unstale":"2023-07-18T03:29:28.084887+0000","last_undegraded":"2023-07-18T03:29:28.084887+0000","last_fullsized":"2023-07-18T03:29:28.084887+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:35:54.060369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.598395+0000","last_change":"2023-07-18T03:29:26.291546+0000","last_active":"2023-07-18T03:29:31.598395+0000","last_peered":"2023-07-18T03:29:31.598395+0000","last_clean":"2023-07-18T03:29:31.598395+0000","last_became_active":"2023-07-18T03:29:26.291319+0000","last_became_peered":"2023-07-18T03:29:26.291319+0000","last_unstale":"2023-07-18T03:29:31.598395+0000","last_undegraded":"2023-07-18T03:29:31.598395+0000","last_fullsized":"2023-07-18T03:29:31.598395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:23:25.182800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085688+0000","last_change":"2023-07-18T03:29:27.098479+0000","last_active":"2023-07-18T03:29:28.085688+0000","last_peered":"2023-07-18T03:29:28.085688+0000","last_clean":"2023-07-18T03:29:28.085688+0000","last_became_active":"2023-07-18T03:29:27.098153+0000","last_became_peered":"2023-07-18T03:29:27.098153+0000","last_unstale":"2023-07-18T03:29:28.085688+0000","last_undegraded":"2023-07-18T03:29:28.085688+0000","last_fullsized":"2023-07-18T03:29:28.085688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:53:01.674305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:29.873882+0000","last_change":"2023-07-18T03:29:26.290953+0000","last_active":"2023-07-18T03:29:29.873882+0000","last_peered":"2023-07-18T03:29:29.873882+0000","last_clean":"2023-07-18T03:29:29.873882+0000","last_became_active":"2023-07-18T03:29:26.290633+0000","last_became_peered":"2023-07-18T03:29:26.290633+0000","last_unstale":"2023-07-18T03:29:29.873882+0000","last_undegraded":"2023-07-18T03:29:29.873882+0000","last_fullsized":"2023-07-18T03:29:29.873882+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:40:49.185595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.611541+0000","last_change":"2023-07-18T03:29:27.093651+0000","last_active":"2023-07-18T03:29:30.611541+0000","last_peered":"2023-07-18T03:29:30.611541+0000","last_clean":"2023-07-18T03:29:30.611541+0000","last_became_active":"2023-07-18T03:29:27.093307+0000","last_became_peered":"2023-07-18T03:29:27.093307+0000","last_unstale":"2023-07-18T03:29:30.611541+0000","last_undegraded":"2023-07-18T03:29:30.611541+0000","last_fullsized":"2023-07-18T03:29:30.611541+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:31:10.315148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:30:05.710526+0000","last_change":"2023-07-18T03:29:26.287405+0000","last_active":"2023-07-18T03:30:05.710526+0000","last_peered":"2023-07-18T03:30:05.710526+0000","last_clean":"2023-07-18T03:30:05.710526+0000","last_became_active":"2023-07-18T03:29:26.287048+0000","last_became_peered":"2023-07-18T03:29:26.287048+0000","last_unstale":"2023-07-18T03:30:05.710526+0000","last_undegraded":"2023-07-18T03:30:05.710526+0000","last_fullsized":"2023-07-18T03:30:05.710526+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:50:53.303258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085803+0000","last_change":"2023-07-18T03:29:27.103218+0000","last_active":"2023-07-18T03:29:28.085803+0000","last_peered":"2023-07-18T03:29:28.085803+0000","last_clean":"2023-07-18T03:29:28.085803+0000","last_became_active":"2023-07-18T03:29:27.102917+0000","last_became_peered":"2023-07-18T03:29:27.102917+0000","last_unstale":"2023-07-18T03:29:28.085803+0000","last_undegraded":"2023-07-18T03:29:28.085803+0000","last_fullsized":"2023-07-18T03:29:28.085803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:23:49.743367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085272+0000","last_change":"2023-07-18T03:29:27.098696+0000","last_active":"2023-07-18T03:29:28.085272+0000","last_peered":"2023-07-18T03:29:28.085272+0000","last_clean":"2023-07-18T03:29:28.085272+0000","last_became_active":"2023-07-18T03:29:27.098253+0000","last_became_peered":"2023-07-18T03:29:27.098253+0000","last_unstale":"2023-07-18T03:29:28.085272+0000","last_undegraded":"2023-07-18T03:29:28.085272+0000","last_fullsized":"2023-07-18T03:29:28.085272+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:40:58.942045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:30:05.709200+0000","last_change":"2023-07-18T03:29:26.290308+0000","last_active":"2023-07-18T03:30:05.709200+0000","last_peered":"2023-07-18T03:30:05.709200+0000","last_clean":"2023-07-18T03:30:05.709200+0000","last_became_active":"2023-07-18T03:29:26.289937+0000","last_became_peered":"2023-07-18T03:29:26.289937+0000","last_unstale":"2023-07-18T03:30:05.709200+0000","last_undegraded":"2023-07-18T03:30:05.709200+0000","last_fullsized":"2023-07-18T03:30:05.709200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:21:51.452197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249450+0000","last_change":"2023-07-18T03:29:27.096469+0000","last_active":"2023-07-18T03:29:28.249450+0000","last_peered":"2023-07-18T03:29:28.249450+0000","last_clean":"2023-07-18T03:29:28.249450+0000","last_became_active":"2023-07-18T03:29:27.096171+0000","last_became_peered":"2023-07-18T03:29:27.096171+0000","last_unstale":"2023-07-18T03:29:28.249450+0000","last_undegraded":"2023-07-18T03:29:28.249450+0000","last_fullsized":"2023-07-18T03:29:28.249450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:14:47.599303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:34.656948+0000","last_change":"2023-07-18T03:29:26.290935+0000","last_active":"2023-07-18T03:29:34.656948+0000","last_peered":"2023-07-18T03:29:34.656948+0000","last_clean":"2023-07-18T03:29:34.656948+0000","last_became_active":"2023-07-18T03:29:26.290595+0000","last_became_peered":"2023-07-18T03:29:26.290595+0000","last_unstale":"2023-07-18T03:29:34.656948+0000","last_undegraded":"2023-07-18T03:29:34.656948+0000","last_fullsized":"2023-07-18T03:29:34.656948+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:34:23.417106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":331,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339337+0000","last_change":"2023-07-18T03:28:28.122385+0000","last_active":"2023-07-18T03:29:28.339337+0000","last_peered":"2023-07-18T03:29:28.339337+0000","last_clean":"2023-07-18T03:29:28.339337+0000","last_became_active":"2023-07-18T03:28:28.121932+0000","last_became_peered":"2023-07-18T03:28:28.121932+0000","last_unstale":"2023-07-18T03:29:28.339337+0000","last_undegraded":"2023-07-18T03:29:28.339337+0000","last_fullsized":"2023-07-18T03:29:28.339337+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:25:38.267627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:25:38.267627+0000","last_clean_scrub_stamp":"2023-07-18T03:25:38.267627+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:46:37.243383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248934+0000","last_change":"2023-07-18T03:29:27.101582+0000","last_active":"2023-07-18T03:29:28.248934+0000","last_peered":"2023-07-18T03:29:28.248934+0000","last_clean":"2023-07-18T03:29:28.248934+0000","last_became_active":"2023-07-18T03:29:27.101226+0000","last_became_peered":"2023-07-18T03:29:27.101226+0000","last_unstale":"2023-07-18T03:29:28.248934+0000","last_undegraded":"2023-07-18T03:29:28.248934+0000","last_fullsized":"2023-07-18T03:29:28.248934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:42:14.571198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:31.608398+0000","last_change":"2023-07-18T03:29:26.291787+0000","last_active":"2023-07-18T03:29:31.608398+0000","last_peered":"2023-07-18T03:29:31.608398+0000","last_clean":"2023-07-18T03:29:31.608398+0000","last_became_active":"2023-07-18T03:29:26.291471+0000","last_became_peered":"2023-07-18T03:29:26.291471+0000","last_unstale":"2023-07-18T03:29:31.608398+0000","last_undegraded":"2023-07-18T03:29:31.608398+0000","last_fullsized":"2023-07-18T03:29:31.608398+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:49:20.527148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:32.635875+0000","last_change":"2023-07-18T03:29:26.296936+0000","last_active":"2023-07-18T03:29:32.635875+0000","last_peered":"2023-07-18T03:29:32.635875+0000","last_clean":"2023-07-18T03:29:32.635875+0000","last_became_active":"2023-07-18T03:29:26.296241+0000","last_became_peered":"2023-07-18T03:29:26.296241+0000","last_unstale":"2023-07-18T03:29:32.635875+0000","last_undegraded":"2023-07-18T03:29:32.635875+0000","last_fullsized":"2023-07-18T03:29:32.635875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:23:24.313635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249296+0000","last_change":"2023-07-18T03:29:27.102715+0000","last_active":"2023-07-18T03:29:28.249296+0000","last_peered":"2023-07-18T03:29:28.249296+0000","last_clean":"2023-07-18T03:29:28.249296+0000","last_became_active":"2023-07-18T03:29:27.100914+0000","last_became_peered":"2023-07-18T03:29:27.100914+0000","last_unstale":"2023-07-18T03:29:28.249296+0000","last_undegraded":"2023-07-18T03:29:28.249296+0000","last_fullsized":"2023-07-18T03:29:28.249296+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:43:38.965763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'9","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:35.629147+0000","last_change":"2023-07-18T03:29:26.290231+0000","last_active":"2023-07-18T03:29:35.629147+0000","last_peered":"2023-07-18T03:29:35.629147+0000","last_clean":"2023-07-18T03:29:35.629147+0000","last_became_active":"2023-07-18T03:29:26.289753+0000","last_became_peered":"2023-07-18T03:29:26.289753+0000","last_unstale":"2023-07-18T03:29:35.629147+0000","last_undegraded":"2023-07-18T03:29:35.629147+0000","last_fullsized":"2023-07-18T03:29:35.629147+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:33:36.792644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249097+0000","last_change":"2023-07-18T03:29:27.103964+0000","last_active":"2023-07-18T03:29:28.249097+0000","last_peered":"2023-07-18T03:29:28.249097+0000","last_clean":"2023-07-18T03:29:28.249097+0000","last_became_active":"2023-07-18T03:29:27.103660+0000","last_became_peered":"2023-07-18T03:29:27.103660+0000","last_unstale":"2023-07-18T03:29:28.249097+0000","last_undegraded":"2023-07-18T03:29:28.249097+0000","last_fullsized":"2023-07-18T03:29:28.249097+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:08:09.167239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.641324+0000","last_change":"2023-07-18T03:29:26.286436+0000","last_active":"2023-07-18T03:29:33.641324+0000","last_peered":"2023-07-18T03:29:33.641324+0000","last_clean":"2023-07-18T03:29:33.641324+0000","last_became_active":"2023-07-18T03:29:26.286070+0000","last_became_peered":"2023-07-18T03:29:26.286070+0000","last_unstale":"2023-07-18T03:29:33.641324+0000","last_undegraded":"2023-07-18T03:29:33.641324+0000","last_fullsized":"2023-07-18T03:29:33.641324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:14:53.466702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249329+0000","last_change":"2023-07-18T03:29:27.102303+0000","last_active":"2023-07-18T03:29:28.249329+0000","last_peered":"2023-07-18T03:29:28.249329+0000","last_clean":"2023-07-18T03:29:28.249329+0000","last_became_active":"2023-07-18T03:29:27.101980+0000","last_became_peered":"2023-07-18T03:29:27.101980+0000","last_unstale":"2023-07-18T03:29:28.249329+0000","last_undegraded":"2023-07-18T03:29:28.249329+0000","last_fullsized":"2023-07-18T03:29:28.249329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:46:48.299459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.624991+0000","last_change":"2023-07-18T03:29:26.289628+0000","last_active":"2023-07-18T03:29:30.624991+0000","last_peered":"2023-07-18T03:29:30.624991+0000","last_clean":"2023-07-18T03:29:30.624991+0000","last_became_active":"2023-07-18T03:29:26.289013+0000","last_became_peered":"2023-07-18T03:29:26.289013+0000","last_unstale":"2023-07-18T03:29:30.624991+0000","last_undegraded":"2023-07-18T03:29:30.624991+0000","last_fullsized":"2023-07-18T03:29:30.624991+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:58:20.665607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248709+0000","last_change":"2023-07-18T03:29:27.090943+0000","last_active":"2023-07-18T03:29:28.248709+0000","last_peered":"2023-07-18T03:29:28.248709+0000","last_clean":"2023-07-18T03:29:28.248709+0000","last_became_active":"2023-07-18T03:29:27.090465+0000","last_became_peered":"2023-07-18T03:29:27.090465+0000","last_unstale":"2023-07-18T03:29:28.248709+0000","last_undegraded":"2023-07-18T03:29:28.248709+0000","last_fullsized":"2023-07-18T03:29:28.248709+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:49:17.548432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.642141+0000","last_change":"2023-07-18T03:29:26.288576+0000","last_active":"2023-07-18T03:29:33.642141+0000","last_peered":"2023-07-18T03:29:33.642141+0000","last_clean":"2023-07-18T03:29:33.642141+0000","last_became_active":"2023-07-18T03:29:26.287514+0000","last_became_peered":"2023-07-18T03:29:26.287514+0000","last_unstale":"2023-07-18T03:29:33.642141+0000","last_undegraded":"2023-07-18T03:29:33.642141+0000","last_fullsized":"2023-07-18T03:29:33.642141+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:37:47.805842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085745+0000","last_change":"2023-07-18T03:29:27.101111+0000","last_active":"2023-07-18T03:29:28.085745+0000","last_peered":"2023-07-18T03:29:28.085745+0000","last_clean":"2023-07-18T03:29:28.085745+0000","last_became_active":"2023-07-18T03:29:27.100737+0000","last_became_peered":"2023-07-18T03:29:27.100737+0000","last_unstale":"2023-07-18T03:29:28.085745+0000","last_undegraded":"2023-07-18T03:29:28.085745+0000","last_fullsized":"2023-07-18T03:29:28.085745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:00:58.906730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:34.044421+0000","last_change":"2023-07-18T03:29:26.296974+0000","last_active":"2023-07-18T03:29:34.044421+0000","last_peered":"2023-07-18T03:29:34.044421+0000","last_clean":"2023-07-18T03:29:34.044421+0000","last_became_active":"2023-07-18T03:29:26.296451+0000","last_became_peered":"2023-07-18T03:29:26.296451+0000","last_unstale":"2023-07-18T03:29:34.044421+0000","last_undegraded":"2023-07-18T03:29:34.044421+0000","last_fullsized":"2023-07-18T03:29:34.044421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:35:42.821600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249813+0000","last_change":"2023-07-18T03:29:27.095857+0000","last_active":"2023-07-18T03:29:28.249813+0000","last_peered":"2023-07-18T03:29:28.249813+0000","last_clean":"2023-07-18T03:29:28.249813+0000","last_became_active":"2023-07-18T03:29:27.095553+0000","last_became_peered":"2023-07-18T03:29:27.095553+0000","last_unstale":"2023-07-18T03:29:28.249813+0000","last_undegraded":"2023-07-18T03:29:28.249813+0000","last_fullsized":"2023-07-18T03:29:28.249813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:30:11.142073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:30.627680+0000","last_change":"2023-07-18T03:29:26.295936+0000","last_active":"2023-07-18T03:29:30.627680+0000","last_peered":"2023-07-18T03:29:30.627680+0000","last_clean":"2023-07-18T03:29:30.627680+0000","last_became_active":"2023-07-18T03:29:26.295467+0000","last_became_peered":"2023-07-18T03:29:26.295467+0000","last_unstale":"2023-07-18T03:29:30.627680+0000","last_undegraded":"2023-07-18T03:29:30.627680+0000","last_fullsized":"2023-07-18T03:29:30.627680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:45:43.716125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339616+0000","last_change":"2023-07-18T03:29:27.108528+0000","last_active":"2023-07-18T03:29:28.339616+0000","last_peered":"2023-07-18T03:29:28.339616+0000","last_clean":"2023-07-18T03:29:28.339616+0000","last_became_active":"2023-07-18T03:29:27.108039+0000","last_became_peered":"2023-07-18T03:29:27.108039+0000","last_unstale":"2023-07-18T03:29:28.339616+0000","last_undegraded":"2023-07-18T03:29:28.339616+0000","last_fullsized":"2023-07-18T03:29:28.339616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:57:20.088291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'5","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:30:06.713918+0000","last_change":"2023-07-18T03:29:26.294006+0000","last_active":"2023-07-18T03:30:06.713918+0000","last_peered":"2023-07-18T03:30:06.713918+0000","last_clean":"2023-07-18T03:30:06.713918+0000","last_became_active":"2023-07-18T03:29:26.293700+0000","last_became_peered":"2023-07-18T03:29:26.293700+0000","last_unstale":"2023-07-18T03:30:06.713918+0000","last_undegraded":"2023-07-18T03:30:06.713918+0000","last_fullsized":"2023-07-18T03:30:06.713918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:44:37.451806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339715+0000","last_change":"2023-07-18T03:29:27.102764+0000","last_active":"2023-07-18T03:29:28.339715+0000","last_peered":"2023-07-18T03:29:28.339715+0000","last_clean":"2023-07-18T03:29:28.339715+0000","last_became_active":"2023-07-18T03:29:27.102200+0000","last_became_peered":"2023-07-18T03:29:27.102200+0000","last_unstale":"2023-07-18T03:29:28.339715+0000","last_undegraded":"2023-07-18T03:29:28.339715+0000","last_fullsized":"2023-07-18T03:29:28.339715+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:54:13.608760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:32.635822+0000","last_change":"2023-07-18T03:29:26.294271+0000","last_active":"2023-07-18T03:29:32.635822+0000","last_peered":"2023-07-18T03:29:32.635822+0000","last_clean":"2023-07-18T03:29:32.635822+0000","last_became_active":"2023-07-18T03:29:26.293938+0000","last_became_peered":"2023-07-18T03:29:26.293938+0000","last_unstale":"2023-07-18T03:29:32.635822+0000","last_undegraded":"2023-07-18T03:29:32.635822+0000","last_fullsized":"2023-07-18T03:29:32.635822+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:10:38.556865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339451+0000","last_change":"2023-07-18T03:29:27.106510+0000","last_active":"2023-07-18T03:29:28.339451+0000","last_peered":"2023-07-18T03:29:28.339451+0000","last_clean":"2023-07-18T03:29:28.339451+0000","last_became_active":"2023-07-18T03:29:27.106222+0000","last_became_peered":"2023-07-18T03:29:27.106222+0000","last_unstale":"2023-07-18T03:29:28.339451+0000","last_undegraded":"2023-07-18T03:29:28.339451+0000","last_fullsized":"2023-07-18T03:29:28.339451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:27:44.790790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:33.640406+0000","last_change":"2023-07-18T03:29:26.293583+0000","last_active":"2023-07-18T03:29:33.640406+0000","last_peered":"2023-07-18T03:29:33.640406+0000","last_clean":"2023-07-18T03:29:33.640406+0000","last_became_active":"2023-07-18T03:29:26.293076+0000","last_became_peered":"2023-07-18T03:29:26.293076+0000","last_unstale":"2023-07-18T03:29:33.640406+0000","last_undegraded":"2023-07-18T03:29:33.640406+0000","last_fullsized":"2023-07-18T03:29:33.640406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:37:21.591425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.339559+0000","last_change":"2023-07-18T03:29:27.109175+0000","last_active":"2023-07-18T03:29:28.339559+0000","last_peered":"2023-07-18T03:29:28.339559+0000","last_clean":"2023-07-18T03:29:28.339559+0000","last_became_active":"2023-07-18T03:29:27.108669+0000","last_became_peered":"2023-07-18T03:29:27.108669+0000","last_unstale":"2023-07-18T03:29:28.339559+0000","last_undegraded":"2023-07-18T03:29:28.339559+0000","last_fullsized":"2023-07-18T03:29:28.339559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:00:09.903589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085918+0000","last_change":"2023-07-18T03:29:27.099770+0000","last_active":"2023-07-18T03:29:28.085918+0000","last_peered":"2023-07-18T03:29:28.085918+0000","last_clean":"2023-07-18T03:29:28.085918+0000","last_became_active":"2023-07-18T03:29:27.099487+0000","last_became_peered":"2023-07-18T03:29:27.099487+0000","last_unstale":"2023-07-18T03:29:28.085918+0000","last_undegraded":"2023-07-18T03:29:28.085918+0000","last_fullsized":"2023-07-18T03:29:28.085918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:22:59.316119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129653+0000","last_change":"2023-07-18T03:29:27.100189+0000","last_active":"2023-07-18T03:29:28.129653+0000","last_peered":"2023-07-18T03:29:28.129653+0000","last_clean":"2023-07-18T03:29:28.129653+0000","last_became_active":"2023-07-18T03:29:27.099884+0000","last_became_peered":"2023-07-18T03:29:27.099884+0000","last_unstale":"2023-07-18T03:29:28.129653+0000","last_undegraded":"2023-07-18T03:29:28.129653+0000","last_fullsized":"2023-07-18T03:29:28.129653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:04:36.761236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129593+0000","last_change":"2023-07-18T03:29:27.099196+0000","last_active":"2023-07-18T03:29:28.129593+0000","last_peered":"2023-07-18T03:29:28.129593+0000","last_clean":"2023-07-18T03:29:28.129593+0000","last_became_active":"2023-07-18T03:29:27.098903+0000","last_became_peered":"2023-07-18T03:29:27.098903+0000","last_unstale":"2023-07-18T03:29:28.129593+0000","last_undegraded":"2023-07-18T03:29:28.129593+0000","last_fullsized":"2023-07-18T03:29:28.129593+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:46:05.998503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248469+0000","last_change":"2023-07-18T03:29:27.098138+0000","last_active":"2023-07-18T03:29:28.248469+0000","last_peered":"2023-07-18T03:29:28.248469+0000","last_clean":"2023-07-18T03:29:28.248469+0000","last_became_active":"2023-07-18T03:29:27.097793+0000","last_became_peered":"2023-07-18T03:29:27.097793+0000","last_unstale":"2023-07-18T03:29:28.248469+0000","last_undegraded":"2023-07-18T03:29:28.248469+0000","last_fullsized":"2023-07-18T03:29:28.248469+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:07:31.466874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249184+0000","last_change":"2023-07-18T03:29:27.101610+0000","last_active":"2023-07-18T03:29:28.249184+0000","last_peered":"2023-07-18T03:29:28.249184+0000","last_clean":"2023-07-18T03:29:28.249184+0000","last_became_active":"2023-07-18T03:29:27.101332+0000","last_became_peered":"2023-07-18T03:29:27.101332+0000","last_unstale":"2023-07-18T03:29:28.249184+0000","last_undegraded":"2023-07-18T03:29:28.249184+0000","last_fullsized":"2023-07-18T03:29:28.249184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:43:01.583313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085879+0000","last_change":"2023-07-18T03:29:27.093345+0000","last_active":"2023-07-18T03:29:28.085879+0000","last_peered":"2023-07-18T03:29:28.085879+0000","last_clean":"2023-07-18T03:29:28.085879+0000","last_became_active":"2023-07-18T03:29:27.093048+0000","last_became_peered":"2023-07-18T03:29:27.093048+0000","last_unstale":"2023-07-18T03:29:28.085879+0000","last_undegraded":"2023-07-18T03:29:28.085879+0000","last_fullsized":"2023-07-18T03:29:28.085879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:00:19.659948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.129830+0000","last_change":"2023-07-18T03:29:27.098311+0000","last_active":"2023-07-18T03:29:28.129830+0000","last_peered":"2023-07-18T03:29:28.129830+0000","last_clean":"2023-07-18T03:29:28.129830+0000","last_became_active":"2023-07-18T03:29:27.097980+0000","last_became_peered":"2023-07-18T03:29:27.097980+0000","last_unstale":"2023-07-18T03:29:28.129830+0000","last_undegraded":"2023-07-18T03:29:28.129830+0000","last_fullsized":"2023-07-18T03:29:28.129830+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:34:48.849264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248965+0000","last_change":"2023-07-18T03:29:27.093796+0000","last_active":"2023-07-18T03:29:28.248965+0000","last_peered":"2023-07-18T03:29:28.248965+0000","last_clean":"2023-07-18T03:29:28.248965+0000","last_became_active":"2023-07-18T03:29:27.093474+0000","last_became_peered":"2023-07-18T03:29:27.093474+0000","last_unstale":"2023-07-18T03:29:28.248965+0000","last_undegraded":"2023-07-18T03:29:28.248965+0000","last_fullsized":"2023-07-18T03:29:28.248965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:00:09.316029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085099+0000","last_change":"2023-07-18T03:29:27.098541+0000","last_active":"2023-07-18T03:29:28.085099+0000","last_peered":"2023-07-18T03:29:28.085099+0000","last_clean":"2023-07-18T03:29:28.085099+0000","last_became_active":"2023-07-18T03:29:27.097824+0000","last_became_peered":"2023-07-18T03:29:27.097824+0000","last_unstale":"2023-07-18T03:29:28.085099+0000","last_undegraded":"2023-07-18T03:29:28.085099+0000","last_fullsized":"2023-07-18T03:29:28.085099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:28:38.837863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.248860+0000","last_change":"2023-07-18T03:29:27.096983+0000","last_active":"2023-07-18T03:29:28.248860+0000","last_peered":"2023-07-18T03:29:28.248860+0000","last_clean":"2023-07-18T03:29:28.248860+0000","last_became_active":"2023-07-18T03:29:27.096632+0000","last_became_peered":"2023-07-18T03:29:27.096632+0000","last_unstale":"2023-07-18T03:29:28.248860+0000","last_undegraded":"2023-07-18T03:29:28.248860+0000","last_fullsized":"2023-07-18T03:29:28.248860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:45:45.965235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249073+0000","last_change":"2023-07-18T03:29:27.105236+0000","last_active":"2023-07-18T03:29:28.249073+0000","last_peered":"2023-07-18T03:29:28.249073+0000","last_clean":"2023-07-18T03:29:28.249073+0000","last_became_active":"2023-07-18T03:29:27.104906+0000","last_became_peered":"2023-07-18T03:29:27.104906+0000","last_unstale":"2023-07-18T03:29:28.249073+0000","last_undegraded":"2023-07-18T03:29:28.249073+0000","last_fullsized":"2023-07-18T03:29:28.249073+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:53:05.499609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.249970+0000","last_change":"2023-07-18T03:29:27.103461+0000","last_active":"2023-07-18T03:29:28.249970+0000","last_peered":"2023-07-18T03:29:28.249970+0000","last_clean":"2023-07-18T03:29:28.249970+0000","last_became_active":"2023-07-18T03:29:27.103162+0000","last_became_peered":"2023-07-18T03:29:27.103162+0000","last_unstale":"2023-07-18T03:29:28.249970+0000","last_undegraded":"2023-07-18T03:29:28.249970+0000","last_fullsized":"2023-07-18T03:29:28.249970+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:26:14.600362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-18T03:29:28.085635+0000","last_change":"2023-07-18T03:29:27.095300+0000","last_active":"2023-07-18T03:29:28.085635+0000","last_peered":"2023-07-18T03:29:28.085635+0000","last_clean":"2023-07-18T03:29:28.085635+0000","last_became_active":"2023-07-18T03:29:27.094985+0000","last_became_peered":"2023-07-18T03:29:27.094985+0000","last_unstale":"2023-07-18T03:29:28.085635+0000","last_undegraded":"2023-07-18T03:29:28.085635+0000","last_fullsized":"2023-07-18T03:29:28.085635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:09:05.373008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65147,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":370,"num_read_kb":254,"num_write":229,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":195441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514516,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":418360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":0.42599999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.72699999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.495},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":4,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60099999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.79600000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.77500000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":9,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.627}]},{"osd":10,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.86499999999999999}]}]},{"osd":10,"up_from":69,"seq":296352743448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28496,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716656,"statfs":{"total":95995035648,"available":95965855744,"internally_reserved":0,"allocated":1581056,"data_stored":1067922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.628},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.751}]},{"osd":2,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1,"5min":1,"15min":1},"last":0.78400000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.57799999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.64500000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.88600000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.81899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":8,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.87},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.82799999999999996}]},{"osd":9,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.80500000000000005}]},{"osd":11,"last update":"Tue Jul 18 03:29:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.84999999999999998}]}]},{"osd":9,"up_from":64,"seq":274877906972,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":418338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.34599999999999997}]},{"osd":1,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.501}]},{"osd":2,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.44500000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.40999999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53200000000000003}]},{"osd":5,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.70799999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.51800000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.63200000000000001}]},{"osd":8,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.754},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.73399999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.71599999999999997}]}]},{"osd":8,"up_from":56,"seq":240518168608,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":418385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.39500000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.747,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57499999999999996}]},{"osd":1,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.49399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.41199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86599999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65700000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.439,"15min":0.433},"min":{"1min":0.35099999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.45700000000000002,"15min":0.44700000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.45700000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57899999999999996,"15min":0.57299999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.63400000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.749},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.63600000000000001,"15min":0.62},"min":{"1min":0.42999999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.76800000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.373,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54500000000000004,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.68100000000000005}]},{"osd":6,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.52000000000000002,"5min":0.437,"15min":0.437},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69599999999999995,"15min":0.69399999999999995},"min":{"1min":0.49399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89200000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.78300000000000003}]},{"osd":7,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.5,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.47099999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.81399999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:30:05 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.752,"15min":0.72999999999999998},"min":{"1min":0.56499999999999995,"5min":0.505,"15min":0.505},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.78300000000000003,"15min":0.77300000000000002},"min":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.89400000000000002}]},{"osd":10,"last update":"Tue Jul 18 03:29:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.85199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":7,"up_from":49,"seq":210453397540,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28416,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716736,"statfs":{"total":95995035648,"available":95965937664,"internally_reserved":0,"allocated":1499136,"data_stored":1011095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46200000000000002,"15min":0.45900000000000002},"min":{"1min":0.30599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.753,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.48399999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.61299999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.379,"15min":0.379},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.621,"15min":0.60899999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.81399999999999995}]},{"osd":3,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.64900000000000002,"15min":0.63900000000000001},"min":{"1min":0.40600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93400000000000005,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66400000000000003,"15min":0.65800000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.79100000000000004}]},{"osd":4,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.63800000000000001}]},{"osd":5,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66800000000000004,"15min":0.66000000000000003},"min":{"1min":0.48299999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.70899999999999996,"15min":0.69599999999999995},"min":{"1min":0.46600000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":1.0609999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.74399999999999999,"15min":0.73399999999999999},"min":{"1min":0.499,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.04,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.52900000000000003,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97599999999999998,"5min":1.103,"15min":1.103},"last":0.92200000000000004}]},{"osd":8,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.80400000000000005,"15min":0.77600000000000002},"min":{"1min":0.51300000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":4.5880000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.76900000000000002,"15min":0.73299999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.6050000000000004,"5min":4.6050000000000004,"15min":4.6050000000000004},"last":1.0860000000000001}]},{"osd":9,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":1.024},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":1.0069999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.94399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]}]},{"osd":6,"up_from":44,"seq":188978561065,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":418013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.40699999999999997,"15min":0.40200000000000002},"min":{"1min":0.309,"5min":0.254,"15min":0.254},"max":{"1min":0.60299999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.32100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.58099999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.39600000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.45600000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.42099999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78800000000000003,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.48899999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.626,"15min":0.61799999999999999},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83599999999999997,"5min":1.008,"15min":1.008},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.63500000000000001,"15min":0.61899999999999999},"min":{"1min":0.48499999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.52400000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.498,"15min":0.48899999999999999},"min":{"1min":0.36399999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.373},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.36699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71099999999999997,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.42599999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.623,"15min":0.61399999999999999},"min":{"1min":0.52000000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85299999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.68500000000000005,"15min":0.67100000000000004},"min":{"1min":0.55100000000000005,"5min":0.438,"15min":0.438},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.60099999999999998}]},{"osd":5,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.68100000000000005,"15min":0.67100000000000004},"min":{"1min":0.54400000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.503,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.96399999999999997,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.61699999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73199999999999998,"15min":0.72799999999999998},"min":{"1min":0.47199999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56899999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0580000000000001,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.56499999999999995}]},{"osd":8,"last update":"Tue Jul 18 03:29:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.86399999999999999,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":5.1760000000000002,"5min":5.1760000000000002,"15min":5.1760000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.77400000000000002,"15min":0.78000000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.59299999999999997}]},{"osd":9,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.63900000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.71999999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":477447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.51500000000000001,"15min":0.47899999999999998},"min":{"1min":0.34999999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81100000000000005,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.52600000000000002,"15min":0.503},"min":{"1min":0.38300000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.89100000000000001,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58699999999999997,"15min":0.54800000000000004},"min":{"1min":0.50600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88400000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56699999999999995,"15min":0.52500000000000002},"min":{"1min":0.51200000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.68799999999999994}]},{"osd":3,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.59099999999999997,"15min":0.55500000000000005},"min":{"1min":0.441,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.60799999999999998,"15min":0.57799999999999996},"min":{"1min":0.54700000000000004,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.81999999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.44700000000000001,"15min":0.40699999999999997},"min":{"1min":0.33200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.45000000000000001,"15min":0.42199999999999999},"min":{"1min":0.30499999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.752,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.63700000000000001}]},{"osd":5,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.63600000000000001,"15min":0.58899999999999997},"min":{"1min":0.53000000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.66400000000000003,"15min":0.622},"min":{"1min":0.56799999999999995,"5min":0.433,"15min":0.433},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.89400000000000002}]},{"osd":6,"last update":"Tue Jul 18 03:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.63200000000000001,"15min":0.61699999999999999},"min":{"1min":0.498,"5min":0.443,"15min":0.443},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.68500000000000005,"15min":0.66900000000000004},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.69599999999999995,"15min":0.67700000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.97699999999999998}]},{"osd":8,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.81299999999999994,"15min":0.80600000000000005},"min":{"1min":0.51900000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.01},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.78900000000000003,"15min":0.77900000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.96399999999999997}]},{"osd":9,"last update":"Tue Jul 18 03:29:04 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.88400000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:29:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.038}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":419346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53600000000000003,"15min":0.51400000000000001},"min":{"1min":0.35999999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.374,"5min":0.318,"15min":0.318},"max":{"1min":0.82299999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.38900000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57899999999999996,"15min":0.53100000000000003},"min":{"1min":0.46200000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.879,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56299999999999994,"15min":0.52700000000000002},"min":{"1min":0.39300000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80400000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.60199999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.42299999999999999,"15min":0.41499999999999998},"min":{"1min":0.315,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.628,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.32700000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56899999999999995,"15min":0.55100000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.92800000000000005,"5min":1.018,"15min":1.018},"last":0.40799999999999997}]},{"osd":5,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.67500000000000004,"15min":0.63},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":7.1079999999999997,"5min":7.1079999999999997,"15min":7.1079999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.63200000000000001,"15min":0.623},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.57799999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56499999999999995,"15min":0.55200000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.437}]},{"osd":7,"last update":"Tue Jul 18 03:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.42999999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.81499999999999995,"15min":0.79200000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.1449999999999996,"5min":7.1449999999999996,"15min":7.1449999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.77000000000000002,"15min":0.79000000000000004},"min":{"1min":0.47999999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96799999999999997,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.54700000000000004}]},{"osd":9,"last update":"Tue Jul 18 03:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.63700000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.65500000000000003}]},{"osd":11,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":7.1790000000000003,"5min":7.1790000000000003,"15min":7.1790000000000003},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.58699999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":478009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.39800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.55500000000000005},"min":{"1min":0.36099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82199999999999995,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.432}]},{"osd":1,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78400000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59699999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61699999999999999,"15min":0.58599999999999997},"min":{"1min":0.48299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88600000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.61499999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55200000000000005,"15min":0.55500000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71999999999999997,"5min":0.872,"15min":0.872},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76500000000000001,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.65100000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.41699999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.45100000000000001,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.63}]},{"osd":6,"last update":"Tue Jul 18 03:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.59299999999999997,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.61599999999999999,"15min":0.58399999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89900000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.71499999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.505,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.61699999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.378,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.68500000000000005}]},{"osd":9,"last update":"Tue Jul 18 03:29:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.75},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.73299999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.78800000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":417892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.36399999999999999,"15min":0.34200000000000003},"min":{"1min":0.308,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.38600000000000001,"15min":0.36799999999999999},"min":{"1min":0.317,"5min":0.215,"15min":0.215},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.55300000000000005}]},{"osd":1,"last update":"Tue Jul 18 03:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.56599999999999995,"15min":0.52700000000000002},"min":{"1min":0.42399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.53600000000000003,"15min":0.50600000000000001},"min":{"1min":0.42299999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78700000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.72299999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.41199999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84699999999999998,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59499999999999997,"15min":0.56200000000000006},"min":{"1min":0.47599999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94299999999999995,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.63600000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.61199999999999999,"15min":0.57899999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.64900000000000002,"15min":0.60499999999999998},"min":{"1min":0.51500000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.93300000000000005}]},{"osd":5,"last update":"Tue Jul 18 03:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.50600000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.357,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.436,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3460000000000001,"5min":1.827,"15min":1.827},"last":0.68400000000000005}]},{"osd":6,"last update":"Tue Jul 18 03:29:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.58899999999999997,"15min":0.56699999999999995},"min":{"1min":0.39200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.749},{"interface":"front","average":{"1min":0.628,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.41199999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85599999999999998,"5min":0.88,"15min":0.88},"last":0.65000000000000002}]},{"osd":7,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.871,"5min":0.79900000000000004,"15min":0.78700000000000003},"min":{"1min":0.55900000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.95999999999999996}]},{"osd":8,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.49399999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.82999999999999996,"15min":0.85199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92500000000000004,"5min":1.113,"15min":1.113},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.74399999999999999,"15min":0.76200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.995,"5min":0.997,"15min":0.997},"last":0.76400000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":0.91800000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]}]},{"osd":4,"up_from":32,"seq":137438953520,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":417868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53200000000000003,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.54700000000000004,"15min":0.50700000000000001},"min":{"1min":0.48099999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89500000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.502}]},{"osd":1,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.39900000000000002,"15min":0.38600000000000001},"min":{"1min":0.32800000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.57299999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.40600000000000003,"15min":0.38100000000000001},"min":{"1min":0.32500000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65900000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.443}]},{"osd":2,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.60199999999999998,"15min":0.57199999999999995},"min":{"1min":0.434,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.745},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.47199999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.85599999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.54100000000000004}]},{"osd":3,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.58499999999999996,"15min":0.54900000000000004},"min":{"1min":0.39800000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.57599999999999996,"15min":0.54800000000000004},"min":{"1min":0.41599999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.48599999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63,"15min":0.61399999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89200000000000002,"5min":1.038,"15min":1.038},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.64300000000000002,"15min":0.61599999999999999},"min":{"1min":0.58599999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.70199999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.50700000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.40100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51000000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.434,"5min":0.38,"15min":0.38},"max":{"1min":0.89900000000000002,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.40699999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88600000000000001,"5min":1.121,"15min":1.121},"last":0.78300000000000003}]},{"osd":8,"last update":"Tue Jul 18 03:29:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71399999999999997,"15min":0.71499999999999997},"min":{"1min":0.53000000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91100000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.78200000000000003,"15min":0.77500000000000002},"min":{"1min":0.59999999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0920000000000001,"5min":1.208,"15min":1.208},"last":0.80400000000000005}]},{"osd":9,"last update":"Tue Jul 18 03:29:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.82499999999999996}]},{"osd":10,"last update":"Tue Jul 18 03:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.872}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]}]},{"osd":5,"up_from":37,"seq":158913789997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28428,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716724,"statfs":{"total":95995035648,"available":95965925376,"internally_reserved":0,"allocated":1511424,"data_stored":1010869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.78900000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.31900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.50800000000000001}]},{"osd":1,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.54800000000000004,"15min":0.51700000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.55800000000000005,"15min":0.52900000000000003},"min":{"1min":0.41399999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.60799999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.40799999999999997,"15min":0.38400000000000001},"min":{"1min":0.29099999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.43099999999999999,"15min":0.41499999999999998},"min":{"1min":0.29899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71299999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.56999999999999995}]},{"osd":3,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.55900000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75800000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.59699999999999998,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.59899999999999998,"15min":0.56599999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.79500000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:30:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.623},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82599999999999996,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.72499999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70999999999999996,"15min":0.70799999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.871},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71799999999999997,"15min":0.71399999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.81799999999999995}]},{"osd":8,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85199999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94099999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.83699999999999997}]},{"osd":9,"last update":"Tue Jul 18 03:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.85199999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.88900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:30:10.934 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-07-18T03:30:10.934 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-18T03:30:10.934 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-07-18T03:30:10.935 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph health --format=json 2023-07-18T03:30:11.494 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:30:11.990 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:11 smithi040 ceph-mon[96439]: pgmap v270: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2023-07-18T03:30:11.990 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:11 smithi040 ceph-mon[96439]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:30:12.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:11 smithi134 ceph-mon[98216]: pgmap v270: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2023-07-18T03:30:12.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:11 smithi134 ceph-mon[98216]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:30:12.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:11 smithi064 ceph-mon[98059]: pgmap v270: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2023-07-18T03:30:12.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:11 smithi064 ceph-mon[98059]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:30:12.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.770+0000 7f4360830700 1 -- 172.21.15.40:0/1326772940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f435c0ff450 msgr2=0x7f435c0ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:12.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.770+0000 7f4360830700 1 --2- 172.21.15.40:0/1326772940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f435c0ff450 0x7f435c0ff820 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f434c0099c0 tx=0x7f434c009cd0 comp rx=0 tx=0).stop 2023-07-18T03:30:12.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.770+0000 7f4360830700 1 -- 172.21.15.40:0/1326772940 shutdown_connections 2023-07-18T03:30:12.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.770+0000 7f4360830700 1 --2- 172.21.15.40:0/1326772940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f435c103e20 0x7f435c10b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:12.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.770+0000 7f4360830700 1 --2- 172.21.15.40:0/1326772940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f435c0ff450 0x7f435c0ff820 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:12.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.770+0000 7f4360830700 1 --2- 172.21.15.40:0/1326772940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f435c0ffdf0 0x7f435c1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:12.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.770+0000 7f4360830700 1 -- 172.21.15.40:0/1326772940 >> 172.21.15.40:0/1326772940 conn(0x7f435c075150 msgr2=0x7f435c075550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:12.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.770+0000 7f4360830700 1 -- 172.21.15.40:0/1326772940 shutdown_connections 2023-07-18T03:30:12.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.770+0000 7f4360830700 1 -- 172.21.15.40:0/1326772940 wait complete. 2023-07-18T03:30:12.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.771+0000 7f4360830700 1 Processor -- start 2023-07-18T03:30:12.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f4360830700 1 -- start start 2023-07-18T03:30:12.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f4360830700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f435c0ff450 0x7f435c196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:12.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f4360830700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f435c0ffdf0 0x7f435c197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:12.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f4360830700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f435c103e20 0x7f435c1a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:12.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f4360830700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c197a20 con 0x7f435c103e20 2023-07-18T03:30:12.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f4360830700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1a2190 con 0x7f435c0ffdf0 2023-07-18T03:30:12.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f4360830700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1a2300 con 0x7f435c0ff450 2023-07-18T03:30:12.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f435ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f435c103e20 0x7f435c1a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:12.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f435ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f435c103e20 0x7f435c1a1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56768/0 (socket says 172.21.15.40:56768) 2023-07-18T03:30:12.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f435ad9d700 1 -- 172.21.15.40:0/65122187 learned_addr learned my addr 172.21.15.40:0/65122187 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:12.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f435a59c700 1 --2- 172.21.15.40:0/65122187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f435c0ff450 0x7f435c196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:12.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f4351d9b700 1 --2- 172.21.15.40:0/65122187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f435c0ffdf0 0x7f435c197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:12.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f435ad9d700 1 -- 172.21.15.40:0/65122187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f435c0ff450 msgr2=0x7f435c196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:12.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f435ad9d700 1 --2- 172.21.15.40:0/65122187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f435c0ff450 0x7f435c196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:12.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f435ad9d700 1 -- 172.21.15.40:0/65122187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f435c0ffdf0 msgr2=0x7f435c197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:12.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f435ad9d700 1 --2- 172.21.15.40:0/65122187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f435c0ffdf0 0x7f435c197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:12.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f435ad9d700 1 -- 172.21.15.40:0/65122187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c0095f0 con 0x7f435c103e20 2023-07-18T03:30:12.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.772+0000 7f4351d9b700 1 --2- 172.21.15.40:0/65122187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f435c0ffdf0 0x7f435c197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:12.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.773+0000 7f435ad9d700 1 --2- 172.21.15.40:0/65122187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f435c103e20 0x7f435c1a1c50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f435400b970 tx=0x7f435400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:12.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.773+0000 7f4353fff700 1 -- 172.21.15.40:0/65122187 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435400c9b0 con 0x7f435c103e20 2023-07-18T03:30:12.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.773+0000 7f4353fff700 1 -- 172.21.15.40:0/65122187 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4354014440 con 0x7f435c103e20 2023-07-18T03:30:12.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.773+0000 7f4360830700 1 -- 172.21.15.40:0/65122187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435c1a25e0 con 0x7f435c103e20 2023-07-18T03:30:12.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.773+0000 7f4353fff700 1 -- 172.21.15.40:0/65122187 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43540075d0 con 0x7f435c103e20 2023-07-18T03:30:12.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.773+0000 7f4360830700 1 -- 172.21.15.40:0/65122187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435c1a2a90 con 0x7f435c103e20 2023-07-18T03:30:12.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.774+0000 7f4360830700 1 -- 172.21.15.40:0/65122187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435c04e210 con 0x7f435c103e20 2023-07-18T03:30:12.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.775+0000 7f4353fff700 1 -- 172.21.15.40:0/65122187 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f43540145b0 con 0x7f435c103e20 2023-07-18T03:30:12.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.775+0000 7f4353fff700 1 --2- 172.21.15.40:0/65122187 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f43480a11e0 0x7f43480a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:12.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.776+0000 7f435a59c700 1 --2- 172.21.15.40:0/65122187 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f43480a11e0 0x7f43480a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:12.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.776+0000 7f4353fff700 1 -- 172.21.15.40:0/65122187 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4354006e40 con 0x7f435c103e20 2023-07-18T03:30:12.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.776+0000 7f435a59c700 1 --2- 172.21.15.40:0/65122187 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f43480a11e0 0x7f43480a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f434c000c00 tx=0x7f434c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:12.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:12.779+0000 7f4353fff700 1 -- 172.21.15.40:0/65122187 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4354095ca0 con 0x7f435c103e20 2023-07-18T03:30:13.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.019+0000 7f4360830700 1 -- 172.21.15.40:0/65122187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f435c0fe6d0 con 0x7f435c103e20 2023-07-18T03:30:13.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.020+0000 7f4353fff700 1 -- 172.21.15.40:0/65122187 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f43540992c0 con 0x7f435c103e20 2023-07-18T03:30:13.017 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:13.018 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-18T03:30:13.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.022+0000 7f4360830700 1 -- 172.21.15.40:0/65122187 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f43480a11e0 msgr2=0x7f43480a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:13.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.022+0000 7f4360830700 1 --2- 172.21.15.40:0/65122187 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f43480a11e0 0x7f43480a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f434c000c00 tx=0x7f434c005fb0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.022+0000 7f4360830700 1 -- 172.21.15.40:0/65122187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f435c103e20 msgr2=0x7f435c1a1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:13.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.022+0000 7f4360830700 1 --2- 172.21.15.40:0/65122187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f435c103e20 0x7f435c1a1c50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f435400b970 tx=0x7f435400bd30 comp rx=0 tx=0).stop 2023-07-18T03:30:13.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.022+0000 7f4360830700 1 -- 172.21.15.40:0/65122187 shutdown_connections 2023-07-18T03:30:13.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.022+0000 7f4360830700 1 --2- 172.21.15.40:0/65122187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f435c103e20 0x7f435c1a1c50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.022+0000 7f4360830700 1 --2- 172.21.15.40:0/65122187 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f43480a11e0 0x7f43480a3690 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.022+0000 7f4360830700 1 --2- 172.21.15.40:0/65122187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f435c0ff450 0x7f435c196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.022+0000 7f4360830700 1 --2- 172.21.15.40:0/65122187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f435c0ffdf0 0x7f435c197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.022+0000 7f4360830700 1 -- 172.21.15.40:0/65122187 >> 172.21.15.40:0/65122187 conn(0x7f435c075150 msgr2=0x7f435c0fbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:13.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.022+0000 7f4360830700 1 -- 172.21.15.40:0/65122187 shutdown_connections 2023-07-18T03:30:13.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.022+0000 7f4360830700 1 -- 172.21.15.40:0/65122187 wait complete. 2023-07-18T03:30:13.442 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-07-18T03:30:13.443 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:30:13.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.575+0000 7fc106302700 1 -- 172.21.15.40:0/44029171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1000febf0 msgr2=0x7fc1000ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:13.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.575+0000 7fc106302700 1 --2- 172.21.15.40:0/44029171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1000febf0 0x7fc1000ff060 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009990 tx=0x7fc0f4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.575+0000 7fc106302700 1 -- 172.21.15.40:0/44029171 shutdown_connections 2023-07-18T03:30:13.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.575+0000 7fc106302700 1 --2- 172.21.15.40:0/44029171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1000ff5a0 0x7fc100106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.575+0000 7fc106302700 1 --2- 172.21.15.40:0/44029171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1000fffa0 0x7fc1001003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.575+0000 7fc106302700 1 --2- 172.21.15.40:0/44029171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1000febf0 0x7fc1000ff060 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.575+0000 7fc106302700 1 -- 172.21.15.40:0/44029171 >> 172.21.15.40:0/44029171 conn(0x7fc1000f9770 msgr2=0x7fc1000fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:13.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.575+0000 7fc106302700 1 -- 172.21.15.40:0/44029171 shutdown_connections 2023-07-18T03:30:13.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.575+0000 7fc106302700 1 -- 172.21.15.40:0/44029171 wait complete. 2023-07-18T03:30:13.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc106302700 1 Processor -- start 2023-07-18T03:30:13.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc106302700 1 -- start start 2023-07-18T03:30:13.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc106302700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1000febf0 0x7fc100197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:13.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc106302700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1000ff5a0 0x7fc100197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:13.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc106302700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1000fffa0 0x7fc10019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:13.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc106302700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10019f580 con 0x7fc1000febf0 2023-07-18T03:30:13.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc106302700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10019f6c0 con 0x7fc1000fffa0 2023-07-18T03:30:13.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc106302700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10019f850 con 0x7fc1000ff5a0 2023-07-18T03:30:13.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc0fffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1000febf0 0x7fc100197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:13.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc0fffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1000febf0 0x7fc100197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56776/0 (socket says 172.21.15.40:56776) 2023-07-18T03:30:13.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc0fffff700 1 -- 172.21.15.40:0/394771698 learned_addr learned my addr 172.21.15.40:0/394771698 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:13.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc0ff7fe700 1 --2- 172.21.15.40:0/394771698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1000ff5a0 0x7fc100197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:13.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc0fffff700 1 -- 172.21.15.40:0/394771698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1000ff5a0 msgr2=0x7fc100197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:13.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc10489f700 1 --2- 172.21.15.40:0/394771698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1000fffa0 0x7fc10019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:13.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc0fffff700 1 --2- 172.21.15.40:0/394771698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1000ff5a0 0x7fc100197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc0fffff700 1 -- 172.21.15.40:0/394771698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1000fffa0 msgr2=0x7fc10019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:13.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc0fffff700 1 --2- 172.21.15.40:0/394771698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1000fffa0 0x7fc10019efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.576+0000 7fc0fffff700 1 -- 172.21.15.40:0/394771698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f40095f0 con 0x7fc1000febf0 2023-07-18T03:30:13.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.577+0000 7fc0fffff700 1 --2- 172.21.15.40:0/394771698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1000febf0 0x7fc100197a00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000d980 tx=0x7fc0f000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:13.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.577+0000 7fc0fd7fa700 1 -- 172.21.15.40:0/394771698 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f000bb60 con 0x7fc1000febf0 2023-07-18T03:30:13.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.577+0000 7fc106302700 1 -- 172.21.15.40:0/394771698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10019fb30 con 0x7fc1000febf0 2023-07-18T03:30:13.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.577+0000 7fc0fd7fa700 1 -- 172.21.15.40:0/394771698 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f0007620 con 0x7fc1000febf0 2023-07-18T03:30:13.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.577+0000 7fc0fd7fa700 1 -- 172.21.15.40:0/394771698 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f000f620 con 0x7fc1000febf0 2023-07-18T03:30:13.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.577+0000 7fc106302700 1 -- 172.21.15.40:0/394771698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1001a0040 con 0x7fc1000febf0 2023-07-18T03:30:13.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.578+0000 7fc106302700 1 -- 172.21.15.40:0/394771698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc10010a1b0 con 0x7fc1000febf0 2023-07-18T03:30:13.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.579+0000 7fc0fd7fa700 1 -- 172.21.15.40:0/394771698 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc0f0010460 con 0x7fc1000febf0 2023-07-18T03:30:13.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.579+0000 7fc0fd7fa700 1 --2- 172.21.15.40:0/394771698 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc0e80a1220 0x7fc0e80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:13.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.580+0000 7fc0fd7fa700 1 -- 172.21.15.40:0/394771698 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc0f00cc890 con 0x7fc1000febf0 2023-07-18T03:30:13.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.580+0000 7fc0ff7fe700 1 --2- 172.21.15.40:0/394771698 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc0e80a1220 0x7fc0e80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:13.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.580+0000 7fc0ff7fe700 1 --2- 172.21.15.40:0/394771698 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc0e80a1220 0x7fc0e80a36d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40095c0 tx=0x7fc0f4009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:13.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.584+0000 7fc0fd7fa700 1 -- 172.21.15.40:0/394771698 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc0f0096030 con 0x7fc1000febf0 2023-07-18T03:30:13.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.787+0000 7fc106302700 1 -- 172.21.15.40:0/394771698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1001a05f0 con 0x7fc1000febf0 2023-07-18T03:30:13.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.788+0000 7fc0fd7fa700 1 -- 172.21.15.40:0/394771698 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22071 (secure 0 0 0) 0x7fc0f0016090 con 0x7fc1000febf0 2023-07-18T03:30:13.786 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:13.787 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":81,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:29:28.081282+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:30:13.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.791+0000 7fc106302700 1 -- 172.21.15.40:0/394771698 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc0e80a1220 msgr2=0x7fc0e80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:13.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.791+0000 7fc106302700 1 --2- 172.21.15.40:0/394771698 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc0e80a1220 0x7fc0e80a36d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40095c0 tx=0x7fc0f4009530 comp rx=0 tx=0).stop 2023-07-18T03:30:13.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.791+0000 7fc106302700 1 -- 172.21.15.40:0/394771698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1000febf0 msgr2=0x7fc100197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:13.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.791+0000 7fc106302700 1 --2- 172.21.15.40:0/394771698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1000febf0 0x7fc100197a00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000d980 tx=0x7fc0f000dd40 comp rx=0 tx=0).stop 2023-07-18T03:30:13.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.791+0000 7fc106302700 1 -- 172.21.15.40:0/394771698 shutdown_connections 2023-07-18T03:30:13.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.791+0000 7fc106302700 1 --2- 172.21.15.40:0/394771698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1000fffa0 0x7fc10019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.791+0000 7fc106302700 1 --2- 172.21.15.40:0/394771698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1000febf0 0x7fc100197a00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.791+0000 7fc106302700 1 --2- 172.21.15.40:0/394771698 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc0e80a1220 0x7fc0e80a36d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.791+0000 7fc106302700 1 --2- 172.21.15.40:0/394771698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1000ff5a0 0x7fc100197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.791+0000 7fc106302700 1 -- 172.21.15.40:0/394771698 >> 172.21.15.40:0/394771698 conn(0x7fc1000f9770 msgr2=0x7fc1000f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:13.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.791+0000 7fc106302700 1 -- 172.21.15.40:0/394771698 shutdown_connections 2023-07-18T03:30:13.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.791+0000 7fc106302700 1 -- 172.21.15.40:0/394771698 wait complete. 2023-07-18T03:30:13.801 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:25:38.267262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:29:24.727790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:29:26.078507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:30:13.802 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:30:13.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.935+0000 7fc3dc6fa700 1 -- 172.21.15.40:0/2575875501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d40fe9f0 msgr2=0x7fc3d40fee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:13.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.935+0000 7fc3dc6fa700 1 --2- 172.21.15.40:0/2575875501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d40fe9f0 0x7fc3d40fee00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009990 tx=0x7fc3c4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.935+0000 7fc3dc6fa700 1 -- 172.21.15.40:0/2575875501 shutdown_connections 2023-07-18T03:30:13.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.935+0000 7fc3dc6fa700 1 --2- 172.21.15.40:0/2575875501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3d4100570 0x7fc3d4106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.935+0000 7fc3dc6fa700 1 --2- 172.21.15.40:0/2575875501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d40fe9f0 0x7fc3d40fee00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.935+0000 7fc3dc6fa700 1 --2- 172.21.15.40:0/2575875501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3d40ffbe0 0x7fc3d4100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.935+0000 7fc3dc6fa700 1 -- 172.21.15.40:0/2575875501 >> 172.21.15.40:0/2575875501 conn(0x7fc3d40fa030 msgr2=0x7fc3d40fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.935+0000 7fc3dc6fa700 1 -- 172.21.15.40:0/2575875501 shutdown_connections 2023-07-18T03:30:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.935+0000 7fc3dc6fa700 1 -- 172.21.15.40:0/2575875501 wait complete. 2023-07-18T03:30:13.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.935+0000 7fc3dc6fa700 1 Processor -- start 2023-07-18T03:30:13.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3dc6fa700 1 -- start start 2023-07-18T03:30:13.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3dc6fa700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3d40fe9f0 0x7fc3d41947c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:13.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3dc6fa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3d40ffbe0 0x7fc3d4194d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:13.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3dc6fa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d4100570 0x7fc3d419bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:13.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3dc6fa700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d419c340 con 0x7fc3d4100570 2023-07-18T03:30:13.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3dc6fa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d419c480 con 0x7fc3d40ffbe0 2023-07-18T03:30:13.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3dc6fa700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d419c610 con 0x7fc3d40fe9f0 2023-07-18T03:30:13.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3dac97700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d4100570 0x7fc3d419bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:13.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3dac97700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d4100570 0x7fc3d419bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56792/0 (socket says 172.21.15.40:56792) 2023-07-18T03:30:13.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3dac97700 1 -- 172.21.15.40:0/1241589965 learned_addr learned my addr 172.21.15.40:0/1241589965 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:13.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3da496700 1 --2- 172.21.15.40:0/1241589965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3d40fe9f0 0x7fc3d41947c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:13.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3dac97700 1 -- 172.21.15.40:0/1241589965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3d40fe9f0 msgr2=0x7fc3d41947c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:13.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3dac97700 1 --2- 172.21.15.40:0/1241589965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3d40fe9f0 0x7fc3d41947c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3dac97700 1 -- 172.21.15.40:0/1241589965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3d40ffbe0 msgr2=0x7fc3d4194d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:13.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3d9c95700 1 --2- 172.21.15.40:0/1241589965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3d40ffbe0 0x7fc3d4194d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:13.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3dac97700 1 --2- 172.21.15.40:0/1241589965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3d40ffbe0 0x7fc3d4194d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:13.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3dac97700 1 -- 172.21.15.40:0/1241589965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c40095f0 con 0x7fc3d4100570 2023-07-18T03:30:13.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.936+0000 7fc3dac97700 1 --2- 172.21.15.40:0/1241589965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d4100570 0x7fc3d419bd70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc007ee0 tx=0x7fc3cc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:13.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.937+0000 7fc3cb7fe700 1 -- 172.21.15.40:0/1241589965 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc00ddd0 con 0x7fc3d4100570 2023-07-18T03:30:13.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.937+0000 7fc3dc6fa700 1 -- 172.21.15.40:0/1241589965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d419c8f0 con 0x7fc3d4100570 2023-07-18T03:30:13.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.937+0000 7fc3cb7fe700 1 -- 172.21.15.40:0/1241589965 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3cc00f040 con 0x7fc3d4100570 2023-07-18T03:30:13.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.937+0000 7fc3cb7fe700 1 -- 172.21.15.40:0/1241589965 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc015400 con 0x7fc3d4100570 2023-07-18T03:30:13.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.937+0000 7fc3dc6fa700 1 -- 172.21.15.40:0/1241589965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d419ce00 con 0x7fc3d4100570 2023-07-18T03:30:13.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.937+0000 7fc3dc6fa700 1 -- 172.21.15.40:0/1241589965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d4105fd0 con 0x7fc3d4100570 2023-07-18T03:30:13.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.939+0000 7fc3cb7fe700 1 -- 172.21.15.40:0/1241589965 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc3cc004000 con 0x7fc3d4100570 2023-07-18T03:30:13.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.939+0000 7fc3cb7fe700 1 --2- 172.21.15.40:0/1241589965 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3c00a11e0 0x7fc3c00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:13.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.939+0000 7fc3cb7fe700 1 -- 172.21.15.40:0/1241589965 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc3cc0cca50 con 0x7fc3d4100570 2023-07-18T03:30:13.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.939+0000 7fc3da496700 1 --2- 172.21.15.40:0/1241589965 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3c00a11e0 0x7fc3c00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:13.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.940+0000 7fc3da496700 1 --2- 172.21.15.40:0/1241589965 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3c00a11e0 0x7fc3c00a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400b5c0 tx=0x7fc3c4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:13.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:13.943+0000 7fc3cb7fe700 1 -- 172.21.15.40:0/1241589965 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc3cc0961f0 con 0x7fc3d4100570 2023-07-18T03:30:14.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:13 smithi134 ceph-mon[98216]: pgmap v271: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2023-07-18T03:30:14.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:13 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/65122187' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-18T03:30:14.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:13 smithi040 ceph-mon[96439]: pgmap v271: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2023-07-18T03:30:14.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:13 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/65122187' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-18T03:30:14.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:13 smithi064 ceph-mon[98059]: pgmap v271: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2023-07-18T03:30:14.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:13 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/65122187' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-18T03:30:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.142+0000 7fc3dc6fa700 1 -- 172.21.15.40:0/1241589965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3d402c9e0 con 0x7fc3d4100570 2023-07-18T03:30:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.142+0000 7fc3cb7fe700 1 -- 172.21.15.40:0/1241589965 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc3cc099810 con 0x7fc3d4100570 2023-07-18T03:30:14.140 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-07-18T03:30:14.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.144+0000 7fc3dc6fa700 1 -- 172.21.15.40:0/1241589965 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3c00a11e0 msgr2=0x7fc3c00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:14.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.144+0000 7fc3dc6fa700 1 --2- 172.21.15.40:0/1241589965 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3c00a11e0 0x7fc3c00a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400b5c0 tx=0x7fc3c4005fb0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.144+0000 7fc3dc6fa700 1 -- 172.21.15.40:0/1241589965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d4100570 msgr2=0x7fc3d419bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:14.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.144+0000 7fc3dc6fa700 1 --2- 172.21.15.40:0/1241589965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d4100570 0x7fc3d419bd70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc007ee0 tx=0x7fc3cc00d3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.144+0000 7fc3dc6fa700 1 -- 172.21.15.40:0/1241589965 shutdown_connections 2023-07-18T03:30:14.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.144+0000 7fc3dc6fa700 1 --2- 172.21.15.40:0/1241589965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d4100570 0x7fc3d419bd70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.144+0000 7fc3dc6fa700 1 --2- 172.21.15.40:0/1241589965 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3c00a11e0 0x7fc3c00a3690 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.144+0000 7fc3dc6fa700 1 --2- 172.21.15.40:0/1241589965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3d40fe9f0 0x7fc3d41947c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.144+0000 7fc3dc6fa700 1 --2- 172.21.15.40:0/1241589965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3d40ffbe0 0x7fc3d4194d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.144+0000 7fc3dc6fa700 1 -- 172.21.15.40:0/1241589965 >> 172.21.15.40:0/1241589965 conn(0x7fc3d40fa030 msgr2=0x7fc3d4103110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:14.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.144+0000 7fc3dc6fa700 1 -- 172.21.15.40:0/1241589965 shutdown_connections 2023-07-18T03:30:14.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.144+0000 7fc3dc6fa700 1 -- 172.21.15.40:0/1241589965 wait complete. 2023-07-18T03:30:14.154 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:30:14.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.285+0000 7f53a4ded700 1 -- 172.21.15.40:0/3203772130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53a00ffbd0 msgr2=0x7f53a0100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:14.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.285+0000 7f53a4ded700 1 --2- 172.21.15.40:0/3203772130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53a00ffbd0 0x7f53a0100020 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5390009990 tx=0x7f5390009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.285+0000 7f53a4ded700 1 -- 172.21.15.40:0/3203772130 shutdown_connections 2023-07-18T03:30:14.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.285+0000 7f53a4ded700 1 --2- 172.21.15.40:0/3203772130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a0100560 0x7f53a0106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.285+0000 7f53a4ded700 1 --2- 172.21.15.40:0/3203772130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53a00fe9e0 0x7f53a00fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.285+0000 7f53a4ded700 1 --2- 172.21.15.40:0/3203772130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53a00ffbd0 0x7f53a0100020 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.285+0000 7f53a4ded700 1 -- 172.21.15.40:0/3203772130 >> 172.21.15.40:0/3203772130 conn(0x7f53a00fa000 msgr2=0x7f53a00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:14.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.285+0000 7f53a4ded700 1 -- 172.21.15.40:0/3203772130 shutdown_connections 2023-07-18T03:30:14.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.285+0000 7f53a4ded700 1 -- 172.21.15.40:0/3203772130 wait complete. 2023-07-18T03:30:14.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.286+0000 7f53a4ded700 1 Processor -- start 2023-07-18T03:30:14.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.286+0000 7f53a4ded700 1 -- start start 2023-07-18T03:30:14.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.286+0000 7f53a4ded700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a00fe9e0 0x7f53a0194790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:14.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.286+0000 7f53a4ded700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53a00ffbd0 0x7f53a0194cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:14.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.286+0000 7f53a4ded700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53a0100560 0x7f53a019bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:14.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.286+0000 7f53a4ded700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a0067670 con 0x7f53a00ffbd0 2023-07-18T03:30:14.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.286+0000 7f53a4ded700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a00677e0 con 0x7f53a0100560 2023-07-18T03:30:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.286+0000 7f53a4ded700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a00679a0 con 0x7f53a00fe9e0 2023-07-18T03:30:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.287+0000 7f539dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53a00ffbd0 0x7f53a0194cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.287+0000 7f539dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53a00ffbd0 0x7f53a0194cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56804/0 (socket says 172.21.15.40:56804) 2023-07-18T03:30:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.287+0000 7f539dd9b700 1 -- 172.21.15.40:0/561141590 learned_addr learned my addr 172.21.15.40:0/561141590 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.287+0000 7f539ed9d700 1 --2- 172.21.15.40:0/561141590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53a0100560 0x7f53a019bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.287+0000 7f539e59c700 1 --2- 172.21.15.40:0/561141590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a00fe9e0 0x7f53a0194790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:14.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.287+0000 7f539dd9b700 1 -- 172.21.15.40:0/561141590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a00fe9e0 msgr2=0x7f53a0194790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:14.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.287+0000 7f539dd9b700 1 --2- 172.21.15.40:0/561141590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a00fe9e0 0x7f53a0194790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.287+0000 7f539dd9b700 1 -- 172.21.15.40:0/561141590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53a0100560 msgr2=0x7f53a019bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:14.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.287+0000 7f539dd9b700 1 --2- 172.21.15.40:0/561141590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53a0100560 0x7f53a019bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.287+0000 7f539dd9b700 1 -- 172.21.15.40:0/561141590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53900095f0 con 0x7f53a00ffbd0 2023-07-18T03:30:14.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.287+0000 7f539dd9b700 1 --2- 172.21.15.40:0/561141590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53a00ffbd0 0x7f53a0194cd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f539000c010 tx=0x7f5390004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:14.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.287+0000 7f538f7fe700 1 -- 172.21.15.40:0/561141590 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539001d070 con 0x7f53a00ffbd0 2023-07-18T03:30:14.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.287+0000 7f53a4ded700 1 -- 172.21.15.40:0/561141590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a0067bc0 con 0x7f53a00ffbd0 2023-07-18T03:30:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.287+0000 7f538f7fe700 1 -- 172.21.15.40:0/561141590 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5390005030 con 0x7f53a00ffbd0 2023-07-18T03:30:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.287+0000 7f538f7fe700 1 -- 172.21.15.40:0/561141590 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390004b90 con 0x7f53a00ffbd0 2023-07-18T03:30:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.288+0000 7f53a4ded700 1 -- 172.21.15.40:0/561141590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a0068010 con 0x7f53a00ffbd0 2023-07-18T03:30:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.288+0000 7f53a4ded700 1 -- 172.21.15.40:0/561141590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a0105960 con 0x7f53a00ffbd0 2023-07-18T03:30:14.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.290+0000 7f538f7fe700 1 -- 172.21.15.40:0/561141590 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f53900051a0 con 0x7f53a00ffbd0 2023-07-18T03:30:14.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.291+0000 7f538f7fe700 1 --2- 172.21.15.40:0/561141590 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f53880a11e0 0x7f53880a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:14.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.291+0000 7f539e59c700 1 --2- 172.21.15.40:0/561141590 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f53880a11e0 0x7f53880a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:14.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.291+0000 7f538f7fe700 1 -- 172.21.15.40:0/561141590 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f53900ce020 con 0x7f53a00ffbd0 2023-07-18T03:30:14.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.291+0000 7f539e59c700 1 --2- 172.21.15.40:0/561141590 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f53880a11e0 0x7f53880a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f53a00ffa30 tx=0x7f539400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:14.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.294+0000 7f538f7fe700 1 -- 172.21.15.40:0/561141590 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5390097850 con 0x7f53a00ffbd0 2023-07-18T03:30:14.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.495+0000 7f53a4ded700 1 -- 172.21.15.40:0/561141590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f53a00684a0 con 0x7f53a00ffbd0 2023-07-18T03:30:14.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.496+0000 7f538f7fe700 1 -- 172.21.15.40:0/561141590 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7f539009ae70 con 0x7f53a00ffbd0 2023-07-18T03:30:14.494 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-07-18T03:30:14.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.498+0000 7f53a4ded700 1 -- 172.21.15.40:0/561141590 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f53880a11e0 msgr2=0x7f53880a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:14.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.498+0000 7f53a4ded700 1 --2- 172.21.15.40:0/561141590 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f53880a11e0 0x7f53880a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f53a00ffa30 tx=0x7f539400b410 comp rx=0 tx=0).stop 2023-07-18T03:30:14.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.498+0000 7f53a4ded700 1 -- 172.21.15.40:0/561141590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53a00ffbd0 msgr2=0x7f53a0194cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:14.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.498+0000 7f53a4ded700 1 --2- 172.21.15.40:0/561141590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53a00ffbd0 0x7f53a0194cd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f539000c010 tx=0x7f5390004730 comp rx=0 tx=0).stop 2023-07-18T03:30:14.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.498+0000 7f53a4ded700 1 -- 172.21.15.40:0/561141590 shutdown_connections 2023-07-18T03:30:14.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.498+0000 7f53a4ded700 1 --2- 172.21.15.40:0/561141590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53a0100560 0x7f53a019bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.498+0000 7f53a4ded700 1 --2- 172.21.15.40:0/561141590 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f53880a11e0 0x7f53880a3690 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.498+0000 7f53a4ded700 1 --2- 172.21.15.40:0/561141590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a00fe9e0 0x7f53a0194790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.498+0000 7f53a4ded700 1 --2- 172.21.15.40:0/561141590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53a00ffbd0 0x7f53a0194cd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.498+0000 7f53a4ded700 1 -- 172.21.15.40:0/561141590 >> 172.21.15.40:0/561141590 conn(0x7f53a00fa000 msgr2=0x7f53a00fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:14.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.498+0000 7f53a4ded700 1 -- 172.21.15.40:0/561141590 shutdown_connections 2023-07-18T03:30:14.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.498+0000 7f53a4ded700 1 -- 172.21.15.40:0/561141590 wait complete. 2023-07-18T03:30:14.508 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:30:14.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.639+0000 7f4cde786700 1 -- 172.21.15.40:0/562215141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd80ffc40 msgr2=0x7f4cd8100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:14.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.639+0000 7f4cde786700 1 --2- 172.21.15.40:0/562215141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd80ffc40 0x7f4cd8100090 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8009990 tx=0x7f4cc8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.639+0000 7f4cde786700 1 -- 172.21.15.40:0/562215141 shutdown_connections 2023-07-18T03:30:14.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.639+0000 7f4cde786700 1 --2- 172.21.15.40:0/562215141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd81005d0 0x7f4cd8106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.639+0000 7f4cde786700 1 --2- 172.21.15.40:0/562215141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cd80fea50 0x7f4cd80fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.639+0000 7f4cde786700 1 --2- 172.21.15.40:0/562215141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd80ffc40 0x7f4cd8100090 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.640+0000 7f4cde786700 1 -- 172.21.15.40:0/562215141 >> 172.21.15.40:0/562215141 conn(0x7f4cd80fa070 msgr2=0x7f4cd80fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:14.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.640+0000 7f4cde786700 1 -- 172.21.15.40:0/562215141 shutdown_connections 2023-07-18T03:30:14.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.640+0000 7f4cde786700 1 -- 172.21.15.40:0/562215141 wait complete. 2023-07-18T03:30:14.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.640+0000 7f4cde786700 1 Processor -- start 2023-07-18T03:30:14.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.640+0000 7f4cde786700 1 -- start start 2023-07-18T03:30:14.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cde786700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd80fea50 0x7f4cd8197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:14.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cde786700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd80ffc40 0x7f4cd8197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:14.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cde786700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cd81005d0 0x7f4cd819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:14.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cde786700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd819f620 con 0x7f4cd80fea50 2023-07-18T03:30:14.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cde786700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd819f760 con 0x7f4cd81005d0 2023-07-18T03:30:14.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cde786700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd819f8f0 con 0x7f4cd80ffc40 2023-07-18T03:30:14.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cd77fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd80ffc40 0x7f4cd8197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:14.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cd77fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd80ffc40 0x7f4cd8197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:45844/0 (socket says 172.21.15.40:45844) 2023-07-18T03:30:14.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cd7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd80fea50 0x7f4cd8197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:14.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cd77fe700 1 -- 172.21.15.40:0/937008798 learned_addr learned my addr 172.21.15.40:0/937008798 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:14.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cdcd23700 1 --2- 172.21.15.40:0/937008798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cd81005d0 0x7f4cd819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:14.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cd7fff700 1 -- 172.21.15.40:0/937008798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd80ffc40 msgr2=0x7f4cd8197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:14.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cd7fff700 1 --2- 172.21.15.40:0/937008798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd80ffc40 0x7f4cd8197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cd7fff700 1 -- 172.21.15.40:0/937008798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cd81005d0 msgr2=0x7f4cd819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:14.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cd7fff700 1 --2- 172.21.15.40:0/937008798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cd81005d0 0x7f4cd819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cd7fff700 1 -- 172.21.15.40:0/937008798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc80095f0 con 0x7f4cd80fea50 2023-07-18T03:30:14.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cdcd23700 1 --2- 172.21.15.40:0/937008798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cd81005d0 0x7f4cd819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:14.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cd7fff700 1 --2- 172.21.15.40:0/937008798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd80fea50 0x7f4cd8197aa0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000c940 tx=0x7f4cc000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:14.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.641+0000 7f4cd57fa700 1 -- 172.21.15.40:0/937008798 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0004220 con 0x7f4cd80fea50 2023-07-18T03:30:14.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.642+0000 7f4cd57fa700 1 -- 172.21.15.40:0/937008798 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cc000ce30 con 0x7f4cd80fea50 2023-07-18T03:30:14.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.642+0000 7f4cde786700 1 -- 172.21.15.40:0/937008798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd819fbd0 con 0x7f4cd80fea50 2023-07-18T03:30:14.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.642+0000 7f4cd57fa700 1 -- 172.21.15.40:0/937008798 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0003c00 con 0x7f4cd80fea50 2023-07-18T03:30:14.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.642+0000 7f4cde786700 1 -- 172.21.15.40:0/937008798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd81a00e0 con 0x7f4cd80fea50 2023-07-18T03:30:14.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.643+0000 7f4cde786700 1 -- 172.21.15.40:0/937008798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd810a230 con 0x7f4cd80fea50 2023-07-18T03:30:14.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.644+0000 7f4cd57fa700 1 -- 172.21.15.40:0/937008798 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4cc0003d60 con 0x7f4cd80fea50 2023-07-18T03:30:14.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.644+0000 7f4cd57fa700 1 --2- 172.21.15.40:0/937008798 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4cc40a11e0 0x7f4cc40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:14.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.644+0000 7f4cd77fe700 1 --2- 172.21.15.40:0/937008798 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4cc40a11e0 0x7f4cc40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:14.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.644+0000 7f4cd57fa700 1 -- 172.21.15.40:0/937008798 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4cc00cc3c0 con 0x7f4cd80fea50 2023-07-18T03:30:14.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.645+0000 7f4cd77fe700 1 --2- 172.21.15.40:0/937008798 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4cc40a11e0 0x7f4cc40a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8006010 tx=0x7f4cc800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:14.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.648+0000 7f4cd57fa700 1 -- 172.21.15.40:0/937008798 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4cc0095b60 con 0x7f4cd80fea50 2023-07-18T03:30:14.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:14 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/394771698' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:14.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:14 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1241589965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:14.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:14 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/561141590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:14.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.847+0000 7f4cde786700 1 -- 172.21.15.40:0/937008798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4cd810a3f0 con 0x7f4cd80fea50 2023-07-18T03:30:14.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.847+0000 7f4cd57fa700 1 -- 172.21.15.40:0/937008798 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4cc0099180 con 0x7f4cd80fea50 2023-07-18T03:30:14.845 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-07-18T03:30:14.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.849+0000 7f4cde786700 1 -- 172.21.15.40:0/937008798 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4cc40a11e0 msgr2=0x7f4cc40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.849+0000 7f4cde786700 1 --2- 172.21.15.40:0/937008798 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4cc40a11e0 0x7f4cc40a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8006010 tx=0x7f4cc800b540 comp rx=0 tx=0).stop 2023-07-18T03:30:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.849+0000 7f4cde786700 1 -- 172.21.15.40:0/937008798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd80fea50 msgr2=0x7f4cd8197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.849+0000 7f4cde786700 1 --2- 172.21.15.40:0/937008798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd80fea50 0x7f4cd8197aa0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000c940 tx=0x7f4cc000cc50 comp rx=0 tx=0).stop 2023-07-18T03:30:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.849+0000 7f4cde786700 1 -- 172.21.15.40:0/937008798 shutdown_connections 2023-07-18T03:30:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.849+0000 7f4cde786700 1 --2- 172.21.15.40:0/937008798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cd81005d0 0x7f4cd819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.849+0000 7f4cde786700 1 --2- 172.21.15.40:0/937008798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd80fea50 0x7f4cd8197aa0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.849+0000 7f4cde786700 1 --2- 172.21.15.40:0/937008798 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4cc40a11e0 0x7f4cc40a3690 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.849+0000 7f4cde786700 1 --2- 172.21.15.40:0/937008798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd80ffc40 0x7f4cd8197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.849+0000 7f4cde786700 1 -- 172.21.15.40:0/937008798 >> 172.21.15.40:0/937008798 conn(0x7f4cd80fa070 msgr2=0x7f4cd8068860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.849+0000 7f4cde786700 1 -- 172.21.15.40:0/937008798 shutdown_connections 2023-07-18T03:30:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.849+0000 7f4cde786700 1 -- 172.21.15.40:0/937008798 wait complete. 2023-07-18T03:30:14.859 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:30:14.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.988+0000 7f68b2c8b700 1 -- 172.21.15.40:0/566161017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac100550 msgr2=0x7f68ac106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:14.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.988+0000 7f68b2c8b700 1 --2- 172.21.15.40:0/566161017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac100550 0x7f68ac106e10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f68a400b0a0 tx=0x7f68a400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.988+0000 7f68b2c8b700 1 -- 172.21.15.40:0/566161017 shutdown_connections 2023-07-18T03:30:14.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.988+0000 7f68b2c8b700 1 --2- 172.21.15.40:0/566161017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac100550 0x7f68ac106e10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.988+0000 7f68b2c8b700 1 --2- 172.21.15.40:0/566161017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ac0fe9d0 0x7f68ac0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.988+0000 7f68b2c8b700 1 --2- 172.21.15.40:0/566161017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac0ffbc0 0x7f68ac100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.988+0000 7f68b2c8b700 1 -- 172.21.15.40:0/566161017 >> 172.21.15.40:0/566161017 conn(0x7f68ac0f9fd0 msgr2=0x7f68ac0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:14.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.988+0000 7f68b2c8b700 1 -- 172.21.15.40:0/566161017 shutdown_connections 2023-07-18T03:30:14.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.988+0000 7f68b2c8b700 1 -- 172.21.15.40:0/566161017 wait complete. 2023-07-18T03:30:14.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.988+0000 7f68b2c8b700 1 Processor -- start 2023-07-18T03:30:14.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.988+0000 7f68b2c8b700 1 -- start start 2023-07-18T03:30:14.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.988+0000 7f68b2c8b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac0fe9d0 0x7f68ac068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:14.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68b2c8b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac0ffbc0 0x7f68ac19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:14.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68b2c8b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ac100550 0x7f68ac19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:14.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68b2c8b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac19d410 con 0x7f68ac0ffbc0 2023-07-18T03:30:14.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68b2c8b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac19d580 con 0x7f68ac0fe9d0 2023-07-18T03:30:14.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68b2c8b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac19d740 con 0x7f68ac100550 2023-07-18T03:30:14.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68abfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac0ffbc0 0x7f68ac19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:14.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68abfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac0ffbc0 0x7f68ac19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56830/0 (socket says 172.21.15.40:56830) 2023-07-18T03:30:14.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68abfff700 1 -- 172.21.15.40:0/757834397 learned_addr learned my addr 172.21.15.40:0/757834397 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:14.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68b0a27700 1 --2- 172.21.15.40:0/757834397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac0fe9d0 0x7f68ac068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:14.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68b1228700 1 --2- 172.21.15.40:0/757834397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ac100550 0x7f68ac19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:14.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68abfff700 1 -- 172.21.15.40:0/757834397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ac100550 msgr2=0x7f68ac19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:14.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68abfff700 1 --2- 172.21.15.40:0/757834397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ac100550 0x7f68ac19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68abfff700 1 -- 172.21.15.40:0/757834397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac0fe9d0 msgr2=0x7f68ac068200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:14.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68abfff700 1 --2- 172.21.15.40:0/757834397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac0fe9d0 0x7f68ac068200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68abfff700 1 -- 172.21.15.40:0/757834397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a4009d00 con 0x7f68ac0ffbc0 2023-07-18T03:30:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68abfff700 1 --2- 172.21.15.40:0/757834397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac0ffbc0 0x7f68ac19a5f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f689800c920 tx=0x7f689800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68a9ffb700 1 -- 172.21.15.40:0/757834397 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898007b00 con 0x7f68ac0ffbc0 2023-07-18T03:30:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68a9ffb700 1 -- 172.21.15.40:0/757834397 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6898004cf0 con 0x7f68ac0ffbc0 2023-07-18T03:30:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68b2c8b700 1 -- 172.21.15.40:0/757834397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ac19da20 con 0x7f68ac0ffbc0 2023-07-18T03:30:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.989+0000 7f68a9ffb700 1 -- 172.21.15.40:0/757834397 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898005610 con 0x7f68ac0ffbc0 2023-07-18T03:30:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.990+0000 7f68b2c8b700 1 -- 172.21.15.40:0/757834397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac19df80 con 0x7f68ac0ffbc0 2023-07-18T03:30:14.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.991+0000 7f68b2c8b700 1 -- 172.21.15.40:0/757834397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ac105ec0 con 0x7f68ac0ffbc0 2023-07-18T03:30:14.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.992+0000 7f68a9ffb700 1 -- 172.21.15.40:0/757834397 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6898004750 con 0x7f68ac0ffbc0 2023-07-18T03:30:14.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.992+0000 7f68a9ffb700 1 --2- 172.21.15.40:0/757834397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f689c0a12a0 0x7f689c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:14.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.992+0000 7f68a9ffb700 1 -- 172.21.15.40:0/757834397 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f68980cc800 con 0x7f68ac0ffbc0 2023-07-18T03:30:14.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.992+0000 7f68b0a27700 1 --2- 172.21.15.40:0/757834397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f689c0a12a0 0x7f689c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:14.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.993+0000 7f68b0a27700 1 --2- 172.21.15.40:0/757834397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f689c0a12a0 0x7f689c0a3750 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f68a0005950 tx=0x7f68a00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:14.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:14.996+0000 7f68a9ffb700 1 -- 172.21.15.40:0/757834397 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6898095fa0 con 0x7f68ac0ffbc0 2023-07-18T03:30:15.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:14 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/394771698' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:15.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:14 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1241589965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:15.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:14 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/561141590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:15.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:14 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/394771698' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:15.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:14 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1241589965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:15.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:14 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/561141590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:15.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.246+0000 7f68b2c8b700 1 -- 172.21.15.40:0/757834397 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68ac02c850 con 0x7f68ac0ffbc0 2023-07-18T03:30:15.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.247+0000 7f68a9ffb700 1 -- 172.21.15.40:0/757834397 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10888 (secure 0 0 0) 0x7f689801d020 con 0x7f68ac0ffbc0 2023-07-18T03:30:15.244 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:15.245 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:29:26.718789+0000","modified":"2023-07-18T03:29:34.650353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24523,"mds_2":24482,"mds_3":24452,"mds_4":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14601":{"gid":14601,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1818167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1818167554},{"type":"v1","addr":"172.21.15.40:6839","nonce":1818167554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14613":{"gid":14613,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/3953074960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":3953074960},{"type":"v1","addr":"172.21.15.40:6841","nonce":3953074960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3509655945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3509655945},{"type":"v1","addr":"172.21.15.134:6837","nonce":3509655945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/3007039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3007039043},{"type":"v1","addr":"172.21.15.64:6839","nonce":3007039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:30:15.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.249+0000 7f68b2c8b700 1 -- 172.21.15.40:0/757834397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f689c0a12a0 msgr2=0x7f689c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:15.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.249+0000 7f68b2c8b700 1 --2- 172.21.15.40:0/757834397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f689c0a12a0 0x7f689c0a3750 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f68a0005950 tx=0x7f68a00058e0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.249+0000 7f68b2c8b700 1 -- 172.21.15.40:0/757834397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac0ffbc0 msgr2=0x7f68ac19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:15.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.249+0000 7f68b2c8b700 1 --2- 172.21.15.40:0/757834397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac0ffbc0 0x7f68ac19a5f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f689800c920 tx=0x7f689800cce0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.249+0000 7f68b2c8b700 1 -- 172.21.15.40:0/757834397 shutdown_connections 2023-07-18T03:30:15.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.249+0000 7f68b2c8b700 1 --2- 172.21.15.40:0/757834397 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f689c0a12a0 0x7f689c0a3750 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.249+0000 7f68b2c8b700 1 --2- 172.21.15.40:0/757834397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ac100550 0x7f68ac19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.249+0000 7f68b2c8b700 1 --2- 172.21.15.40:0/757834397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac0fe9d0 0x7f68ac068200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.249+0000 7f68b2c8b700 1 --2- 172.21.15.40:0/757834397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac0ffbc0 0x7f68ac19a5f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.249+0000 7f68b2c8b700 1 -- 172.21.15.40:0/757834397 >> 172.21.15.40:0/757834397 conn(0x7f68ac0f9fd0 msgr2=0x7f68ac103000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:15.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.249+0000 7f68b2c8b700 1 -- 172.21.15.40:0/757834397 shutdown_connections 2023-07-18T03:30:15.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.249+0000 7f68b2c8b700 1 -- 172.21.15.40:0/757834397 wait complete. 2023-07-18T03:30:15.249 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 27 2023-07-18T03:30:15.259 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:30:15.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.387+0000 7f33335e0700 1 -- 172.21.15.40:0/149480317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f332c100590 msgr2=0x7f332c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:15.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.387+0000 7f33335e0700 1 --2- 172.21.15.40:0/149480317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f332c100590 0x7f332c106e50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f332400b0a0 tx=0x7f332400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.387+0000 7f33335e0700 1 -- 172.21.15.40:0/149480317 shutdown_connections 2023-07-18T03:30:15.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.387+0000 7f33335e0700 1 --2- 172.21.15.40:0/149480317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f332c100590 0x7f332c106e50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.387+0000 7f33335e0700 1 --2- 172.21.15.40:0/149480317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f332c0fea10 0x7f332c0fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.387+0000 7f33335e0700 1 --2- 172.21.15.40:0/149480317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f332c0ffc00 0x7f332c100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.387+0000 7f33335e0700 1 -- 172.21.15.40:0/149480317 >> 172.21.15.40:0/149480317 conn(0x7f332c0fa030 msgr2=0x7f332c0fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:15.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.387+0000 7f33335e0700 1 -- 172.21.15.40:0/149480317 shutdown_connections 2023-07-18T03:30:15.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.387+0000 7f33335e0700 1 -- 172.21.15.40:0/149480317 wait complete. 2023-07-18T03:30:15.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.388+0000 7f33335e0700 1 Processor -- start 2023-07-18T03:30:15.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.388+0000 7f33335e0700 1 -- start start 2023-07-18T03:30:15.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.388+0000 7f33335e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f332c0fea10 0x7f332c1029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:15.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.388+0000 7f33335e0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f332c0ffc00 0x7f332c100fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:15.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.388+0000 7f33335e0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f332c100590 0x7f332c101520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:15.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.388+0000 7f33335e0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c101cc0 con 0x7f332c100590 2023-07-18T03:30:15.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.388+0000 7f33335e0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c101e30 con 0x7f332c0fea10 2023-07-18T03:30:15.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.388+0000 7f33335e0700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c19f920 con 0x7f332c0ffc00 2023-07-18T03:30:15.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.388+0000 7f3331b7d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f332c100590 0x7f332c101520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:15.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.388+0000 7f3331b7d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f332c100590 0x7f332c101520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56846/0 (socket says 172.21.15.40:56846) 2023-07-18T03:30:15.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.388+0000 7f3331b7d700 1 -- 172.21.15.40:0/570654086 learned_addr learned my addr 172.21.15.40:0/570654086 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:15.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.388+0000 7f333137c700 1 --2- 172.21.15.40:0/570654086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f332c0fea10 0x7f332c1029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:15.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.388+0000 7f3330b7b700 1 --2- 172.21.15.40:0/570654086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f332c0ffc00 0x7f332c100fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:15.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.389+0000 7f3331b7d700 1 -- 172.21.15.40:0/570654086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f332c0ffc00 msgr2=0x7f332c100fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:15.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.389+0000 7f3331b7d700 1 --2- 172.21.15.40:0/570654086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f332c0ffc00 0x7f332c100fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.389+0000 7f3331b7d700 1 -- 172.21.15.40:0/570654086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f332c0fea10 msgr2=0x7f332c1029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:15.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.389+0000 7f3331b7d700 1 --2- 172.21.15.40:0/570654086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f332c0fea10 0x7f332c1029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.389+0000 7f3331b7d700 1 -- 172.21.15.40:0/570654086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3324009d00 con 0x7f332c100590 2023-07-18T03:30:15.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.389+0000 7f3331b7d700 1 --2- 172.21.15.40:0/570654086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f332c100590 0x7f332c101520 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f332400b830 tx=0x7f3324004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:15.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.389+0000 7f33227fc700 1 -- 172.21.15.40:0/570654086 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332400e040 con 0x7f332c100590 2023-07-18T03:30:15.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.389+0000 7f33227fc700 1 -- 172.21.15.40:0/570654086 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f332401db50 con 0x7f332c100590 2023-07-18T03:30:15.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.389+0000 7f33335e0700 1 -- 172.21.15.40:0/570654086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332c19fba0 con 0x7f332c100590 2023-07-18T03:30:15.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.389+0000 7f33335e0700 1 -- 172.21.15.40:0/570654086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332c1a0020 con 0x7f332c100590 2023-07-18T03:30:15.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.389+0000 7f33227fc700 1 -- 172.21.15.40:0/570654086 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332401bac0 con 0x7f332c100590 2023-07-18T03:30:15.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.390+0000 7f33335e0700 1 -- 172.21.15.40:0/570654086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332c10a1f0 con 0x7f332c100590 2023-07-18T03:30:15.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.391+0000 7f33227fc700 1 -- 172.21.15.40:0/570654086 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f332401dcc0 con 0x7f332c100590 2023-07-18T03:30:15.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.392+0000 7f33227fc700 1 --2- 172.21.15.40:0/570654086 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33180a1220 0x7f33180a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:15.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.392+0000 7f333137c700 1 --2- 172.21.15.40:0/570654086 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33180a1220 0x7f33180a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:15.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.393+0000 7f33227fc700 1 -- 172.21.15.40:0/570654086 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f33240cea60 con 0x7f332c100590 2023-07-18T03:30:15.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.393+0000 7f333137c700 1 --2- 172.21.15.40:0/570654086 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33180a1220 0x7f33180a36d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f332c0ffa60 tx=0x7f332800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:15.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.396+0000 7f33227fc700 1 -- 172.21.15.40:0/570654086 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3324098290 con 0x7f332c100590 2023-07-18T03:30:15.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.597+0000 7f33335e0700 1 -- 172.21.15.40:0/570654086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f332c1a0630 con 0x7f332c100590 2023-07-18T03:30:15.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.599+0000 7f33227fc700 1 -- 172.21.15.40:0/570654086 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22071 (secure 0 0 0) 0x7f332409b8b0 con 0x7f332c100590 2023-07-18T03:30:15.597 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:15.598 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":81,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:29:28.081282+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:30:15.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.601+0000 7f33335e0700 1 -- 172.21.15.40:0/570654086 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33180a1220 msgr2=0x7f33180a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:15.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.601+0000 7f33335e0700 1 --2- 172.21.15.40:0/570654086 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33180a1220 0x7f33180a36d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f332c0ffa60 tx=0x7f332800b410 comp rx=0 tx=0).stop 2023-07-18T03:30:15.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.601+0000 7f33335e0700 1 -- 172.21.15.40:0/570654086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f332c100590 msgr2=0x7f332c101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:15.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.601+0000 7f33335e0700 1 --2- 172.21.15.40:0/570654086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f332c100590 0x7f332c101520 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f332400b830 tx=0x7f3324004c80 comp rx=0 tx=0).stop 2023-07-18T03:30:15.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.601+0000 7f33335e0700 1 -- 172.21.15.40:0/570654086 shutdown_connections 2023-07-18T03:30:15.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.601+0000 7f33335e0700 1 --2- 172.21.15.40:0/570654086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f332c100590 0x7f332c101520 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.601+0000 7f33335e0700 1 --2- 172.21.15.40:0/570654086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f332c0fea10 0x7f332c1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.601+0000 7f33335e0700 1 --2- 172.21.15.40:0/570654086 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33180a1220 0x7f33180a36d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.601+0000 7f33335e0700 1 --2- 172.21.15.40:0/570654086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f332c0ffc00 0x7f332c100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.601+0000 7f33335e0700 1 -- 172.21.15.40:0/570654086 >> 172.21.15.40:0/570654086 conn(0x7f332c0fa030 msgr2=0x7f332c068680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:15.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.601+0000 7f33335e0700 1 -- 172.21.15.40:0/570654086 shutdown_connections 2023-07-18T03:30:15.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.601+0000 7f33335e0700 1 -- 172.21.15.40:0/570654086 wait complete. 2023-07-18T03:30:15.611 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:25:38.267262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:29:24.727790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:29:26.078507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:30:15.612 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:30:15.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.743+0000 7fc584901700 1 -- 172.21.15.40:0/1185659616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5800fea50 msgr2=0x7fc5800fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:15.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.743+0000 7fc584901700 1 --2- 172.21.15.40:0/1185659616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5800fea50 0x7fc5800fee60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc570009990 tx=0x7fc570009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.743+0000 7fc584901700 1 -- 172.21.15.40:0/1185659616 shutdown_connections 2023-07-18T03:30:15.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.743+0000 7fc584901700 1 --2- 172.21.15.40:0/1185659616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5801005d0 0x7fc580106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.743+0000 7fc584901700 1 --2- 172.21.15.40:0/1185659616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5800fea50 0x7fc5800fee60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.743+0000 7fc584901700 1 --2- 172.21.15.40:0/1185659616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5800ffc40 0x7fc580100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.743+0000 7fc584901700 1 -- 172.21.15.40:0/1185659616 >> 172.21.15.40:0/1185659616 conn(0x7fc5800fa070 msgr2=0x7fc5800fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:15.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.743+0000 7fc584901700 1 -- 172.21.15.40:0/1185659616 shutdown_connections 2023-07-18T03:30:15.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.743+0000 7fc584901700 1 -- 172.21.15.40:0/1185659616 wait complete. 2023-07-18T03:30:15.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.744+0000 7fc584901700 1 Processor -- start 2023-07-18T03:30:15.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.744+0000 7fc584901700 1 -- start start 2023-07-18T03:30:15.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.744+0000 7fc584901700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5800fea50 0x7fc580069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:15.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.744+0000 7fc584901700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5800ffc40 0x7fc58019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:15.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.744+0000 7fc584901700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5801005d0 0x7fc58019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:15.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.744+0000 7fc584901700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58019d350 con 0x7fc5801005d0 2023-07-18T03:30:15.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.744+0000 7fc584901700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58019d4c0 con 0x7fc5800fea50 2023-07-18T03:30:15.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.744+0000 7fc584901700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58019d680 con 0x7fc5800ffc40 2023-07-18T03:30:15.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.744+0000 7fc57ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5801005d0 0x7fc58019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:15.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.744+0000 7fc57ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5801005d0 0x7fc58019cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56860/0 (socket says 172.21.15.40:56860) 2023-07-18T03:30:15.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.744+0000 7fc57ed9d700 1 -- 172.21.15.40:0/4068415097 learned_addr learned my addr 172.21.15.40:0/4068415097 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:15.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.744+0000 7fc57e59c700 1 --2- 172.21.15.40:0/4068415097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5800fea50 0x7fc580069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:15.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.745+0000 7fc57dd9b700 1 --2- 172.21.15.40:0/4068415097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5800ffc40 0x7fc58019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:15.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.745+0000 7fc57ed9d700 1 -- 172.21.15.40:0/4068415097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5800ffc40 msgr2=0x7fc58019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:15.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.745+0000 7fc57ed9d700 1 --2- 172.21.15.40:0/4068415097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5800ffc40 0x7fc58019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.745+0000 7fc57ed9d700 1 -- 172.21.15.40:0/4068415097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5800fea50 msgr2=0x7fc580069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:15.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.745+0000 7fc57ed9d700 1 --2- 172.21.15.40:0/4068415097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5800fea50 0x7fc580069390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.745+0000 7fc57ed9d700 1 -- 172.21.15.40:0/4068415097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5700095f0 con 0x7fc5801005d0 2023-07-18T03:30:15.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.745+0000 7fc57dd9b700 1 --2- 172.21.15.40:0/4068415097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5800ffc40 0x7fc58019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:15.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.745+0000 7fc57ed9d700 1 --2- 172.21.15.40:0/4068415097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5801005d0 0x7fc58019cbb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc57800ef30 tx=0x7fc57800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:15.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.745+0000 7fc56f7fe700 1 -- 172.21.15.40:0/4068415097 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc578018070 con 0x7fc5801005d0 2023-07-18T03:30:15.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.745+0000 7fc56f7fe700 1 -- 172.21.15.40:0/4068415097 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc57800f040 con 0x7fc5801005d0 2023-07-18T03:30:15.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.745+0000 7fc584901700 1 -- 172.21.15.40:0/4068415097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc58019d900 con 0x7fc5801005d0 2023-07-18T03:30:15.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.745+0000 7fc56f7fe700 1 -- 172.21.15.40:0/4068415097 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57800cdb0 con 0x7fc5801005d0 2023-07-18T03:30:15.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.745+0000 7fc584901700 1 -- 172.21.15.40:0/4068415097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58019de10 con 0x7fc5801005d0 2023-07-18T03:30:15.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.746+0000 7fc584901700 1 -- 172.21.15.40:0/4068415097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58010a1b0 con 0x7fc5801005d0 2023-07-18T03:30:15.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.747+0000 7fc56f7fe700 1 -- 172.21.15.40:0/4068415097 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc578003a00 con 0x7fc5801005d0 2023-07-18T03:30:15.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.748+0000 7fc56f7fe700 1 --2- 172.21.15.40:0/4068415097 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5680a1260 0x7fc5680a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:15.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.748+0000 7fc56f7fe700 1 -- 172.21.15.40:0/4068415097 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc5780ccec0 con 0x7fc5801005d0 2023-07-18T03:30:15.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.748+0000 7fc57e59c700 1 --2- 172.21.15.40:0/4068415097 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5680a1260 0x7fc5680a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:15.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.748+0000 7fc57e59c700 1 --2- 172.21.15.40:0/4068415097 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5680a1260 0x7fc5680a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc570006010 tx=0x7fc570005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:15.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.751+0000 7fc56f7fe700 1 -- 172.21.15.40:0/4068415097 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc5780965f0 con 0x7fc5801005d0 2023-07-18T03:30:15.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.951+0000 7fc584901700 1 -- 172.21.15.40:0/4068415097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc580105990 con 0x7fc5801005d0 2023-07-18T03:30:15.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.952+0000 7fc56f7fe700 1 -- 172.21.15.40:0/4068415097 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc578099c10 con 0x7fc5801005d0 2023-07-18T03:30:15.949 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-07-18T03:30:15.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.954+0000 7fc584901700 1 -- 172.21.15.40:0/4068415097 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5680a1260 msgr2=0x7fc5680a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:15.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.954+0000 7fc584901700 1 --2- 172.21.15.40:0/4068415097 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5680a1260 0x7fc5680a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc570006010 tx=0x7fc570005960 comp rx=0 tx=0).stop 2023-07-18T03:30:15.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.954+0000 7fc584901700 1 -- 172.21.15.40:0/4068415097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5801005d0 msgr2=0x7fc58019cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:15.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.954+0000 7fc584901700 1 --2- 172.21.15.40:0/4068415097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5801005d0 0x7fc58019cbb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc57800ef30 tx=0x7fc57800c3a0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.954+0000 7fc584901700 1 -- 172.21.15.40:0/4068415097 shutdown_connections 2023-07-18T03:30:15.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.954+0000 7fc584901700 1 --2- 172.21.15.40:0/4068415097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5801005d0 0x7fc58019cbb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.954+0000 7fc584901700 1 --2- 172.21.15.40:0/4068415097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5800fea50 0x7fc580069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.954+0000 7fc584901700 1 --2- 172.21.15.40:0/4068415097 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5680a1260 0x7fc5680a3710 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.954+0000 7fc584901700 1 --2- 172.21.15.40:0/4068415097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5800ffc40 0x7fc58019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:15.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.954+0000 7fc584901700 1 -- 172.21.15.40:0/4068415097 >> 172.21.15.40:0/4068415097 conn(0x7fc5800fa070 msgr2=0x7fc5800fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:15.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.954+0000 7fc584901700 1 -- 172.21.15.40:0/4068415097 shutdown_connections 2023-07-18T03:30:15.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:15.954+0000 7fc584901700 1 -- 172.21.15.40:0/4068415097 wait complete. 2023-07-18T03:30:15.963 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:15 smithi040 ceph-mon[96439]: pgmap v272: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2023-07-18T03:30:15.963 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:15 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/937008798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:15.963 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:15 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/757834397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:15.963 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:15 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/570654086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:15.964 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:30:16.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:15 smithi134 ceph-mon[98216]: pgmap v272: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2023-07-18T03:30:16.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:15 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/937008798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:16.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:15 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/757834397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:16.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:15 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/570654086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:16.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.095+0000 7f4ff686d700 1 -- 172.21.15.40:0/3655477307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ff0102f40 msgr2=0x7f4ff0105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:16.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.095+0000 7f4ff686d700 1 --2- 172.21.15.40:0/3655477307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ff0102f40 0x7f4ff0105320 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009990 tx=0x7f4fd8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.095+0000 7f4ff686d700 1 -- 172.21.15.40:0/3655477307 shutdown_connections 2023-07-18T03:30:16.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.095+0000 7f4ff686d700 1 --2- 172.21.15.40:0/3655477307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ff0105860 0x7f4ff0107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.095+0000 7f4ff686d700 1 --2- 172.21.15.40:0/3655477307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0100620 0x7f4ff0102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.095+0000 7f4ff686d700 1 --2- 172.21.15.40:0/3655477307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ff0102f40 0x7f4ff0105320 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.095+0000 7f4ff686d700 1 -- 172.21.15.40:0/3655477307 >> 172.21.15.40:0/3655477307 conn(0x7f4ff00fa000 msgr2=0x7f4ff00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:16.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.095+0000 7f4ff686d700 1 -- 172.21.15.40:0/3655477307 shutdown_connections 2023-07-18T03:30:16.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.095+0000 7f4ff686d700 1 -- 172.21.15.40:0/3655477307 wait complete. 2023-07-18T03:30:16.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4ff686d700 1 Processor -- start 2023-07-18T03:30:16.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4ff686d700 1 -- start start 2023-07-18T03:30:16.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4ff686d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ff0100620 0x7f4ff0195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:16.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4ff686d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0102f40 0x7f4ff0195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:16.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4ff686d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ff0105860 0x7f4ff019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:16.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4ff686d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff019d380 con 0x7f4ff0105860 2023-07-18T03:30:16.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4ff686d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff019d4c0 con 0x7f4ff0100620 2023-07-18T03:30:16.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4ff686d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff019d650 con 0x7f4ff0102f40 2023-07-18T03:30:16.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4ff4e0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ff0105860 0x7f4ff019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:16.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4ff4e0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ff0105860 0x7f4ff019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56874/0 (socket says 172.21.15.40:56874) 2023-07-18T03:30:16.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4ff4e0a700 1 -- 172.21.15.40:0/2716789136 learned_addr learned my addr 172.21.15.40:0/2716789136 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:16.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4fe7fff700 1 --2- 172.21.15.40:0/2716789136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0102f40 0x7f4ff0195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:16.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4feffff700 1 --2- 172.21.15.40:0/2716789136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ff0100620 0x7f4ff0195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:16.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4ff4e0a700 1 -- 172.21.15.40:0/2716789136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0102f40 msgr2=0x7f4ff0195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:16.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4ff4e0a700 1 --2- 172.21.15.40:0/2716789136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0102f40 0x7f4ff0195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4ff4e0a700 1 -- 172.21.15.40:0/2716789136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ff0100620 msgr2=0x7f4ff0195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:16.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4ff4e0a700 1 --2- 172.21.15.40:0/2716789136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ff0100620 0x7f4ff0195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.096+0000 7f4ff4e0a700 1 -- 172.21.15.40:0/2716789136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd80095f0 con 0x7f4ff0105860 2023-07-18T03:30:16.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.097+0000 7f4ff4e0a700 1 --2- 172.21.15.40:0/2716789136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ff0105860 0x7f4ff019cdb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8007ee0 tx=0x7f4fe800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:16.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.097+0000 7f4fedffb700 1 -- 172.21.15.40:0/2716789136 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8017070 con 0x7f4ff0105860 2023-07-18T03:30:16.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.097+0000 7f4ff686d700 1 -- 172.21.15.40:0/2716789136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff019d930 con 0x7f4ff0105860 2023-07-18T03:30:16.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.097+0000 7f4fedffb700 1 -- 172.21.15.40:0/2716789136 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fe800f040 con 0x7f4ff0105860 2023-07-18T03:30:16.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.097+0000 7f4fedffb700 1 -- 172.21.15.40:0/2716789136 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8005740 con 0x7f4ff0105860 2023-07-18T03:30:16.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.097+0000 7f4ff686d700 1 -- 172.21.15.40:0/2716789136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff019ddf0 con 0x7f4ff0105860 2023-07-18T03:30:16.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.098+0000 7f4ff686d700 1 -- 172.21.15.40:0/2716789136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff010b290 con 0x7f4ff0105860 2023-07-18T03:30:16.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.099+0000 7f4fedffb700 1 -- 172.21.15.40:0/2716789136 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4fe800dc60 con 0x7f4ff0105860 2023-07-18T03:30:16.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.099+0000 7f4fedffb700 1 --2- 172.21.15.40:0/2716789136 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4fdc0a11e0 0x7f4fdc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:16.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.100+0000 7f4feffff700 1 --2- 172.21.15.40:0/2716789136 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4fdc0a11e0 0x7f4fdc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:16.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.100+0000 7f4fedffb700 1 -- 172.21.15.40:0/2716789136 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4fe80cc840 con 0x7f4ff0105860 2023-07-18T03:30:16.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.100+0000 7f4feffff700 1 --2- 172.21.15.40:0/2716789136 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4fdc0a11e0 0x7f4fdc0a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0005950 tx=0x7f4fe00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:16.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:15 smithi064 ceph-mon[98059]: pgmap v272: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2023-07-18T03:30:16.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:15 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/937008798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:16.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:15 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/757834397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:16.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:15 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/570654086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:16.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.103+0000 7f4fedffb700 1 -- 172.21.15.40:0/2716789136 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4fe8095ee0 con 0x7f4ff0105860 2023-07-18T03:30:16.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.301+0000 7f4ff686d700 1 -- 172.21.15.40:0/2716789136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ff019e3d0 con 0x7f4ff0105860 2023-07-18T03:30:16.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.302+0000 7f4fedffb700 1 -- 172.21.15.40:0/2716789136 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4fe8099500 con 0x7f4ff0105860 2023-07-18T03:30:16.300 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-07-18T03:30:16.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.304+0000 7f4ff686d700 1 -- 172.21.15.40:0/2716789136 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4fdc0a11e0 msgr2=0x7f4fdc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:16.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.304+0000 7f4ff686d700 1 --2- 172.21.15.40:0/2716789136 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4fdc0a11e0 0x7f4fdc0a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0005950 tx=0x7f4fe00058e0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.304+0000 7f4ff686d700 1 -- 172.21.15.40:0/2716789136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ff0105860 msgr2=0x7f4ff019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:16.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.304+0000 7f4ff686d700 1 --2- 172.21.15.40:0/2716789136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ff0105860 0x7f4ff019cdb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8007ee0 tx=0x7f4fe800d3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.304+0000 7f4ff686d700 1 -- 172.21.15.40:0/2716789136 shutdown_connections 2023-07-18T03:30:16.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.304+0000 7f4ff686d700 1 --2- 172.21.15.40:0/2716789136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ff0105860 0x7f4ff019cdb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.304+0000 7f4ff686d700 1 --2- 172.21.15.40:0/2716789136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4ff0100620 0x7f4ff0195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.304+0000 7f4ff686d700 1 --2- 172.21.15.40:0/2716789136 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4fdc0a11e0 0x7f4fdc0a3690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.304+0000 7f4ff686d700 1 --2- 172.21.15.40:0/2716789136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff0102f40 0x7f4ff0195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.304+0000 7f4ff686d700 1 -- 172.21.15.40:0/2716789136 >> 172.21.15.40:0/2716789136 conn(0x7f4ff00fa000 msgr2=0x7f4ff0101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:16.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.304+0000 7f4ff686d700 1 -- 172.21.15.40:0/2716789136 shutdown_connections 2023-07-18T03:30:16.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.304+0000 7f4ff686d700 1 -- 172.21.15.40:0/2716789136 wait complete. 2023-07-18T03:30:16.314 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:30:16.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.445+0000 7f99cce94700 1 -- 172.21.15.40:0/1766062299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99c8100560 msgr2=0x7f99c8106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:16.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.445+0000 7f99cce94700 1 --2- 172.21.15.40:0/1766062299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99c8100560 0x7f99c8106e20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f99c000b0a0 tx=0x7f99c000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.445+0000 7f99cce94700 1 -- 172.21.15.40:0/1766062299 shutdown_connections 2023-07-18T03:30:16.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.445+0000 7f99cce94700 1 --2- 172.21.15.40:0/1766062299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99c8100560 0x7f99c8106e20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.445+0000 7f99cce94700 1 --2- 172.21.15.40:0/1766062299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c80fe9e0 0x7f99c80fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.445+0000 7f99cce94700 1 --2- 172.21.15.40:0/1766062299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c80ffbd0 0x7f99c8100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.445+0000 7f99cce94700 1 -- 172.21.15.40:0/1766062299 >> 172.21.15.40:0/1766062299 conn(0x7f99c80fa000 msgr2=0x7f99c80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:16.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.445+0000 7f99cce94700 1 -- 172.21.15.40:0/1766062299 shutdown_connections 2023-07-18T03:30:16.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.445+0000 7f99cce94700 1 -- 172.21.15.40:0/1766062299 wait complete. 2023-07-18T03:30:16.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99cce94700 1 Processor -- start 2023-07-18T03:30:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99cce94700 1 -- start start 2023-07-18T03:30:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99cce94700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99c80fe9e0 0x7f99c8195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99cce94700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c80ffbd0 0x7f99c8195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99cce94700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c8100560 0x7f99c819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99cce94700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c819d510 con 0x7f99c80fe9e0 2023-07-18T03:30:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99cce94700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c819d650 con 0x7f99c8100560 2023-07-18T03:30:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99cce94700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c819d7e0 con 0x7f99c80ffbd0 2023-07-18T03:30:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99c659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99c80fe9e0 0x7f99c8195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99c659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99c80fe9e0 0x7f99c8195990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56884/0 (socket says 172.21.15.40:56884) 2023-07-18T03:30:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99c659c700 1 -- 172.21.15.40:0/398530324 learned_addr learned my addr 172.21.15.40:0/398530324 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99c5d9b700 1 --2- 172.21.15.40:0/398530324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c80ffbd0 0x7f99c8195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99c6d9d700 1 --2- 172.21.15.40:0/398530324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c8100560 0x7f99c819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99c659c700 1 -- 172.21.15.40:0/398530324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c80ffbd0 msgr2=0x7f99c8195ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:16.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99c659c700 1 --2- 172.21.15.40:0/398530324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c80ffbd0 0x7f99c8195ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99c659c700 1 -- 172.21.15.40:0/398530324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c8100560 msgr2=0x7f99c819cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99c659c700 1 --2- 172.21.15.40:0/398530324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c8100560 0x7f99c819cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.446+0000 7f99c659c700 1 -- 172.21.15.40:0/398530324 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c0009d00 con 0x7f99c80fe9e0 2023-07-18T03:30:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.447+0000 7f99c659c700 1 --2- 172.21.15.40:0/398530324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99c80fe9e0 0x7f99c8195990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f99b000d980 tx=0x7f99b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.447+0000 7f99bf7fe700 1 -- 172.21.15.40:0/398530324 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b000b950 con 0x7f99c80fe9e0 2023-07-18T03:30:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.447+0000 7f99bf7fe700 1 -- 172.21.15.40:0/398530324 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99b0007620 con 0x7f99c80fe9e0 2023-07-18T03:30:16.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.447+0000 7f99cce94700 1 -- 172.21.15.40:0/398530324 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c819dac0 con 0x7f99c80fe9e0 2023-07-18T03:30:16.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.447+0000 7f99bf7fe700 1 -- 172.21.15.40:0/398530324 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b000f3c0 con 0x7f99c80fe9e0 2023-07-18T03:30:16.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.447+0000 7f99cce94700 1 -- 172.21.15.40:0/398530324 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c819dfd0 con 0x7f99c80fe9e0 2023-07-18T03:30:16.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.448+0000 7f99cce94700 1 -- 172.21.15.40:0/398530324 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c8105890 con 0x7f99c80fe9e0 2023-07-18T03:30:16.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.449+0000 7f99bf7fe700 1 -- 172.21.15.40:0/398530324 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f99b0010640 con 0x7f99c80fe9e0 2023-07-18T03:30:16.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.449+0000 7f99bf7fe700 1 --2- 172.21.15.40:0/398530324 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f99b40a11e0 0x7f99b40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:16.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.449+0000 7f99c5d9b700 1 --2- 172.21.15.40:0/398530324 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f99b40a11e0 0x7f99b40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:16.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.450+0000 7f99bf7fe700 1 -- 172.21.15.40:0/398530324 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f99b00cc7d0 con 0x7f99c80fe9e0 2023-07-18T03:30:16.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.450+0000 7f99c5d9b700 1 --2- 172.21.15.40:0/398530324 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f99b40a11e0 0x7f99b40a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009be0 tx=0x7f99b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:16.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.453+0000 7f99bf7fe700 1 -- 172.21.15.40:0/398530324 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f99b0095f70 con 0x7f99c80fe9e0 2023-07-18T03:30:16.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.652+0000 7f99cce94700 1 -- 172.21.15.40:0/398530324 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f99c802c850 con 0x7f99c80fe9e0 2023-07-18T03:30:16.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.653+0000 7f99bf7fe700 1 -- 172.21.15.40:0/398530324 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f99b0099590 con 0x7f99c80fe9e0 2023-07-18T03:30:16.651 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-07-18T03:30:16.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.655+0000 7f99cce94700 1 -- 172.21.15.40:0/398530324 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f99b40a11e0 msgr2=0x7f99b40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:16.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.655+0000 7f99cce94700 1 --2- 172.21.15.40:0/398530324 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f99b40a11e0 0x7f99b40a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009be0 tx=0x7f99b8009260 comp rx=0 tx=0).stop 2023-07-18T03:30:16.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.655+0000 7f99cce94700 1 -- 172.21.15.40:0/398530324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99c80fe9e0 msgr2=0x7f99c8195990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:16.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.655+0000 7f99cce94700 1 --2- 172.21.15.40:0/398530324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99c80fe9e0 0x7f99c8195990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f99b000d980 tx=0x7f99b000dc90 comp rx=0 tx=0).stop 2023-07-18T03:30:16.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.655+0000 7f99cce94700 1 -- 172.21.15.40:0/398530324 shutdown_connections 2023-07-18T03:30:16.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.655+0000 7f99cce94700 1 --2- 172.21.15.40:0/398530324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99c8100560 0x7f99c819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.655+0000 7f99cce94700 1 --2- 172.21.15.40:0/398530324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99c80fe9e0 0x7f99c8195990 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.655+0000 7f99cce94700 1 --2- 172.21.15.40:0/398530324 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f99b40a11e0 0x7f99b40a3690 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.655+0000 7f99cce94700 1 --2- 172.21.15.40:0/398530324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99c80ffbd0 0x7f99c8195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.655+0000 7f99cce94700 1 -- 172.21.15.40:0/398530324 >> 172.21.15.40:0/398530324 conn(0x7f99c80fa000 msgr2=0x7f99c80fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:16.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.655+0000 7f99cce94700 1 -- 172.21.15.40:0/398530324 shutdown_connections 2023-07-18T03:30:16.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.655+0000 7f99cce94700 1 -- 172.21.15.40:0/398530324 wait complete. 2023-07-18T03:30:16.665 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:30:16.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.795+0000 7fd9c8b53700 1 -- 172.21.15.40:0/149038353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c4100550 msgr2=0x7fd9c4106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.795+0000 7fd9c8b53700 1 --2- 172.21.15.40:0/149038353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c4100550 0x7fd9c4106e10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00b0a0 tx=0x7fd9bc00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.795+0000 7fd9c8b53700 1 -- 172.21.15.40:0/149038353 shutdown_connections 2023-07-18T03:30:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.795+0000 7fd9c8b53700 1 --2- 172.21.15.40:0/149038353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c4100550 0x7fd9c4106e10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.795+0000 7fd9c8b53700 1 --2- 172.21.15.40:0/149038353 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9c40fe9d0 0x7fd9c40fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.795+0000 7fd9c8b53700 1 --2- 172.21.15.40:0/149038353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9c40ffbc0 0x7fd9c4100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.795+0000 7fd9c8b53700 1 -- 172.21.15.40:0/149038353 >> 172.21.15.40:0/149038353 conn(0x7fd9c40f9fd0 msgr2=0x7fd9c40fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:16.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.795+0000 7fd9c8b53700 1 -- 172.21.15.40:0/149038353 shutdown_connections 2023-07-18T03:30:16.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.795+0000 7fd9c8b53700 1 -- 172.21.15.40:0/149038353 wait complete. 2023-07-18T03:30:16.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c8b53700 1 Processor -- start 2023-07-18T03:30:16.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c8b53700 1 -- start start 2023-07-18T03:30:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c8b53700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9c40fe9d0 0x7fd9c4194780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c8b53700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c40ffbc0 0x7fd9c4194cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c8b53700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9c4100550 0x7fd9c419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c8b53700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c419c300 con 0x7fd9c40ffbc0 2023-07-18T03:30:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c8b53700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c419c440 con 0x7fd9c40fe9d0 2023-07-18T03:30:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c8b53700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c419c5d0 con 0x7fd9c4100550 2023-07-18T03:30:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c1d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c40ffbc0 0x7fd9c4194cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c1d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c40ffbc0 0x7fd9c4194cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56900/0 (socket says 172.21.15.40:56900) 2023-07-18T03:30:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c1d9b700 1 -- 172.21.15.40:0/4160823012 learned_addr learned my addr 172.21.15.40:0/4160823012 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c2d9d700 1 --2- 172.21.15.40:0/4160823012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9c4100550 0x7fd9c419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c259c700 1 --2- 172.21.15.40:0/4160823012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9c40fe9d0 0x7fd9c4194780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c1d9b700 1 -- 172.21.15.40:0/4160823012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9c4100550 msgr2=0x7fd9c419bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c1d9b700 1 --2- 172.21.15.40:0/4160823012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9c4100550 0x7fd9c419bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c1d9b700 1 -- 172.21.15.40:0/4160823012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9c40fe9d0 msgr2=0x7fd9c4194780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c1d9b700 1 --2- 172.21.15.40:0/4160823012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9c40fe9d0 0x7fd9c4194780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c1d9b700 1 -- 172.21.15.40:0/4160823012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc009d00 con 0x7fd9c40ffbc0 2023-07-18T03:30:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.796+0000 7fd9c1d9b700 1 --2- 172.21.15.40:0/4160823012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c40ffbc0 0x7fd9c4194cc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400d980 tx=0x7fd9b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.797+0000 7fd9bb7fe700 1 -- 172.21.15.40:0/4160823012 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4004220 con 0x7fd9c40ffbc0 2023-07-18T03:30:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.797+0000 7fd9c8b53700 1 -- 172.21.15.40:0/4160823012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c419c8b0 con 0x7fd9c40ffbc0 2023-07-18T03:30:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.797+0000 7fd9bb7fe700 1 -- 172.21.15.40:0/4160823012 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9b4007640 con 0x7fd9c40ffbc0 2023-07-18T03:30:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.797+0000 7fd9bb7fe700 1 -- 172.21.15.40:0/4160823012 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4005610 con 0x7fd9c40ffbc0 2023-07-18T03:30:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.797+0000 7fd9c8b53700 1 -- 172.21.15.40:0/4160823012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c419cdc0 con 0x7fd9c40ffbc0 2023-07-18T03:30:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.798+0000 7fd9c8b53700 1 -- 172.21.15.40:0/4160823012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c410a1b0 con 0x7fd9c40ffbc0 2023-07-18T03:30:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.799+0000 7fd9bb7fe700 1 -- 172.21.15.40:0/4160823012 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd9b4004750 con 0x7fd9c40ffbc0 2023-07-18T03:30:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.799+0000 7fd9bb7fe700 1 --2- 172.21.15.40:0/4160823012 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd9b00a1220 0x7fd9b00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:16.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.799+0000 7fd9bb7fe700 1 -- 172.21.15.40:0/4160823012 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd9b40ccc50 con 0x7fd9c40ffbc0 2023-07-18T03:30:16.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.799+0000 7fd9c259c700 1 --2- 172.21.15.40:0/4160823012 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd9b00a1220 0x7fd9b00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:16.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.800+0000 7fd9c259c700 1 --2- 172.21.15.40:0/4160823012 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd9b00a1220 0x7fd9b00a36d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac005fd0 tx=0x7fd9ac005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:16.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:16.803+0000 7fd9bb7fe700 1 -- 172.21.15.40:0/4160823012 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd9b40963f0 con 0x7fd9c40ffbc0 2023-07-18T03:30:17.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.046+0000 7fd9c8b53700 1 -- 172.21.15.40:0/4160823012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9c4105850 con 0x7fd9c40ffbc0 2023-07-18T03:30:17.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.047+0000 7fd9bb7fe700 1 -- 172.21.15.40:0/4160823012 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10888 (secure 0 0 0) 0x7fd9b4016090 con 0x7fd9c40ffbc0 2023-07-18T03:30:17.044 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:17.045 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:29:26.718789+0000","modified":"2023-07-18T03:29:34.650353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24523,"mds_2":24482,"mds_3":24452,"mds_4":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14601":{"gid":14601,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1818167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1818167554},{"type":"v1","addr":"172.21.15.40:6839","nonce":1818167554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14613":{"gid":14613,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/3953074960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":3953074960},{"type":"v1","addr":"172.21.15.40:6841","nonce":3953074960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3509655945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3509655945},{"type":"v1","addr":"172.21.15.134:6837","nonce":3509655945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/3007039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3007039043},{"type":"v1","addr":"172.21.15.64:6839","nonce":3007039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:30:17.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.049+0000 7fd9c8b53700 1 -- 172.21.15.40:0/4160823012 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd9b00a1220 msgr2=0x7fd9b00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.049+0000 7fd9c8b53700 1 --2- 172.21.15.40:0/4160823012 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd9b00a1220 0x7fd9b00a36d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac005fd0 tx=0x7fd9ac005dd0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.049+0000 7fd9c8b53700 1 -- 172.21.15.40:0/4160823012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c40ffbc0 msgr2=0x7fd9c4194cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.049+0000 7fd9c8b53700 1 --2- 172.21.15.40:0/4160823012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c40ffbc0 0x7fd9c4194cc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400d980 tx=0x7fd9b400dc90 comp rx=0 tx=0).stop 2023-07-18T03:30:17.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.049+0000 7fd9c8b53700 1 -- 172.21.15.40:0/4160823012 shutdown_connections 2023-07-18T03:30:17.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.049+0000 7fd9c8b53700 1 --2- 172.21.15.40:0/4160823012 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd9b00a1220 0x7fd9b00a36d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.049+0000 7fd9c8b53700 1 --2- 172.21.15.40:0/4160823012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9c4100550 0x7fd9c419bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.049+0000 7fd9c8b53700 1 --2- 172.21.15.40:0/4160823012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9c40fe9d0 0x7fd9c4194780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.049+0000 7fd9c8b53700 1 --2- 172.21.15.40:0/4160823012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9c40ffbc0 0x7fd9c4194cc0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.049+0000 7fd9c8b53700 1 -- 172.21.15.40:0/4160823012 >> 172.21.15.40:0/4160823012 conn(0x7fd9c40f9fd0 msgr2=0x7fd9c40fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:17.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.049+0000 7fd9c8b53700 1 -- 172.21.15.40:0/4160823012 shutdown_connections 2023-07-18T03:30:17.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.049+0000 7fd9c8b53700 1 -- 172.21.15.40:0/4160823012 wait complete. 2023-07-18T03:30:17.048 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 27 2023-07-18T03:30:17.058 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:16 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/4068415097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:17.058 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:16 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2716789136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:17.058 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:16 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/398530324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:17.059 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:30:17.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:16 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/4068415097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:17.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:16 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2716789136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:17.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:16 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/398530324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:17.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:16 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/4068415097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:17.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:16 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2716789136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:17.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:16 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/398530324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.188+0000 7f726474e700 1 -- 172.21.15.40:0/3958375907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c0ffb00 msgr2=0x7f725c0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.188+0000 7f726474e700 1 --2- 172.21.15.40:0/3958375907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c0ffb00 0x7f725c0fff10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7250009990 tx=0x7f7250009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.188+0000 7f726474e700 1 -- 172.21.15.40:0/3958375907 shutdown_connections 2023-07-18T03:30:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.188+0000 7f726474e700 1 --2- 172.21.15.40:0/3958375907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f725c101680 0x7f725c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.188+0000 7f726474e700 1 --2- 172.21.15.40:0/3958375907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c0ffb00 0x7f725c0fff10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.188+0000 7f726474e700 1 --2- 172.21.15.40:0/3958375907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f725c100cf0 0x7f725c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.188+0000 7f726474e700 1 -- 172.21.15.40:0/3958375907 >> 172.21.15.40:0/3958375907 conn(0x7f725c0fb0e0 msgr2=0x7f725c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.188+0000 7f726474e700 1 -- 172.21.15.40:0/3958375907 shutdown_connections 2023-07-18T03:30:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.188+0000 7f726474e700 1 -- 172.21.15.40:0/3958375907 wait complete. 2023-07-18T03:30:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.188+0000 7f726474e700 1 Processor -- start 2023-07-18T03:30:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f726474e700 1 -- start start 2023-07-18T03:30:17.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f726474e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f725c0ffb00 0x7f725c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f726474e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f725c100cf0 0x7f725c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f726474e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c101680 0x7f725c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f726474e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c19f540 con 0x7f725c101680 2023-07-18T03:30:17.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f726474e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c19f680 con 0x7f725c0ffb00 2023-07-18T03:30:17.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f726474e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c19f810 con 0x7f725c100cf0 2023-07-18T03:30:17.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f7262ceb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c101680 0x7f725c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f7262ceb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c101680 0x7f725c19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56916/0 (socket says 172.21.15.40:56916) 2023-07-18T03:30:17.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f7262ceb700 1 -- 172.21.15.40:0/2719000639 learned_addr learned my addr 172.21.15.40:0/2719000639 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:17.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f72624ea700 1 --2- 172.21.15.40:0/2719000639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f725c0ffb00 0x7f725c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f7261ce9700 1 --2- 172.21.15.40:0/2719000639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f725c100cf0 0x7f725c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f7262ceb700 1 -- 172.21.15.40:0/2719000639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f725c100cf0 msgr2=0x7f725c197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f7262ceb700 1 --2- 172.21.15.40:0/2719000639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f725c100cf0 0x7f725c197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f7262ceb700 1 -- 172.21.15.40:0/2719000639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f725c0ffb00 msgr2=0x7f725c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f7262ceb700 1 --2- 172.21.15.40:0/2719000639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f725c0ffb00 0x7f725c1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f7262ceb700 1 -- 172.21.15.40:0/2719000639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72500095f0 con 0x7f725c101680 2023-07-18T03:30:17.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f7262ceb700 1 --2- 172.21.15.40:0/2719000639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c101680 0x7f725c19ef70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7254007ee0 tx=0x7f725400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:17.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.189+0000 7f724f7fe700 1 -- 172.21.15.40:0/2719000639 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254017070 con 0x7f725c101680 2023-07-18T03:30:17.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.190+0000 7f726474e700 1 -- 172.21.15.40:0/2719000639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725c19faf0 con 0x7f725c101680 2023-07-18T03:30:17.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.190+0000 7f724f7fe700 1 -- 172.21.15.40:0/2719000639 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f725400f040 con 0x7f725c101680 2023-07-18T03:30:17.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.190+0000 7f724f7fe700 1 -- 172.21.15.40:0/2719000639 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254005740 con 0x7f725c101680 2023-07-18T03:30:17.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.190+0000 7f726474e700 1 -- 172.21.15.40:0/2719000639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c19ffb0 con 0x7f725c101680 2023-07-18T03:30:17.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.191+0000 7f726474e700 1 -- 172.21.15.40:0/2719000639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725c10b320 con 0x7f725c101680 2023-07-18T03:30:17.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.192+0000 7f724f7fe700 1 -- 172.21.15.40:0/2719000639 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f725400dc60 con 0x7f725c101680 2023-07-18T03:30:17.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.192+0000 7f724f7fe700 1 --2- 172.21.15.40:0/2719000639 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72480a1220 0x7f72480a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.193+0000 7f724f7fe700 1 -- 172.21.15.40:0/2719000639 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f72540cbb00 con 0x7f725c101680 2023-07-18T03:30:17.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.193+0000 7f72624ea700 1 --2- 172.21.15.40:0/2719000639 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72480a1220 0x7f72480a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.193+0000 7f72624ea700 1 --2- 172.21.15.40:0/2719000639 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72480a1220 0x7f72480a36d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f725000b5c0 tx=0x7f7250005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:17.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.197+0000 7f724f7fe700 1 -- 172.21.15.40:0/2719000639 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f72540952a0 con 0x7f725c101680 2023-07-18T03:30:17.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.398+0000 7f726474e700 1 -- 172.21.15.40:0/2719000639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f725c1a0560 con 0x7f725c101680 2023-07-18T03:30:17.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.399+0000 7f724f7fe700 1 -- 172.21.15.40:0/2719000639 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22071 (secure 0 0 0) 0x7f72540988c0 con 0x7f725c101680 2023-07-18T03:30:17.397 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:17.397 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":81,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:29:28.081282+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:30:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.401+0000 7f726474e700 1 -- 172.21.15.40:0/2719000639 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72480a1220 msgr2=0x7f72480a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.401+0000 7f726474e700 1 --2- 172.21.15.40:0/2719000639 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72480a1220 0x7f72480a36d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f725000b5c0 tx=0x7f7250005fb0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.401+0000 7f726474e700 1 -- 172.21.15.40:0/2719000639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c101680 msgr2=0x7f725c19ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.401+0000 7f726474e700 1 --2- 172.21.15.40:0/2719000639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c101680 0x7f725c19ef70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7254007ee0 tx=0x7f725400d3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.401+0000 7f726474e700 1 -- 172.21.15.40:0/2719000639 shutdown_connections 2023-07-18T03:30:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.401+0000 7f726474e700 1 --2- 172.21.15.40:0/2719000639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f725c101680 0x7f725c19ef70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.401+0000 7f726474e700 1 --2- 172.21.15.40:0/2719000639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f725c0ffb00 0x7f725c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.401+0000 7f726474e700 1 --2- 172.21.15.40:0/2719000639 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72480a1220 0x7f72480a36d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.401+0000 7f726474e700 1 --2- 172.21.15.40:0/2719000639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f725c100cf0 0x7f725c197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.401+0000 7f726474e700 1 -- 172.21.15.40:0/2719000639 >> 172.21.15.40:0/2719000639 conn(0x7f725c0fb0e0 msgr2=0x7f725c1064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:17.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.401+0000 7f726474e700 1 -- 172.21.15.40:0/2719000639 shutdown_connections 2023-07-18T03:30:17.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.401+0000 7f726474e700 1 -- 172.21.15.40:0/2719000639 wait complete. 2023-07-18T03:30:17.411 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:30:17.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.539+0000 7f54004f7700 1 -- 172.21.15.40:0/2641818285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f8100590 msgr2=0x7f53f8106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.539+0000 7f54004f7700 1 --2- 172.21.15.40:0/2641818285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f8100590 0x7f53f8106e50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f53f000b0a0 tx=0x7f53f000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f54004f7700 1 -- 172.21.15.40:0/2641818285 shutdown_connections 2023-07-18T03:30:17.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f54004f7700 1 --2- 172.21.15.40:0/2641818285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f8100590 0x7f53f8106e50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f54004f7700 1 --2- 172.21.15.40:0/2641818285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53f80fea10 0x7f53f80fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f54004f7700 1 --2- 172.21.15.40:0/2641818285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53f80ffc00 0x7f53f8100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f54004f7700 1 -- 172.21.15.40:0/2641818285 >> 172.21.15.40:0/2641818285 conn(0x7f53f80fa030 msgr2=0x7f53f80fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:17.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f54004f7700 1 -- 172.21.15.40:0/2641818285 shutdown_connections 2023-07-18T03:30:17.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f54004f7700 1 -- 172.21.15.40:0/2641818285 wait complete. 2023-07-18T03:30:17.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f54004f7700 1 Processor -- start 2023-07-18T03:30:17.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f54004f7700 1 -- start start 2023-07-18T03:30:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f54004f7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80fea10 0x7f53f81029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f54004f7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53f80ffc00 0x7f53f8100fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f54004f7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53f8100590 0x7f53f8101520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f54004f7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f8101cc0 con 0x7f53f80fea10 2023-07-18T03:30:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f54004f7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f8101e30 con 0x7f53f8100590 2023-07-18T03:30:17.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f54004f7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f819f840 con 0x7f53f80ffc00 2023-07-18T03:30:17.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f53fe293700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80fea10 0x7f53f81029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f53fe293700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80fea10 0x7f53f81029a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56926/0 (socket says 172.21.15.40:56926) 2023-07-18T03:30:17.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.540+0000 7f53fe293700 1 -- 172.21.15.40:0/2646017032 learned_addr learned my addr 172.21.15.40:0/2646017032 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:17.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.541+0000 7f53fea94700 1 --2- 172.21.15.40:0/2646017032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53f8100590 0x7f53f8101520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.541+0000 7f53fda92700 1 --2- 172.21.15.40:0/2646017032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53f80ffc00 0x7f53f8100fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.541+0000 7f53fe293700 1 -- 172.21.15.40:0/2646017032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53f80ffc00 msgr2=0x7f53f8100fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.541+0000 7f53fe293700 1 --2- 172.21.15.40:0/2646017032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53f80ffc00 0x7f53f8100fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.541+0000 7f53fe293700 1 -- 172.21.15.40:0/2646017032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53f8100590 msgr2=0x7f53f8101520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.541+0000 7f53fe293700 1 --2- 172.21.15.40:0/2646017032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53f8100590 0x7f53f8101520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.541+0000 7f53fe293700 1 -- 172.21.15.40:0/2646017032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f0009d00 con 0x7f53f80fea10 2023-07-18T03:30:17.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.541+0000 7f53fda92700 1 --2- 172.21.15.40:0/2646017032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53f80ffc00 0x7f53f8100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:17.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.541+0000 7f53fe293700 1 --2- 172.21.15.40:0/2646017032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80fea10 0x7f53f81029a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f53f400c990 tx=0x7f53f400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:17.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.541+0000 7f53ef7fe700 1 -- 172.21.15.40:0/2646017032 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4007b00 con 0x7f53f80fea10 2023-07-18T03:30:17.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.541+0000 7f53ef7fe700 1 -- 172.21.15.40:0/2646017032 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53f4007c60 con 0x7f53f80fea10 2023-07-18T03:30:17.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.541+0000 7f54004f7700 1 -- 172.21.15.40:0/2646017032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f819fb20 con 0x7f53f80fea10 2023-07-18T03:30:17.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.541+0000 7f53ef7fe700 1 -- 172.21.15.40:0/2646017032 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f4018660 con 0x7f53f80fea10 2023-07-18T03:30:17.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.541+0000 7f54004f7700 1 -- 172.21.15.40:0/2646017032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f81a0080 con 0x7f53f80fea10 2023-07-18T03:30:17.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.542+0000 7f54004f7700 1 -- 172.21.15.40:0/2646017032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f810a1f0 con 0x7f53f80fea10 2023-07-18T03:30:17.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.543+0000 7f53ef7fe700 1 -- 172.21.15.40:0/2646017032 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f53f400f450 con 0x7f53f80fea10 2023-07-18T03:30:17.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.544+0000 7f53ef7fe700 1 --2- 172.21.15.40:0/2646017032 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f53e40a12a0 0x7f53e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.544+0000 7f53ef7fe700 1 -- 172.21.15.40:0/2646017032 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f53f40cc800 con 0x7f53f80fea10 2023-07-18T03:30:17.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.544+0000 7f53fda92700 1 --2- 172.21.15.40:0/2646017032 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f53e40a12a0 0x7f53e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.544+0000 7f53fda92700 1 --2- 172.21.15.40:0/2646017032 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f53e40a12a0 0x7f53e40a3750 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f53e8005950 tx=0x7f53e80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:17.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.548+0000 7f53ef7fe700 1 -- 172.21.15.40:0/2646017032 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f53f4095fa0 con 0x7f53f80fea10 2023-07-18T03:30:17.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.793+0000 7f54004f7700 1 -- 172.21.15.40:0/2646017032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53f810a410 con 0x7f53f80fea10 2023-07-18T03:30:17.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.794+0000 7f53ef7fe700 1 -- 172.21.15.40:0/2646017032 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10888 (secure 0 0 0) 0x7f53f40995c0 con 0x7f53f80fea10 2023-07-18T03:30:17.792 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:17.792 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:29:26.718789+0000","modified":"2023-07-18T03:29:34.650353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24523,"mds_2":24482,"mds_3":24452,"mds_4":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14601":{"gid":14601,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1818167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1818167554},{"type":"v1","addr":"172.21.15.40:6839","nonce":1818167554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14613":{"gid":14613,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/3953074960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":3953074960},{"type":"v1","addr":"172.21.15.40:6841","nonce":3953074960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3509655945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3509655945},{"type":"v1","addr":"172.21.15.134:6837","nonce":3509655945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/3007039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3007039043},{"type":"v1","addr":"172.21.15.64:6839","nonce":3007039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:30:17.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.796+0000 7f54004f7700 1 -- 172.21.15.40:0/2646017032 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f53e40a12a0 msgr2=0x7f53e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.796+0000 7f54004f7700 1 --2- 172.21.15.40:0/2646017032 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f53e40a12a0 0x7f53e40a3750 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f53e8005950 tx=0x7f53e80058e0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.796+0000 7f54004f7700 1 -- 172.21.15.40:0/2646017032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80fea10 msgr2=0x7f53f81029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.796+0000 7f54004f7700 1 --2- 172.21.15.40:0/2646017032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80fea10 0x7f53f81029a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f53f400c990 tx=0x7f53f400cca0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.796+0000 7f54004f7700 1 -- 172.21.15.40:0/2646017032 shutdown_connections 2023-07-18T03:30:17.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.796+0000 7f54004f7700 1 --2- 172.21.15.40:0/2646017032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f53f8100590 0x7f53f8101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.796+0000 7f54004f7700 1 --2- 172.21.15.40:0/2646017032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53f80fea10 0x7f53f81029a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.796+0000 7f54004f7700 1 --2- 172.21.15.40:0/2646017032 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f53e40a12a0 0x7f53e40a3750 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.796+0000 7f54004f7700 1 --2- 172.21.15.40:0/2646017032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53f80ffc00 0x7f53f8100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.796+0000 7f54004f7700 1 -- 172.21.15.40:0/2646017032 >> 172.21.15.40:0/2646017032 conn(0x7f53f80fa030 msgr2=0x7f53f80fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:17.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.796+0000 7f54004f7700 1 -- 172.21.15.40:0/2646017032 shutdown_connections 2023-07-18T03:30:17.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.796+0000 7f54004f7700 1 -- 172.21.15.40:0/2646017032 wait complete. 2023-07-18T03:30:17.796 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 27 2023-07-18T03:30:17.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-18T03:29:26.718789+0000', 'modified': '2023-07-18T03:29:34.650353+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24527, 'mds_1': 24523, 'mds_2': 24482, 'mds_3': 24452, 'mds_4': 24442}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14538': {'gid': 14538, 'name': 'd', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6837/3377503144', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6836', 'nonce': 3377503144}, {'type': 'v1', 'addr': '172.21.15.40:6837', 'nonce': 3377503144}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14601': {'gid': 14601, 'name': 'g', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6839/1818167554', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6838', 'nonce': 1818167554}, {'type': 'v1', 'addr': '172.21.15.40:6839', 'nonce': 1818167554}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14613': {'gid': 14613, 'name': 'a', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6841/3953074960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6840', 'nonce': 3953074960}, {'type': 'v1', 'addr': '172.21.15.40:6841', 'nonce': 3953074960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24442': {'gid': 24442, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.134:6833/2345264306', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 2345264306}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 2345264306}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.64:6835/1864495009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 1864495009}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 1864495009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.64:6837/3077792296', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6836', 'nonce': 3077792296}, {'type': 'v1', 'addr': '172.21.15.64:6837', 'nonce': 3077792296}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/2946854745', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 2946854745}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 2946854745}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6837/3509655945', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 3509655945}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 3509655945}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24523': {'gid': 24523, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.134:6839/3192254101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 3192254101}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 3192254101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24527': {'gid': 24527, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6839/3007039043', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6838', 'nonce': 3007039043}, {'type': 'v1', 'addr': '172.21.15.64:6839', 'nonce': 3007039043}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-18T03:30:17.807 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-07-18T03:30:17.807 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2023-07-18T03:30:17.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.936+0000 7f0792414700 1 -- 172.21.15.40:0/1317022186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f078c0fed20 msgr2=0x7f078c0ff190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.936+0000 7f0792414700 1 --2- 172.21.15.40:0/1317022186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f078c0fed20 0x7f078c0ff190 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0774009990 tx=0x7f0774009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.936+0000 7f0792414700 1 -- 172.21.15.40:0/1317022186 shutdown_connections 2023-07-18T03:30:17.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.936+0000 7f0792414700 1 --2- 172.21.15.40:0/1317022186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f078c0ff6d0 0x7f078c0ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.936+0000 7f0792414700 1 --2- 172.21.15.40:0/1317022186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f078c0fe3d0 0x7f078c0fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.936+0000 7f0792414700 1 --2- 172.21.15.40:0/1317022186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f078c0fed20 0x7f078c0ff190 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.936+0000 7f0792414700 1 -- 172.21.15.40:0/1317022186 >> 172.21.15.40:0/1317022186 conn(0x7f078c0fa000 msgr2=0x7f078c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:17.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.936+0000 7f0792414700 1 -- 172.21.15.40:0/1317022186 shutdown_connections 2023-07-18T03:30:17.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.936+0000 7f0792414700 1 -- 172.21.15.40:0/1317022186 wait complete. 2023-07-18T03:30:17.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.937+0000 7f0792414700 1 Processor -- start 2023-07-18T03:30:17.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.937+0000 7f0792414700 1 -- start start 2023-07-18T03:30:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.937+0000 7f0792414700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f078c0fe3d0 0x7f078c197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.937+0000 7f0792414700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f078c0fed20 0x7f078c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.937+0000 7f0792414700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f078c0ff6d0 0x7f078c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.937+0000 7f0792414700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c19f590 con 0x7f078c0fed20 2023-07-18T03:30:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.937+0000 7f0792414700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c19f6d0 con 0x7f078c0fe3d0 2023-07-18T03:30:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.937+0000 7f0792414700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c19f860 con 0x7f078c0ff6d0 2023-07-18T03:30:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.937+0000 7f078b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f078c0fed20 0x7f078c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.937+0000 7f078b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f078c0fed20 0x7f078c197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56942/0 (socket says 172.21.15.40:56942) 2023-07-18T03:30:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.937+0000 7f078b7fe700 1 -- 172.21.15.40:0/3833937953 learned_addr learned my addr 172.21.15.40:0/3833937953 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.937+0000 7f07909b1700 1 --2- 172.21.15.40:0/3833937953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f078c0ff6d0 0x7f078c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.937+0000 7f078bfff700 1 --2- 172.21.15.40:0/3833937953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f078c0fe3d0 0x7f078c197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.938+0000 7f078b7fe700 1 -- 172.21.15.40:0/3833937953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f078c0ff6d0 msgr2=0x7f078c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.938+0000 7f078b7fe700 1 --2- 172.21.15.40:0/3833937953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f078c0ff6d0 0x7f078c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.938+0000 7f078b7fe700 1 -- 172.21.15.40:0/3833937953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f078c0fe3d0 msgr2=0x7f078c197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.938+0000 7f078b7fe700 1 --2- 172.21.15.40:0/3833937953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f078c0fe3d0 0x7f078c197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.938+0000 7f078b7fe700 1 -- 172.21.15.40:0/3833937953 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07740095f0 con 0x7f078c0fed20 2023-07-18T03:30:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.938+0000 7f078b7fe700 1 --2- 172.21.15.40:0/3833937953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f078c0fed20 0x7f078c197f50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0774000c00 tx=0x7f07740047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.938+0000 7f07897fa700 1 -- 172.21.15.40:0/3833937953 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077401d070 con 0x7f078c0fed20 2023-07-18T03:30:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.938+0000 7f07897fa700 1 -- 172.21.15.40:0/3833937953 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0774022470 con 0x7f078c0fed20 2023-07-18T03:30:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.938+0000 7f0792414700 1 -- 172.21.15.40:0/3833937953 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c19fae0 con 0x7f078c0fed20 2023-07-18T03:30:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.938+0000 7f07897fa700 1 -- 172.21.15.40:0/3833937953 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774017650 con 0x7f078c0fed20 2023-07-18T03:30:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.938+0000 7f0792414700 1 -- 172.21.15.40:0/3833937953 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078c19ff00 con 0x7f078c0fed20 2023-07-18T03:30:17.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.940+0000 7f07897fa700 1 -- 172.21.15.40:0/3833937953 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f07740225e0 con 0x7f078c0fed20 2023-07-18T03:30:17.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.941+0000 7f07897fa700 1 --2- 172.21.15.40:0/3833937953 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f07780a11e0 0x7f07780a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.941+0000 7f078bfff700 1 --2- 172.21.15.40:0/3833937953 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f07780a11e0 0x7f07780a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.941+0000 7f07897fa700 1 -- 172.21.15.40:0/3833937953 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f07740cdf90 con 0x7f078c0fed20 2023-07-18T03:30:17.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.941+0000 7f078bfff700 1 --2- 172.21.15.40:0/3833937953 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f07780a11e0 0x7f07780a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f077c005950 tx=0x7f077c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f079e59ab80 1 Processor -- start 2023-07-18T03:30:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f079e59ab80 1 -- start start 2023-07-18T03:30:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f079e59ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124480 0x555803124890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f079e59ab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124dd0 0x555803125690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f079e59ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803125c20 0x555803126090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f079e59ab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555803029c60 con 0x555803124dd0 2023-07-18T03:30:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f079e59ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55580307c9f0 con 0x555803124480 2023-07-18T03:30:17.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f079e59ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555803075e80 con 0x555803125c20 2023-07-18T03:30:17.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f078b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124dd0 0x555803125690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f078b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124dd0 0x555803125690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56950/0 (socket says 172.21.15.40:56950) 2023-07-18T03:30:17.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f078b7fe700 1 -- 172.21.15.40:0/56122340 learned_addr learned my addr 172.21.15.40:0/56122340 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:17.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f078bfff700 1 --2- 172.21.15.40:0/56122340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124480 0x555803124890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f07909b1700 1 --2- 172.21.15.40:0/56122340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803125c20 0x555803126090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f078b7fe700 1 -- 172.21.15.40:0/56122340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803125c20 msgr2=0x555803126090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:17.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f078b7fe700 1 --2- 172.21.15.40:0/56122340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803125c20 0x555803126090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f078b7fe700 1 -- 172.21.15.40:0/56122340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124480 msgr2=0x555803124890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:17.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f078b7fe700 1 --2- 172.21.15.40:0/56122340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124480 0x555803124890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f078b7fe700 1 -- 172.21.15.40:0/56122340 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555802ea3d90 con 0x555803124dd0 2023-07-18T03:30:17.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f078bfff700 1 --2- 172.21.15.40:0/56122340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124480 0x555803124890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:17.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.942+0000 7f07909b1700 1 --2- 172.21.15.40:0/56122340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803125c20 0x555803126090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:17.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.943+0000 7f078b7fe700 1 --2- 172.21.15.40:0/56122340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124dd0 0x555803125690 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f077409b080 tx=0x7f0774009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeac5c2330fb4b7a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:17.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.943+0000 7f07827fc700 1 -- 172.21.15.40:0/56122340 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07740a3740 con 0x555803124dd0 2023-07-18T03:30:17.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.943+0000 7f07827fc700 1 -- 172.21.15.40:0/56122340 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0774099720 con 0x555803124dd0 2023-07-18T03:30:17.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.943+0000 7f07827fc700 1 -- 172.21.15.40:0/56122340 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077400fcf0 con 0x555803124dd0 2023-07-18T03:30:17.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.943+0000 7f079e59ab80 1 -- 172.21.15.40:0/56122340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124dd0 msgr2=0x555803125690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.943+0000 7f079e59ab80 1 --2- 172.21.15.40:0/56122340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124dd0 0x555803125690 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f077409b080 tx=0x7f0774009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.943+0000 7f079e59ab80 1 -- 172.21.15.40:0/56122340 shutdown_connections 2023-07-18T03:30:17.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.943+0000 7f079e59ab80 1 --2- 172.21.15.40:0/56122340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803125c20 0x555803126090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.943+0000 7f079e59ab80 1 --2- 172.21.15.40:0/56122340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124480 0x555803124890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.943+0000 7f079e59ab80 1 --2- 172.21.15.40:0/56122340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124dd0 0x555803125690 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.943+0000 7f079e59ab80 1 -- 172.21.15.40:0/56122340 >> 172.21.15.40:0/56122340 conn(0x555803119d50 msgr2=0x55580311f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:17.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.943+0000 7f079e59ab80 1 -- 172.21.15.40:0/56122340 shutdown_connections 2023-07-18T03:30:17.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.943+0000 7f079e59ab80 1 -- 172.21.15.40:0/56122340 wait complete. 2023-07-18T03:30:17.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.944+0000 7f079e59ab80 1 -- start start 2023-07-18T03:30:17.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.944+0000 7f079e59ab80 1 Processor -- start 2023-07-18T03:30:17.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.944+0000 7f079e59ab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124480 0x555803136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.944+0000 7f079e59ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803124dd0 0x555803136950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.944+0000 7f079e59ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803125c20 0x55580313d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.944+0000 7f079e59ab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555803015a70 con 0x555803124480 2023-07-18T03:30:17.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.944+0000 7f079e59ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555803015770 con 0x555803125c20 2023-07-18T03:30:17.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.944+0000 7f079e59ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555802ef98d0 con 0x555803124dd0 2023-07-18T03:30:17.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.944+0000 7f079e59ab80 10 client.0 _ll_register_callbacks cb 0x555803013710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:17.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.944+0000 7f078bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124480 0x555803136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.944+0000 7f078bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124480 0x555803136410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56954/0 (socket says 172.21.15.40:56954) 2023-07-18T03:30:17.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.944+0000 7f078bfff700 1 -- 172.21.15.40:0/42976711 learned_addr learned my addr 172.21.15.40:0/42976711 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:17.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.944+0000 7f078b7fe700 1 --2- 172.21.15.40:0/42976711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803124dd0 0x555803136950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.945+0000 7f07909b1700 1 --2- 172.21.15.40:0/42976711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803125c20 0x55580313d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.945+0000 7f078bfff700 1 -- 172.21.15.40:0/42976711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803124dd0 msgr2=0x555803136950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:17.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.945+0000 7f078bfff700 1 --2- 172.21.15.40:0/42976711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803124dd0 0x555803136950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.945+0000 7f078bfff700 1 -- 172.21.15.40:0/42976711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803125c20 msgr2=0x55580313d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:17.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.945+0000 7f078bfff700 1 --2- 172.21.15.40:0/42976711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803125c20 0x55580313d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.945+0000 7f078bfff700 1 -- 172.21.15.40:0/42976711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078c19ff00 con 0x555803124480 2023-07-18T03:30:17.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.945+0000 7f078b7fe700 1 --2- 172.21.15.40:0/42976711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803124dd0 0x555803136950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.945+0000 7f078bfff700 1 --2- 172.21.15.40:0/42976711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124480 0x555803136410 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f077c005d90 tx=0x7f077c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d3b3af06263b9d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.945+0000 7f076b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.945+0000 7f076b7fe700 1 -- 172.21.15.40:0/42976711 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c027470 con 0x555803124480 2023-07-18T03:30:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.945+0000 7f076b7fe700 1 -- 172.21.15.40:0/42976711 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f077c017070 con 0x555803124480 2023-07-18T03:30:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.945+0000 7f079e59ab80 1 -- 172.21.15.40:0/42976711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555802fcf4a0 con 0x555803124480 2023-07-18T03:30:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.945+0000 7f076b7fe700 1 -- 172.21.15.40:0/42976711 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f077c0275d0 con 0x555803124480 2023-07-18T03:30:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.945+0000 7f079e59ab80 10 client.15024 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:17.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.945+0000 7f079e59ab80 1 -- 172.21.15.40:0/42976711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555802e8cc10 con 0x555803124480 2023-07-18T03:30:17.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.945+0000 7f076b7fe700 1 -- 172.21.15.40:0/42976711 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555802fcf4a0 con 0x555803124480 2023-07-18T03:30:17.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.946+0000 7f076b7fe700 1 -- 172.21.15.40:0/42976711 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f077c01b590 con 0x555803124480 2023-07-18T03:30:17.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.946+0000 7f079e59ab80 10 client.15024 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:17.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.946+0000 7f079e59ab80 10 client.15024 resolve_mds: resolved name 'l' to daemon mds.l 2023-07-18T03:30:17.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.946+0000 7f079e59ab80 20 client.15024 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:17.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.946+0000 7f079e59ab80 1 --2- 172.21.15.40:0/42976711 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x5558031416b0 0x555803140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.946+0000 7f079e59ab80 4 client.15024 mds_command: new command op to 24442 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:30:17.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.946+0000 7f079e59ab80 1 -- 172.21.15.40:0/42976711 --> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55580306ee60 con 0x5558031416b0 2023-07-18T03:30:17.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.946+0000 7f078b7fe700 1 --2- 172.21.15.40:0/42976711 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x5558031416b0 0x555803140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.946+0000 7f076b7fe700 10 client.15024 ms_handle_connect on v2:172.21.15.134:6832/2345264306 2023-07-18T03:30:17.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.946+0000 7f078b7fe700 1 --2- 172.21.15.40:0/42976711 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x5558031416b0 0x555803140ab0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=104a434335e30cba server_cookie=c15c070dfc398ae0 in_seq=0 out_seq=0 2023-07-18T03:30:17.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.947+0000 7f076b7fe700 1 -- 172.21.15.40:0/42976711 <== mds.4 v2:172.21.15.134:6832/2345264306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55580306ee60 con 0x5558031416b0 2023-07-18T03:30:17.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.947+0000 7f076b7fe700 10 client.15024 handle_command_reply: tid=0 2023-07-18T03:30:17.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.947+0000 7f079e59ab80 1 client.15024 shutdown 2023-07-18T03:30:17.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.947+0000 7f079e59ab80 1 -- 172.21.15.40:0/42976711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124480 msgr2=0x555803136410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.947+0000 7f079e59ab80 1 --2- 172.21.15.40:0/42976711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124480 0x555803136410 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f077c005d90 tx=0x7f077c0047c0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.947+0000 7f079e59ab80 1 -- 172.21.15.40:0/42976711 shutdown_connections 2023-07-18T03:30:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.947+0000 7f079e59ab80 1 --2- 172.21.15.40:0/42976711 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x5558031416b0 0x555803140ab0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.947+0000 7f079e59ab80 1 --2- 172.21.15.40:0/42976711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803125c20 0x55580313d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.947+0000 7f079e59ab80 1 --2- 172.21.15.40:0/42976711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124480 0x555803136410 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.947+0000 7f079e59ab80 1 --2- 172.21.15.40:0/42976711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803124dd0 0x555803136950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.947+0000 7f076b7fe700 0 client.15024 ms_handle_reset on v2:172.21.15.134:6832/2345264306 2023-07-18T03:30:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.947+0000 7f079e59ab80 1 -- 172.21.15.40:0/42976711 >> 172.21.15.40:0/42976711 conn(0x555803119d50 msgr2=0x55580311e860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:17.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.947+0000 7f079e59ab80 1 -- 172.21.15.40:0/42976711 shutdown_connections 2023-07-18T03:30:17.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.947+0000 7f079e59ab80 1 -- 172.21.15.40:0/42976711 wait complete. 2023-07-18T03:30:17.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.948+0000 7f079e59ab80 20 client.15024 trim_cache size 0 max 16384 2023-07-18T03:30:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.965+0000 7f079e59ab80 1 Processor -- start 2023-07-18T03:30:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.965+0000 7f079e59ab80 1 -- start start 2023-07-18T03:30:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f079e59ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124480 0x555803140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f079e59ab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124dd0 0x555803140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f079e59ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803125c20 0x55580313d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f079e59ab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555802fcf4a0 con 0x555803124dd0 2023-07-18T03:30:17.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f079e59ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555803044790 con 0x555803124480 2023-07-18T03:30:17.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f079e59ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555802f05de0 con 0x555803125c20 2023-07-18T03:30:17.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f078b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124dd0 0x555803140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f078b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124dd0 0x555803140ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56964/0 (socket says 172.21.15.40:56964) 2023-07-18T03:30:17.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f078b7fe700 1 -- 172.21.15.40:0/3879284283 learned_addr learned my addr 172.21.15.40:0/3879284283 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:17.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f078bfff700 1 --2- 172.21.15.40:0/3879284283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124480 0x555803140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f078b7fe700 1 -- 172.21.15.40:0/3879284283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803125c20 msgr2=0x55580313d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:17.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f07909b1700 1 --2- 172.21.15.40:0/3879284283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803125c20 0x55580313d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f078b7fe700 1 --2- 172.21.15.40:0/3879284283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803125c20 0x55580313d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f078b7fe700 1 -- 172.21.15.40:0/3879284283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124480 msgr2=0x555803140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:17.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f078b7fe700 1 --2- 172.21.15.40:0/3879284283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124480 0x555803140ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f078b7fe700 1 -- 172.21.15.40:0/3879284283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555802eca200 con 0x555803124dd0 2023-07-18T03:30:17.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f078bfff700 1 --2- 172.21.15.40:0/3879284283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124480 0x555803140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:17.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f078b7fe700 1 --2- 172.21.15.40:0/3879284283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124dd0 0x555803140ff0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f07740cb930 tx=0x7f07740977f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07225be6a532d57 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:17.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f0782ffd700 1 -- 172.21.15.40:0/3879284283 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07740a12a0 con 0x555803124dd0 2023-07-18T03:30:17.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f0782ffd700 1 -- 172.21.15.40:0/3879284283 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07740a10b0 con 0x555803124dd0 2023-07-18T03:30:17.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.966+0000 7f0782ffd700 1 -- 172.21.15.40:0/3879284283 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077409aad0 con 0x555803124dd0 2023-07-18T03:30:17.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.967+0000 7f079e59ab80 1 -- 172.21.15.40:0/3879284283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124dd0 msgr2=0x555803140ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.967+0000 7f079e59ab80 1 --2- 172.21.15.40:0/3879284283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124dd0 0x555803140ff0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f07740cb930 tx=0x7f07740977f0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.967+0000 7f079e59ab80 1 -- 172.21.15.40:0/3879284283 shutdown_connections 2023-07-18T03:30:17.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.967+0000 7f079e59ab80 1 --2- 172.21.15.40:0/3879284283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803125c20 0x55580313d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.967+0000 7f079e59ab80 1 --2- 172.21.15.40:0/3879284283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124480 0x555803140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.967+0000 7f079e59ab80 1 --2- 172.21.15.40:0/3879284283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803124dd0 0x555803140ff0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.967+0000 7f079e59ab80 1 -- 172.21.15.40:0/3879284283 >> 172.21.15.40:0/3879284283 conn(0x555803119d50 msgr2=0x55580312ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:17.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.967+0000 7f079e59ab80 1 -- 172.21.15.40:0/3879284283 shutdown_connections 2023-07-18T03:30:17.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.967+0000 7f079e59ab80 1 -- 172.21.15.40:0/3879284283 wait complete. 2023-07-18T03:30:17.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.967+0000 7f079e59ab80 1 -- start start 2023-07-18T03:30:17.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f079e59ab80 1 Processor -- start 2023-07-18T03:30:17.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f079e59ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803124480 0x55580313abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f079e59ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124dd0 0x55580313d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f079e59ab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803125c20 0x55580313d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f079e59ab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55580313de20 con 0x555803125c20 2023-07-18T03:30:17.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f079e59ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555803195510 con 0x555803124dd0 2023-07-18T03:30:17.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f079e59ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558031956d0 con 0x555803124480 2023-07-18T03:30:17.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f079e59ab80 10 client.0 _ll_register_callbacks cb 0x7f077c010c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:17.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f07909b1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803125c20 0x55580313d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f07909b1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803125c20 0x55580313d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56972/0 (socket says 172.21.15.40:56972) 2023-07-18T03:30:17.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f07909b1700 1 -- 172.21.15.40:0/972076306 learned_addr learned my addr 172.21.15.40:0/972076306 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:17.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f078b7fe700 1 --2- 172.21.15.40:0/972076306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124dd0 0x55580313d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f078bfff700 1 --2- 172.21.15.40:0/972076306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803124480 0x55580313abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f07909b1700 1 -- 172.21.15.40:0/972076306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803124480 msgr2=0x55580313abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:17.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f07909b1700 1 --2- 172.21.15.40:0/972076306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803124480 0x55580313abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f07909b1700 1 -- 172.21.15.40:0/972076306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124dd0 msgr2=0x55580313d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:17.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f07909b1700 1 --2- 172.21.15.40:0/972076306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124dd0 0x55580313d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f07909b1700 1 -- 172.21.15.40:0/972076306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555802ea3d90 con 0x555803125c20 2023-07-18T03:30:17.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f078b7fe700 1 --2- 172.21.15.40:0/972076306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124dd0 0x55580313d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:17.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f078bfff700 1 --2- 172.21.15.40:0/972076306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803124480 0x55580313abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:17.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f07909b1700 1 --2- 172.21.15.40:0/972076306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803125c20 0x55580313d750 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0784007f00 tx=0x7f0784009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957ffc3a35133d4e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:17.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f076b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:17.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.968+0000 7f076b7fe700 1 -- 172.21.15.40:0/972076306 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078400e040 con 0x555803125c20 2023-07-18T03:30:17.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.969+0000 7f076b7fe700 1 -- 172.21.15.40:0/972076306 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0784015c70 con 0x555803125c20 2023-07-18T03:30:17.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.969+0000 7f079e59ab80 1 -- 172.21.15.40:0/972076306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555803031000 con 0x555803125c20 2023-07-18T03:30:17.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.969+0000 7f076b7fe700 1 -- 172.21.15.40:0/972076306 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0784015e30 con 0x555803125c20 2023-07-18T03:30:17.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.969+0000 7f079e59ab80 10 client.15030 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:17.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.969+0000 7f079e59ab80 1 -- 172.21.15.40:0/972076306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555802ea2260 con 0x555803125c20 2023-07-18T03:30:17.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.969+0000 7f076b7fe700 1 -- 172.21.15.40:0/972076306 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555803031000 con 0x555803125c20 2023-07-18T03:30:17.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.969+0000 7f076b7fe700 1 -- 172.21.15.40:0/972076306 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f07840175a0 con 0x555803125c20 2023-07-18T03:30:17.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.969+0000 7f079e59ab80 10 client.15030 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:17.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.969+0000 7f079e59ab80 10 client.15030 resolve_mds: resolved name 'l' to daemon mds.l 2023-07-18T03:30:17.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.969+0000 7f079e59ab80 20 client.15030 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:17.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.969+0000 7f079e59ab80 1 --2- 172.21.15.40:0/972076306 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55580319a550 0x555803196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:17.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.969+0000 7f079e59ab80 4 client.15030 mds_command: new command op to 24442 tid=0[{"prefix": "status"}] 2023-07-18T03:30:17.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.969+0000 7f079e59ab80 1 -- 172.21.15.40:0/972076306 --> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555802ec4f20 con 0x55580319a550 2023-07-18T03:30:17.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.969+0000 7f078bfff700 1 --2- 172.21.15.40:0/972076306 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55580319a550 0x555803196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:17.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.970+0000 7f076b7fe700 10 client.15030 ms_handle_connect on v2:172.21.15.134:6832/2345264306 2023-07-18T03:30:17.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.970+0000 7f078bfff700 1 --2- 172.21.15.40:0/972076306 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55580319a550 0x555803196ec0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ecd49fee4350c9dc server_cookie=7c723f146f4f8152 in_seq=0 out_seq=0 2023-07-18T03:30:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.970+0000 7f076b7fe700 1 -- 172.21.15.40:0/972076306 <== mds.4 v2:172.21.15.134:6832/2345264306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x555802ec4f20 con 0x55580319a550 2023-07-18T03:30:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.970+0000 7f076b7fe700 10 client.15030 handle_command_reply: tid=0 2023-07-18T03:30:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.970+0000 7f079e59ab80 1 client.15030 shutdown 2023-07-18T03:30:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.970+0000 7f079e59ab80 1 -- 172.21.15.40:0/972076306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803125c20 msgr2=0x55580313d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.970+0000 7f079e59ab80 1 --2- 172.21.15.40:0/972076306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803125c20 0x55580313d750 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0784007f00 tx=0x7f0784009380 comp rx=0 tx=0).stop 2023-07-18T03:30:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.970+0000 7f079e59ab80 1 -- 172.21.15.40:0/972076306 shutdown_connections 2023-07-18T03:30:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.970+0000 7f079e59ab80 1 --2- 172.21.15.40:0/972076306 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55580319a550 0x555803196ec0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.970+0000 7f079e59ab80 1 --2- 172.21.15.40:0/972076306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555803125c20 0x55580313d750 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.970+0000 7f076b7fe700 0 client.15030 ms_handle_reset on v2:172.21.15.134:6832/2345264306 2023-07-18T03:30:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.970+0000 7f079e59ab80 1 --2- 172.21.15.40:0/972076306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555803124480 0x55580313abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.970+0000 7f079e59ab80 1 --2- 172.21.15.40:0/972076306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555803124dd0 0x55580313d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.970+0000 7f079e59ab80 1 -- 172.21.15.40:0/972076306 >> 172.21.15.40:0/972076306 conn(0x555803119d50 msgr2=0x555803140ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.970+0000 7f079e59ab80 1 -- 172.21.15.40:0/972076306 shutdown_connections 2023-07-18T03:30:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.970+0000 7f079e59ab80 1 -- 172.21.15.40:0/972076306 wait complete. 2023-07-18T03:30:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.970+0000 7f079e59ab80 20 client.15030 trim_cache size 0 max 16384 2023-07-18T03:30:17.979 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:30:17.979 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "3af3dba8-251a-11ee-9b34-001a4aab830c", 2023-07-18T03:30:17.979 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 4, 2023-07-18T03:30:17.979 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24442, 2023-07-18T03:30:17.979 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-07-18T03:30:17.979 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-07-18T03:30:17.980 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-07-18T03:30:17.980 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 44.339405192000001, 2023-07-18T03:30:17.980 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 27, 2023-07-18T03:30:17.980 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 81, 2023-07-18T03:30:17.980 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 81, 2023-07-18T03:30:17.980 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 81.445071212000002 2023-07-18T03:30:17.980 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:30:17.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.971+0000 7f0792414700 1 -- 172.21.15.40:0/3833937953 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f07780a11e0 msgr2=0x7f07780a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.971+0000 7f0792414700 1 --2- 172.21.15.40:0/3833937953 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f07780a11e0 0x7f07780a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f077c005950 tx=0x7f077c00b410 comp rx=0 tx=0).stop 2023-07-18T03:30:17.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.971+0000 7f0792414700 1 -- 172.21.15.40:0/3833937953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f078c0fed20 msgr2=0x7f078c197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:17.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.971+0000 7f0792414700 1 --2- 172.21.15.40:0/3833937953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f078c0fed20 0x7f078c197f50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0774000c00 tx=0x7f07740047a0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.971+0000 7f0792414700 1 -- 172.21.15.40:0/3833937953 shutdown_connections 2023-07-18T03:30:17.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.971+0000 7f0792414700 1 --2- 172.21.15.40:0/3833937953 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f07780a11e0 0x7f07780a3690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.971+0000 7f0792414700 1 --2- 172.21.15.40:0/3833937953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f078c0ff6d0 0x7f078c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.971+0000 7f0792414700 1 --2- 172.21.15.40:0/3833937953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f078c0fe3d0 0x7f078c197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.971+0000 7f0792414700 1 --2- 172.21.15.40:0/3833937953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f078c0fed20 0x7f078c197f50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:17.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.971+0000 7f0792414700 1 -- 172.21.15.40:0/3833937953 >> 172.21.15.40:0/3833937953 conn(0x7f078c0fa000 msgr2=0x7f078c0fbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:17.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.971+0000 7f0792414700 1 -- 172.21.15.40:0/3833937953 shutdown_connections 2023-07-18T03:30:17.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:17.971+0000 7f0792414700 1 -- 172.21.15.40:0/3833937953 wait complete. 2023-07-18T03:30:17.983 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-07-18T03:30:18.007 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:17 smithi040 ceph-mon[96439]: pgmap v273: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2023-07-18T03:30:18.007 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:17 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/4160823012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:18.007 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:17 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2719000639' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:18.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:17 smithi134 ceph-mon[98216]: pgmap v273: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2023-07-18T03:30:18.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:17 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/4160823012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:18.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:17 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2719000639' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:18.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:17 smithi064 ceph-mon[98059]: pgmap v273: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2023-07-18T03:30:18.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:17 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/4160823012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:18.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:17 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2719000639' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:18.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.113+0000 7fbad7fe2700 1 -- 172.21.15.40:0/3945823575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00fe9e0 msgr2=0x7fbad00fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.113+0000 7fbad7fe2700 1 --2- 172.21.15.40:0/3945823575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00fe9e0 0x7fbad00fedf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009990 tx=0x7fbacc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.113+0000 7fbad7fe2700 1 -- 172.21.15.40:0/3945823575 shutdown_connections 2023-07-18T03:30:18.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.113+0000 7fbad7fe2700 1 --2- 172.21.15.40:0/3945823575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbad0100560 0x7fbad0106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.113+0000 7fbad7fe2700 1 --2- 172.21.15.40:0/3945823575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00fe9e0 0x7fbad00fedf0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.113+0000 7fbad7fe2700 1 --2- 172.21.15.40:0/3945823575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad00ffbd0 0x7fbad0100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.113+0000 7fbad7fe2700 1 -- 172.21.15.40:0/3945823575 >> 172.21.15.40:0/3945823575 conn(0x7fbad00fa000 msgr2=0x7fbad00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.114+0000 7fbad7fe2700 1 -- 172.21.15.40:0/3945823575 shutdown_connections 2023-07-18T03:30:18.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.114+0000 7fbad7fe2700 1 -- 172.21.15.40:0/3945823575 wait complete. 2023-07-18T03:30:18.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.114+0000 7fbad7fe2700 1 Processor -- start 2023-07-18T03:30:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.114+0000 7fbad7fe2700 1 -- start start 2023-07-18T03:30:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.114+0000 7fbad7fe2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad00fe9e0 0x7fbad0194640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.114+0000 7fbad7fe2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00ffbd0 0x7fbad0194b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.114+0000 7fbad7fe2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbad0100560 0x7fbad019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.114+0000 7fbad7fe2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad019c1c0 con 0x7fbad00ffbd0 2023-07-18T03:30:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.114+0000 7fbad7fe2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad019c300 con 0x7fbad0100560 2023-07-18T03:30:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.114+0000 7fbad7fe2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad019c490 con 0x7fbad00fe9e0 2023-07-18T03:30:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.114+0000 7fbad557d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00ffbd0 0x7fbad0194b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.115+0000 7fbad557d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00ffbd0 0x7fbad0194b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56982/0 (socket says 172.21.15.40:56982) 2023-07-18T03:30:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.115+0000 7fbad557d700 1 -- 172.21.15.40:0/909235133 learned_addr learned my addr 172.21.15.40:0/909235133 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.115+0000 7fbad657f700 1 --2- 172.21.15.40:0/909235133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbad0100560 0x7fbad019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.115+0000 7fbad5d7e700 1 --2- 172.21.15.40:0/909235133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad00fe9e0 0x7fbad0194640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.115+0000 7fbad557d700 1 -- 172.21.15.40:0/909235133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad00fe9e0 msgr2=0x7fbad0194640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.115+0000 7fbad557d700 1 --2- 172.21.15.40:0/909235133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad00fe9e0 0x7fbad0194640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.115+0000 7fbad557d700 1 -- 172.21.15.40:0/909235133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbad0100560 msgr2=0x7fbad019bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.115+0000 7fbad557d700 1 --2- 172.21.15.40:0/909235133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbad0100560 0x7fbad019bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.115+0000 7fbad557d700 1 -- 172.21.15.40:0/909235133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc0095f0 con 0x7fbad00ffbd0 2023-07-18T03:30:18.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.115+0000 7fbad657f700 1 --2- 172.21.15.40:0/909235133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbad0100560 0x7fbad019bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.115+0000 7fbad557d700 1 --2- 172.21.15.40:0/909235133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00ffbd0 0x7fbad0194b80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbac000c9b0 tx=0x7fbac000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.115+0000 7fbac6ffd700 1 -- 172.21.15.40:0/909235133 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0007b00 con 0x7fbad00ffbd0 2023-07-18T03:30:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.115+0000 7fbac6ffd700 1 -- 172.21.15.40:0/909235133 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbac0007c60 con 0x7fbad00ffbd0 2023-07-18T03:30:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.115+0000 7fbad7fe2700 1 -- 172.21.15.40:0/909235133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad019c770 con 0x7fbad00ffbd0 2023-07-18T03:30:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.115+0000 7fbac6ffd700 1 -- 172.21.15.40:0/909235133 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac0018660 con 0x7fbad00ffbd0 2023-07-18T03:30:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.115+0000 7fbad7fe2700 1 -- 172.21.15.40:0/909235133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad019ccd0 con 0x7fbad00ffbd0 2023-07-18T03:30:18.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.117+0000 7fbac6ffd700 1 -- 172.21.15.40:0/909235133 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbac000f450 con 0x7fbad00ffbd0 2023-07-18T03:30:18.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.118+0000 7fbac6ffd700 1 --2- 172.21.15.40:0/909235133 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbabc0a1260 0x7fbabc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.118+0000 7fbad5d7e700 1 --2- 172.21.15.40:0/909235133 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbabc0a1260 0x7fbabc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.118+0000 7fbac6ffd700 1 -- 172.21.15.40:0/909235133 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbac00cc870 con 0x7fbad00ffbd0 2023-07-18T03:30:18.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.118+0000 7fbad5d7e700 1 --2- 172.21.15.40:0/909235133 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbabc0a1260 0x7fbabc0a3710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00b5c0 tx=0x7fbacc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.119+0000 7fbae4168b80 1 Processor -- start 2023-07-18T03:30:18.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.119+0000 7fbae4168b80 1 -- start start 2023-07-18T03:30:18.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.119+0000 7fbae4168b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decd480 0x556f1decd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.119+0000 7fbae4168b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decddd0 0x556f1dece690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.119+0000 7fbae4168b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decec20 0x556f1decf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.119+0000 7fbae4168b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1ddd2c60 con 0x556f1decec20 2023-07-18T03:30:18.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.119+0000 7fbae4168b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1de259f0 con 0x556f1decd480 2023-07-18T03:30:18.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.119+0000 7fbae4168b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1de1ee80 con 0x556f1decddd0 2023-07-18T03:30:18.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.119+0000 7fbad657f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decec20 0x556f1decf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.119+0000 7fbad657f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decec20 0x556f1decf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56992/0 (socket says 172.21.15.40:56992) 2023-07-18T03:30:18.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.119+0000 7fbad657f700 1 -- 172.21.15.40:0/1738809345 learned_addr learned my addr 172.21.15.40:0/1738809345 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.119+0000 7fbad557d700 1 --2- 172.21.15.40:0/1738809345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decddd0 0x556f1dece690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.119+0000 7fbad5d7e700 1 --2- 172.21.15.40:0/1738809345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decd480 0x556f1decd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.120+0000 7fbad657f700 1 -- 172.21.15.40:0/1738809345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decddd0 msgr2=0x556f1dece690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.120+0000 7fbad657f700 1 --2- 172.21.15.40:0/1738809345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decddd0 0x556f1dece690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.120+0000 7fbad657f700 1 -- 172.21.15.40:0/1738809345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decd480 msgr2=0x556f1decd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.120+0000 7fbad657f700 1 --2- 172.21.15.40:0/1738809345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decd480 0x556f1decd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.120+0000 7fbad657f700 1 -- 172.21.15.40:0/1738809345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f1dc4cd90 con 0x556f1decec20 2023-07-18T03:30:18.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.120+0000 7fbad657f700 1 --2- 172.21.15.40:0/1738809345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decec20 0x556f1decf090 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbac800e260 tx=0x7fbac800e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42d55e648128e23 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.120+0000 7fbabbfff700 1 -- 172.21.15.40:0/1738809345 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8018070 con 0x556f1decec20 2023-07-18T03:30:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.120+0000 7fbabbfff700 1 -- 172.21.15.40:0/1738809345 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbac8007d00 con 0x556f1decec20 2023-07-18T03:30:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.120+0000 7fbabbfff700 1 -- 172.21.15.40:0/1738809345 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8014c90 con 0x556f1decec20 2023-07-18T03:30:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.121+0000 7fbae4168b80 1 -- 172.21.15.40:0/1738809345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decec20 msgr2=0x556f1decf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.121+0000 7fbae4168b80 1 --2- 172.21.15.40:0/1738809345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decec20 0x556f1decf090 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbac800e260 tx=0x7fbac800e570 comp rx=0 tx=0).stop 2023-07-18T03:30:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.121+0000 7fbae4168b80 1 -- 172.21.15.40:0/1738809345 shutdown_connections 2023-07-18T03:30:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.121+0000 7fbae4168b80 1 --2- 172.21.15.40:0/1738809345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decec20 0x556f1decf090 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.121+0000 7fbae4168b80 1 --2- 172.21.15.40:0/1738809345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decd480 0x556f1decd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.121+0000 7fbae4168b80 1 --2- 172.21.15.40:0/1738809345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decddd0 0x556f1dece690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.121+0000 7fbae4168b80 1 -- 172.21.15.40:0/1738809345 >> 172.21.15.40:0/1738809345 conn(0x556f1dec2d50 msgr2=0x556f1dec8180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.121+0000 7fbae4168b80 1 -- 172.21.15.40:0/1738809345 shutdown_connections 2023-07-18T03:30:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.121+0000 7fbae4168b80 1 -- 172.21.15.40:0/1738809345 wait complete. 2023-07-18T03:30:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.121+0000 7fbae4168b80 1 -- start start 2023-07-18T03:30:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.121+0000 7fbae4168b80 1 Processor -- start 2023-07-18T03:30:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbae4168b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decd480 0x556f1ded7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbae4168b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decddd0 0x556f1ded7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbae4168b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decec20 0x556f1dede680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbae4168b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1ddbea70 con 0x556f1decddd0 2023-07-18T03:30:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbae4168b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1ddbe770 con 0x556f1decec20 2023-07-18T03:30:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbae4168b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1dca28d0 con 0x556f1decd480 2023-07-18T03:30:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbad557d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decddd0 0x556f1ded7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbae4168b80 10 client.0 _ll_register_callbacks cb 0x556f1ddbc710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbad557d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decddd0 0x556f1ded7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56996/0 (socket says 172.21.15.40:56996) 2023-07-18T03:30:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbad557d700 1 -- 172.21.15.40:0/1832338370 learned_addr learned my addr 172.21.15.40:0/1832338370 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbad657f700 1 --2- 172.21.15.40:0/1832338370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decec20 0x556f1dede680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbad5d7e700 1 --2- 172.21.15.40:0/1832338370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decd480 0x556f1ded7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbad557d700 1 -- 172.21.15.40:0/1832338370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decd480 msgr2=0x556f1ded7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbad557d700 1 --2- 172.21.15.40:0/1832338370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decd480 0x556f1ded7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbad557d700 1 -- 172.21.15.40:0/1832338370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decec20 msgr2=0x556f1dede680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbad557d700 1 --2- 172.21.15.40:0/1832338370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decec20 0x556f1dede680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbad557d700 1 -- 172.21.15.40:0/1832338370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac800f040 con 0x556f1decddd0 2023-07-18T03:30:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbad5d7e700 1 --2- 172.21.15.40:0/1832338370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decd480 0x556f1ded7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbad557d700 1 --2- 172.21.15.40:0/1832338370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decddd0 0x556f1ded7640 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbac00961d0 tx=0x7fbac0099d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fd4cce7b883119 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.122+0000 7fbab97fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.123+0000 7fbab97fa700 1 -- 172.21.15.40:0/1832338370 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac009c040 con 0x556f1decddd0 2023-07-18T03:30:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.123+0000 7fbab97fa700 1 -- 172.21.15.40:0/1832338370 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbac0099f30 con 0x556f1decddd0 2023-07-18T03:30:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.123+0000 7fbae4168b80 1 -- 172.21.15.40:0/1832338370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f1dd784a0 con 0x556f1decddd0 2023-07-18T03:30:18.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.123+0000 7fbab97fa700 1 -- 172.21.15.40:0/1832338370 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbac009a0f0 con 0x556f1decddd0 2023-07-18T03:30:18.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.123+0000 7fbae4168b80 10 client.15042 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:18.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.123+0000 7fbae4168b80 1 -- 172.21.15.40:0/1832338370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556f1dc35c10 con 0x556f1decddd0 2023-07-18T03:30:18.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.123+0000 7fbab97fa700 1 -- 172.21.15.40:0/1832338370 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556f1dd784a0 con 0x556f1decddd0 2023-07-18T03:30:18.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.123+0000 7fbab97fa700 1 -- 172.21.15.40:0/1832338370 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbac00a1d00 con 0x556f1decddd0 2023-07-18T03:30:18.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.123+0000 7fbae4168b80 10 client.15042 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:18.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.123+0000 7fbae4168b80 10 client.15042 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-18T03:30:18.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.123+0000 7fbae4168b80 20 client.15042 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.123+0000 7fbae4168b80 1 --2- 172.21.15.40:0/1832338370 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x556f1dee21d0 0x556f1dee1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.123+0000 7fbae4168b80 4 client.15042 mds_command: new command op to 24452 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:30:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.123+0000 7fbae4168b80 1 -- 172.21.15.40:0/1832338370 --> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f1de17e60 con 0x556f1dee21d0 2023-07-18T03:30:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.123+0000 7fbad5d7e700 1 --2- 172.21.15.40:0/1832338370 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x556f1dee21d0 0x556f1dee1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.124+0000 7fbab97fa700 10 client.15042 ms_handle_connect on v2:172.21.15.64:6834/1864495009 2023-07-18T03:30:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.124+0000 7fbad5d7e700 1 --2- 172.21.15.40:0/1832338370 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x556f1dee21d0 0x556f1dee1660 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1939d14360cc638a server_cookie=e8ba000ff3cff57b in_seq=0 out_seq=0 2023-07-18T03:30:18.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.125+0000 7fbab97fa700 1 -- 172.21.15.40:0/1832338370 <== mds.3 v2:172.21.15.64:6834/1864495009 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556f1de17e60 con 0x556f1dee21d0 2023-07-18T03:30:18.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.125+0000 7fbab97fa700 10 client.15042 handle_command_reply: tid=0 2023-07-18T03:30:18.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.125+0000 7fbae4168b80 1 client.15042 shutdown 2023-07-18T03:30:18.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.125+0000 7fbae4168b80 1 -- 172.21.15.40:0/1832338370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decddd0 msgr2=0x556f1ded7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.125+0000 7fbae4168b80 1 --2- 172.21.15.40:0/1832338370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decddd0 0x556f1ded7640 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbac00961d0 tx=0x7fbac0099d90 comp rx=0 tx=0).stop 2023-07-18T03:30:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.125+0000 7fbae4168b80 1 -- 172.21.15.40:0/1832338370 shutdown_connections 2023-07-18T03:30:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.125+0000 7fbae4168b80 1 --2- 172.21.15.40:0/1832338370 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x556f1dee21d0 0x556f1dee1660 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.125+0000 7fbae4168b80 1 --2- 172.21.15.40:0/1832338370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decec20 0x556f1dede680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.125+0000 7fbae4168b80 1 --2- 172.21.15.40:0/1832338370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decd480 0x556f1ded7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.125+0000 7fbae4168b80 1 --2- 172.21.15.40:0/1832338370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decddd0 0x556f1ded7640 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.125+0000 7fbab97fa700 0 client.15042 ms_handle_reset on v2:172.21.15.64:6834/1864495009 2023-07-18T03:30:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.125+0000 7fbae4168b80 1 -- 172.21.15.40:0/1832338370 >> 172.21.15.40:0/1832338370 conn(0x556f1dec2d50 msgr2=0x556f1dec7860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.125+0000 7fbae4168b80 1 -- 172.21.15.40:0/1832338370 shutdown_connections 2023-07-18T03:30:18.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.125+0000 7fbae4168b80 1 -- 172.21.15.40:0/1832338370 wait complete. 2023-07-18T03:30:18.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.125+0000 7fbae4168b80 20 client.15042 trim_cache size 0 max 16384 2023-07-18T03:30:18.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.143+0000 7fbae4168b80 1 Processor -- start 2023-07-18T03:30:18.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.143+0000 7fbae4168b80 1 -- start start 2023-07-18T03:30:18.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.143+0000 7fbae4168b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decd480 0x556f1ded4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.143+0000 7fbae4168b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decddd0 0x556f1dede5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.143+0000 7fbae4168b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decec20 0x556f1ded1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.143+0000 7fbae4168b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1ddc8470 con 0x556f1decd480 2023-07-18T03:30:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.143+0000 7fbae4168b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1dded790 con 0x556f1decec20 2023-07-18T03:30:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.143+0000 7fbae4168b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1dddd480 con 0x556f1decddd0 2023-07-18T03:30:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.143+0000 7fbad5d7e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decd480 0x556f1ded4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.143+0000 7fbad5d7e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decd480 0x556f1ded4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57002/0 (socket says 172.21.15.40:57002) 2023-07-18T03:30:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.143+0000 7fbad5d7e700 1 -- 172.21.15.40:0/4129528092 learned_addr learned my addr 172.21.15.40:0/4129528092 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.144+0000 7fbad657f700 1 --2- 172.21.15.40:0/4129528092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decec20 0x556f1ded1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.144+0000 7fbad557d700 1 --2- 172.21.15.40:0/4129528092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decddd0 0x556f1dede5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.144+0000 7fbad5d7e700 1 -- 172.21.15.40:0/4129528092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decddd0 msgr2=0x556f1dede5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.144+0000 7fbad5d7e700 1 --2- 172.21.15.40:0/4129528092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decddd0 0x556f1dede5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.144+0000 7fbad5d7e700 1 -- 172.21.15.40:0/4129528092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decec20 msgr2=0x556f1ded1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.144+0000 7fbad5d7e700 1 --2- 172.21.15.40:0/4129528092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decec20 0x556f1ded1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.144+0000 7fbad5d7e700 1 -- 172.21.15.40:0/4129528092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f1decf510 con 0x556f1decd480 2023-07-18T03:30:18.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.144+0000 7fbad5d7e700 1 --2- 172.21.15.40:0/4129528092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decd480 0x556f1ded4fd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009990 tx=0x7fbacc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7d64b7daca9c4c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.144+0000 7fbac4ff9700 1 -- 172.21.15.40:0/4129528092 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00b770 con 0x556f1decd480 2023-07-18T03:30:18.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.144+0000 7fbac4ff9700 1 -- 172.21.15.40:0/4129528092 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbacc01e040 con 0x556f1decd480 2023-07-18T03:30:18.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.144+0000 7fbac4ff9700 1 -- 172.21.15.40:0/4129528092 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc003dd0 con 0x556f1decd480 2023-07-18T03:30:18.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.145+0000 7fbae4168b80 1 -- 172.21.15.40:0/4129528092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decd480 msgr2=0x556f1ded4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.145+0000 7fbae4168b80 1 --2- 172.21.15.40:0/4129528092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decd480 0x556f1ded4fd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009990 tx=0x7fbacc020000 comp rx=0 tx=0).stop 2023-07-18T03:30:18.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.145+0000 7fbae4168b80 1 -- 172.21.15.40:0/4129528092 shutdown_connections 2023-07-18T03:30:18.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.145+0000 7fbae4168b80 1 --2- 172.21.15.40:0/4129528092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decec20 0x556f1ded1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.145+0000 7fbae4168b80 1 --2- 172.21.15.40:0/4129528092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decd480 0x556f1ded4fd0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.145+0000 7fbae4168b80 1 --2- 172.21.15.40:0/4129528092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decddd0 0x556f1dede5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.145+0000 7fbae4168b80 1 -- 172.21.15.40:0/4129528092 >> 172.21.15.40:0/4129528092 conn(0x556f1dec2d50 msgr2=0x556f1dee1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.145+0000 7fbae4168b80 1 -- 172.21.15.40:0/4129528092 shutdown_connections 2023-07-18T03:30:18.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.145+0000 7fbae4168b80 1 -- 172.21.15.40:0/4129528092 wait complete. 2023-07-18T03:30:18.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.145+0000 7fbae4168b80 1 -- start start 2023-07-18T03:30:18.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.145+0000 7fbae4168b80 1 Processor -- start 2023-07-18T03:30:18.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbae4168b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decd480 0x556f1dedc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbae4168b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decddd0 0x556f1df34040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbae4168b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decec20 0x556f1df36590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbae4168b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1ded4800 con 0x556f1decd480 2023-07-18T03:30:18.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbae4168b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1df36ad0 con 0x556f1decec20 2023-07-18T03:30:18.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbae4168b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f1df36c10 con 0x556f1decddd0 2023-07-18T03:30:18.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbae4168b80 10 client.0 _ll_register_callbacks cb 0x556f1ddbc710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:18.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbad5d7e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decd480 0x556f1dedc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbad5d7e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decd480 0x556f1dedc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57008/0 (socket says 172.21.15.40:57008) 2023-07-18T03:30:18.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbad5d7e700 1 -- 172.21.15.40:0/289955352 learned_addr learned my addr 172.21.15.40:0/289955352 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbad657f700 1 --2- 172.21.15.40:0/289955352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decec20 0x556f1df36590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbad557d700 1 --2- 172.21.15.40:0/289955352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decddd0 0x556f1df34040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbad5d7e700 1 -- 172.21.15.40:0/289955352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decddd0 msgr2=0x556f1df34040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbad5d7e700 1 --2- 172.21.15.40:0/289955352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decddd0 0x556f1df34040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbad5d7e700 1 -- 172.21.15.40:0/289955352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decec20 msgr2=0x556f1df36590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbad5d7e700 1 --2- 172.21.15.40:0/289955352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decec20 0x556f1df36590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbad5d7e700 1 -- 172.21.15.40:0/289955352 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbacc02c040 con 0x556f1decd480 2023-07-18T03:30:18.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbad5d7e700 1 --2- 172.21.15.40:0/289955352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decd480 0x556f1dedc590 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00b830 tx=0x7fbacc00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f48b75ec0dc06cc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbab97fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:18.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbab97fa700 1 -- 172.21.15.40:0/289955352 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc034050 con 0x556f1decd480 2023-07-18T03:30:18.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbab97fa700 1 -- 172.21.15.40:0/289955352 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbacc0341b0 con 0x556f1decd480 2023-07-18T03:30:18.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.146+0000 7fbae4168b80 1 -- 172.21.15.40:0/289955352 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbac8007d00 con 0x556f1decd480 2023-07-18T03:30:18.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.147+0000 7fbab97fa700 1 -- 172.21.15.40:0/289955352 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbacc034370 con 0x556f1decd480 2023-07-18T03:30:18.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.147+0000 7fbae4168b80 10 client.15048 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:18.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.147+0000 7fbae4168b80 1 -- 172.21.15.40:0/289955352 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556f1dc73200 con 0x556f1decd480 2023-07-18T03:30:18.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.147+0000 7fbab97fa700 1 -- 172.21.15.40:0/289955352 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbac8007d00 con 0x556f1decd480 2023-07-18T03:30:18.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.147+0000 7fbab97fa700 1 -- 172.21.15.40:0/289955352 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbacc046730 con 0x556f1decd480 2023-07-18T03:30:18.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.147+0000 7fbae4168b80 10 client.15048 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:18.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.147+0000 7fbae4168b80 10 client.15048 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-18T03:30:18.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.147+0000 7fbae4168b80 20 client.15048 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:18.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.147+0000 7fbae4168b80 1 --2- 172.21.15.40:0/289955352 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x556f1df3b3b0 0x556f1df37d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.147+0000 7fbae4168b80 4 client.15048 mds_command: new command op to 24452 tid=0[{"prefix": "status"}] 2023-07-18T03:30:18.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.147+0000 7fbae4168b80 1 -- 172.21.15.40:0/289955352 --> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556f1ddda000 con 0x556f1df3b3b0 2023-07-18T03:30:18.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.147+0000 7fbad557d700 1 --2- 172.21.15.40:0/289955352 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x556f1df3b3b0 0x556f1df37d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.148+0000 7fbab97fa700 10 client.15048 ms_handle_connect on v2:172.21.15.64:6834/1864495009 2023-07-18T03:30:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.148+0000 7fbad557d700 1 --2- 172.21.15.40:0/289955352 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x556f1df3b3b0 0x556f1df37d40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=856b26a265c06539 server_cookie=cf0787b4b2a137a in_seq=0 out_seq=0 2023-07-18T03:30:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.148+0000 7fbab97fa700 1 -- 172.21.15.40:0/289955352 <== mds.3 v2:172.21.15.64:6834/1864495009 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x556f1ddda000 con 0x556f1df3b3b0 2023-07-18T03:30:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.148+0000 7fbab97fa700 10 client.15048 handle_command_reply: tid=0 2023-07-18T03:30:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.148+0000 7fbae4168b80 1 client.15048 shutdown 2023-07-18T03:30:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.148+0000 7fbae4168b80 1 -- 172.21.15.40:0/289955352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decd480 msgr2=0x556f1dedc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.148+0000 7fbae4168b80 1 --2- 172.21.15.40:0/289955352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decd480 0x556f1dedc590 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00b830 tx=0x7fbacc00b860 comp rx=0 tx=0).stop 2023-07-18T03:30:18.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.148+0000 7fbae4168b80 1 -- 172.21.15.40:0/289955352 shutdown_connections 2023-07-18T03:30:18.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.148+0000 7fbae4168b80 1 --2- 172.21.15.40:0/289955352 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x556f1df3b3b0 0x556f1df37d40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.148+0000 7fbae4168b80 1 --2- 172.21.15.40:0/289955352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1decec20 0x556f1df36590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.148+0000 7fbae4168b80 1 --2- 172.21.15.40:0/289955352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f1decd480 0x556f1dedc590 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.148+0000 7fbab97fa700 0 client.15048 ms_handle_reset on v2:172.21.15.64:6834/1864495009 2023-07-18T03:30:18.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.148+0000 7fbae4168b80 1 --2- 172.21.15.40:0/289955352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556f1decddd0 0x556f1df34040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.148+0000 7fbae4168b80 1 -- 172.21.15.40:0/289955352 >> 172.21.15.40:0/289955352 conn(0x556f1dec2d50 msgr2=0x556f1ded4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.148+0000 7fbae4168b80 1 -- 172.21.15.40:0/289955352 shutdown_connections 2023-07-18T03:30:18.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.148+0000 7fbae4168b80 1 -- 172.21.15.40:0/289955352 wait complete. 2023-07-18T03:30:18.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.148+0000 7fbae4168b80 20 client.15048 trim_cache size 0 max 16384 2023-07-18T03:30:18.158 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:30:18.158 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "3af3dba8-251a-11ee-9b34-001a4aab830c", 2023-07-18T03:30:18.158 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 3, 2023-07-18T03:30:18.159 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24452, 2023-07-18T03:30:18.159 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-07-18T03:30:18.159 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-07-18T03:30:18.159 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-07-18T03:30:18.159 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 45.524453694000002, 2023-07-18T03:30:18.159 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 27, 2023-07-18T03:30:18.159 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 81, 2023-07-18T03:30:18.160 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 81, 2023-07-18T03:30:18.160 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 73.080529730999999 2023-07-18T03:30:18.160 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:30:18.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.149+0000 7fbad7fe2700 1 -- 172.21.15.40:0/909235133 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbabc0a1260 msgr2=0x7fbabc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.149+0000 7fbad7fe2700 1 --2- 172.21.15.40:0/909235133 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbabc0a1260 0x7fbabc0a3710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00b5c0 tx=0x7fbacc005960 comp rx=0 tx=0).stop 2023-07-18T03:30:18.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.149+0000 7fbad7fe2700 1 -- 172.21.15.40:0/909235133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00ffbd0 msgr2=0x7fbad0194b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.149+0000 7fbad7fe2700 1 --2- 172.21.15.40:0/909235133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00ffbd0 0x7fbad0194b80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbac000c9b0 tx=0x7fbac000ccc0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.149+0000 7fbad7fe2700 1 -- 172.21.15.40:0/909235133 shutdown_connections 2023-07-18T03:30:18.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.149+0000 7fbad7fe2700 1 --2- 172.21.15.40:0/909235133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbad0100560 0x7fbad019bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.149+0000 7fbad7fe2700 1 --2- 172.21.15.40:0/909235133 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbabc0a1260 0x7fbabc0a3710 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.149+0000 7fbad7fe2700 1 --2- 172.21.15.40:0/909235133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbad00fe9e0 0x7fbad0194640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.149+0000 7fbad7fe2700 1 --2- 172.21.15.40:0/909235133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbad00ffbd0 0x7fbad0194b80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.149+0000 7fbad7fe2700 1 -- 172.21.15.40:0/909235133 >> 172.21.15.40:0/909235133 conn(0x7fbad00fa000 msgr2=0x7fbad00fc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.149+0000 7fbad7fe2700 1 -- 172.21.15.40:0/909235133 shutdown_connections 2023-07-18T03:30:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.149+0000 7fbad7fe2700 1 -- 172.21.15.40:0/909235133 wait complete. 2023-07-18T03:30:18.163 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-07-18T03:30:18.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.332+0000 7f198b7d1700 1 -- 172.21.15.40:0/4278254480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840ff580 msgr2=0x7f1984106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.332+0000 7f198b7d1700 1 --2- 172.21.15.40:0/4278254480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840ff580 0x7f1984106e30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f197c00b0a0 tx=0x7f197c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.332+0000 7f198b7d1700 1 -- 172.21.15.40:0/4278254480 shutdown_connections 2023-07-18T03:30:18.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.332+0000 7f198b7d1700 1 --2- 172.21.15.40:0/4278254480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840ff580 0x7f1984106e30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.332+0000 7f198b7d1700 1 --2- 172.21.15.40:0/4278254480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19840fff80 0x7f1984100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.332+0000 7f198b7d1700 1 --2- 172.21.15.40:0/4278254480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19840febd0 0x7f19840ff040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.332+0000 7f198b7d1700 1 -- 172.21.15.40:0/4278254480 >> 172.21.15.40:0/4278254480 conn(0x7f19840f9770 msgr2=0x7f19840fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.332+0000 7f198b7d1700 1 -- 172.21.15.40:0/4278254480 shutdown_connections 2023-07-18T03:30:18.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.332+0000 7f198b7d1700 1 -- 172.21.15.40:0/4278254480 wait complete. 2023-07-18T03:30:18.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.332+0000 7f198b7d1700 1 Processor -- start 2023-07-18T03:30:18.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.332+0000 7f198b7d1700 1 -- start start 2023-07-18T03:30:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f198b7d1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19840febd0 0x7f1984194820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f198b7d1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840ff580 0x7f1984194d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f198b7d1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19840fff80 0x7f198419bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f198b7d1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198419c3a0 con 0x7f19840ff580 2023-07-18T03:30:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f198b7d1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198419c4e0 con 0x7f19840fff80 2023-07-18T03:30:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f198b7d1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198419c670 con 0x7f19840febd0 2023-07-18T03:30:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f1988d6c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840ff580 0x7f1984194d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f1988d6c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840ff580 0x7f1984194d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57024/0 (socket says 172.21.15.40:57024) 2023-07-18T03:30:18.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f1988d6c700 1 -- 172.21.15.40:0/1081273236 learned_addr learned my addr 172.21.15.40:0/1081273236 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f198956d700 1 --2- 172.21.15.40:0/1081273236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19840febd0 0x7f1984194820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f1989d6e700 1 --2- 172.21.15.40:0/1081273236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19840fff80 0x7f198419bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f1988d6c700 1 -- 172.21.15.40:0/1081273236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19840febd0 msgr2=0x7f1984194820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f1988d6c700 1 --2- 172.21.15.40:0/1081273236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19840febd0 0x7f1984194820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f1988d6c700 1 -- 172.21.15.40:0/1081273236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19840fff80 msgr2=0x7f198419bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f1988d6c700 1 --2- 172.21.15.40:0/1081273236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19840fff80 0x7f198419bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f1988d6c700 1 -- 172.21.15.40:0/1081273236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c009d00 con 0x7f19840ff580 2023-07-18T03:30:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f1989d6e700 1 --2- 172.21.15.40:0/1081273236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19840fff80 0x7f198419bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f1988d6c700 1 --2- 172.21.15.40:0/1081273236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840ff580 0x7f1984194d60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f198000d980 tx=0x7f198000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f197a7fc700 1 -- 172.21.15.40:0/1081273236 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1980004220 con 0x7f19840ff580 2023-07-18T03:30:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f197a7fc700 1 -- 172.21.15.40:0/1081273236 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1980007640 con 0x7f19840ff580 2023-07-18T03:30:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f197a7fc700 1 -- 172.21.15.40:0/1081273236 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1980005610 con 0x7f19840ff580 2023-07-18T03:30:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.333+0000 7f198b7d1700 1 -- 172.21.15.40:0/1081273236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198419c950 con 0x7f19840ff580 2023-07-18T03:30:18.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.334+0000 7f198b7d1700 1 -- 172.21.15.40:0/1081273236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198419ce10 con 0x7f19840ff580 2023-07-18T03:30:18.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.336+0000 7f197a7fc700 1 -- 172.21.15.40:0/1081273236 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1980004750 con 0x7f19840ff580 2023-07-18T03:30:18.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.336+0000 7f197a7fc700 1 --2- 172.21.15.40:0/1081273236 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f19700a11e0 0x7f19700a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.336+0000 7f197a7fc700 1 -- 172.21.15.40:0/1081273236 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f19800cc800 con 0x7f19840ff580 2023-07-18T03:30:18.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.336+0000 7f198956d700 1 --2- 172.21.15.40:0/1081273236 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f19700a11e0 0x7f19700a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.337+0000 7f198956d700 1 --2- 172.21.15.40:0/1081273236 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f19700a11e0 0x7f19700a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1974005950 tx=0x7f197400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.337+0000 7f1997957b80 1 Processor -- start 2023-07-18T03:30:18.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.337+0000 7f1997957b80 1 -- start start 2023-07-18T03:30:18.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.337+0000 7f1997957b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450e690 0x55d1c450eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.337+0000 7f1997957b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450efe0 0x55d1c450f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.337+0000 7f1997957b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450fe30 0x55d1c45102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.337+0000 7f1997957b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c4411c60 con 0x55d1c450fe30 2023-07-18T03:30:18.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.337+0000 7f1997957b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c44649f0 con 0x55d1c450efe0 2023-07-18T03:30:18.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.337+0000 7f1997957b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c445de80 con 0x55d1c450e690 2023-07-18T03:30:18.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.338+0000 7f1989d6e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450fe30 0x55d1c45102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.338+0000 7f1989d6e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450fe30 0x55d1c45102a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57034/0 (socket says 172.21.15.40:57034) 2023-07-18T03:30:18.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.338+0000 7f1989d6e700 1 -- 172.21.15.40:0/3237984964 learned_addr learned my addr 172.21.15.40:0/3237984964 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.338+0000 7f198956d700 1 --2- 172.21.15.40:0/3237984964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450e690 0x55d1c450eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.338+0000 7f1988d6c700 1 --2- 172.21.15.40:0/3237984964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450efe0 0x55d1c450f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.338+0000 7f1989d6e700 1 -- 172.21.15.40:0/3237984964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450e690 msgr2=0x55d1c450eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.338+0000 7f1989d6e700 1 --2- 172.21.15.40:0/3237984964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450e690 0x55d1c450eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.338+0000 7f1989d6e700 1 -- 172.21.15.40:0/3237984964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450efe0 msgr2=0x55d1c450f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.338+0000 7f1989d6e700 1 --2- 172.21.15.40:0/3237984964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450efe0 0x55d1c450f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.338+0000 7f1989d6e700 1 -- 172.21.15.40:0/3237984964 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1c428bd90 con 0x55d1c450fe30 2023-07-18T03:30:18.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.338+0000 7f1989d6e700 1 --2- 172.21.15.40:0/3237984964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450fe30 0x55d1c45102a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f197c00b070 tx=0x7f197c009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7767140bc30fb231 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.338+0000 7f196f7fe700 1 -- 172.21.15.40:0/3237984964 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c00e050 con 0x55d1c450fe30 2023-07-18T03:30:18.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.338+0000 7f196f7fe700 1 -- 172.21.15.40:0/3237984964 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f197c008ef0 con 0x55d1c450fe30 2023-07-18T03:30:18.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.338+0000 7f196f7fe700 1 -- 172.21.15.40:0/3237984964 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c0052a0 con 0x55d1c450fe30 2023-07-18T03:30:18.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.339+0000 7f1997957b80 1 -- 172.21.15.40:0/3237984964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450fe30 msgr2=0x55d1c45102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.339+0000 7f1997957b80 1 --2- 172.21.15.40:0/3237984964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450fe30 0x55d1c45102a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f197c00b070 tx=0x7f197c009c80 comp rx=0 tx=0).stop 2023-07-18T03:30:18.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.339+0000 7f1997957b80 1 -- 172.21.15.40:0/3237984964 shutdown_connections 2023-07-18T03:30:18.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.339+0000 7f1997957b80 1 --2- 172.21.15.40:0/3237984964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450fe30 0x55d1c45102a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.339+0000 7f1997957b80 1 --2- 172.21.15.40:0/3237984964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450e690 0x55d1c450eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.339+0000 7f1997957b80 1 --2- 172.21.15.40:0/3237984964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450efe0 0x55d1c450f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.339+0000 7f1997957b80 1 -- 172.21.15.40:0/3237984964 >> 172.21.15.40:0/3237984964 conn(0x55d1c4501d50 msgr2=0x55d1c4507180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.339+0000 7f1997957b80 1 -- 172.21.15.40:0/3237984964 shutdown_connections 2023-07-18T03:30:18.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.339+0000 7f1997957b80 1 -- 172.21.15.40:0/3237984964 wait complete. 2023-07-18T03:30:18.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.339+0000 7f1997957b80 1 -- start start 2023-07-18T03:30:18.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.339+0000 7f1997957b80 1 Processor -- start 2023-07-18T03:30:18.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f1997957b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c4518210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f1997957b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450efe0 0x55d1c4518750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f1997957b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450fe30 0x55d1c451f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f1997957b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c43fda70 con 0x55d1c450e690 2023-07-18T03:30:18.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f1997957b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c43fd770 con 0x55d1c450fe30 2023-07-18T03:30:18.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f1997957b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c42e18d0 con 0x55d1c450efe0 2023-07-18T03:30:18.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f1997957b80 10 client.0 _ll_register_callbacks cb 0x55d1c43fb710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:18.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f198956d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c4518210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f198956d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c4518210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57036/0 (socket says 172.21.15.40:57036) 2023-07-18T03:30:18.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f198956d700 1 -- 172.21.15.40:0/971223388 learned_addr learned my addr 172.21.15.40:0/971223388 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f1989d6e700 1 --2- 172.21.15.40:0/971223388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450fe30 0x55d1c451f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f1988d6c700 1 --2- 172.21.15.40:0/971223388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450efe0 0x55d1c4518750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f198956d700 1 -- 172.21.15.40:0/971223388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450efe0 msgr2=0x55d1c4518750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f198956d700 1 --2- 172.21.15.40:0/971223388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450efe0 0x55d1c4518750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f198956d700 1 -- 172.21.15.40:0/971223388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450fe30 msgr2=0x55d1c451f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f198956d700 1 --2- 172.21.15.40:0/971223388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450fe30 0x55d1c451f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f198956d700 1 -- 172.21.15.40:0/971223388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1984107720 con 0x55d1c450e690 2023-07-18T03:30:18.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f198956d700 1 --2- 172.21.15.40:0/971223388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c4518210 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f197401da60 tx=0x7f19740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c1c6dac0c74836 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.340+0000 7f196cff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:18.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.341+0000 7f196cff9700 1 -- 172.21.15.40:0/971223388 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974004a30 con 0x55d1c450e690 2023-07-18T03:30:18.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.341+0000 7f196cff9700 1 -- 172.21.15.40:0/971223388 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1974004b90 con 0x55d1c450e690 2023-07-18T03:30:18.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.341+0000 7f1997957b80 1 -- 172.21.15.40:0/971223388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1c4456e60 con 0x55d1c450e690 2023-07-18T03:30:18.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.341+0000 7f196cff9700 1 -- 172.21.15.40:0/971223388 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1974004d50 con 0x55d1c450e690 2023-07-18T03:30:18.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.341+0000 7f1997957b80 10 client.15060 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:18.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.341+0000 7f1997957b80 1 -- 172.21.15.40:0/971223388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d1c4274c10 con 0x55d1c450e690 2023-07-18T03:30:18.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.341+0000 7f196cff9700 1 -- 172.21.15.40:0/971223388 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d1c4456e60 con 0x55d1c450e690 2023-07-18T03:30:18.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.341+0000 7f196cff9700 1 -- 172.21.15.40:0/971223388 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1974031070 con 0x55d1c450e690 2023-07-18T03:30:18.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.341+0000 7f1997957b80 10 client.15060 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:18.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.341+0000 7f1997957b80 10 client.15060 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-18T03:30:18.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.341+0000 7f1997957b80 20 client.15060 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:18.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.341+0000 7f1997957b80 1 --2- 172.21.15.40:0/971223388 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55d1c4523590 0x55d1c4522940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.341+0000 7f1997957b80 4 client.15060 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:30:18.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.341+0000 7f1997957b80 1 -- 172.21.15.40:0/971223388 --> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1c4419000 con 0x55d1c4523590 2023-07-18T03:30:18.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.341+0000 7f1988d6c700 1 --2- 172.21.15.40:0/971223388 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55d1c4523590 0x55d1c4522940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.342+0000 7f1988d6c700 1 --2- 172.21.15.40:0/971223388 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55d1c4523590 0x55d1c4522940 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e6991d4c82445c49 server_cookie=a3d23fe8f17d6995 in_seq=0 out_seq=0 2023-07-18T03:30:18.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.342+0000 7f196cff9700 10 client.15060 ms_handle_connect on v2:172.21.15.64:6836/3077792296 2023-07-18T03:30:18.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.343+0000 7f196cff9700 1 -- 172.21.15.40:0/971223388 <== mds.2 v2:172.21.15.64:6836/3077792296 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d1c4419000 con 0x55d1c4523590 2023-07-18T03:30:18.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.343+0000 7f196cff9700 10 client.15060 handle_command_reply: tid=0 2023-07-18T03:30:18.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.343+0000 7f1997957b80 1 client.15060 shutdown 2023-07-18T03:30:18.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.343+0000 7f1997957b80 1 -- 172.21.15.40:0/971223388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 msgr2=0x55d1c4518210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.343+0000 7f1997957b80 1 --2- 172.21.15.40:0/971223388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c4518210 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f197401da60 tx=0x7f19740047c0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.343+0000 7f1997957b80 1 -- 172.21.15.40:0/971223388 shutdown_connections 2023-07-18T03:30:18.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.343+0000 7f1997957b80 1 --2- 172.21.15.40:0/971223388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450fe30 0x55d1c451f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.343+0000 7f1997957b80 1 --2- 172.21.15.40:0/971223388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c4518210 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.343+0000 7f1997957b80 1 --2- 172.21.15.40:0/971223388 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55d1c4523590 0x55d1c4522940 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.343+0000 7f1997957b80 1 --2- 172.21.15.40:0/971223388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450efe0 0x55d1c4518750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.343+0000 7f196cff9700 0 client.15060 ms_handle_reset on v2:172.21.15.64:6836/3077792296 2023-07-18T03:30:18.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.343+0000 7f1997957b80 1 -- 172.21.15.40:0/971223388 >> 172.21.15.40:0/971223388 conn(0x55d1c4501d50 msgr2=0x55d1c4506860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.343+0000 7f1997957b80 1 -- 172.21.15.40:0/971223388 shutdown_connections 2023-07-18T03:30:18.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.343+0000 7f1997957b80 1 -- 172.21.15.40:0/971223388 wait complete. 2023-07-18T03:30:18.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.343+0000 7f1997957b80 20 client.15060 trim_cache size 0 max 16384 2023-07-18T03:30:18.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.361+0000 7f1997957b80 1 Processor -- start 2023-07-18T03:30:18.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.361+0000 7f1997957b80 1 -- start start 2023-07-18T03:30:18.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.361+0000 7f1997957b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c4516200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.361+0000 7f1997957b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450efe0 0x55d1c451f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.361+0000 7f1997957b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450fe30 0x55d1c451ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.361+0000 7f1997957b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c441c480 con 0x55d1c450e690 2023-07-18T03:30:18.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.361+0000 7f1997957b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c43f9d50 con 0x55d1c450efe0 2023-07-18T03:30:18.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.361+0000 7f1997957b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c45176c0 con 0x55d1c450fe30 2023-07-18T03:30:18.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.361+0000 7f198956d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c4516200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.361+0000 7f198956d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c4516200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57044/0 (socket says 172.21.15.40:57044) 2023-07-18T03:30:18.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.361+0000 7f198956d700 1 -- 172.21.15.40:0/3750098733 learned_addr learned my addr 172.21.15.40:0/3750098733 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.362+0000 7f1989d6e700 1 --2- 172.21.15.40:0/3750098733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450fe30 0x55d1c451ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.362+0000 7f1988d6c700 1 --2- 172.21.15.40:0/3750098733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450efe0 0x55d1c451f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.362+0000 7f198956d700 1 -- 172.21.15.40:0/3750098733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450fe30 msgr2=0x55d1c451ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.362+0000 7f198956d700 1 --2- 172.21.15.40:0/3750098733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450fe30 0x55d1c451ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.362+0000 7f198956d700 1 -- 172.21.15.40:0/3750098733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450efe0 msgr2=0x55d1c451f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.362+0000 7f198956d700 1 --2- 172.21.15.40:0/3750098733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450efe0 0x55d1c451f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.362+0000 7f198956d700 1 -- 172.21.15.40:0/3750098733 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1c42b2200 con 0x55d1c450e690 2023-07-18T03:30:18.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.362+0000 7f1989d6e700 1 --2- 172.21.15.40:0/3750098733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450fe30 0x55d1c451ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:18.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.362+0000 7f198956d700 1 --2- 172.21.15.40:0/3750098733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c4516200 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f19740109c0 tx=0x7f197400a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69d6955966a9fd8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.362+0000 7f196ffff700 1 -- 172.21.15.40:0/3750098733 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974033650 con 0x55d1c450e690 2023-07-18T03:30:18.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.362+0000 7f196ffff700 1 -- 172.21.15.40:0/3750098733 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f197402e070 con 0x55d1c450e690 2023-07-18T03:30:18.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.362+0000 7f196ffff700 1 -- 172.21.15.40:0/3750098733 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19740279d0 con 0x55d1c450e690 2023-07-18T03:30:18.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.363+0000 7f1997957b80 1 -- 172.21.15.40:0/3750098733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 msgr2=0x55d1c4516200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.363+0000 7f1997957b80 1 --2- 172.21.15.40:0/3750098733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c4516200 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f19740109c0 tx=0x7f197400a510 comp rx=0 tx=0).stop 2023-07-18T03:30:18.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.363+0000 7f1997957b80 1 -- 172.21.15.40:0/3750098733 shutdown_connections 2023-07-18T03:30:18.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.363+0000 7f1997957b80 1 --2- 172.21.15.40:0/3750098733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450fe30 0x55d1c451ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.363+0000 7f1997957b80 1 --2- 172.21.15.40:0/3750098733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c4516200 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.363+0000 7f1997957b80 1 --2- 172.21.15.40:0/3750098733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450efe0 0x55d1c451f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.363+0000 7f1997957b80 1 -- 172.21.15.40:0/3750098733 >> 172.21.15.40:0/3750098733 conn(0x55d1c4501d50 msgr2=0x55d1c4516fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.363+0000 7f1997957b80 1 -- 172.21.15.40:0/3750098733 shutdown_connections 2023-07-18T03:30:18.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.363+0000 7f1997957b80 1 -- 172.21.15.40:0/3750098733 wait complete. 2023-07-18T03:30:18.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.363+0000 7f1997957b80 1 -- start start 2023-07-18T03:30:18.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.363+0000 7f1997957b80 1 Processor -- start 2023-07-18T03:30:18.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f1997957b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c451b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f1997957b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450efe0 0x55d1c451b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f1997957b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450fe30 0x55d1c451e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f1997957b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c451ec80 con 0x55d1c450e690 2023-07-18T03:30:18.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f1997957b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c451edf0 con 0x55d1c450efe0 2023-07-18T03:30:18.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f1997957b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c451efb0 con 0x55d1c450fe30 2023-07-18T03:30:18.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f1997957b80 10 client.0 _ll_register_callbacks cb 0x7f197401d430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:18.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f198956d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c451b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f198956d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c451b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57050/0 (socket says 172.21.15.40:57050) 2023-07-18T03:30:18.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f198956d700 1 -- 172.21.15.40:0/2410364919 learned_addr learned my addr 172.21.15.40:0/2410364919 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f198956d700 1 -- 172.21.15.40:0/2410364919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450fe30 msgr2=0x55d1c451e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f198956d700 1 --2- 172.21.15.40:0/2410364919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450fe30 0x55d1c451e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f198956d700 1 -- 172.21.15.40:0/2410364919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450efe0 msgr2=0x55d1c451b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f198956d700 1 --2- 172.21.15.40:0/2410364919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450efe0 0x55d1c451b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f198956d700 1 -- 172.21.15.40:0/2410364919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1c4274c10 con 0x55d1c450e690 2023-07-18T03:30:18.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f198956d700 1 --2- 172.21.15.40:0/2410364919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c451b1e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1974027620 tx=0x7f1974005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ce4cee2b036022 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f196cff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:18.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f196cff9700 1 -- 172.21.15.40:0/2410364919 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974027b60 con 0x55d1c450e690 2023-07-18T03:30:18.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f196cff9700 1 -- 172.21.15.40:0/2410364919 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1974027cc0 con 0x55d1c450e690 2023-07-18T03:30:18.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.364+0000 7f1997957b80 1 -- 172.21.15.40:0/2410364919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1c43b74a0 con 0x55d1c450e690 2023-07-18T03:30:18.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.365+0000 7f196cff9700 1 -- 172.21.15.40:0/2410364919 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1974010800 con 0x55d1c450e690 2023-07-18T03:30:18.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.365+0000 7f1997957b80 10 client.15066 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:18.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.365+0000 7f1997957b80 1 -- 172.21.15.40:0/2410364919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d1c428a260 con 0x55d1c450e690 2023-07-18T03:30:18.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.365+0000 7f196cff9700 1 -- 172.21.15.40:0/2410364919 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d1c43b74a0 con 0x55d1c450e690 2023-07-18T03:30:18.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.365+0000 7f196cff9700 1 -- 172.21.15.40:0/2410364919 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f197403a070 con 0x55d1c450e690 2023-07-18T03:30:18.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.365+0000 7f1997957b80 10 client.15066 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:18.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.365+0000 7f1997957b80 10 client.15066 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-18T03:30:18.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.365+0000 7f1997957b80 20 client.15066 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:18.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.365+0000 7f1997957b80 1 --2- 172.21.15.40:0/2410364919 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55d1c457d060 0x55d1c4579970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.365+0000 7f1997957b80 4 client.15066 mds_command: new command op to 24482 tid=0[{"prefix": "status"}] 2023-07-18T03:30:18.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.365+0000 7f1997957b80 1 -- 172.21.15.40:0/2410364919 --> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d1c42acf20 con 0x55d1c457d060 2023-07-18T03:30:18.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.365+0000 7f1988d6c700 1 --2- 172.21.15.40:0/2410364919 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55d1c457d060 0x55d1c4579970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.366+0000 7f196cff9700 10 client.15066 ms_handle_connect on v2:172.21.15.64:6836/3077792296 2023-07-18T03:30:18.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.366+0000 7f1988d6c700 1 --2- 172.21.15.40:0/2410364919 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55d1c457d060 0x55d1c4579970 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b58aecc0915ab24 server_cookie=ee5f2fa01ce66ae5 in_seq=0 out_seq=0 2023-07-18T03:30:18.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.366+0000 7f196cff9700 1 -- 172.21.15.40:0/2410364919 <== mds.2 v2:172.21.15.64:6836/3077792296 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d1c42acf20 con 0x55d1c457d060 2023-07-18T03:30:18.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.366+0000 7f196cff9700 10 client.15066 handle_command_reply: tid=0 2023-07-18T03:30:18.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.366+0000 7f1997957b80 1 client.15066 shutdown 2023-07-18T03:30:18.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.366+0000 7f1997957b80 1 -- 172.21.15.40:0/2410364919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 msgr2=0x55d1c451b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.366+0000 7f1997957b80 1 --2- 172.21.15.40:0/2410364919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c451b1e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1974027620 tx=0x7f1974005610 comp rx=0 tx=0).stop 2023-07-18T03:30:18.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.366+0000 7f1997957b80 1 -- 172.21.15.40:0/2410364919 shutdown_connections 2023-07-18T03:30:18.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.366+0000 7f1997957b80 1 --2- 172.21.15.40:0/2410364919 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55d1c457d060 0x55d1c4579970 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.366+0000 7f1997957b80 1 --2- 172.21.15.40:0/2410364919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d1c450fe30 0x55d1c451e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.366+0000 7f196cff9700 0 client.15066 ms_handle_reset on v2:172.21.15.64:6836/3077792296 2023-07-18T03:30:18.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.366+0000 7f1997957b80 1 --2- 172.21.15.40:0/2410364919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d1c450e690 0x55d1c451b1e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.366+0000 7f1997957b80 1 --2- 172.21.15.40:0/2410364919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1c450efe0 0x55d1c451b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.366+0000 7f1997957b80 1 -- 172.21.15.40:0/2410364919 >> 172.21.15.40:0/2410364919 conn(0x55d1c4501d50 msgr2=0x55d1c4516200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.366+0000 7f1997957b80 1 -- 172.21.15.40:0/2410364919 shutdown_connections 2023-07-18T03:30:18.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.366+0000 7f1997957b80 1 -- 172.21.15.40:0/2410364919 wait complete. 2023-07-18T03:30:18.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.366+0000 7f1997957b80 20 client.15066 trim_cache size 0 max 16384 2023-07-18T03:30:18.374 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:30:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "3af3dba8-251a-11ee-9b34-001a4aab830c", 2023-07-18T03:30:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 2, 2023-07-18T03:30:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24482, 2023-07-18T03:30:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-07-18T03:30:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-07-18T03:30:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-07-18T03:30:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 46.769773016999999, 2023-07-18T03:30:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 27, 2023-07-18T03:30:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 81, 2023-07-18T03:30:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 81, 2023-07-18T03:30:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 66.548420880999998 2023-07-18T03:30:18.375 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:30:18.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.367+0000 7f198b7d1700 1 -- 172.21.15.40:0/1081273236 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f19700a11e0 msgr2=0x7f19700a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.367+0000 7f198b7d1700 1 --2- 172.21.15.40:0/1081273236 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f19700a11e0 0x7f19700a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1974005950 tx=0x7f197400a300 comp rx=0 tx=0).stop 2023-07-18T03:30:18.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.367+0000 7f198b7d1700 1 -- 172.21.15.40:0/1081273236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840ff580 msgr2=0x7f1984194d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.367+0000 7f198b7d1700 1 --2- 172.21.15.40:0/1081273236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840ff580 0x7f1984194d60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f198000d980 tx=0x7f198000dc90 comp rx=0 tx=0).stop 2023-07-18T03:30:18.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.367+0000 7f198b7d1700 1 -- 172.21.15.40:0/1081273236 shutdown_connections 2023-07-18T03:30:18.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.367+0000 7f198b7d1700 1 --2- 172.21.15.40:0/1081273236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19840fff80 0x7f198419bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.367+0000 7f198b7d1700 1 --2- 172.21.15.40:0/1081273236 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f19700a11e0 0x7f19700a3690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.367+0000 7f198b7d1700 1 --2- 172.21.15.40:0/1081273236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19840febd0 0x7f1984194820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.367+0000 7f198b7d1700 1 --2- 172.21.15.40:0/1081273236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19840ff580 0x7f1984194d60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.367+0000 7f198b7d1700 1 -- 172.21.15.40:0/1081273236 >> 172.21.15.40:0/1081273236 conn(0x7f19840f9770 msgr2=0x7f19840fb970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.367+0000 7f198b7d1700 1 -- 172.21.15.40:0/1081273236 shutdown_connections 2023-07-18T03:30:18.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.367+0000 7f198b7d1700 1 -- 172.21.15.40:0/1081273236 wait complete. 2023-07-18T03:30:18.378 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-18T03:30:18.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.507+0000 7fc643316700 1 -- 172.21.15.40:0/917796699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc63c0fea20 msgr2=0x7fc63c0fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.507+0000 7fc643316700 1 --2- 172.21.15.40:0/917796699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc63c0fea20 0x7fc63c0fee30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc638009990 tx=0x7fc638009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.507+0000 7fc643316700 1 -- 172.21.15.40:0/917796699 shutdown_connections 2023-07-18T03:30:18.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.507+0000 7fc643316700 1 --2- 172.21.15.40:0/917796699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc63c1005a0 0x7fc63c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.507+0000 7fc643316700 1 --2- 172.21.15.40:0/917796699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc63c0fea20 0x7fc63c0fee30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.507+0000 7fc643316700 1 --2- 172.21.15.40:0/917796699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc63c0ffc10 0x7fc63c100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.507+0000 7fc643316700 1 -- 172.21.15.40:0/917796699 >> 172.21.15.40:0/917796699 conn(0x7fc63c0fa020 msgr2=0x7fc63c0fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.507+0000 7fc643316700 1 -- 172.21.15.40:0/917796699 shutdown_connections 2023-07-18T03:30:18.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.507+0000 7fc643316700 1 -- 172.21.15.40:0/917796699 wait complete. 2023-07-18T03:30:18.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc643316700 1 Processor -- start 2023-07-18T03:30:18.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc643316700 1 -- start start 2023-07-18T03:30:18.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc643316700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc63c0fea20 0x7fc63c069190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc643316700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc63c0ffc10 0x7fc63c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc643316700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc63c1005a0 0x7fc63c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc643316700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c19d3c0 con 0x7fc63c0fea20 2023-07-18T03:30:18.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc643316700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c19d530 con 0x7fc63c1005a0 2023-07-18T03:30:18.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc643316700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c19d6f0 con 0x7fc63c0ffc10 2023-07-18T03:30:18.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc6410b2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc63c0fea20 0x7fc63c069190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc6410b2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc63c0fea20 0x7fc63c069190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57064/0 (socket says 172.21.15.40:57064) 2023-07-18T03:30:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc6410b2700 1 -- 172.21.15.40:0/264898772 learned_addr learned my addr 172.21.15.40:0/264898772 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc6418b3700 1 --2- 172.21.15.40:0/264898772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc63c1005a0 0x7fc63c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc6408b1700 1 --2- 172.21.15.40:0/264898772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc63c0ffc10 0x7fc63c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc6410b2700 1 -- 172.21.15.40:0/264898772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc63c0ffc10 msgr2=0x7fc63c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc6410b2700 1 --2- 172.21.15.40:0/264898772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc63c0ffc10 0x7fc63c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc6410b2700 1 -- 172.21.15.40:0/264898772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc63c1005a0 msgr2=0x7fc63c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc6410b2700 1 --2- 172.21.15.40:0/264898772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc63c1005a0 0x7fc63c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc6410b2700 1 -- 172.21.15.40:0/264898772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6380095f0 con 0x7fc63c0fea20 2023-07-18T03:30:18.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.508+0000 7fc6410b2700 1 --2- 172.21.15.40:0/264898772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc63c0fea20 0x7fc63c069190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc638004730 tx=0x7fc638004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.509+0000 7fc62e7fc700 1 -- 172.21.15.40:0/264898772 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63801d070 con 0x7fc63c0fea20 2023-07-18T03:30:18.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.509+0000 7fc643316700 1 -- 172.21.15.40:0/264898772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc63c19d970 con 0x7fc63c0fea20 2023-07-18T03:30:18.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.509+0000 7fc62e7fc700 1 -- 172.21.15.40:0/264898772 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc638021410 con 0x7fc63c0fea20 2023-07-18T03:30:18.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.509+0000 7fc62e7fc700 1 -- 172.21.15.40:0/264898772 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc638017650 con 0x7fc63c0fea20 2023-07-18T03:30:18.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.509+0000 7fc643316700 1 -- 172.21.15.40:0/264898772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc63c19ddf0 con 0x7fc63c0fea20 2023-07-18T03:30:18.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.511+0000 7fc62e7fc700 1 -- 172.21.15.40:0/264898772 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc638021810 con 0x7fc63c0fea20 2023-07-18T03:30:18.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.511+0000 7fc62e7fc700 1 --2- 172.21.15.40:0/264898772 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc6280a11e0 0x7fc6280a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.512+0000 7fc62e7fc700 1 -- 172.21.15.40:0/264898772 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc6380cddc0 con 0x7fc63c0fea20 2023-07-18T03:30:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.512+0000 7fc6408b1700 1 --2- 172.21.15.40:0/264898772 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc6280a11e0 0x7fc6280a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.512+0000 7fc6408b1700 1 --2- 172.21.15.40:0/264898772 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc6280a11e0 0x7fc6280a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc63c068f00 tx=0x7fc630009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc64f49cb80 1 Processor -- start 2023-07-18T03:30:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc64f49cb80 1 -- start start 2023-07-18T03:30:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc64f49cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b4480 0x55ae6c6b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc64f49cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6b5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc64f49cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc64f49cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c5b9c60 con 0x55ae6c6b5c20 2023-07-18T03:30:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc64f49cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c60c9f0 con 0x55ae6c6b4dd0 2023-07-18T03:30:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc64f49cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c605e80 con 0x55ae6c6b4480 2023-07-18T03:30:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc6418b3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc6418b3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57076/0 (socket says 172.21.15.40:57076) 2023-07-18T03:30:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc6418b3700 1 -- 172.21.15.40:0/3632579252 learned_addr learned my addr 172.21.15.40:0/3632579252 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc6410b2700 1 --2- 172.21.15.40:0/3632579252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b4480 0x55ae6c6b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc6408b1700 1 --2- 172.21.15.40:0/3632579252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc6418b3700 1 -- 172.21.15.40:0/3632579252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b4480 msgr2=0x55ae6c6b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc6418b3700 1 --2- 172.21.15.40:0/3632579252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b4480 0x55ae6c6b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc6418b3700 1 -- 172.21.15.40:0/3632579252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4dd0 msgr2=0x55ae6c6b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc6418b3700 1 --2- 172.21.15.40:0/3632579252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6b5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc6418b3700 1 -- 172.21.15.40:0/3632579252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae6c433d90 con 0x55ae6c6b5c20 2023-07-18T03:30:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.513+0000 7fc6418b3700 1 --2- 172.21.15.40:0/3632579252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6b6090 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc634011680 tx=0x7fc634009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9298c9900f51558a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.514+0000 7fc6277fe700 1 -- 172.21.15.40:0/3632579252 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63400e040 con 0x55ae6c6b5c20 2023-07-18T03:30:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.514+0000 7fc6277fe700 1 -- 172.21.15.40:0/3632579252 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc63400f040 con 0x55ae6c6b5c20 2023-07-18T03:30:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.514+0000 7fc6277fe700 1 -- 172.21.15.40:0/3632579252 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc634004860 con 0x55ae6c6b5c20 2023-07-18T03:30:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.514+0000 7fc64f49cb80 1 -- 172.21.15.40:0/3632579252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 msgr2=0x55ae6c6b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.514+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/3632579252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6b6090 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc634011680 tx=0x7fc634009920 comp rx=0 tx=0).stop 2023-07-18T03:30:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.514+0000 7fc64f49cb80 1 -- 172.21.15.40:0/3632579252 shutdown_connections 2023-07-18T03:30:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.514+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/3632579252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6b6090 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.514+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/3632579252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b4480 0x55ae6c6b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.514+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/3632579252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.514+0000 7fc64f49cb80 1 -- 172.21.15.40:0/3632579252 >> 172.21.15.40:0/3632579252 conn(0x55ae6c6a9d50 msgr2=0x55ae6c6af180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.514+0000 7fc64f49cb80 1 -- 172.21.15.40:0/3632579252 shutdown_connections 2023-07-18T03:30:18.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.514+0000 7fc64f49cb80 1 -- 172.21.15.40:0/3632579252 wait complete. 2023-07-18T03:30:18.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.515+0000 7fc64f49cb80 1 -- start start 2023-07-18T03:30:18.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.515+0000 7fc64f49cb80 1 Processor -- start 2023-07-18T03:30:18.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.515+0000 7fc64f49cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4480 0x55ae6c6be0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.515+0000 7fc64f49cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6be5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.515+0000 7fc64f49cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6c5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.515+0000 7fc64f49cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c5a5a70 con 0x55ae6c6b5c20 2023-07-18T03:30:18.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.515+0000 7fc64f49cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c5a5770 con 0x55ae6c6b4480 2023-07-18T03:30:18.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.515+0000 7fc64f49cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c4898d0 con 0x55ae6c6b4dd0 2023-07-18T03:30:18.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.515+0000 7fc64f49cb80 10 client.0 _ll_register_callbacks cb 0x55ae6c5a3710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:18.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.515+0000 7fc6418b3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.515+0000 7fc6418b3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6c5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57082/0 (socket says 172.21.15.40:57082) 2023-07-18T03:30:18.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.515+0000 7fc6418b3700 1 -- 172.21.15.40:0/3950590020 learned_addr learned my addr 172.21.15.40:0/3950590020 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.515+0000 7fc6410b2700 1 --2- 172.21.15.40:0/3950590020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4480 0x55ae6c6be0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.515+0000 7fc6408b1700 1 --2- 172.21.15.40:0/3950590020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6be5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.516+0000 7fc6418b3700 1 -- 172.21.15.40:0/3950590020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b4dd0 msgr2=0x55ae6c6be5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.516+0000 7fc6418b3700 1 --2- 172.21.15.40:0/3950590020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6be5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.516+0000 7fc6418b3700 1 -- 172.21.15.40:0/3950590020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4480 msgr2=0x55ae6c6be0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.516+0000 7fc6418b3700 1 --2- 172.21.15.40:0/3950590020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4480 0x55ae6c6be0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.516+0000 7fc6418b3700 1 -- 172.21.15.40:0/3950590020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc634011330 con 0x55ae6c6b5c20 2023-07-18T03:30:18.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.516+0000 7fc6418b3700 1 --2- 172.21.15.40:0/3950590020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6c5620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc634009690 tx=0x7fc634009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9424a80e949a6331 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.516+0000 7fc624ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:18.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.516+0000 7fc624ff9700 1 -- 172.21.15.40:0/3950590020 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63400e070 con 0x55ae6c6b5c20 2023-07-18T03:30:18.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.516+0000 7fc624ff9700 1 -- 172.21.15.40:0/3950590020 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc634003c10 con 0x55ae6c6b5c20 2023-07-18T03:30:18.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.516+0000 7fc64f49cb80 1 -- 172.21.15.40:0/3950590020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae6c55f4a0 con 0x55ae6c6b5c20 2023-07-18T03:30:18.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.516+0000 7fc624ff9700 1 -- 172.21.15.40:0/3950590020 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc634003dd0 con 0x55ae6c6b5c20 2023-07-18T03:30:18.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.516+0000 7fc64f49cb80 10 client.15078 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:18.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.516+0000 7fc64f49cb80 1 -- 172.21.15.40:0/3950590020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ae6c41cc10 con 0x55ae6c6b5c20 2023-07-18T03:30:18.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.516+0000 7fc624ff9700 1 -- 172.21.15.40:0/3950590020 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ae6c55f4a0 con 0x55ae6c6b5c20 2023-07-18T03:30:18.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.517+0000 7fc624ff9700 1 -- 172.21.15.40:0/3950590020 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc634015450 con 0x55ae6c6b5c20 2023-07-18T03:30:18.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.517+0000 7fc64f49cb80 10 client.15078 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:18.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.517+0000 7fc64f49cb80 10 client.15078 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-18T03:30:18.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.517+0000 7fc64f49cb80 20 client.15078 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:18.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.517+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/3950590020 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x55ae6c6c92c0 0x55ae6c6c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.517+0000 7fc64f49cb80 4 client.15078 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:30:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.517+0000 7fc64f49cb80 1 -- 172.21.15.40:0/3950590020 --> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae6c5fee60 con 0x55ae6c6c92c0 2023-07-18T03:30:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.517+0000 7fc6410b2700 1 --2- 172.21.15.40:0/3950590020 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x55ae6c6c92c0 0x55ae6c6c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.517+0000 7fc624ff9700 10 client.15078 ms_handle_connect on v2:172.21.15.134:6838/3192254101 2023-07-18T03:30:18.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.517+0000 7fc6410b2700 1 --2- 172.21.15.40:0/3950590020 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x55ae6c6c92c0 0x55ae6c6c8720 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2481d46cfd7441f server_cookie=a03f98425d4e7aa4 in_seq=0 out_seq=0 2023-07-18T03:30:18.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.518+0000 7fc624ff9700 1 -- 172.21.15.40:0/3950590020 <== mds.1 v2:172.21.15.134:6838/3192254101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ae6c5fee60 con 0x55ae6c6c92c0 2023-07-18T03:30:18.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.518+0000 7fc624ff9700 10 client.15078 handle_command_reply: tid=0 2023-07-18T03:30:18.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.518+0000 7fc64f49cb80 1 client.15078 shutdown 2023-07-18T03:30:18.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.518+0000 7fc64f49cb80 1 -- 172.21.15.40:0/3950590020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 msgr2=0x55ae6c6c5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.518+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/3950590020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6c5620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc634009690 tx=0x7fc634009510 comp rx=0 tx=0).stop 2023-07-18T03:30:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.518+0000 7fc64f49cb80 1 -- 172.21.15.40:0/3950590020 shutdown_connections 2023-07-18T03:30:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.518+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/3950590020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6c5620 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.518+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/3950590020 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x55ae6c6c92c0 0x55ae6c6c8720 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.518+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/3950590020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4480 0x55ae6c6be0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.518+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/3950590020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6be5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.518+0000 7fc624ff9700 0 client.15078 ms_handle_reset on v2:172.21.15.134:6838/3192254101 2023-07-18T03:30:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.518+0000 7fc64f49cb80 1 -- 172.21.15.40:0/3950590020 >> 172.21.15.40:0/3950590020 conn(0x55ae6c6a9d50 msgr2=0x55ae6c6ae860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.518+0000 7fc64f49cb80 1 -- 172.21.15.40:0/3950590020 shutdown_connections 2023-07-18T03:30:18.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.518+0000 7fc64f49cb80 1 -- 172.21.15.40:0/3950590020 wait complete. 2023-07-18T03:30:18.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.519+0000 7fc64f49cb80 20 client.15078 trim_cache size 0 max 16384 2023-07-18T03:30:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.536+0000 7fc64f49cb80 1 Processor -- start 2023-07-18T03:30:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.536+0000 7fc64f49cb80 1 -- start start 2023-07-18T03:30:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc64f49cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b4480 0x55ae6c6d2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc64f49cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc64f49cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc64f49cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c55f4d0 con 0x55ae6c6b5c20 2023-07-18T03:30:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc64f49cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c6bf2c0 con 0x55ae6c6b4dd0 2023-07-18T03:30:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc64f49cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c495de0 con 0x55ae6c6b4480 2023-07-18T03:30:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc6418b3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc6418b3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57090/0 (socket says 172.21.15.40:57090) 2023-07-18T03:30:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc6418b3700 1 -- 172.21.15.40:0/2950890979 learned_addr learned my addr 172.21.15.40:0/2950890979 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc6408b1700 1 --2- 172.21.15.40:0/2950890979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc6410b2700 1 --2- 172.21.15.40:0/2950890979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b4480 0x55ae6c6d2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc6418b3700 1 -- 172.21.15.40:0/2950890979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b4480 msgr2=0x55ae6c6d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc6418b3700 1 --2- 172.21.15.40:0/2950890979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b4480 0x55ae6c6d2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc6418b3700 1 -- 172.21.15.40:0/2950890979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4dd0 msgr2=0x55ae6c6c55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc6418b3700 1 --2- 172.21.15.40:0/2950890979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6c55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc6418b3700 1 -- 172.21.15.40:0/2950890979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae6c45a200 con 0x55ae6c6b5c20 2023-07-18T03:30:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc6418b3700 1 --2- 172.21.15.40:0/2950890979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6c5df0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc6340157b0 tx=0x7fc634005a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef319ebb88a5827 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.537+0000 7fc627fff700 1 -- 172.21.15.40:0/2950890979 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63400e040 con 0x55ae6c6b5c20 2023-07-18T03:30:18.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.538+0000 7fc627fff700 1 -- 172.21.15.40:0/2950890979 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc634015450 con 0x55ae6c6b5c20 2023-07-18T03:30:18.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.538+0000 7fc627fff700 1 -- 172.21.15.40:0/2950890979 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63401e630 con 0x55ae6c6b5c20 2023-07-18T03:30:18.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.538+0000 7fc64f49cb80 1 -- 172.21.15.40:0/2950890979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 msgr2=0x55ae6c6c5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.538+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/2950890979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6c5df0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc6340157b0 tx=0x7fc634005a30 comp rx=0 tx=0).stop 2023-07-18T03:30:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.538+0000 7fc64f49cb80 1 -- 172.21.15.40:0/2950890979 shutdown_connections 2023-07-18T03:30:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.538+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/2950890979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6c5df0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.538+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/2950890979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b4480 0x55ae6c6d2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.538+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/2950890979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.538+0000 7fc64f49cb80 1 -- 172.21.15.40:0/2950890979 >> 172.21.15.40:0/2950890979 conn(0x55ae6c6a9d50 msgr2=0x55ae6c6ab2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.538+0000 7fc64f49cb80 1 -- 172.21.15.40:0/2950890979 shutdown_connections 2023-07-18T03:30:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.538+0000 7fc64f49cb80 1 -- 172.21.15.40:0/2950890979 wait complete. 2023-07-18T03:30:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.538+0000 7fc64f49cb80 1 -- start start 2023-07-18T03:30:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.538+0000 7fc64f49cb80 1 Processor -- start 2023-07-18T03:30:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc64f49cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4480 0x55ae6c6c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc64f49cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc64f49cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc64f49cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c6c5920 con 0x55ae6c6b4dd0 2023-07-18T03:30:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc64f49cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c71d160 con 0x55ae6c6b4480 2023-07-18T03:30:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc64f49cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c71d2a0 con 0x55ae6c6b5c20 2023-07-18T03:30:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc64f49cb80 10 client.0 _ll_register_callbacks cb 0x7fc638005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc6408b1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc6408b1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6c4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57094/0 (socket says 172.21.15.40:57094) 2023-07-18T03:30:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc6408b1700 1 -- 172.21.15.40:0/1370209431 learned_addr learned my addr 172.21.15.40:0/1370209431 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc6410b2700 1 --2- 172.21.15.40:0/1370209431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4480 0x55ae6c6c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc6418b3700 1 --2- 172.21.15.40:0/1370209431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc6408b1700 1 -- 172.21.15.40:0/1370209431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b5c20 msgr2=0x55ae6c6c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc6408b1700 1 --2- 172.21.15.40:0/1370209431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc6408b1700 1 -- 172.21.15.40:0/1370209431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4480 msgr2=0x55ae6c6c27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc6408b1700 1 --2- 172.21.15.40:0/1370209431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4480 0x55ae6c6c27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc6408b1700 1 -- 172.21.15.40:0/1370209431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae6c41cc10 con 0x55ae6c6b4dd0 2023-07-18T03:30:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc6418b3700 1 --2- 172.21.15.40:0/1370209431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc6408b1700 1 --2- 172.21.15.40:0/1370209431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6c4d10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc630012940 tx=0x7fc6300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eece62eca44b64f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.539+0000 7fc624ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.540+0000 7fc624ff9700 1 -- 172.21.15.40:0/1370209431 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc630025070 con 0x55ae6c6b4dd0 2023-07-18T03:30:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.540+0000 7fc624ff9700 1 -- 172.21.15.40:0/1370209431 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc630018070 con 0x55ae6c6b4dd0 2023-07-18T03:30:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.540+0000 7fc64f49cb80 1 -- 172.21.15.40:0/1370209431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc63400f040 con 0x55ae6c6b4dd0 2023-07-18T03:30:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.540+0000 7fc624ff9700 1 -- 172.21.15.40:0/1370209431 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc630004020 con 0x55ae6c6b4dd0 2023-07-18T03:30:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.540+0000 7fc64f49cb80 10 client.15084 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.540+0000 7fc64f49cb80 1 -- 172.21.15.40:0/1370209431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ae6c432260 con 0x55ae6c6b4dd0 2023-07-18T03:30:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.540+0000 7fc624ff9700 1 -- 172.21.15.40:0/1370209431 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc63400f040 con 0x55ae6c6b4dd0 2023-07-18T03:30:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.540+0000 7fc624ff9700 1 -- 172.21.15.40:0/1370209431 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc630003680 con 0x55ae6c6b4dd0 2023-07-18T03:30:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.540+0000 7fc64f49cb80 10 client.15084 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.540+0000 7fc64f49cb80 10 client.15084 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-18T03:30:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.540+0000 7fc64f49cb80 20 client.15084 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.540+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/1370209431 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x55ae6c722120 0x55ae6c71ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.540+0000 7fc64f49cb80 4 client.15084 mds_command: new command op to 24523 tid=0[{"prefix": "status"}] 2023-07-18T03:30:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.540+0000 7fc64f49cb80 1 -- 172.21.15.40:0/1370209431 --> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ae6c5c1000 con 0x55ae6c722120 2023-07-18T03:30:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.540+0000 7fc6410b2700 1 --2- 172.21.15.40:0/1370209431 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x55ae6c722120 0x55ae6c71ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.541+0000 7fc624ff9700 10 client.15084 ms_handle_connect on v2:172.21.15.134:6838/3192254101 2023-07-18T03:30:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.541+0000 7fc6410b2700 1 --2- 172.21.15.40:0/1370209431 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x55ae6c722120 0x55ae6c71ea30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=632e17c3c17d96fd server_cookie=2356c0db1bc477da in_seq=0 out_seq=0 2023-07-18T03:30:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.541+0000 7fc624ff9700 1 -- 172.21.15.40:0/1370209431 <== mds.1 v2:172.21.15.134:6838/3192254101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ae6c5c1000 con 0x55ae6c722120 2023-07-18T03:30:18.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.541+0000 7fc624ff9700 10 client.15084 handle_command_reply: tid=0 2023-07-18T03:30:18.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.541+0000 7fc64f49cb80 1 client.15084 shutdown 2023-07-18T03:30:18.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.541+0000 7fc64f49cb80 1 -- 172.21.15.40:0/1370209431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b4dd0 msgr2=0x55ae6c6c4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.541+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/1370209431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6c4d10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc630012940 tx=0x7fc6300047c0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.541+0000 7fc64f49cb80 1 -- 172.21.15.40:0/1370209431 shutdown_connections 2023-07-18T03:30:18.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.541+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/1370209431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ae6c6b5c20 0x55ae6c6c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.541+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/1370209431 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x55ae6c722120 0x55ae6c71ea30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.541+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/1370209431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae6c6b4480 0x55ae6c6c27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.541+0000 7fc64f49cb80 1 --2- 172.21.15.40:0/1370209431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ae6c6b4dd0 0x55ae6c6c4d10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.541+0000 7fc624ff9700 0 client.15084 ms_handle_reset on v2:172.21.15.134:6838/3192254101 2023-07-18T03:30:18.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.541+0000 7fc64f49cb80 1 -- 172.21.15.40:0/1370209431 >> 172.21.15.40:0/1370209431 conn(0x55ae6c6a9d50 msgr2=0x55ae6c6d2030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.541+0000 7fc64f49cb80 1 -- 172.21.15.40:0/1370209431 shutdown_connections 2023-07-18T03:30:18.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.541+0000 7fc64f49cb80 1 -- 172.21.15.40:0/1370209431 wait complete. 2023-07-18T03:30:18.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.541+0000 7fc64f49cb80 20 client.15084 trim_cache size 0 max 16384 2023-07-18T03:30:18.551 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:30:18.551 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "3af3dba8-251a-11ee-9b34-001a4aab830c", 2023-07-18T03:30:18.551 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 1, 2023-07-18T03:30:18.551 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24523, 2023-07-18T03:30:18.551 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-07-18T03:30:18.551 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-07-18T03:30:18.551 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-07-18T03:30:18.552 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 47.925562380999999, 2023-07-18T03:30:18.552 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 27, 2023-07-18T03:30:18.552 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 81, 2023-07-18T03:30:18.552 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 81, 2023-07-18T03:30:18.552 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 64.502181069000002 2023-07-18T03:30:18.552 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:30:18.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.542+0000 7fc643316700 1 -- 172.21.15.40:0/264898772 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc6280a11e0 msgr2=0x7fc6280a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.542+0000 7fc643316700 1 --2- 172.21.15.40:0/264898772 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc6280a11e0 0x7fc6280a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc63c068f00 tx=0x7fc630009260 comp rx=0 tx=0).stop 2023-07-18T03:30:18.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.542+0000 7fc643316700 1 -- 172.21.15.40:0/264898772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc63c0fea20 msgr2=0x7fc63c069190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.542+0000 7fc643316700 1 --2- 172.21.15.40:0/264898772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc63c0fea20 0x7fc63c069190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc638004730 tx=0x7fc638004760 comp rx=0 tx=0).stop 2023-07-18T03:30:18.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.542+0000 7fc643316700 1 -- 172.21.15.40:0/264898772 shutdown_connections 2023-07-18T03:30:18.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.542+0000 7fc643316700 1 --2- 172.21.15.40:0/264898772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc63c1005a0 0x7fc63c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.542+0000 7fc643316700 1 --2- 172.21.15.40:0/264898772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc63c0fea20 0x7fc63c069190 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.542+0000 7fc643316700 1 --2- 172.21.15.40:0/264898772 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc6280a11e0 0x7fc6280a3690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.542+0000 7fc643316700 1 --2- 172.21.15.40:0/264898772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc63c0ffc10 0x7fc63c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.542+0000 7fc643316700 1 -- 172.21.15.40:0/264898772 >> 172.21.15.40:0/264898772 conn(0x7fc63c0fa020 msgr2=0x7fc63c1030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.542+0000 7fc643316700 1 -- 172.21.15.40:0/264898772 shutdown_connections 2023-07-18T03:30:18.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.542+0000 7fc643316700 1 -- 172.21.15.40:0/264898772 wait complete. 2023-07-18T03:30:18.556 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-18T03:30:18.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.685+0000 7ff6e87ce700 1 -- 172.21.15.40:0/2931461651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6e0100560 msgr2=0x7ff6e0106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.685+0000 7ff6e87ce700 1 --2- 172.21.15.40:0/2931461651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6e0100560 0x7ff6e0106e20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800b0a0 tx=0x7ff6d800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.686+0000 7ff6e87ce700 1 -- 172.21.15.40:0/2931461651 shutdown_connections 2023-07-18T03:30:18.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.686+0000 7ff6e87ce700 1 --2- 172.21.15.40:0/2931461651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6e0100560 0x7ff6e0106e20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.686+0000 7ff6e87ce700 1 --2- 172.21.15.40:0/2931461651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6e00fe9e0 0x7ff6e00fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.686+0000 7ff6e87ce700 1 --2- 172.21.15.40:0/2931461651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6e00ffbd0 0x7ff6e0100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.686+0000 7ff6e87ce700 1 -- 172.21.15.40:0/2931461651 >> 172.21.15.40:0/2931461651 conn(0x7ff6e00fa000 msgr2=0x7ff6e00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.686+0000 7ff6e87ce700 1 -- 172.21.15.40:0/2931461651 shutdown_connections 2023-07-18T03:30:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.686+0000 7ff6e87ce700 1 -- 172.21.15.40:0/2931461651 wait complete. 2023-07-18T03:30:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.686+0000 7ff6e87ce700 1 Processor -- start 2023-07-18T03:30:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.686+0000 7ff6e87ce700 1 -- start start 2023-07-18T03:30:18.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.686+0000 7ff6e87ce700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6e00fe9e0 0x7ff6e0195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.686+0000 7ff6e87ce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6e00ffbd0 0x7ff6e0195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.686+0000 7ff6e87ce700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6e0100560 0x7ff6e019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.686+0000 7ff6e87ce700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e019d4f0 con 0x7ff6e00fe9e0 2023-07-18T03:30:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.686+0000 7ff6e87ce700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e019d630 con 0x7ff6e00ffbd0 2023-07-18T03:30:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.686+0000 7ff6e87ce700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e019d7c0 con 0x7ff6e0100560 2023-07-18T03:30:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.687+0000 7ff6e656a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6e00fe9e0 0x7ff6e0195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.687+0000 7ff6e656a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6e00fe9e0 0x7ff6e0195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57106/0 (socket says 172.21.15.40:57106) 2023-07-18T03:30:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.687+0000 7ff6e656a700 1 -- 172.21.15.40:0/3710346101 learned_addr learned my addr 172.21.15.40:0/3710346101 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.687+0000 7ff6e6d6b700 1 --2- 172.21.15.40:0/3710346101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6e0100560 0x7ff6e019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.687+0000 7ff6e5d69700 1 --2- 172.21.15.40:0/3710346101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6e00ffbd0 0x7ff6e0195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.687+0000 7ff6e656a700 1 -- 172.21.15.40:0/3710346101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6e0100560 msgr2=0x7ff6e019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.687+0000 7ff6e656a700 1 --2- 172.21.15.40:0/3710346101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6e0100560 0x7ff6e019cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.687+0000 7ff6e656a700 1 -- 172.21.15.40:0/3710346101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6e00ffbd0 msgr2=0x7ff6e0195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.687+0000 7ff6e656a700 1 --2- 172.21.15.40:0/3710346101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6e00ffbd0 0x7ff6e0195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.687+0000 7ff6e656a700 1 -- 172.21.15.40:0/3710346101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d8009d00 con 0x7ff6e00fe9e0 2023-07-18T03:30:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.687+0000 7ff6e656a700 1 --2- 172.21.15.40:0/3710346101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6e00fe9e0 0x7ff6e0195970 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00d980 tx=0x7ff6dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.687+0000 7ff6d77fe700 1 -- 172.21.15.40:0/3710346101 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc004020 con 0x7ff6e00fe9e0 2023-07-18T03:30:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.687+0000 7ff6d77fe700 1 -- 172.21.15.40:0/3710346101 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6dc00f690 con 0x7ff6e00fe9e0 2023-07-18T03:30:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.687+0000 7ff6e87ce700 1 -- 172.21.15.40:0/3710346101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e019daa0 con 0x7ff6e00fe9e0 2023-07-18T03:30:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.687+0000 7ff6d77fe700 1 -- 172.21.15.40:0/3710346101 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc003a60 con 0x7ff6e00fe9e0 2023-07-18T03:30:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.687+0000 7ff6e87ce700 1 -- 172.21.15.40:0/3710346101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e019dfb0 con 0x7ff6e00fe9e0 2023-07-18T03:30:18.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.689+0000 7ff6d77fe700 1 -- 172.21.15.40:0/3710346101 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff6dc00fb00 con 0x7ff6e00fe9e0 2023-07-18T03:30:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.690+0000 7ff6d77fe700 1 --2- 172.21.15.40:0/3710346101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff6cc0a1330 0x7ff6cc0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.690+0000 7ff6d77fe700 1 -- 172.21.15.40:0/3710346101 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff6dc0cc4c0 con 0x7ff6e00fe9e0 2023-07-18T03:30:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.690+0000 7ff6e5d69700 1 --2- 172.21.15.40:0/3710346101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff6cc0a1330 0x7ff6cc0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.691+0000 7ff6e5d69700 1 --2- 172.21.15.40:0/3710346101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff6cc0a1330 0x7ff6cc0a37e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0005950 tx=0x7ff6d000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.691+0000 7ff6f4954b80 1 Processor -- start 2023-07-18T03:30:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.691+0000 7ff6f4954b80 1 -- start start 2023-07-18T03:30:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.691+0000 7ff6f4954b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e09890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.691+0000 7ff6f4954b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e09dd0 0x562880e0a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.691+0000 7ff6f4954b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e0ac20 0x562880e0b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.691+0000 7ff6f4954b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x562880d0ec60 con 0x562880e0ac20 2023-07-18T03:30:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.691+0000 7ff6f4954b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562880d619f0 con 0x562880e09dd0 2023-07-18T03:30:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.691+0000 7ff6f4954b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562880d5ae80 con 0x562880e09480 2023-07-18T03:30:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.691+0000 7ff6e6d6b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e0ac20 0x562880e0b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.691+0000 7ff6e6d6b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e0ac20 0x562880e0b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57118/0 (socket says 172.21.15.40:57118) 2023-07-18T03:30:18.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.691+0000 7ff6e6d6b700 1 -- 172.21.15.40:0/3056371488 learned_addr learned my addr 172.21.15.40:0/3056371488 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.691+0000 7ff6e656a700 1 --2- 172.21.15.40:0/3056371488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e09890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.691+0000 7ff6e5d69700 1 --2- 172.21.15.40:0/3056371488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e09dd0 0x562880e0a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.692+0000 7ff6e6d6b700 1 -- 172.21.15.40:0/3056371488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 msgr2=0x562880e09890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.692+0000 7ff6e6d6b700 1 --2- 172.21.15.40:0/3056371488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e09890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.692+0000 7ff6e6d6b700 1 -- 172.21.15.40:0/3056371488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e09dd0 msgr2=0x562880e0a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.692+0000 7ff6e6d6b700 1 --2- 172.21.15.40:0/3056371488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e09dd0 0x562880e0a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.692+0000 7ff6e6d6b700 1 -- 172.21.15.40:0/3056371488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562880b88d90 con 0x562880e0ac20 2023-07-18T03:30:18.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.692+0000 7ff6e656a700 1 --2- 172.21.15.40:0/3056371488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e09890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:18.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.692+0000 7ff6e6d6b700 1 --2- 172.21.15.40:0/3056371488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e0ac20 0x562880e0b090 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8007a60 tx=0x7ff6d8007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e461955bf3ac5fa server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.692+0000 7ff6d4ff9700 1 -- 172.21.15.40:0/3056371488 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d800e040 con 0x562880e0ac20 2023-07-18T03:30:18.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.692+0000 7ff6d4ff9700 1 -- 172.21.15.40:0/3056371488 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6d8007830 con 0x562880e0ac20 2023-07-18T03:30:18.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.692+0000 7ff6d4ff9700 1 -- 172.21.15.40:0/3056371488 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d801ca00 con 0x562880e0ac20 2023-07-18T03:30:18.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.693+0000 7ff6f4954b80 1 -- 172.21.15.40:0/3056371488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e0ac20 msgr2=0x562880e0b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.693+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/3056371488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e0ac20 0x562880e0b090 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8007a60 tx=0x7ff6d8007590 comp rx=0 tx=0).stop 2023-07-18T03:30:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.693+0000 7ff6f4954b80 1 -- 172.21.15.40:0/3056371488 shutdown_connections 2023-07-18T03:30:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.693+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/3056371488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e0ac20 0x562880e0b090 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.693+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/3056371488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e09890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.693+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/3056371488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e09dd0 0x562880e0a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.693+0000 7ff6f4954b80 1 -- 172.21.15.40:0/3056371488 >> 172.21.15.40:0/3056371488 conn(0x562880dfed50 msgr2=0x562880e04180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.693+0000 7ff6f4954b80 1 -- 172.21.15.40:0/3056371488 shutdown_connections 2023-07-18T03:30:18.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.693+0000 7ff6f4954b80 1 -- 172.21.15.40:0/3056371488 wait complete. 2023-07-18T03:30:18.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.693+0000 7ff6f4954b80 1 -- start start 2023-07-18T03:30:18.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.693+0000 7ff6f4954b80 1 Processor -- start 2023-07-18T03:30:18.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.693+0000 7ff6f4954b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e13040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6f4954b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e09dd0 0x562880e13580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6f4954b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e0ac20 0x562880e1a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6f4954b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x562880cfaa70 con 0x562880e09dd0 2023-07-18T03:30:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6f4954b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562880cfa770 con 0x562880e0ac20 2023-07-18T03:30:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6f4954b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562880bde8d0 con 0x562880e09480 2023-07-18T03:30:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6f4954b80 10 client.0 _ll_register_callbacks cb 0x562880cf8710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6e5d69700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e09dd0 0x562880e13580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6e5d69700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e09dd0 0x562880e13580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57122/0 (socket says 172.21.15.40:57122) 2023-07-18T03:30:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6e5d69700 1 -- 172.21.15.40:0/2967607471 learned_addr learned my addr 172.21.15.40:0/2967607471 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6e656a700 1 --2- 172.21.15.40:0/2967607471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e13040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6e5d69700 1 -- 172.21.15.40:0/2967607471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 msgr2=0x562880e13040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6e5d69700 1 --2- 172.21.15.40:0/2967607471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e13040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6e5d69700 1 -- 172.21.15.40:0/2967607471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e0ac20 msgr2=0x562880e1a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6e5d69700 1 --2- 172.21.15.40:0/2967607471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e0ac20 0x562880e1a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6e5d69700 1 -- 172.21.15.40:0/2967607471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e0107750 con 0x562880e09dd0 2023-07-18T03:30:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6e5d69700 1 --2- 172.21.15.40:0/2967607471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e09dd0 0x562880e13580 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff6d00116c0 tx=0x7ff6d00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402240f59125afd9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:18.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6c9ffb700 1 -- 172.21.15.40:0/2967607471 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0004a30 con 0x562880e09dd0 2023-07-18T03:30:18.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6c9ffb700 1 -- 172.21.15.40:0/2967607471 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff6d0017070 con 0x562880e09dd0 2023-07-18T03:30:18.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6f4954b80 1 -- 172.21.15.40:0/2967607471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562880cb44a0 con 0x562880e09dd0 2023-07-18T03:30:18.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6c9ffb700 1 -- 172.21.15.40:0/2967607471 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6d0004b90 con 0x562880e09dd0 2023-07-18T03:30:18.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6f4954b80 10 client.15096 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:18.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.694+0000 7ff6f4954b80 1 -- 172.21.15.40:0/2967607471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562880b71c10 con 0x562880e09dd0 2023-07-18T03:30:18.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.695+0000 7ff6c9ffb700 1 -- 172.21.15.40:0/2967607471 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562880cb44a0 con 0x562880e09dd0 2023-07-18T03:30:18.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.695+0000 7ff6c9ffb700 1 -- 172.21.15.40:0/2967607471 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff6d001f760 con 0x562880e09dd0 2023-07-18T03:30:18.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.695+0000 7ff6f4954b80 10 client.15096 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:18.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.695+0000 7ff6f4954b80 10 client.15096 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-18T03:30:18.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.695+0000 7ff6f4954b80 20 client.15096 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:18.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.695+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/2967607471 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x562880e1e3b0 0x562880e1d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.695+0000 7ff6f4954b80 4 client.15096 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:30:18.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.695+0000 7ff6f4954b80 1 -- 172.21.15.40:0/2967607471 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562880d53e60 con 0x562880e1e3b0 2023-07-18T03:30:18.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.695+0000 7ff6e656a700 1 --2- 172.21.15.40:0/2967607471 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x562880e1e3b0 0x562880e1d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.695+0000 7ff6e656a700 1 --2- 172.21.15.40:0/2967607471 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x562880e1e3b0 0x562880e1d720 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b56ad02b1968f4b5 server_cookie=72cae01a7f541527 in_seq=0 out_seq=0 2023-07-18T03:30:18.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.695+0000 7ff6c9ffb700 10 client.15096 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:30:18.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.696+0000 7ff6c9ffb700 1 -- 172.21.15.40:0/2967607471 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562880d53e60 con 0x562880e1e3b0 2023-07-18T03:30:18.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.696+0000 7ff6c9ffb700 10 client.15096 handle_command_reply: tid=0 2023-07-18T03:30:18.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.697+0000 7ff6f4954b80 1 client.15096 shutdown 2023-07-18T03:30:18.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.697+0000 7ff6f4954b80 1 -- 172.21.15.40:0/2967607471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e09dd0 msgr2=0x562880e13580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.697+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/2967607471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e09dd0 0x562880e13580 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff6d00116c0 tx=0x7ff6d00047c0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.697+0000 7ff6f4954b80 1 -- 172.21.15.40:0/2967607471 shutdown_connections 2023-07-18T03:30:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.697+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/2967607471 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x562880e1e3b0 0x562880e1d720 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.697+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/2967607471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e0ac20 0x562880e1a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.697+0000 7ff6c9ffb700 0 client.15096 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:30:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.697+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/2967607471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e13040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.697+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/2967607471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e09dd0 0x562880e13580 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.697+0000 7ff6f4954b80 1 -- 172.21.15.40:0/2967607471 >> 172.21.15.40:0/2967607471 conn(0x562880dfed50 msgr2=0x562880e03860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.697+0000 7ff6f4954b80 1 -- 172.21.15.40:0/2967607471 shutdown_connections 2023-07-18T03:30:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.697+0000 7ff6f4954b80 1 -- 172.21.15.40:0/2967607471 wait complete. 2023-07-18T03:30:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.697+0000 7ff6f4954b80 20 client.15096 trim_cache size 0 max 16384 2023-07-18T03:30:18.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6f4954b80 1 Processor -- start 2023-07-18T03:30:18.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6f4954b80 1 -- start start 2023-07-18T03:30:18.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6f4954b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e1adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6f4954b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e09dd0 0x562880e1a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6f4954b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e0ac20 0x562880e10f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6f4954b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x562880d29790 con 0x562880e09dd0 2023-07-18T03:30:18.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6f4954b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562880beade0 con 0x562880e0ac20 2023-07-18T03:30:18.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6f4954b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562880d19480 con 0x562880e09480 2023-07-18T03:30:18.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6e5d69700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e09dd0 0x562880e1a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6e5d69700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e09dd0 0x562880e1a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57130/0 (socket says 172.21.15.40:57130) 2023-07-18T03:30:18.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6e5d69700 1 -- 172.21.15.40:0/4164535350 learned_addr learned my addr 172.21.15.40:0/4164535350 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6e6d6b700 1 --2- 172.21.15.40:0/4164535350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e0ac20 0x562880e10f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6e5d69700 1 -- 172.21.15.40:0/4164535350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 msgr2=0x562880e1adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6e656a700 1 --2- 172.21.15.40:0/4164535350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e1adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6e5d69700 1 --2- 172.21.15.40:0/4164535350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e1adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6e5d69700 1 -- 172.21.15.40:0/4164535350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e0ac20 msgr2=0x562880e10f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6e5d69700 1 --2- 172.21.15.40:0/4164535350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e0ac20 0x562880e10f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6e5d69700 1 -- 172.21.15.40:0/4164535350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562880baf200 con 0x562880e09dd0 2023-07-18T03:30:18.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6e6d6b700 1 --2- 172.21.15.40:0/4164535350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e0ac20 0x562880e10f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:18.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6e656a700 1 --2- 172.21.15.40:0/4164535350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e1adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:18.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.715+0000 7ff6e5d69700 1 --2- 172.21.15.40:0/4164535350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e09dd0 0x562880e1a580 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0000c00 tx=0x7ff6d0010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abed71c6b3de346c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.716+0000 7ff6d57fa700 1 -- 172.21.15.40:0/4164535350 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d001f840 con 0x562880e09dd0 2023-07-18T03:30:18.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.716+0000 7ff6d57fa700 1 -- 172.21.15.40:0/4164535350 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6d0015070 con 0x562880e09dd0 2023-07-18T03:30:18.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.716+0000 7ff6d57fa700 1 -- 172.21.15.40:0/4164535350 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0005720 con 0x562880e09dd0 2023-07-18T03:30:18.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.716+0000 7ff6f4954b80 1 -- 172.21.15.40:0/4164535350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e09dd0 msgr2=0x562880e1a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.716+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/4164535350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e09dd0 0x562880e1a580 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0000c00 tx=0x7ff6d0010fa0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.716+0000 7ff6f4954b80 1 -- 172.21.15.40:0/4164535350 shutdown_connections 2023-07-18T03:30:18.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.716+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/4164535350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e0ac20 0x562880e10f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.716+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/4164535350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e1adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.716+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/4164535350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e09dd0 0x562880e1a580 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.716+0000 7ff6f4954b80 1 -- 172.21.15.40:0/4164535350 >> 172.21.15.40:0/4164535350 conn(0x562880dfed50 msgr2=0x562880e27140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.716+0000 7ff6f4954b80 1 -- 172.21.15.40:0/4164535350 shutdown_connections 2023-07-18T03:30:18.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.716+0000 7ff6f4954b80 1 -- 172.21.15.40:0/4164535350 wait complete. 2023-07-18T03:30:18.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.716+0000 7ff6f4954b80 1 -- start start 2023-07-18T03:30:18.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6f4954b80 1 Processor -- start 2023-07-18T03:30:18.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6f4954b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e158d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6f4954b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e09dd0 0x562880e15e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6f4954b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e0ac20 0x562880e18e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6f4954b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x562880e19370 con 0x562880e0ac20 2023-07-18T03:30:18.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6f4954b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562880e194e0 con 0x562880e09dd0 2023-07-18T03:30:18.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6f4954b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562880e196a0 con 0x562880e09480 2023-07-18T03:30:18.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6f4954b80 10 client.0 _ll_register_callbacks cb 0x7ff6d0010da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:18.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6e6d6b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e0ac20 0x562880e18e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6e6d6b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e0ac20 0x562880e18e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57138/0 (socket says 172.21.15.40:57138) 2023-07-18T03:30:18.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6e6d6b700 1 -- 172.21.15.40:0/4095109979 learned_addr learned my addr 172.21.15.40:0/4095109979 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:18.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6e656a700 1 --2- 172.21.15.40:0/4095109979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e158d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6e5d69700 1 --2- 172.21.15.40:0/4095109979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e09dd0 0x562880e15e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6e6d6b700 1 -- 172.21.15.40:0/4095109979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 msgr2=0x562880e158d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6e6d6b700 1 --2- 172.21.15.40:0/4095109979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e158d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6e6d6b700 1 -- 172.21.15.40:0/4095109979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e09dd0 msgr2=0x562880e15e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:18.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6e6d6b700 1 --2- 172.21.15.40:0/4095109979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e09dd0 0x562880e15e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6e6d6b700 1 -- 172.21.15.40:0/4095109979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562880b71c10 con 0x562880e0ac20 2023-07-18T03:30:18.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.717+0000 7ff6e5d69700 1 --2- 172.21.15.40:0/4095109979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e09dd0 0x562880e15e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:18.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.718+0000 7ff6e6d6b700 1 --2- 172.21.15.40:0/4095109979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e0ac20 0x562880e18e30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x562880e00fc0 tx=0x7ff6d8003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3080447ea8213d1d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:18.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.718+0000 7ff6c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:18.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.718+0000 7ff6c9ffb700 1 -- 172.21.15.40:0/4095109979 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d800e040 con 0x562880e0ac20 2023-07-18T03:30:18.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.718+0000 7ff6c9ffb700 1 -- 172.21.15.40:0/4095109979 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff6d800b7b0 con 0x562880e0ac20 2023-07-18T03:30:18.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.718+0000 7ff6f4954b80 1 -- 172.21.15.40:0/4095109979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6d8007830 con 0x562880e0ac20 2023-07-18T03:30:18.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.718+0000 7ff6c9ffb700 1 -- 172.21.15.40:0/4095109979 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6d80094d0 con 0x562880e0ac20 2023-07-18T03:30:18.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.718+0000 7ff6f4954b80 10 client.15102 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:18.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.718+0000 7ff6f4954b80 1 -- 172.21.15.40:0/4095109979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562880b87260 con 0x562880e0ac20 2023-07-18T03:30:18.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.718+0000 7ff6c9ffb700 1 -- 172.21.15.40:0/4095109979 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6d8007830 con 0x562880e0ac20 2023-07-18T03:30:18.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.718+0000 7ff6c9ffb700 1 -- 172.21.15.40:0/4095109979 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff6d8038070 con 0x562880e0ac20 2023-07-18T03:30:18.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.718+0000 7ff6f4954b80 10 client.15102 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:18.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.718+0000 7ff6f4954b80 10 client.15102 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-18T03:30:18.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.718+0000 7ff6f4954b80 20 client.15102 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:18.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.718+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/4095109979 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x562880e77830 0x562880e74280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:18.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.718+0000 7ff6f4954b80 4 client.15102 mds_command: new command op to 24527 tid=0[{"prefix": "status"}] 2023-07-18T03:30:18.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.718+0000 7ff6f4954b80 1 -- 172.21.15.40:0/4095109979 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562880d16000 con 0x562880e77830 2023-07-18T03:30:18.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.718+0000 7ff6e656a700 1 --2- 172.21.15.40:0/4095109979 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x562880e77830 0x562880e74280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:18.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.719+0000 7ff6e656a700 1 --2- 172.21.15.40:0/4095109979 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x562880e77830 0x562880e74280 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd8dbf5293e43abe server_cookie=67efe7034e9102b9 in_seq=0 out_seq=0 2023-07-18T03:30:18.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.719+0000 7ff6c9ffb700 10 client.15102 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:30:18.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.719+0000 7ff6c9ffb700 1 -- 172.21.15.40:0/4095109979 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x562880d16000 con 0x562880e77830 2023-07-18T03:30:18.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.719+0000 7ff6c9ffb700 10 client.15102 handle_command_reply: tid=0 2023-07-18T03:30:18.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.719+0000 7ff6f4954b80 1 client.15102 shutdown 2023-07-18T03:30:18.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.719+0000 7ff6f4954b80 1 -- 172.21.15.40:0/4095109979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e0ac20 msgr2=0x562880e18e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.719+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/4095109979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e0ac20 0x562880e18e30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x562880e00fc0 tx=0x7ff6d8003c30 comp rx=0 tx=0).stop 2023-07-18T03:30:18.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.719+0000 7ff6f4954b80 1 -- 172.21.15.40:0/4095109979 shutdown_connections 2023-07-18T03:30:18.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.719+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/4095109979 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x562880e77830 0x562880e74280 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.719+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/4095109979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562880e0ac20 0x562880e18e30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.719+0000 7ff6c9ffb700 0 client.15102 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:30:18.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.719+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/4095109979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562880e09480 0x562880e158d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.719+0000 7ff6f4954b80 1 --2- 172.21.15.40:0/4095109979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562880e09dd0 0x562880e15e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.719+0000 7ff6f4954b80 1 -- 172.21.15.40:0/4095109979 >> 172.21.15.40:0/4095109979 conn(0x562880dfed50 msgr2=0x562880e1adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.719+0000 7ff6f4954b80 1 -- 172.21.15.40:0/4095109979 shutdown_connections 2023-07-18T03:30:18.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.719+0000 7ff6f4954b80 1 -- 172.21.15.40:0/4095109979 wait complete. 2023-07-18T03:30:18.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.719+0000 7ff6f4954b80 20 client.15102 trim_cache size 0 max 16384 2023-07-18T03:30:18.735 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:30:18.736 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "3af3dba8-251a-11ee-9b34-001a4aab830c", 2023-07-18T03:30:18.736 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-07-18T03:30:18.736 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24527, 2023-07-18T03:30:18.736 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-07-18T03:30:18.736 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-07-18T03:30:18.736 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-07-18T03:30:18.737 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 51.126033286999998, 2023-07-18T03:30:18.737 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 27, 2023-07-18T03:30:18.737 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 81, 2023-07-18T03:30:18.737 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 81, 2023-07-18T03:30:18.737 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 58.095972129000003 2023-07-18T03:30:18.738 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:30:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.720+0000 7ff6e87ce700 1 -- 172.21.15.40:0/3710346101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff6cc0a1330 msgr2=0x7ff6cc0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.720+0000 7ff6e87ce700 1 --2- 172.21.15.40:0/3710346101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff6cc0a1330 0x7ff6cc0a37e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0005950 tx=0x7ff6d000b410 comp rx=0 tx=0).stop 2023-07-18T03:30:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.720+0000 7ff6e87ce700 1 -- 172.21.15.40:0/3710346101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6e00fe9e0 msgr2=0x7ff6e0195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.720+0000 7ff6e87ce700 1 --2- 172.21.15.40:0/3710346101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6e00fe9e0 0x7ff6e0195970 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00d980 tx=0x7ff6dc00dc90 comp rx=0 tx=0).stop 2023-07-18T03:30:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.720+0000 7ff6e87ce700 1 -- 172.21.15.40:0/3710346101 shutdown_connections 2023-07-18T03:30:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.720+0000 7ff6e87ce700 1 --2- 172.21.15.40:0/3710346101 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff6cc0a1330 0x7ff6cc0a37e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.720+0000 7ff6e87ce700 1 --2- 172.21.15.40:0/3710346101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6e0100560 0x7ff6e019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.720+0000 7ff6e87ce700 1 --2- 172.21.15.40:0/3710346101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6e00fe9e0 0x7ff6e0195970 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.720+0000 7ff6e87ce700 1 --2- 172.21.15.40:0/3710346101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff6e00ffbd0 0x7ff6e0195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:18.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.720+0000 7ff6e87ce700 1 -- 172.21.15.40:0/3710346101 >> 172.21.15.40:0/3710346101 conn(0x7ff6e00fa000 msgr2=0x7ff6e00676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:18.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.720+0000 7ff6e87ce700 1 -- 172.21.15.40:0/3710346101 shutdown_connections 2023-07-18T03:30:18.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:18.720+0000 7ff6e87ce700 1 -- 172.21.15.40:0/3710346101 wait complete. 2023-07-18T03:30:18.741 INFO:tasks.cephadm:Setup complete, yielding 2023-07-18T03:30:18.741 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-18T03:30:18.751 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi040.front.sepia.ceph.com 2023-07-18T03:30:18.752 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- bash -c 'ceph orch status' 2023-07-18T03:30:19.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:18 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2646017032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:19.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:18 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2646017032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:19.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:18 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2646017032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:19.308 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:30:19.799 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:19 smithi040 ceph-mon[96439]: pgmap v274: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-18T03:30:20.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:19 smithi134 ceph-mon[98216]: pgmap v274: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-18T03:30:20.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:19 smithi064 ceph-mon[98059]: pgmap v274: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-18T03:30:20.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.512+0000 7f1012720700 1 -- 172.21.15.40:0/2803157894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c0ffc80 msgr2=0x7f100c1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:20.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.512+0000 7f1012720700 1 --2- 172.21.15.40:0/2803157894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c0ffc80 0x7f100c1000d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1000009990 tx=0x7f1000009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:20.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.512+0000 7f1012720700 1 -- 172.21.15.40:0/2803157894 shutdown_connections 2023-07-18T03:30:20.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.512+0000 7f1012720700 1 --2- 172.21.15.40:0/2803157894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f100c100610 0x7f100c106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:20.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.512+0000 7f1012720700 1 --2- 172.21.15.40:0/2803157894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f100c0fea80 0x7f100c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:20.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.512+0000 7f1012720700 1 --2- 172.21.15.40:0/2803157894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c0ffc80 0x7f100c1000d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:20.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.512+0000 7f1012720700 1 -- 172.21.15.40:0/2803157894 >> 172.21.15.40:0/2803157894 conn(0x7f100c0fa0b0 msgr2=0x7f100c0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:20.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.512+0000 7f1012720700 1 -- 172.21.15.40:0/2803157894 shutdown_connections 2023-07-18T03:30:20.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.512+0000 7f1012720700 1 -- 172.21.15.40:0/2803157894 wait complete. 2023-07-18T03:30:20.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.512+0000 7f1012720700 1 Processor -- start 2023-07-18T03:30:20.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.512+0000 7f1012720700 1 -- start start 2023-07-18T03:30:20.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f1012720700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f100c0fea80 0x7f100c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:20.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f1012720700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c0ffc80 0x7f100c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:20.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f1012720700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f100c100610 0x7f100c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:20.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f1012720700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c19d400 con 0x7f100c0ffc80 2023-07-18T03:30:20.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f1012720700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c19d570 con 0x7f100c0fea80 2023-07-18T03:30:20.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f1012720700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c19d730 con 0x7f100c100610 2023-07-18T03:30:20.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f100b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c0ffc80 0x7f100c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:20.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f100b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c0ffc80 0x7f100c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57148/0 (socket says 172.21.15.40:57148) 2023-07-18T03:30:20.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f100b7fe700 1 -- 172.21.15.40:0/2417613193 learned_addr learned my addr 172.21.15.40:0/2417613193 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:20.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f100bfff700 1 --2- 172.21.15.40:0/2417613193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f100c0fea80 0x7f100c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:20.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f1010cbd700 1 --2- 172.21.15.40:0/2417613193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f100c100610 0x7f100c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:20.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f100b7fe700 1 -- 172.21.15.40:0/2417613193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f100c100610 msgr2=0x7f100c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:20.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f100b7fe700 1 --2- 172.21.15.40:0/2417613193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f100c100610 0x7f100c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:20.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f100b7fe700 1 -- 172.21.15.40:0/2417613193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f100c0fea80 msgr2=0x7f100c0693e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:20.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f100b7fe700 1 --2- 172.21.15.40:0/2417613193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f100c0fea80 0x7f100c0693e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:20.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f100b7fe700 1 -- 172.21.15.40:0/2417613193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10000095f0 con 0x7f100c0ffc80 2023-07-18T03:30:20.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f100bfff700 1 --2- 172.21.15.40:0/2417613193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f100c0fea80 0x7f100c0693e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:20.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f1010cbd700 1 --2- 172.21.15.40:0/2417613193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f100c100610 0x7f100c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:20.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f100b7fe700 1 --2- 172.21.15.40:0/2417613193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c0ffc80 0x7f100c19a610 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1000005910 tx=0x7f100000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:20.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.513+0000 7f10097fa700 1 -- 172.21.15.40:0/2417613193 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100001d070 con 0x7f100c0ffc80 2023-07-18T03:30:20.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.514+0000 7f10097fa700 1 -- 172.21.15.40:0/2417613193 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f100000fc30 con 0x7f100c0ffc80 2023-07-18T03:30:20.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.514+0000 7f1012720700 1 -- 172.21.15.40:0/2417613193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100c19d9b0 con 0x7f100c0ffc80 2023-07-18T03:30:20.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.514+0000 7f10097fa700 1 -- 172.21.15.40:0/2417613193 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1000017650 con 0x7f100c0ffc80 2023-07-18T03:30:20.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.514+0000 7f1012720700 1 -- 172.21.15.40:0/2417613193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100c19ddc0 con 0x7f100c0ffc80 2023-07-18T03:30:20.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.515+0000 7f1012720700 1 -- 172.21.15.40:0/2417613193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100c10a290 con 0x7f100c0ffc80 2023-07-18T03:30:20.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.516+0000 7f10097fa700 1 -- 172.21.15.40:0/2417613193 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1000022470 con 0x7f100c0ffc80 2023-07-18T03:30:20.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.516+0000 7f10097fa700 1 --2- 172.21.15.40:0/2417613193 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0ff40a11e0 0x7f0ff40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:20.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.516+0000 7f100bfff700 1 --2- 172.21.15.40:0/2417613193 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0ff40a11e0 0x7f0ff40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:20.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.516+0000 7f10097fa700 1 -- 172.21.15.40:0/2417613193 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f10000cdd40 con 0x7f100c0ffc80 2023-07-18T03:30:20.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.517+0000 7f100bfff700 1 --2- 172.21.15.40:0/2417613193 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0ff40a11e0 0x7f0ff40a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc005950 tx=0x7f0ffc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:20.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.520+0000 7f10097fa700 1 -- 172.21.15.40:0/2417613193 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1000097600 con 0x7f100c0ffc80 2023-07-18T03:30:20.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.733+0000 7f1012720700 1 -- 172.21.15.40:0/2417613193 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f100c060950 con 0x7f0ff40a11e0 2023-07-18T03:30:20.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.735+0000 7f10097fa700 1 -- 172.21.15.40:0/2417613193 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f100c060950 con 0x7f0ff40a11e0 2023-07-18T03:30:20.732 INFO:teuthology.orchestra.run.smithi040.stdout:Backend: cephadm 2023-07-18T03:30:20.732 INFO:teuthology.orchestra.run.smithi040.stdout:Available: Yes 2023-07-18T03:30:20.732 INFO:teuthology.orchestra.run.smithi040.stdout:Paused: No 2023-07-18T03:30:20.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.736+0000 7f1012720700 1 -- 172.21.15.40:0/2417613193 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0ff40a11e0 msgr2=0x7f0ff40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:20.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.736+0000 7f1012720700 1 --2- 172.21.15.40:0/2417613193 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0ff40a11e0 0x7f0ff40a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc005950 tx=0x7f0ffc0058e0 comp rx=0 tx=0).stop 2023-07-18T03:30:20.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.736+0000 7f1012720700 1 -- 172.21.15.40:0/2417613193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c0ffc80 msgr2=0x7f100c19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:20.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.736+0000 7f1012720700 1 --2- 172.21.15.40:0/2417613193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c0ffc80 0x7f100c19a610 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1000005910 tx=0x7f100000f740 comp rx=0 tx=0).stop 2023-07-18T03:30:20.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.737+0000 7f1012720700 1 -- 172.21.15.40:0/2417613193 shutdown_connections 2023-07-18T03:30:20.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.737+0000 7f1012720700 1 --2- 172.21.15.40:0/2417613193 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0ff40a11e0 0x7f0ff40a3690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:20.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.737+0000 7f1012720700 1 --2- 172.21.15.40:0/2417613193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f100c100610 0x7f100c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:20.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.737+0000 7f1012720700 1 --2- 172.21.15.40:0/2417613193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f100c0fea80 0x7f100c0693e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:20.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.737+0000 7f1012720700 1 --2- 172.21.15.40:0/2417613193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f100c0ffc80 0x7f100c19a610 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:20.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.737+0000 7f1012720700 1 -- 172.21.15.40:0/2417613193 >> 172.21.15.40:0/2417613193 conn(0x7f100c0fa0b0 msgr2=0x7f100c0fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:20.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.737+0000 7f1012720700 1 -- 172.21.15.40:0/2417613193 shutdown_connections 2023-07-18T03:30:20.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:20.737+0000 7f1012720700 1 -- 172.21.15.40:0/2417613193 wait complete. 2023-07-18T03:30:21.158 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- bash -c 'ceph orch ps' 2023-07-18T03:30:21.718 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:30:22.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:21 smithi134 ceph-mon[98216]: pgmap v275: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-07-18T03:30:22.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:21 smithi134 ceph-mon[98216]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:30:22.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:21 smithi040 ceph-mon[96439]: pgmap v275: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-07-18T03:30:22.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:21 smithi040 ceph-mon[96439]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:30:22.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:21 smithi064 ceph-mon[98059]: pgmap v275: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2023-07-18T03:30:22.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:21 smithi064 ceph-mon[98059]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:30:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.895+0000 7f1c6bed9700 1 -- 172.21.15.40:0/4276485897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c640fea80 msgr2=0x7f1c640fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.895+0000 7f1c6bed9700 1 --2- 172.21.15.40:0/4276485897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c640fea80 0x7f1c640fee90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009990 tx=0x7f1c60009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.895+0000 7f1c6bed9700 1 -- 172.21.15.40:0/4276485897 shutdown_connections 2023-07-18T03:30:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.895+0000 7f1c6bed9700 1 --2- 172.21.15.40:0/4276485897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c64100610 0x7f1c64106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.895+0000 7f1c6bed9700 1 --2- 172.21.15.40:0/4276485897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c640fea80 0x7f1c640fee90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.895+0000 7f1c6bed9700 1 --2- 172.21.15.40:0/4276485897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c640ffc80 0x7f1c641000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.895+0000 7f1c6bed9700 1 -- 172.21.15.40:0/4276485897 >> 172.21.15.40:0/4276485897 conn(0x7f1c640fa0b0 msgr2=0x7f1c640fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.895+0000 7f1c6bed9700 1 -- 172.21.15.40:0/4276485897 shutdown_connections 2023-07-18T03:30:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.895+0000 7f1c6bed9700 1 -- 172.21.15.40:0/4276485897 wait complete. 2023-07-18T03:30:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.896+0000 7f1c6bed9700 1 Processor -- start 2023-07-18T03:30:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.896+0000 7f1c6bed9700 1 -- start start 2023-07-18T03:30:22.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.896+0000 7f1c6bed9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c640fea80 0x7f1c64195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.896+0000 7f1c6bed9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c640ffc80 0x7f1c64195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.896+0000 7f1c6bed9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c64100610 0x7f1c6419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:22.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.896+0000 7f1c6bed9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6419d3b0 con 0x7f1c640fea80 2023-07-18T03:30:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.896+0000 7f1c6bed9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6419d4f0 con 0x7f1c640ffc80 2023-07-18T03:30:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.896+0000 7f1c6bed9700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6419d680 con 0x7f1c64100610 2023-07-18T03:30:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.896+0000 7f1c69c75700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c640fea80 0x7f1c64195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:22.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.896+0000 7f1c69c75700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c640fea80 0x7f1c64195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57156/0 (socket says 172.21.15.40:57156) 2023-07-18T03:30:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.896+0000 7f1c69c75700 1 -- 172.21.15.40:0/1401268844 learned_addr learned my addr 172.21.15.40:0/1401268844 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.896+0000 7f1c69474700 1 --2- 172.21.15.40:0/1401268844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c640ffc80 0x7f1c64195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.896+0000 7f1c6a476700 1 --2- 172.21.15.40:0/1401268844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c64100610 0x7f1c6419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.897+0000 7f1c69c75700 1 -- 172.21.15.40:0/1401268844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c64100610 msgr2=0x7f1c6419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:22.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.897+0000 7f1c69c75700 1 --2- 172.21.15.40:0/1401268844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c64100610 0x7f1c6419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.897+0000 7f1c69c75700 1 -- 172.21.15.40:0/1401268844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c640ffc80 msgr2=0x7f1c64195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.897+0000 7f1c69c75700 1 --2- 172.21.15.40:0/1401268844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c640ffc80 0x7f1c64195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.897+0000 7f1c69c75700 1 -- 172.21.15.40:0/1401268844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c600095f0 con 0x7f1c640fea80 2023-07-18T03:30:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.897+0000 7f1c69c75700 1 --2- 172.21.15.40:0/1401268844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c640fea80 0x7f1c64195820 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1c60004730 tx=0x7f1c60004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.897+0000 7f1c5affd700 1 -- 172.21.15.40:0/1401268844 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6001d070 con 0x7f1c640fea80 2023-07-18T03:30:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.897+0000 7f1c5affd700 1 -- 172.21.15.40:0/1401268844 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c600054f0 con 0x7f1c640fea80 2023-07-18T03:30:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.897+0000 7f1c6bed9700 1 -- 172.21.15.40:0/1401268844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6419d900 con 0x7f1c640fea80 2023-07-18T03:30:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.897+0000 7f1c5affd700 1 -- 172.21.15.40:0/1401268844 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c60004c60 con 0x7f1c640fea80 2023-07-18T03:30:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.897+0000 7f1c6bed9700 1 -- 172.21.15.40:0/1401268844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6419dcb0 con 0x7f1c640fea80 2023-07-18T03:30:22.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.898+0000 7f1c6bed9700 1 -- 172.21.15.40:0/1401268844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6410b3b0 con 0x7f1c640fea80 2023-07-18T03:30:22.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.899+0000 7f1c5affd700 1 -- 172.21.15.40:0/1401268844 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1c6000fbb0 con 0x7f1c640fea80 2023-07-18T03:30:22.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.899+0000 7f1c5affd700 1 --2- 172.21.15.40:0/1401268844 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1c500a11e0 0x7f1c500a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:22.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.900+0000 7f1c69474700 1 --2- 172.21.15.40:0/1401268844 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1c500a11e0 0x7f1c500a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:22.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.900+0000 7f1c5affd700 1 -- 172.21.15.40:0/1401268844 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1c600ce0b0 con 0x7f1c640fea80 2023-07-18T03:30:22.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.900+0000 7f1c69474700 1 --2- 172.21.15.40:0/1401268844 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1c500a11e0 0x7f1c500a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009c60 tx=0x7f1c54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:22.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:22.903+0000 7f1c5affd700 1 -- 172.21.15.40:0/1401268844 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1c600978e0 con 0x7f1c640fea80 2023-07-18T03:30:23.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:23.113+0000 7f1c6bed9700 1 -- 172.21.15.40:0/1401268844 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c6419e350 con 0x7f1c500a11e0 2023-07-18T03:30:23.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:23.133+0000 7f1c5affd700 1 -- 172.21.15.40:0/1401268844 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f1c6419e350 con 0x7f1c500a11e0 2023-07-18T03:30:23.131 INFO:teuthology.orchestra.run.smithi040.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-07-18T03:30:23.131 INFO:teuthology.orchestra.run.smithi040.stdout:mds.a smithi040 running (65s) 55s ago 64s 13.2M - 18.0.0-4746-g81cf785a 66a4f7b015b5 dd6806ce7cb5 2023-07-18T03:30:23.131 INFO:teuthology.orchestra.run.smithi040.stdout:mds.b smithi064 running (62s) 55s ago 62s 13.2M - 18.0.0-4746-g81cf785a 66a4f7b015b5 789ae0704aeb 2023-07-18T03:30:23.131 INFO:teuthology.orchestra.run.smithi040.stdout:mds.c smithi134 running (73s) 55s ago 73s 14.3M - 18.0.0-4746-g81cf785a 66a4f7b015b5 e1f021a73de1 2023-07-18T03:30:23.131 INFO:teuthology.orchestra.run.smithi040.stdout:mds.d smithi040 running (80s) 55s ago 80s 14.3M - 18.0.0-4746-g81cf785a 66a4f7b015b5 fbf659c313e7 2023-07-18T03:30:23.131 INFO:teuthology.orchestra.run.smithi040.stdout:mds.e smithi064 running (78s) 55s ago 77s 15.7M - 18.0.0-4746-g81cf785a 66a4f7b015b5 d28733e6182f 2023-07-18T03:30:23.132 INFO:teuthology.orchestra.run.smithi040.stdout:mds.f smithi134 running (76s) 55s ago 75s 14.0M - 18.0.0-4746-g81cf785a 66a4f7b015b5 842d678a97b3 2023-07-18T03:30:23.132 INFO:teuthology.orchestra.run.smithi040.stdout:mds.g smithi040 running (67s) 55s ago 66s 15.7M - 18.0.0-4746-g81cf785a 66a4f7b015b5 80f792591c15 2023-07-18T03:30:23.132 INFO:teuthology.orchestra.run.smithi040.stdout:mds.h smithi064 running (71s) 55s ago 71s 15.7M - 18.0.0-4746-g81cf785a 66a4f7b015b5 23568a0fb5f7 2023-07-18T03:30:23.132 INFO:teuthology.orchestra.run.smithi040.stdout:mds.i smithi134 running (69s) 55s ago 69s 15.9M - 18.0.0-4746-g81cf785a 66a4f7b015b5 4ecaf9f3f7ec 2023-07-18T03:30:23.132 INFO:teuthology.orchestra.run.smithi040.stdout:mds.j smithi040 running (82s) 55s ago 82s 16.7M - 18.0.0-4746-g81cf785a 66a4f7b015b5 f532e0aed9aa 2023-07-18T03:30:23.132 INFO:teuthology.orchestra.run.smithi040.stdout:mds.k smithi064 running (84s) 55s ago 84s 15.7M - 18.0.0-4746-g81cf785a 66a4f7b015b5 c856b53d03ad 2023-07-18T03:30:23.133 INFO:teuthology.orchestra.run.smithi040.stdout:mds.l smithi134 running (86s) 55s ago 86s 14.0M - 18.0.0-4746-g81cf785a 66a4f7b015b5 7ae02476fcba 2023-07-18T03:30:23.133 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.x smithi040 *:9283,8765 running (7m) 55s ago 7m 510M - 18.0.0-4746-g81cf785a 66a4f7b015b5 91c5a2cd4a99 2023-07-18T03:30:23.133 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.y smithi064 *:8443,8765 running (5m) 55s ago 5m 445M - 18.0.0-4746-g81cf785a 66a4f7b015b5 60e8e108da70 2023-07-18T03:30:23.133 INFO:teuthology.orchestra.run.smithi040.stdout:mgr.z smithi134 *:8443,8765 running (5m) 55s ago 5m 446M - 18.0.0-4746-g81cf785a 66a4f7b015b5 a107047838e2 2023-07-18T03:30:23.133 INFO:teuthology.orchestra.run.smithi040.stdout:mon.a smithi040 running (7m) 55s ago 7m 82.7M 2048M 18.0.0-4746-g81cf785a 66a4f7b015b5 c90877839340 2023-07-18T03:30:23.133 INFO:teuthology.orchestra.run.smithi040.stdout:mon.b smithi064 running (6m) 55s ago 6m 59.0M 2048M 18.0.0-4746-g81cf785a 66a4f7b015b5 06bbfda0118c 2023-07-18T03:30:23.133 INFO:teuthology.orchestra.run.smithi040.stdout:mon.c smithi134 running (6m) 55s ago 6m 57.4M 2048M 18.0.0-4746-g81cf785a 66a4f7b015b5 0a3fcd582172 2023-07-18T03:30:23.133 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 smithi040 running (5m) 55s ago 5m 54.2M 4293M 18.0.0-4746-g81cf785a 66a4f7b015b5 2692e06e061d 2023-07-18T03:30:23.134 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 smithi064 running (5m) 55s ago 5m 58.8M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 318b063debce 2023-07-18T03:30:23.134 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 smithi134 running (4m) 55s ago 4m 54.4M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 700e4acde872 2023-07-18T03:30:23.134 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 smithi040 running (4m) 55s ago 4m 54.5M 4293M 18.0.0-4746-g81cf785a 66a4f7b015b5 33d9b22fc1da 2023-07-18T03:30:23.134 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 smithi064 running (4m) 55s ago 4m 47.0M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 c9ecf4b9db5b 2023-07-18T03:30:23.134 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 smithi134 running (3m) 55s ago 3m 54.2M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 a6351b80c180 2023-07-18T03:30:23.134 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6 smithi040 running (3m) 55s ago 3m 47.1M 4293M 18.0.0-4746-g81cf785a 66a4f7b015b5 4925290a0973 2023-07-18T03:30:23.134 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7 smithi064 running (3m) 55s ago 3m 51.1M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 4baf09ad854f 2023-07-18T03:30:23.134 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8 smithi134 running (2m) 55s ago 2m 49.0M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 43b22b8b5570 2023-07-18T03:30:23.135 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9 smithi040 running (2m) 55s ago 2m 46.2M 4293M 18.0.0-4746-g81cf785a 66a4f7b015b5 836c094ed9f2 2023-07-18T03:30:23.135 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10 smithi064 running (2m) 55s ago 2m 48.1M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 9516533ca03e 2023-07-18T03:30:23.135 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11 smithi134 running (104s) 55s ago 104s 45.3M 4292M 18.0.0-4746-g81cf785a 66a4f7b015b5 406bbee31a25 2023-07-18T03:30:23.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:23.135+0000 7f1c6bed9700 1 -- 172.21.15.40:0/1401268844 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1c500a11e0 msgr2=0x7f1c500a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:23.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:23.135+0000 7f1c6bed9700 1 --2- 172.21.15.40:0/1401268844 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1c500a11e0 0x7f1c500a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009c60 tx=0x7f1c54009260 comp rx=0 tx=0).stop 2023-07-18T03:30:23.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:23.135+0000 7f1c6bed9700 1 -- 172.21.15.40:0/1401268844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c640fea80 msgr2=0x7f1c64195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:23.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:23.135+0000 7f1c6bed9700 1 --2- 172.21.15.40:0/1401268844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c640fea80 0x7f1c64195820 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1c60004730 tx=0x7f1c60004760 comp rx=0 tx=0).stop 2023-07-18T03:30:23.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:23.135+0000 7f1c6bed9700 1 -- 172.21.15.40:0/1401268844 shutdown_connections 2023-07-18T03:30:23.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:23.135+0000 7f1c6bed9700 1 --2- 172.21.15.40:0/1401268844 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1c500a11e0 0x7f1c500a3690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:23.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:23.135+0000 7f1c6bed9700 1 --2- 172.21.15.40:0/1401268844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c64100610 0x7f1c6419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:23.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:23.135+0000 7f1c6bed9700 1 --2- 172.21.15.40:0/1401268844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c640fea80 0x7f1c64195820 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:23.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:23.135+0000 7f1c6bed9700 1 --2- 172.21.15.40:0/1401268844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c640ffc80 0x7f1c64195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:23.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:23.135+0000 7f1c6bed9700 1 -- 172.21.15.40:0/1401268844 >> 172.21.15.40:0/1401268844 conn(0x7f1c640fa0b0 msgr2=0x7f1c640fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:23.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:23.135+0000 7f1c6bed9700 1 -- 172.21.15.40:0/1401268844 shutdown_connections 2023-07-18T03:30:23.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:23.135+0000 7f1c6bed9700 1 -- 172.21.15.40:0/1401268844 wait complete. 2023-07-18T03:30:23.473 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- bash -c 'ceph orch ls' 2023-07-18T03:30:24.036 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:30:24.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:23 smithi134 ceph-mon[98216]: pgmap v276: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-18T03:30:24.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:23 smithi040 ceph-mon[96439]: pgmap v276: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-18T03:30:24.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:23 smithi064 ceph-mon[98059]: pgmap v276: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-18T03:30:25.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:24 smithi134 ceph-mon[98216]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:30:25.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.078+0000 7f0a28e5e700 1 -- 172.21.15.40:0/379875070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a24101720 msgr2=0x7f0a24107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:25.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.078+0000 7f0a28e5e700 1 --2- 172.21.15.40:0/379875070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a24101720 0x7f0a24107fe0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00b0a0 tx=0x7f0a1c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:25.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.078+0000 7f0a28e5e700 1 -- 172.21.15.40:0/379875070 shutdown_connections 2023-07-18T03:30:25.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.078+0000 7f0a28e5e700 1 --2- 172.21.15.40:0/379875070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a24101720 0x7f0a24107fe0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:25.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.078+0000 7f0a28e5e700 1 --2- 172.21.15.40:0/379875070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a240ffb90 0x7f0a240fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:25.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.078+0000 7f0a28e5e700 1 --2- 172.21.15.40:0/379875070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a24100d90 0x7f0a241011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:25.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.078+0000 7f0a28e5e700 1 -- 172.21.15.40:0/379875070 >> 172.21.15.40:0/379875070 conn(0x7f0a240fb1a0 msgr2=0x7f0a240fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:25.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.078+0000 7f0a28e5e700 1 -- 172.21.15.40:0/379875070 shutdown_connections 2023-07-18T03:30:25.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.078+0000 7f0a28e5e700 1 -- 172.21.15.40:0/379875070 wait complete. 2023-07-18T03:30:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.079+0000 7f0a28e5e700 1 Processor -- start 2023-07-18T03:30:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.079+0000 7f0a28e5e700 1 -- start start 2023-07-18T03:30:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.079+0000 7f0a28e5e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a240ffb90 0x7f0a24195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.079+0000 7f0a28e5e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a24100d90 0x7f0a24195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.079+0000 7f0a28e5e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a24101720 0x7f0a2419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.079+0000 7f0a28e5e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2419d500 con 0x7f0a240ffb90 2023-07-18T03:30:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a28e5e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2419d640 con 0x7f0a24101720 2023-07-18T03:30:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a28e5e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2419d7d0 con 0x7f0a24100d90 2023-07-18T03:30:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a2259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a240ffb90 0x7f0a24195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a2259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a240ffb90 0x7f0a24195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57170/0 (socket says 172.21.15.40:57170) 2023-07-18T03:30:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a2259c700 1 -- 172.21.15.40:0/3947808562 learned_addr learned my addr 172.21.15.40:0/3947808562 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a22d9d700 1 --2- 172.21.15.40:0/3947808562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a24101720 0x7f0a2419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a21d9b700 1 --2- 172.21.15.40:0/3947808562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a24100d90 0x7f0a24195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a2259c700 1 -- 172.21.15.40:0/3947808562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a24100d90 msgr2=0x7f0a24195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a2259c700 1 --2- 172.21.15.40:0/3947808562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a24100d90 0x7f0a24195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a2259c700 1 -- 172.21.15.40:0/3947808562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a24101720 msgr2=0x7f0a2419cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:25.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a2259c700 1 --2- 172.21.15.40:0/3947808562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a24101720 0x7f0a2419cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a2259c700 1 -- 172.21.15.40:0/3947808562 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c009d00 con 0x7f0a240ffb90 2023-07-18T03:30:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a2259c700 1 --2- 172.21.15.40:0/3947808562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a240ffb90 0x7f0a24195970 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00c9a0 tx=0x7f0a0c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a1b7fe700 1 -- 172.21.15.40:0/3947808562 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c004220 con 0x7f0a240ffb90 2023-07-18T03:30:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a28e5e700 1 -- 172.21.15.40:0/3947808562 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a2419dab0 con 0x7f0a240ffb90 2023-07-18T03:30:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a1b7fe700 1 -- 172.21.15.40:0/3947808562 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a0c007d20 con 0x7f0a240ffb90 2023-07-18T03:30:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a1b7fe700 1 -- 172.21.15.40:0/3947808562 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c003c00 con 0x7f0a240ffb90 2023-07-18T03:30:25.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.080+0000 7f0a28e5e700 1 -- 172.21.15.40:0/3947808562 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a2419e000 con 0x7f0a240ffb90 2023-07-18T03:30:25.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.081+0000 7f0a28e5e700 1 -- 172.21.15.40:0/3947808562 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a24107130 con 0x7f0a240ffb90 2023-07-18T03:30:25.083 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:24 smithi040 ceph-mon[96439]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:30:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.083+0000 7f0a1b7fe700 1 -- 172.21.15.40:0/3947808562 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0a0c007860 con 0x7f0a240ffb90 2023-07-18T03:30:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.083+0000 7f0a1b7fe700 1 --2- 172.21.15.40:0/3947808562 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0a100a11e0 0x7f0a100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.083+0000 7f0a21d9b700 1 --2- 172.21.15.40:0/3947808562 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0a100a11e0 0x7f0a100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.083+0000 7f0a1b7fe700 1 -- 172.21.15.40:0/3947808562 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0a0c0cc8b0 con 0x7f0a240ffb90 2023-07-18T03:30:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.084+0000 7f0a21d9b700 1 --2- 172.21.15.40:0/3947808562 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0a100a11e0 0x7f0a100a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0a14005950 tx=0x7f0a140058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:25.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.087+0000 7f0a1b7fe700 1 -- 172.21.15.40:0/3947808562 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0a0c096050 con 0x7f0a240ffb90 2023-07-18T03:30:25.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:24 smithi064 ceph-mon[98059]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:30:25.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.291+0000 7f0a28e5e700 1 -- 172.21.15.40:0/3947808562 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a24060950 con 0x7f0a100a11e0 2023-07-18T03:30:25.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.298+0000 7f0a1b7fe700 1 -- 172.21.15.40:0/3947808562 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f0a24060950 con 0x7f0a100a11e0 2023-07-18T03:30:25.296 INFO:teuthology.orchestra.run.smithi040.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-07-18T03:30:25.296 INFO:teuthology.orchestra.run.smithi040.stdout:mds.all 12/12 57s ago 90s smithi040=a;smithi040=d;smithi040=g;smithi040=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2023-07-18T03:30:25.297 INFO:teuthology.orchestra.run.smithi040.stdout:mgr 3/3 57s ago 5m smithi040=x;smithi064=y;smithi134=z;count:3 2023-07-18T03:30:25.297 INFO:teuthology.orchestra.run.smithi040.stdout:mon 3/3 57s ago 6m smithi040:172.21.15.40=a;smithi064:172.21.15.64=b;smithi134:172.21.15.134=c;count:3 2023-07-18T03:30:25.297 INFO:teuthology.orchestra.run.smithi040.stdout:osd 12 57s ago - 2023-07-18T03:30:25.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.300+0000 7f0a28e5e700 1 -- 172.21.15.40:0/3947808562 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0a100a11e0 msgr2=0x7f0a100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:25.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.300+0000 7f0a28e5e700 1 --2- 172.21.15.40:0/3947808562 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0a100a11e0 0x7f0a100a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0a14005950 tx=0x7f0a140058e0 comp rx=0 tx=0).stop 2023-07-18T03:30:25.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.300+0000 7f0a28e5e700 1 -- 172.21.15.40:0/3947808562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a240ffb90 msgr2=0x7f0a24195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:25.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.300+0000 7f0a28e5e700 1 --2- 172.21.15.40:0/3947808562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a240ffb90 0x7f0a24195970 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00c9a0 tx=0x7f0a0c00ccb0 comp rx=0 tx=0).stop 2023-07-18T03:30:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.301+0000 7f0a28e5e700 1 -- 172.21.15.40:0/3947808562 shutdown_connections 2023-07-18T03:30:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.301+0000 7f0a28e5e700 1 --2- 172.21.15.40:0/3947808562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a24101720 0x7f0a2419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.301+0000 7f0a28e5e700 1 --2- 172.21.15.40:0/3947808562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a240ffb90 0x7f0a24195970 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.301+0000 7f0a28e5e700 1 --2- 172.21.15.40:0/3947808562 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0a100a11e0 0x7f0a100a3690 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.301+0000 7f0a28e5e700 1 --2- 172.21.15.40:0/3947808562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a24100d90 0x7f0a24195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.301+0000 7f0a28e5e700 1 -- 172.21.15.40:0/3947808562 >> 172.21.15.40:0/3947808562 conn(0x7f0a240fb1a0 msgr2=0x7f0a24104270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:25.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.301+0000 7f0a28e5e700 1 -- 172.21.15.40:0/3947808562 shutdown_connections 2023-07-18T03:30:25.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:25.301+0000 7f0a28e5e700 1 -- 172.21.15.40:0/3947808562 wait complete. 2023-07-18T03:30:25.687 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- bash -c 'ceph orch host ls' 2023-07-18T03:30:26.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:25 smithi134 ceph-mon[98216]: pgmap v277: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-07-18T03:30:26.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:25 smithi040 ceph-mon[96439]: pgmap v277: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-07-18T03:30:26.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:25 smithi064 ceph-mon[98059]: pgmap v277: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-07-18T03:30:26.250 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:30:27.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:26 smithi134 ceph-mon[98216]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:30:27.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:26 smithi040 ceph-mon[96439]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:30:27.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:26 smithi064 ceph-mon[98059]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:30:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.348+0000 7eff06e30700 1 -- 172.21.15.40:0/1947316527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff00100610 msgr2=0x7eff00106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.348+0000 7eff06e30700 1 --2- 172.21.15.40:0/1947316527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff00100610 0x7eff00106ed0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efef800b0a0 tx=0x7efef800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.349+0000 7eff06e30700 1 -- 172.21.15.40:0/1947316527 shutdown_connections 2023-07-18T03:30:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.349+0000 7eff06e30700 1 --2- 172.21.15.40:0/1947316527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff00100610 0x7eff00106ed0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.349+0000 7eff06e30700 1 --2- 172.21.15.40:0/1947316527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff000fea80 0x7eff000fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:27.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.349+0000 7eff06e30700 1 --2- 172.21.15.40:0/1947316527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff000ffc80 0x7eff001000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:27.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.349+0000 7eff06e30700 1 -- 172.21.15.40:0/1947316527 >> 172.21.15.40:0/1947316527 conn(0x7eff000fa0b0 msgr2=0x7eff000fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:27.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.349+0000 7eff06e30700 1 -- 172.21.15.40:0/1947316527 shutdown_connections 2023-07-18T03:30:27.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.349+0000 7eff06e30700 1 -- 172.21.15.40:0/1947316527 wait complete. 2023-07-18T03:30:27.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.349+0000 7eff06e30700 1 Processor -- start 2023-07-18T03:30:27.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.349+0000 7eff06e30700 1 -- start start 2023-07-18T03:30:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7eff06e30700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff000fea80 0x7eff00195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7eff06e30700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff000ffc80 0x7eff00195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7eff06e30700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff00100610 0x7eff0019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7eff06e30700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0019d4e0 con 0x7eff000ffc80 2023-07-18T03:30:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7eff06e30700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0019d620 con 0x7eff000fea80 2023-07-18T03:30:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7eff06e30700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0019d7b0 con 0x7eff00100610 2023-07-18T03:30:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7efefffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff000ffc80 0x7eff00195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:27.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7efefffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff000ffc80 0x7eff00195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57184/0 (socket says 172.21.15.40:57184) 2023-07-18T03:30:27.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7efefffff700 1 -- 172.21.15.40:0/1047734894 learned_addr learned my addr 172.21.15.40:0/1047734894 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:27.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7eff04bcc700 1 --2- 172.21.15.40:0/1047734894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff000fea80 0x7eff00195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:27.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7eff053cd700 1 --2- 172.21.15.40:0/1047734894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff00100610 0x7eff0019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:27.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7efefffff700 1 -- 172.21.15.40:0/1047734894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff00100610 msgr2=0x7eff0019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:27.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7efefffff700 1 --2- 172.21.15.40:0/1047734894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff00100610 0x7eff0019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:27.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7efefffff700 1 -- 172.21.15.40:0/1047734894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff000fea80 msgr2=0x7eff00195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7efefffff700 1 --2- 172.21.15.40:0/1047734894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff000fea80 0x7eff00195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7efefffff700 1 -- 172.21.15.40:0/1047734894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef8009d00 con 0x7eff000ffc80 2023-07-18T03:30:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7eff053cd700 1 --2- 172.21.15.40:0/1047734894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff00100610 0x7eff0019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7efefffff700 1 --2- 172.21.15.40:0/1047734894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff000ffc80 0x7eff00195e90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efeec00d9b0 tx=0x7efeec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.350+0000 7efefdffb700 1 -- 172.21.15.40:0/1047734894 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec004220 con 0x7eff000ffc80 2023-07-18T03:30:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.351+0000 7efefdffb700 1 -- 172.21.15.40:0/1047734894 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeec007640 con 0x7eff000ffc80 2023-07-18T03:30:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.351+0000 7eff06e30700 1 -- 172.21.15.40:0/1047734894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0019da90 con 0x7eff000ffc80 2023-07-18T03:30:27.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.351+0000 7efefdffb700 1 -- 172.21.15.40:0/1047734894 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec005650 con 0x7eff000ffc80 2023-07-18T03:30:27.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.351+0000 7eff06e30700 1 -- 172.21.15.40:0/1047734894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0019dfe0 con 0x7eff000ffc80 2023-07-18T03:30:27.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.352+0000 7eff06e30700 1 -- 172.21.15.40:0/1047734894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0010a250 con 0x7eff000ffc80 2023-07-18T03:30:27.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.353+0000 7efefdffb700 1 -- 172.21.15.40:0/1047734894 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7efeec004750 con 0x7eff000ffc80 2023-07-18T03:30:27.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.353+0000 7efefdffb700 1 --2- 172.21.15.40:0/1047734894 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efef00a1260 0x7efef00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:27.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.354+0000 7eff04bcc700 1 --2- 172.21.15.40:0/1047734894 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efef00a1260 0x7efef00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:27.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.354+0000 7efefdffb700 1 -- 172.21.15.40:0/1047734894 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efeec0cc6e0 con 0x7eff000ffc80 2023-07-18T03:30:27.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.354+0000 7eff04bcc700 1 --2- 172.21.15.40:0/1047734894 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efef00a1260 0x7efef00a3710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7eff000ffae0 tx=0x7efef400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.358+0000 7efefdffb700 1 -- 172.21.15.40:0/1047734894 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7efeec095e80 con 0x7eff000ffc80 2023-07-18T03:30:27.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.566+0000 7eff06e30700 1 -- 172.21.15.40:0/1047734894 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7eff00068700 con 0x7efef00a1260 2023-07-18T03:30:27.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.568+0000 7efefdffb700 1 -- 172.21.15.40:0/1047734894 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7eff00068700 con 0x7efef00a1260 2023-07-18T03:30:27.566 INFO:teuthology.orchestra.run.smithi040.stdout:HOST ADDR LABELS STATUS 2023-07-18T03:30:27.566 INFO:teuthology.orchestra.run.smithi040.stdout:smithi040 172.21.15.40 2023-07-18T03:30:27.566 INFO:teuthology.orchestra.run.smithi040.stdout:smithi064 172.21.15.64 2023-07-18T03:30:27.566 INFO:teuthology.orchestra.run.smithi040.stdout:smithi134 172.21.15.134 2023-07-18T03:30:27.566 INFO:teuthology.orchestra.run.smithi040.stdout:3 hosts in cluster 2023-07-18T03:30:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.570+0000 7eff06e30700 1 -- 172.21.15.40:0/1047734894 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efef00a1260 msgr2=0x7efef00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.570+0000 7eff06e30700 1 --2- 172.21.15.40:0/1047734894 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efef00a1260 0x7efef00a3710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7eff000ffae0 tx=0x7efef400b410 comp rx=0 tx=0).stop 2023-07-18T03:30:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.570+0000 7eff06e30700 1 -- 172.21.15.40:0/1047734894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff000ffc80 msgr2=0x7eff00195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.570+0000 7eff06e30700 1 --2- 172.21.15.40:0/1047734894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff000ffc80 0x7eff00195e90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efeec00d9b0 tx=0x7efeec00dcc0 comp rx=0 tx=0).stop 2023-07-18T03:30:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.570+0000 7eff06e30700 1 -- 172.21.15.40:0/1047734894 shutdown_connections 2023-07-18T03:30:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.570+0000 7eff06e30700 1 --2- 172.21.15.40:0/1047734894 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efef00a1260 0x7efef00a3710 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.570+0000 7eff06e30700 1 --2- 172.21.15.40:0/1047734894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff00100610 0x7eff0019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.570+0000 7eff06e30700 1 --2- 172.21.15.40:0/1047734894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff000fea80 0x7eff00195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.570+0000 7eff06e30700 1 --2- 172.21.15.40:0/1047734894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff000ffc80 0x7eff00195e90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.570+0000 7eff06e30700 1 -- 172.21.15.40:0/1047734894 >> 172.21.15.40:0/1047734894 conn(0x7eff000fa0b0 msgr2=0x7eff00067ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.570+0000 7eff06e30700 1 -- 172.21.15.40:0/1047734894 shutdown_connections 2023-07-18T03:30:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:27.570+0000 7eff06e30700 1 -- 172.21.15.40:0/1047734894 wait complete. 2023-07-18T03:30:27.999 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:27 smithi040 ceph-mon[96439]: pgmap v278: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-07-18T03:30:27.999 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:27 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:30:28.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- bash -c 'ceph orch device ls' 2023-07-18T03:30:28.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:27 smithi134 ceph-mon[98216]: pgmap v278: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-07-18T03:30:28.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:27 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:30:28.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:27 smithi064 ceph-mon[98059]: pgmap v278: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-07-18T03:30:28.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:27 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:30:28.566 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:30:29.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:28 smithi134 ceph-mon[98216]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:30:29.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:28 smithi040 ceph-mon[96439]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:30:29.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:28 smithi064 ceph-mon[98059]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:30:29.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.956+0000 7fda8316b700 1 -- 172.21.15.40:0/2203152132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c0fea80 msgr2=0x7fda7c0fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:29.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.956+0000 7fda8316b700 1 --2- 172.21.15.40:0/2203152132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c0fea80 0x7fda7c0fee90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fda700098f0 tx=0x7fda70009c00 comp rx=0 tx=0).stop 2023-07-18T03:30:29.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.956+0000 7fda8316b700 1 -- 172.21.15.40:0/2203152132 shutdown_connections 2023-07-18T03:30:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.956+0000 7fda8316b700 1 --2- 172.21.15.40:0/2203152132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda7c100610 0x7fda7c106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.956+0000 7fda8316b700 1 --2- 172.21.15.40:0/2203152132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c0fea80 0x7fda7c0fee90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.956+0000 7fda8316b700 1 --2- 172.21.15.40:0/2203152132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda7c0ffc80 0x7fda7c1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.956+0000 7fda8316b700 1 -- 172.21.15.40:0/2203152132 >> 172.21.15.40:0/2203152132 conn(0x7fda7c0fa0b0 msgr2=0x7fda7c0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.956+0000 7fda8316b700 1 -- 172.21.15.40:0/2203152132 shutdown_connections 2023-07-18T03:30:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.957+0000 7fda8316b700 1 -- 172.21.15.40:0/2203152132 wait complete. 2023-07-18T03:30:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.957+0000 7fda8316b700 1 Processor -- start 2023-07-18T03:30:29.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.957+0000 7fda8316b700 1 -- start start 2023-07-18T03:30:29.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.957+0000 7fda8316b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda7c0fea80 0x7fda7c195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:29.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.957+0000 7fda8316b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda7c0ffc80 0x7fda7c195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:29.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.957+0000 7fda8316b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c100610 0x7fda7c19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:29.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.957+0000 7fda8316b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c19d4a0 con 0x7fda7c100610 2023-07-18T03:30:29.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.957+0000 7fda8316b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c19d5e0 con 0x7fda7c0ffc80 2023-07-18T03:30:29.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.957+0000 7fda8316b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c19d770 con 0x7fda7c0fea80 2023-07-18T03:30:29.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.957+0000 7fda80f07700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda7c0fea80 0x7fda7c195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:29.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.957+0000 7fda80f07700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda7c0fea80 0x7fda7c195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:46220/0 (socket says 172.21.15.40:46220) 2023-07-18T03:30:29.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.958+0000 7fda81708700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c100610 0x7fda7c19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:29.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.958+0000 7fda80f07700 1 -- 172.21.15.40:0/2089205718 learned_addr learned my addr 172.21.15.40:0/2089205718 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:29.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.958+0000 7fda7bfff700 1 --2- 172.21.15.40:0/2089205718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda7c0ffc80 0x7fda7c195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:29.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.958+0000 7fda80f07700 1 -- 172.21.15.40:0/2089205718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda7c0ffc80 msgr2=0x7fda7c195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:29.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.958+0000 7fda80f07700 1 --2- 172.21.15.40:0/2089205718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda7c0ffc80 0x7fda7c195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:29.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.958+0000 7fda80f07700 1 -- 172.21.15.40:0/2089205718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c100610 msgr2=0x7fda7c19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:29.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.958+0000 7fda80f07700 1 --2- 172.21.15.40:0/2089205718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c100610 0x7fda7c19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:29.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.958+0000 7fda80f07700 1 -- 172.21.15.40:0/2089205718 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda70009520 con 0x7fda7c0fea80 2023-07-18T03:30:29.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.958+0000 7fda81708700 1 --2- 172.21.15.40:0/2089205718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c100610 0x7fda7c19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:29.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.958+0000 7fda80f07700 1 --2- 172.21.15.40:0/2089205718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda7c0fea80 0x7fda7c195910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fda70000c00 tx=0x7fda7000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:29.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.958+0000 7fda79ffb700 1 -- 172.21.15.40:0/2089205718 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7001c070 con 0x7fda7c0fea80 2023-07-18T03:30:29.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.958+0000 7fda8316b700 1 -- 172.21.15.40:0/2089205718 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7c19d9f0 con 0x7fda7c0fea80 2023-07-18T03:30:29.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.958+0000 7fda79ffb700 1 -- 172.21.15.40:0/2089205718 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda7000f860 con 0x7fda7c0fea80 2023-07-18T03:30:29.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.958+0000 7fda79ffb700 1 -- 172.21.15.40:0/2089205718 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70003d60 con 0x7fda7c0fea80 2023-07-18T03:30:29.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.958+0000 7fda8316b700 1 -- 172.21.15.40:0/2089205718 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7c19de60 con 0x7fda7c0fea80 2023-07-18T03:30:29.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.959+0000 7fda8316b700 1 -- 172.21.15.40:0/2089205718 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7c10b3b0 con 0x7fda7c0fea80 2023-07-18T03:30:29.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.961+0000 7fda79ffb700 1 -- 172.21.15.40:0/2089205718 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fda70005590 con 0x7fda7c0fea80 2023-07-18T03:30:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.961+0000 7fda79ffb700 1 --2- 172.21.15.40:0/2089205718 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fda6c0a11e0 0x7fda6c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.961+0000 7fda7bfff700 1 --2- 172.21.15.40:0/2089205718 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fda6c0a11e0 0x7fda6c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.961+0000 7fda79ffb700 1 -- 172.21.15.40:0/2089205718 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fda700cd800 con 0x7fda7c0fea80 2023-07-18T03:30:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.962+0000 7fda7bfff700 1 --2- 172.21.15.40:0/2089205718 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fda6c0a11e0 0x7fda6c0a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fda68005950 tx=0x7fda6800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:29.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:29.965+0000 7fda79ffb700 1 -- 172.21.15.40:0/2089205718 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fda70096f30 con 0x7fda7c0fea80 2023-07-18T03:30:30.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:29 smithi134 ceph-mon[98216]: pgmap v279: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-18T03:30:30.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:29 smithi040 ceph-mon[96439]: pgmap v279: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-18T03:30:30.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:29 smithi064 ceph-mon[98059]: pgmap v279: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-18T03:30:30.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:30.171+0000 7fda8316b700 1 -- 172.21.15.40:0/2089205718 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fda7c060950 con 0x7fda6c0a11e0 2023-07-18T03:30:30.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:30.173+0000 7fda79ffb700 1 -- 172.21.15.40:0/2089205718 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7fda7c060950 con 0x7fda6c0a11e0 2023-07-18T03:30:30.170 INFO:teuthology.orchestra.run.smithi040.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-07-18T03:30:30.170 INFO:teuthology.orchestra.run.smithi040.stdout:smithi040 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009J400BGN 372G No 2m ago LVM detected, locked 2023-07-18T03:30:30.170 INFO:teuthology.orchestra.run.smithi040.stdout:smithi064 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT534400C2400BGN 372G No 2m ago LVM detected, locked 2023-07-18T03:30:30.170 INFO:teuthology.orchestra.run.smithi040.stdout:smithi134 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62420002400BGN 372G No 106s ago LVM detected, locked 2023-07-18T03:30:30.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:30.175+0000 7fda8316b700 1 -- 172.21.15.40:0/2089205718 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fda6c0a11e0 msgr2=0x7fda6c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:30.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:30.175+0000 7fda8316b700 1 --2- 172.21.15.40:0/2089205718 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fda6c0a11e0 0x7fda6c0a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fda68005950 tx=0x7fda6800a300 comp rx=0 tx=0).stop 2023-07-18T03:30:30.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:30.175+0000 7fda8316b700 1 -- 172.21.15.40:0/2089205718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda7c0fea80 msgr2=0x7fda7c195910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:30.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:30.175+0000 7fda8316b700 1 --2- 172.21.15.40:0/2089205718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda7c0fea80 0x7fda7c195910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fda70000c00 tx=0x7fda7000b790 comp rx=0 tx=0).stop 2023-07-18T03:30:30.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:30.175+0000 7fda8316b700 1 -- 172.21.15.40:0/2089205718 shutdown_connections 2023-07-18T03:30:30.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:30.175+0000 7fda8316b700 1 --2- 172.21.15.40:0/2089205718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda7c100610 0x7fda7c19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:30.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:30.175+0000 7fda8316b700 1 --2- 172.21.15.40:0/2089205718 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fda6c0a11e0 0x7fda6c0a3690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:30.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:30.175+0000 7fda8316b700 1 --2- 172.21.15.40:0/2089205718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda7c0fea80 0x7fda7c195910 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:30.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:30.175+0000 7fda8316b700 1 --2- 172.21.15.40:0/2089205718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda7c0ffc80 0x7fda7c195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:30.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:30.175+0000 7fda8316b700 1 -- 172.21.15.40:0/2089205718 >> 172.21.15.40:0/2089205718 conn(0x7fda7c0fa0b0 msgr2=0x7fda7c0fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:30.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:30.175+0000 7fda8316b700 1 -- 172.21.15.40:0/2089205718 shutdown_connections 2023-07-18T03:30:30.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:30.175+0000 7fda8316b700 1 -- 172.21.15.40:0/2089205718 wait complete. 2023-07-18T03:30:30.535 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-18T03:30:30.544 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi040.front.sepia.ceph.com 2023-07-18T03:30:30.545 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- bash -c 'ceph fs dump' 2023-07-18T03:30:31.114 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:30:31.565 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:31 smithi040 ceph-mon[96439]: pgmap v280: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-18T03:30:31.565 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:31 smithi040 ceph-mon[96439]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:30:31.565 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:31 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:31.566 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:31 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:31.566 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:31 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:31.566 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:31 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:31.566 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:31 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:31.566 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:31 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:31.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:31 smithi134 ceph-mon[98216]: pgmap v280: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-18T03:30:31.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:31 smithi134 ceph-mon[98216]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:30:31.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:31 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:31.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:31 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:31.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:31 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:31.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:31 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:31.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:31 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:31.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:31 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:31.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:31 smithi064 ceph-mon[98059]: pgmap v280: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-18T03:30:31.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:31 smithi064 ceph-mon[98059]: from='client.24919 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-18T03:30:31.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:31 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:31 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:31 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:31 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:31 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:31.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:31 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.199+0000 7fc5b41ea700 1 -- 172.21.15.40:0/1200745758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5ac1058d0 msgr2=0x7fc5ac107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.199+0000 7fc5b41ea700 1 --2- 172.21.15.40:0/1200745758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5ac1058d0 0x7fc5ac107cb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400b0a0 tx=0x7fc5a400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.199+0000 7fc5b41ea700 1 -- 172.21.15.40:0/1200745758 shutdown_connections 2023-07-18T03:30:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.199+0000 7fc5b41ea700 1 --2- 172.21.15.40:0/1200745758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5ac1058d0 0x7fc5ac107cb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.199+0000 7fc5b41ea700 1 --2- 172.21.15.40:0/1200745758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5ac100690 0x7fc5ac102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.199+0000 7fc5b41ea700 1 --2- 172.21.15.40:0/1200745758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5ac102fb0 0x7fc5ac105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.199+0000 7fc5b41ea700 1 -- 172.21.15.40:0/1200745758 >> 172.21.15.40:0/1200745758 conn(0x7fc5ac0fa0b0 msgr2=0x7fc5ac0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:32.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.199+0000 7fc5b41ea700 1 -- 172.21.15.40:0/1200745758 shutdown_connections 2023-07-18T03:30:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.199+0000 7fc5b41ea700 1 -- 172.21.15.40:0/1200745758 wait complete. 2023-07-18T03:30:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.199+0000 7fc5b41ea700 1 Processor -- start 2023-07-18T03:30:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.199+0000 7fc5b41ea700 1 -- start start 2023-07-18T03:30:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b41ea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5ac100690 0x7fc5ac0ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b41ea700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5ac102fb0 0x7fc5ac0fe640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b41ea700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5ac1058d0 0x7fc5ac0feb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b41ea700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0ff300 con 0x7fc5ac100690 2023-07-18T03:30:32.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b41ea700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0ff470 con 0x7fc5ac102fb0 2023-07-18T03:30:32.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b41ea700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac19fa50 con 0x7fc5ac1058d0 2023-07-18T03:30:32.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b1f86700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5ac100690 0x7fc5ac0ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:32.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b1f86700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5ac100690 0x7fc5ac0ffff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57206/0 (socket says 172.21.15.40:57206) 2023-07-18T03:30:32.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b1f86700 1 -- 172.21.15.40:0/2225331773 learned_addr learned my addr 172.21.15.40:0/2225331773 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:32.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b2787700 1 --2- 172.21.15.40:0/2225331773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5ac1058d0 0x7fc5ac0feb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b1785700 1 --2- 172.21.15.40:0/2225331773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5ac102fb0 0x7fc5ac0fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b1f86700 1 -- 172.21.15.40:0/2225331773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5ac1058d0 msgr2=0x7fc5ac0feb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b1f86700 1 --2- 172.21.15.40:0/2225331773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5ac1058d0 0x7fc5ac0feb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b1f86700 1 -- 172.21.15.40:0/2225331773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5ac102fb0 msgr2=0x7fc5ac0fe640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b1f86700 1 --2- 172.21.15.40:0/2225331773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5ac102fb0 0x7fc5ac0fe640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b1f86700 1 -- 172.21.15.40:0/2225331773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a4009d00 con 0x7fc5ac100690 2023-07-18T03:30:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b1f86700 1 --2- 172.21.15.40:0/2225331773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5ac100690 0x7fc5ac0ffff0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800d950 tx=0x7fc5a800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5a2ffd700 1 -- 172.21.15.40:0/2225331773 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8009940 con 0x7fc5ac100690 2023-07-18T03:30:32.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5a2ffd700 1 -- 172.21.15.40:0/2225331773 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5a8010460 con 0x7fc5ac100690 2023-07-18T03:30:32.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5a2ffd700 1 -- 172.21.15.40:0/2225331773 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a800f620 con 0x7fc5ac100690 2023-07-18T03:30:32.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.200+0000 7fc5b41ea700 1 -- 172.21.15.40:0/2225331773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ac19fd30 con 0x7fc5ac100690 2023-07-18T03:30:32.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.201+0000 7fc5b41ea700 1 -- 172.21.15.40:0/2225331773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ac1a0180 con 0x7fc5ac100690 2023-07-18T03:30:32.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.202+0000 7fc5b41ea700 1 -- 172.21.15.40:0/2225331773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ac0fbc20 con 0x7fc5ac100690 2023-07-18T03:30:32.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.203+0000 7fc5a2ffd700 1 -- 172.21.15.40:0/2225331773 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc5a80108d0 con 0x7fc5ac100690 2023-07-18T03:30:32.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.203+0000 7fc5a2ffd700 1 --2- 172.21.15.40:0/2225331773 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5980a11e0 0x7fc5980a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:32.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.203+0000 7fc5b1785700 1 --2- 172.21.15.40:0/2225331773 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5980a11e0 0x7fc5980a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:32.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.203+0000 7fc5a2ffd700 1 -- 172.21.15.40:0/2225331773 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc5a80cc770 con 0x7fc5ac100690 2023-07-18T03:30:32.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.204+0000 7fc5b1785700 1 --2- 172.21.15.40:0/2225331773 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5980a11e0 0x7fc5980a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005fd0 tx=0x7fc59c005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:32.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.208+0000 7fc5a2ffd700 1 -- 172.21.15.40:0/2225331773 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc5a8095f10 con 0x7fc5ac100690 2023-07-18T03:30:32.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.451+0000 7fc5b41ea700 1 -- 172.21.15.40:0/2225331773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc5ac02cc70 con 0x7fc5ac100690 2023-07-18T03:30:32.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.451+0000 7fc5a2ffd700 1 -- 172.21.15.40:0/2225331773 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2924 (secure 0 0 0) 0x7fc5a8016b80 con 0x7fc5ac100690 2023-07-18T03:30:32.448 INFO:teuthology.orchestra.run.smithi040.stdout:e27 2023-07-18T03:30:32.449 INFO:teuthology.orchestra.run.smithi040.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-07-18T03:30:32.449 INFO:teuthology.orchestra.run.smithi040.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-18T03:30:32.449 INFO:teuthology.orchestra.run.smithi040.stdout:legacy client fscid: 1 2023-07-18T03:30:32.449 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:32.449 INFO:teuthology.orchestra.run.smithi040.stdout:Filesystem 'cephfs' (1) 2023-07-18T03:30:32.449 INFO:teuthology.orchestra.run.smithi040.stdout:fs_name cephfs 2023-07-18T03:30:32.450 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 27 2023-07-18T03:30:32.450 INFO:teuthology.orchestra.run.smithi040.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-07-18T03:30:32.450 INFO:teuthology.orchestra.run.smithi040.stdout:created 2023-07-18T03:29:26.718789+0000 2023-07-18T03:30:32.450 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2023-07-18T03:29:34.650353+0000 2023-07-18T03:30:32.450 INFO:teuthology.orchestra.run.smithi040.stdout:tableserver 0 2023-07-18T03:30:32.450 INFO:teuthology.orchestra.run.smithi040.stdout:root 0 2023-07-18T03:30:32.450 INFO:teuthology.orchestra.run.smithi040.stdout:session_timeout 300 2023-07-18T03:30:32.450 INFO:teuthology.orchestra.run.smithi040.stdout:session_autoclose 300 2023-07-18T03:30:32.451 INFO:teuthology.orchestra.run.smithi040.stdout:max_file_size 1099511627776 2023-07-18T03:30:32.451 INFO:teuthology.orchestra.run.smithi040.stdout:max_xattr_size 65536 2023-07-18T03:30:32.451 INFO:teuthology.orchestra.run.smithi040.stdout:required_client_features {} 2023-07-18T03:30:32.451 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure 0 2023-07-18T03:30:32.451 INFO:teuthology.orchestra.run.smithi040.stdout:last_failure_osd_epoch 0 2023-07-18T03:30:32.451 INFO:teuthology.orchestra.run.smithi040.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-18T03:30:32.451 INFO:teuthology.orchestra.run.smithi040.stdout:max_mds 5 2023-07-18T03:30:32.451 INFO:teuthology.orchestra.run.smithi040.stdout:in 0,1,2,3,4 2023-07-18T03:30:32.452 INFO:teuthology.orchestra.run.smithi040.stdout:up {0=24527,1=24523,2=24482,3=24452,4=24442} 2023-07-18T03:30:32.452 INFO:teuthology.orchestra.run.smithi040.stdout:failed 2023-07-18T03:30:32.452 INFO:teuthology.orchestra.run.smithi040.stdout:damaged 2023-07-18T03:30:32.452 INFO:teuthology.orchestra.run.smithi040.stdout:stopped 2023-07-18T03:30:32.452 INFO:teuthology.orchestra.run.smithi040.stdout:data_pools [3] 2023-07-18T03:30:32.452 INFO:teuthology.orchestra.run.smithi040.stdout:metadata_pool 2 2023-07-18T03:30:32.452 INFO:teuthology.orchestra.run.smithi040.stdout:inline_data disabled 2023-07-18T03:30:32.452 INFO:teuthology.orchestra.run.smithi040.stdout:balancer 2023-07-18T03:30:32.453 INFO:teuthology.orchestra.run.smithi040.stdout:bal_rank_mask -1 2023-07-18T03:30:32.453 INFO:teuthology.orchestra.run.smithi040.stdout:standby_count_wanted 0 2023-07-18T03:30:32.453 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b{0:24527} state up:active seq 3 addr [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:30:32.453 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c{0:24499} state up:standby-replay seq 1 addr [v2:172.21.15.134:6836/3509655945,v1:172.21.15.134:6837/3509655945] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:30:32.453 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.i{1:24523} state up:active seq 6 addr [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:30:32.453 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.f{1:24487} state up:standby-replay seq 1 addr [v2:172.21.15.134:6834/2946854745,v1:172.21.15.134:6835/2946854745] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:30:32.453 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.h{2:24482} state up:active seq 6 addr [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:30:32.454 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a{2:14613} state up:standby-replay seq 1 addr [v2:172.21.15.40:6840/3953074960,v1:172.21.15.40:6841/3953074960] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:30:32.454 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.e{3:24452} state up:active seq 8 addr [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:30:32.454 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.g{3:14601} state up:standby-replay seq 1 addr [v2:172.21.15.40:6838/1818167554,v1:172.21.15.40:6839/1818167554] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:30:32.454 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.l{4:24442} state up:active seq 11 addr [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:30:32.454 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d{4:14538} state up:standby-replay seq 1 addr [v2:172.21.15.40:6836/3377503144,v1:172.21.15.40:6837/3377503144] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:30:32.454 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:32.454 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:32.455 INFO:teuthology.orchestra.run.smithi040.stdout:Standby daemons: 2023-07-18T03:30:32.455 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:32.455 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.k{-1:14520} state up:standby seq 1 addr [v2:172.21.15.64:6832/646383396,v1:172.21.15.64:6833/646383396] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:30:32.455 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.j{-1:14532} state up:standby seq 1 addr [v2:172.21.15.40:6834/2289812880,v1:172.21.15.40:6835/2289812880] compat {c=[1],r=[1],i=[7ff]}] 2023-07-18T03:30:32.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.453+0000 7fc5b41ea700 1 -- 172.21.15.40:0/2225331773 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5980a11e0 msgr2=0x7fc5980a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:32.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.454+0000 7fc5b41ea700 1 --2- 172.21.15.40:0/2225331773 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5980a11e0 0x7fc5980a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005fd0 tx=0x7fc59c005f60 comp rx=0 tx=0).stop 2023-07-18T03:30:32.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.454+0000 7fc5b41ea700 1 -- 172.21.15.40:0/2225331773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5ac100690 msgr2=0x7fc5ac0ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:32.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.454+0000 7fc5b41ea700 1 --2- 172.21.15.40:0/2225331773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5ac100690 0x7fc5ac0ffff0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800d950 tx=0x7fc5a800dc60 comp rx=0 tx=0).stop 2023-07-18T03:30:32.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.454+0000 7fc5b41ea700 1 -- 172.21.15.40:0/2225331773 shutdown_connections 2023-07-18T03:30:32.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.454+0000 7fc5b41ea700 1 --2- 172.21.15.40:0/2225331773 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5980a11e0 0x7fc5980a3690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.454+0000 7fc5b41ea700 1 --2- 172.21.15.40:0/2225331773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5ac1058d0 0x7fc5ac0feb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.454+0000 7fc5b41ea700 1 --2- 172.21.15.40:0/2225331773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5ac100690 0x7fc5ac0ffff0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.454+0000 7fc5b41ea700 1 --2- 172.21.15.40:0/2225331773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5ac102fb0 0x7fc5ac0fe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.454+0000 7fc5b41ea700 1 -- 172.21.15.40:0/2225331773 >> 172.21.15.40:0/2225331773 conn(0x7fc5ac0fa0b0 msgr2=0x7fc5ac1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.454+0000 7fc5b41ea700 1 -- 172.21.15.40:0/2225331773 shutdown_connections 2023-07-18T03:30:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:32.454+0000 7fc5b41ea700 1 -- 172.21.15.40:0/2225331773 wait complete. 2023-07-18T03:30:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 27 2023-07-18T03:30:32.753 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:30:32.754 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:30:32.754 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:32.754 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-07-18T03:30:32.754 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-07-18T03:30:32.755 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2023-07-18T03:30:32.755 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-07-18T03:30:32.755 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2023-07-18T03:30:32.755 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2023-07-18T03:30:32.755 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2023-07-18T03:30:32.755 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2023-07-18T03:30:32.755 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-07-18T03:30:32.755 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2023-07-18T03:30:32.756 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:32 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2225331773' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-18T03:30:32.768 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- bash -c 'ceph osd dump' 2023-07-18T03:30:33.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:30:33.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:30:33.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:33.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-07-18T03:30:33.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-07-18T03:30:33.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2023-07-18T03:30:33.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-07-18T03:30:33.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2023-07-18T03:30:33.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2023-07-18T03:30:33.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2023-07-18T03:30:33.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2023-07-18T03:30:33.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-07-18T03:30:33.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2023-07-18T03:30:33.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:32 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2225331773' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-18T03:30:33.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:30:33.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:30:33.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:30:33.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-07-18T03:30:33.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-07-18T03:30:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2023-07-18T03:30:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-07-18T03:30:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2023-07-18T03:30:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2023-07-18T03:30:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2023-07-18T03:30:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2023-07-18T03:30:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2023-07-18T03:30:33.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2023-07-18T03:30:33.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:32 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2225331773' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-18T03:30:33.331 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:30:33.797 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:33 smithi040 ceph-mon[96439]: pgmap v281: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-18T03:30:33.798 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:33 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-07-18T03:30:33.798 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:33 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-07-18T03:30:33.798 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:33 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2023-07-18T03:30:33.798 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:33 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-07-18T03:30:33.798 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:33 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2023-07-18T03:30:33.798 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:33 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2023-07-18T03:30:33.799 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:33 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2023-07-18T03:30:33.799 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:33 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2023-07-18T03:30:33.799 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:33 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-07-18T03:30:33.799 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:33 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2023-07-18T03:30:33.799 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:33 smithi040 ceph-mon[96439]: osdmap e82: 12 total, 12 up, 12 in 2023-07-18T03:30:34.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:33 smithi134 ceph-mon[98216]: pgmap v281: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-18T03:30:34.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:33 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-07-18T03:30:34.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:33 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-07-18T03:30:34.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:33 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2023-07-18T03:30:34.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:33 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-07-18T03:30:34.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:33 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2023-07-18T03:30:34.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:33 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2023-07-18T03:30:34.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:33 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2023-07-18T03:30:34.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:33 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2023-07-18T03:30:34.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:33 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-07-18T03:30:34.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:33 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2023-07-18T03:30:34.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:33 smithi134 ceph-mon[98216]: osdmap e82: 12 total, 12 up, 12 in 2023-07-18T03:30:34.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:33 smithi064 ceph-mon[98059]: pgmap v281: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-07-18T03:30:34.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:33 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-07-18T03:30:34.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:33 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-07-18T03:30:34.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:33 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2023-07-18T03:30:34.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:33 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-07-18T03:30:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:33 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2023-07-18T03:30:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:33 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2023-07-18T03:30:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:33 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2023-07-18T03:30:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:33 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2023-07-18T03:30:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:33 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2023-07-18T03:30:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:33 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2023-07-18T03:30:34.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:33 smithi064 ceph-mon[98059]: osdmap e82: 12 total, 12 up, 12 in 2023-07-18T03:30:34.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.398+0000 7f6d5186c700 1 -- 172.21.15.40:0/2251857435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d4c100610 msgr2=0x7f6d4c106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:34.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.398+0000 7f6d5186c700 1 --2- 172.21.15.40:0/2251857435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d4c100610 0x7f6d4c106ed0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400b0a0 tx=0x7f6d4400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:34.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.398+0000 7f6d5186c700 1 -- 172.21.15.40:0/2251857435 shutdown_connections 2023-07-18T03:30:34.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.398+0000 7f6d5186c700 1 --2- 172.21.15.40:0/2251857435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d4c100610 0x7f6d4c106ed0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:34.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.398+0000 7f6d5186c700 1 --2- 172.21.15.40:0/2251857435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d4c0fea80 0x7f6d4c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:34.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.398+0000 7f6d5186c700 1 --2- 172.21.15.40:0/2251857435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d4c0ffc80 0x7f6d4c1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:34.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.398+0000 7f6d5186c700 1 -- 172.21.15.40:0/2251857435 >> 172.21.15.40:0/2251857435 conn(0x7f6d4c0fa0b0 msgr2=0x7f6d4c0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:34.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.399+0000 7f6d5186c700 1 -- 172.21.15.40:0/2251857435 shutdown_connections 2023-07-18T03:30:34.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.399+0000 7f6d5186c700 1 -- 172.21.15.40:0/2251857435 wait complete. 2023-07-18T03:30:34.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.399+0000 7f6d5186c700 1 Processor -- start 2023-07-18T03:30:34.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.399+0000 7f6d5186c700 1 -- start start 2023-07-18T03:30:34.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.399+0000 7f6d5186c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d4c0fea80 0x7f6d4c195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:34.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.399+0000 7f6d5186c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d4c0ffc80 0x7f6d4c195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:34.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d5186c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d4c100610 0x7f6d4c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:34.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d5186c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c19d4f0 con 0x7f6d4c0fea80 2023-07-18T03:30:34.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d5186c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c19d630 con 0x7f6d4c100610 2023-07-18T03:30:34.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d5186c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c19d7c0 con 0x7f6d4c0ffc80 2023-07-18T03:30:34.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d4affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d4c0fea80 0x7f6d4c195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:34.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d4affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d4c0fea80 0x7f6d4c195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57222/0 (socket says 172.21.15.40:57222) 2023-07-18T03:30:34.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d4affd700 1 -- 172.21.15.40:0/2575360000 learned_addr learned my addr 172.21.15.40:0/2575360000 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:34.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d4b7fe700 1 --2- 172.21.15.40:0/2575360000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d4c100610 0x7f6d4c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:34.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d4a7fc700 1 --2- 172.21.15.40:0/2575360000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d4c0ffc80 0x7f6d4c195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:34.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d4affd700 1 -- 172.21.15.40:0/2575360000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d4c0ffc80 msgr2=0x7f6d4c195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:34.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d4affd700 1 --2- 172.21.15.40:0/2575360000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d4c0ffc80 0x7f6d4c195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:34.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d4affd700 1 -- 172.21.15.40:0/2575360000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d4c100610 msgr2=0x7f6d4c19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:34.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d4affd700 1 --2- 172.21.15.40:0/2575360000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d4c100610 0x7f6d4c19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:34.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d4affd700 1 -- 172.21.15.40:0/2575360000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d44009d00 con 0x7f6d4c0fea80 2023-07-18T03:30:34.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d4affd700 1 --2- 172.21.15.40:0/2575360000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d4c0fea80 0x7f6d4c195960 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00d9b0 tx=0x7f6d3c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:34.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d5086a700 1 -- 172.21.15.40:0/2575360000 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c004020 con 0x7f6d4c0fea80 2023-07-18T03:30:34.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d5086a700 1 -- 172.21.15.40:0/2575360000 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d3c009730 con 0x7f6d4c0fea80 2023-07-18T03:30:34.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d5186c700 1 -- 172.21.15.40:0/2575360000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4c19daa0 con 0x7f6d4c0fea80 2023-07-18T03:30:34.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d5086a700 1 -- 172.21.15.40:0/2575360000 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c003a30 con 0x7f6d4c0fea80 2023-07-18T03:30:34.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.400+0000 7f6d5186c700 1 -- 172.21.15.40:0/2575360000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4c19dfa0 con 0x7f6d4c0fea80 2023-07-18T03:30:34.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.401+0000 7f6d5186c700 1 -- 172.21.15.40:0/2575360000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4c10a250 con 0x7f6d4c0fea80 2023-07-18T03:30:34.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.402+0000 7f6d5086a700 1 -- 172.21.15.40:0/2575360000 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6d3c0098a0 con 0x7f6d4c0fea80 2023-07-18T03:30:34.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.403+0000 7f6d5086a700 1 --2- 172.21.15.40:0/2575360000 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d340a11e0 0x7f6d340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:34.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.403+0000 7f6d4a7fc700 1 --2- 172.21.15.40:0/2575360000 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d340a11e0 0x7f6d340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:34.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.403+0000 7f6d5086a700 1 -- 172.21.15.40:0/2575360000 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6d3c0ccb90 con 0x7f6d4c0fea80 2023-07-18T03:30:34.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.404+0000 7f6d4a7fc700 1 --2- 172.21.15.40:0/2575360000 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d340a11e0 0x7f6d340a3690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6d40005fd0 tx=0x7f6d40005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:34.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.406+0000 7f6d5086a700 1 -- 172.21.15.40:0/2575360000 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6d3c096100 con 0x7f6d4c0fea80 2023-07-18T03:30:34.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.604+0000 7f6d5186c700 1 -- 172.21.15.40:0/2575360000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f6d4c19e610 con 0x7f6d4c0fea80 2023-07-18T03:30:34.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.605+0000 7f6d5086a700 1 -- 172.21.15.40:0/2575360000 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) v1 ==== 56+0+5319 (secure 0 0 0) 0x7f6d3c099720 con 0x7f6d4c0fea80 2023-07-18T03:30:34.602 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 83 2023-07-18T03:30:34.602 INFO:teuthology.orchestra.run.smithi040.stdout:fsid 3af3dba8-251a-11ee-9b34-001a4aab830c 2023-07-18T03:30:34.603 INFO:teuthology.orchestra.run.smithi040.stdout:created 2023-07-18T03:22:30.461503+0000 2023-07-18T03:30:34.603 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2023-07-18T03:30:33.646627+0000 2023-07-18T03:30:34.603 INFO:teuthology.orchestra.run.smithi040.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-07-18T03:30:34.604 INFO:teuthology.orchestra.run.smithi040.stdout:crush_version 26 2023-07-18T03:30:34.604 INFO:teuthology.orchestra.run.smithi040.stdout:full_ratio 0.95 2023-07-18T03:30:34.604 INFO:teuthology.orchestra.run.smithi040.stdout:backfillfull_ratio 0.9 2023-07-18T03:30:34.604 INFO:teuthology.orchestra.run.smithi040.stdout:nearfull_ratio 0.85 2023-07-18T03:30:34.604 INFO:teuthology.orchestra.run.smithi040.stdout:require_min_compat_client luminous 2023-07-18T03:30:34.605 INFO:teuthology.orchestra.run.smithi040.stdout:min_compat_client luminous 2023-07-18T03:30:34.605 INFO:teuthology.orchestra.run.smithi040.stdout:require_osd_release reef 2023-07-18T03:30:34.605 INFO:teuthology.orchestra.run.smithi040.stdout:stretch_mode_enabled false 2023-07-18T03:30:34.605 INFO:teuthology.orchestra.run.smithi040.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-07-18T03:30:34.605 INFO:teuthology.orchestra.run.smithi040.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-07-18T03:30:34.606 INFO:teuthology.orchestra.run.smithi040.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-07-18T03:30:34.606 INFO:teuthology.orchestra.run.smithi040.stdout:max_osd 12 2023-07-18T03:30:34.606 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 up in weight 1 up_from 8 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6802/3043492254,v1:172.21.15.40:6803/3043492254] [v2:172.21.15.40:6804/3043492254,v1:172.21.15.40:6805/3043492254] exists,up f924e504-7de2-48fb-820f-7b74c5a46ae3 2023-07-18T03:30:34.607 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 up in weight 1 up_from 13 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6800/2992805245,v1:172.21.15.64:6801/2992805245] [v2:172.21.15.64:6802/2992805245,v1:172.21.15.64:6803/2992805245] exists,up ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049 2023-07-18T03:30:34.607 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 up in weight 1 up_from 18 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6800/1877339886,v1:172.21.15.134:6801/1877339886] [v2:172.21.15.134:6802/1877339886,v1:172.21.15.134:6803/1877339886] exists,up e48fbcee-7d01-4dcf-845f-fc64645dca43 2023-07-18T03:30:34.607 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 up in weight 1 up_from 25 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6810/3916741213,v1:172.21.15.40:6811/3916741213] [v2:172.21.15.40:6812/3916741213,v1:172.21.15.40:6813/3916741213] exists,up 571c77cb-1acd-4d6f-a297-529dd5de162d 2023-07-18T03:30:34.607 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 up in weight 1 up_from 32 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6808/788347693,v1:172.21.15.64:6809/788347693] [v2:172.21.15.64:6810/788347693,v1:172.21.15.64:6811/788347693] exists,up 0b567530-058b-44db-b773-7c6f7224d36a 2023-07-18T03:30:34.607 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 up in weight 1 up_from 37 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6808/3544269956,v1:172.21.15.134:6809/3544269956] [v2:172.21.15.134:6810/3544269956,v1:172.21.15.134:6811/3544269956] exists,up 37241e63-af2a-400b-b5df-a15096c201e9 2023-07-18T03:30:34.608 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6 up in weight 1 up_from 44 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6818/380513677,v1:172.21.15.40:6819/380513677] [v2:172.21.15.40:6820/380513677,v1:172.21.15.40:6821/380513677] exists,up 36ea6ecd-85aa-42cb-ab41-006ab774af0e 2023-07-18T03:30:34.608 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7 up in weight 1 up_from 49 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6816/589664358,v1:172.21.15.64:6817/589664358] [v2:172.21.15.64:6818/589664358,v1:172.21.15.64:6819/589664358] exists,up 80a76d08-32dd-429e-b6e9-4a70a0f2b7e5 2023-07-18T03:30:34.608 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8 up in weight 1 up_from 56 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6816/2602435562,v1:172.21.15.134:6817/2602435562] [v2:172.21.15.134:6818/2602435562,v1:172.21.15.134:6819/2602435562] exists,up a0442a0d-4350-4f98-a6a1-346905e55e52 2023-07-18T03:30:34.608 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9 up in weight 1 up_from 64 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6826/2771612323,v1:172.21.15.40:6827/2771612323] [v2:172.21.15.40:6828/2771612323,v1:172.21.15.40:6829/2771612323] exists,up 56920a5e-7f3e-4776-921c-682aa946a6b7 2023-07-18T03:30:34.608 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10 up in weight 1 up_from 69 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6824/2791256757,v1:172.21.15.64:6825/2791256757] [v2:172.21.15.64:6826/2791256757,v1:172.21.15.64:6827/2791256757] exists,up 50f2740d-5e2a-43b5-9dd0-11d322238231 2023-07-18T03:30:34.608 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11 up in weight 1 up_from 76 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6824/1938250059,v1:172.21.15.134:6825/1938250059] [v2:172.21.15.134:6826/1938250059,v1:172.21.15.134:6827/1938250059] exists,up 735e8994-4fce-4cab-a980-dbfe3259b2ae 2023-07-18T03:30:34.609 INFO:teuthology.orchestra.run.smithi040.stdout:pg_upmap_items 2.f [8,11] 2023-07-18T03:30:34.609 INFO:teuthology.orchestra.run.smithi040.stdout:pg_upmap_items 2.13 [2,11] 2023-07-18T03:30:34.609 INFO:teuthology.orchestra.run.smithi040.stdout:pg_upmap_items 3.6 [4,7] 2023-07-18T03:30:34.609 INFO:teuthology.orchestra.run.smithi040.stdout:pg_upmap_items 3.11 [5,9] 2023-07-18T03:30:34.609 INFO:teuthology.orchestra.run.smithi040.stdout:pg_upmap_items 3.12 [3,9] 2023-07-18T03:30:34.609 INFO:teuthology.orchestra.run.smithi040.stdout:pg_upmap_items 3.17 [4,6] 2023-07-18T03:30:34.610 INFO:teuthology.orchestra.run.smithi040.stdout:pg_upmap_items 3.1a [5,6] 2023-07-18T03:30:34.610 INFO:teuthology.orchestra.run.smithi040.stdout:pg_upmap_items 3.21 [4,8] 2023-07-18T03:30:34.610 INFO:teuthology.orchestra.run.smithi040.stdout:pg_upmap_items 3.23 [4,8] 2023-07-18T03:30:34.610 INFO:teuthology.orchestra.run.smithi040.stdout:pg_upmap_items 3.27 [3,9] 2023-07-18T03:30:34.610 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:0/2185187717 expires 2023-07-19T03:23:31.734610+0000 2023-07-18T03:30:34.610 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:0/908447335 expires 2023-07-19T03:23:31.734610+0000 2023-07-18T03:30:34.610 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:6800/1112125515 expires 2023-07-19T03:23:31.734610+0000 2023-07-18T03:30:34.611 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:6801/1112125515 expires 2023-07-19T03:23:31.734610+0000 2023-07-18T03:30:34.611 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:0/3890436231 expires 2023-07-19T03:23:05.482888+0000 2023-07-18T03:30:34.611 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:0/3748912525 expires 2023-07-19T03:23:05.482888+0000 2023-07-18T03:30:34.611 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:6800/1353038297 expires 2023-07-19T03:23:05.482888+0000 2023-07-18T03:30:34.611 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:0/101292551 expires 2023-07-19T03:23:05.482888+0000 2023-07-18T03:30:34.612 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:6801/1353038297 expires 2023-07-19T03:23:05.482888+0000 2023-07-18T03:30:34.612 INFO:teuthology.orchestra.run.smithi040.stdout:blocklist 172.21.15.40:0/1171137337 expires 2023-07-19T03:23:31.734610+0000 2023-07-18T03:30:34.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.607+0000 7f6d5186c700 1 -- 172.21.15.40:0/2575360000 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d340a11e0 msgr2=0x7f6d340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:34.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.607+0000 7f6d5186c700 1 --2- 172.21.15.40:0/2575360000 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d340a11e0 0x7f6d340a3690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6d40005fd0 tx=0x7f6d40005db0 comp rx=0 tx=0).stop 2023-07-18T03:30:34.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.607+0000 7f6d5186c700 1 -- 172.21.15.40:0/2575360000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d4c0fea80 msgr2=0x7f6d4c195960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:34.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.607+0000 7f6d5186c700 1 --2- 172.21.15.40:0/2575360000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d4c0fea80 0x7f6d4c195960 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00d9b0 tx=0x7f6d3c00dcc0 comp rx=0 tx=0).stop 2023-07-18T03:30:34.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.607+0000 7f6d5186c700 1 -- 172.21.15.40:0/2575360000 shutdown_connections 2023-07-18T03:30:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.607+0000 7f6d5186c700 1 --2- 172.21.15.40:0/2575360000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d4c100610 0x7f6d4c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.607+0000 7f6d5186c700 1 --2- 172.21.15.40:0/2575360000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d4c0fea80 0x7f6d4c195960 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.607+0000 7f6d5186c700 1 --2- 172.21.15.40:0/2575360000 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d340a11e0 0x7f6d340a3690 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.607+0000 7f6d5186c700 1 --2- 172.21.15.40:0/2575360000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d4c0ffc80 0x7f6d4c195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.607+0000 7f6d5186c700 1 -- 172.21.15.40:0/2575360000 >> 172.21.15.40:0/2575360000 conn(0x7f6d4c0fa0b0 msgr2=0x7f6d4c0fa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.608+0000 7f6d5186c700 1 -- 172.21.15.40:0/2575360000 shutdown_connections 2023-07-18T03:30:34.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:34.608+0000 7f6d5186c700 1 -- 172.21.15.40:0/2575360000 wait complete. 2023-07-18T03:30:34.919 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:34 smithi040 ceph-mon[96439]: osdmap e83: 12 total, 12 up, 12 in 2023-07-18T03:30:34.919 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:34 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2575360000' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-18T03:30:34.993 INFO:teuthology.run_tasks:Running task fs.ready... 2023-07-18T03:30:35.008 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:30:35.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:34 smithi134 ceph-mon[98216]: osdmap e83: 12 total, 12 up, 12 in 2023-07-18T03:30:35.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:34 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2575360000' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-18T03:30:35.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:34 smithi064 ceph-mon[98059]: osdmap e83: 12 total, 12 up, 12 in 2023-07-18T03:30:35.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:34 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2575360000' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-18T03:30:35.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.141+0000 7f8bdd47a700 1 -- 172.21.15.40:0/4169892233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bd80fffd0 msgr2=0x7f8bd81003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:35.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.141+0000 7f8bdd47a700 1 --2- 172.21.15.40:0/4169892233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bd80fffd0 0x7f8bd81003e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009990 tx=0x7f8bc8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.141+0000 7f8bdd47a700 1 -- 172.21.15.40:0/4169892233 shutdown_connections 2023-07-18T03:30:35.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.141+0000 7f8bdd47a700 1 --2- 172.21.15.40:0/4169892233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd80ff5d0 0x7f8bd8106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.141+0000 7f8bdd47a700 1 --2- 172.21.15.40:0/4169892233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bd80fffd0 0x7f8bd81003e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.141+0000 7f8bdd47a700 1 --2- 172.21.15.40:0/4169892233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bd80fec20 0x7f8bd80ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.141+0000 7f8bdd47a700 1 -- 172.21.15.40:0/4169892233 >> 172.21.15.40:0/4169892233 conn(0x7f8bd80f97a0 msgr2=0x7f8bd80fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.141+0000 7f8bdd47a700 1 -- 172.21.15.40:0/4169892233 shutdown_connections 2023-07-18T03:30:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.141+0000 7f8bdd47a700 1 -- 172.21.15.40:0/4169892233 wait complete. 2023-07-18T03:30:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.142+0000 7f8bdd47a700 1 Processor -- start 2023-07-18T03:30:35.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.142+0000 7f8bdd47a700 1 -- start start 2023-07-18T03:30:35.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.142+0000 7f8bdd47a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd80fec20 0x7f8bd8193580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:35.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.142+0000 7f8bdd47a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bd80ff5d0 0x7f8bd8193ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:35.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.142+0000 7f8bdd47a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bd80fffd0 0x7f8bd819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:35.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.142+0000 7f8bdd47a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd819b100 con 0x7f8bd80fffd0 2023-07-18T03:30:35.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.142+0000 7f8bdd47a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd819b240 con 0x7f8bd80fec20 2023-07-18T03:30:35.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.142+0000 7f8bdd47a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd819b3d0 con 0x7f8bd80ff5d0 2023-07-18T03:30:35.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.142+0000 7f8bd77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bd80fffd0 0x7f8bd819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:35.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.142+0000 7f8bd77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bd80fffd0 0x7f8bd819ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57240/0 (socket says 172.21.15.40:57240) 2023-07-18T03:30:35.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.142+0000 7f8bd77fe700 1 -- 172.21.15.40:0/311055310 learned_addr learned my addr 172.21.15.40:0/311055310 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:35.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.142+0000 7f8bd6ffd700 1 --2- 172.21.15.40:0/311055310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd80fec20 0x7f8bd8193580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:35.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.142+0000 7f8bd67fc700 1 --2- 172.21.15.40:0/311055310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bd80ff5d0 0x7f8bd8193ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:35.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.142+0000 7f8bd77fe700 1 -- 172.21.15.40:0/311055310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bd80ff5d0 msgr2=0x7f8bd8193ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:35.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.143+0000 7f8bd77fe700 1 --2- 172.21.15.40:0/311055310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bd80ff5d0 0x7f8bd8193ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.143+0000 7f8bd77fe700 1 -- 172.21.15.40:0/311055310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd80fec20 msgr2=0x7f8bd8193580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:35.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.143+0000 7f8bd77fe700 1 --2- 172.21.15.40:0/311055310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd80fec20 0x7f8bd8193580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.143+0000 7f8bd77fe700 1 -- 172.21.15.40:0/311055310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc80095f0 con 0x7f8bd80fffd0 2023-07-18T03:30:35.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.143+0000 7f8bd77fe700 1 --2- 172.21.15.40:0/311055310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bd80fffd0 0x7f8bd819ab30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0007ee0 tx=0x7f8bd000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:35.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.143+0000 7f8bbffff700 1 -- 172.21.15.40:0/311055310 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0017070 con 0x7f8bd80fffd0 2023-07-18T03:30:35.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.143+0000 7f8bbffff700 1 -- 172.21.15.40:0/311055310 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bd000f040 con 0x7f8bd80fffd0 2023-07-18T03:30:35.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.143+0000 7f8bdd47a700 1 -- 172.21.15.40:0/311055310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd819b6b0 con 0x7f8bd80fffd0 2023-07-18T03:30:35.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.143+0000 7f8bbffff700 1 -- 172.21.15.40:0/311055310 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0005580 con 0x7f8bd80fffd0 2023-07-18T03:30:35.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.143+0000 7f8bdd47a700 1 -- 172.21.15.40:0/311055310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd819bb70 con 0x7f8bd80fffd0 2023-07-18T03:30:35.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.144+0000 7f8bdd47a700 1 -- 172.21.15.40:0/311055310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd81058d0 con 0x7f8bd80fffd0 2023-07-18T03:30:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.145+0000 7f8bbffff700 1 -- 172.21.15.40:0/311055310 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8bd000db20 con 0x7f8bd80fffd0 2023-07-18T03:30:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.146+0000 7f8bbffff700 1 --2- 172.21.15.40:0/311055310 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8bc00a3430 0x7f8bc00a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.146+0000 7f8bd6ffd700 1 --2- 172.21.15.40:0/311055310 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8bc00a3430 0x7f8bc00a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.146+0000 7f8bbffff700 1 -- 172.21.15.40:0/311055310 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8bd00cc250 con 0x7f8bd80fffd0 2023-07-18T03:30:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.146+0000 7f8bd6ffd700 1 --2- 172.21.15.40:0/311055310 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8bc00a3430 0x7f8bc00a58e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800b5c0 tx=0x7f8bc8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.150+0000 7f8bbffff700 1 -- 172.21.15.40:0/311055310 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8bd00957c0 con 0x7f8bd80fffd0 2023-07-18T03:30:35.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.348+0000 7f8bdd47a700 1 -- 172.21.15.40:0/311055310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bd802ca20 con 0x7f8bd80fffd0 2023-07-18T03:30:35.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.350+0000 7f8bbffff700 1 -- 172.21.15.40:0/311055310 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22599 (secure 0 0 0) 0x7f8bd0098de0 con 0x7f8bd80fffd0 2023-07-18T03:30:35.348 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:35.349 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":83,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:30:33.646627+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:30:35.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.352+0000 7f8bdd47a700 1 -- 172.21.15.40:0/311055310 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8bc00a3430 msgr2=0x7f8bc00a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:35.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.352+0000 7f8bdd47a700 1 --2- 172.21.15.40:0/311055310 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8bc00a3430 0x7f8bc00a58e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800b5c0 tx=0x7f8bc8005fd0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.352+0000 7f8bdd47a700 1 -- 172.21.15.40:0/311055310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bd80fffd0 msgr2=0x7f8bd819ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:35.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.352+0000 7f8bdd47a700 1 --2- 172.21.15.40:0/311055310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bd80fffd0 0x7f8bd819ab30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0007ee0 tx=0x7f8bd000d3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.352+0000 7f8bdd47a700 1 -- 172.21.15.40:0/311055310 shutdown_connections 2023-07-18T03:30:35.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.352+0000 7f8bdd47a700 1 --2- 172.21.15.40:0/311055310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bd80fffd0 0x7f8bd819ab30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.352+0000 7f8bdd47a700 1 --2- 172.21.15.40:0/311055310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd80fec20 0x7f8bd8193580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.352+0000 7f8bdd47a700 1 --2- 172.21.15.40:0/311055310 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8bc00a3430 0x7f8bc00a58e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.352+0000 7f8bdd47a700 1 --2- 172.21.15.40:0/311055310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8bd80ff5d0 0x7f8bd8193ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.352+0000 7f8bdd47a700 1 -- 172.21.15.40:0/311055310 >> 172.21.15.40:0/311055310 conn(0x7f8bd80f97a0 msgr2=0x7f8bd80fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:35.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.352+0000 7f8bdd47a700 1 -- 172.21.15.40:0/311055310 shutdown_connections 2023-07-18T03:30:35.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.352+0000 7f8bdd47a700 1 -- 172.21.15.40:0/311055310 wait complete. 2023-07-18T03:30:35.361 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:25:38.267262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:29:24.727790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:29:26.078507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:30:35.362 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:30:35.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.494+0000 7ff36b445700 1 -- 172.21.15.40:0/381854641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3640fe9e0 msgr2=0x7ff3640fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:35.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.495+0000 7ff36b445700 1 --2- 172.21.15.40:0/381854641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3640fe9e0 0x7ff3640fedf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff354009990 tx=0x7ff354009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.495+0000 7ff36b445700 1 -- 172.21.15.40:0/381854641 shutdown_connections 2023-07-18T03:30:35.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.495+0000 7ff36b445700 1 --2- 172.21.15.40:0/381854641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff364100560 0x7ff364106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.495+0000 7ff36b445700 1 --2- 172.21.15.40:0/381854641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3640fe9e0 0x7ff3640fedf0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.495+0000 7ff36b445700 1 --2- 172.21.15.40:0/381854641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3640ffbd0 0x7ff364100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.495+0000 7ff36b445700 1 -- 172.21.15.40:0/381854641 >> 172.21.15.40:0/381854641 conn(0x7ff3640fa000 msgr2=0x7ff3640fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:35.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.495+0000 7ff36b445700 1 -- 172.21.15.40:0/381854641 shutdown_connections 2023-07-18T03:30:35.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.495+0000 7ff36b445700 1 -- 172.21.15.40:0/381854641 wait complete. 2023-07-18T03:30:35.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.495+0000 7ff36b445700 1 Processor -- start 2023-07-18T03:30:35.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.495+0000 7ff36b445700 1 -- start start 2023-07-18T03:30:35.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.495+0000 7ff36b445700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3640fe9e0 0x7ff364195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:35.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.495+0000 7ff36b445700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3640ffbd0 0x7ff364195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:35.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.495+0000 7ff36b445700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff364100560 0x7ff36419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:35.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.495+0000 7ff36b445700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36419d3d0 con 0x7ff364100560 2023-07-18T03:30:35.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.495+0000 7ff36b445700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36419d510 con 0x7ff3640fe9e0 2023-07-18T03:30:35.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.495+0000 7ff36b445700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36419d6a0 con 0x7ff3640ffbd0 2023-07-18T03:30:35.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.496+0000 7ff3699e2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff364100560 0x7ff36419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:35.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.496+0000 7ff3699e2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff364100560 0x7ff36419ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57254/0 (socket says 172.21.15.40:57254) 2023-07-18T03:30:35.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.496+0000 7ff3699e2700 1 -- 172.21.15.40:0/1827322349 learned_addr learned my addr 172.21.15.40:0/1827322349 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:35.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.496+0000 7ff3689e0700 1 --2- 172.21.15.40:0/1827322349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3640ffbd0 0x7ff364195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:35.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.496+0000 7ff3691e1700 1 --2- 172.21.15.40:0/1827322349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3640fe9e0 0x7ff364195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:35.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.496+0000 7ff3699e2700 1 -- 172.21.15.40:0/1827322349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3640ffbd0 msgr2=0x7ff364195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:35.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.496+0000 7ff3699e2700 1 --2- 172.21.15.40:0/1827322349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3640ffbd0 0x7ff364195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.496+0000 7ff3699e2700 1 -- 172.21.15.40:0/1827322349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3640fe9e0 msgr2=0x7ff364195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:35.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.496+0000 7ff3699e2700 1 --2- 172.21.15.40:0/1827322349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3640fe9e0 0x7ff364195850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.496+0000 7ff3699e2700 1 -- 172.21.15.40:0/1827322349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3540095f0 con 0x7ff364100560 2023-07-18T03:30:35.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.496+0000 7ff3699e2700 1 --2- 172.21.15.40:0/1827322349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff364100560 0x7ff36419ce00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff35c007ee0 tx=0x7ff35c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:35.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.496+0000 7ff35a7fc700 1 -- 172.21.15.40:0/1827322349 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c017070 con 0x7ff364100560 2023-07-18T03:30:35.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.496+0000 7ff35a7fc700 1 -- 172.21.15.40:0/1827322349 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff35c00f040 con 0x7ff364100560 2023-07-18T03:30:35.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.496+0000 7ff36b445700 1 -- 172.21.15.40:0/1827322349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff36419d980 con 0x7ff364100560 2023-07-18T03:30:35.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.496+0000 7ff35a7fc700 1 -- 172.21.15.40:0/1827322349 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c005580 con 0x7ff364100560 2023-07-18T03:30:35.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.496+0000 7ff36b445700 1 -- 172.21.15.40:0/1827322349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36419de90 con 0x7ff364100560 2023-07-18T03:30:35.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.497+0000 7ff36b445700 1 -- 172.21.15.40:0/1827322349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff364105960 con 0x7ff364100560 2023-07-18T03:30:35.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.498+0000 7ff35a7fc700 1 -- 172.21.15.40:0/1827322349 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff35c00db20 con 0x7ff364100560 2023-07-18T03:30:35.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.499+0000 7ff35a7fc700 1 --2- 172.21.15.40:0/1827322349 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff3500a11e0 0x7ff3500a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:35.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.499+0000 7ff35a7fc700 1 -- 172.21.15.40:0/1827322349 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff35c0cc830 con 0x7ff364100560 2023-07-18T03:30:35.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.499+0000 7ff3691e1700 1 --2- 172.21.15.40:0/1827322349 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff3500a11e0 0x7ff3500a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:35.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.499+0000 7ff3691e1700 1 --2- 172.21.15.40:0/1827322349 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff3500a11e0 0x7ff3500a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff35400b5c0 tx=0x7ff354005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:35.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.502+0000 7ff35a7fc700 1 -- 172.21.15.40:0/1827322349 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff35c095e00 con 0x7ff364100560 2023-07-18T03:30:35.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.704+0000 7ff36b445700 1 -- 172.21.15.40:0/1827322349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff36402ca60 con 0x7ff364100560 2023-07-18T03:30:35.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.704+0000 7ff35a7fc700 1 -- 172.21.15.40:0/1827322349 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff35c099420 con 0x7ff364100560 2023-07-18T03:30:35.701 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-07-18T03:30:35.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.707+0000 7ff36b445700 1 -- 172.21.15.40:0/1827322349 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff3500a11e0 msgr2=0x7ff3500a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:35.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.707+0000 7ff36b445700 1 --2- 172.21.15.40:0/1827322349 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff3500a11e0 0x7ff3500a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff35400b5c0 tx=0x7ff354005960 comp rx=0 tx=0).stop 2023-07-18T03:30:35.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.707+0000 7ff36b445700 1 -- 172.21.15.40:0/1827322349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff364100560 msgr2=0x7ff36419ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:35.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.707+0000 7ff36b445700 1 --2- 172.21.15.40:0/1827322349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff364100560 0x7ff36419ce00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff35c007ee0 tx=0x7ff35c00d3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.707+0000 7ff36b445700 1 -- 172.21.15.40:0/1827322349 shutdown_connections 2023-07-18T03:30:35.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.707+0000 7ff36b445700 1 --2- 172.21.15.40:0/1827322349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff364100560 0x7ff36419ce00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.707+0000 7ff36b445700 1 --2- 172.21.15.40:0/1827322349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3640fe9e0 0x7ff364195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.707+0000 7ff36b445700 1 --2- 172.21.15.40:0/1827322349 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff3500a11e0 0x7ff3500a3690 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.707+0000 7ff36b445700 1 --2- 172.21.15.40:0/1827322349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3640ffbd0 0x7ff364195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.707+0000 7ff36b445700 1 -- 172.21.15.40:0/1827322349 >> 172.21.15.40:0/1827322349 conn(0x7ff3640fa000 msgr2=0x7ff3640fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:35.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.707+0000 7ff36b445700 1 -- 172.21.15.40:0/1827322349 shutdown_connections 2023-07-18T03:30:35.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.707+0000 7ff36b445700 1 -- 172.21.15.40:0/1827322349 wait complete. 2023-07-18T03:30:35.716 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:30:35.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.849+0000 7f4990f7d700 1 -- 172.21.15.40:0/1596971623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c0ffbd0 msgr2=0x7f498c100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:35.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.849+0000 7f4990f7d700 1 --2- 172.21.15.40:0/1596971623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c0ffbd0 0x7f498c100020 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f497c009990 tx=0x7f497c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.849+0000 7f4990f7d700 1 -- 172.21.15.40:0/1596971623 shutdown_connections 2023-07-18T03:30:35.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.849+0000 7f4990f7d700 1 --2- 172.21.15.40:0/1596971623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f498c100560 0x7f498c106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.849+0000 7f4990f7d700 1 --2- 172.21.15.40:0/1596971623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f498c0fe9e0 0x7f498c0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.849+0000 7f4990f7d700 1 --2- 172.21.15.40:0/1596971623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c0ffbd0 0x7f498c100020 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.849+0000 7f4990f7d700 1 -- 172.21.15.40:0/1596971623 >> 172.21.15.40:0/1596971623 conn(0x7f498c0fa000 msgr2=0x7f498c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:35.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.849+0000 7f4990f7d700 1 -- 172.21.15.40:0/1596971623 shutdown_connections 2023-07-18T03:30:35.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.849+0000 7f4990f7d700 1 -- 172.21.15.40:0/1596971623 wait complete. 2023-07-18T03:30:35.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.850+0000 7f4990f7d700 1 Processor -- start 2023-07-18T03:30:35.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.850+0000 7f4990f7d700 1 -- start start 2023-07-18T03:30:35.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.850+0000 7f4990f7d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f498c0fe9e0 0x7f498c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:35.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.850+0000 7f4990f7d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f498c0ffbd0 0x7f498c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:35.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.850+0000 7f4990f7d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c100560 0x7f498c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:35.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.850+0000 7f4990f7d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c19f5d0 con 0x7f498c100560 2023-07-18T03:30:35.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.850+0000 7f4990f7d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c19f710 con 0x7f498c0fe9e0 2023-07-18T03:30:35.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.850+0000 7f4990f7d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c19f8a0 con 0x7f498c0ffbd0 2023-07-18T03:30:35.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.850+0000 7f498ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c100560 0x7f498c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:35.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.851+0000 7f498ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c100560 0x7f498c19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57268/0 (socket says 172.21.15.40:57268) 2023-07-18T03:30:35.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.851+0000 7f498ad9d700 1 -- 172.21.15.40:0/1601601319 learned_addr learned my addr 172.21.15.40:0/1601601319 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:35.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.851+0000 7f498a59c700 1 --2- 172.21.15.40:0/1601601319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f498c0fe9e0 0x7f498c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:35.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.851+0000 7f4989d9b700 1 --2- 172.21.15.40:0/1601601319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f498c0ffbd0 0x7f498c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:35.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.851+0000 7f498ad9d700 1 -- 172.21.15.40:0/1601601319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f498c0ffbd0 msgr2=0x7f498c197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:35.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.851+0000 7f498ad9d700 1 --2- 172.21.15.40:0/1601601319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f498c0ffbd0 0x7f498c197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.851+0000 7f498ad9d700 1 -- 172.21.15.40:0/1601601319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f498c0fe9e0 msgr2=0x7f498c197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:35.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.851+0000 7f498ad9d700 1 --2- 172.21.15.40:0/1601601319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f498c0fe9e0 0x7f498c197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:35.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.851+0000 7f498ad9d700 1 -- 172.21.15.40:0/1601601319 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497c0095f0 con 0x7f498c100560 2023-07-18T03:30:35.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.851+0000 7f498ad9d700 1 --2- 172.21.15.40:0/1601601319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c100560 0x7f498c19f000 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f498400ef30 tx=0x7f498400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:35.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.851+0000 7f49837fe700 1 -- 172.21.15.40:0/1601601319 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984018070 con 0x7f498c100560 2023-07-18T03:30:35.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.851+0000 7f49837fe700 1 -- 172.21.15.40:0/1601601319 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f498400f040 con 0x7f498c100560 2023-07-18T03:30:35.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.851+0000 7f4990f7d700 1 -- 172.21.15.40:0/1601601319 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498c19fb20 con 0x7f498c100560 2023-07-18T03:30:35.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.851+0000 7f49837fe700 1 -- 172.21.15.40:0/1601601319 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498400cdb0 con 0x7f498c100560 2023-07-18T03:30:35.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.851+0000 7f4990f7d700 1 -- 172.21.15.40:0/1601601319 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498c19ffe0 con 0x7f498c100560 2023-07-18T03:30:35.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.852+0000 7f4990f7d700 1 -- 172.21.15.40:0/1601601319 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498c10b320 con 0x7f498c100560 2023-07-18T03:30:35.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.853+0000 7f49837fe700 1 -- 172.21.15.40:0/1601601319 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4984003a00 con 0x7f498c100560 2023-07-18T03:30:35.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.854+0000 7f49837fe700 1 --2- 172.21.15.40:0/1601601319 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f49780a11e0 0x7f49780a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:35.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.854+0000 7f49837fe700 1 -- 172.21.15.40:0/1601601319 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f49840cc650 con 0x7f498c100560 2023-07-18T03:30:35.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.854+0000 7f498a59c700 1 --2- 172.21.15.40:0/1601601319 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f49780a11e0 0x7f49780a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:35.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.854+0000 7f498a59c700 1 --2- 172.21.15.40:0/1601601319 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f49780a11e0 0x7f49780a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f498c0ffa30 tx=0x7f4974007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:35.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:35.858+0000 7f49837fe700 1 -- 172.21.15.40:0/1601601319 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4984095cb0 con 0x7f498c100560 2023-07-18T03:30:36.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.057+0000 7f4990f7d700 1 -- 172.21.15.40:0/1601601319 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f498c1a0590 con 0x7f498c100560 2023-07-18T03:30:36.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.058+0000 7f49837fe700 1 -- 172.21.15.40:0/1601601319 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f49840992d0 con 0x7f498c100560 2023-07-18T03:30:36.055 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-07-18T03:30:36.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.060+0000 7f4990f7d700 1 -- 172.21.15.40:0/1601601319 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f49780a11e0 msgr2=0x7f49780a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:36.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.060+0000 7f4990f7d700 1 --2- 172.21.15.40:0/1601601319 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f49780a11e0 0x7f49780a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f498c0ffa30 tx=0x7f4974007400 comp rx=0 tx=0).stop 2023-07-18T03:30:36.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.060+0000 7f4990f7d700 1 -- 172.21.15.40:0/1601601319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c100560 msgr2=0x7f498c19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:36.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.060+0000 7f4990f7d700 1 --2- 172.21.15.40:0/1601601319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c100560 0x7f498c19f000 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f498400ef30 tx=0x7f498400c3a0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.060+0000 7f4990f7d700 1 -- 172.21.15.40:0/1601601319 shutdown_connections 2023-07-18T03:30:36.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.060+0000 7f4990f7d700 1 --2- 172.21.15.40:0/1601601319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f498c100560 0x7f498c19f000 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.060+0000 7f4990f7d700 1 --2- 172.21.15.40:0/1601601319 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f498c0fe9e0 0x7f498c197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.060+0000 7f4990f7d700 1 --2- 172.21.15.40:0/1601601319 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f49780a11e0 0x7f49780a3690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.060+0000 7f4990f7d700 1 --2- 172.21.15.40:0/1601601319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f498c0ffbd0 0x7f498c197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.060+0000 7f4990f7d700 1 -- 172.21.15.40:0/1601601319 >> 172.21.15.40:0/1601601319 conn(0x7f498c0fa000 msgr2=0x7f498c0fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:36.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.060+0000 7f4990f7d700 1 -- 172.21.15.40:0/1601601319 shutdown_connections 2023-07-18T03:30:36.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.060+0000 7f4990f7d700 1 -- 172.21.15.40:0/1601601319 wait complete. 2023-07-18T03:30:36.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:35 smithi134 ceph-mon[98216]: pgmap v284: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2023-07-18T03:30:36.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:35 smithi134 ceph-mon[98216]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-07-18T03:30:36.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:35 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/311055310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:36.067 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:35 smithi040 ceph-mon[96439]: pgmap v284: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2023-07-18T03:30:36.068 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:35 smithi040 ceph-mon[96439]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-07-18T03:30:36.068 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:35 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/311055310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:36.069 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:30:36.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:35 smithi064 ceph-mon[98059]: pgmap v284: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2023-07-18T03:30:36.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:35 smithi064 ceph-mon[98059]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-07-18T03:30:36.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:35 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/311055310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.201+0000 7f1486f91700 1 -- 172.21.15.40:0/2756061321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14800fe9e0 msgr2=0x7f14800fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.201+0000 7f1486f91700 1 --2- 172.21.15.40:0/2756061321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14800fe9e0 0x7f14800fedf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1474009990 tx=0x7f1474009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.201+0000 7f1486f91700 1 -- 172.21.15.40:0/2756061321 shutdown_connections 2023-07-18T03:30:36.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.201+0000 7f1486f91700 1 --2- 172.21.15.40:0/2756061321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1480100560 0x7f1480106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.201+0000 7f1486f91700 1 --2- 172.21.15.40:0/2756061321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14800fe9e0 0x7f14800fedf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.201+0000 7f1486f91700 1 --2- 172.21.15.40:0/2756061321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14800ffbd0 0x7f1480100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.201+0000 7f1486f91700 1 -- 172.21.15.40:0/2756061321 >> 172.21.15.40:0/2756061321 conn(0x7f14800fa000 msgr2=0x7f14800fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.201+0000 7f1486f91700 1 -- 172.21.15.40:0/2756061321 shutdown_connections 2023-07-18T03:30:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.201+0000 7f1486f91700 1 -- 172.21.15.40:0/2756061321 wait complete. 2023-07-18T03:30:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f1486f91700 1 Processor -- start 2023-07-18T03:30:36.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f1486f91700 1 -- start start 2023-07-18T03:30:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f1486f91700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14800fe9e0 0x7f14800681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f1486f91700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14800ffbd0 0x7f148019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f1486f91700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1480100560 0x7f148019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:36.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f1486f91700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148019d340 con 0x7f1480100560 2023-07-18T03:30:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f1486f91700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148019d4b0 con 0x7f14800fe9e0 2023-07-18T03:30:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f1486f91700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148019d670 con 0x7f14800ffbd0 2023-07-18T03:30:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f148552e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1480100560 0x7f148019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f148552e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1480100560 0x7f148019cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57282/0 (socket says 172.21.15.40:57282) 2023-07-18T03:30:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f148552e700 1 -- 172.21.15.40:0/1577320034 learned_addr learned my addr 172.21.15.40:0/1577320034 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f1484d2d700 1 --2- 172.21.15.40:0/1577320034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14800fe9e0 0x7f14800681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:36.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f148552e700 1 -- 172.21.15.40:0/1577320034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14800ffbd0 msgr2=0x7f148019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:36.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f147ffff700 1 --2- 172.21.15.40:0/1577320034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14800ffbd0 0x7f148019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:36.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f148552e700 1 --2- 172.21.15.40:0/1577320034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14800ffbd0 0x7f148019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f148552e700 1 -- 172.21.15.40:0/1577320034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14800fe9e0 msgr2=0x7f14800681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:36.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f148552e700 1 --2- 172.21.15.40:0/1577320034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14800fe9e0 0x7f14800681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.202+0000 7f148552e700 1 -- 172.21.15.40:0/1577320034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14740095f0 con 0x7f1480100560 2023-07-18T03:30:36.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.203+0000 7f148552e700 1 --2- 172.21.15.40:0/1577320034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1480100560 0x7f148019cba0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f147800ef30 tx=0x7f147800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:36.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.203+0000 7f147dffb700 1 -- 172.21.15.40:0/1577320034 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478018070 con 0x7f1480100560 2023-07-18T03:30:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.203+0000 7f147dffb700 1 -- 172.21.15.40:0/1577320034 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f147800f040 con 0x7f1480100560 2023-07-18T03:30:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.203+0000 7f1486f91700 1 -- 172.21.15.40:0/1577320034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148019d8f0 con 0x7f1480100560 2023-07-18T03:30:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.203+0000 7f147dffb700 1 -- 172.21.15.40:0/1577320034 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147800cdb0 con 0x7f1480100560 2023-07-18T03:30:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.203+0000 7f1486f91700 1 -- 172.21.15.40:0/1577320034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148019ddb0 con 0x7f1480100560 2023-07-18T03:30:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.204+0000 7f1486f91700 1 -- 172.21.15.40:0/1577320034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1480105960 con 0x7f1480100560 2023-07-18T03:30:36.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.205+0000 7f147dffb700 1 -- 172.21.15.40:0/1577320034 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1478003a00 con 0x7f1480100560 2023-07-18T03:30:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.206+0000 7f147dffb700 1 --2- 172.21.15.40:0/1577320034 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14700a11e0 0x7f14700a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.206+0000 7f1484d2d700 1 --2- 172.21.15.40:0/1577320034 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14700a11e0 0x7f14700a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.206+0000 7f147dffb700 1 -- 172.21.15.40:0/1577320034 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f14780cc750 con 0x7f1480100560 2023-07-18T03:30:36.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.206+0000 7f1484d2d700 1 --2- 172.21.15.40:0/1577320034 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14700a11e0 0x7f14700a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f147400b2e0 tx=0x7f1474005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:36.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.210+0000 7f147dffb700 1 -- 172.21.15.40:0/1577320034 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1478095d50 con 0x7f1480100560 2023-07-18T03:30:36.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.409+0000 7f1486f91700 1 -- 172.21.15.40:0/1577320034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f148002ca50 con 0x7f1480100560 2023-07-18T03:30:36.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.409+0000 7f147dffb700 1 -- 172.21.15.40:0/1577320034 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1478099370 con 0x7f1480100560 2023-07-18T03:30:36.406 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-07-18T03:30:36.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.411+0000 7f1486f91700 1 -- 172.21.15.40:0/1577320034 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14700a11e0 msgr2=0x7f14700a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:36.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.411+0000 7f1486f91700 1 --2- 172.21.15.40:0/1577320034 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14700a11e0 0x7f14700a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f147400b2e0 tx=0x7f1474005960 comp rx=0 tx=0).stop 2023-07-18T03:30:36.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.411+0000 7f1486f91700 1 -- 172.21.15.40:0/1577320034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1480100560 msgr2=0x7f148019cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:36.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.411+0000 7f1486f91700 1 --2- 172.21.15.40:0/1577320034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1480100560 0x7f148019cba0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f147800ef30 tx=0x7f147800c3a0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.411+0000 7f1486f91700 1 -- 172.21.15.40:0/1577320034 shutdown_connections 2023-07-18T03:30:36.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.411+0000 7f1486f91700 1 --2- 172.21.15.40:0/1577320034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1480100560 0x7f148019cba0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.411+0000 7f1486f91700 1 --2- 172.21.15.40:0/1577320034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14800fe9e0 0x7f14800681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.411+0000 7f1486f91700 1 --2- 172.21.15.40:0/1577320034 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f14700a11e0 0x7f14700a3690 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.411+0000 7f1486f91700 1 --2- 172.21.15.40:0/1577320034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14800ffbd0 0x7f148019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.411+0000 7f1486f91700 1 -- 172.21.15.40:0/1577320034 >> 172.21.15.40:0/1577320034 conn(0x7f14800fa000 msgr2=0x7f14800fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:36.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.412+0000 7f1486f91700 1 -- 172.21.15.40:0/1577320034 shutdown_connections 2023-07-18T03:30:36.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.412+0000 7f1486f91700 1 -- 172.21.15.40:0/1577320034 wait complete. 2023-07-18T03:30:36.420 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:30:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.549+0000 7f671aee7700 1 -- 172.21.15.40:0/2625901878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6714100600 msgr2=0x7f67141029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:36.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.549+0000 7f671aee7700 1 --2- 172.21.15.40:0/2625901878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6714100600 0x7f67141029e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6708009990 tx=0x7f6708009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.549+0000 7f671aee7700 1 -- 172.21.15.40:0/2625901878 shutdown_connections 2023-07-18T03:30:36.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.549+0000 7f671aee7700 1 --2- 172.21.15.40:0/2625901878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6714105840 0x7f6714107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.549+0000 7f671aee7700 1 --2- 172.21.15.40:0/2625901878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6714100600 0x7f67141029e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.549+0000 7f671aee7700 1 --2- 172.21.15.40:0/2625901878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6714102f20 0x7f6714105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.549+0000 7f671aee7700 1 -- 172.21.15.40:0/2625901878 >> 172.21.15.40:0/2625901878 conn(0x7f67140f9fe0 msgr2=0x7f67140fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:36.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.549+0000 7f671aee7700 1 -- 172.21.15.40:0/2625901878 shutdown_connections 2023-07-18T03:30:36.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.549+0000 7f671aee7700 1 -- 172.21.15.40:0/2625901878 wait complete. 2023-07-18T03:30:36.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.550+0000 7f671aee7700 1 Processor -- start 2023-07-18T03:30:36.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.550+0000 7f671aee7700 1 -- start start 2023-07-18T03:30:36.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.550+0000 7f671aee7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6714100600 0x7f67140fff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:36.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.550+0000 7f671aee7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6714102f20 0x7f67140fe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.550+0000 7f671aee7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6714105840 0x7f67140feaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.550+0000 7f671aee7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67140ff290 con 0x7f6714100600 2023-07-18T03:30:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.550+0000 7f671aee7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67140ff400 con 0x7f6714102f20 2023-07-18T03:30:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.550+0000 7f671aee7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671419f850 con 0x7f6714105840 2023-07-18T03:30:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.550+0000 7f6718c83700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6714100600 0x7f67140fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.550+0000 7f6718c83700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6714100600 0x7f67140fff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57292/0 (socket says 172.21.15.40:57292) 2023-07-18T03:30:36.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.550+0000 7f6718c83700 1 -- 172.21.15.40:0/2196187745 learned_addr learned my addr 172.21.15.40:0/2196187745 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.550+0000 7f6713fff700 1 --2- 172.21.15.40:0/2196187745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6714102f20 0x7f67140fe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.551+0000 7f6719484700 1 --2- 172.21.15.40:0/2196187745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6714105840 0x7f67140feaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.551+0000 7f6718c83700 1 -- 172.21.15.40:0/2196187745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6714105840 msgr2=0x7f67140feaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.551+0000 7f6718c83700 1 --2- 172.21.15.40:0/2196187745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6714105840 0x7f67140feaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.551+0000 7f6718c83700 1 -- 172.21.15.40:0/2196187745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6714102f20 msgr2=0x7f67140fe5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.551+0000 7f6718c83700 1 --2- 172.21.15.40:0/2196187745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6714102f20 0x7f67140fe5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.551+0000 7f6718c83700 1 -- 172.21.15.40:0/2196187745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67080095f0 con 0x7f6714100600 2023-07-18T03:30:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.551+0000 7f6718c83700 1 --2- 172.21.15.40:0/2196187745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6714100600 0x7f67140fff70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6708000c00 tx=0x7f670800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.551+0000 7f6711ffb700 1 -- 172.21.15.40:0/2196187745 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670801d070 con 0x7f6714100600 2023-07-18T03:30:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.551+0000 7f671aee7700 1 -- 172.21.15.40:0/2196187745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671419fad0 con 0x7f6714100600 2023-07-18T03:30:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.551+0000 7f6711ffb700 1 -- 172.21.15.40:0/2196187745 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f670800fb50 con 0x7f6714100600 2023-07-18T03:30:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.551+0000 7f6711ffb700 1 -- 172.21.15.40:0/2196187745 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6708017650 con 0x7f6714100600 2023-07-18T03:30:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.551+0000 7f671aee7700 1 -- 172.21.15.40:0/2196187745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671419ffd0 con 0x7f6714100600 2023-07-18T03:30:36.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.552+0000 7f671aee7700 1 -- 172.21.15.40:0/2196187745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67140fbb50 con 0x7f6714100600 2023-07-18T03:30:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.553+0000 7f6711ffb700 1 -- 172.21.15.40:0/2196187745 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f670800fcc0 con 0x7f6714100600 2023-07-18T03:30:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.554+0000 7f6711ffb700 1 --2- 172.21.15.40:0/2196187745 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67040a1220 0x7f67040a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.554+0000 7f6713fff700 1 --2- 172.21.15.40:0/2196187745 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67040a1220 0x7f67040a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:36.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.554+0000 7f6711ffb700 1 -- 172.21.15.40:0/2196187745 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f67080cde10 con 0x7f6714100600 2023-07-18T03:30:36.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.555+0000 7f6713fff700 1 --2- 172.21.15.40:0/2196187745 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67040a1220 0x7f67040a36d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6700005950 tx=0x7f670000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:36.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.559+0000 7f6711ffb700 1 -- 172.21.15.40:0/2196187745 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6708097470 con 0x7f6714100600 2023-07-18T03:30:36.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.804+0000 7f671aee7700 1 -- 172.21.15.40:0/2196187745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f671402ca00 con 0x7f6714100600 2023-07-18T03:30:36.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.805+0000 7f6711ffb700 1 -- 172.21.15.40:0/2196187745 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10888 (secure 0 0 0) 0x7f670809aa90 con 0x7f6714100600 2023-07-18T03:30:36.802 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:36.803 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:29:26.718789+0000","modified":"2023-07-18T03:29:34.650353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24523,"mds_2":24482,"mds_3":24452,"mds_4":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14601":{"gid":14601,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1818167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1818167554},{"type":"v1","addr":"172.21.15.40:6839","nonce":1818167554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14613":{"gid":14613,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/3953074960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":3953074960},{"type":"v1","addr":"172.21.15.40:6841","nonce":3953074960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3509655945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3509655945},{"type":"v1","addr":"172.21.15.134:6837","nonce":3509655945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/3007039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3007039043},{"type":"v1","addr":"172.21.15.64:6839","nonce":3007039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:30:36.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.807+0000 7f671aee7700 1 -- 172.21.15.40:0/2196187745 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67040a1220 msgr2=0x7f67040a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:36.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.807+0000 7f671aee7700 1 --2- 172.21.15.40:0/2196187745 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67040a1220 0x7f67040a36d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6700005950 tx=0x7f670000b410 comp rx=0 tx=0).stop 2023-07-18T03:30:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.807+0000 7f671aee7700 1 -- 172.21.15.40:0/2196187745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6714100600 msgr2=0x7f67140fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.807+0000 7f671aee7700 1 --2- 172.21.15.40:0/2196187745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6714100600 0x7f67140fff70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6708000c00 tx=0x7f670800f710 comp rx=0 tx=0).stop 2023-07-18T03:30:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.807+0000 7f671aee7700 1 -- 172.21.15.40:0/2196187745 shutdown_connections 2023-07-18T03:30:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.807+0000 7f671aee7700 1 --2- 172.21.15.40:0/2196187745 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67040a1220 0x7f67040a36d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.807+0000 7f671aee7700 1 --2- 172.21.15.40:0/2196187745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6714105840 0x7f67140feaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.807+0000 7f671aee7700 1 --2- 172.21.15.40:0/2196187745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6714100600 0x7f67140fff70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.807+0000 7f671aee7700 1 --2- 172.21.15.40:0/2196187745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6714102f20 0x7f67140fe5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.807+0000 7f671aee7700 1 -- 172.21.15.40:0/2196187745 >> 172.21.15.40:0/2196187745 conn(0x7f67140f9fe0 msgr2=0x7f6714101280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:36.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.807+0000 7f671aee7700 1 -- 172.21.15.40:0/2196187745 shutdown_connections 2023-07-18T03:30:36.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.808+0000 7f671aee7700 1 -- 172.21.15.40:0/2196187745 wait complete. 2023-07-18T03:30:36.807 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 27 2023-07-18T03:30:36.815 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:36 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1827322349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:36.815 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:36 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1601601319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:36.816 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:36 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1577320034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:36.817 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:30:36.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.947+0000 7fb636e32700 1 -- 172.21.15.40:0/4185393812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb630100550 msgr2=0x7fb630106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:36.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.947+0000 7fb636e32700 1 --2- 172.21.15.40:0/4185393812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb630100550 0x7fb630106e10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb62800b0a0 tx=0x7fb62800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.947+0000 7fb636e32700 1 -- 172.21.15.40:0/4185393812 shutdown_connections 2023-07-18T03:30:36.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.947+0000 7fb636e32700 1 --2- 172.21.15.40:0/4185393812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb630100550 0x7fb630106e10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.947+0000 7fb636e32700 1 --2- 172.21.15.40:0/4185393812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6300fe9d0 0x7fb6300fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.947+0000 7fb636e32700 1 --2- 172.21.15.40:0/4185393812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6300ffbc0 0x7fb630100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.947+0000 7fb636e32700 1 -- 172.21.15.40:0/4185393812 >> 172.21.15.40:0/4185393812 conn(0x7fb6300f9fd0 msgr2=0x7fb6300fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:36.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.947+0000 7fb636e32700 1 -- 172.21.15.40:0/4185393812 shutdown_connections 2023-07-18T03:30:36.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.947+0000 7fb636e32700 1 -- 172.21.15.40:0/4185393812 wait complete. 2023-07-18T03:30:36.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.948+0000 7fb636e32700 1 Processor -- start 2023-07-18T03:30:36.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.948+0000 7fb636e32700 1 -- start start 2023-07-18T03:30:36.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.948+0000 7fb636e32700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6300fe9d0 0x7fb6301958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:36.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.948+0000 7fb636e32700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6300ffbc0 0x7fb630195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:36.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.948+0000 7fb636e32700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb630100550 0x7fb63019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:36.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.948+0000 7fb636e32700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63019d430 con 0x7fb6300ffbc0 2023-07-18T03:30:36.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.948+0000 7fb636e32700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63019d570 con 0x7fb6300fe9d0 2023-07-18T03:30:36.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.948+0000 7fb636e32700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63019d700 con 0x7fb630100550 2023-07-18T03:30:36.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.948+0000 7fb62ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6300ffbc0 0x7fb630195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:36.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.948+0000 7fb62ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6300ffbc0 0x7fb630195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57308/0 (socket says 172.21.15.40:57308) 2023-07-18T03:30:36.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.948+0000 7fb634bce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6300fe9d0 0x7fb6301958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:36.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.948+0000 7fb62ffff700 1 -- 172.21.15.40:0/1111751818 learned_addr learned my addr 172.21.15.40:0/1111751818 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:36.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.948+0000 7fb6353cf700 1 --2- 172.21.15.40:0/1111751818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb630100550 0x7fb63019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:36.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.949+0000 7fb62ffff700 1 -- 172.21.15.40:0/1111751818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb630100550 msgr2=0x7fb63019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:36.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.949+0000 7fb62ffff700 1 --2- 172.21.15.40:0/1111751818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb630100550 0x7fb63019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.949+0000 7fb62ffff700 1 -- 172.21.15.40:0/1111751818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6300fe9d0 msgr2=0x7fb6301958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:36.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.949+0000 7fb62ffff700 1 --2- 172.21.15.40:0/1111751818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6300fe9d0 0x7fb6301958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:36.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.949+0000 7fb62ffff700 1 -- 172.21.15.40:0/1111751818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb628009d00 con 0x7fb6300ffbc0 2023-07-18T03:30:36.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.949+0000 7fb6353cf700 1 --2- 172.21.15.40:0/1111751818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb630100550 0x7fb63019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:36.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.949+0000 7fb62ffff700 1 --2- 172.21.15.40:0/1111751818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6300ffbc0 0x7fb630195df0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00c920 tx=0x7fb61c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:36.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.949+0000 7fb62dffb700 1 -- 172.21.15.40:0/1111751818 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c007b00 con 0x7fb6300ffbc0 2023-07-18T03:30:36.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.949+0000 7fb62dffb700 1 -- 172.21.15.40:0/1111751818 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb61c00ce70 con 0x7fb6300ffbc0 2023-07-18T03:30:36.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.949+0000 7fb636e32700 1 -- 172.21.15.40:0/1111751818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63019d9e0 con 0x7fb6300ffbc0 2023-07-18T03:30:36.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.949+0000 7fb62dffb700 1 -- 172.21.15.40:0/1111751818 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0054e0 con 0x7fb6300ffbc0 2023-07-18T03:30:36.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.949+0000 7fb636e32700 1 -- 172.21.15.40:0/1111751818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63019df40 con 0x7fb6300ffbc0 2023-07-18T03:30:36.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.950+0000 7fb636e32700 1 -- 172.21.15.40:0/1111751818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb630106990 con 0x7fb6300ffbc0 2023-07-18T03:30:36.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.951+0000 7fb62dffb700 1 -- 172.21.15.40:0/1111751818 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb61c004750 con 0x7fb6300ffbc0 2023-07-18T03:30:36.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.952+0000 7fb62dffb700 1 --2- 172.21.15.40:0/1111751818 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb6200a12a0 0x7fb6200a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.952+0000 7fb62dffb700 1 -- 172.21.15.40:0/1111751818 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb61c0cc570 con 0x7fb6300ffbc0 2023-07-18T03:30:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.952+0000 7fb634bce700 1 --2- 172.21.15.40:0/1111751818 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb6200a12a0 0x7fb6200a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.952+0000 7fb634bce700 1 --2- 172.21.15.40:0/1111751818 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb6200a12a0 0x7fb6200a3750 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb6300ffa20 tx=0x7fb62400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:36.956+0000 7fb62dffb700 1 -- 172.21.15.40:0/1111751818 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb61c095b40 con 0x7fb6300ffbc0 2023-07-18T03:30:37.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:36 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1827322349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:37.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:36 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1601601319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:37.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:36 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1577320034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:37.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:36 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1827322349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:37.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:36 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1601601319' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:37.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:36 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1577320034' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:37.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.156+0000 7fb636e32700 1 -- 172.21.15.40:0/1111751818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb63002ca90 con 0x7fb6300ffbc0 2023-07-18T03:30:37.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.157+0000 7fb62dffb700 1 -- 172.21.15.40:0/1111751818 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22599 (secure 0 0 0) 0x7fb61c099160 con 0x7fb6300ffbc0 2023-07-18T03:30:37.154 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:37.155 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":83,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:30:33.646627+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:30:37.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.159+0000 7fb636e32700 1 -- 172.21.15.40:0/1111751818 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb6200a12a0 msgr2=0x7fb6200a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:37.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.159+0000 7fb636e32700 1 --2- 172.21.15.40:0/1111751818 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb6200a12a0 0x7fb6200a3750 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb6300ffa20 tx=0x7fb62400b410 comp rx=0 tx=0).stop 2023-07-18T03:30:37.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.159+0000 7fb636e32700 1 -- 172.21.15.40:0/1111751818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6300ffbc0 msgr2=0x7fb630195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:37.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.159+0000 7fb636e32700 1 --2- 172.21.15.40:0/1111751818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6300ffbc0 0x7fb630195df0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00c920 tx=0x7fb61c00cc30 comp rx=0 tx=0).stop 2023-07-18T03:30:37.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.160+0000 7fb636e32700 1 -- 172.21.15.40:0/1111751818 shutdown_connections 2023-07-18T03:30:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.160+0000 7fb636e32700 1 --2- 172.21.15.40:0/1111751818 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb6200a12a0 0x7fb6200a3750 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.160+0000 7fb636e32700 1 --2- 172.21.15.40:0/1111751818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb630100550 0x7fb63019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.160+0000 7fb636e32700 1 --2- 172.21.15.40:0/1111751818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb6300fe9d0 0x7fb6301958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.160+0000 7fb636e32700 1 --2- 172.21.15.40:0/1111751818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6300ffbc0 0x7fb630195df0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.160+0000 7fb636e32700 1 -- 172.21.15.40:0/1111751818 >> 172.21.15.40:0/1111751818 conn(0x7fb6300f9fd0 msgr2=0x7fb6300fc100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.160+0000 7fb636e32700 1 -- 172.21.15.40:0/1111751818 shutdown_connections 2023-07-18T03:30:37.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.160+0000 7fb636e32700 1 -- 172.21.15.40:0/1111751818 wait complete. 2023-07-18T03:30:37.169 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:25:38.267262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:29:24.727790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:29:26.078507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:30:37.170 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:30:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.303+0000 7f2fa9c65700 1 -- 172.21.15.40:0/3056697522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fa4100cd0 msgr2=0x7f2fa4101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.303+0000 7f2fa9c65700 1 --2- 172.21.15.40:0/3056697522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fa4100cd0 0x7f2fa4101120 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2f940098f0 tx=0x7f2f94009c00 comp rx=0 tx=0).stop 2023-07-18T03:30:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.303+0000 7f2fa9c65700 1 -- 172.21.15.40:0/3056697522 shutdown_connections 2023-07-18T03:30:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.303+0000 7f2fa9c65700 1 --2- 172.21.15.40:0/3056697522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa4101660 0x7f2fa4107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.303+0000 7f2fa9c65700 1 --2- 172.21.15.40:0/3056697522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fa40ffae0 0x7f2fa40ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.303+0000 7f2fa9c65700 1 --2- 172.21.15.40:0/3056697522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fa4100cd0 0x7f2fa4101120 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.303+0000 7f2fa9c65700 1 -- 172.21.15.40:0/3056697522 >> 172.21.15.40:0/3056697522 conn(0x7f2fa40fb120 msgr2=0x7f2fa40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:37.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.303+0000 7f2fa9c65700 1 -- 172.21.15.40:0/3056697522 shutdown_connections 2023-07-18T03:30:37.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.303+0000 7f2fa9c65700 1 -- 172.21.15.40:0/3056697522 wait complete. 2023-07-18T03:30:37.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.303+0000 7f2fa9c65700 1 Processor -- start 2023-07-18T03:30:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa9c65700 1 -- start start 2023-07-18T03:30:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa9c65700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fa40ffae0 0x7f2fa4195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa9c65700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa4100cd0 0x7f2fa4195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa9c65700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fa4101660 0x7f2fa419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:37.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa9c65700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa419d3e0 con 0x7f2fa4100cd0 2023-07-18T03:30:37.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa9c65700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa419d520 con 0x7f2fa40ffae0 2023-07-18T03:30:37.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa9c65700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa419d6b0 con 0x7f2fa4101660 2023-07-18T03:30:37.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa4100cd0 0x7f2fa4195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:37.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fa40ffae0 0x7f2fa4195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:37.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa4100cd0 0x7f2fa4195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57322/0 (socket says 172.21.15.40:57322) 2023-07-18T03:30:37.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa2ffd700 1 -- 172.21.15.40:0/4168584401 learned_addr learned my addr 172.21.15.40:0/4168584401 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa3fff700 1 --2- 172.21.15.40:0/4168584401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fa4101660 0x7f2fa419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa2ffd700 1 -- 172.21.15.40:0/4168584401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fa4101660 msgr2=0x7f2fa419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa2ffd700 1 --2- 172.21.15.40:0/4168584401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fa4101660 0x7f2fa419ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa2ffd700 1 -- 172.21.15.40:0/4168584401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fa40ffae0 msgr2=0x7f2fa4195860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa2ffd700 1 --2- 172.21.15.40:0/4168584401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fa40ffae0 0x7f2fa4195860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa2ffd700 1 -- 172.21.15.40:0/4168584401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9c009d00 con 0x7f2fa4100cd0 2023-07-18T03:30:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.304+0000 7f2fa37fe700 1 --2- 172.21.15.40:0/4168584401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fa40ffae0 0x7f2fa4195860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.305+0000 7f2fa2ffd700 1 --2- 172.21.15.40:0/4168584401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa4100cd0 0x7f2fa4195da0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400b730 tx=0x7f2f9400b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:37.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.305+0000 7f2fa0ff9700 1 -- 172.21.15.40:0/4168584401 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9401c070 con 0x7f2fa4100cd0 2023-07-18T03:30:37.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.305+0000 7f2fa9c65700 1 -- 172.21.15.40:0/4168584401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f94009520 con 0x7f2fa4100cd0 2023-07-18T03:30:37.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.305+0000 7f2fa0ff9700 1 -- 172.21.15.40:0/4168584401 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f9400f7b0 con 0x7f2fa4100cd0 2023-07-18T03:30:37.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.305+0000 7f2fa0ff9700 1 -- 172.21.15.40:0/4168584401 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f94004a00 con 0x7f2fa4100cd0 2023-07-18T03:30:37.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.305+0000 7f2fa9c65700 1 -- 172.21.15.40:0/4168584401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa419dbc0 con 0x7f2fa4100cd0 2023-07-18T03:30:37.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.306+0000 7f2fa9c65700 1 -- 172.21.15.40:0/4168584401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa4106a60 con 0x7f2fa4100cd0 2023-07-18T03:30:37.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.307+0000 7f2fa0ff9700 1 -- 172.21.15.40:0/4168584401 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2f94020070 con 0x7f2fa4100cd0 2023-07-18T03:30:37.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.307+0000 7f2fa0ff9700 1 --2- 172.21.15.40:0/4168584401 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2f900a11e0 0x7f2f900a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:37.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.307+0000 7f2fa0ff9700 1 -- 172.21.15.40:0/4168584401 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2f940cded0 con 0x7f2fa4100cd0 2023-07-18T03:30:37.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.307+0000 7f2fa37fe700 1 --2- 172.21.15.40:0/4168584401 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2f900a11e0 0x7f2f900a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:37.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.308+0000 7f2fa37fe700 1 --2- 172.21.15.40:0/4168584401 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2f900a11e0 0x7f2f900a3690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4100b30 tx=0x7f2f8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:37.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.311+0000 7f2fa0ff9700 1 -- 172.21.15.40:0/4168584401 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2f94097530 con 0x7f2fa4100cd0 2023-07-18T03:30:37.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.510+0000 7f2fa9c65700 1 -- 172.21.15.40:0/4168584401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2fa402caa0 con 0x7f2fa4100cd0 2023-07-18T03:30:37.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.511+0000 7f2fa0ff9700 1 -- 172.21.15.40:0/4168584401 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f9409ab50 con 0x7f2fa4100cd0 2023-07-18T03:30:37.507 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-07-18T03:30:37.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.513+0000 7f2fa9c65700 1 -- 172.21.15.40:0/4168584401 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2f900a11e0 msgr2=0x7f2f900a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:37.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.513+0000 7f2fa9c65700 1 --2- 172.21.15.40:0/4168584401 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2f900a11e0 0x7f2f900a3690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4100b30 tx=0x7f2f8c009260 comp rx=0 tx=0).stop 2023-07-18T03:30:37.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.513+0000 7f2fa9c65700 1 -- 172.21.15.40:0/4168584401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa4100cd0 msgr2=0x7f2fa4195da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:37.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.513+0000 7f2fa9c65700 1 --2- 172.21.15.40:0/4168584401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa4100cd0 0x7f2fa4195da0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400b730 tx=0x7f2f9400b760 comp rx=0 tx=0).stop 2023-07-18T03:30:37.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.513+0000 7f2fa9c65700 1 -- 172.21.15.40:0/4168584401 shutdown_connections 2023-07-18T03:30:37.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.513+0000 7f2fa9c65700 1 --2- 172.21.15.40:0/4168584401 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2f900a11e0 0x7f2f900a3690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.513+0000 7f2fa9c65700 1 --2- 172.21.15.40:0/4168584401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fa4101660 0x7f2fa419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.513+0000 7f2fa9c65700 1 --2- 172.21.15.40:0/4168584401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fa40ffae0 0x7f2fa4195860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.513+0000 7f2fa9c65700 1 --2- 172.21.15.40:0/4168584401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa4100cd0 0x7f2fa4195da0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.513+0000 7f2fa9c65700 1 -- 172.21.15.40:0/4168584401 >> 172.21.15.40:0/4168584401 conn(0x7f2fa40fb120 msgr2=0x7f2fa40fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:37.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.513+0000 7f2fa9c65700 1 -- 172.21.15.40:0/4168584401 shutdown_connections 2023-07-18T03:30:37.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.513+0000 7f2fa9c65700 1 -- 172.21.15.40:0/4168584401 wait complete. 2023-07-18T03:30:37.522 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:30:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.655+0000 7f67b7fff700 1 -- 172.21.15.40:0/3908092572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b80ffae0 msgr2=0x7f67b80fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.655+0000 7f67b7fff700 1 --2- 172.21.15.40:0/3908092572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b80ffae0 0x7f67b80fff30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009990 tx=0x7f67ac009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.655+0000 7f67b7fff700 1 -- 172.21.15.40:0/3908092572 shutdown_connections 2023-07-18T03:30:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.655+0000 7f67b7fff700 1 --2- 172.21.15.40:0/3908092572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67b8100470 0x7f67b8106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.655+0000 7f67b7fff700 1 --2- 172.21.15.40:0/3908092572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67b80fe8f0 0x7f67b80fed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.655+0000 7f67b7fff700 1 --2- 172.21.15.40:0/3908092572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b80ffae0 0x7f67b80fff30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.655+0000 7f67b7fff700 1 -- 172.21.15.40:0/3908092572 >> 172.21.15.40:0/3908092572 conn(0x7f67b80f9ef0 msgr2=0x7f67b80fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:37.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.655+0000 7f67b7fff700 1 -- 172.21.15.40:0/3908092572 shutdown_connections 2023-07-18T03:30:37.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.655+0000 7f67b7fff700 1 -- 172.21.15.40:0/3908092572 wait complete. 2023-07-18T03:30:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.656+0000 7f67b7fff700 1 Processor -- start 2023-07-18T03:30:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.656+0000 7f67b7fff700 1 -- start start 2023-07-18T03:30:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.656+0000 7f67b7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67b80fe8f0 0x7f67b8068e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.656+0000 7f67b7fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67b80ffae0 0x7f67b819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.656+0000 7f67b7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b8100470 0x7f67b819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.656+0000 7f67b7fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b819d120 con 0x7f67b8100470 2023-07-18T03:30:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.656+0000 7f67b7fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b819d290 con 0x7f67b80fe8f0 2023-07-18T03:30:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.656+0000 7f67b7fff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b819d450 con 0x7f67b80ffae0 2023-07-18T03:30:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.657+0000 7f67b6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67b80fe8f0 0x7f67b8068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.657+0000 7f67b77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b8100470 0x7f67b819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.657+0000 7f67b67fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67b80ffae0 0x7f67b819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.657+0000 7f67b6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67b80fe8f0 0x7f67b8068e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:40660/0 (socket says 172.21.15.40:40660) 2023-07-18T03:30:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.657+0000 7f67b6ffd700 1 -- 172.21.15.40:0/25028505 learned_addr learned my addr 172.21.15.40:0/25028505 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.657+0000 7f67b77fe700 1 -- 172.21.15.40:0/25028505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67b80ffae0 msgr2=0x7f67b819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.657+0000 7f67b77fe700 1 --2- 172.21.15.40:0/25028505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67b80ffae0 0x7f67b819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.657+0000 7f67b77fe700 1 -- 172.21.15.40:0/25028505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67b80fe8f0 msgr2=0x7f67b8068e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.657+0000 7f67b77fe700 1 --2- 172.21.15.40:0/25028505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67b80fe8f0 0x7f67b8068e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.657+0000 7f67b77fe700 1 -- 172.21.15.40:0/25028505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac0095f0 con 0x7f67b8100470 2023-07-18T03:30:37.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.657+0000 7f67b6ffd700 1 --2- 172.21.15.40:0/25028505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67b80fe8f0 0x7f67b8068e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:37.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.657+0000 7f67b77fe700 1 --2- 172.21.15.40:0/25028505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b8100470 0x7f67b819c950 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f67b000eb80 tx=0x7f67b000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:37.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.657+0000 7f679ffff700 1 -- 172.21.15.40:0/25028505 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0018070 con 0x7f67b8100470 2023-07-18T03:30:37.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.657+0000 7f679ffff700 1 -- 172.21.15.40:0/25028505 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67b000f040 con 0x7f67b8100470 2023-07-18T03:30:37.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.657+0000 7f67b7fff700 1 -- 172.21.15.40:0/25028505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b819d6d0 con 0x7f67b8100470 2023-07-18T03:30:37.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.657+0000 7f679ffff700 1 -- 172.21.15.40:0/25028505 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b000cd50 con 0x7f67b8100470 2023-07-18T03:30:37.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.657+0000 7f67b7fff700 1 -- 172.21.15.40:0/25028505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b819db50 con 0x7f67b8100470 2023-07-18T03:30:37.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.658+0000 7f67b7fff700 1 -- 172.21.15.40:0/25028505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b8105d90 con 0x7f67b8100470 2023-07-18T03:30:37.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.659+0000 7f679ffff700 1 -- 172.21.15.40:0/25028505 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f67b0003a50 con 0x7f67b8100470 2023-07-18T03:30:37.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.660+0000 7f679ffff700 1 --2- 172.21.15.40:0/25028505 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67a00a1290 0x7f67a00a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:37.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.660+0000 7f67b6ffd700 1 --2- 172.21.15.40:0/25028505 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67a00a1290 0x7f67a00a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:37.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.660+0000 7f679ffff700 1 -- 172.21.15.40:0/25028505 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f67b00ccfa0 con 0x7f67b8100470 2023-07-18T03:30:37.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.661+0000 7f67b6ffd700 1 --2- 172.21.15.40:0/25028505 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67a00a1290 0x7f67a00a3740 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f67a8005950 tx=0x7f67a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:37.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.663+0000 7f679ffff700 1 -- 172.21.15.40:0/25028505 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f67b0096600 con 0x7f67b8100470 2023-07-18T03:30:37.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:37 smithi040 ceph-mon[96439]: pgmap v285: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2023-07-18T03:30:37.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:37 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2196187745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:37.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:37 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1111751818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:37.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:37 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/4168584401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:37.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.866+0000 7f67b7fff700 1 -- 172.21.15.40:0/25028505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67b802c9e0 con 0x7f67b8100470 2023-07-18T03:30:37.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.866+0000 7f679ffff700 1 -- 172.21.15.40:0/25028505 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f67b0099c20 con 0x7f67b8100470 2023-07-18T03:30:37.863 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-07-18T03:30:37.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.868+0000 7f67b7fff700 1 -- 172.21.15.40:0/25028505 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67a00a1290 msgr2=0x7f67a00a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:37.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.868+0000 7f67b7fff700 1 --2- 172.21.15.40:0/25028505 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67a00a1290 0x7f67a00a3740 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f67a8005950 tx=0x7f67a8009260 comp rx=0 tx=0).stop 2023-07-18T03:30:37.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.868+0000 7f67b7fff700 1 -- 172.21.15.40:0/25028505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b8100470 msgr2=0x7f67b819c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:37.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.868+0000 7f67b7fff700 1 --2- 172.21.15.40:0/25028505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b8100470 0x7f67b819c950 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f67b000eb80 tx=0x7f67b000ee90 comp rx=0 tx=0).stop 2023-07-18T03:30:37.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.868+0000 7f67b7fff700 1 -- 172.21.15.40:0/25028505 shutdown_connections 2023-07-18T03:30:37.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.868+0000 7f67b7fff700 1 --2- 172.21.15.40:0/25028505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b8100470 0x7f67b819c950 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.868+0000 7f67b7fff700 1 --2- 172.21.15.40:0/25028505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67b80fe8f0 0x7f67b8068e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.868+0000 7f67b7fff700 1 --2- 172.21.15.40:0/25028505 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f67a00a1290 0x7f67a00a3740 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.868+0000 7f67b7fff700 1 --2- 172.21.15.40:0/25028505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67b80ffae0 0x7f67b819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:37.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.868+0000 7f67b7fff700 1 -- 172.21.15.40:0/25028505 >> 172.21.15.40:0/25028505 conn(0x7f67b80f9ef0 msgr2=0x7f67b8102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:37.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.868+0000 7f67b7fff700 1 -- 172.21.15.40:0/25028505 shutdown_connections 2023-07-18T03:30:37.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:37.868+0000 7f67b7fff700 1 -- 172.21.15.40:0/25028505 wait complete. 2023-07-18T03:30:37.877 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:30:38.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.009+0000 7ff57031c700 1 -- 172.21.15.40:0/2340844943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5680fe9e0 msgr2=0x7ff5680fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:38.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.009+0000 7ff57031c700 1 --2- 172.21.15.40:0/2340844943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5680fe9e0 0x7ff5680fedf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff564009990 tx=0x7ff564009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.009+0000 7ff57031c700 1 -- 172.21.15.40:0/2340844943 shutdown_connections 2023-07-18T03:30:38.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.009+0000 7ff57031c700 1 --2- 172.21.15.40:0/2340844943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff568100560 0x7ff568106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.009+0000 7ff57031c700 1 --2- 172.21.15.40:0/2340844943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5680fe9e0 0x7ff5680fedf0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.009+0000 7ff57031c700 1 --2- 172.21.15.40:0/2340844943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5680ffbd0 0x7ff568100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.009+0000 7ff57031c700 1 -- 172.21.15.40:0/2340844943 >> 172.21.15.40:0/2340844943 conn(0x7ff5680fa000 msgr2=0x7ff5680fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:38.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.009+0000 7ff57031c700 1 -- 172.21.15.40:0/2340844943 shutdown_connections 2023-07-18T03:30:38.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.009+0000 7ff57031c700 1 -- 172.21.15.40:0/2340844943 wait complete. 2023-07-18T03:30:38.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.010+0000 7ff57031c700 1 Processor -- start 2023-07-18T03:30:38.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.010+0000 7ff57031c700 1 -- start start 2023-07-18T03:30:38.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.010+0000 7ff57031c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5680fe9e0 0x7ff5680681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:38.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.010+0000 7ff57031c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5680ffbd0 0x7ff56819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:38.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.010+0000 7ff57031c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff568100560 0x7ff56819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:38.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.010+0000 7ff57031c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56819d310 con 0x7ff5680fe9e0 2023-07-18T03:30:38.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.010+0000 7ff57031c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56819d480 con 0x7ff5680ffbd0 2023-07-18T03:30:38.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.010+0000 7ff57031c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56819d640 con 0x7ff568100560 2023-07-18T03:30:38.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.010+0000 7ff56e0b8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5680fe9e0 0x7ff5680681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:38.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.010+0000 7ff56e0b8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5680fe9e0 0x7ff5680681e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57348/0 (socket says 172.21.15.40:57348) 2023-07-18T03:30:38.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.010+0000 7ff56e0b8700 1 -- 172.21.15.40:0/112679855 learned_addr learned my addr 172.21.15.40:0/112679855 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:38.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.010+0000 7ff56e0b8700 1 -- 172.21.15.40:0/112679855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff568100560 msgr2=0x7ff56819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:38.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.010+0000 7ff56e0b8700 1 --2- 172.21.15.40:0/112679855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff568100560 0x7ff56819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.010+0000 7ff56e0b8700 1 -- 172.21.15.40:0/112679855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5680ffbd0 msgr2=0x7ff56819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:38.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.010+0000 7ff56d8b7700 1 --2- 172.21.15.40:0/112679855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5680ffbd0 0x7ff56819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:38.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.010+0000 7ff56e0b8700 1 --2- 172.21.15.40:0/112679855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5680ffbd0 0x7ff56819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.010+0000 7ff56e0b8700 1 -- 172.21.15.40:0/112679855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5640095f0 con 0x7ff5680fe9e0 2023-07-18T03:30:38.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.011+0000 7ff56e0b8700 1 --2- 172.21.15.40:0/112679855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5680fe9e0 0x7ff5680681e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff5640059d0 tx=0x7ff564005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:38.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.011+0000 7ff55f7fe700 1 -- 172.21.15.40:0/112679855 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56401d070 con 0x7ff5680fe9e0 2023-07-18T03:30:38.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.011+0000 7ff55f7fe700 1 -- 172.21.15.40:0/112679855 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff564004120 con 0x7ff5680fe9e0 2023-07-18T03:30:38.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.011+0000 7ff57031c700 1 -- 172.21.15.40:0/112679855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56819d8c0 con 0x7ff5680fe9e0 2023-07-18T03:30:38.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.011+0000 7ff55f7fe700 1 -- 172.21.15.40:0/112679855 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff564022de0 con 0x7ff5680fe9e0 2023-07-18T03:30:38.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.011+0000 7ff57031c700 1 -- 172.21.15.40:0/112679855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56819dd40 con 0x7ff5680fe9e0 2023-07-18T03:30:38.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.012+0000 7ff57031c700 1 -- 172.21.15.40:0/112679855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff568105910 con 0x7ff5680fe9e0 2023-07-18T03:30:38.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.013+0000 7ff55f7fe700 1 -- 172.21.15.40:0/112679855 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff564022470 con 0x7ff5680fe9e0 2023-07-18T03:30:38.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.013+0000 7ff55f7fe700 1 --2- 172.21.15.40:0/112679855 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff5540a1260 0x7ff5540a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:38.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.014+0000 7ff56d8b7700 1 --2- 172.21.15.40:0/112679855 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff5540a1260 0x7ff5540a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:38.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.014+0000 7ff55f7fe700 1 -- 172.21.15.40:0/112679855 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff5640cdec0 con 0x7ff5680fe9e0 2023-07-18T03:30:38.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.014+0000 7ff56d8b7700 1 --2- 172.21.15.40:0/112679855 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff5540a1260 0x7ff5540a3710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff558005fd0 tx=0x7ff558005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:38.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.017+0000 7ff55f7fe700 1 -- 172.21.15.40:0/112679855 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff5640974c0 con 0x7ff5680fe9e0 2023-07-18T03:30:38.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:37 smithi134 ceph-mon[98216]: pgmap v285: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2023-07-18T03:30:38.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:37 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2196187745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:38.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:37 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1111751818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:38.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:37 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/4168584401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:38.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:37 smithi064 ceph-mon[98059]: pgmap v285: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2023-07-18T03:30:38.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:37 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2196187745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:38.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:37 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1111751818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:38.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:37 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/4168584401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:38.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.216+0000 7ff57031c700 1 -- 172.21.15.40:0/112679855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff56802cab0 con 0x7ff5680fe9e0 2023-07-18T03:30:38.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.216+0000 7ff55f7fe700 1 -- 172.21.15.40:0/112679855 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff564027020 con 0x7ff5680fe9e0 2023-07-18T03:30:38.213 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-07-18T03:30:38.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.218+0000 7ff57031c700 1 -- 172.21.15.40:0/112679855 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff5540a1260 msgr2=0x7ff5540a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:38.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.218+0000 7ff57031c700 1 --2- 172.21.15.40:0/112679855 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff5540a1260 0x7ff5540a3710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff558005fd0 tx=0x7ff558005f60 comp rx=0 tx=0).stop 2023-07-18T03:30:38.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.218+0000 7ff57031c700 1 -- 172.21.15.40:0/112679855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5680fe9e0 msgr2=0x7ff5680681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:38.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.218+0000 7ff57031c700 1 --2- 172.21.15.40:0/112679855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5680fe9e0 0x7ff5680681e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff5640059d0 tx=0x7ff564005ef0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.218+0000 7ff57031c700 1 -- 172.21.15.40:0/112679855 shutdown_connections 2023-07-18T03:30:38.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.218+0000 7ff57031c700 1 --2- 172.21.15.40:0/112679855 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff5540a1260 0x7ff5540a3710 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.218+0000 7ff57031c700 1 --2- 172.21.15.40:0/112679855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff568100560 0x7ff56819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.218+0000 7ff57031c700 1 --2- 172.21.15.40:0/112679855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5680fe9e0 0x7ff5680681e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.218+0000 7ff57031c700 1 --2- 172.21.15.40:0/112679855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5680ffbd0 0x7ff56819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.219+0000 7ff57031c700 1 -- 172.21.15.40:0/112679855 >> 172.21.15.40:0/112679855 conn(0x7ff5680fa000 msgr2=0x7ff5680fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:38.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.219+0000 7ff57031c700 1 -- 172.21.15.40:0/112679855 shutdown_connections 2023-07-18T03:30:38.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.219+0000 7ff57031c700 1 -- 172.21.15.40:0/112679855 wait complete. 2023-07-18T03:30:38.227 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:30:38.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.358+0000 7f12a4a83700 1 -- 172.21.15.40:0/329499681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a00ffbc0 msgr2=0x7f12a0100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:38.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.358+0000 7f12a4a83700 1 --2- 172.21.15.40:0/329499681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a00ffbc0 0x7f12a0100010 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1294009990 tx=0x7f1294009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.358+0000 7f12a4a83700 1 -- 172.21.15.40:0/329499681 shutdown_connections 2023-07-18T03:30:38.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.358+0000 7f12a4a83700 1 --2- 172.21.15.40:0/329499681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12a0100550 0x7f12a0106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.358+0000 7f12a4a83700 1 --2- 172.21.15.40:0/329499681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12a00fe9d0 0x7f12a00fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.358+0000 7f12a4a83700 1 --2- 172.21.15.40:0/329499681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a00ffbc0 0x7f12a0100010 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.358+0000 7f12a4a83700 1 -- 172.21.15.40:0/329499681 >> 172.21.15.40:0/329499681 conn(0x7f12a00f9fd0 msgr2=0x7f12a00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:38.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.358+0000 7f12a4a83700 1 -- 172.21.15.40:0/329499681 shutdown_connections 2023-07-18T03:30:38.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.358+0000 7f12a4a83700 1 -- 172.21.15.40:0/329499681 wait complete. 2023-07-18T03:30:38.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.358+0000 7f12a4a83700 1 Processor -- start 2023-07-18T03:30:38.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f12a4a83700 1 -- start start 2023-07-18T03:30:38.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f12a4a83700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12a00fe9d0 0x7f12a0068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:38.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f12a4a83700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12a00ffbc0 0x7f12a019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:38.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f12a4a83700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a0100550 0x7f12a019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:38.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f12a4a83700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a019d340 con 0x7f12a0100550 2023-07-18T03:30:38.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f12a4a83700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a019d4b0 con 0x7f12a00ffbc0 2023-07-18T03:30:38.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f12a4a83700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a019d670 con 0x7f12a00fe9d0 2023-07-18T03:30:38.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f129ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a0100550 0x7f12a019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:38.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f129ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a0100550 0x7f12a019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57364/0 (socket says 172.21.15.40:57364) 2023-07-18T03:30:38.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f129ed9d700 1 -- 172.21.15.40:0/3870402229 learned_addr learned my addr 172.21.15.40:0/3870402229 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:38.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f129dd9b700 1 --2- 172.21.15.40:0/3870402229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12a00ffbc0 0x7f12a019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:38.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f129ed9d700 1 -- 172.21.15.40:0/3870402229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12a00fe9d0 msgr2=0x7f12a0068200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:38.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f129ed9d700 1 --2- 172.21.15.40:0/3870402229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12a00fe9d0 0x7f12a0068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f129ed9d700 1 -- 172.21.15.40:0/3870402229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12a00ffbc0 msgr2=0x7f12a019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:38.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f129ed9d700 1 --2- 172.21.15.40:0/3870402229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12a00ffbc0 0x7f12a019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f129ed9d700 1 -- 172.21.15.40:0/3870402229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12940095f0 con 0x7f12a0100550 2023-07-18T03:30:38.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f129ed9d700 1 --2- 172.21.15.40:0/3870402229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a0100550 0x7f12a019cb70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1298007ee0 tx=0x7f129800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:38.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f128f7fe700 1 -- 172.21.15.40:0/3870402229 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298017070 con 0x7f12a0100550 2023-07-18T03:30:38.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f128f7fe700 1 -- 172.21.15.40:0/3870402229 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f129800f040 con 0x7f12a0100550 2023-07-18T03:30:38.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f12a4a83700 1 -- 172.21.15.40:0/3870402229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a019d950 con 0x7f12a0100550 2023-07-18T03:30:38.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.359+0000 7f128f7fe700 1 -- 172.21.15.40:0/3870402229 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298005740 con 0x7f12a0100550 2023-07-18T03:30:38.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.360+0000 7f12a4a83700 1 -- 172.21.15.40:0/3870402229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a019de60 con 0x7f12a0100550 2023-07-18T03:30:38.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.361+0000 7f12a4a83700 1 -- 172.21.15.40:0/3870402229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a0105f20 con 0x7f12a0100550 2023-07-18T03:30:38.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.361+0000 7f128f7fe700 1 -- 172.21.15.40:0/3870402229 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f129800dc60 con 0x7f12a0100550 2023-07-18T03:30:38.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.362+0000 7f128f7fe700 1 --2- 172.21.15.40:0/3870402229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f12880a12a0 0x7f12880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:38.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.362+0000 7f128f7fe700 1 -- 172.21.15.40:0/3870402229 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f12980cc260 con 0x7f12a0100550 2023-07-18T03:30:38.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.362+0000 7f129e59c700 1 --2- 172.21.15.40:0/3870402229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f12880a12a0 0x7f12880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:38.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.363+0000 7f129e59c700 1 --2- 172.21.15.40:0/3870402229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f12880a12a0 0x7f12880a3750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f12a00ffa20 tx=0x7f1290009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:38.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.366+0000 7f128f7fe700 1 -- 172.21.15.40:0/3870402229 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1298095830 con 0x7f12a0100550 2023-07-18T03:30:38.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.611+0000 7f12a4a83700 1 -- 172.21.15.40:0/3870402229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12a002ca00 con 0x7f12a0100550 2023-07-18T03:30:38.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.611+0000 7f128f7fe700 1 -- 172.21.15.40:0/3870402229 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10888 (secure 0 0 0) 0x7f1298098e50 con 0x7f12a0100550 2023-07-18T03:30:38.608 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:38.608 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:29:26.718789+0000","modified":"2023-07-18T03:29:34.650353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24523,"mds_2":24482,"mds_3":24452,"mds_4":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14601":{"gid":14601,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1818167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1818167554},{"type":"v1","addr":"172.21.15.40:6839","nonce":1818167554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14613":{"gid":14613,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/3953074960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":3953074960},{"type":"v1","addr":"172.21.15.40:6841","nonce":3953074960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3509655945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3509655945},{"type":"v1","addr":"172.21.15.134:6837","nonce":3509655945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/3007039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3007039043},{"type":"v1","addr":"172.21.15.64:6839","nonce":3007039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:30:38.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.613+0000 7f12a4a83700 1 -- 172.21.15.40:0/3870402229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f12880a12a0 msgr2=0x7f12880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:38.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.613+0000 7f12a4a83700 1 --2- 172.21.15.40:0/3870402229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f12880a12a0 0x7f12880a3750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f12a00ffa20 tx=0x7f1290009470 comp rx=0 tx=0).stop 2023-07-18T03:30:38.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.613+0000 7f12a4a83700 1 -- 172.21.15.40:0/3870402229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a0100550 msgr2=0x7f12a019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:38.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.613+0000 7f12a4a83700 1 --2- 172.21.15.40:0/3870402229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a0100550 0x7f12a019cb70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1298007ee0 tx=0x7f129800d3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.613+0000 7f12a4a83700 1 -- 172.21.15.40:0/3870402229 shutdown_connections 2023-07-18T03:30:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.613+0000 7f12a4a83700 1 --2- 172.21.15.40:0/3870402229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12a0100550 0x7f12a019cb70 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.613+0000 7f12a4a83700 1 --2- 172.21.15.40:0/3870402229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f12880a12a0 0x7f12880a3750 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.613+0000 7f12a4a83700 1 --2- 172.21.15.40:0/3870402229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12a00fe9d0 0x7f12a0068200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.613+0000 7f12a4a83700 1 --2- 172.21.15.40:0/3870402229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12a00ffbc0 0x7f12a019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.613+0000 7f12a4a83700 1 -- 172.21.15.40:0/3870402229 >> 172.21.15.40:0/3870402229 conn(0x7f12a00f9fd0 msgr2=0x7f12a0103060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.614+0000 7f12a4a83700 1 -- 172.21.15.40:0/3870402229 shutdown_connections 2023-07-18T03:30:38.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.614+0000 7f12a4a83700 1 -- 172.21.15.40:0/3870402229 wait complete. 2023-07-18T03:30:38.612 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 27 2023-07-18T03:30:38.623 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:30:38.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.754+0000 7f1ec15a6700 1 -- 172.21.15.40:0/2648835264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ebc0ffbc0 msgr2=0x7f1ebc100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:38.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.754+0000 7f1ec15a6700 1 --2- 172.21.15.40:0/2648835264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ebc0ffbc0 0x7f1ebc100010 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009990 tx=0x7f1eb0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.754+0000 7f1ec15a6700 1 -- 172.21.15.40:0/2648835264 shutdown_connections 2023-07-18T03:30:38.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.754+0000 7f1ec15a6700 1 --2- 172.21.15.40:0/2648835264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ebc100550 0x7f1ebc106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.754+0000 7f1ec15a6700 1 --2- 172.21.15.40:0/2648835264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc0fe9d0 0x7f1ebc0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.754+0000 7f1ec15a6700 1 --2- 172.21.15.40:0/2648835264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ebc0ffbc0 0x7f1ebc100010 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.754+0000 7f1ec15a6700 1 -- 172.21.15.40:0/2648835264 >> 172.21.15.40:0/2648835264 conn(0x7f1ebc0f9fd0 msgr2=0x7f1ebc0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:38.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.754+0000 7f1ec15a6700 1 -- 172.21.15.40:0/2648835264 shutdown_connections 2023-07-18T03:30:38.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.754+0000 7f1ec15a6700 1 -- 172.21.15.40:0/2648835264 wait complete. 2023-07-18T03:30:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.754+0000 7f1ec15a6700 1 Processor -- start 2023-07-18T03:30:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1ec15a6700 1 -- start start 2023-07-18T03:30:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1ec15a6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc0fe9d0 0x7f1ebc195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1ec15a6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ebc0ffbc0 0x7f1ebc195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1ec15a6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ebc100550 0x7f1ebc19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1ec15a6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc19d3a0 con 0x7f1ebc0ffbc0 2023-07-18T03:30:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1ec15a6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc19d4e0 con 0x7f1ebc0fe9d0 2023-07-18T03:30:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1ec15a6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc19d670 con 0x7f1ebc100550 2023-07-18T03:30:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1eba7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ebc0ffbc0 0x7f1ebc195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:38.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1eba7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ebc0ffbc0 0x7f1ebc195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57376/0 (socket says 172.21.15.40:57376) 2023-07-18T03:30:38.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1eba7fc700 1 -- 172.21.15.40:0/3832458061 learned_addr learned my addr 172.21.15.40:0/3832458061 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:38.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1ebaffd700 1 --2- 172.21.15.40:0/3832458061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc0fe9d0 0x7f1ebc195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:38.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1eba7fc700 1 -- 172.21.15.40:0/3832458061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ebc100550 msgr2=0x7f1ebc19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:38.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1ebb7fe700 1 --2- 172.21.15.40:0/3832458061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ebc100550 0x7f1ebc19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:38.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1eba7fc700 1 --2- 172.21.15.40:0/3832458061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ebc100550 0x7f1ebc19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1eba7fc700 1 -- 172.21.15.40:0/3832458061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc0fe9d0 msgr2=0x7f1ebc195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:38.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1eba7fc700 1 --2- 172.21.15.40:0/3832458061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc0fe9d0 0x7f1ebc195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1eba7fc700 1 -- 172.21.15.40:0/3832458061 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb00095f0 con 0x7f1ebc0ffbc0 2023-07-18T03:30:38.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.755+0000 7f1ebaffd700 1 --2- 172.21.15.40:0/3832458061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc0fe9d0 0x7f1ebc195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:38.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.756+0000 7f1eba7fc700 1 --2- 172.21.15.40:0/3832458061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ebc0ffbc0 0x7f1ebc195d60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c010 tx=0x7f1eb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:38.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.756+0000 7f1ea3fff700 1 -- 172.21.15.40:0/3832458061 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb001d070 con 0x7f1ebc0ffbc0 2023-07-18T03:30:38.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.756+0000 7f1ec15a6700 1 -- 172.21.15.40:0/3832458061 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc19d8f0 con 0x7f1ebc0ffbc0 2023-07-18T03:30:38.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.756+0000 7f1ea3fff700 1 -- 172.21.15.40:0/3832458061 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1eb00054f0 con 0x7f1ebc0ffbc0 2023-07-18T03:30:38.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.756+0000 7f1ea3fff700 1 -- 172.21.15.40:0/3832458061 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0004ca0 con 0x7f1ebc0ffbc0 2023-07-18T03:30:38.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.756+0000 7f1ec15a6700 1 -- 172.21.15.40:0/3832458061 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc19dd10 con 0x7f1ebc0ffbc0 2023-07-18T03:30:38.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.757+0000 7f1ec15a6700 1 -- 172.21.15.40:0/3832458061 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc10a1f0 con 0x7f1ebc0ffbc0 2023-07-18T03:30:38.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.758+0000 7f1ea3fff700 1 -- 172.21.15.40:0/3832458061 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1eb000fbc0 con 0x7f1ebc0ffbc0 2023-07-18T03:30:38.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.758+0000 7f1ea3fff700 1 --2- 172.21.15.40:0/3832458061 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1ea40a1220 0x7f1ea40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:38.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.758+0000 7f1ebaffd700 1 --2- 172.21.15.40:0/3832458061 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1ea40a1220 0x7f1ea40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:38.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.758+0000 7f1ea3fff700 1 -- 172.21.15.40:0/3832458061 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1eb00ce240 con 0x7f1ebc0ffbc0 2023-07-18T03:30:38.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.759+0000 7f1ebaffd700 1 --2- 172.21.15.40:0/3832458061 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1ea40a1220 0x7f1ea40a36d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0ffa20 tx=0x7f1eac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:38.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.762+0000 7f1ea3fff700 1 -- 172.21.15.40:0/3832458061 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1eb00978a0 con 0x7f1ebc0ffbc0 2023-07-18T03:30:38.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.961+0000 7f1ec15a6700 1 -- 172.21.15.40:0/3832458061 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ebc19e3b0 con 0x7f1ebc0ffbc0 2023-07-18T03:30:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.963+0000 7f1ea3fff700 1 -- 172.21.15.40:0/3832458061 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22599 (secure 0 0 0) 0x7f1eb0097020 con 0x7f1ebc0ffbc0 2023-07-18T03:30:38.960 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:38.960 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":83,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:30:33.646627+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:30:38.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.965+0000 7f1ec15a6700 1 -- 172.21.15.40:0/3832458061 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1ea40a1220 msgr2=0x7f1ea40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:38.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.965+0000 7f1ec15a6700 1 --2- 172.21.15.40:0/3832458061 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1ea40a1220 0x7f1ea40a36d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0ffa20 tx=0x7f1eac009260 comp rx=0 tx=0).stop 2023-07-18T03:30:38.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.965+0000 7f1ec15a6700 1 -- 172.21.15.40:0/3832458061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ebc0ffbc0 msgr2=0x7f1ebc195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:38.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.965+0000 7f1ec15a6700 1 --2- 172.21.15.40:0/3832458061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ebc0ffbc0 0x7f1ebc195d60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c010 tx=0x7f1eb0004730 comp rx=0 tx=0).stop 2023-07-18T03:30:38.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.965+0000 7f1ec15a6700 1 -- 172.21.15.40:0/3832458061 shutdown_connections 2023-07-18T03:30:38.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.965+0000 7f1ec15a6700 1 --2- 172.21.15.40:0/3832458061 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1ea40a1220 0x7f1ea40a36d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.965+0000 7f1ec15a6700 1 --2- 172.21.15.40:0/3832458061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1ebc100550 0x7f1ebc19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.965+0000 7f1ec15a6700 1 --2- 172.21.15.40:0/3832458061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ebc0fe9d0 0x7f1ebc195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.965+0000 7f1ec15a6700 1 --2- 172.21.15.40:0/3832458061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ebc0ffbc0 0x7f1ebc195d60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:38.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.965+0000 7f1ec15a6700 1 -- 172.21.15.40:0/3832458061 >> 172.21.15.40:0/3832458061 conn(0x7f1ebc0f9fd0 msgr2=0x7f1ebc067670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:38.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.965+0000 7f1ec15a6700 1 -- 172.21.15.40:0/3832458061 shutdown_connections 2023-07-18T03:30:38.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:38.965+0000 7f1ec15a6700 1 -- 172.21.15.40:0/3832458061 wait complete. 2023-07-18T03:30:38.973 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:38 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/25028505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:38.973 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:38 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/112679855' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:38.973 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:38 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3870402229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:38.974 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-18T03:29:26.718789+0000', 'modified': '2023-07-18T03:29:34.650353+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24527, 'mds_1': 24523, 'mds_2': 24482, 'mds_3': 24452, 'mds_4': 24442}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14538': {'gid': 14538, 'name': 'd', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6837/3377503144', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6836', 'nonce': 3377503144}, {'type': 'v1', 'addr': '172.21.15.40:6837', 'nonce': 3377503144}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14601': {'gid': 14601, 'name': 'g', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6839/1818167554', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6838', 'nonce': 1818167554}, {'type': 'v1', 'addr': '172.21.15.40:6839', 'nonce': 1818167554}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14613': {'gid': 14613, 'name': 'a', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.40:6841/3953074960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6840', 'nonce': 3953074960}, {'type': 'v1', 'addr': '172.21.15.40:6841', 'nonce': 3953074960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24442': {'gid': 24442, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.134:6833/2345264306', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 2345264306}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 2345264306}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.64:6835/1864495009', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 1864495009}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 1864495009}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.64:6837/3077792296', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6836', 'nonce': 3077792296}, {'type': 'v1', 'addr': '172.21.15.64:6837', 'nonce': 3077792296}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6835/2946854745', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 2946854745}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 2946854745}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6837/3509655945', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 3509655945}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 3509655945}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24523': {'gid': 24523, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.134:6839/3192254101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 3192254101}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 3192254101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24527': {'gid': 24527, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6839/3007039043', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6838', 'nonce': 3007039043}, {'type': 'v1', 'addr': '172.21.15.64:6839', 'nonce': 3007039043}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-18T03:30:38.975 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-07-18T03:30:38.975 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2023-07-18T03:30:39.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:38 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/25028505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:39.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:38 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/112679855' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:39.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:38 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3870402229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:39.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:38 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/25028505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:39.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:38 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/112679855' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:39.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:38 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3870402229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:39.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.107+0000 7fb54687a700 1 -- 172.21.15.40:0/3539721354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb540100560 msgr2=0x7fb540106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.107+0000 7fb54687a700 1 --2- 172.21.15.40:0/3539721354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb540100560 0x7fb540106e20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb53800b0a0 tx=0x7fb53800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.108+0000 7fb54687a700 1 -- 172.21.15.40:0/3539721354 shutdown_connections 2023-07-18T03:30:39.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.108+0000 7fb54687a700 1 --2- 172.21.15.40:0/3539721354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb540100560 0x7fb540106e20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.108+0000 7fb54687a700 1 --2- 172.21.15.40:0/3539721354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5400fe9e0 0x7fb5400fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.108+0000 7fb54687a700 1 --2- 172.21.15.40:0/3539721354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5400ffbd0 0x7fb540100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.108+0000 7fb54687a700 1 -- 172.21.15.40:0/3539721354 >> 172.21.15.40:0/3539721354 conn(0x7fb5400fa000 msgr2=0x7fb5400fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.108+0000 7fb54687a700 1 -- 172.21.15.40:0/3539721354 shutdown_connections 2023-07-18T03:30:39.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.108+0000 7fb54687a700 1 -- 172.21.15.40:0/3539721354 wait complete. 2023-07-18T03:30:39.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.108+0000 7fb54687a700 1 Processor -- start 2023-07-18T03:30:39.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.108+0000 7fb54687a700 1 -- start start 2023-07-18T03:30:39.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb54687a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5400fe9e0 0x7fb540197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb54687a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5400ffbd0 0x7fb540198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb54687a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb540100560 0x7fb54019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb54687a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54019f690 con 0x7fb5400fe9e0 2023-07-18T03:30:39.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb54687a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54019f7d0 con 0x7fb5400ffbd0 2023-07-18T03:30:39.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb54687a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54019f960 con 0x7fb540100560 2023-07-18T03:30:39.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb53ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5400fe9e0 0x7fb540197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb53ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5400fe9e0 0x7fb540197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57388/0 (socket says 172.21.15.40:57388) 2023-07-18T03:30:39.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb53ffff700 1 -- 172.21.15.40:0/1842864655 learned_addr learned my addr 172.21.15.40:0/1842864655 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb544e17700 1 --2- 172.21.15.40:0/1842864655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb540100560 0x7fb54019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb53f7fe700 1 --2- 172.21.15.40:0/1842864655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5400ffbd0 0x7fb540198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb53ffff700 1 -- 172.21.15.40:0/1842864655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb540100560 msgr2=0x7fb54019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb53ffff700 1 --2- 172.21.15.40:0/1842864655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb540100560 0x7fb54019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb53ffff700 1 -- 172.21.15.40:0/1842864655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5400ffbd0 msgr2=0x7fb540198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb53ffff700 1 --2- 172.21.15.40:0/1842864655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5400ffbd0 0x7fb540198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb53ffff700 1 -- 172.21.15.40:0/1842864655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb538009d00 con 0x7fb5400fe9e0 2023-07-18T03:30:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb53ffff700 1 --2- 172.21.15.40:0/1842864655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5400fe9e0 0x7fb540197b10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb53000e970 tx=0x7fb53000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb53d7fa700 1 -- 172.21.15.40:0/1842864655 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530009790 con 0x7fb5400fe9e0 2023-07-18T03:30:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb54687a700 1 -- 172.21.15.40:0/1842864655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb54019fc40 con 0x7fb5400fe9e0 2023-07-18T03:30:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb53d7fa700 1 -- 172.21.15.40:0/1842864655 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb53000eec0 con 0x7fb5400fe9e0 2023-07-18T03:30:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb53d7fa700 1 -- 172.21.15.40:0/1842864655 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530003cf0 con 0x7fb5400fe9e0 2023-07-18T03:30:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.109+0000 7fb54687a700 1 -- 172.21.15.40:0/1842864655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5401a0150 con 0x7fb5400fe9e0 2023-07-18T03:30:39.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.112+0000 7fb53d7fa700 1 -- 172.21.15.40:0/1842864655 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb53000ccd0 con 0x7fb5400fe9e0 2023-07-18T03:30:39.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.113+0000 7fb53d7fa700 1 --2- 172.21.15.40:0/1842864655 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb52c0a11e0 0x7fb52c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.113+0000 7fb53f7fe700 1 --2- 172.21.15.40:0/1842864655 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb52c0a11e0 0x7fb52c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.113+0000 7fb53d7fa700 1 -- 172.21.15.40:0/1842864655 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb530014070 con 0x7fb5400fe9e0 2023-07-18T03:30:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.113+0000 7fb53f7fe700 1 --2- 172.21.15.40:0/1842864655 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb52c0a11e0 0x7fb52c0a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb52800b3c0 tx=0x7fb52800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.113+0000 7fb552a00b80 1 Processor -- start 2023-07-18T03:30:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.113+0000 7fb552a00b80 1 -- start start 2023-07-18T03:30:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb552a00b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dc6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb552a00b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dc7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb552a00b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dc8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb552a00b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f3ccbc60 con 0x55f0f3dc6480 2023-07-18T03:30:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb552a00b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f3d1e9f0 con 0x55f0f3dc6dd0 2023-07-18T03:30:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb552a00b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f3d17e80 con 0x55f0f3dc7c20 2023-07-18T03:30:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb53ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dc6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb53ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dc6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57396/0 (socket says 172.21.15.40:57396) 2023-07-18T03:30:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb53ffff700 1 -- 172.21.15.40:0/1836439571 learned_addr learned my addr 172.21.15.40:0/1836439571 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb53f7fe700 1 --2- 172.21.15.40:0/1836439571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dc7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb544e17700 1 --2- 172.21.15.40:0/1836439571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dc8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb53ffff700 1 -- 172.21.15.40:0/1836439571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc7c20 msgr2=0x55f0f3dc8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb53ffff700 1 --2- 172.21.15.40:0/1836439571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dc8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb53ffff700 1 -- 172.21.15.40:0/1836439571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc6dd0 msgr2=0x55f0f3dc7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb53ffff700 1 --2- 172.21.15.40:0/1836439571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dc7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb53ffff700 1 -- 172.21.15.40:0/1836439571 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0f3b45d90 con 0x55f0f3dc6480 2023-07-18T03:30:39.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb544e17700 1 --2- 172.21.15.40:0/1836439571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dc8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:39.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb53ffff700 1 --2- 172.21.15.40:0/1836439571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dc6890 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb5300a1ec0 tx=0x7fb530099980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725739642f96806f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb5367fc700 1 -- 172.21.15.40:0/1836439571 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53009dfd0 con 0x55f0f3dc6480 2023-07-18T03:30:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb5367fc700 1 -- 172.21.15.40:0/1836439571 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5300a23f0 con 0x55f0f3dc6480 2023-07-18T03:30:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.114+0000 7fb5367fc700 1 -- 172.21.15.40:0/1836439571 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300996f0 con 0x55f0f3dc6480 2023-07-18T03:30:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.115+0000 7fb552a00b80 1 -- 172.21.15.40:0/1836439571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 msgr2=0x55f0f3dc6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.115+0000 7fb552a00b80 1 --2- 172.21.15.40:0/1836439571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dc6890 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb5300a1ec0 tx=0x7fb530099980 comp rx=0 tx=0).stop 2023-07-18T03:30:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.115+0000 7fb552a00b80 1 -- 172.21.15.40:0/1836439571 shutdown_connections 2023-07-18T03:30:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.115+0000 7fb552a00b80 1 --2- 172.21.15.40:0/1836439571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dc8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.115+0000 7fb552a00b80 1 --2- 172.21.15.40:0/1836439571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dc6890 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.115+0000 7fb552a00b80 1 --2- 172.21.15.40:0/1836439571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dc7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.115+0000 7fb552a00b80 1 -- 172.21.15.40:0/1836439571 >> 172.21.15.40:0/1836439571 conn(0x55f0f3dbbd50 msgr2=0x55f0f3dc1180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.115+0000 7fb552a00b80 1 -- 172.21.15.40:0/1836439571 shutdown_connections 2023-07-18T03:30:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.115+0000 7fb552a00b80 1 -- 172.21.15.40:0/1836439571 wait complete. 2023-07-18T03:30:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.115+0000 7fb552a00b80 1 -- start start 2023-07-18T03:30:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb552a00b80 1 Processor -- start 2023-07-18T03:30:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb552a00b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc6480 0x55f0f3dd0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb552a00b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dd06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb552a00b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dd7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb552a00b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f3cb7a70 con 0x55f0f3dc6dd0 2023-07-18T03:30:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb552a00b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f3cb7770 con 0x55f0f3dc6480 2023-07-18T03:30:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb552a00b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f3b9b8d0 con 0x55f0f3dc7c20 2023-07-18T03:30:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb552a00b80 10 client.0 _ll_register_callbacks cb 0x55f0f3cb5710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb53f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dd06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb53f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dd06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57404/0 (socket says 172.21.15.40:57404) 2023-07-18T03:30:39.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb53f7fe700 1 -- 172.21.15.40:0/1455256687 learned_addr learned my addr 172.21.15.40:0/1455256687 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb53ffff700 1 --2- 172.21.15.40:0/1455256687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc6480 0x55f0f3dd0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb544e17700 1 --2- 172.21.15.40:0/1455256687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dd7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb53f7fe700 1 -- 172.21.15.40:0/1455256687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc7c20 msgr2=0x55f0f3dd7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb53f7fe700 1 --2- 172.21.15.40:0/1455256687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dd7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb53f7fe700 1 -- 172.21.15.40:0/1455256687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc6480 msgr2=0x55f0f3dd0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb53f7fe700 1 --2- 172.21.15.40:0/1455256687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc6480 0x55f0f3dd0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb53f7fe700 1 -- 172.21.15.40:0/1455256687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5401a0150 con 0x55f0f3dc6dd0 2023-07-18T03:30:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb544e17700 1 --2- 172.21.15.40:0/1455256687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dd7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.116+0000 7fb53ffff700 1 --2- 172.21.15.40:0/1455256687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc6480 0x55f0f3dd0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.117+0000 7fb53f7fe700 1 --2- 172.21.15.40:0/1455256687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dd06c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb52800bfd0 tx=0x7fb528004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac6bd695060cea6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.117+0000 7fb51f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.117+0000 7fb51f7fe700 1 -- 172.21.15.40:0/1455256687 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb528023600 con 0x55f0f3dc6dd0 2023-07-18T03:30:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.117+0000 7fb552a00b80 1 -- 172.21.15.40:0/1455256687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0f3c714a0 con 0x55f0f3dc6dd0 2023-07-18T03:30:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.117+0000 7fb51f7fe700 1 -- 172.21.15.40:0/1455256687 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb528023760 con 0x55f0f3dc6dd0 2023-07-18T03:30:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.117+0000 7fb51f7fe700 1 -- 172.21.15.40:0/1455256687 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb528023920 con 0x55f0f3dc6dd0 2023-07-18T03:30:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.117+0000 7fb552a00b80 10 client.15222 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.117+0000 7fb552a00b80 1 -- 172.21.15.40:0/1455256687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f0f3b2ec10 con 0x55f0f3dc6dd0 2023-07-18T03:30:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.117+0000 7fb51f7fe700 1 -- 172.21.15.40:0/1455256687 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb528032020 con 0x55f0f3dc6dd0 2023-07-18T03:30:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.117+0000 7fb51f7fe700 1 -- 172.21.15.40:0/1455256687 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb528022070 con 0x55f0f3dc6dd0 2023-07-18T03:30:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.117+0000 7fb552a00b80 10 client.15222 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.117+0000 7fb552a00b80 10 client.15222 resolve_mds: resolved name 'l' to daemon mds.l 2023-07-18T03:30:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.117+0000 7fb552a00b80 20 client.15222 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.117+0000 7fb552a00b80 1 --2- 172.21.15.40:0/1455256687 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55f0f3ddb4b0 0x55f0f3dda860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.117+0000 7fb552a00b80 4 client.15222 mds_command: new command op to 24442 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:30:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.117+0000 7fb552a00b80 1 -- 172.21.15.40:0/1455256687 --> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0f3d10e60 con 0x55f0f3ddb4b0 2023-07-18T03:30:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.118+0000 7fb53ffff700 1 --2- 172.21.15.40:0/1455256687 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55f0f3ddb4b0 0x55f0f3dda860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.118+0000 7fb51f7fe700 10 client.15222 ms_handle_connect on v2:172.21.15.134:6832/2345264306 2023-07-18T03:30:39.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.118+0000 7fb53ffff700 1 --2- 172.21.15.40:0/1455256687 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55f0f3ddb4b0 0x55f0f3dda860 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=952b84066325e1ef server_cookie=16b43132c543aaaf in_seq=0 out_seq=0 2023-07-18T03:30:39.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.119+0000 7fb51f7fe700 1 -- 172.21.15.40:0/1455256687 <== mds.4 v2:172.21.15.134:6832/2345264306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f0f3d10e60 con 0x55f0f3ddb4b0 2023-07-18T03:30:39.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.119+0000 7fb51f7fe700 10 client.15222 handle_command_reply: tid=0 2023-07-18T03:30:39.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.119+0000 7fb552a00b80 1 client.15222 shutdown 2023-07-18T03:30:39.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.119+0000 7fb552a00b80 1 -- 172.21.15.40:0/1455256687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6dd0 msgr2=0x55f0f3dd06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.119+0000 7fb552a00b80 1 --2- 172.21.15.40:0/1455256687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dd06c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb52800bfd0 tx=0x7fb528004750 comp rx=0 tx=0).stop 2023-07-18T03:30:39.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.119+0000 7fb552a00b80 1 -- 172.21.15.40:0/1455256687 shutdown_connections 2023-07-18T03:30:39.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.119+0000 7fb552a00b80 1 --2- 172.21.15.40:0/1455256687 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55f0f3ddb4b0 0x55f0f3dda860 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.119+0000 7fb552a00b80 1 --2- 172.21.15.40:0/1455256687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dd7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.119+0000 7fb51f7fe700 0 client.15222 ms_handle_reset on v2:172.21.15.134:6832/2345264306 2023-07-18T03:30:39.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.119+0000 7fb552a00b80 1 --2- 172.21.15.40:0/1455256687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc6480 0x55f0f3dd0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.119+0000 7fb552a00b80 1 --2- 172.21.15.40:0/1455256687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dd06c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.119+0000 7fb552a00b80 1 -- 172.21.15.40:0/1455256687 >> 172.21.15.40:0/1455256687 conn(0x55f0f3dbbd50 msgr2=0x55f0f3dc0860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.119+0000 7fb552a00b80 1 -- 172.21.15.40:0/1455256687 shutdown_connections 2023-07-18T03:30:39.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.119+0000 7fb552a00b80 1 -- 172.21.15.40:0/1455256687 wait complete. 2023-07-18T03:30:39.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.119+0000 7fb552a00b80 20 client.15222 trim_cache size 0 max 16384 2023-07-18T03:30:39.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb552a00b80 1 Processor -- start 2023-07-18T03:30:39.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb552a00b80 1 -- start start 2023-07-18T03:30:39.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb552a00b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dc0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb552a00b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dd76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb552a00b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb552a00b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f3cc1470 con 0x55f0f3dc6480 2023-07-18T03:30:39.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb552a00b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f3ce6790 con 0x55f0f3dc7c20 2023-07-18T03:30:39.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb552a00b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f3ba7de0 con 0x55f0f3dc6dd0 2023-07-18T03:30:39.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb53ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dc0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb53ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dc0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57410/0 (socket says 172.21.15.40:57410) 2023-07-18T03:30:39.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb53ffff700 1 -- 172.21.15.40:0/73535072 learned_addr learned my addr 172.21.15.40:0/73535072 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb544e17700 1 --2- 172.21.15.40:0/73535072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb53f7fe700 1 --2- 172.21.15.40:0/73535072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dd76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb53ffff700 1 -- 172.21.15.40:0/73535072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc6dd0 msgr2=0x55f0f3dd76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb53ffff700 1 --2- 172.21.15.40:0/73535072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dd76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb53ffff700 1 -- 172.21.15.40:0/73535072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc7c20 msgr2=0x55f0f3dce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb53ffff700 1 --2- 172.21.15.40:0/73535072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb53ffff700 1 -- 172.21.15.40:0/73535072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0f3dbd5d0 con 0x55f0f3dc6480 2023-07-18T03:30:39.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb53ffff700 1 --2- 172.21.15.40:0/73535072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dc0860 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55f0f3dc8670 tx=0x7fb530003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69552db089023acf server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.137+0000 7fb536ffd700 1 -- 172.21.15.40:0/73535072 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53009e050 con 0x55f0f3dc6480 2023-07-18T03:30:39.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.138+0000 7fb536ffd700 1 -- 172.21.15.40:0/73535072 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb530099900 con 0x55f0f3dc6480 2023-07-18T03:30:39.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.138+0000 7fb536ffd700 1 -- 172.21.15.40:0/73535072 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530099490 con 0x55f0f3dc6480 2023-07-18T03:30:39.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.138+0000 7fb552a00b80 1 -- 172.21.15.40:0/73535072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 msgr2=0x55f0f3dc0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.138+0000 7fb552a00b80 1 --2- 172.21.15.40:0/73535072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dc0860 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55f0f3dc8670 tx=0x7fb530003780 comp rx=0 tx=0).stop 2023-07-18T03:30:39.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.138+0000 7fb552a00b80 1 -- 172.21.15.40:0/73535072 shutdown_connections 2023-07-18T03:30:39.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.138+0000 7fb552a00b80 1 --2- 172.21.15.40:0/73535072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.138+0000 7fb552a00b80 1 --2- 172.21.15.40:0/73535072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dc0860 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.138+0000 7fb552a00b80 1 --2- 172.21.15.40:0/73535072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dd76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.138+0000 7fb552a00b80 1 -- 172.21.15.40:0/73535072 >> 172.21.15.40:0/73535072 conn(0x55f0f3dbbd50 msgr2=0x55f0f3dcee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.138+0000 7fb552a00b80 1 -- 172.21.15.40:0/73535072 shutdown_connections 2023-07-18T03:30:39.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.138+0000 7fb552a00b80 1 -- 172.21.15.40:0/73535072 wait complete. 2023-07-18T03:30:39.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.138+0000 7fb552a00b80 1 -- start start 2023-07-18T03:30:39.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb552a00b80 1 Processor -- start 2023-07-18T03:30:39.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb552a00b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dd2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb552a00b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dd2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb552a00b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dd5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb552a00b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f3dd6510 con 0x55f0f3dc6480 2023-07-18T03:30:39.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb552a00b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f3dd6680 con 0x55f0f3dc7c20 2023-07-18T03:30:39.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb552a00b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f3dd6840 con 0x55f0f3dc6dd0 2023-07-18T03:30:39.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb552a00b80 10 client.0 _ll_register_callbacks cb 0x7fb52800aec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:39.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb53ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dd2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb53ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dd2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57416/0 (socket says 172.21.15.40:57416) 2023-07-18T03:30:39.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb53ffff700 1 -- 172.21.15.40:0/510162735 learned_addr learned my addr 172.21.15.40:0/510162735 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb544e17700 1 --2- 172.21.15.40:0/510162735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dd5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb53f7fe700 1 --2- 172.21.15.40:0/510162735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dd2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb53ffff700 1 -- 172.21.15.40:0/510162735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc6dd0 msgr2=0x55f0f3dd2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb53ffff700 1 --2- 172.21.15.40:0/510162735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dd2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb53ffff700 1 -- 172.21.15.40:0/510162735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc7c20 msgr2=0x55f0f3dd5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb53ffff700 1 --2- 172.21.15.40:0/510162735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dd5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb53ffff700 1 -- 172.21.15.40:0/510162735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0f3b45d90 con 0x55f0f3dc6480 2023-07-18T03:30:39.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb544e17700 1 --2- 172.21.15.40:0/510162735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dd5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:39.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb53ffff700 1 --2- 172.21.15.40:0/510162735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dd2a70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb5300982f0 tx=0x7fb5300cab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b867dab27534a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb51f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:39.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb51f7fe700 1 -- 172.21.15.40:0/510162735 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300b81e0 con 0x55f0f3dc6480 2023-07-18T03:30:39.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb51f7fe700 1 -- 172.21.15.40:0/510162735 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb5300b8340 con 0x55f0f3dc6480 2023-07-18T03:30:39.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.139+0000 7fb552a00b80 1 -- 172.21.15.40:0/510162735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5300a23f0 con 0x55f0f3dc6480 2023-07-18T03:30:39.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.140+0000 7fb51f7fe700 1 -- 172.21.15.40:0/510162735 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb530099490 con 0x55f0f3dc6480 2023-07-18T03:30:39.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.140+0000 7fb552a00b80 10 client.15228 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:39.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.140+0000 7fb552a00b80 1 -- 172.21.15.40:0/510162735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f0f3b6c200 con 0x55f0f3dc6480 2023-07-18T03:30:39.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.140+0000 7fb51f7fe700 1 -- 172.21.15.40:0/510162735 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb530080020 con 0x55f0f3dc6480 2023-07-18T03:30:39.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.140+0000 7fb51f7fe700 1 -- 172.21.15.40:0/510162735 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb530094bb0 con 0x55f0f3dc6480 2023-07-18T03:30:39.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.140+0000 7fb552a00b80 10 client.15228 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:39.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.140+0000 7fb552a00b80 10 client.15228 resolve_mds: resolved name 'l' to daemon mds.l 2023-07-18T03:30:39.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.140+0000 7fb552a00b80 20 client.15228 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:39.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.140+0000 7fb552a00b80 1 --2- 172.21.15.40:0/510162735 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55f0f3e347d0 0x55f0f3e31380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.140+0000 7fb552a00b80 4 client.15228 mds_command: new command op to 24442 tid=0[{"prefix": "status"}] 2023-07-18T03:30:39.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.140+0000 7fb552a00b80 1 -- 172.21.15.40:0/510162735 --> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f0f3cd3000 con 0x55f0f3e347d0 2023-07-18T03:30:39.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.140+0000 7fb53f7fe700 1 --2- 172.21.15.40:0/510162735 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55f0f3e347d0 0x55f0f3e31380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.141+0000 7fb51f7fe700 10 client.15228 ms_handle_connect on v2:172.21.15.134:6832/2345264306 2023-07-18T03:30:39.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.141+0000 7fb53f7fe700 1 --2- 172.21.15.40:0/510162735 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55f0f3e347d0 0x55f0f3e31380 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9fa54d3abc4ff35b server_cookie=5cc487d26251601a in_seq=0 out_seq=0 2023-07-18T03:30:39.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.141+0000 7fb51f7fe700 1 -- 172.21.15.40:0/510162735 <== mds.4 v2:172.21.15.134:6832/2345264306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55f0f3cd3000 con 0x55f0f3e347d0 2023-07-18T03:30:39.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.141+0000 7fb51f7fe700 10 client.15228 handle_command_reply: tid=0 2023-07-18T03:30:39.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.141+0000 7fb552a00b80 1 client.15228 shutdown 2023-07-18T03:30:39.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.141+0000 7fb552a00b80 1 -- 172.21.15.40:0/510162735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 msgr2=0x55f0f3dd2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.141+0000 7fb552a00b80 1 --2- 172.21.15.40:0/510162735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dd2a70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb5300982f0 tx=0x7fb5300cab30 comp rx=0 tx=0).stop 2023-07-18T03:30:39.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.141+0000 7fb552a00b80 1 -- 172.21.15.40:0/510162735 shutdown_connections 2023-07-18T03:30:39.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.141+0000 7fb552a00b80 1 --2- 172.21.15.40:0/510162735 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55f0f3e347d0 0x55f0f3e31380 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.141+0000 7fb552a00b80 1 --2- 172.21.15.40:0/510162735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f0f3dc7c20 0x55f0f3dd5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.141+0000 7fb552a00b80 1 --2- 172.21.15.40:0/510162735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f0f3dc6480 0x55f0f3dd2a70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.141+0000 7fb51f7fe700 0 client.15228 ms_handle_reset on v2:172.21.15.134:6832/2345264306 2023-07-18T03:30:39.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.141+0000 7fb552a00b80 1 --2- 172.21.15.40:0/510162735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f0f3dc6dd0 0x55f0f3dd2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.141+0000 7fb552a00b80 1 -- 172.21.15.40:0/510162735 >> 172.21.15.40:0/510162735 conn(0x55f0f3dbbd50 msgr2=0x55f0f3dc0860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.141+0000 7fb552a00b80 1 -- 172.21.15.40:0/510162735 shutdown_connections 2023-07-18T03:30:39.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.141+0000 7fb552a00b80 1 -- 172.21.15.40:0/510162735 wait complete. 2023-07-18T03:30:39.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.141+0000 7fb552a00b80 20 client.15228 trim_cache size 0 max 16384 2023-07-18T03:30:39.149 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:30:39.149 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "3af3dba8-251a-11ee-9b34-001a4aab830c", 2023-07-18T03:30:39.149 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 4, 2023-07-18T03:30:39.150 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24442, 2023-07-18T03:30:39.150 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-07-18T03:30:39.150 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-07-18T03:30:39.150 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-07-18T03:30:39.150 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 65.509294183999998, 2023-07-18T03:30:39.150 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 27, 2023-07-18T03:30:39.150 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 83, 2023-07-18T03:30:39.150 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 81, 2023-07-18T03:30:39.151 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 102.614961518 2023-07-18T03:30:39.151 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:30:39.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.142+0000 7fb54687a700 1 -- 172.21.15.40:0/1842864655 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb52c0a11e0 msgr2=0x7fb52c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.142+0000 7fb54687a700 1 --2- 172.21.15.40:0/1842864655 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb52c0a11e0 0x7fb52c0a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb52800b3c0 tx=0x7fb52800d040 comp rx=0 tx=0).stop 2023-07-18T03:30:39.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.142+0000 7fb54687a700 1 -- 172.21.15.40:0/1842864655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5400fe9e0 msgr2=0x7fb540197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.142+0000 7fb54687a700 1 --2- 172.21.15.40:0/1842864655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5400fe9e0 0x7fb540197b10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb53000e970 tx=0x7fb53000ec80 comp rx=0 tx=0).stop 2023-07-18T03:30:39.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.142+0000 7fb54687a700 1 -- 172.21.15.40:0/1842864655 shutdown_connections 2023-07-18T03:30:39.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.142+0000 7fb54687a700 1 --2- 172.21.15.40:0/1842864655 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb52c0a11e0 0x7fb52c0a3690 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.142+0000 7fb54687a700 1 --2- 172.21.15.40:0/1842864655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb540100560 0x7fb54019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.142+0000 7fb54687a700 1 --2- 172.21.15.40:0/1842864655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5400fe9e0 0x7fb540197b10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.142+0000 7fb54687a700 1 --2- 172.21.15.40:0/1842864655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5400ffbd0 0x7fb540198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.142+0000 7fb54687a700 1 -- 172.21.15.40:0/1842864655 >> 172.21.15.40:0/1842864655 conn(0x7fb5400fa000 msgr2=0x7fb5400fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.142+0000 7fb54687a700 1 -- 172.21.15.40:0/1842864655 shutdown_connections 2023-07-18T03:30:39.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.142+0000 7fb54687a700 1 -- 172.21.15.40:0/1842864655 wait complete. 2023-07-18T03:30:39.154 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-07-18T03:30:39.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.283+0000 7fe074631700 1 -- 172.21.15.40:0/342011100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06c0ffc40 msgr2=0x7fe06c100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.283+0000 7fe074631700 1 --2- 172.21.15.40:0/342011100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06c0ffc40 0x7fe06c100090 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009990 tx=0x7fe05c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.284+0000 7fe074631700 1 -- 172.21.15.40:0/342011100 shutdown_connections 2023-07-18T03:30:39.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.284+0000 7fe074631700 1 --2- 172.21.15.40:0/342011100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe06c1005d0 0x7fe06c106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.284+0000 7fe074631700 1 --2- 172.21.15.40:0/342011100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe06c0fea50 0x7fe06c0fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.284+0000 7fe074631700 1 --2- 172.21.15.40:0/342011100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06c0ffc40 0x7fe06c100090 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.284+0000 7fe074631700 1 -- 172.21.15.40:0/342011100 >> 172.21.15.40:0/342011100 conn(0x7fe06c0fa070 msgr2=0x7fe06c0fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.284+0000 7fe074631700 1 -- 172.21.15.40:0/342011100 shutdown_connections 2023-07-18T03:30:39.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.284+0000 7fe074631700 1 -- 172.21.15.40:0/342011100 wait complete. 2023-07-18T03:30:39.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.284+0000 7fe074631700 1 Processor -- start 2023-07-18T03:30:39.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.284+0000 7fe074631700 1 -- start start 2023-07-18T03:30:39.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.284+0000 7fe074631700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe06c0fea50 0x7fe06c1957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.284+0000 7fe074631700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe06c0ffc40 0x7fe06c195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.284+0000 7fe074631700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06c1005d0 0x7fe06c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.284+0000 7fe074631700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c19d320 con 0x7fe06c1005d0 2023-07-18T03:30:39.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.284+0000 7fe074631700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c19d460 con 0x7fe06c0ffc40 2023-07-18T03:30:39.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.284+0000 7fe074631700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c19d5f0 con 0x7fe06c0fea50 2023-07-18T03:30:39.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.284+0000 7fe072bce700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06c1005d0 0x7fe06c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.284+0000 7fe072bce700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06c1005d0 0x7fe06c19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57432/0 (socket says 172.21.15.40:57432) 2023-07-18T03:30:39.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.285+0000 7fe072bce700 1 -- 172.21.15.40:0/3456009493 learned_addr learned my addr 172.21.15.40:0/3456009493 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.285+0000 7fe071bcc700 1 --2- 172.21.15.40:0/3456009493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe06c0ffc40 0x7fe06c195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.285+0000 7fe0723cd700 1 --2- 172.21.15.40:0/3456009493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe06c0fea50 0x7fe06c1957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.285+0000 7fe072bce700 1 -- 172.21.15.40:0/3456009493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe06c0fea50 msgr2=0x7fe06c1957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.285+0000 7fe072bce700 1 --2- 172.21.15.40:0/3456009493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe06c0fea50 0x7fe06c1957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.285+0000 7fe072bce700 1 -- 172.21.15.40:0/3456009493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe06c0ffc40 msgr2=0x7fe06c195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.285+0000 7fe072bce700 1 --2- 172.21.15.40:0/3456009493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe06c0ffc40 0x7fe06c195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.285+0000 7fe072bce700 1 -- 172.21.15.40:0/3456009493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c0095f0 con 0x7fe06c1005d0 2023-07-18T03:30:39.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.285+0000 7fe072bce700 1 --2- 172.21.15.40:0/3456009493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06c1005d0 0x7fe06c19cd50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe064007d70 tx=0x7fe06400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.285+0000 7fe0637fe700 1 -- 172.21.15.40:0/3456009493 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06400ad40 con 0x7fe06c1005d0 2023-07-18T03:30:39.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.285+0000 7fe074631700 1 -- 172.21.15.40:0/3456009493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe06c19d8d0 con 0x7fe06c1005d0 2023-07-18T03:30:39.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.285+0000 7fe0637fe700 1 -- 172.21.15.40:0/3456009493 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0640128e0 con 0x7fe06c1005d0 2023-07-18T03:30:39.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.285+0000 7fe0637fe700 1 -- 172.21.15.40:0/3456009493 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0640116b0 con 0x7fe06c1005d0 2023-07-18T03:30:39.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.285+0000 7fe074631700 1 -- 172.21.15.40:0/3456009493 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06c19dd90 con 0x7fe06c1005d0 2023-07-18T03:30:39.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.287+0000 7fe0637fe700 1 -- 172.21.15.40:0/3456009493 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe064012420 con 0x7fe06c1005d0 2023-07-18T03:30:39.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.288+0000 7fe0637fe700 1 --2- 172.21.15.40:0/3456009493 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0580a11e0 0x7fe0580a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.288+0000 7fe0637fe700 1 -- 172.21.15.40:0/3456009493 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe0640ccc30 con 0x7fe06c1005d0 2023-07-18T03:30:39.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.288+0000 7fe0723cd700 1 --2- 172.21.15.40:0/3456009493 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0580a11e0 0x7fe0580a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.288+0000 7fe0723cd700 1 --2- 172.21.15.40:0/3456009493 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0580a11e0 0x7fe0580a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe068005950 tx=0x7fe0680058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.289+0000 7fe0807b7b80 1 Processor -- start 2023-07-18T03:30:39.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.289+0000 7fe0807b7b80 1 -- start start 2023-07-18T03:30:39.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.289+0000 7fe0807b7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937ab480 0x55e5937ab890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.289+0000 7fe0807b7b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937abdd0 0x55e5937ac690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.289+0000 7fe0807b7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937acc20 0x55e5937ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.289+0000 7fe0807b7b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5936b0c60 con 0x55e5937abdd0 2023-07-18T03:30:39.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.289+0000 7fe0807b7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5937039f0 con 0x55e5937acc20 2023-07-18T03:30:39.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.289+0000 7fe0807b7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5936fce80 con 0x55e5937ab480 2023-07-18T03:30:39.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.289+0000 7fe071bcc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937abdd0 0x55e5937ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.289+0000 7fe071bcc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937abdd0 0x55e5937ac690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57438/0 (socket says 172.21.15.40:57438) 2023-07-18T03:30:39.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.289+0000 7fe071bcc700 1 -- 172.21.15.40:0/1258468069 learned_addr learned my addr 172.21.15.40:0/1258468069 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.289+0000 7fe072bce700 1 --2- 172.21.15.40:0/1258468069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937acc20 0x55e5937ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.289+0000 7fe0723cd700 1 --2- 172.21.15.40:0/1258468069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937ab480 0x55e5937ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.289+0000 7fe071bcc700 1 -- 172.21.15.40:0/1258468069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937ab480 msgr2=0x55e5937ab890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.289+0000 7fe071bcc700 1 --2- 172.21.15.40:0/1258468069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937ab480 0x55e5937ab890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.290+0000 7fe071bcc700 1 -- 172.21.15.40:0/1258468069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937acc20 msgr2=0x55e5937ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.290+0000 7fe071bcc700 1 --2- 172.21.15.40:0/1258468069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937acc20 0x55e5937ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.290+0000 7fe071bcc700 1 -- 172.21.15.40:0/1258468069 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e59352ad90 con 0x55e5937abdd0 2023-07-18T03:30:39.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.290+0000 7fe0723cd700 1 --2- 172.21.15.40:0/1258468069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937ab480 0x55e5937ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:39.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.290+0000 7fe071bcc700 1 --2- 172.21.15.40:0/1258468069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937abdd0 0x55e5937ac690 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe05c015000 tx=0x7fe05c005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ecc420aacd77be5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.290+0000 7fe060ff9700 1 -- 172.21.15.40:0/1258468069 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c01d070 con 0x55e5937abdd0 2023-07-18T03:30:39.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.290+0000 7fe060ff9700 1 -- 172.21.15.40:0/1258468069 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe05c01e040 con 0x55e5937abdd0 2023-07-18T03:30:39.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.290+0000 7fe060ff9700 1 -- 172.21.15.40:0/1258468069 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c004760 con 0x55e5937abdd0 2023-07-18T03:30:39.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 1 -- 172.21.15.40:0/1258468069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937abdd0 msgr2=0x55e5937ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/1258468069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937abdd0 0x55e5937ac690 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe05c015000 tx=0x7fe05c005c70 comp rx=0 tx=0).stop 2023-07-18T03:30:39.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 1 -- 172.21.15.40:0/1258468069 shutdown_connections 2023-07-18T03:30:39.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/1258468069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937acc20 0x55e5937ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/1258468069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937ab480 0x55e5937ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/1258468069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937abdd0 0x55e5937ac690 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 1 -- 172.21.15.40:0/1258468069 >> 172.21.15.40:0/1258468069 conn(0x55e5937a0d50 msgr2=0x55e5937a6180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 1 -- 172.21.15.40:0/1258468069 shutdown_connections 2023-07-18T03:30:39.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 1 -- 172.21.15.40:0/1258468069 wait complete. 2023-07-18T03:30:39.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 1 -- start start 2023-07-18T03:30:39.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 1 Processor -- start 2023-07-18T03:30:39.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937ab480 0x55e5937b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937abdd0 0x55e5937b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937acc20 0x55e5937bc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59369ca70 con 0x55e5937acc20 2023-07-18T03:30:39.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59369c770 con 0x55e5937ab480 2023-07-18T03:30:39.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5935808d0 con 0x55e5937abdd0 2023-07-18T03:30:39.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0807b7b80 10 client.0 _ll_register_callbacks cb 0x55e59369a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:39.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe072bce700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937acc20 0x55e5937bc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe072bce700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937acc20 0x55e5937bc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57446/0 (socket says 172.21.15.40:57446) 2023-07-18T03:30:39.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe072bce700 1 -- 172.21.15.40:0/2513483152 learned_addr learned my addr 172.21.15.40:0/2513483152 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.291+0000 7fe0723cd700 1 --2- 172.21.15.40:0/2513483152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937ab480 0x55e5937b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.292+0000 7fe071bcc700 1 --2- 172.21.15.40:0/2513483152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937abdd0 0x55e5937b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.292+0000 7fe072bce700 1 -- 172.21.15.40:0/2513483152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937abdd0 msgr2=0x55e5937b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.292+0000 7fe072bce700 1 --2- 172.21.15.40:0/2513483152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937abdd0 0x55e5937b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.292+0000 7fe072bce700 1 -- 172.21.15.40:0/2513483152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937ab480 msgr2=0x55e5937b5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.292+0000 7fe072bce700 1 --2- 172.21.15.40:0/2513483152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937ab480 0x55e5937b5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.292+0000 7fe072bce700 1 -- 172.21.15.40:0/2513483152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06c068820 con 0x55e5937acc20 2023-07-18T03:30:39.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.292+0000 7fe072bce700 1 --2- 172.21.15.40:0/2513483152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937acc20 0x55e5937bc5e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe064099f60 tx=0x7fe06409a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aedfa555fe438d6d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.292+0000 7fe055ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:39.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.292+0000 7fe055ffb700 1 -- 172.21.15.40:0/2513483152 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0640ca810 con 0x55e5937acc20 2023-07-18T03:30:39.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.292+0000 7fe055ffb700 1 -- 172.21.15.40:0/2513483152 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe06409a230 con 0x55e5937acc20 2023-07-18T03:30:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.292+0000 7fe0807b7b80 1 -- 172.21.15.40:0/2513483152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5936564a0 con 0x55e5937acc20 2023-07-18T03:30:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.292+0000 7fe055ffb700 1 -- 172.21.15.40:0/2513483152 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe06409a3f0 con 0x55e5937acc20 2023-07-18T03:30:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.292+0000 7fe0807b7b80 10 client.15240 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.292+0000 7fe0807b7b80 1 -- 172.21.15.40:0/2513483152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e593513c10 con 0x55e5937acc20 2023-07-18T03:30:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.292+0000 7fe055ffb700 1 -- 172.21.15.40:0/2513483152 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0640c8020 con 0x55e5937acc20 2023-07-18T03:30:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.292+0000 7fe055ffb700 1 -- 172.21.15.40:0/2513483152 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe0640995e0 con 0x55e5937acc20 2023-07-18T03:30:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.293+0000 7fe0807b7b80 10 client.15240 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.293+0000 7fe0807b7b80 10 client.15240 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-18T03:30:39.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.293+0000 7fe0807b7b80 20 client.15240 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:39.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.293+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/2513483152 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x55e5937c02e0 0x55e5937bf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.293+0000 7fe0807b7b80 4 client.15240 mds_command: new command op to 24452 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:30:39.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.293+0000 7fe0807b7b80 1 -- 172.21.15.40:0/2513483152 --> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5936f5e60 con 0x55e5937c02e0 2023-07-18T03:30:39.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.293+0000 7fe0723cd700 1 --2- 172.21.15.40:0/2513483152 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x55e5937c02e0 0x55e5937bf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.293+0000 7fe055ffb700 10 client.15240 ms_handle_connect on v2:172.21.15.64:6834/1864495009 2023-07-18T03:30:39.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.293+0000 7fe0723cd700 1 --2- 172.21.15.40:0/2513483152 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x55e5937c02e0 0x55e5937bf740 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b4d215114584d23a server_cookie=82d8864d1f528b04 in_seq=0 out_seq=0 2023-07-18T03:30:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.294+0000 7fe055ffb700 1 -- 172.21.15.40:0/2513483152 <== mds.3 v2:172.21.15.64:6834/1864495009 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e5936f5e60 con 0x55e5937c02e0 2023-07-18T03:30:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.294+0000 7fe055ffb700 10 client.15240 handle_command_reply: tid=0 2023-07-18T03:30:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.294+0000 7fe0807b7b80 1 client.15240 shutdown 2023-07-18T03:30:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.294+0000 7fe0807b7b80 1 -- 172.21.15.40:0/2513483152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937acc20 msgr2=0x55e5937bc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.294+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/2513483152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937acc20 0x55e5937bc5e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe064099f60 tx=0x7fe06409a050 comp rx=0 tx=0).stop 2023-07-18T03:30:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.294+0000 7fe0807b7b80 1 -- 172.21.15.40:0/2513483152 shutdown_connections 2023-07-18T03:30:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.294+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/2513483152 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x55e5937c02e0 0x55e5937bf740 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.294+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/2513483152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937acc20 0x55e5937bc5e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.294+0000 7fe055ffb700 0 client.15240 ms_handle_reset on v2:172.21.15.64:6834/1864495009 2023-07-18T03:30:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.294+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/2513483152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937ab480 0x55e5937b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.294+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/2513483152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937abdd0 0x55e5937b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.294+0000 7fe0807b7b80 1 -- 172.21.15.40:0/2513483152 >> 172.21.15.40:0/2513483152 conn(0x55e5937a0d50 msgr2=0x55e5937a5860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.294+0000 7fe0807b7b80 1 -- 172.21.15.40:0/2513483152 shutdown_connections 2023-07-18T03:30:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.294+0000 7fe0807b7b80 1 -- 172.21.15.40:0/2513483152 wait complete. 2023-07-18T03:30:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.294+0000 7fe0807b7b80 20 client.15240 trim_cache size 0 max 16384 2023-07-18T03:30:39.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.312+0000 7fe0807b7b80 1 Processor -- start 2023-07-18T03:30:39.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.312+0000 7fe0807b7b80 1 -- start start 2023-07-18T03:30:39.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.312+0000 7fe0807b7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937ab480 0x55e5937ad070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.312+0000 7fe0807b7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937abdd0 0x55e5937a5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.312+0000 7fe0807b7b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937acc20 0x55e5937b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.312+0000 7fe0807b7b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5937b4c30 con 0x55e5937acc20 2023-07-18T03:30:39.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.312+0000 7fe0807b7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5936cb790 con 0x55e5937abdd0 2023-07-18T03:30:39.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.312+0000 7fe0807b7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e59358cde0 con 0x55e5937ab480 2023-07-18T03:30:39.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.312+0000 7fe072bce700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937acc20 0x55e5937b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.312+0000 7fe072bce700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937acc20 0x55e5937b3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57454/0 (socket says 172.21.15.40:57454) 2023-07-18T03:30:39.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.312+0000 7fe072bce700 1 -- 172.21.15.40:0/3962964268 learned_addr learned my addr 172.21.15.40:0/3962964268 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.312+0000 7fe0723cd700 1 --2- 172.21.15.40:0/3962964268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937ab480 0x55e5937ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.312+0000 7fe071bcc700 1 --2- 172.21.15.40:0/3962964268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937abdd0 0x55e5937a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.313+0000 7fe072bce700 1 -- 172.21.15.40:0/3962964268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937ab480 msgr2=0x55e5937ad070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.313+0000 7fe072bce700 1 --2- 172.21.15.40:0/3962964268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937ab480 0x55e5937ad070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.313+0000 7fe072bce700 1 -- 172.21.15.40:0/3962964268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937abdd0 msgr2=0x55e5937a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.313+0000 7fe072bce700 1 --2- 172.21.15.40:0/3962964268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937abdd0 0x55e5937a5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.313+0000 7fe072bce700 1 -- 172.21.15.40:0/3962964268 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e593551200 con 0x55e5937acc20 2023-07-18T03:30:39.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.313+0000 7fe072bce700 1 --2- 172.21.15.40:0/3962964268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937acc20 0x55e5937b3030 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe06400afd0 tx=0x7fe064006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61700b3700220f17 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.313+0000 7fe0617fa700 1 -- 172.21.15.40:0/3962964268 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064007aa0 con 0x55e5937acc20 2023-07-18T03:30:39.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.313+0000 7fe0617fa700 1 -- 172.21.15.40:0/3962964268 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0640c8020 con 0x55e5937acc20 2023-07-18T03:30:39.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.313+0000 7fe0617fa700 1 -- 172.21.15.40:0/3962964268 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0640994c0 con 0x55e5937acc20 2023-07-18T03:30:39.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.313+0000 7fe0807b7b80 1 -- 172.21.15.40:0/3962964268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937acc20 msgr2=0x55e5937b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.313+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/3962964268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937acc20 0x55e5937b3030 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe06400afd0 tx=0x7fe064006040 comp rx=0 tx=0).stop 2023-07-18T03:30:39.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.313+0000 7fe0807b7b80 1 -- 172.21.15.40:0/3962964268 shutdown_connections 2023-07-18T03:30:39.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.313+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/3962964268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937acc20 0x55e5937b3030 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.313+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/3962964268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937ab480 0x55e5937ad070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.313+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/3962964268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937abdd0 0x55e5937a5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe0807b7b80 1 -- 172.21.15.40:0/3962964268 >> 172.21.15.40:0/3962964268 conn(0x55e5937a0d50 msgr2=0x55e5937b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe0807b7b80 1 -- 172.21.15.40:0/3962964268 shutdown_connections 2023-07-18T03:30:39.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe0807b7b80 1 -- 172.21.15.40:0/3962964268 wait complete. 2023-07-18T03:30:39.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe0807b7b80 1 -- start start 2023-07-18T03:30:39.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe0807b7b80 1 Processor -- start 2023-07-18T03:30:39.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe0807b7b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937ab480 0x55e5937b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe0807b7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937abdd0 0x55e5937b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe0807b7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937acc20 0x55e5938181c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe0807b7b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e593818700 con 0x55e5937ab480 2023-07-18T03:30:39.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe0807b7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e593818840 con 0x55e5937acc20 2023-07-18T03:30:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe0807b7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e593818980 con 0x55e5937abdd0 2023-07-18T03:30:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe0807b7b80 10 client.0 _ll_register_callbacks cb 0x7fe064099950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe0723cd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937ab480 0x55e5937b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe0723cd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937ab480 0x55e5937b1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57456/0 (socket says 172.21.15.40:57456) 2023-07-18T03:30:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe0723cd700 1 -- 172.21.15.40:0/1174937092 learned_addr learned my addr 172.21.15.40:0/1174937092 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe072bce700 1 --2- 172.21.15.40:0/1174937092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937acc20 0x55e5938181c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe071bcc700 1 --2- 172.21.15.40:0/1174937092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937abdd0 0x55e5937b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.314+0000 7fe0723cd700 1 -- 172.21.15.40:0/1174937092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937abdd0 msgr2=0x55e5937b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe0723cd700 1 --2- 172.21.15.40:0/1174937092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937abdd0 0x55e5937b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe0723cd700 1 -- 172.21.15.40:0/1174937092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937acc20 msgr2=0x55e5938181c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe0723cd700 1 --2- 172.21.15.40:0/1174937092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937acc20 0x55e5938181c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe0723cd700 1 -- 172.21.15.40:0/1174937092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e593513c10 con 0x55e5937ab480 2023-07-18T03:30:39.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe072bce700 1 --2- 172.21.15.40:0/1174937092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937acc20 0x55e5938181c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:39.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe071bcc700 1 --2- 172.21.15.40:0/1174937092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937abdd0 0x55e5937b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:39.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe0723cd700 1 --2- 172.21.15.40:0/1174937092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937ab480 0x55e5937b1da0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe068014040 tx=0x7fe0680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c17a6deb93cc51e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe055ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:39.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe055ffb700 1 -- 172.21.15.40:0/1174937092 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe068025040 con 0x55e5937ab480 2023-07-18T03:30:39.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe0807b7b80 1 -- 172.21.15.40:0/1174937092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe05c01e040 con 0x55e5937ab480 2023-07-18T03:30:39.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe055ffb700 1 -- 172.21.15.40:0/1174937092 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe068011850 con 0x55e5937ab480 2023-07-18T03:30:39.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe055ffb700 1 -- 172.21.15.40:0/1174937092 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe068011a10 con 0x55e5937ab480 2023-07-18T03:30:39.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe0807b7b80 10 client.15246 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:39.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe0807b7b80 1 -- 172.21.15.40:0/1174937092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e593529260 con 0x55e5937ab480 2023-07-18T03:30:39.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe055ffb700 1 -- 172.21.15.40:0/1174937092 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe06803e080 con 0x55e5937ab480 2023-07-18T03:30:39.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe055ffb700 1 -- 172.21.15.40:0/1174937092 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe068038030 con 0x55e5937ab480 2023-07-18T03:30:39.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe0807b7b80 10 client.15246 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:39.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe0807b7b80 10 client.15246 resolve_mds: resolved name 'e' to daemon mds.e 2023-07-18T03:30:39.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe0807b7b80 20 client.15246 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:39.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.315+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/1174937092 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x55e59381d780 0x55e59381a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.316+0000 7fe0807b7b80 4 client.15246 mds_command: new command op to 24452 tid=0[{"prefix": "status"}] 2023-07-18T03:30:39.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.316+0000 7fe0807b7b80 1 -- 172.21.15.40:0/1174937092 --> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e5936b8000 con 0x55e59381d780 2023-07-18T03:30:39.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.316+0000 7fe071bcc700 1 --2- 172.21.15.40:0/1174937092 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x55e59381d780 0x55e59381a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.316+0000 7fe055ffb700 10 client.15246 ms_handle_connect on v2:172.21.15.64:6834/1864495009 2023-07-18T03:30:39.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.316+0000 7fe071bcc700 1 --2- 172.21.15.40:0/1174937092 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x55e59381d780 0x55e59381a090 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9d8b5a83d99c234d server_cookie=ba319734b2b8a080 in_seq=0 out_seq=0 2023-07-18T03:30:39.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.316+0000 7fe055ffb700 1 -- 172.21.15.40:0/1174937092 <== mds.3 v2:172.21.15.64:6834/1864495009 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55e5936b8000 con 0x55e59381d780 2023-07-18T03:30:39.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.316+0000 7fe055ffb700 10 client.15246 handle_command_reply: tid=0 2023-07-18T03:30:39.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.316+0000 7fe0807b7b80 1 client.15246 shutdown 2023-07-18T03:30:39.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.316+0000 7fe0807b7b80 1 -- 172.21.15.40:0/1174937092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937ab480 msgr2=0x55e5937b1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.316+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/1174937092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937ab480 0x55e5937b1da0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe068014040 tx=0x7fe0680047c0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.316+0000 7fe0807b7b80 1 -- 172.21.15.40:0/1174937092 shutdown_connections 2023-07-18T03:30:39.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/1174937092 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x55e59381d780 0x55e59381a090 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/1174937092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5937acc20 0x55e5938181c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/1174937092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e5937ab480 0x55e5937b1da0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe055ffb700 0 client.15246 ms_handle_reset on v2:172.21.15.64:6834/1864495009 2023-07-18T03:30:39.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe0807b7b80 1 --2- 172.21.15.40:0/1174937092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e5937abdd0 0x55e5937b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe0807b7b80 1 -- 172.21.15.40:0/1174937092 >> 172.21.15.40:0/1174937092 conn(0x55e5937a0d50 msgr2=0x55e5937b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe0807b7b80 1 -- 172.21.15.40:0/1174937092 shutdown_connections 2023-07-18T03:30:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe0807b7b80 1 -- 172.21.15.40:0/1174937092 wait complete. 2023-07-18T03:30:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe0807b7b80 20 client.15246 trim_cache size 0 max 16384 2023-07-18T03:30:39.324 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:30:39.324 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "3af3dba8-251a-11ee-9b34-001a4aab830c", 2023-07-18T03:30:39.324 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 3, 2023-07-18T03:30:39.325 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24452, 2023-07-18T03:30:39.325 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-07-18T03:30:39.325 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-07-18T03:30:39.325 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-07-18T03:30:39.325 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 66.691940556000006, 2023-07-18T03:30:39.325 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 27, 2023-07-18T03:30:39.325 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 83, 2023-07-18T03:30:39.326 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 81, 2023-07-18T03:30:39.326 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 94.248016308000004 2023-07-18T03:30:39.327 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:30:39.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe074631700 1 -- 172.21.15.40:0/3456009493 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0580a11e0 msgr2=0x7fe0580a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe074631700 1 --2- 172.21.15.40:0/3456009493 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0580a11e0 0x7fe0580a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe068005950 tx=0x7fe0680058e0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe074631700 1 -- 172.21.15.40:0/3456009493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06c1005d0 msgr2=0x7fe06c19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe074631700 1 --2- 172.21.15.40:0/3456009493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06c1005d0 0x7fe06c19cd50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe064007d70 tx=0x7fe06400a340 comp rx=0 tx=0).stop 2023-07-18T03:30:39.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe074631700 1 -- 172.21.15.40:0/3456009493 shutdown_connections 2023-07-18T03:30:39.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe074631700 1 --2- 172.21.15.40:0/3456009493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06c1005d0 0x7fe06c19cd50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe074631700 1 --2- 172.21.15.40:0/3456009493 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0580a11e0 0x7fe0580a3690 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe074631700 1 --2- 172.21.15.40:0/3456009493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe06c0fea50 0x7fe06c1957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe074631700 1 --2- 172.21.15.40:0/3456009493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe06c0ffc40 0x7fe06c195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe074631700 1 -- 172.21.15.40:0/3456009493 >> 172.21.15.40:0/3456009493 conn(0x7fe06c0fa070 msgr2=0x7fe06c0fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe074631700 1 -- 172.21.15.40:0/3456009493 shutdown_connections 2023-07-18T03:30:39.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.317+0000 7fe074631700 1 -- 172.21.15.40:0/3456009493 wait complete. 2023-07-18T03:30:39.330 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-07-18T03:30:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.460+0000 7ffbe220e700 1 -- 172.21.15.40:0/556069399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbdc0ffbd0 msgr2=0x7ffbdc100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.460+0000 7ffbe220e700 1 --2- 172.21.15.40:0/556069399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbdc0ffbd0 0x7ffbdc100020 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009990 tx=0x7ffbc4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.460+0000 7ffbe220e700 1 -- 172.21.15.40:0/556069399 shutdown_connections 2023-07-18T03:30:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.460+0000 7ffbe220e700 1 --2- 172.21.15.40:0/556069399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbdc100560 0x7ffbdc106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.460+0000 7ffbe220e700 1 --2- 172.21.15.40:0/556069399 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbdc0fe9e0 0x7ffbdc0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.460+0000 7ffbe220e700 1 --2- 172.21.15.40:0/556069399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbdc0ffbd0 0x7ffbdc100020 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.460+0000 7ffbe220e700 1 -- 172.21.15.40:0/556069399 >> 172.21.15.40:0/556069399 conn(0x7ffbdc0fa000 msgr2=0x7ffbdc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.460+0000 7ffbe220e700 1 -- 172.21.15.40:0/556069399 shutdown_connections 2023-07-18T03:30:39.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.460+0000 7ffbe220e700 1 -- 172.21.15.40:0/556069399 wait complete. 2023-07-18T03:30:39.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.461+0000 7ffbe220e700 1 Processor -- start 2023-07-18T03:30:39.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.461+0000 7ffbe220e700 1 -- start start 2023-07-18T03:30:39.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.461+0000 7ffbe220e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbdc0fe9e0 0x7ffbdc068240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.461+0000 7ffbe220e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbdc0ffbd0 0x7ffbdc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.461+0000 7ffbe220e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbdc100560 0x7ffbdc19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.461+0000 7ffbe220e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc19d2f0 con 0x7ffbdc100560 2023-07-18T03:30:39.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.461+0000 7ffbe220e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc19d460 con 0x7ffbdc0ffbd0 2023-07-18T03:30:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.461+0000 7ffbe220e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc19d620 con 0x7ffbdc0fe9e0 2023-07-18T03:30:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.461+0000 7ffbdbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbdc100560 0x7ffbdc19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.461+0000 7ffbdbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbdc100560 0x7ffbdc19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57474/0 (socket says 172.21.15.40:57474) 2023-07-18T03:30:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.461+0000 7ffbdbfff700 1 -- 172.21.15.40:0/1896191229 learned_addr learned my addr 172.21.15.40:0/1896191229 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.461+0000 7ffbdb7fe700 1 --2- 172.21.15.40:0/1896191229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbdc0fe9e0 0x7ffbdc068240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.462+0000 7ffbdbfff700 1 -- 172.21.15.40:0/1896191229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbdc0fe9e0 msgr2=0x7ffbdc068240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.462+0000 7ffbdbfff700 1 --2- 172.21.15.40:0/1896191229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbdc0fe9e0 0x7ffbdc068240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.462+0000 7ffbdbfff700 1 -- 172.21.15.40:0/1896191229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbdc0ffbd0 msgr2=0x7ffbdc19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.462+0000 7ffbdaffd700 1 --2- 172.21.15.40:0/1896191229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbdc0ffbd0 0x7ffbdc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.462+0000 7ffbdbfff700 1 --2- 172.21.15.40:0/1896191229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbdc0ffbd0 0x7ffbdc19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.462+0000 7ffbdbfff700 1 -- 172.21.15.40:0/1896191229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc40095f0 con 0x7ffbdc100560 2023-07-18T03:30:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.462+0000 7ffbdaffd700 1 --2- 172.21.15.40:0/1896191229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbdc0ffbd0 0x7ffbdc19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.462+0000 7ffbdbfff700 1 --2- 172.21.15.40:0/1896191229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbdc100560 0x7ffbdc19cb50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400ada0 tx=0x7ffbd400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.462+0000 7ffbd8ff9700 1 -- 172.21.15.40:0/1896191229 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd400f070 con 0x7ffbdc100560 2023-07-18T03:30:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.462+0000 7ffbd8ff9700 1 -- 172.21.15.40:0/1896191229 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbd4007b30 con 0x7ffbdc100560 2023-07-18T03:30:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.462+0000 7ffbe220e700 1 -- 172.21.15.40:0/1896191229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbdc19d900 con 0x7ffbdc100560 2023-07-18T03:30:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.462+0000 7ffbd8ff9700 1 -- 172.21.15.40:0/1896191229 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4005440 con 0x7ffbdc100560 2023-07-18T03:30:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.462+0000 7ffbe220e700 1 -- 172.21.15.40:0/1896191229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbdc19ddc0 con 0x7ffbdc100560 2023-07-18T03:30:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.464+0000 7ffbd8ff9700 1 -- 172.21.15.40:0/1896191229 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ffbd4004ad0 con 0x7ffbdc100560 2023-07-18T03:30:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.465+0000 7ffbd8ff9700 1 --2- 172.21.15.40:0/1896191229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ffbc80a11e0 0x7ffbc80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.465+0000 7ffbdb7fe700 1 --2- 172.21.15.40:0/1896191229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ffbc80a11e0 0x7ffbc80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.465+0000 7ffbd8ff9700 1 -- 172.21.15.40:0/1896191229 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ffbd40cc1a0 con 0x7ffbdc100560 2023-07-18T03:30:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.465+0000 7ffbdb7fe700 1 --2- 172.21.15.40:0/1896191229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ffbc80a11e0 0x7ffbc80a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc005fd0 tx=0x7ffbcc005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.465+0000 7ffbee394b80 1 Processor -- start 2023-07-18T03:30:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.465+0000 7ffbee394b80 1 -- start start 2023-07-18T03:30:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbee394b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6480 0x564b920e6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbee394b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e6dd0 0x564b920e7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbee394b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e7c20 0x564b920e8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbee394b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b91febc60 con 0x564b920e6480 2023-07-18T03:30:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbee394b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9203e9f0 con 0x564b920e7c20 2023-07-18T03:30:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbee394b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b92037e80 con 0x564b920e6dd0 2023-07-18T03:30:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbdb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6480 0x564b920e6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbdb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6480 0x564b920e6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57478/0 (socket says 172.21.15.40:57478) 2023-07-18T03:30:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbdb7fe700 1 -- 172.21.15.40:0/2424388658 learned_addr learned my addr 172.21.15.40:0/2424388658 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbdaffd700 1 --2- 172.21.15.40:0/2424388658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e6dd0 0x564b920e7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbdbfff700 1 --2- 172.21.15.40:0/2424388658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e7c20 0x564b920e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbdb7fe700 1 -- 172.21.15.40:0/2424388658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e6dd0 msgr2=0x564b920e7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbdb7fe700 1 --2- 172.21.15.40:0/2424388658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e6dd0 0x564b920e7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbdb7fe700 1 -- 172.21.15.40:0/2424388658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e7c20 msgr2=0x564b920e8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbdb7fe700 1 --2- 172.21.15.40:0/2424388658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e7c20 0x564b920e8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbdb7fe700 1 -- 172.21.15.40:0/2424388658 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b91e65d90 con 0x564b920e6480 2023-07-18T03:30:39.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbdb7fe700 1 --2- 172.21.15.40:0/2424388658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6480 0x564b920e6890 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00ecd0 tx=0x7ffbcc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1b190ab1a962e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbd1ffb700 1 -- 172.21.15.40:0/2424388658 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc010d80 con 0x564b920e6480 2023-07-18T03:30:39.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbd1ffb700 1 -- 172.21.15.40:0/2424388658 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbcc01d910 con 0x564b920e6480 2023-07-18T03:30:39.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.466+0000 7ffbd1ffb700 1 -- 172.21.15.40:0/2424388658 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc00f900 con 0x564b920e6480 2023-07-18T03:30:39.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.467+0000 7ffbee394b80 1 -- 172.21.15.40:0/2424388658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6480 msgr2=0x564b920e6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.467+0000 7ffbee394b80 1 --2- 172.21.15.40:0/2424388658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6480 0x564b920e6890 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00ecd0 tx=0x7ffbcc011000 comp rx=0 tx=0).stop 2023-07-18T03:30:39.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.467+0000 7ffbee394b80 1 -- 172.21.15.40:0/2424388658 shutdown_connections 2023-07-18T03:30:39.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.467+0000 7ffbee394b80 1 --2- 172.21.15.40:0/2424388658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e7c20 0x564b920e8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.467+0000 7ffbee394b80 1 --2- 172.21.15.40:0/2424388658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6480 0x564b920e6890 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.467+0000 7ffbee394b80 1 --2- 172.21.15.40:0/2424388658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e6dd0 0x564b920e7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.467+0000 7ffbee394b80 1 -- 172.21.15.40:0/2424388658 >> 172.21.15.40:0/2424388658 conn(0x564b920dbd50 msgr2=0x564b920e1180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.467+0000 7ffbee394b80 1 -- 172.21.15.40:0/2424388658 shutdown_connections 2023-07-18T03:30:39.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.467+0000 7ffbee394b80 1 -- 172.21.15.40:0/2424388658 wait complete. 2023-07-18T03:30:39.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.467+0000 7ffbee394b80 1 -- start start 2023-07-18T03:30:39.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbee394b80 1 Processor -- start 2023-07-18T03:30:39.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbee394b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e6480 0x564b920f0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbee394b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6dd0 0x564b920f0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbee394b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e7c20 0x564b920f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbee394b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b91fd7a70 con 0x564b920e6dd0 2023-07-18T03:30:39.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbee394b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b91fd7770 con 0x564b920e6480 2023-07-18T03:30:39.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbee394b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b91ebb8d0 con 0x564b920e7c20 2023-07-18T03:30:39.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbee394b80 10 client.0 _ll_register_callbacks cb 0x564b91fd5710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:39.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbdaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6dd0 0x564b920f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbdaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6dd0 0x564b920f0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57486/0 (socket says 172.21.15.40:57486) 2023-07-18T03:30:39.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbdaffd700 1 -- 172.21.15.40:0/1021342601 learned_addr learned my addr 172.21.15.40:0/1021342601 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbdbfff700 1 --2- 172.21.15.40:0/1021342601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e7c20 0x564b920f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbdb7fe700 1 --2- 172.21.15.40:0/1021342601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e6480 0x564b920f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbdaffd700 1 -- 172.21.15.40:0/1021342601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e7c20 msgr2=0x564b920f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbdaffd700 1 --2- 172.21.15.40:0/1021342601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e7c20 0x564b920f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbdaffd700 1 -- 172.21.15.40:0/1021342601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e6480 msgr2=0x564b920f0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbdaffd700 1 --2- 172.21.15.40:0/1021342601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e6480 0x564b920f0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbdaffd700 1 -- 172.21.15.40:0/1021342601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbcc00e980 con 0x564b920e6dd0 2023-07-18T03:30:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbdb7fe700 1 --2- 172.21.15.40:0/1021342601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e6480 0x564b920f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.468+0000 7ffbdbfff700 1 --2- 172.21.15.40:0/1021342601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e7c20 0x564b920f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.469+0000 7ffbdaffd700 1 --2- 172.21.15.40:0/1021342601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6dd0 0x564b920f0660 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40057f0 tx=0x7ffbc400b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b531ee0ced628d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.469+0000 7ffbbaffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.469+0000 7ffbbaffd700 1 -- 172.21.15.40:0/1021342601 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc400bdd0 con 0x564b920e6dd0 2023-07-18T03:30:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.469+0000 7ffbbaffd700 1 -- 172.21.15.40:0/1021342601 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ffbc4004500 con 0x564b920e6dd0 2023-07-18T03:30:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.469+0000 7ffbee394b80 1 -- 172.21.15.40:0/1021342601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b91f914a0 con 0x564b920e6dd0 2023-07-18T03:30:39.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.469+0000 7ffbbaffd700 1 -- 172.21.15.40:0/1021342601 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbc40046c0 con 0x564b920e6dd0 2023-07-18T03:30:39.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.469+0000 7ffbee394b80 10 client.15258 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:39.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.469+0000 7ffbee394b80 1 -- 172.21.15.40:0/1021342601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564b91e4ec10 con 0x564b920e6dd0 2023-07-18T03:30:39.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.469+0000 7ffbbaffd700 1 -- 172.21.15.40:0/1021342601 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbc4031020 con 0x564b920e6dd0 2023-07-18T03:30:39.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.469+0000 7ffbbaffd700 1 -- 172.21.15.40:0/1021342601 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ffbc4003e40 con 0x564b920e6dd0 2023-07-18T03:30:39.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.469+0000 7ffbee394b80 10 client.15258 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:39.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.469+0000 7ffbee394b80 10 client.15258 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-18T03:30:39.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.469+0000 7ffbee394b80 20 client.15258 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:39.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.469+0000 7ffbee394b80 1 --2- 172.21.15.40:0/1021342601 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x564b920fb410 0x564b920fa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.469+0000 7ffbee394b80 4 client.15258 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:30:39.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.469+0000 7ffbee394b80 1 -- 172.21.15.40:0/1021342601 --> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b92030e60 con 0x564b920fb410 2023-07-18T03:30:39.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.470+0000 7ffbdb7fe700 1 --2- 172.21.15.40:0/1021342601 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x564b920fb410 0x564b920fa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.470+0000 7ffbdb7fe700 1 --2- 172.21.15.40:0/1021342601 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x564b920fb410 0x564b920fa7f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a828fd9e15a3e35a server_cookie=161e870083e89ffc in_seq=0 out_seq=0 2023-07-18T03:30:39.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.470+0000 7ffbbaffd700 10 client.15258 ms_handle_connect on v2:172.21.15.64:6836/3077792296 2023-07-18T03:30:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.471+0000 7ffbbaffd700 1 -- 172.21.15.40:0/1021342601 <== mds.2 v2:172.21.15.64:6836/3077792296 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564b92030e60 con 0x564b920fb410 2023-07-18T03:30:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.471+0000 7ffbbaffd700 10 client.15258 handle_command_reply: tid=0 2023-07-18T03:30:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.471+0000 7ffbee394b80 1 client.15258 shutdown 2023-07-18T03:30:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.471+0000 7ffbee394b80 1 -- 172.21.15.40:0/1021342601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6dd0 msgr2=0x564b920f0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.471+0000 7ffbee394b80 1 --2- 172.21.15.40:0/1021342601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6dd0 0x564b920f0660 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40057f0 tx=0x7ffbc400b770 comp rx=0 tx=0).stop 2023-07-18T03:30:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.471+0000 7ffbee394b80 1 -- 172.21.15.40:0/1021342601 shutdown_connections 2023-07-18T03:30:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.471+0000 7ffbee394b80 1 --2- 172.21.15.40:0/1021342601 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x564b920fb410 0x564b920fa7f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.471+0000 7ffbee394b80 1 --2- 172.21.15.40:0/1021342601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e7c20 0x564b920f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.471+0000 7ffbbaffd700 0 client.15258 ms_handle_reset on v2:172.21.15.64:6836/3077792296 2023-07-18T03:30:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.471+0000 7ffbee394b80 1 --2- 172.21.15.40:0/1021342601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e6480 0x564b920f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.471+0000 7ffbee394b80 1 --2- 172.21.15.40:0/1021342601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6dd0 0x564b920f0660 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.471+0000 7ffbee394b80 1 -- 172.21.15.40:0/1021342601 >> 172.21.15.40:0/1021342601 conn(0x564b920dbd50 msgr2=0x564b920e0860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.471+0000 7ffbee394b80 1 -- 172.21.15.40:0/1021342601 shutdown_connections 2023-07-18T03:30:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.471+0000 7ffbee394b80 1 -- 172.21.15.40:0/1021342601 wait complete. 2023-07-18T03:30:39.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.471+0000 7ffbee394b80 20 client.15258 trim_cache size 0 max 16384 2023-07-18T03:30:39.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbee394b80 1 Processor -- start 2023-07-18T03:30:39.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbee394b80 1 -- start start 2023-07-18T03:30:39.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbee394b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6480 0x564b920eee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbee394b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e6dd0 0x564b920f7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbee394b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e7c20 0x564b920e0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbee394b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b920f1310 con 0x564b920e6480 2023-07-18T03:30:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbee394b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b92006790 con 0x564b920e7c20 2023-07-18T03:30:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbee394b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b91ec7de0 con 0x564b920e6dd0 2023-07-18T03:30:39.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbdb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6480 0x564b920eee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbdb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6480 0x564b920eee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57492/0 (socket says 172.21.15.40:57492) 2023-07-18T03:30:39.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbdb7fe700 1 -- 172.21.15.40:0/1789503062 learned_addr learned my addr 172.21.15.40:0/1789503062 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbdbfff700 1 --2- 172.21.15.40:0/1789503062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e7c20 0x564b920e0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbdb7fe700 1 -- 172.21.15.40:0/1789503062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e6dd0 msgr2=0x564b920f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbdb7fe700 1 --2- 172.21.15.40:0/1789503062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e6dd0 0x564b920f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbdb7fe700 1 -- 172.21.15.40:0/1789503062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e7c20 msgr2=0x564b920e0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbdb7fe700 1 --2- 172.21.15.40:0/1789503062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e7c20 0x564b920e0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbdb7fe700 1 -- 172.21.15.40:0/1789503062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b91e8c200 con 0x564b920e6480 2023-07-18T03:30:39.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.489+0000 7ffbdb7fe700 1 --2- 172.21.15.40:0/1789503062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6480 0x564b920eee00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0107c0 tx=0x7ffbcc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a0135f7431de27 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.490+0000 7ffbd27fc700 1 -- 172.21.15.40:0/1789503062 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0325c0 con 0x564b920e6480 2023-07-18T03:30:39.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.490+0000 7ffbd27fc700 1 -- 172.21.15.40:0/1789503062 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbcc02d040 con 0x564b920e6480 2023-07-18T03:30:39.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.490+0000 7ffbd27fc700 1 -- 172.21.15.40:0/1789503062 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc032ce0 con 0x564b920e6480 2023-07-18T03:30:39.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.490+0000 7ffbee394b80 1 -- 172.21.15.40:0/1789503062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6480 msgr2=0x564b920eee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.490+0000 7ffbee394b80 1 --2- 172.21.15.40:0/1789503062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6480 0x564b920eee00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0107c0 tx=0x7ffbcc014000 comp rx=0 tx=0).stop 2023-07-18T03:30:39.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.490+0000 7ffbee394b80 1 -- 172.21.15.40:0/1789503062 shutdown_connections 2023-07-18T03:30:39.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.490+0000 7ffbee394b80 1 --2- 172.21.15.40:0/1789503062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e7c20 0x564b920e0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.490+0000 7ffbee394b80 1 --2- 172.21.15.40:0/1789503062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6480 0x564b920eee00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.490+0000 7ffbee394b80 1 --2- 172.21.15.40:0/1789503062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e6dd0 0x564b920f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.490+0000 7ffbee394b80 1 -- 172.21.15.40:0/1789503062 >> 172.21.15.40:0/1789503062 conn(0x564b920dbd50 msgr2=0x564b92104160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.490+0000 7ffbee394b80 1 -- 172.21.15.40:0/1789503062 shutdown_connections 2023-07-18T03:30:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.490+0000 7ffbee394b80 1 -- 172.21.15.40:0/1789503062 wait complete. 2023-07-18T03:30:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbee394b80 1 -- start start 2023-07-18T03:30:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbee394b80 1 Processor -- start 2023-07-18T03:30:39.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbee394b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e6480 0x564b920ed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbee394b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6dd0 0x564b920f7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbee394b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e7c20 0x564b9214f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbee394b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9214f860 con 0x564b920e6dd0 2023-07-18T03:30:39.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbee394b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9214f9d0 con 0x564b920e6480 2023-07-18T03:30:39.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbee394b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9214fb90 con 0x564b920e7c20 2023-07-18T03:30:39.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbee394b80 10 client.0 _ll_register_callbacks cb 0x7ffbc4009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:39.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbdaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6dd0 0x564b920f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbdaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6dd0 0x564b920f7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57500/0 (socket says 172.21.15.40:57500) 2023-07-18T03:30:39.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbdaffd700 1 -- 172.21.15.40:0/2620569750 learned_addr learned my addr 172.21.15.40:0/2620569750 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbdb7fe700 1 --2- 172.21.15.40:0/2620569750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e6480 0x564b920ed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbdbfff700 1 --2- 172.21.15.40:0/2620569750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e7c20 0x564b9214f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbdaffd700 1 -- 172.21.15.40:0/2620569750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e7c20 msgr2=0x564b9214f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbdaffd700 1 --2- 172.21.15.40:0/2620569750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e7c20 0x564b9214f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbdaffd700 1 -- 172.21.15.40:0/2620569750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e6480 msgr2=0x564b920ed350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbdaffd700 1 --2- 172.21.15.40:0/2620569750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e6480 0x564b920ed350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbdaffd700 1 -- 172.21.15.40:0/2620569750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b91e65d90 con 0x564b920e6dd0 2023-07-18T03:30:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.491+0000 7ffbdb7fe700 1 --2- 172.21.15.40:0/2620569750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e6480 0x564b920ed350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.492+0000 7ffbdaffd700 1 --2- 172.21.15.40:0/2620569750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6dd0 0x564b920f7630 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40057c0 tx=0x7ffbc400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e917eee3a02d397e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.492+0000 7ffbbaffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:39.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.492+0000 7ffbbaffd700 1 -- 172.21.15.40:0/2620569750 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc402cbd0 con 0x564b920e6dd0 2023-07-18T03:30:39.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.492+0000 7ffbbaffd700 1 -- 172.21.15.40:0/2620569750 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ffbc4032080 con 0x564b920e6dd0 2023-07-18T03:30:39.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.492+0000 7ffbee394b80 1 -- 172.21.15.40:0/2620569750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffbcc01d910 con 0x564b920e6dd0 2023-07-18T03:30:39.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.492+0000 7ffbbaffd700 1 -- 172.21.15.40:0/2620569750 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbc400f580 con 0x564b920e6dd0 2023-07-18T03:30:39.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.492+0000 7ffbee394b80 10 client.15264 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:39.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.492+0000 7ffbee394b80 1 -- 172.21.15.40:0/2620569750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564b91e64260 con 0x564b920e6dd0 2023-07-18T03:30:39.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.492+0000 7ffbbaffd700 1 -- 172.21.15.40:0/2620569750 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbcc01d910 con 0x564b920e6dd0 2023-07-18T03:30:39.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.492+0000 7ffbbaffd700 1 -- 172.21.15.40:0/2620569750 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ffbc403e020 con 0x564b920e6dd0 2023-07-18T03:30:39.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.492+0000 7ffbee394b80 10 client.15264 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:39.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.492+0000 7ffbee394b80 10 client.15264 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-18T03:30:39.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.492+0000 7ffbee394b80 20 client.15264 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:39.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.492+0000 7ffbee394b80 1 --2- 172.21.15.40:0/2620569750 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x564b921544b0 0x564b92150e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.492+0000 7ffbee394b80 4 client.15264 mds_command: new command op to 24482 tid=0[{"prefix": "status"}] 2023-07-18T03:30:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.492+0000 7ffbee394b80 1 -- 172.21.15.40:0/2620569750 --> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564b91ff3000 con 0x564b921544b0 2023-07-18T03:30:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.493+0000 7ffbdb7fe700 1 --2- 172.21.15.40:0/2620569750 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x564b921544b0 0x564b92150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.493+0000 7ffbdb7fe700 1 --2- 172.21.15.40:0/2620569750 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x564b921544b0 0x564b92150e40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=284c25b71d503888 server_cookie=f502be95ba55a6d1 in_seq=0 out_seq=0 2023-07-18T03:30:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.493+0000 7ffbbaffd700 10 client.15264 ms_handle_connect on v2:172.21.15.64:6836/3077792296 2023-07-18T03:30:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.493+0000 7ffbbaffd700 1 -- 172.21.15.40:0/2620569750 <== mds.2 v2:172.21.15.64:6836/3077792296 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x564b91ff3000 con 0x564b921544b0 2023-07-18T03:30:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.493+0000 7ffbbaffd700 10 client.15264 handle_command_reply: tid=0 2023-07-18T03:30:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.493+0000 7ffbee394b80 1 client.15264 shutdown 2023-07-18T03:30:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.493+0000 7ffbee394b80 1 -- 172.21.15.40:0/2620569750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6dd0 msgr2=0x564b920f7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.493+0000 7ffbee394b80 1 --2- 172.21.15.40:0/2620569750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6dd0 0x564b920f7630 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40057c0 tx=0x7ffbc400bf10 comp rx=0 tx=0).stop 2023-07-18T03:30:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.493+0000 7ffbee394b80 1 -- 172.21.15.40:0/2620569750 shutdown_connections 2023-07-18T03:30:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.493+0000 7ffbee394b80 1 --2- 172.21.15.40:0/2620569750 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x564b921544b0 0x564b92150e40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.493+0000 7ffbee394b80 1 --2- 172.21.15.40:0/2620569750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564b920e7c20 0x564b9214f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.493+0000 7ffbee394b80 1 --2- 172.21.15.40:0/2620569750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b920e6480 0x564b920ed350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.493+0000 7ffbee394b80 1 --2- 172.21.15.40:0/2620569750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564b920e6dd0 0x564b920f7630 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.493+0000 7ffbbaffd700 0 client.15264 ms_handle_reset on v2:172.21.15.64:6836/3077792296 2023-07-18T03:30:39.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.493+0000 7ffbee394b80 1 -- 172.21.15.40:0/2620569750 >> 172.21.15.40:0/2620569750 conn(0x564b920dbd50 msgr2=0x564b920eee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.493+0000 7ffbee394b80 1 -- 172.21.15.40:0/2620569750 shutdown_connections 2023-07-18T03:30:39.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.493+0000 7ffbee394b80 1 -- 172.21.15.40:0/2620569750 wait complete. 2023-07-18T03:30:39.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.494+0000 7ffbee394b80 20 client.15264 trim_cache size 0 max 16384 2023-07-18T03:30:39.503 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:30:39.503 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "3af3dba8-251a-11ee-9b34-001a4aab830c", 2023-07-18T03:30:39.503 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 2, 2023-07-18T03:30:39.503 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24482, 2023-07-18T03:30:39.503 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-07-18T03:30:39.503 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-07-18T03:30:39.503 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-07-18T03:30:39.504 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 67.896113428999996, 2023-07-18T03:30:39.504 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 27, 2023-07-18T03:30:39.504 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 83, 2023-07-18T03:30:39.504 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 81, 2023-07-18T03:30:39.504 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 87.674761355000001 2023-07-18T03:30:39.504 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:30:39.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.494+0000 7ffbe220e700 1 -- 172.21.15.40:0/1896191229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ffbc80a11e0 msgr2=0x7ffbc80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.494+0000 7ffbe220e700 1 --2- 172.21.15.40:0/1896191229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ffbc80a11e0 0x7ffbc80a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc005fd0 tx=0x7ffbcc005e50 comp rx=0 tx=0).stop 2023-07-18T03:30:39.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.494+0000 7ffbe220e700 1 -- 172.21.15.40:0/1896191229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbdc100560 msgr2=0x7ffbdc19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.494+0000 7ffbe220e700 1 --2- 172.21.15.40:0/1896191229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbdc100560 0x7ffbdc19cb50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400ada0 tx=0x7ffbd400c330 comp rx=0 tx=0).stop 2023-07-18T03:30:39.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.494+0000 7ffbe220e700 1 -- 172.21.15.40:0/1896191229 shutdown_connections 2023-07-18T03:30:39.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.494+0000 7ffbe220e700 1 --2- 172.21.15.40:0/1896191229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbdc100560 0x7ffbdc19cb50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.494+0000 7ffbe220e700 1 --2- 172.21.15.40:0/1896191229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ffbc80a11e0 0x7ffbc80a3690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.494+0000 7ffbe220e700 1 --2- 172.21.15.40:0/1896191229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbdc0fe9e0 0x7ffbdc068240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.494+0000 7ffbe220e700 1 --2- 172.21.15.40:0/1896191229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffbdc0ffbd0 0x7ffbdc19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.494+0000 7ffbe220e700 1 -- 172.21.15.40:0/1896191229 >> 172.21.15.40:0/1896191229 conn(0x7ffbdc0fa000 msgr2=0x7ffbdc1030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.494+0000 7ffbe220e700 1 -- 172.21.15.40:0/1896191229 shutdown_connections 2023-07-18T03:30:39.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.494+0000 7ffbe220e700 1 -- 172.21.15.40:0/1896191229 wait complete. 2023-07-18T03:30:39.508 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-18T03:30:39.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.638+0000 7f73c4dc6700 1 -- 172.21.15.40:0/1522562456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73c0100560 msgr2=0x7f73c0106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.638+0000 7f73c4dc6700 1 --2- 172.21.15.40:0/1522562456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73c0100560 0x7f73c0106e20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f73b800b0a0 tx=0x7f73b800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.638+0000 7f73c4dc6700 1 -- 172.21.15.40:0/1522562456 shutdown_connections 2023-07-18T03:30:39.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.638+0000 7f73c4dc6700 1 --2- 172.21.15.40:0/1522562456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73c0100560 0x7f73c0106e20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.638+0000 7f73c4dc6700 1 --2- 172.21.15.40:0/1522562456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73c00fe9e0 0x7f73c00fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.638+0000 7f73c4dc6700 1 --2- 172.21.15.40:0/1522562456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73c00ffbd0 0x7f73c0100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.638+0000 7f73c4dc6700 1 -- 172.21.15.40:0/1522562456 >> 172.21.15.40:0/1522562456 conn(0x7f73c00fa000 msgr2=0x7f73c00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.638+0000 7f73c4dc6700 1 -- 172.21.15.40:0/1522562456 shutdown_connections 2023-07-18T03:30:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.638+0000 7f73c4dc6700 1 -- 172.21.15.40:0/1522562456 wait complete. 2023-07-18T03:30:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.639+0000 7f73c4dc6700 1 Processor -- start 2023-07-18T03:30:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.639+0000 7f73c4dc6700 1 -- start start 2023-07-18T03:30:39.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.639+0000 7f73c4dc6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73c00fe9e0 0x7f73c0069210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.639+0000 7f73c4dc6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73c00ffbd0 0x7f73c019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.639+0000 7f73c4dc6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73c0100560 0x7f73c019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.639+0000 7f73c4dc6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c019d3a0 con 0x7f73c0100560 2023-07-18T03:30:39.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.639+0000 7f73c4dc6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c019d510 con 0x7f73c00ffbd0 2023-07-18T03:30:39.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.639+0000 7f73c4dc6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c019d6d0 con 0x7f73c00fe9e0 2023-07-18T03:30:39.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.640+0000 7f73bed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73c0100560 0x7f73c019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.640+0000 7f73bed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73c0100560 0x7f73c019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57514/0 (socket says 172.21.15.40:57514) 2023-07-18T03:30:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.640+0000 7f73bed9d700 1 -- 172.21.15.40:0/3375603520 learned_addr learned my addr 172.21.15.40:0/3375603520 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.640+0000 7f73bdd9b700 1 --2- 172.21.15.40:0/3375603520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73c00ffbd0 0x7f73c019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.640+0000 7f73be59c700 1 --2- 172.21.15.40:0/3375603520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73c00fe9e0 0x7f73c0069210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.640+0000 7f73bed9d700 1 -- 172.21.15.40:0/3375603520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73c00fe9e0 msgr2=0x7f73c0069210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.640+0000 7f73bed9d700 1 --2- 172.21.15.40:0/3375603520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73c00fe9e0 0x7f73c0069210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.640+0000 7f73bed9d700 1 -- 172.21.15.40:0/3375603520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73c00ffbd0 msgr2=0x7f73c019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.640+0000 7f73bed9d700 1 --2- 172.21.15.40:0/3375603520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73c00ffbd0 0x7f73c019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.640+0000 7f73bed9d700 1 -- 172.21.15.40:0/3375603520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b8009d00 con 0x7f73c0100560 2023-07-18T03:30:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.640+0000 7f73bed9d700 1 --2- 172.21.15.40:0/3375603520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73c0100560 0x7f73c019cc00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f73b800b830 tx=0x7f73b8004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.640+0000 7f73b77fe700 1 -- 172.21.15.40:0/3375603520 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b800e040 con 0x7f73c0100560 2023-07-18T03:30:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.640+0000 7f73b77fe700 1 -- 172.21.15.40:0/3375603520 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73b801db50 con 0x7f73c0100560 2023-07-18T03:30:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.640+0000 7f73c4dc6700 1 -- 172.21.15.40:0/3375603520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c019d950 con 0x7f73c0100560 2023-07-18T03:30:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.640+0000 7f73b77fe700 1 -- 172.21.15.40:0/3375603520 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b801bbd0 con 0x7f73c0100560 2023-07-18T03:30:39.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.640+0000 7f73c4dc6700 1 -- 172.21.15.40:0/3375603520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c019ddd0 con 0x7f73c0100560 2023-07-18T03:30:39.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.642+0000 7f73b77fe700 1 -- 172.21.15.40:0/3375603520 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f73b801dcc0 con 0x7f73c0100560 2023-07-18T03:30:39.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.643+0000 7f73b77fe700 1 --2- 172.21.15.40:0/3375603520 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f73ac0a11e0 0x7f73ac0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.643+0000 7f73b77fe700 1 -- 172.21.15.40:0/3375603520 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f73b80ce770 con 0x7f73c0100560 2023-07-18T03:30:39.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.643+0000 7f73be59c700 1 --2- 172.21.15.40:0/3375603520 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f73ac0a11e0 0x7f73ac0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.643+0000 7f73be59c700 1 --2- 172.21.15.40:0/3375603520 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f73ac0a11e0 0x7f73ac0a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f73c00ffa30 tx=0x7f73b000b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.644+0000 7f73d0f4cb80 1 Processor -- start 2023-07-18T03:30:39.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.644+0000 7f73d0f4cb80 1 -- start start 2023-07-18T03:30:39.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.644+0000 7f73d0f4cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.644+0000 7f73d0f4cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 0x56040a9c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.644+0000 7f73d0f4cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c8030 0x56040a9c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.644+0000 7f73d0f4cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040a8c7c60 con 0x56040a9c6890 2023-07-18T03:30:39.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.644+0000 7f73d0f4cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040a91a9f0 con 0x56040a9c71e0 2023-07-18T03:30:39.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.644+0000 7f73d0f4cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040a913e80 con 0x56040a9c8030 2023-07-18T03:30:39.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.644+0000 7f73be59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.644+0000 7f73be59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57518/0 (socket says 172.21.15.40:57518) 2023-07-18T03:30:39.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.644+0000 7f73be59c700 1 -- 172.21.15.40:0/1156742036 learned_addr learned my addr 172.21.15.40:0/1156742036 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.644+0000 7f73bed9d700 1 --2- 172.21.15.40:0/1156742036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c8030 0x56040a9c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.645+0000 7f73bdd9b700 1 --2- 172.21.15.40:0/1156742036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 0x56040a9c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.645+0000 7f73be59c700 1 -- 172.21.15.40:0/1156742036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c8030 msgr2=0x56040a9c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.645+0000 7f73be59c700 1 --2- 172.21.15.40:0/1156742036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c8030 0x56040a9c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.645+0000 7f73be59c700 1 -- 172.21.15.40:0/1156742036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 msgr2=0x56040a9c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.645+0000 7f73be59c700 1 --2- 172.21.15.40:0/1156742036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 0x56040a9c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.645+0000 7f73be59c700 1 -- 172.21.15.40:0/1156742036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56040a741d90 con 0x56040a9c6890 2023-07-18T03:30:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.645+0000 7f73be59c700 1 --2- 172.21.15.40:0/1156742036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9c6ca0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f73b000fcf0 tx=0x7f73b0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fea5e5948216bb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.645+0000 7f73b4ff9700 1 -- 172.21.15.40:0/1156742036 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0011ad0 con 0x56040a9c6890 2023-07-18T03:30:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.645+0000 7f73b4ff9700 1 -- 172.21.15.40:0/1156742036 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73b0011c30 con 0x56040a9c6890 2023-07-18T03:30:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.645+0000 7f73b4ff9700 1 -- 172.21.15.40:0/1156742036 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b001d610 con 0x56040a9c6890 2023-07-18T03:30:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.646+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/1156742036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 msgr2=0x56040a9c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.646+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/1156742036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9c6ca0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f73b000fcf0 tx=0x7f73b0012000 comp rx=0 tx=0).stop 2023-07-18T03:30:39.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.646+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/1156742036 shutdown_connections 2023-07-18T03:30:39.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.646+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/1156742036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c8030 0x56040a9c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.646+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/1156742036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9c6ca0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.646+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/1156742036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 0x56040a9c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.646+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/1156742036 >> 172.21.15.40:0/1156742036 conn(0x56040a9b7d50 msgr2=0x56040a9bd180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.646+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/1156742036 shutdown_connections 2023-07-18T03:30:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.646+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/1156742036 wait complete. 2023-07-18T03:30:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.646+0000 7f73d0f4cb80 1 -- start start 2023-07-18T03:30:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.646+0000 7f73d0f4cb80 1 Processor -- start 2023-07-18T03:30:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.646+0000 7f73d0f4cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.646+0000 7f73d0f4cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 0x56040a9d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73d0f4cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c8030 0x56040a9d7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73d0f4cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040a8b3a70 con 0x56040a9c6890 2023-07-18T03:30:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73d0f4cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040a8b3770 con 0x56040a9c71e0 2023-07-18T03:30:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73d0f4cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040a7978d0 con 0x56040a9c8030 2023-07-18T03:30:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73d0f4cb80 10 client.0 _ll_register_callbacks cb 0x56040a8b1710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73be59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73be59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9d0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57524/0 (socket says 172.21.15.40:57524) 2023-07-18T03:30:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73be59c700 1 -- 172.21.15.40:0/493586691 learned_addr learned my addr 172.21.15.40:0/493586691 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73bdd9b700 1 --2- 172.21.15.40:0/493586691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 0x56040a9d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73bed9d700 1 --2- 172.21.15.40:0/493586691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c8030 0x56040a9d7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73be59c700 1 -- 172.21.15.40:0/493586691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c8030 msgr2=0x56040a9d7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73be59c700 1 --2- 172.21.15.40:0/493586691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c8030 0x56040a9d7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73be59c700 1 -- 172.21.15.40:0/493586691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 msgr2=0x56040a9d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73be59c700 1 --2- 172.21.15.40:0/493586691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 0x56040a9d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73be59c700 1 -- 172.21.15.40:0/493586691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b000f9a0 con 0x56040a9c6890 2023-07-18T03:30:39.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73be59c700 1 --2- 172.21.15.40:0/493586691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9d0500 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f73b000f940 tx=0x7f73b00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea36657e09ac92cd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:39.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73a5ffb700 1 -- 172.21.15.40:0/493586691 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b002d070 con 0x56040a9c6890 2023-07-18T03:30:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73a5ffb700 1 -- 172.21.15.40:0/493586691 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f73b001e720 con 0x56040a9c6890 2023-07-18T03:30:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/493586691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56040a86d4a0 con 0x56040a9c6890 2023-07-18T03:30:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73a5ffb700 1 -- 172.21.15.40:0/493586691 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73b001eea0 con 0x56040a9c6890 2023-07-18T03:30:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73d0f4cb80 10 client.15276 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.647+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/493586691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56040a72ac10 con 0x56040a9c6890 2023-07-18T03:30:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.648+0000 7f73a5ffb700 1 -- 172.21.15.40:0/493586691 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73b0041080 con 0x56040a9c6890 2023-07-18T03:30:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.648+0000 7f73a5ffb700 1 -- 172.21.15.40:0/493586691 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f73b003b030 con 0x56040a9c6890 2023-07-18T03:30:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.648+0000 7f73d0f4cb80 10 client.15276 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.648+0000 7f73d0f4cb80 10 client.15276 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-18T03:30:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.648+0000 7f73d0f4cb80 20 client.15276 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.648+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/493586691 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x56040a9db6f0 0x56040a9dab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.648+0000 7f73d0f4cb80 4 client.15276 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:30:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.648+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/493586691 --> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56040a90ce60 con 0x56040a9db6f0 2023-07-18T03:30:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.648+0000 7f73bdd9b700 1 --2- 172.21.15.40:0/493586691 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x56040a9db6f0 0x56040a9dab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.649+0000 7f73a5ffb700 10 client.15276 ms_handle_connect on v2:172.21.15.134:6838/3192254101 2023-07-18T03:30:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.649+0000 7f73bdd9b700 1 --2- 172.21.15.40:0/493586691 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x56040a9db6f0 0x56040a9dab20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2dfd0dc6c93382ff server_cookie=f61510d9e9ba3b92 in_seq=0 out_seq=0 2023-07-18T03:30:39.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.649+0000 7f73a5ffb700 1 -- 172.21.15.40:0/493586691 <== mds.1 v2:172.21.15.134:6838/3192254101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56040a90ce60 con 0x56040a9db6f0 2023-07-18T03:30:39.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.649+0000 7f73a5ffb700 10 client.15276 handle_command_reply: tid=0 2023-07-18T03:30:39.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.649+0000 7f73d0f4cb80 1 client.15276 shutdown 2023-07-18T03:30:39.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.649+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/493586691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 msgr2=0x56040a9d0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.649+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/493586691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9d0500 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f73b000f940 tx=0x7f73b00118d0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.649+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/493586691 shutdown_connections 2023-07-18T03:30:39.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.649+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/493586691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c8030 0x56040a9d7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.649+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/493586691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9d0500 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.649+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/493586691 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x56040a9db6f0 0x56040a9dab20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.649+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/493586691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 0x56040a9d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.650+0000 7f73a5ffb700 0 client.15276 ms_handle_reset on v2:172.21.15.134:6838/3192254101 2023-07-18T03:30:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.650+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/493586691 >> 172.21.15.40:0/493586691 conn(0x56040a9b7d50 msgr2=0x56040a9bc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.650+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/493586691 shutdown_connections 2023-07-18T03:30:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.650+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/493586691 wait complete. 2023-07-18T03:30:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.650+0000 7f73d0f4cb80 20 client.15276 trim_cache size 0 max 16384 2023-07-18T03:30:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.667+0000 7f73d0f4cb80 1 Processor -- start 2023-07-18T03:30:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.667+0000 7f73d0f4cb80 1 -- start start 2023-07-18T03:30:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.667+0000 7f73d0f4cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9dab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.667+0000 7f73d0f4cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 0x56040a9d7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.667+0000 7f73d0f4cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c8030 0x56040a9caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.667+0000 7f73d0f4cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040a9b9e70 con 0x56040a9c6890 2023-07-18T03:30:39.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.667+0000 7f73d0f4cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040a9dbf50 con 0x56040a9c71e0 2023-07-18T03:30:39.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.667+0000 7f73d0f4cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040a9b7920 con 0x56040a9c8030 2023-07-18T03:30:39.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.668+0000 7f73be59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9dab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.668+0000 7f73be59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9dab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57532/0 (socket says 172.21.15.40:57532) 2023-07-18T03:30:39.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.668+0000 7f73be59c700 1 -- 172.21.15.40:0/2694483762 learned_addr learned my addr 172.21.15.40:0/2694483762 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.668+0000 7f73bdd9b700 1 --2- 172.21.15.40:0/2694483762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 0x56040a9d7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.668+0000 7f73be59c700 1 -- 172.21.15.40:0/2694483762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c8030 msgr2=0x56040a9caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.668+0000 7f73be59c700 1 --2- 172.21.15.40:0/2694483762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c8030 0x56040a9caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.668+0000 7f73be59c700 1 -- 172.21.15.40:0/2694483762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 msgr2=0x56040a9d7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.668+0000 7f73be59c700 1 --2- 172.21.15.40:0/2694483762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 0x56040a9d7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.668+0000 7f73be59c700 1 -- 172.21.15.40:0/2694483762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56040a768200 con 0x56040a9c6890 2023-07-18T03:30:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.668+0000 7f73bdd9b700 1 --2- 172.21.15.40:0/2694483762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 0x56040a9d7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.668+0000 7f73be59c700 1 --2- 172.21.15.40:0/2694483762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9dab20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f73b0031000 tx=0x7f73b0005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34ebaca3e5ab537 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.668+0000 7f73b57fa700 1 -- 172.21.15.40:0/2694483762 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0044070 con 0x56040a9c6890 2023-07-18T03:30:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.668+0000 7f73b57fa700 1 -- 172.21.15.40:0/2694483762 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56040a86d4a0 con 0x56040a9c6890 2023-07-18T03:30:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.668+0000 7f73b57fa700 1 -- 172.21.15.40:0/2694483762 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b002d030 con 0x56040a9c6890 2023-07-18T03:30:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/2694483762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 msgr2=0x56040a9dab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/2694483762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9dab20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f73b0031000 tx=0x7f73b0005d00 comp rx=0 tx=0).stop 2023-07-18T03:30:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/2694483762 shutdown_connections 2023-07-18T03:30:39.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/2694483762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c8030 0x56040a9caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/2694483762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c6890 0x56040a9dab20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/2694483762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 0x56040a9d7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/2694483762 >> 172.21.15.40:0/2694483762 conn(0x56040a9b7d50 msgr2=0x56040a9cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/2694483762 shutdown_connections 2023-07-18T03:30:39.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/2694483762 wait complete. 2023-07-18T03:30:39.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 1 -- start start 2023-07-18T03:30:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 1 Processor -- start 2023-07-18T03:30:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c6890 0x56040a9d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 0x56040a9d3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c8030 0x56040aa2f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040a9d3860 con 0x56040a9c8030 2023-07-18T03:30:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040aa2fab0 con 0x56040a9c71e0 2023-07-18T03:30:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040aa2fbf0 con 0x56040a9c6890 2023-07-18T03:30:39.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73d0f4cb80 10 client.0 _ll_register_callbacks cb 0x7f73a800a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:39.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73bed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c8030 0x56040aa2f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73bed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c8030 0x56040aa2f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57542/0 (socket says 172.21.15.40:57542) 2023-07-18T03:30:39.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73bed9d700 1 -- 172.21.15.40:0/3343247042 learned_addr learned my addr 172.21.15.40:0/3343247042 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.669+0000 7f73be59c700 1 --2- 172.21.15.40:0/3343247042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c6890 0x56040a9d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73bed9d700 1 -- 172.21.15.40:0/3343247042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c6890 msgr2=0x56040a9d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73bed9d700 1 --2- 172.21.15.40:0/3343247042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c6890 0x56040a9d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73bed9d700 1 -- 172.21.15.40:0/3343247042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 msgr2=0x56040a9d3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73bed9d700 1 --2- 172.21.15.40:0/3343247042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 0x56040a9d3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73bed9d700 1 -- 172.21.15.40:0/3343247042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56040a72ac10 con 0x56040a9c8030 2023-07-18T03:30:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73bed9d700 1 --2- 172.21.15.40:0/3343247042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c8030 0x56040aa2f570 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f73b80cc130 tx=0x7f73b809b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfe29935ac03835 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73a5ffb700 1 -- 172.21.15.40:0/3343247042 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b80a3e50 con 0x56040a9c8030 2023-07-18T03:30:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/3343247042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73b0011c30 con 0x56040a9c8030 2023-07-18T03:30:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73a5ffb700 1 -- 172.21.15.40:0/3343247042 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f73b80cc350 con 0x56040a9c8030 2023-07-18T03:30:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73a5ffb700 1 -- 172.21.15.40:0/3343247042 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73b801bd30 con 0x56040a9c8030 2023-07-18T03:30:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73d0f4cb80 10 client.15282 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/3343247042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56040a740260 con 0x56040a9c8030 2023-07-18T03:30:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73a5ffb700 1 -- 172.21.15.40:0/3343247042 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73b0011c30 con 0x56040a9c8030 2023-07-18T03:30:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73a5ffb700 1 -- 172.21.15.40:0/3343247042 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f73b8011cf0 con 0x56040a9c8030 2023-07-18T03:30:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73d0f4cb80 10 client.15282 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73d0f4cb80 10 client.15282 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-18T03:30:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73d0f4cb80 20 client.15282 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/3343247042 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x56040aa34680 0x56040aa30fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73d0f4cb80 4 client.15282 mds_command: new command op to 24523 tid=0[{"prefix": "status"}] 2023-07-18T03:30:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.670+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/3343247042 --> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56040a8cf000 con 0x56040aa34680 2023-07-18T03:30:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.671+0000 7f73be59c700 1 --2- 172.21.15.40:0/3343247042 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x56040aa34680 0x56040aa30fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.671+0000 7f73a5ffb700 10 client.15282 ms_handle_connect on v2:172.21.15.134:6838/3192254101 2023-07-18T03:30:39.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.671+0000 7f73be59c700 1 --2- 172.21.15.40:0/3343247042 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x56040aa34680 0x56040aa30fe0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0cbbbd9dc079d8e server_cookie=98e6e28358a92ce5 in_seq=0 out_seq=0 2023-07-18T03:30:39.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.671+0000 7f73a5ffb700 1 -- 172.21.15.40:0/3343247042 <== mds.1 v2:172.21.15.134:6838/3192254101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56040a8cf000 con 0x56040aa34680 2023-07-18T03:30:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.671+0000 7f73a5ffb700 10 client.15282 handle_command_reply: tid=0 2023-07-18T03:30:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.671+0000 7f73d0f4cb80 1 client.15282 shutdown 2023-07-18T03:30:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.671+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/3343247042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c8030 msgr2=0x56040aa2f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.671+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/3343247042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c8030 0x56040aa2f570 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f73b80cc130 tx=0x7f73b809b4a0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/3343247042 shutdown_connections 2023-07-18T03:30:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/3343247042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56040a9c8030 0x56040aa2f570 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/3343247042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56040a9c6890 0x56040a9d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/3343247042 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x56040aa34680 0x56040aa30fe0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73d0f4cb80 1 --2- 172.21.15.40:0/3343247042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56040a9c71e0 0x56040a9d3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73a5ffb700 0 client.15282 ms_handle_reset on v2:172.21.15.134:6838/3192254101 2023-07-18T03:30:39.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/3343247042 >> 172.21.15.40:0/3343247042 conn(0x56040a9b7d50 msgr2=0x56040a9dab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/3343247042 shutdown_connections 2023-07-18T03:30:39.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73d0f4cb80 1 -- 172.21.15.40:0/3343247042 wait complete. 2023-07-18T03:30:39.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73d0f4cb80 20 client.15282 trim_cache size 0 max 16384 2023-07-18T03:30:39.681 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:30:39.681 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "3af3dba8-251a-11ee-9b34-001a4aab830c", 2023-07-18T03:30:39.681 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 1, 2023-07-18T03:30:39.681 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24523, 2023-07-18T03:30:39.682 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-07-18T03:30:39.682 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-07-18T03:30:39.682 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-07-18T03:30:39.682 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 69.054893258999996, 2023-07-18T03:30:39.682 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 27, 2023-07-18T03:30:39.682 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 83, 2023-07-18T03:30:39.682 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 81, 2023-07-18T03:30:39.683 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 85.631511509000006 2023-07-18T03:30:39.683 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:30:39.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73c4dc6700 1 -- 172.21.15.40:0/3375603520 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f73ac0a11e0 msgr2=0x7f73ac0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73c4dc6700 1 --2- 172.21.15.40:0/3375603520 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f73ac0a11e0 0x7f73ac0a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f73c00ffa30 tx=0x7f73b000b5b0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73c4dc6700 1 -- 172.21.15.40:0/3375603520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73c0100560 msgr2=0x7f73c019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73c4dc6700 1 --2- 172.21.15.40:0/3375603520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73c0100560 0x7f73c019cc00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f73b800b830 tx=0x7f73b8004c80 comp rx=0 tx=0).stop 2023-07-18T03:30:39.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73c4dc6700 1 -- 172.21.15.40:0/3375603520 shutdown_connections 2023-07-18T03:30:39.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73c4dc6700 1 --2- 172.21.15.40:0/3375603520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73c0100560 0x7f73c019cc00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73c4dc6700 1 --2- 172.21.15.40:0/3375603520 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f73ac0a11e0 0x7f73ac0a3690 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73c4dc6700 1 --2- 172.21.15.40:0/3375603520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73c00fe9e0 0x7f73c0069210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73c4dc6700 1 --2- 172.21.15.40:0/3375603520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73c00ffbd0 0x7f73c019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73c4dc6700 1 -- 172.21.15.40:0/3375603520 >> 172.21.15.40:0/3375603520 conn(0x7f73c00fa000 msgr2=0x7f73c00fc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73c4dc6700 1 -- 172.21.15.40:0/3375603520 shutdown_connections 2023-07-18T03:30:39.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.672+0000 7f73c4dc6700 1 -- 172.21.15.40:0/3375603520 wait complete. 2023-07-18T03:30:39.686 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-18T03:30:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.817+0000 7f96ef7fd700 1 -- 172.21.15.40:0/738718704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96e80ffbd0 msgr2=0x7f96e8100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.817+0000 7f96ef7fd700 1 --2- 172.21.15.40:0/738718704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96e80ffbd0 0x7f96e8100020 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009990 tx=0x7f96d8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.817+0000 7f96ef7fd700 1 -- 172.21.15.40:0/738718704 shutdown_connections 2023-07-18T03:30:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.817+0000 7f96ef7fd700 1 --2- 172.21.15.40:0/738718704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e8100560 0x7f96e8106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.817+0000 7f96ef7fd700 1 --2- 172.21.15.40:0/738718704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96e80fe9e0 0x7f96e80fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.817+0000 7f96ef7fd700 1 --2- 172.21.15.40:0/738718704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96e80ffbd0 0x7f96e8100020 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.817+0000 7f96ef7fd700 1 -- 172.21.15.40:0/738718704 >> 172.21.15.40:0/738718704 conn(0x7f96e80fa000 msgr2=0x7f96e80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.817+0000 7f96ef7fd700 1 -- 172.21.15.40:0/738718704 shutdown_connections 2023-07-18T03:30:39.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.817+0000 7f96ef7fd700 1 -- 172.21.15.40:0/738718704 wait complete. 2023-07-18T03:30:39.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.817+0000 7f96ef7fd700 1 Processor -- start 2023-07-18T03:30:39.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ef7fd700 1 -- start start 2023-07-18T03:30:39.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ef7fd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96e80fe9e0 0x7f96e8195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ef7fd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e80ffbd0 0x7f96e8195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ef7fd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96e8100560 0x7f96e819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ef7fd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e819d3c0 con 0x7f96e80fe9e0 2023-07-18T03:30:39.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ef7fd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e819d500 con 0x7f96e80ffbd0 2023-07-18T03:30:39.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ef7fd700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e819d690 con 0x7f96e8100560 2023-07-18T03:30:39.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ed599700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96e80fe9e0 0x7f96e8195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ed599700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96e80fe9e0 0x7f96e8195840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57552/0 (socket says 172.21.15.40:57552) 2023-07-18T03:30:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ecd98700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e80ffbd0 0x7f96e8195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ed599700 1 -- 172.21.15.40:0/3322091678 learned_addr learned my addr 172.21.15.40:0/3322091678 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96edd9a700 1 --2- 172.21.15.40:0/3322091678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96e8100560 0x7f96e819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ed599700 1 -- 172.21.15.40:0/3322091678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96e8100560 msgr2=0x7f96e819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ed599700 1 --2- 172.21.15.40:0/3322091678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96e8100560 0x7f96e819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ed599700 1 -- 172.21.15.40:0/3322091678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e80ffbd0 msgr2=0x7f96e8195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ed599700 1 --2- 172.21.15.40:0/3322091678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e80ffbd0 0x7f96e8195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ed599700 1 -- 172.21.15.40:0/3322091678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d80095f0 con 0x7f96e80fe9e0 2023-07-18T03:30:39.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ecd98700 1 --2- 172.21.15.40:0/3322091678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e80ffbd0 0x7f96e8195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:39.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96ed599700 1 --2- 172.21.15.40:0/3322091678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96e80fe9e0 0x7f96e8195840 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f96e400d980 tx=0x7f96e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.818+0000 7f96de7fc700 1 -- 172.21.15.40:0/3322091678 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4004020 con 0x7f96e80fe9e0 2023-07-18T03:30:39.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.819+0000 7f96de7fc700 1 -- 172.21.15.40:0/3322091678 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96e400f690 con 0x7f96e80fe9e0 2023-07-18T03:30:39.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.819+0000 7f96ef7fd700 1 -- 172.21.15.40:0/3322091678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e819d970 con 0x7f96e80fe9e0 2023-07-18T03:30:39.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.819+0000 7f96de7fc700 1 -- 172.21.15.40:0/3322091678 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4003a60 con 0x7f96e80fe9e0 2023-07-18T03:30:39.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.819+0000 7f96ef7fd700 1 -- 172.21.15.40:0/3322091678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e819de80 con 0x7f96e80fe9e0 2023-07-18T03:30:39.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.821+0000 7f96de7fc700 1 -- 172.21.15.40:0/3322091678 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f96e400fb00 con 0x7f96e80fe9e0 2023-07-18T03:30:39.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.821+0000 7f96de7fc700 1 --2- 172.21.15.40:0/3322091678 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f96d40a11e0 0x7f96d40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.821+0000 7f96ecd98700 1 --2- 172.21.15.40:0/3322091678 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f96d40a11e0 0x7f96d40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.821+0000 7f96de7fc700 1 -- 172.21.15.40:0/3322091678 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f96e40cc6b0 con 0x7f96e80fe9e0 2023-07-18T03:30:39.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.822+0000 7f96ecd98700 1 --2- 172.21.15.40:0/3322091678 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f96d40a11e0 0x7f96d40a3690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f96d800b2e0 tx=0x7f96d8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.822+0000 7f96fb983b80 1 Processor -- start 2023-07-18T03:30:39.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.822+0000 7f96fb983b80 1 -- start start 2023-07-18T03:30:39.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.822+0000 7f96fb983b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d098ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.822+0000 7f96fb983b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d0991e0 0x55be0d099aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.822+0000 7f96fb983b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d09a030 0x55be0d09a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.822+0000 7f96fb983b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0cf99c60 con 0x55be0d098890 2023-07-18T03:30:39.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.823+0000 7f96fb983b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0cfec9f0 con 0x55be0d0991e0 2023-07-18T03:30:39.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.823+0000 7f96fb983b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0cfe5e80 con 0x55be0d09a030 2023-07-18T03:30:39.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.823+0000 7f96ed599700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d098ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.823+0000 7f96ed599700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d098ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57560/0 (socket says 172.21.15.40:57560) 2023-07-18T03:30:39.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.823+0000 7f96ed599700 1 -- 172.21.15.40:0/1284462738 learned_addr learned my addr 172.21.15.40:0/1284462738 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.823+0000 7f96edd9a700 1 --2- 172.21.15.40:0/1284462738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d09a030 0x55be0d09a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.823+0000 7f96ecd98700 1 --2- 172.21.15.40:0/1284462738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d0991e0 0x55be0d099aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.823+0000 7f96ed599700 1 -- 172.21.15.40:0/1284462738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d09a030 msgr2=0x55be0d09a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.823+0000 7f96ed599700 1 --2- 172.21.15.40:0/1284462738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d09a030 0x55be0d09a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.823+0000 7f96ed599700 1 -- 172.21.15.40:0/1284462738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d0991e0 msgr2=0x55be0d099aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.823+0000 7f96ed599700 1 --2- 172.21.15.40:0/1284462738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d0991e0 0x55be0d099aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.823+0000 7f96ed599700 1 -- 172.21.15.40:0/1284462738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be0ce13d90 con 0x55be0d098890 2023-07-18T03:30:39.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.823+0000 7f96ed599700 1 --2- 172.21.15.40:0/1284462738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d098ca0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f96e40963b0 tx=0x7f96e4099820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a0c258efee1bb3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.823+0000 7f96d37fe700 1 -- 172.21.15.40:0/1284462738 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e40a1d90 con 0x55be0d098890 2023-07-18T03:30:39.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.823+0000 7f96d37fe700 1 -- 172.21.15.40:0/1284462738 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96e40ca290 con 0x55be0d098890 2023-07-18T03:30:39.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.823+0000 7f96d37fe700 1 -- 172.21.15.40:0/1284462738 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e40a1900 con 0x55be0d098890 2023-07-18T03:30:39.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.824+0000 7f96fb983b80 1 -- 172.21.15.40:0/1284462738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 msgr2=0x55be0d098ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.824+0000 7f96fb983b80 1 --2- 172.21.15.40:0/1284462738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d098ca0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f96e40963b0 tx=0x7f96e4099820 comp rx=0 tx=0).stop 2023-07-18T03:30:39.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.824+0000 7f96fb983b80 1 -- 172.21.15.40:0/1284462738 shutdown_connections 2023-07-18T03:30:39.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.824+0000 7f96fb983b80 1 --2- 172.21.15.40:0/1284462738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d09a030 0x55be0d09a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.824+0000 7f96fb983b80 1 --2- 172.21.15.40:0/1284462738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d098ca0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.824+0000 7f96fb983b80 1 --2- 172.21.15.40:0/1284462738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d0991e0 0x55be0d099aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.824+0000 7f96fb983b80 1 -- 172.21.15.40:0/1284462738 >> 172.21.15.40:0/1284462738 conn(0x55be0d089d50 msgr2=0x55be0d08f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.824+0000 7f96fb983b80 1 -- 172.21.15.40:0/1284462738 shutdown_connections 2023-07-18T03:30:39.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.824+0000 7f96fb983b80 1 -- 172.21.15.40:0/1284462738 wait complete. 2023-07-18T03:30:39.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.824+0000 7f96fb983b80 1 -- start start 2023-07-18T03:30:39.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.825+0000 7f96fb983b80 1 Processor -- start 2023-07-18T03:30:39.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.825+0000 7f96fb983b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d098890 0x55be0d0a2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.825+0000 7f96fb983b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d0991e0 0x55be0d0a2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.825+0000 7f96fb983b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d09a030 0x55be0d0a9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.825+0000 7f96fb983b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0cf85a70 con 0x55be0d0991e0 2023-07-18T03:30:39.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.825+0000 7f96fb983b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0cf85770 con 0x55be0d09a030 2023-07-18T03:30:39.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.825+0000 7f96fb983b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0ce698d0 con 0x55be0d098890 2023-07-18T03:30:39.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.825+0000 7f96fb983b80 10 client.0 _ll_register_callbacks cb 0x55be0cf83710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:39.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.825+0000 7f96ecd98700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d0991e0 0x55be0d0a2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.825+0000 7f96ecd98700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d0991e0 0x55be0d0a2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57568/0 (socket says 172.21.15.40:57568) 2023-07-18T03:30:39.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.825+0000 7f96ecd98700 1 -- 172.21.15.40:0/2825648171 learned_addr learned my addr 172.21.15.40:0/2825648171 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.825+0000 7f96edd9a700 1 --2- 172.21.15.40:0/2825648171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d09a030 0x55be0d0a9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.825+0000 7f96ed599700 1 --2- 172.21.15.40:0/2825648171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d098890 0x55be0d0a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.826+0000 7f96ecd98700 1 -- 172.21.15.40:0/2825648171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d098890 msgr2=0x55be0d0a2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.826+0000 7f96ecd98700 1 --2- 172.21.15.40:0/2825648171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d098890 0x55be0d0a2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.826+0000 7f96ecd98700 1 -- 172.21.15.40:0/2825648171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d09a030 msgr2=0x55be0d0a9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.826+0000 7f96ecd98700 1 --2- 172.21.15.40:0/2825648171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d09a030 0x55be0d0a9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.826+0000 7f96ecd98700 1 -- 172.21.15.40:0/2825648171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e819de80 con 0x55be0d0991e0 2023-07-18T03:30:39.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.826+0000 7f96ecd98700 1 --2- 172.21.15.40:0/2825648171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d0991e0 0x55be0d0a2a80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f96d8015000 tx=0x7f96d8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1551bba4fce3d18 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.826+0000 7f96d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:39.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.826+0000 7f96d0ff9700 1 -- 172.21.15.40:0/2825648171 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8024c30 con 0x55be0d0991e0 2023-07-18T03:30:39.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.826+0000 7f96d0ff9700 1 -- 172.21.15.40:0/2825648171 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f96d8024d90 con 0x55be0d0991e0 2023-07-18T03:30:39.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.826+0000 7f96fb983b80 1 -- 172.21.15.40:0/2825648171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be0cf3f4a0 con 0x55be0d0991e0 2023-07-18T03:30:39.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.826+0000 7f96d0ff9700 1 -- 172.21.15.40:0/2825648171 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96d8004830 con 0x55be0d0991e0 2023-07-18T03:30:39.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.826+0000 7f96fb983b80 10 client.15294 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:39.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.826+0000 7f96fb983b80 1 -- 172.21.15.40:0/2825648171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55be0cdfcc10 con 0x55be0d0991e0 2023-07-18T03:30:39.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.826+0000 7f96d0ff9700 1 -- 172.21.15.40:0/2825648171 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be0cf3f4a0 con 0x55be0d0991e0 2023-07-18T03:30:39.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.826+0000 7f96d0ff9700 1 -- 172.21.15.40:0/2825648171 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f96d80303f0 con 0x55be0d0991e0 2023-07-18T03:30:39.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.826+0000 7f96fb983b80 10 client.15294 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:39.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.827+0000 7f96fb983b80 10 client.15294 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-18T03:30:39.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.827+0000 7f96fb983b80 20 client.15294 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:39.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.827+0000 7f96fb983b80 1 --2- 172.21.15.40:0/2825648171 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55be0d0ad810 0x55be0d0acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.827+0000 7f96fb983b80 4 client.15294 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:30:39.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.827+0000 7f96fb983b80 1 -- 172.21.15.40:0/2825648171 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be0cfdee60 con 0x55be0d0ad810 2023-07-18T03:30:39.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.827+0000 7f96ed599700 1 --2- 172.21.15.40:0/2825648171 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55be0d0ad810 0x55be0d0acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.827+0000 7f96d0ff9700 10 client.15294 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:30:39.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.827+0000 7f96ed599700 1 --2- 172.21.15.40:0/2825648171 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55be0d0ad810 0x55be0d0acbc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6ee10e533791bac server_cookie=6b6590dc047e2a00 in_seq=0 out_seq=0 2023-07-18T03:30:39.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.828+0000 7f96d0ff9700 1 -- 172.21.15.40:0/2825648171 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55be0cfdee60 con 0x55be0d0ad810 2023-07-18T03:30:39.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.828+0000 7f96d0ff9700 10 client.15294 handle_command_reply: tid=0 2023-07-18T03:30:39.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.828+0000 7f96fb983b80 1 client.15294 shutdown 2023-07-18T03:30:39.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.828+0000 7f96fb983b80 1 -- 172.21.15.40:0/2825648171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d0991e0 msgr2=0x55be0d0a2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.828+0000 7f96fb983b80 1 --2- 172.21.15.40:0/2825648171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d0991e0 0x55be0d0a2a80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f96d8015000 tx=0x7f96d8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.828+0000 7f96fb983b80 1 -- 172.21.15.40:0/2825648171 shutdown_connections 2023-07-18T03:30:39.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.828+0000 7f96fb983b80 1 --2- 172.21.15.40:0/2825648171 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55be0d0ad810 0x55be0d0acbc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.828+0000 7f96fb983b80 1 --2- 172.21.15.40:0/2825648171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d09a030 0x55be0d0a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.828+0000 7f96fb983b80 1 --2- 172.21.15.40:0/2825648171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d098890 0x55be0d0a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.828+0000 7f96d0ff9700 0 client.15294 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:30:39.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.828+0000 7f96fb983b80 1 --2- 172.21.15.40:0/2825648171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d0991e0 0x55be0d0a2a80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.828+0000 7f96fb983b80 1 -- 172.21.15.40:0/2825648171 >> 172.21.15.40:0/2825648171 conn(0x55be0d089d50 msgr2=0x55be0d08e860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.828+0000 7f96fb983b80 1 -- 172.21.15.40:0/2825648171 shutdown_connections 2023-07-18T03:30:39.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.828+0000 7f96fb983b80 1 -- 172.21.15.40:0/2825648171 wait complete. 2023-07-18T03:30:39.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.828+0000 7f96fb983b80 20 client.15294 trim_cache size 0 max 16384 2023-07-18T03:30:39.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.846+0000 7f96fb983b80 1 Processor -- start 2023-07-18T03:30:39.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.846+0000 7f96fb983b80 1 -- start start 2023-07-18T03:30:39.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.846+0000 7f96fb983b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d0a1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.846+0000 7f96fb983b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d0991e0 0x55be0d0a9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.846+0000 7f96fb983b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d09a030 0x55be0d08e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.846+0000 7f96fb983b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0cf3f4a0 con 0x55be0d098890 2023-07-18T03:30:39.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.846+0000 7f96fb983b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0d0a3730 con 0x55be0d09a030 2023-07-18T03:30:39.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.846+0000 7f96fb983b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0cfb4790 con 0x55be0d0991e0 2023-07-18T03:30:39.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.846+0000 7f96ed599700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d0a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.846+0000 7f96ed599700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d0a1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57574/0 (socket says 172.21.15.40:57574) 2023-07-18T03:30:39.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.846+0000 7f96ed599700 1 -- 172.21.15.40:0/3512939770 learned_addr learned my addr 172.21.15.40:0/3512939770 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.847+0000 7f96edd9a700 1 --2- 172.21.15.40:0/3512939770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d09a030 0x55be0d08e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.847+0000 7f96ecd98700 1 --2- 172.21.15.40:0/3512939770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d0991e0 0x55be0d0a9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.847+0000 7f96ed599700 1 -- 172.21.15.40:0/3512939770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d0991e0 msgr2=0x55be0d0a9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.847+0000 7f96ed599700 1 --2- 172.21.15.40:0/3512939770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d0991e0 0x55be0d0a9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.847+0000 7f96ed599700 1 -- 172.21.15.40:0/3512939770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d09a030 msgr2=0x55be0d08e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.847+0000 7f96ed599700 1 --2- 172.21.15.40:0/3512939770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d09a030 0x55be0d08e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.847+0000 7f96ed599700 1 -- 172.21.15.40:0/3512939770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be0ce3a200 con 0x55be0d098890 2023-07-18T03:30:39.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.847+0000 7f96ed599700 1 --2- 172.21.15.40:0/3512939770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d0a1210 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f96e400be80 tx=0x7f96e400ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f0771ea2785bbb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.847+0000 7f96d3fff700 1 -- 172.21.15.40:0/3512939770 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4099ba0 con 0x55be0d098890 2023-07-18T03:30:39.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.847+0000 7f96d3fff700 1 -- 172.21.15.40:0/3512939770 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96e4099d00 con 0x55be0d098890 2023-07-18T03:30:39.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.847+0000 7f96d3fff700 1 -- 172.21.15.40:0/3512939770 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4099090 con 0x55be0d098890 2023-07-18T03:30:39.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.848+0000 7f96fb983b80 1 -- 172.21.15.40:0/3512939770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 msgr2=0x55be0d0a1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.848+0000 7f96fb983b80 1 --2- 172.21.15.40:0/3512939770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d0a1210 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f96e400be80 tx=0x7f96e400ff00 comp rx=0 tx=0).stop 2023-07-18T03:30:39.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.848+0000 7f96fb983b80 1 -- 172.21.15.40:0/3512939770 shutdown_connections 2023-07-18T03:30:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.848+0000 7f96fb983b80 1 --2- 172.21.15.40:0/3512939770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d09a030 0x55be0d08e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.848+0000 7f96fb983b80 1 --2- 172.21.15.40:0/3512939770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d0a1210 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.848+0000 7f96fb983b80 1 --2- 172.21.15.40:0/3512939770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d0991e0 0x55be0d0a9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.848+0000 7f96fb983b80 1 -- 172.21.15.40:0/3512939770 >> 172.21.15.40:0/3512939770 conn(0x55be0d089d50 msgr2=0x55be0d0b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.848+0000 7f96fb983b80 1 -- 172.21.15.40:0/3512939770 shutdown_connections 2023-07-18T03:30:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.848+0000 7f96fb983b80 1 -- 172.21.15.40:0/3512939770 wait complete. 2023-07-18T03:30:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.848+0000 7f96fb983b80 1 -- start start 2023-07-18T03:30:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.848+0000 7f96fb983b80 1 Processor -- start 2023-07-18T03:30:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.848+0000 7f96fb983b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d09f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.848+0000 7f96fb983b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d0991e0 0x55be0d0a9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.848+0000 7f96fb983b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d09a030 0x55be0d1016b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.848+0000 7f96fb983b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0d101d70 con 0x55be0d098890 2023-07-18T03:30:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.848+0000 7f96fb983b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0d101ee0 con 0x55be0d09a030 2023-07-18T03:30:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.848+0000 7f96fb983b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0d1020a0 con 0x55be0d0991e0 2023-07-18T03:30:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96fb983b80 10 client.0 _ll_register_callbacks cb 0x7f96d800f850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96ed599700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d09f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96ed599700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d09f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57580/0 (socket says 172.21.15.40:57580) 2023-07-18T03:30:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96ed599700 1 -- 172.21.15.40:0/2411884664 learned_addr learned my addr 172.21.15.40:0/2411884664 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96edd9a700 1 --2- 172.21.15.40:0/2411884664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d09a030 0x55be0d1016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96ecd98700 1 --2- 172.21.15.40:0/2411884664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d0991e0 0x55be0d0a9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96ed599700 1 -- 172.21.15.40:0/2411884664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d0991e0 msgr2=0x55be0d0a9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96ed599700 1 --2- 172.21.15.40:0/2411884664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d0991e0 0x55be0d0a9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96ed599700 1 -- 172.21.15.40:0/2411884664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d09a030 msgr2=0x55be0d1016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:39.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96ed599700 1 --2- 172.21.15.40:0/2411884664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d09a030 0x55be0d1016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96ed599700 1 -- 172.21.15.40:0/2411884664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be0ce13d90 con 0x55be0d098890 2023-07-18T03:30:39.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96ecd98700 1 --2- 172.21.15.40:0/2411884664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d0991e0 0x55be0d0a9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:39.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96ed599700 1 --2- 172.21.15.40:0/2411884664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d09f840 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f96e40963b0 tx=0x7f96e4095680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0a4062e5e9f40a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:39.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96d0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:39.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96d0ff9700 1 -- 172.21.15.40:0/2411884664 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4099ba0 con 0x55be0d098890 2023-07-18T03:30:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96d0ff9700 1 -- 172.21.15.40:0/2411884664 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f96e4084040 con 0x55be0d098890 2023-07-18T03:30:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96fb983b80 1 -- 172.21.15.40:0/2411884664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96e40ca290 con 0x55be0d098890 2023-07-18T03:30:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96d0ff9700 1 -- 172.21.15.40:0/2411884664 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96e40992e0 con 0x55be0d098890 2023-07-18T03:30:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96fb983b80 10 client.15300 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.849+0000 7f96fb983b80 1 -- 172.21.15.40:0/2411884664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55be0ce12260 con 0x55be0d098890 2023-07-18T03:30:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.850+0000 7f96d0ff9700 1 -- 172.21.15.40:0/2411884664 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96e40ca290 con 0x55be0d098890 2023-07-18T03:30:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.850+0000 7f96d0ff9700 1 -- 172.21.15.40:0/2411884664 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f96e4003bc0 con 0x55be0d098890 2023-07-18T03:30:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.850+0000 7f96fb983b80 10 client.15300 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.850+0000 7f96fb983b80 10 client.15300 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-18T03:30:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.850+0000 7f96fb983b80 20 client.15300 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.850+0000 7f96fb983b80 1 --2- 172.21.15.40:0/2411884664 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55be0d106960 0x55be0d1032f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.850+0000 7f96fb983b80 4 client.15300 mds_command: new command op to 24527 tid=0[{"prefix": "status"}] 2023-07-18T03:30:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.850+0000 7f96fb983b80 1 -- 172.21.15.40:0/2411884664 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55be0cfa1000 con 0x55be0d106960 2023-07-18T03:30:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.850+0000 7f96ecd98700 1 --2- 172.21.15.40:0/2411884664 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55be0d106960 0x55be0d1032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.850+0000 7f96d0ff9700 10 client.15300 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:30:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.850+0000 7f96ecd98700 1 --2- 172.21.15.40:0/2411884664 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55be0d106960 0x55be0d1032f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c63a82213d84e30 server_cookie=d6a11376f788f20b in_seq=0 out_seq=0 2023-07-18T03:30:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.851+0000 7f96d0ff9700 1 -- 172.21.15.40:0/2411884664 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55be0cfa1000 con 0x55be0d106960 2023-07-18T03:30:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.851+0000 7f96d0ff9700 10 client.15300 handle_command_reply: tid=0 2023-07-18T03:30:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.851+0000 7f96fb983b80 1 client.15300 shutdown 2023-07-18T03:30:39.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.851+0000 7f96fb983b80 1 -- 172.21.15.40:0/2411884664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 msgr2=0x55be0d09f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.851+0000 7f96fb983b80 1 --2- 172.21.15.40:0/2411884664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d09f840 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f96e40963b0 tx=0x7f96e4095680 comp rx=0 tx=0).stop 2023-07-18T03:30:39.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.851+0000 7f96fb983b80 1 -- 172.21.15.40:0/2411884664 shutdown_connections 2023-07-18T03:30:39.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.851+0000 7f96fb983b80 1 --2- 172.21.15.40:0/2411884664 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55be0d106960 0x55be0d1032f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.851+0000 7f96fb983b80 1 --2- 172.21.15.40:0/2411884664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be0d09a030 0x55be0d1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.851+0000 7f96fb983b80 1 --2- 172.21.15.40:0/2411884664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55be0d098890 0x55be0d09f840 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.851+0000 7f96fb983b80 1 --2- 172.21.15.40:0/2411884664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55be0d0991e0 0x55be0d0a9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.851+0000 7f96d0ff9700 0 client.15300 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:30:39.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.851+0000 7f96fb983b80 1 -- 172.21.15.40:0/2411884664 >> 172.21.15.40:0/2411884664 conn(0x55be0d089d50 msgr2=0x55be0d0a1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.851+0000 7f96fb983b80 1 -- 172.21.15.40:0/2411884664 shutdown_connections 2023-07-18T03:30:39.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.851+0000 7f96fb983b80 1 -- 172.21.15.40:0/2411884664 wait complete. 2023-07-18T03:30:39.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.851+0000 7f96fb983b80 20 client.15300 trim_cache size 0 max 16384 2023-07-18T03:30:39.859 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:30:39.860 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "3af3dba8-251a-11ee-9b34-001a4aab830c", 2023-07-18T03:30:39.860 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-07-18T03:30:39.860 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24527, 2023-07-18T03:30:39.860 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-07-18T03:30:39.860 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-07-18T03:30:39.860 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-07-18T03:30:39.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 72.256624373999998, 2023-07-18T03:30:39.861 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 27, 2023-07-18T03:30:39.861 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 83, 2023-07-18T03:30:39.861 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 81, 2023-07-18T03:30:39.861 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 79.226564468999996 2023-07-18T03:30:39.861 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:30:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.852+0000 7f96ef7fd700 1 -- 172.21.15.40:0/3322091678 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f96d40a11e0 msgr2=0x7f96d40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.852+0000 7f96ef7fd700 1 --2- 172.21.15.40:0/3322091678 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f96d40a11e0 0x7f96d40a3690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f96d800b2e0 tx=0x7f96d8005960 comp rx=0 tx=0).stop 2023-07-18T03:30:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.852+0000 7f96ef7fd700 1 -- 172.21.15.40:0/3322091678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96e80fe9e0 msgr2=0x7f96e8195840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.852+0000 7f96ef7fd700 1 --2- 172.21.15.40:0/3322091678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96e80fe9e0 0x7f96e8195840 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f96e400d980 tx=0x7f96e400dc90 comp rx=0 tx=0).stop 2023-07-18T03:30:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.852+0000 7f96ef7fd700 1 -- 172.21.15.40:0/3322091678 shutdown_connections 2023-07-18T03:30:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.852+0000 7f96ef7fd700 1 --2- 172.21.15.40:0/3322091678 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f96d40a11e0 0x7f96d40a3690 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.852+0000 7f96ef7fd700 1 --2- 172.21.15.40:0/3322091678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96e8100560 0x7f96e819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.852+0000 7f96ef7fd700 1 --2- 172.21.15.40:0/3322091678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96e80fe9e0 0x7f96e8195840 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.852+0000 7f96ef7fd700 1 --2- 172.21.15.40:0/3322091678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96e80ffbd0 0x7f96e8195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:39.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.852+0000 7f96ef7fd700 1 -- 172.21.15.40:0/3322091678 >> 172.21.15.40:0/3322091678 conn(0x7f96e80fa000 msgr2=0x7f96e80fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:39.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.852+0000 7f96ef7fd700 1 -- 172.21.15.40:0/3322091678 shutdown_connections 2023-07-18T03:30:39.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:39.852+0000 7f96ef7fd700 1 -- 172.21.15.40:0/3322091678 wait complete. 2023-07-18T03:30:39.864 INFO:teuthology.run_tasks:Running task kclient... 2023-07-18T03:30:39.893 INFO:tasks.kclient:Mounting kernel clients... 2023-07-18T03:30:39.894 INFO:tasks.kclient:config is {'client.0': {}} 2023-07-18T03:30:39.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:39.895 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-07-18T03:30:39.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:39.913 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2023-07-18T03:30:39.976 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:39 smithi040 ceph-mon[96439]: pgmap v286: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2023-07-18T03:30:39.976 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:39 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3832458061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:39.982 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2023-07-18T03:30:39.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:30:39.983 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2023-07-18T03:30:39.984 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-18T03:30:39.984 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-18T03:30:39.984 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph fs ls 2023-07-18T03:30:40.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:39 smithi134 ceph-mon[98216]: pgmap v286: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2023-07-18T03:30:40.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:39 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3832458061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:40.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:39 smithi064 ceph-mon[98059]: pgmap v286: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2023-07-18T03:30:40.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:39 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3832458061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:40.582 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:30:41.055 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:40 smithi040 ceph-mon[96439]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-07-18T03:30:41.055 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:40 smithi040 ceph-mon[96439]: Cluster is now healthy 2023-07-18T03:30:41.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:40 smithi134 ceph-mon[98216]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-07-18T03:30:41.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:40 smithi134 ceph-mon[98216]: Cluster is now healthy 2023-07-18T03:30:41.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:40 smithi064 ceph-mon[98059]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2023-07-18T03:30:41.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:40 smithi064 ceph-mon[98059]: Cluster is now healthy 2023-07-18T03:30:41.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.667+0000 7fa6b52dc700 1 -- 172.21.15.40:0/2385555798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6b0101700 msgr2=0x7fa6b0109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:41.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.667+0000 7fa6b52dc700 1 --2- 172.21.15.40:0/2385555798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6b0101700 0x7fa6b0109c00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa698009990 tx=0x7fa698009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:41.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.667+0000 7fa6b52dc700 1 -- 172.21.15.40:0/2385555798 shutdown_connections 2023-07-18T03:30:41.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.667+0000 7fa6b52dc700 1 --2- 172.21.15.40:0/2385555798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6b010a140 0x7fa6b010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:41.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.667+0000 7fa6b52dc700 1 --2- 172.21.15.40:0/2385555798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6b0100df0 0x7fa6b01011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:41.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.667+0000 7fa6b52dc700 1 --2- 172.21.15.40:0/2385555798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6b0101700 0x7fa6b0109c00 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:41.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.667+0000 7fa6b52dc700 1 -- 172.21.15.40:0/2385555798 >> 172.21.15.40:0/2385555798 conn(0x7fa6b0075b70 msgr2=0x7fa6b0075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:41.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.667+0000 7fa6b52dc700 1 -- 172.21.15.40:0/2385555798 shutdown_connections 2023-07-18T03:30:41.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.667+0000 7fa6b52dc700 1 -- 172.21.15.40:0/2385555798 wait complete. 2023-07-18T03:30:41.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.668+0000 7fa6b52dc700 1 Processor -- start 2023-07-18T03:30:41.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.668+0000 7fa6b52dc700 1 -- start start 2023-07-18T03:30:41.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.668+0000 7fa6b52dc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6b0100df0 0x7fa6b019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:41.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.668+0000 7fa6b52dc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6b0101700 0x7fa6b019bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:41.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.668+0000 7fa6b52dc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6b010a140 0x7fa6b0195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:41.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.668+0000 7fa6b52dc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b0196140 con 0x7fa6b0101700 2023-07-18T03:30:41.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.668+0000 7fa6b52dc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b01962b0 con 0x7fa6b0100df0 2023-07-18T03:30:41.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.668+0000 7fa6b52dc700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b0196470 con 0x7fa6b010a140 2023-07-18T03:30:41.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.668+0000 7fa6ae7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6b0101700 0x7fa6b019bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:41.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.669+0000 7fa6ae7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6b0101700 0x7fa6b019bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57596/0 (socket says 172.21.15.40:57596) 2023-07-18T03:30:41.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.669+0000 7fa6ae7fc700 1 -- 172.21.15.40:0/1730799627 learned_addr learned my addr 172.21.15.40:0/1730799627 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:41.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.669+0000 7fa6aeffd700 1 --2- 172.21.15.40:0/1730799627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6b0100df0 0x7fa6b019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:41.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.669+0000 7fa6af7fe700 1 --2- 172.21.15.40:0/1730799627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6b010a140 0x7fa6b0195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:41.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.669+0000 7fa6ae7fc700 1 -- 172.21.15.40:0/1730799627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6b010a140 msgr2=0x7fa6b0195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:41.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.669+0000 7fa6ae7fc700 1 --2- 172.21.15.40:0/1730799627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6b010a140 0x7fa6b0195ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:41.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.669+0000 7fa6ae7fc700 1 -- 172.21.15.40:0/1730799627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6b0100df0 msgr2=0x7fa6b019ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:41.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.669+0000 7fa6ae7fc700 1 --2- 172.21.15.40:0/1730799627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6b0100df0 0x7fa6b019ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:41.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.669+0000 7fa6ae7fc700 1 -- 172.21.15.40:0/1730799627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6980095f0 con 0x7fa6b0101700 2023-07-18T03:30:41.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.669+0000 7fa6ae7fc700 1 --2- 172.21.15.40:0/1730799627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6b0101700 0x7fa6b019bf90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa698000c00 tx=0x7fa6980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:41.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.669+0000 7fa6a7fff700 1 -- 172.21.15.40:0/1730799627 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69801d070 con 0x7fa6b0101700 2023-07-18T03:30:41.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.669+0000 7fa6a7fff700 1 -- 172.21.15.40:0/1730799627 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa698021410 con 0x7fa6b0101700 2023-07-18T03:30:41.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.669+0000 7fa6b52dc700 1 -- 172.21.15.40:0/1730799627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b01966f0 con 0x7fa6b0101700 2023-07-18T03:30:41.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.669+0000 7fa6a7fff700 1 -- 172.21.15.40:0/1730799627 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698017650 con 0x7fa6b0101700 2023-07-18T03:30:41.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.669+0000 7fa6b52dc700 1 -- 172.21.15.40:0/1730799627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b01a2a40 con 0x7fa6b0101700 2023-07-18T03:30:41.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.670+0000 7fa6b52dc700 1 -- 172.21.15.40:0/1730799627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b01049d0 con 0x7fa6b0101700 2023-07-18T03:30:41.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.671+0000 7fa6a7fff700 1 -- 172.21.15.40:0/1730799627 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa6980217f0 con 0x7fa6b0101700 2023-07-18T03:30:41.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.672+0000 7fa6a7fff700 1 --2- 172.21.15.40:0/1730799627 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa69c0a11e0 0x7fa69c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:41.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.672+0000 7fa6aeffd700 1 --2- 172.21.15.40:0/1730799627 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa69c0a11e0 0x7fa69c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:41.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.672+0000 7fa6a7fff700 1 -- 172.21.15.40:0/1730799627 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa6980cde80 con 0x7fa6b0101700 2023-07-18T03:30:41.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.672+0000 7fa6aeffd700 1 --2- 172.21.15.40:0/1730799627 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa69c0a11e0 0x7fa69c0a3690 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0005950 tx=0x7fa6a000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:41.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.676+0000 7fa6a7fff700 1 -- 172.21.15.40:0/1730799627 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa698097480 con 0x7fa6b0101700 2023-07-18T03:30:41.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.919+0000 7fa6b52dc700 1 -- 172.21.15.40:0/1730799627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa6b004e210 con 0x7fa6b0101700 2023-07-18T03:30:41.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.919+0000 7fa6a7fff700 1 -- 172.21.15.40:0/1730799627 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa69809aaa0 con 0x7fa6b0101700 2023-07-18T03:30:41.916 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-18T03:30:41.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.921+0000 7fa6b52dc700 1 -- 172.21.15.40:0/1730799627 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa69c0a11e0 msgr2=0x7fa69c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:41.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.921+0000 7fa6b52dc700 1 --2- 172.21.15.40:0/1730799627 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa69c0a11e0 0x7fa69c0a3690 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0005950 tx=0x7fa6a000a300 comp rx=0 tx=0).stop 2023-07-18T03:30:41.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.921+0000 7fa6b52dc700 1 -- 172.21.15.40:0/1730799627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6b0101700 msgr2=0x7fa6b019bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:41.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.921+0000 7fa6b52dc700 1 --2- 172.21.15.40:0/1730799627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6b0101700 0x7fa6b019bf90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa698000c00 tx=0x7fa6980047b0 comp rx=0 tx=0).stop 2023-07-18T03:30:41.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.922+0000 7fa6b52dc700 1 -- 172.21.15.40:0/1730799627 shutdown_connections 2023-07-18T03:30:41.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.922+0000 7fa6b52dc700 1 --2- 172.21.15.40:0/1730799627 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa69c0a11e0 0x7fa69c0a3690 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:41.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.922+0000 7fa6b52dc700 1 --2- 172.21.15.40:0/1730799627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6b010a140 0x7fa6b0195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:41.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.922+0000 7fa6b52dc700 1 --2- 172.21.15.40:0/1730799627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6b0100df0 0x7fa6b019ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:41.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.922+0000 7fa6b52dc700 1 --2- 172.21.15.40:0/1730799627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6b0101700 0x7fa6b019bf90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:41.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.922+0000 7fa6b52dc700 1 -- 172.21.15.40:0/1730799627 >> 172.21.15.40:0/1730799627 conn(0x7fa6b0075b70 msgr2=0x7fa6b0101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:41.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.922+0000 7fa6b52dc700 1 -- 172.21.15.40:0/1730799627 shutdown_connections 2023-07-18T03:30:41.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:41.922+0000 7fa6b52dc700 1 -- 172.21.15.40:0/1730799627 wait complete. 2023-07-18T03:30:42.055 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:41 smithi040 ceph-mon[96439]: pgmap v287: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2023-07-18T03:30:42.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:41 smithi134 ceph-mon[98216]: pgmap v287: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2023-07-18T03:30:42.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:41 smithi064 ceph-mon[98059]: pgmap v287: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2023-07-18T03:30:42.446 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-18T03:30:42.446 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-18T03:30:42.446 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-07-18T03:30:42.446 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-18T03:30:42.447 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-18T03:30:42.447 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-18T03:30:42.447 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-18T03:30:42.447 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-18T03:30:42.447 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-18T03:30:42.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:42.448 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-07-18T03:30:42.466 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-18T03:30:42.467 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-18T03:30:42.467 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-07-18T03:30:42.467 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:30:42.467 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-07-18T03:30:42.467 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:30:42.467 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:30:42.467 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:30:42.468 INFO:teuthology.orchestra.run.smithi040.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-18T03:30:42.468 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-07-18T03:30:42.468 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-18T03:30:42.468 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 41800sec preferred_lft 41800sec 2023-07-18T03:30:42.468 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link 2023-07-18T03:30:42.468 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-07-18T03:30:42.468 INFO:teuthology.orchestra.run.smithi040.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:30:42.469 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-07-18T03:30:42.469 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-18T03:30:42.469 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-07-18T03:30:42.469 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-18T03:30:42.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:42.470 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-07-18T03:30:42.470 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-07-18T03:30:42.470 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add name ceph-brx type bridge 2023-07-18T03:30:42.471 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr flush dev ceph-brx 2023-07-18T03:30:42.471 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set ceph-brx up 2023-07-18T03:30:42.471 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-18T03:30:42.471 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-07-18T03:30:42.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:42.639 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-18T03:30:42.668 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-07-18T03:30:42.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:42.669 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-07-18T03:30:42.748 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-07-18T03:30:42.749 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-18T03:30:42.749 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-18T03:30:42.749 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-18T03:30:42.749 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-18T03:30:42.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:42.750 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-07-18T03:30:42.750 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-07-18T03:30:42.750 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-18T03:30:42.751 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-18T03:30:42.751 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-18T03:30:42.751 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-07-18T03:30:42.932 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:42 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1730799627' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-18T03:30:43.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:43.006 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-07-18T03:30:43.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:43.023 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2023-07-18T03:30:43.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:42 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1730799627' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-18T03:30:43.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:43.080 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-07-18T03:30:43.081 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-07-18T03:30:43.081 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-18T03:30:43.081 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-18T03:30:43.081 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-07-18T03:30:43.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:42 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1730799627' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-18T03:30:43.177 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-18T03:30:43.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:43.178 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-07-18T03:30:43.178 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-07-18T03:30:43.178 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-18T03:30:43.179 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-18T03:30:43.179 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-18T03:30:43.179 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-18T03:30:43.179 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-18T03:30:43.180 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-07-18T03:30:43.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:43.425 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-07-18T03:30:43.426 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-07-18T03:30:43.426 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-07-18T03:30:43.426 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.0 master ceph-brx 2023-07-18T03:30:43.426 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-07-18T03:30:43.526 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:30:43.527 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-18T03:30:43.543 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-18T03:30:43.544 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-18T03:30:43.544 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-18T03:30:43.600 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-07-18T03:30:43.600 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-07-18T03:30:43.601 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c 2023-07-18T03:30:43.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:43.601 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-07-18T03:30:43.696 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-07-18T03:30:43.696 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2023-07-18T03:30:43.824 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2023-07-18T03:30:43.825 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2023-07-18T03:30:43.825 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:6789,172.21.15.64:6789,172.21.15.134:6789" 2023-07-18T03:30:43.825 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@3af3dba8-251a-11ee-9b34-001a4aab830c.cephfs=/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c 2023-07-18T03:30:43.825 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.40:6789/172.21.15.64:6789/172.21.15.134:6789" will pass to kernel 2023-07-18T03:30:43.825 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: v2 mount support check returned 2 2023-07-18T03:30:43.825 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: kernel does not support v2 syntax 2023-07-18T03:30:43.826 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.40:6789,172.21.15.64:6789,172.21.15.134:6789:/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c 2023-07-18T03:30:43.826 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mds_namespace=cephfs,fsid=3af3dba8-251a-11ee-9b34-001a4aab830c" will pass to kernel 2023-07-18T03:30:43.831 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-18T03:30:43.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:43.832 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-18T03:30:43.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:43.891 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:30:43.959 INFO:teuthology.orchestra.run.smithi040.stdout:client15309 2023-07-18T03:30:43.959 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:30:43.960 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:30:43.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:43.978 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:30:44.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:43 smithi134 ceph-mon[98216]: pgmap v288: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 9 B/s, 1 keys/s, 0 objects/s recovering 2023-07-18T03:30:44.077 INFO:teuthology.orchestra.run.smithi040.stdout:client15309 2023-07-18T03:30:44.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:44.078 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-18T03:30:44.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:43 smithi040 ceph-mon[96439]: pgmap v288: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 9 B/s, 1 keys/s, 0 objects/s recovering 2023-07-18T03:30:44.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:43 smithi064 ceph-mon[98059]: pgmap v288: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 9 B/s, 1 keys/s, 0 objects/s recovering 2023-07-18T03:30:44.136 INFO:teuthology.orchestra.run.smithi040.stdout:client15309 2023-07-18T03:30:44.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:30:44.137 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c.client15309/status') 2023-07-18T03:30:44.199 INFO:teuthology.orchestra.run.smithi040.stdout:instance: client.15309 (3)192.168.0.1:0/3997537511 2023-07-18T03:30:44.199 INFO:teuthology.orchestra.run.smithi040.stdout:blocklisted: false 2023-07-18T03:30:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2023-07-18T03:30:44.200 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2023-07-18T03:30:44.201 INFO:teuthology.orchestra.run.smithi040.stderr:70 bytes copied, 2.4686e-05 s, 2.8 MB/s 2023-07-18T03:30:44.201 INFO:teuthology.run_tasks:Running task check-counter... 2023-07-18T03:30:44.215 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-07-18T03:30:44.226 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:30:44.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.356+0000 7f6198f8c700 1 -- 172.21.15.40:0/3694896158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6194101680 msgr2=0x7f6194107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:44.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.356+0000 7f6198f8c700 1 --2- 172.21.15.40:0/3694896158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6194101680 0x7f6194107f40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f618c00b0a0 tx=0x7f618c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.356+0000 7f6198f8c700 1 -- 172.21.15.40:0/3694896158 shutdown_connections 2023-07-18T03:30:44.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.356+0000 7f6198f8c700 1 --2- 172.21.15.40:0/3694896158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6194101680 0x7f6194107f40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.356+0000 7f6198f8c700 1 --2- 172.21.15.40:0/3694896158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f61940ffb00 0x7f61940fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.356+0000 7f6198f8c700 1 --2- 172.21.15.40:0/3694896158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6194100cf0 0x7f6194101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.356+0000 7f6198f8c700 1 -- 172.21.15.40:0/3694896158 >> 172.21.15.40:0/3694896158 conn(0x7f61940fb0e0 msgr2=0x7f61940fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:44.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.356+0000 7f6198f8c700 1 -- 172.21.15.40:0/3694896158 shutdown_connections 2023-07-18T03:30:44.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.356+0000 7f6198f8c700 1 -- 172.21.15.40:0/3694896158 wait complete. 2023-07-18T03:30:44.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f6198f8c700 1 Processor -- start 2023-07-18T03:30:44.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f6198f8c700 1 -- start start 2023-07-18T03:30:44.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f6198f8c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61940ffb00 0x7f6194195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:44.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f6198f8c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6194100cf0 0x7f6194195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:44.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f6198f8c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6194101680 0x7f619419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:44.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f6198f8c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619419d380 con 0x7f61940ffb00 2023-07-18T03:30:44.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f6198f8c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619419d4c0 con 0x7f6194100cf0 2023-07-18T03:30:44.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f6198f8c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6194072740 con 0x7f6194101680 2023-07-18T03:30:44.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f619259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61940ffb00 0x7f6194195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:44.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f619259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61940ffb00 0x7f6194195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57616/0 (socket says 172.21.15.40:57616) 2023-07-18T03:30:44.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f619259c700 1 -- 172.21.15.40:0/2929824593 learned_addr learned my addr 172.21.15.40:0/2929824593 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:44.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f6191d9b700 1 --2- 172.21.15.40:0/2929824593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6194100cf0 0x7f6194195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:44.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f6192d9d700 1 --2- 172.21.15.40:0/2929824593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6194101680 0x7f619419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:44.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f619259c700 1 -- 172.21.15.40:0/2929824593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6194101680 msgr2=0x7f619419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:44.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f619259c700 1 --2- 172.21.15.40:0/2929824593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6194101680 0x7f619419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f619259c700 1 -- 172.21.15.40:0/2929824593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6194100cf0 msgr2=0x7f6194195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:44.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f619259c700 1 --2- 172.21.15.40:0/2929824593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6194100cf0 0x7f6194195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.357+0000 7f619259c700 1 -- 172.21.15.40:0/2929824593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c009d00 con 0x7f61940ffb00 2023-07-18T03:30:44.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.358+0000 7f6191d9b700 1 --2- 172.21.15.40:0/2929824593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6194100cf0 0x7f6194195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:44.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.358+0000 7f619259c700 1 --2- 172.21.15.40:0/2929824593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61940ffb00 0x7f6194195800 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f618400e970 tx=0x7f618400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:44.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.358+0000 7f61837fe700 1 -- 172.21.15.40:0/2929824593 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6184009790 con 0x7f61940ffb00 2023-07-18T03:30:44.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.358+0000 7f61837fe700 1 -- 172.21.15.40:0/2929824593 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f618400c7e0 con 0x7f61940ffb00 2023-07-18T03:30:44.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.358+0000 7f6198f8c700 1 -- 172.21.15.40:0/2929824593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6194072a20 con 0x7f61940ffb00 2023-07-18T03:30:44.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.358+0000 7f61837fe700 1 -- 172.21.15.40:0/2929824593 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6184003c70 con 0x7f61940ffb00 2023-07-18T03:30:44.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.358+0000 7f6198f8c700 1 -- 172.21.15.40:0/2929824593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6194072f30 con 0x7f61940ffb00 2023-07-18T03:30:44.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.359+0000 7f6198f8c700 1 -- 172.21.15.40:0/2929824593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619410b2e0 con 0x7f61940ffb00 2023-07-18T03:30:44.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.360+0000 7f61837fe700 1 -- 172.21.15.40:0/2929824593 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f618400cc50 con 0x7f61940ffb00 2023-07-18T03:30:44.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.361+0000 7f61837fe700 1 --2- 172.21.15.40:0/2929824593 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f617c0a1220 0x7f617c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:44.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.361+0000 7f61837fe700 1 -- 172.21.15.40:0/2929824593 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6184014070 con 0x7f61940ffb00 2023-07-18T03:30:44.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.361+0000 7f6191d9b700 1 --2- 172.21.15.40:0/2929824593 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f617c0a1220 0x7f617c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:44.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.361+0000 7f6191d9b700 1 --2- 172.21.15.40:0/2929824593 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f617c0a1220 0x7f617c0a36d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f618800afd0 tx=0x7f618800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:44.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.364+0000 7f61837fe700 1 -- 172.21.15.40:0/2929824593 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6184096380 con 0x7f61940ffb00 2023-07-18T03:30:44.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.564+0000 7f6198f8c700 1 -- 172.21.15.40:0/2929824593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f619402cc70 con 0x7f61940ffb00 2023-07-18T03:30:44.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.565+0000 7f61837fe700 1 -- 172.21.15.40:0/2929824593 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22599 (secure 0 0 0) 0x7f6184003dd0 con 0x7f61940ffb00 2023-07-18T03:30:44.562 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:44.562 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":83,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:30:33.646627+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:30:44.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.567+0000 7f6198f8c700 1 -- 172.21.15.40:0/2929824593 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f617c0a1220 msgr2=0x7f617c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:44.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.567+0000 7f6198f8c700 1 --2- 172.21.15.40:0/2929824593 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f617c0a1220 0x7f617c0a36d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f618800afd0 tx=0x7f618800a380 comp rx=0 tx=0).stop 2023-07-18T03:30:44.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.567+0000 7f6198f8c700 1 -- 172.21.15.40:0/2929824593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61940ffb00 msgr2=0x7f6194195800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:44.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.567+0000 7f6198f8c700 1 --2- 172.21.15.40:0/2929824593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61940ffb00 0x7f6194195800 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f618400e970 tx=0x7f618400ec80 comp rx=0 tx=0).stop 2023-07-18T03:30:44.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.567+0000 7f6198f8c700 1 -- 172.21.15.40:0/2929824593 shutdown_connections 2023-07-18T03:30:44.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.567+0000 7f6198f8c700 1 --2- 172.21.15.40:0/2929824593 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f617c0a1220 0x7f617c0a36d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.567+0000 7f6198f8c700 1 --2- 172.21.15.40:0/2929824593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6194101680 0x7f619419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.567+0000 7f6198f8c700 1 --2- 172.21.15.40:0/2929824593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61940ffb00 0x7f6194195800 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.567+0000 7f6198f8c700 1 --2- 172.21.15.40:0/2929824593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6194100cf0 0x7f6194195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.567+0000 7f6198f8c700 1 -- 172.21.15.40:0/2929824593 >> 172.21.15.40:0/2929824593 conn(0x7f61940fb0e0 msgr2=0x7f6194104060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:44.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.568+0000 7f6198f8c700 1 -- 172.21.15.40:0/2929824593 shutdown_connections 2023-07-18T03:30:44.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.568+0000 7f6198f8c700 1 -- 172.21.15.40:0/2929824593 wait complete. 2023-07-18T03:30:44.577 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:25:38.267262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:29:24.727790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:29:26.078507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:30:44.578 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:30:44.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.710+0000 7f7824f76700 1 -- 172.21.15.40:0/2912040343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7820100560 msgr2=0x7f7820106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:44.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.710+0000 7f7824f76700 1 --2- 172.21.15.40:0/2912040343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7820100560 0x7f7820106e20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f781800b0a0 tx=0x7f781800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.710+0000 7f7824f76700 1 -- 172.21.15.40:0/2912040343 shutdown_connections 2023-07-18T03:30:44.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.710+0000 7f7824f76700 1 --2- 172.21.15.40:0/2912040343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7820100560 0x7f7820106e20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.710+0000 7f7824f76700 1 --2- 172.21.15.40:0/2912040343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78200fe9e0 0x7f78200fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.710+0000 7f7824f76700 1 --2- 172.21.15.40:0/2912040343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78200ffbd0 0x7f7820100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.710+0000 7f7824f76700 1 -- 172.21.15.40:0/2912040343 >> 172.21.15.40:0/2912040343 conn(0x7f78200fa000 msgr2=0x7f78200fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:44.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.710+0000 7f7824f76700 1 -- 172.21.15.40:0/2912040343 shutdown_connections 2023-07-18T03:30:44.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.710+0000 7f7824f76700 1 -- 172.21.15.40:0/2912040343 wait complete. 2023-07-18T03:30:44.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f7824f76700 1 Processor -- start 2023-07-18T03:30:44.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f7824f76700 1 -- start start 2023-07-18T03:30:44.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f7824f76700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78200fe9e0 0x7f7820068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:44.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f7824f76700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78200ffbd0 0x7f782019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:44.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f7824f76700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7820100560 0x7f782019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:44.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f7824f76700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782019d4e0 con 0x7f78200fe9e0 2023-07-18T03:30:44.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f7824f76700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782019d650 con 0x7f7820100560 2023-07-18T03:30:44.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f7824f76700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782019d810 con 0x7f78200ffbd0 2023-07-18T03:30:44.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f781e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78200fe9e0 0x7f7820068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:44.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f781e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78200fe9e0 0x7f7820068200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57630/0 (socket says 172.21.15.40:57630) 2023-07-18T03:30:44.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f781e59c700 1 -- 172.21.15.40:0/1237578229 learned_addr learned my addr 172.21.15.40:0/1237578229 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:44.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f781ed9d700 1 --2- 172.21.15.40:0/1237578229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7820100560 0x7f782019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:44.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f781dd9b700 1 --2- 172.21.15.40:0/1237578229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78200ffbd0 0x7f782019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:44.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f781e59c700 1 -- 172.21.15.40:0/1237578229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78200ffbd0 msgr2=0x7f782019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:44.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f781e59c700 1 --2- 172.21.15.40:0/1237578229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78200ffbd0 0x7f782019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f781e59c700 1 -- 172.21.15.40:0/1237578229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7820100560 msgr2=0x7f782019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:44.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f781e59c700 1 --2- 172.21.15.40:0/1237578229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7820100560 0x7f782019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f781e59c700 1 -- 172.21.15.40:0/1237578229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7818009d00 con 0x7f78200fe9e0 2023-07-18T03:30:44.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.711+0000 7f781e59c700 1 --2- 172.21.15.40:0/1237578229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78200fe9e0 0x7f7820068200 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f781400d980 tx=0x7f781400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:44.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.712+0000 7f780f7fe700 1 -- 172.21.15.40:0/1237578229 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7814004020 con 0x7f78200fe9e0 2023-07-18T03:30:44.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.712+0000 7f7824f76700 1 -- 172.21.15.40:0/1237578229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782019daf0 con 0x7f78200fe9e0 2023-07-18T03:30:44.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.712+0000 7f780f7fe700 1 -- 172.21.15.40:0/1237578229 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7814009730 con 0x7f78200fe9e0 2023-07-18T03:30:44.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.712+0000 7f780f7fe700 1 -- 172.21.15.40:0/1237578229 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7814003a30 con 0x7f78200fe9e0 2023-07-18T03:30:44.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.712+0000 7f7824f76700 1 -- 172.21.15.40:0/1237578229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782019e000 con 0x7f78200fe9e0 2023-07-18T03:30:44.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.713+0000 7f7824f76700 1 -- 172.21.15.40:0/1237578229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7820105f20 con 0x7f78200fe9e0 2023-07-18T03:30:44.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.714+0000 7f780f7fe700 1 -- 172.21.15.40:0/1237578229 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f78140098a0 con 0x7f78200fe9e0 2023-07-18T03:30:44.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.714+0000 7f780f7fe700 1 --2- 172.21.15.40:0/1237578229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f78080a11e0 0x7f78080a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:44.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.714+0000 7f781dd9b700 1 --2- 172.21.15.40:0/1237578229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f78080a11e0 0x7f78080a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:44.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.714+0000 7f780f7fe700 1 -- 172.21.15.40:0/1237578229 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f78140ccb20 con 0x7f78200fe9e0 2023-07-18T03:30:44.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.715+0000 7f781dd9b700 1 --2- 172.21.15.40:0/1237578229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f78080a11e0 0x7f78080a3690 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7820067f70 tx=0x7f7810009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:44.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.718+0000 7f780f7fe700 1 -- 172.21.15.40:0/1237578229 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f78140960f0 con 0x7f78200fe9e0 2023-07-18T03:30:44.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:44 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2929824593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:44.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.916+0000 7f7824f76700 1 -- 172.21.15.40:0/1237578229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f782002c850 con 0x7f78200fe9e0 2023-07-18T03:30:44.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.917+0000 7f780f7fe700 1 -- 172.21.15.40:0/1237578229 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7814099710 con 0x7f78200fe9e0 2023-07-18T03:30:44.914 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-07-18T03:30:44.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.919+0000 7f7824f76700 1 -- 172.21.15.40:0/1237578229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f78080a11e0 msgr2=0x7f78080a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:44.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.919+0000 7f7824f76700 1 --2- 172.21.15.40:0/1237578229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f78080a11e0 0x7f78080a3690 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7820067f70 tx=0x7f7810009260 comp rx=0 tx=0).stop 2023-07-18T03:30:44.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.919+0000 7f7824f76700 1 -- 172.21.15.40:0/1237578229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78200fe9e0 msgr2=0x7f7820068200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:44.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.919+0000 7f7824f76700 1 --2- 172.21.15.40:0/1237578229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78200fe9e0 0x7f7820068200 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f781400d980 tx=0x7f781400dc90 comp rx=0 tx=0).stop 2023-07-18T03:30:44.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.919+0000 7f7824f76700 1 -- 172.21.15.40:0/1237578229 shutdown_connections 2023-07-18T03:30:44.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.919+0000 7f7824f76700 1 --2- 172.21.15.40:0/1237578229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7820100560 0x7f782019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.919+0000 7f7824f76700 1 --2- 172.21.15.40:0/1237578229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78200fe9e0 0x7f7820068200 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.919+0000 7f7824f76700 1 --2- 172.21.15.40:0/1237578229 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f78080a11e0 0x7f78080a3690 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.919+0000 7f7824f76700 1 --2- 172.21.15.40:0/1237578229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78200ffbd0 0x7f782019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:44.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.919+0000 7f7824f76700 1 -- 172.21.15.40:0/1237578229 >> 172.21.15.40:0/1237578229 conn(0x7f78200fa000 msgr2=0x7f7820103060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:44.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.919+0000 7f7824f76700 1 -- 172.21.15.40:0/1237578229 shutdown_connections 2023-07-18T03:30:44.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:44.919+0000 7f7824f76700 1 -- 172.21.15.40:0/1237578229 wait complete. 2023-07-18T03:30:44.928 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:30:45.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.060+0000 7f993c965700 1 -- 172.21.15.40:0/1767435079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99380ffbd0 msgr2=0x7f9938100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.060+0000 7f993c965700 1 --2- 172.21.15.40:0/1767435079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99380ffbd0 0x7f9938100020 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9920009990 tx=0x7f9920009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.060+0000 7f993c965700 1 -- 172.21.15.40:0/1767435079 shutdown_connections 2023-07-18T03:30:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.060+0000 7f993c965700 1 --2- 172.21.15.40:0/1767435079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9938100560 0x7f9938106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.060+0000 7f993c965700 1 --2- 172.21.15.40:0/1767435079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99380fe9e0 0x7f99380fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.060+0000 7f993c965700 1 --2- 172.21.15.40:0/1767435079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99380ffbd0 0x7f9938100020 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.060+0000 7f993c965700 1 -- 172.21.15.40:0/1767435079 >> 172.21.15.40:0/1767435079 conn(0x7f99380fa000 msgr2=0x7f99380fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.060+0000 7f993c965700 1 -- 172.21.15.40:0/1767435079 shutdown_connections 2023-07-18T03:30:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.060+0000 7f993c965700 1 -- 172.21.15.40:0/1767435079 wait complete. 2023-07-18T03:30:45.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f993c965700 1 Processor -- start 2023-07-18T03:30:45.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f993c965700 1 -- start start 2023-07-18T03:30:45.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f993c965700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99380fe9e0 0x7f99381947b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f993c965700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99380ffbd0 0x7f9938194cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f993c965700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9938100560 0x7f993819bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f993c965700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993819c330 con 0x7f99380fe9e0 2023-07-18T03:30:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f993c965700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993819c470 con 0x7f99380ffbd0 2023-07-18T03:30:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f993c965700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993819c600 con 0x7f9938100560 2023-07-18T03:30:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f993659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99380fe9e0 0x7f99381947b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f993659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99380fe9e0 0x7f99381947b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57644/0 (socket says 172.21.15.40:57644) 2023-07-18T03:30:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f993659c700 1 -- 172.21.15.40:0/2464421393 learned_addr learned my addr 172.21.15.40:0/2464421393 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f9935d9b700 1 --2- 172.21.15.40:0/2464421393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99380ffbd0 0x7f9938194cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f9936d9d700 1 --2- 172.21.15.40:0/2464421393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9938100560 0x7f993819bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f993659c700 1 -- 172.21.15.40:0/2464421393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9938100560 msgr2=0x7f993819bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f993659c700 1 --2- 172.21.15.40:0/2464421393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9938100560 0x7f993819bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f993659c700 1 -- 172.21.15.40:0/2464421393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99380ffbd0 msgr2=0x7f9938194cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f993659c700 1 --2- 172.21.15.40:0/2464421393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99380ffbd0 0x7f9938194cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.061+0000 7f993659c700 1 -- 172.21.15.40:0/2464421393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99200095f0 con 0x7f99380fe9e0 2023-07-18T03:30:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.062+0000 7f993659c700 1 --2- 172.21.15.40:0/2464421393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99380fe9e0 0x7f99381947b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f992800c970 tx=0x7f992800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.062+0000 7f992f7fe700 1 -- 172.21.15.40:0/2464421393 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928004220 con 0x7f99380fe9e0 2023-07-18T03:30:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.062+0000 7f993c965700 1 -- 172.21.15.40:0/2464421393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993819c8e0 con 0x7f99380fe9e0 2023-07-18T03:30:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.062+0000 7f992f7fe700 1 -- 172.21.15.40:0/2464421393 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9928007d60 con 0x7f99380fe9e0 2023-07-18T03:30:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.062+0000 7f992f7fe700 1 -- 172.21.15.40:0/2464421393 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928003c90 con 0x7f99380fe9e0 2023-07-18T03:30:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.062+0000 7f993c965700 1 -- 172.21.15.40:0/2464421393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993819cdf0 con 0x7f99380fe9e0 2023-07-18T03:30:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.063+0000 7f993c965700 1 -- 172.21.15.40:0/2464421393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9938106020 con 0x7f99380fe9e0 2023-07-18T03:30:45.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.064+0000 7f992f7fe700 1 -- 172.21.15.40:0/2464421393 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9928004380 con 0x7f99380fe9e0 2023-07-18T03:30:45.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.065+0000 7f992f7fe700 1 --2- 172.21.15.40:0/2464421393 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f99240a11e0 0x7f99240a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:45.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.065+0000 7f992f7fe700 1 -- 172.21.15.40:0/2464421393 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f99280cca90 con 0x7f99380fe9e0 2023-07-18T03:30:45.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.065+0000 7f9935d9b700 1 --2- 172.21.15.40:0/2464421393 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f99240a11e0 0x7f99240a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.065+0000 7f9935d9b700 1 --2- 172.21.15.40:0/2464421393 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f99240a11e0 0x7f99240a3690 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9920006010 tx=0x7f992000b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.068+0000 7f992f7fe700 1 -- 172.21.15.40:0/2464421393 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9928096000 con 0x7f99380fe9e0 2023-07-18T03:30:45.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:44 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2929824593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:45.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:44 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2929824593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:45.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.267+0000 7f993c965700 1 -- 172.21.15.40:0/2464421393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f993802ca00 con 0x7f99380fe9e0 2023-07-18T03:30:45.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.267+0000 7f992f7fe700 1 -- 172.21.15.40:0/2464421393 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f992801d020 con 0x7f99380fe9e0 2023-07-18T03:30:45.264 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-07-18T03:30:45.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.269+0000 7f993c965700 1 -- 172.21.15.40:0/2464421393 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f99240a11e0 msgr2=0x7f99240a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:45.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.269+0000 7f993c965700 1 --2- 172.21.15.40:0/2464421393 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f99240a11e0 0x7f99240a3690 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9920006010 tx=0x7f992000b560 comp rx=0 tx=0).stop 2023-07-18T03:30:45.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.269+0000 7f993c965700 1 -- 172.21.15.40:0/2464421393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99380fe9e0 msgr2=0x7f99381947b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:45.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.269+0000 7f993c965700 1 --2- 172.21.15.40:0/2464421393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99380fe9e0 0x7f99381947b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f992800c970 tx=0x7f992800cc80 comp rx=0 tx=0).stop 2023-07-18T03:30:45.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.269+0000 7f993c965700 1 -- 172.21.15.40:0/2464421393 shutdown_connections 2023-07-18T03:30:45.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.269+0000 7f993c965700 1 --2- 172.21.15.40:0/2464421393 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f99240a11e0 0x7f99240a3690 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.269+0000 7f993c965700 1 --2- 172.21.15.40:0/2464421393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9938100560 0x7f993819bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.269+0000 7f993c965700 1 --2- 172.21.15.40:0/2464421393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99380fe9e0 0x7f99381947b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.269+0000 7f993c965700 1 --2- 172.21.15.40:0/2464421393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99380ffbd0 0x7f9938194cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.269+0000 7f993c965700 1 -- 172.21.15.40:0/2464421393 >> 172.21.15.40:0/2464421393 conn(0x7f99380fa000 msgr2=0x7f9938103160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:45.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.269+0000 7f993c965700 1 -- 172.21.15.40:0/2464421393 shutdown_connections 2023-07-18T03:30:45.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.270+0000 7f993c965700 1 -- 172.21.15.40:0/2464421393 wait complete. 2023-07-18T03:30:45.278 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:30:45.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.410+0000 7f7543c40700 1 -- 172.21.15.40:0/552301995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f753c0ffaf0 msgr2=0x7f753c0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:45.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.410+0000 7f7543c40700 1 --2- 172.21.15.40:0/552301995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f753c0ffaf0 0x7f753c0fff00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7538009990 tx=0x7f7538009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.410+0000 7f7543c40700 1 -- 172.21.15.40:0/552301995 shutdown_connections 2023-07-18T03:30:45.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.410+0000 7f7543c40700 1 --2- 172.21.15.40:0/552301995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f753c101670 0x7f753c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.410+0000 7f7543c40700 1 --2- 172.21.15.40:0/552301995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f753c0ffaf0 0x7f753c0fff00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.410+0000 7f7543c40700 1 --2- 172.21.15.40:0/552301995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f753c100ce0 0x7f753c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.410+0000 7f7543c40700 1 -- 172.21.15.40:0/552301995 >> 172.21.15.40:0/552301995 conn(0x7f753c0fb130 msgr2=0x7f753c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:45.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.410+0000 7f7543c40700 1 -- 172.21.15.40:0/552301995 shutdown_connections 2023-07-18T03:30:45.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.410+0000 7f7543c40700 1 -- 172.21.15.40:0/552301995 wait complete. 2023-07-18T03:30:45.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.411+0000 7f7543c40700 1 Processor -- start 2023-07-18T03:30:45.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.411+0000 7f7543c40700 1 -- start start 2023-07-18T03:30:45.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.411+0000 7f7543c40700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f753c0ffaf0 0x7f753c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:45.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.411+0000 7f7543c40700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f753c100ce0 0x7f753c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:45.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.411+0000 7f7543c40700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f753c101670 0x7f753c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:45.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.411+0000 7f7543c40700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c0735e0 con 0x7f753c0ffaf0 2023-07-18T03:30:45.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.411+0000 7f7543c40700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c073750 con 0x7f753c101670 2023-07-18T03:30:45.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.411+0000 7f7543c40700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c19f8a0 con 0x7f753c100ce0 2023-07-18T03:30:45.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.411+0000 7f75419dc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f753c0ffaf0 0x7f753c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:45.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.411+0000 7f75419dc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f753c0ffaf0 0x7f753c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57656/0 (socket says 172.21.15.40:57656) 2023-07-18T03:30:45.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.411+0000 7f75419dc700 1 -- 172.21.15.40:0/793976221 learned_addr learned my addr 172.21.15.40:0/793976221 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:45.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.412+0000 7f75411db700 1 --2- 172.21.15.40:0/793976221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f753c100ce0 0x7f753c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:45.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.412+0000 7f75421dd700 1 --2- 172.21.15.40:0/793976221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f753c101670 0x7f753c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:45.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.412+0000 7f75419dc700 1 -- 172.21.15.40:0/793976221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f753c100ce0 msgr2=0x7f753c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:45.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.412+0000 7f75419dc700 1 --2- 172.21.15.40:0/793976221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f753c100ce0 0x7f753c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.412+0000 7f75419dc700 1 -- 172.21.15.40:0/793976221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f753c101670 msgr2=0x7f753c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:45.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.412+0000 7f75419dc700 1 --2- 172.21.15.40:0/793976221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f753c101670 0x7f753c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.412+0000 7f75419dc700 1 -- 172.21.15.40:0/793976221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75380095f0 con 0x7f753c0ffaf0 2023-07-18T03:30:45.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.412+0000 7f75411db700 1 --2- 172.21.15.40:0/793976221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f753c100ce0 0x7f753c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:45.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.412+0000 7f75419dc700 1 --2- 172.21.15.40:0/793976221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f753c0ffaf0 0x7f753c0742c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7538000c00 tx=0x7f7538009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:45.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.412+0000 7f7532ffd700 1 -- 172.21.15.40:0/793976221 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753801d070 con 0x7f753c0ffaf0 2023-07-18T03:30:45.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.412+0000 7f7532ffd700 1 -- 172.21.15.40:0/793976221 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7538004180 con 0x7f753c0ffaf0 2023-07-18T03:30:45.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.412+0000 7f7543c40700 1 -- 172.21.15.40:0/793976221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753c19fb20 con 0x7f753c0ffaf0 2023-07-18T03:30:45.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.412+0000 7f7532ffd700 1 -- 172.21.15.40:0/793976221 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7538022e00 con 0x7f753c0ffaf0 2023-07-18T03:30:45.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.412+0000 7f7543c40700 1 -- 172.21.15.40:0/793976221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753c19ffd0 con 0x7f753c0ffaf0 2023-07-18T03:30:45.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.413+0000 7f7543c40700 1 -- 172.21.15.40:0/793976221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753c10b2c0 con 0x7f753c0ffaf0 2023-07-18T03:30:45.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.414+0000 7f7532ffd700 1 -- 172.21.15.40:0/793976221 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7538022500 con 0x7f753c0ffaf0 2023-07-18T03:30:45.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.415+0000 7f7532ffd700 1 --2- 172.21.15.40:0/793976221 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f75280a11e0 0x7f75280a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:45.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.415+0000 7f7532ffd700 1 -- 172.21.15.40:0/793976221 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f75380ce160 con 0x7f753c0ffaf0 2023-07-18T03:30:45.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.415+0000 7f75411db700 1 --2- 172.21.15.40:0/793976221 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f75280a11e0 0x7f75280a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:45.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.415+0000 7f75411db700 1 --2- 172.21.15.40:0/793976221 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f75280a11e0 0x7f75280a3690 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f752c005fd0 tx=0x7f752c005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:45.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.418+0000 7f7532ffd700 1 -- 172.21.15.40:0/793976221 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f75380977c0 con 0x7f753c0ffaf0 2023-07-18T03:30:45.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.618+0000 7f7543c40700 1 -- 172.21.15.40:0/793976221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f753c1069f0 con 0x7f753c0ffaf0 2023-07-18T03:30:45.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.619+0000 7f7532ffd700 1 -- 172.21.15.40:0/793976221 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f753809ade0 con 0x7f753c0ffaf0 2023-07-18T03:30:45.616 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-07-18T03:30:45.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.621+0000 7f7543c40700 1 -- 172.21.15.40:0/793976221 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f75280a11e0 msgr2=0x7f75280a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:45.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.621+0000 7f7543c40700 1 --2- 172.21.15.40:0/793976221 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f75280a11e0 0x7f75280a3690 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f752c005fd0 tx=0x7f752c005f60 comp rx=0 tx=0).stop 2023-07-18T03:30:45.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.621+0000 7f7543c40700 1 -- 172.21.15.40:0/793976221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f753c0ffaf0 msgr2=0x7f753c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:45.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.621+0000 7f7543c40700 1 --2- 172.21.15.40:0/793976221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f753c0ffaf0 0x7f753c0742c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7538000c00 tx=0x7f7538009f90 comp rx=0 tx=0).stop 2023-07-18T03:30:45.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.621+0000 7f7543c40700 1 -- 172.21.15.40:0/793976221 shutdown_connections 2023-07-18T03:30:45.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.621+0000 7f7543c40700 1 --2- 172.21.15.40:0/793976221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f753c101670 0x7f753c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.621+0000 7f7543c40700 1 --2- 172.21.15.40:0/793976221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f753c0ffaf0 0x7f753c0742c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.621+0000 7f7543c40700 1 --2- 172.21.15.40:0/793976221 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f75280a11e0 0x7f75280a3690 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.621+0000 7f7543c40700 1 --2- 172.21.15.40:0/793976221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f753c100ce0 0x7f753c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.621+0000 7f7543c40700 1 -- 172.21.15.40:0/793976221 >> 172.21.15.40:0/793976221 conn(0x7f753c0fb130 msgr2=0x7f753c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:45.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.621+0000 7f7543c40700 1 -- 172.21.15.40:0/793976221 shutdown_connections 2023-07-18T03:30:45.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.621+0000 7f7543c40700 1 -- 172.21.15.40:0/793976221 wait complete. 2023-07-18T03:30:45.630 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:30:45.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.760+0000 7f8dcc800700 1 -- 172.21.15.40:0/3119897328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dc4100550 msgr2=0x7f8dc4106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.760+0000 7f8dcc800700 1 --2- 172.21.15.40:0/3119897328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dc4100550 0x7f8dc4106e10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000b0a0 tx=0x7f8dc000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.760+0000 7f8dcc800700 1 -- 172.21.15.40:0/3119897328 shutdown_connections 2023-07-18T03:30:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.760+0000 7f8dcc800700 1 --2- 172.21.15.40:0/3119897328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dc4100550 0x7f8dc4106e10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.760+0000 7f8dcc800700 1 --2- 172.21.15.40:0/3119897328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8dc40fe9d0 0x7f8dc40fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.760+0000 7f8dcc800700 1 --2- 172.21.15.40:0/3119897328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dc40ffbc0 0x7f8dc4100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.760+0000 7f8dcc800700 1 -- 172.21.15.40:0/3119897328 >> 172.21.15.40:0/3119897328 conn(0x7f8dc40f9fd0 msgr2=0x7f8dc40fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:45.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.761+0000 7f8dcc800700 1 -- 172.21.15.40:0/3119897328 shutdown_connections 2023-07-18T03:30:45.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.761+0000 7f8dcc800700 1 -- 172.21.15.40:0/3119897328 wait complete. 2023-07-18T03:30:45.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.761+0000 7f8dcc800700 1 Processor -- start 2023-07-18T03:30:45.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.761+0000 7f8dcc800700 1 -- start start 2023-07-18T03:30:45.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.761+0000 7f8dcc800700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dc40fe9d0 0x7f8dc40681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:45.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dcc800700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8dc40ffbc0 0x7f8dc419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:45.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dcc800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dc4100550 0x7f8dc419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:45.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dcc800700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc419d3f0 con 0x7f8dc40fe9d0 2023-07-18T03:30:45.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dcc800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc419d560 con 0x7f8dc4100550 2023-07-18T03:30:45.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dcc800700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc419d720 con 0x7f8dc40ffbc0 2023-07-18T03:30:45.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dca59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dc40fe9d0 0x7f8dc40681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:45.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dca59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dc40fe9d0 0x7f8dc40681e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57670/0 (socket says 172.21.15.40:57670) 2023-07-18T03:30:45.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dca59c700 1 -- 172.21.15.40:0/3161197506 learned_addr learned my addr 172.21.15.40:0/3161197506 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:45.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dcad9d700 1 --2- 172.21.15.40:0/3161197506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dc4100550 0x7f8dc419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:45.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dc9d9b700 1 --2- 172.21.15.40:0/3161197506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8dc40ffbc0 0x7f8dc419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:45.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dca59c700 1 -- 172.21.15.40:0/3161197506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8dc40ffbc0 msgr2=0x7f8dc419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:45.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dca59c700 1 --2- 172.21.15.40:0/3161197506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8dc40ffbc0 0x7f8dc419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dca59c700 1 -- 172.21.15.40:0/3161197506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dc4100550 msgr2=0x7f8dc419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:45.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dca59c700 1 --2- 172.21.15.40:0/3161197506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dc4100550 0x7f8dc419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dca59c700 1 -- 172.21.15.40:0/3161197506 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc0009d00 con 0x7f8dc40fe9d0 2023-07-18T03:30:45.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dca59c700 1 --2- 172.21.15.40:0/3161197506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dc40fe9d0 0x7f8dc40681e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8db800c970 tx=0x7f8db800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:45.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dbf7fe700 1 -- 172.21.15.40:0/3161197506 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8004220 con 0x7f8dc40fe9d0 2023-07-18T03:30:45.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.762+0000 7f8dcc800700 1 -- 172.21.15.40:0/3161197506 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc419da00 con 0x7f8dc40fe9d0 2023-07-18T03:30:45.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.763+0000 7f8dcc800700 1 -- 172.21.15.40:0/3161197506 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc419df60 con 0x7f8dc40fe9d0 2023-07-18T03:30:45.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.763+0000 7f8dbf7fe700 1 -- 172.21.15.40:0/3161197506 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8db800a550 con 0x7f8dc40fe9d0 2023-07-18T03:30:45.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.763+0000 7f8dbf7fe700 1 -- 172.21.15.40:0/3161197506 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8003ba0 con 0x7f8dc40fe9d0 2023-07-18T03:30:45.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.763+0000 7f8dcc800700 1 -- 172.21.15.40:0/3161197506 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da8005320 con 0x7f8dc40fe9d0 2023-07-18T03:30:45.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.765+0000 7f8dbf7fe700 1 -- 172.21.15.40:0/3161197506 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8db8003d00 con 0x7f8dc40fe9d0 2023-07-18T03:30:45.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.765+0000 7f8dbf7fe700 1 --2- 172.21.15.40:0/3161197506 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8db40a5630 0x7f8db40a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:45.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.765+0000 7f8dc9d9b700 1 --2- 172.21.15.40:0/3161197506 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8db40a5630 0x7f8db40a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:45.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.765+0000 7f8dbf7fe700 1 -- 172.21.15.40:0/3161197506 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8db80cca60 con 0x7f8dc40fe9d0 2023-07-18T03:30:45.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.766+0000 7f8dc9d9b700 1 --2- 172.21.15.40:0/3161197506 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8db40a5630 0x7f8db40a7ae0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8db0005950 tx=0x7f8db00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:45.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.768+0000 7f8dbf7fe700 1 -- 172.21.15.40:0/3161197506 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8db8096030 con 0x7f8dc40fe9d0 2023-07-18T03:30:45.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.968+0000 7f8dcc800700 1 -- 172.21.15.40:0/3161197506 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8da8005cc0 con 0x7f8dc40fe9d0 2023-07-18T03:30:45.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.969+0000 7f8dbf7fe700 1 -- 172.21.15.40:0/3161197506 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22599 (secure 0 0 0) 0x7f8db801d020 con 0x7f8dc40fe9d0 2023-07-18T03:30:45.966 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:45.967 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":83,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:30:33.646627+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:30:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.971+0000 7f8dcc800700 1 -- 172.21.15.40:0/3161197506 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8db40a5630 msgr2=0x7f8db40a7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.971+0000 7f8dcc800700 1 --2- 172.21.15.40:0/3161197506 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8db40a5630 0x7f8db40a7ae0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8db0005950 tx=0x7f8db00058e0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.971+0000 7f8dcc800700 1 -- 172.21.15.40:0/3161197506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dc40fe9d0 msgr2=0x7f8dc40681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.971+0000 7f8dcc800700 1 --2- 172.21.15.40:0/3161197506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dc40fe9d0 0x7f8dc40681e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8db800c970 tx=0x7f8db800cc80 comp rx=0 tx=0).stop 2023-07-18T03:30:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.972+0000 7f8dcc800700 1 -- 172.21.15.40:0/3161197506 shutdown_connections 2023-07-18T03:30:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.972+0000 7f8dcc800700 1 --2- 172.21.15.40:0/3161197506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dc4100550 0x7f8dc419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.972+0000 7f8dcc800700 1 --2- 172.21.15.40:0/3161197506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8dc40fe9d0 0x7f8dc40681e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.972+0000 7f8dcc800700 1 --2- 172.21.15.40:0/3161197506 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f8db40a5630 0x7f8db40a7ae0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.972+0000 7f8dcc800700 1 --2- 172.21.15.40:0/3161197506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8dc40ffbc0 0x7f8dc419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.972+0000 7f8dcc800700 1 -- 172.21.15.40:0/3161197506 >> 172.21.15.40:0/3161197506 conn(0x7f8dc40f9fd0 msgr2=0x7f8dc4103000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.972+0000 7f8dcc800700 1 -- 172.21.15.40:0/3161197506 shutdown_connections 2023-07-18T03:30:45.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:45.972+0000 7f8dcc800700 1 -- 172.21.15.40:0/3161197506 wait complete. 2023-07-18T03:30:45.979 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:45 smithi040 ceph-mon[96439]: pgmap v289: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 8 B/s, 0 keys/s, 0 objects/s recovering 2023-07-18T03:30:45.980 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:45 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1237578229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:45.980 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:45 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2464421393' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:45.980 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:45 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/793976221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:45.981 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:25:38.267262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:29:24.727790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:29:26.078507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:30:45.982 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:30:46.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:45 smithi134 ceph-mon[98216]: pgmap v289: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 8 B/s, 0 keys/s, 0 objects/s recovering 2023-07-18T03:30:46.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:45 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1237578229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:46.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:45 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2464421393' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:46.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:45 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/793976221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:46.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:45 smithi064 ceph-mon[98059]: pgmap v289: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 8 B/s, 0 keys/s, 0 objects/s recovering 2023-07-18T03:30:46.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:45 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1237578229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:46.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:45 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2464421393' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:46.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:45 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/793976221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:46.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.114+0000 7fc32b6d3700 1 -- 172.21.15.40:0/1229589124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3240ffbd0 msgr2=0x7fc324100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:46.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.114+0000 7fc32b6d3700 1 --2- 172.21.15.40:0/1229589124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3240ffbd0 0x7fc324100020 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc320009990 tx=0x7fc320009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.114+0000 7fc32b6d3700 1 -- 172.21.15.40:0/1229589124 shutdown_connections 2023-07-18T03:30:46.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.114+0000 7fc32b6d3700 1 --2- 172.21.15.40:0/1229589124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc324100560 0x7fc324106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.114+0000 7fc32b6d3700 1 --2- 172.21.15.40:0/1229589124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3240fe9e0 0x7fc3240fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.114+0000 7fc32b6d3700 1 --2- 172.21.15.40:0/1229589124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3240ffbd0 0x7fc324100020 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.114+0000 7fc32b6d3700 1 -- 172.21.15.40:0/1229589124 >> 172.21.15.40:0/1229589124 conn(0x7fc3240fa000 msgr2=0x7fc3240fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:46.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.114+0000 7fc32b6d3700 1 -- 172.21.15.40:0/1229589124 shutdown_connections 2023-07-18T03:30:46.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.114+0000 7fc32b6d3700 1 -- 172.21.15.40:0/1229589124 wait complete. 2023-07-18T03:30:46.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc32b6d3700 1 Processor -- start 2023-07-18T03:30:46.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc32b6d3700 1 -- start start 2023-07-18T03:30:46.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc32b6d3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3240fe9e0 0x7fc324195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:46.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc32b6d3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3240ffbd0 0x7fc324195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:46.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc32b6d3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc324100560 0x7fc32419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:46.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc32b6d3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32419d3b0 con 0x7fc3240ffbd0 2023-07-18T03:30:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc32b6d3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32419d4f0 con 0x7fc3240fe9e0 2023-07-18T03:30:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc32b6d3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32419d680 con 0x7fc324100560 2023-07-18T03:30:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc328c6e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3240ffbd0 0x7fc324195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc32946f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3240fe9e0 0x7fc324195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc328c6e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3240ffbd0 0x7fc324195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57686/0 (socket says 172.21.15.40:57686) 2023-07-18T03:30:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc32946f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3240fe9e0 0x7fc324195830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:41010/0 (socket says 172.21.15.40:41010) 2023-07-18T03:30:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc328c6e700 1 -- 172.21.15.40:0/1923866437 learned_addr learned my addr 172.21.15.40:0/1923866437 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:46.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc329c70700 1 --2- 172.21.15.40:0/1923866437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc324100560 0x7fc32419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc328c6e700 1 -- 172.21.15.40:0/1923866437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc324100560 msgr2=0x7fc32419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc328c6e700 1 --2- 172.21.15.40:0/1923866437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc324100560 0x7fc32419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc328c6e700 1 -- 172.21.15.40:0/1923866437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3240fe9e0 msgr2=0x7fc324195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc328c6e700 1 --2- 172.21.15.40:0/1923866437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3240fe9e0 0x7fc324195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.115+0000 7fc328c6e700 1 -- 172.21.15.40:0/1923866437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3200095f0 con 0x7fc3240ffbd0 2023-07-18T03:30:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.116+0000 7fc328c6e700 1 --2- 172.21.15.40:0/1923866437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3240ffbd0 0x7fc324195d70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc32000b620 tx=0x7fc320004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.116+0000 7fc31a7fc700 1 -- 172.21.15.40:0/1923866437 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32001d070 con 0x7fc3240ffbd0 2023-07-18T03:30:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.116+0000 7fc31a7fc700 1 -- 172.21.15.40:0/1923866437 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc320005030 con 0x7fc3240ffbd0 2023-07-18T03:30:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.116+0000 7fc32b6d3700 1 -- 172.21.15.40:0/1923866437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32419d900 con 0x7fc3240ffbd0 2023-07-18T03:30:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.116+0000 7fc31a7fc700 1 -- 172.21.15.40:0/1923866437 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320004bf0 con 0x7fc3240ffbd0 2023-07-18T03:30:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.116+0000 7fc32b6d3700 1 -- 172.21.15.40:0/1923866437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32419dd20 con 0x7fc3240ffbd0 2023-07-18T03:30:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.117+0000 7fc32b6d3700 1 -- 172.21.15.40:0/1923866437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc32410a1b0 con 0x7fc3240ffbd0 2023-07-18T03:30:46.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.118+0000 7fc31a7fc700 1 -- 172.21.15.40:0/1923866437 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc320004d50 con 0x7fc3240ffbd0 2023-07-18T03:30:46.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.118+0000 7fc31a7fc700 1 --2- 172.21.15.40:0/1923866437 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3100a11e0 0x7fc3100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:46.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.119+0000 7fc31a7fc700 1 -- 172.21.15.40:0/1923866437 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc3200cdf60 con 0x7fc3240ffbd0 2023-07-18T03:30:46.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.119+0000 7fc32946f700 1 --2- 172.21.15.40:0/1923866437 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3100a11e0 0x7fc3100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:46.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.119+0000 7fc32946f700 1 --2- 172.21.15.40:0/1923866437 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3100a11e0 0x7fc3100a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc3240ffa30 tx=0x7fc314009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:46.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.122+0000 7fc31a7fc700 1 -- 172.21.15.40:0/1923866437 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc3200975c0 con 0x7fc3240ffbd0 2023-07-18T03:30:46.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.321+0000 7fc32b6d3700 1 -- 172.21.15.40:0/1923866437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc32419e360 con 0x7fc3240ffbd0 2023-07-18T03:30:46.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.321+0000 7fc31a7fc700 1 -- 172.21.15.40:0/1923866437 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc32009abe0 con 0x7fc3240ffbd0 2023-07-18T03:30:46.318 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-07-18T03:30:46.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.323+0000 7fc32b6d3700 1 -- 172.21.15.40:0/1923866437 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3100a11e0 msgr2=0x7fc3100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:46.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.323+0000 7fc32b6d3700 1 --2- 172.21.15.40:0/1923866437 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3100a11e0 0x7fc3100a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc3240ffa30 tx=0x7fc314009260 comp rx=0 tx=0).stop 2023-07-18T03:30:46.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.323+0000 7fc32b6d3700 1 -- 172.21.15.40:0/1923866437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3240ffbd0 msgr2=0x7fc324195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:46.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.323+0000 7fc32b6d3700 1 --2- 172.21.15.40:0/1923866437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3240ffbd0 0x7fc324195d70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc32000b620 tx=0x7fc320004730 comp rx=0 tx=0).stop 2023-07-18T03:30:46.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.323+0000 7fc32b6d3700 1 -- 172.21.15.40:0/1923866437 shutdown_connections 2023-07-18T03:30:46.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.323+0000 7fc32b6d3700 1 --2- 172.21.15.40:0/1923866437 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3100a11e0 0x7fc3100a3690 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.323+0000 7fc32b6d3700 1 --2- 172.21.15.40:0/1923866437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc324100560 0x7fc32419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.323+0000 7fc32b6d3700 1 --2- 172.21.15.40:0/1923866437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3240fe9e0 0x7fc324195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.323+0000 7fc32b6d3700 1 --2- 172.21.15.40:0/1923866437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3240ffbd0 0x7fc324195d70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.323+0000 7fc32b6d3700 1 -- 172.21.15.40:0/1923866437 >> 172.21.15.40:0/1923866437 conn(0x7fc3240fa000 msgr2=0x7fc324067690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:46.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.323+0000 7fc32b6d3700 1 -- 172.21.15.40:0/1923866437 shutdown_connections 2023-07-18T03:30:46.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.324+0000 7fc32b6d3700 1 -- 172.21.15.40:0/1923866437 wait complete. 2023-07-18T03:30:46.332 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:30:46.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.465+0000 7f39d7e67700 1 -- 172.21.15.40:0/1766487401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d00ffc00 msgr2=0x7f39d0100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:46.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.465+0000 7f39d7e67700 1 --2- 172.21.15.40:0/1766487401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d00ffc00 0x7f39d0100050 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009990 tx=0x7f39c0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.465+0000 7f39d7e67700 1 -- 172.21.15.40:0/1766487401 shutdown_connections 2023-07-18T03:30:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.465+0000 7f39d7e67700 1 --2- 172.21.15.40:0/1766487401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39d0100590 0x7f39d0106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.465+0000 7f39d7e67700 1 --2- 172.21.15.40:0/1766487401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d00fea10 0x7f39d00fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.465+0000 7f39d7e67700 1 --2- 172.21.15.40:0/1766487401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d00ffc00 0x7f39d0100050 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.465+0000 7f39d7e67700 1 -- 172.21.15.40:0/1766487401 >> 172.21.15.40:0/1766487401 conn(0x7f39d00fa030 msgr2=0x7f39d00fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.465+0000 7f39d7e67700 1 -- 172.21.15.40:0/1766487401 shutdown_connections 2023-07-18T03:30:46.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.465+0000 7f39d7e67700 1 -- 172.21.15.40:0/1766487401 wait complete. 2023-07-18T03:30:46.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.466+0000 7f39d7e67700 1 Processor -- start 2023-07-18T03:30:46.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.466+0000 7f39d7e67700 1 -- start start 2023-07-18T03:30:46.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.466+0000 7f39d7e67700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d00fea10 0x7f39d01029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:46.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.466+0000 7f39d7e67700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d00ffc00 0x7f39d0100fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:46.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.466+0000 7f39d7e67700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39d0100590 0x7f39d0101520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:46.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.466+0000 7f39d7e67700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d0101cc0 con 0x7f39d00fea10 2023-07-18T03:30:46.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.466+0000 7f39d7e67700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d0101e30 con 0x7f39d00ffc00 2023-07-18T03:30:46.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.466+0000 7f39d7e67700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d019f8e0 con 0x7f39d0100590 2023-07-18T03:30:46.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.466+0000 7f39d5c03700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d00fea10 0x7f39d01029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:46.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.466+0000 7f39d5c03700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d00fea10 0x7f39d01029a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57702/0 (socket says 172.21.15.40:57702) 2023-07-18T03:30:46.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.466+0000 7f39d5c03700 1 -- 172.21.15.40:0/1349046408 learned_addr learned my addr 172.21.15.40:0/1349046408 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:46.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.466+0000 7f39d6404700 1 --2- 172.21.15.40:0/1349046408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39d0100590 0x7f39d0101520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:46.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.466+0000 7f39d5402700 1 --2- 172.21.15.40:0/1349046408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d00ffc00 0x7f39d0100fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:46.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.466+0000 7f39d5c03700 1 -- 172.21.15.40:0/1349046408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39d0100590 msgr2=0x7f39d0101520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:46.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.466+0000 7f39d5c03700 1 --2- 172.21.15.40:0/1349046408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39d0100590 0x7f39d0101520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.467+0000 7f39d5c03700 1 -- 172.21.15.40:0/1349046408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d00ffc00 msgr2=0x7f39d0100fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:46.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.467+0000 7f39d5c03700 1 --2- 172.21.15.40:0/1349046408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d00ffc00 0x7f39d0100fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.467+0000 7f39d5c03700 1 -- 172.21.15.40:0/1349046408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c00095f0 con 0x7f39d00fea10 2023-07-18T03:30:46.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.467+0000 7f39d5402700 1 --2- 172.21.15.40:0/1349046408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d00ffc00 0x7f39d0100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:46.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.467+0000 7f39d5c03700 1 --2- 172.21.15.40:0/1349046408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d00fea10 0x7f39d01029a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00e970 tx=0x7f39cc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:46.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.467+0000 7f39c6ffd700 1 -- 172.21.15.40:0/1349046408 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc009790 con 0x7f39d00fea10 2023-07-18T03:30:46.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.467+0000 7f39d7e67700 1 -- 172.21.15.40:0/1349046408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d019fbc0 con 0x7f39d00fea10 2023-07-18T03:30:46.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.467+0000 7f39c6ffd700 1 -- 172.21.15.40:0/1349046408 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39cc0044e0 con 0x7f39d00fea10 2023-07-18T03:30:46.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.467+0000 7f39c6ffd700 1 -- 172.21.15.40:0/1349046408 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc003e30 con 0x7f39d00fea10 2023-07-18T03:30:46.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.467+0000 7f39d7e67700 1 -- 172.21.15.40:0/1349046408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d01a00d0 con 0x7f39d00fea10 2023-07-18T03:30:46.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.468+0000 7f39d7e67700 1 -- 172.21.15.40:0/1349046408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d010a230 con 0x7f39d00fea10 2023-07-18T03:30:46.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.469+0000 7f39c6ffd700 1 -- 172.21.15.40:0/1349046408 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f39cc003f90 con 0x7f39d00fea10 2023-07-18T03:30:46.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.470+0000 7f39c6ffd700 1 --2- 172.21.15.40:0/1349046408 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f39bc0a11e0 0x7f39bc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:46.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.470+0000 7f39c6ffd700 1 -- 172.21.15.40:0/1349046408 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f39cc014070 con 0x7f39d00fea10 2023-07-18T03:30:46.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.470+0000 7f39d5402700 1 --2- 172.21.15.40:0/1349046408 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f39bc0a11e0 0x7f39bc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:46.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.470+0000 7f39d5402700 1 --2- 172.21.15.40:0/1349046408 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f39bc0a11e0 0x7f39bc0a3690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f39c000b2e0 tx=0x7f39c0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:46.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.473+0000 7f39c6ffd700 1 -- 172.21.15.40:0/1349046408 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f39cc0964d0 con 0x7f39d00fea10 2023-07-18T03:30:46.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.674+0000 7f39d7e67700 1 -- 172.21.15.40:0/1349046408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f39d01a0680 con 0x7f39d00fea10 2023-07-18T03:30:46.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.674+0000 7f39c6ffd700 1 -- 172.21.15.40:0/1349046408 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f39cc00f3c0 con 0x7f39d00fea10 2023-07-18T03:30:46.671 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-07-18T03:30:46.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.677+0000 7f39d7e67700 1 -- 172.21.15.40:0/1349046408 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f39bc0a11e0 msgr2=0x7f39bc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:46.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.677+0000 7f39d7e67700 1 --2- 172.21.15.40:0/1349046408 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f39bc0a11e0 0x7f39bc0a3690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f39c000b2e0 tx=0x7f39c0000bc0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.677+0000 7f39d7e67700 1 -- 172.21.15.40:0/1349046408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d00fea10 msgr2=0x7f39d01029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:46.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.677+0000 7f39d7e67700 1 --2- 172.21.15.40:0/1349046408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d00fea10 0x7f39d01029a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00e970 tx=0x7f39cc00ed30 comp rx=0 tx=0).stop 2023-07-18T03:30:46.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.677+0000 7f39d7e67700 1 -- 172.21.15.40:0/1349046408 shutdown_connections 2023-07-18T03:30:46.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.677+0000 7f39d7e67700 1 --2- 172.21.15.40:0/1349046408 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f39bc0a11e0 0x7f39bc0a3690 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.677+0000 7f39d7e67700 1 --2- 172.21.15.40:0/1349046408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39d0100590 0x7f39d0101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.677+0000 7f39d7e67700 1 --2- 172.21.15.40:0/1349046408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d00fea10 0x7f39d01029a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.677+0000 7f39d7e67700 1 --2- 172.21.15.40:0/1349046408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39d00ffc00 0x7f39d0100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.677+0000 7f39d7e67700 1 -- 172.21.15.40:0/1349046408 >> 172.21.15.40:0/1349046408 conn(0x7f39d00fa030 msgr2=0x7f39d00686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:46.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.677+0000 7f39d7e67700 1 -- 172.21.15.40:0/1349046408 shutdown_connections 2023-07-18T03:30:46.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.677+0000 7f39d7e67700 1 -- 172.21.15.40:0/1349046408 wait complete. 2023-07-18T03:30:46.686 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:30:46.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.824+0000 7f74f2fe2700 1 -- 172.21.15.40:0/3768479111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74ec100d60 msgr2=0x7f74ec1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:46.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.824+0000 7f74f2fe2700 1 --2- 172.21.15.40:0/3768479111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74ec100d60 0x7f74ec1011b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009990 tx=0x7f74d8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.824+0000 7f74f2fe2700 1 -- 172.21.15.40:0/3768479111 shutdown_connections 2023-07-18T03:30:46.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.824+0000 7f74f2fe2700 1 --2- 172.21.15.40:0/3768479111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74ec1016f0 0x7f74ec107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.824+0000 7f74f2fe2700 1 --2- 172.21.15.40:0/3768479111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74ec0fdd90 0x7f74ec0fe1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.824+0000 7f74f2fe2700 1 --2- 172.21.15.40:0/3768479111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74ec100d60 0x7f74ec1011b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.824+0000 7f74f2fe2700 1 -- 172.21.15.40:0/3768479111 >> 172.21.15.40:0/3768479111 conn(0x7f74ec0f9770 msgr2=0x7f74ec0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:46.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.824+0000 7f74f2fe2700 1 -- 172.21.15.40:0/3768479111 shutdown_connections 2023-07-18T03:30:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.824+0000 7f74f2fe2700 1 -- 172.21.15.40:0/3768479111 wait complete. 2023-07-18T03:30:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.824+0000 7f74f2fe2700 1 Processor -- start 2023-07-18T03:30:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.824+0000 7f74f2fe2700 1 -- start start 2023-07-18T03:30:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.824+0000 7f74f2fe2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74ec0fdd90 0x7f74ec1006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.824+0000 7f74f2fe2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74ec100d60 0x7f74ec0fed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.824+0000 7f74f2fe2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74ec1016f0 0x7f74ec0ff250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.824+0000 7f74f2fe2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec0ff9f0 con 0x7f74ec1016f0 2023-07-18T03:30:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.824+0000 7f74f2fe2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec0ffb60 con 0x7f74ec100d60 2023-07-18T03:30:46.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.824+0000 7f74f2fe2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec19f8a0 con 0x7f74ec0fdd90 2023-07-18T03:30:46.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.825+0000 7f74f157f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74ec1016f0 0x7f74ec0ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:46.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.825+0000 7f74f157f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74ec1016f0 0x7f74ec0ff250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57720/0 (socket says 172.21.15.40:57720) 2023-07-18T03:30:46.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.825+0000 7f74f157f700 1 -- 172.21.15.40:0/3356882302 learned_addr learned my addr 172.21.15.40:0/3356882302 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:46.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.825+0000 7f74ebfff700 1 --2- 172.21.15.40:0/3356882302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74ec100d60 0x7f74ec0fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:46.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.825+0000 7f74f0d7e700 1 --2- 172.21.15.40:0/3356882302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74ec0fdd90 0x7f74ec1006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:46.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.825+0000 7f74f157f700 1 -- 172.21.15.40:0/3356882302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74ec0fdd90 msgr2=0x7f74ec1006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:46.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.825+0000 7f74f157f700 1 --2- 172.21.15.40:0/3356882302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74ec0fdd90 0x7f74ec1006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.825+0000 7f74f157f700 1 -- 172.21.15.40:0/3356882302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74ec100d60 msgr2=0x7f74ec0fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:46.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.825+0000 7f74f157f700 1 --2- 172.21.15.40:0/3356882302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74ec100d60 0x7f74ec0fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:46.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.825+0000 7f74f157f700 1 -- 172.21.15.40:0/3356882302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d80095f0 con 0x7f74ec1016f0 2023-07-18T03:30:46.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.825+0000 7f74f157f700 1 --2- 172.21.15.40:0/3356882302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74ec1016f0 0x7f74ec0ff250 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f74e4007ee0 tx=0x7f74e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:46.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.825+0000 7f74e9ffb700 1 -- 172.21.15.40:0/3356882302 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4017070 con 0x7f74ec1016f0 2023-07-18T03:30:46.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.825+0000 7f74e9ffb700 1 -- 172.21.15.40:0/3356882302 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74e400f040 con 0x7f74ec1016f0 2023-07-18T03:30:46.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.825+0000 7f74f2fe2700 1 -- 172.21.15.40:0/3356882302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ec19fb80 con 0x7f74ec1016f0 2023-07-18T03:30:46.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.825+0000 7f74e9ffb700 1 -- 172.21.15.40:0/3356882302 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4005740 con 0x7f74ec1016f0 2023-07-18T03:30:46.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.826+0000 7f74f2fe2700 1 -- 172.21.15.40:0/3356882302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ec1a0090 con 0x7f74ec1016f0 2023-07-18T03:30:46.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.826+0000 7f74f2fe2700 1 -- 172.21.15.40:0/3356882302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ec107160 con 0x7f74ec1016f0 2023-07-18T03:30:46.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.827+0000 7f74e9ffb700 1 -- 172.21.15.40:0/3356882302 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f74e400dc60 con 0x7f74ec1016f0 2023-07-18T03:30:46.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.828+0000 7f74e9ffb700 1 --2- 172.21.15.40:0/3356882302 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74dc0a1260 0x7f74dc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:46.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.828+0000 7f74e9ffb700 1 -- 172.21.15.40:0/3356882302 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f74e40cc260 con 0x7f74ec1016f0 2023-07-18T03:30:46.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.828+0000 7f74f0d7e700 1 --2- 172.21.15.40:0/3356882302 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74dc0a1260 0x7f74dc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:46.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.829+0000 7f74f0d7e700 1 --2- 172.21.15.40:0/3356882302 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74dc0a1260 0x7f74dc0a3710 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f74e0005950 tx=0x7f74e00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:46.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:46.833+0000 7f74e9ffb700 1 -- 172.21.15.40:0/3356882302 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f74e4095830 con 0x7f74ec1016f0 2023-07-18T03:30:47.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.034+0000 7f74f2fe2700 1 -- 172.21.15.40:0/3356882302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f74ec02c850 con 0x7f74ec1016f0 2023-07-18T03:30:47.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.034+0000 7f74e9ffb700 1 -- 172.21.15.40:0/3356882302 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f74e4098e50 con 0x7f74ec1016f0 2023-07-18T03:30:47.031 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-07-18T03:30:47.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.036+0000 7f74f2fe2700 1 -- 172.21.15.40:0/3356882302 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74dc0a1260 msgr2=0x7f74dc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.036+0000 7f74f2fe2700 1 --2- 172.21.15.40:0/3356882302 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74dc0a1260 0x7f74dc0a3710 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f74e0005950 tx=0x7f74e00058e0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.036+0000 7f74f2fe2700 1 -- 172.21.15.40:0/3356882302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74ec1016f0 msgr2=0x7f74ec0ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.036+0000 7f74f2fe2700 1 --2- 172.21.15.40:0/3356882302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74ec1016f0 0x7f74ec0ff250 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f74e4007ee0 tx=0x7f74e400d3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.036+0000 7f74f2fe2700 1 -- 172.21.15.40:0/3356882302 shutdown_connections 2023-07-18T03:30:47.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.036+0000 7f74f2fe2700 1 --2- 172.21.15.40:0/3356882302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74ec1016f0 0x7f74ec0ff250 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.036+0000 7f74f2fe2700 1 --2- 172.21.15.40:0/3356882302 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f74dc0a1260 0x7f74dc0a3710 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.036+0000 7f74f2fe2700 1 --2- 172.21.15.40:0/3356882302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74ec0fdd90 0x7f74ec1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.036+0000 7f74f2fe2700 1 --2- 172.21.15.40:0/3356882302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f74ec100d60 0x7f74ec0fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.036+0000 7f74f2fe2700 1 -- 172.21.15.40:0/3356882302 >> 172.21.15.40:0/3356882302 conn(0x7f74ec0f9770 msgr2=0x7f74ec1042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:47.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.036+0000 7f74f2fe2700 1 -- 172.21.15.40:0/3356882302 shutdown_connections 2023-07-18T03:30:47.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.036+0000 7f74f2fe2700 1 -- 172.21.15.40:0/3356882302 wait complete. 2023-07-18T03:30:47.044 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:46 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3161197506' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:47.045 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:46 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1923866437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:47.045 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:46 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1349046408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:47.046 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-07-18T03:30:47.047 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:30:47.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:46 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3161197506' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:47.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:46 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1923866437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:47.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:46 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1349046408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:47.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:46 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3161197506' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:47.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:46 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1923866437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:47.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:46 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1349046408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:47.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.180+0000 7fef4c53e700 1 -- 172.21.15.40:0/1106760105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef440fe9d0 msgr2=0x7fef440fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.180+0000 7fef4c53e700 1 --2- 172.21.15.40:0/1106760105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef440fe9d0 0x7fef440fede0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fef40009990 tx=0x7fef40009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.180+0000 7fef4c53e700 1 -- 172.21.15.40:0/1106760105 shutdown_connections 2023-07-18T03:30:47.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.180+0000 7fef4c53e700 1 --2- 172.21.15.40:0/1106760105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef44100550 0x7fef44106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.180+0000 7fef4c53e700 1 --2- 172.21.15.40:0/1106760105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef440fe9d0 0x7fef440fede0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.180+0000 7fef4c53e700 1 --2- 172.21.15.40:0/1106760105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef440ffbc0 0x7fef44100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.180+0000 7fef4c53e700 1 -- 172.21.15.40:0/1106760105 >> 172.21.15.40:0/1106760105 conn(0x7fef440f9fd0 msgr2=0x7fef440fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:47.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.180+0000 7fef4c53e700 1 -- 172.21.15.40:0/1106760105 shutdown_connections 2023-07-18T03:30:47.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.180+0000 7fef4c53e700 1 -- 172.21.15.40:0/1106760105 wait complete. 2023-07-18T03:30:47.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4c53e700 1 Processor -- start 2023-07-18T03:30:47.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4c53e700 1 -- start start 2023-07-18T03:30:47.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4c53e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef440fe9d0 0x7fef44195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4c53e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef440ffbc0 0x7fef44195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4c53e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef44100550 0x7fef4419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4c53e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4419d290 con 0x7fef440fe9d0 2023-07-18T03:30:47.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4c53e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4419d3d0 con 0x7fef44100550 2023-07-18T03:30:47.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4c53e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4419d560 con 0x7fef440ffbc0 2023-07-18T03:30:47.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4a2da700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef440fe9d0 0x7fef44195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4a2da700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef440fe9d0 0x7fef44195710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57736/0 (socket says 172.21.15.40:57736) 2023-07-18T03:30:47.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4a2da700 1 -- 172.21.15.40:0/2475316995 learned_addr learned my addr 172.21.15.40:0/2475316995 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:47.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef49ad9700 1 --2- 172.21.15.40:0/2475316995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef440ffbc0 0x7fef44195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4aadb700 1 --2- 172.21.15.40:0/2475316995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef44100550 0x7fef4419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4a2da700 1 -- 172.21.15.40:0/2475316995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef440ffbc0 msgr2=0x7fef44195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4a2da700 1 --2- 172.21.15.40:0/2475316995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef440ffbc0 0x7fef44195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4a2da700 1 -- 172.21.15.40:0/2475316995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef44100550 msgr2=0x7fef4419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4a2da700 1 --2- 172.21.15.40:0/2475316995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef44100550 0x7fef4419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4a2da700 1 -- 172.21.15.40:0/2475316995 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef400095f0 con 0x7fef440fe9d0 2023-07-18T03:30:47.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.181+0000 7fef4a2da700 1 --2- 172.21.15.40:0/2475316995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef440fe9d0 0x7fef44195710 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fef40000c00 tx=0x7fef40005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:47.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.182+0000 7fef3b7fe700 1 -- 172.21.15.40:0/2475316995 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4001d070 con 0x7fef440fe9d0 2023-07-18T03:30:47.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.182+0000 7fef3b7fe700 1 -- 172.21.15.40:0/2475316995 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef40004060 con 0x7fef440fe9d0 2023-07-18T03:30:47.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.182+0000 7fef3b7fe700 1 -- 172.21.15.40:0/2475316995 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef40022d40 con 0x7fef440fe9d0 2023-07-18T03:30:47.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.182+0000 7fef4c53e700 1 -- 172.21.15.40:0/2475316995 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4419d7e0 con 0x7fef440fe9d0 2023-07-18T03:30:47.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.182+0000 7fef4c53e700 1 -- 172.21.15.40:0/2475316995 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4419dbb0 con 0x7fef440fe9d0 2023-07-18T03:30:47.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.183+0000 7fef4c53e700 1 -- 172.21.15.40:0/2475316995 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4410a1f0 con 0x7fef440fe9d0 2023-07-18T03:30:47.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.184+0000 7fef3b7fe700 1 -- 172.21.15.40:0/2475316995 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fef40003ba0 con 0x7fef440fe9d0 2023-07-18T03:30:47.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.185+0000 7fef3b7fe700 1 --2- 172.21.15.40:0/2475316995 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fef300a1220 0x7fef300a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.185+0000 7fef3b7fe700 1 -- 172.21.15.40:0/2475316995 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fef400ce7e0 con 0x7fef440fe9d0 2023-07-18T03:30:47.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.185+0000 7fef49ad9700 1 --2- 172.21.15.40:0/2475316995 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fef300a1220 0x7fef300a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.185+0000 7fef49ad9700 1 --2- 172.21.15.40:0/2475316995 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fef300a1220 0x7fef300a36d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fef34005ea0 tx=0x7fef34005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:47.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.188+0000 7fef3b7fe700 1 -- 172.21.15.40:0/2475316995 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fef40097e40 con 0x7fef440fe9d0 2023-07-18T03:30:47.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.431+0000 7fef4c53e700 1 -- 172.21.15.40:0/2475316995 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef4419e1f0 con 0x7fef440fe9d0 2023-07-18T03:30:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.431+0000 7fef3b7fe700 1 -- 172.21.15.40:0/2475316995 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10888 (secure 0 0 0) 0x7fef4009b460 con 0x7fef440fe9d0 2023-07-18T03:30:47.428 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:47.428 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:29:26.718789+0000","modified":"2023-07-18T03:29:34.650353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24523,"mds_2":24482,"mds_3":24452,"mds_4":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14601":{"gid":14601,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1818167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1818167554},{"type":"v1","addr":"172.21.15.40:6839","nonce":1818167554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14613":{"gid":14613,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/3953074960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":3953074960},{"type":"v1","addr":"172.21.15.40:6841","nonce":3953074960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3509655945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3509655945},{"type":"v1","addr":"172.21.15.134:6837","nonce":3509655945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/3007039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3007039043},{"type":"v1","addr":"172.21.15.64:6839","nonce":3007039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:30:47.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.433+0000 7fef4c53e700 1 -- 172.21.15.40:0/2475316995 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fef300a1220 msgr2=0x7fef300a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.433+0000 7fef4c53e700 1 --2- 172.21.15.40:0/2475316995 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fef300a1220 0x7fef300a36d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fef34005ea0 tx=0x7fef34005e30 comp rx=0 tx=0).stop 2023-07-18T03:30:47.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.433+0000 7fef4c53e700 1 -- 172.21.15.40:0/2475316995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef440fe9d0 msgr2=0x7fef44195710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.433+0000 7fef4c53e700 1 --2- 172.21.15.40:0/2475316995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef440fe9d0 0x7fef44195710 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fef40000c00 tx=0x7fef40005f10 comp rx=0 tx=0).stop 2023-07-18T03:30:47.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.433+0000 7fef4c53e700 1 -- 172.21.15.40:0/2475316995 shutdown_connections 2023-07-18T03:30:47.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.433+0000 7fef4c53e700 1 --2- 172.21.15.40:0/2475316995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef44100550 0x7fef4419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.433+0000 7fef4c53e700 1 --2- 172.21.15.40:0/2475316995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef440fe9d0 0x7fef44195710 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.433+0000 7fef4c53e700 1 --2- 172.21.15.40:0/2475316995 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fef300a1220 0x7fef300a36d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.433+0000 7fef4c53e700 1 --2- 172.21.15.40:0/2475316995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef440ffbc0 0x7fef44195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.433+0000 7fef4c53e700 1 -- 172.21.15.40:0/2475316995 >> 172.21.15.40:0/2475316995 conn(0x7fef440f9fd0 msgr2=0x7fef440676f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.434+0000 7fef4c53e700 1 -- 172.21.15.40:0/2475316995 shutdown_connections 2023-07-18T03:30:47.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.434+0000 7fef4c53e700 1 -- 172.21.15.40:0/2475316995 wait complete. 2023-07-18T03:30:47.433 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 27 2023-07-18T03:30:47.445 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-07-18T03:30:47.445 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-07-18T03:30:47.446 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:30:47.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.576+0000 7f5830f22700 1 -- 172.21.15.40:0/1606499881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f582c100050 msgr2=0x7f582c100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.576+0000 7f5830f22700 1 --2- 172.21.15.40:0/1606499881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f582c100050 0x7f582c100460 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5820009990 tx=0x7f5820009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.576+0000 7f5830f22700 1 -- 172.21.15.40:0/1606499881 shutdown_connections 2023-07-18T03:30:47.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.576+0000 7f5830f22700 1 --2- 172.21.15.40:0/1606499881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f582c0ff5c0 0x7f582c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.576+0000 7f5830f22700 1 --2- 172.21.15.40:0/1606499881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f582c100050 0x7f582c100460 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.576+0000 7f5830f22700 1 --2- 172.21.15.40:0/1606499881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f582c0fec10 0x7f582c0ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.576+0000 7f5830f22700 1 -- 172.21.15.40:0/1606499881 >> 172.21.15.40:0/1606499881 conn(0x7f582c0f9770 msgr2=0x7f582c0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:47.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.576+0000 7f5830f22700 1 -- 172.21.15.40:0/1606499881 shutdown_connections 2023-07-18T03:30:47.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.576+0000 7f5830f22700 1 -- 172.21.15.40:0/1606499881 wait complete. 2023-07-18T03:30:47.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.577+0000 7f5830f22700 1 Processor -- start 2023-07-18T03:30:47.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.577+0000 7f5830f22700 1 -- start start 2023-07-18T03:30:47.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.577+0000 7f5830f22700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f582c0fec10 0x7f582c0fe670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.577+0000 7f5830f22700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f582c0ff5c0 0x7f582c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.577+0000 7f5830f22700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f582c100050 0x7f582c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.577+0000 7f5830f22700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c19d2f0 con 0x7f582c0ff5c0 2023-07-18T03:30:47.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.577+0000 7f5830f22700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c19d460 con 0x7f582c100050 2023-07-18T03:30:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.577+0000 7f5830f22700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c19d620 con 0x7f582c0fec10 2023-07-18T03:30:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.577+0000 7f5829d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f582c0ff5c0 0x7f582c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.577+0000 7f5829d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f582c0ff5c0 0x7f582c19a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57752/0 (socket says 172.21.15.40:57752) 2023-07-18T03:30:47.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.577+0000 7f5829d9b700 1 -- 172.21.15.40:0/4128476266 learned_addr learned my addr 172.21.15.40:0/4128476266 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:47.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.577+0000 7f582a59c700 1 --2- 172.21.15.40:0/4128476266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f582c0fec10 0x7f582c0fe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.577+0000 7f582ad9d700 1 --2- 172.21.15.40:0/4128476266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f582c100050 0x7f582c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.578+0000 7f5829d9b700 1 -- 172.21.15.40:0/4128476266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f582c0fec10 msgr2=0x7f582c0fe670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.578+0000 7f5829d9b700 1 --2- 172.21.15.40:0/4128476266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f582c0fec10 0x7f582c0fe670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.578+0000 7f5829d9b700 1 -- 172.21.15.40:0/4128476266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f582c100050 msgr2=0x7f582c19cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.578+0000 7f5829d9b700 1 --2- 172.21.15.40:0/4128476266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f582c100050 0x7f582c19cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.578+0000 7f5829d9b700 1 -- 172.21.15.40:0/4128476266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58200095f0 con 0x7f582c0ff5c0 2023-07-18T03:30:47.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.578+0000 7f5829d9b700 1 --2- 172.21.15.40:0/4128476266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f582c0ff5c0 0x7f582c19a510 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f581c00d950 tx=0x7f581c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:47.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.578+0000 7f581b7fe700 1 -- 172.21.15.40:0/4128476266 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c00f890 con 0x7f582c0ff5c0 2023-07-18T03:30:47.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.578+0000 7f5830f22700 1 -- 172.21.15.40:0/4128476266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582c19d900 con 0x7f582c0ff5c0 2023-07-18T03:30:47.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.578+0000 7f581b7fe700 1 -- 172.21.15.40:0/4128476266 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f581c010460 con 0x7f582c0ff5c0 2023-07-18T03:30:47.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.578+0000 7f581b7fe700 1 -- 172.21.15.40:0/4128476266 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c00e610 con 0x7f582c0ff5c0 2023-07-18T03:30:47.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.578+0000 7f5830f22700 1 -- 172.21.15.40:0/4128476266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c19de60 con 0x7f582c0ff5c0 2023-07-18T03:30:47.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.579+0000 7f5830f22700 1 -- 172.21.15.40:0/4128476266 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582c105960 con 0x7f582c0ff5c0 2023-07-18T03:30:47.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.580+0000 7f581b7fe700 1 -- 172.21.15.40:0/4128476266 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f581c0108d0 con 0x7f582c0ff5c0 2023-07-18T03:30:47.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.581+0000 7f581b7fe700 1 --2- 172.21.15.40:0/4128476266 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f58140a1220 0x7f58140a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.581+0000 7f581b7fe700 1 -- 172.21.15.40:0/4128476266 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f581c0cc870 con 0x7f582c0ff5c0 2023-07-18T03:30:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.581+0000 7f582a59c700 1 --2- 172.21.15.40:0/4128476266 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f58140a1220 0x7f58140a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.581+0000 7f582a59c700 1 --2- 172.21.15.40:0/4128476266 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f58140a1220 0x7f58140a36d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f582000b5c0 tx=0x7f5820005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:47.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.584+0000 7f581b7fe700 1 -- 172.21.15.40:0/4128476266 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f581c095de0 con 0x7f582c0ff5c0 2023-07-18T03:30:47.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.781+0000 7f5830f22700 1 -- 172.21.15.40:0/4128476266 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f582c060950 con 0x7f58140a1220 2023-07-18T03:30:47.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.793+0000 7f581b7fe700 1 -- 172.21.15.40:0/4128476266 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343956 (secure 0 0 0) 0x7f582c060950 con 0x7f58140a1220 2023-07-18T03:30:47.793 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:47.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.796+0000 7f5830f22700 1 -- 172.21.15.40:0/4128476266 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f58140a1220 msgr2=0x7f58140a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.796+0000 7f5830f22700 1 --2- 172.21.15.40:0/4128476266 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f58140a1220 0x7f58140a36d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f582000b5c0 tx=0x7f5820005fd0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.796+0000 7f5830f22700 1 -- 172.21.15.40:0/4128476266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f582c0ff5c0 msgr2=0x7f582c19a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.796+0000 7f5830f22700 1 --2- 172.21.15.40:0/4128476266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f582c0ff5c0 0x7f582c19a510 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f581c00d950 tx=0x7f581c00dc60 comp rx=0 tx=0).stop 2023-07-18T03:30:47.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.796+0000 7f5830f22700 1 -- 172.21.15.40:0/4128476266 shutdown_connections 2023-07-18T03:30:47.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.796+0000 7f5830f22700 1 --2- 172.21.15.40:0/4128476266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f582c100050 0x7f582c19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.796+0000 7f5830f22700 1 --2- 172.21.15.40:0/4128476266 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f58140a1220 0x7f58140a36d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.796+0000 7f5830f22700 1 --2- 172.21.15.40:0/4128476266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f582c0fec10 0x7f582c0fe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.796+0000 7f5830f22700 1 --2- 172.21.15.40:0/4128476266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f582c0ff5c0 0x7f582c19a510 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.796+0000 7f5830f22700 1 -- 172.21.15.40:0/4128476266 >> 172.21.15.40:0/4128476266 conn(0x7f582c0f9770 msgr2=0x7f582c0fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:47.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.796+0000 7f5830f22700 1 -- 172.21.15.40:0/4128476266 shutdown_connections 2023-07-18T03:30:47.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.796+0000 7f5830f22700 1 -- 172.21.15.40:0/4128476266 wait complete. 2023-07-18T03:30:47.797 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-18T03:30:47.805 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2023-07-18T03:30:45.835426+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656079,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":871,"num_read_kb":558,"num_write":415,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180826,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336296,"kb_used_data":12872,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605528,"statfs":{"total":1151940427776,"available":1151596060672,"internally_reserved":0,"allocated":13180928,"data_stored":7184654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":318,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":71,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003854"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656547+0000","last_change":"2023-07-18T03:29:27.101623+0000","last_active":"2023-07-18T03:30:33.656547+0000","last_peered":"2023-07-18T03:30:33.656547+0000","last_clean":"2023-07-18T03:30:33.656547+0000","last_became_active":"2023-07-18T03:29:27.101297+0000","last_became_peered":"2023-07-18T03:29:27.101297+0000","last_unstale":"2023-07-18T03:30:33.656547+0000","last_undegraded":"2023-07-18T03:30:33.656547+0000","last_fullsized":"2023-07-18T03:30:33.656547+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:38:25.465606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655619+0000","last_change":"2023-07-18T03:29:27.104159+0000","last_active":"2023-07-18T03:30:33.655619+0000","last_peered":"2023-07-18T03:30:33.655619+0000","last_clean":"2023-07-18T03:30:33.655619+0000","last_became_active":"2023-07-18T03:29:27.103830+0000","last_became_peered":"2023-07-18T03:29:27.103830+0000","last_unstale":"2023-07-18T03:30:33.655619+0000","last_undegraded":"2023-07-18T03:30:33.655619+0000","last_fullsized":"2023-07-18T03:30:33.655619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:02:01.106624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655451+0000","last_change":"2023-07-18T03:29:27.098858+0000","last_active":"2023-07-18T03:30:33.655451+0000","last_peered":"2023-07-18T03:30:33.655451+0000","last_clean":"2023-07-18T03:30:33.655451+0000","last_became_active":"2023-07-18T03:29:27.098540+0000","last_became_peered":"2023-07-18T03:29:27.098540+0000","last_unstale":"2023-07-18T03:30:33.655451+0000","last_undegraded":"2023-07-18T03:30:33.655451+0000","last_fullsized":"2023-07-18T03:30:33.655451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:59:37.520654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.658876+0000","last_change":"2023-07-18T03:29:27.103106+0000","last_active":"2023-07-18T03:30:33.658876+0000","last_peered":"2023-07-18T03:30:33.658876+0000","last_clean":"2023-07-18T03:30:33.658876+0000","last_became_active":"2023-07-18T03:29:27.102780+0000","last_became_peered":"2023-07-18T03:29:27.102780+0000","last_unstale":"2023-07-18T03:30:33.658876+0000","last_undegraded":"2023-07-18T03:30:33.658876+0000","last_fullsized":"2023-07-18T03:30:33.658876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:50:45.417621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.657667+0000","last_change":"2023-07-18T03:29:27.095432+0000","last_active":"2023-07-18T03:30:33.657667+0000","last_peered":"2023-07-18T03:30:33.657667+0000","last_clean":"2023-07-18T03:30:33.657667+0000","last_became_active":"2023-07-18T03:29:27.095103+0000","last_became_peered":"2023-07-18T03:29:27.095103+0000","last_unstale":"2023-07-18T03:30:33.657667+0000","last_undegraded":"2023-07-18T03:30:33.657667+0000","last_fullsized":"2023-07-18T03:30:33.657667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:34:25.462492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660813+0000","last_change":"2023-07-18T03:29:27.091412+0000","last_active":"2023-07-18T03:30:33.660813+0000","last_peered":"2023-07-18T03:30:33.660813+0000","last_clean":"2023-07-18T03:30:33.660813+0000","last_became_active":"2023-07-18T03:29:27.091060+0000","last_became_peered":"2023-07-18T03:29:27.091060+0000","last_unstale":"2023-07-18T03:30:33.660813+0000","last_undegraded":"2023-07-18T03:30:33.660813+0000","last_fullsized":"2023-07-18T03:30:33.660813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:56:48.388976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.748981+0000","last_change":"2023-07-18T03:29:27.105223+0000","last_active":"2023-07-18T03:30:33.748981+0000","last_peered":"2023-07-18T03:30:33.748981+0000","last_clean":"2023-07-18T03:30:33.748981+0000","last_became_active":"2023-07-18T03:29:27.104926+0000","last_became_peered":"2023-07-18T03:29:27.104926+0000","last_unstale":"2023-07-18T03:30:33.748981+0000","last_undegraded":"2023-07-18T03:30:33.748981+0000","last_fullsized":"2023-07-18T03:30:33.748981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:53:06.895279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660183+0000","last_change":"2023-07-18T03:29:27.093722+0000","last_active":"2023-07-18T03:30:33.660183+0000","last_peered":"2023-07-18T03:30:33.660183+0000","last_clean":"2023-07-18T03:30:33.660183+0000","last_became_active":"2023-07-18T03:29:27.093103+0000","last_became_peered":"2023-07-18T03:29:27.093103+0000","last_unstale":"2023-07-18T03:30:33.660183+0000","last_undegraded":"2023-07-18T03:30:33.660183+0000","last_fullsized":"2023-07-18T03:30:33.660183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:57:08.943397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.651027+0000","last_change":"2023-07-18T03:29:27.094681+0000","last_active":"2023-07-18T03:30:33.651027+0000","last_peered":"2023-07-18T03:30:33.651027+0000","last_clean":"2023-07-18T03:30:33.651027+0000","last_became_active":"2023-07-18T03:29:27.094284+0000","last_became_peered":"2023-07-18T03:29:27.094284+0000","last_unstale":"2023-07-18T03:30:33.651027+0000","last_undegraded":"2023-07-18T03:30:33.651027+0000","last_fullsized":"2023-07-18T03:30:33.651027+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:25:26.164002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661512+0000","last_change":"2023-07-18T03:29:27.100759+0000","last_active":"2023-07-18T03:30:33.661512+0000","last_peered":"2023-07-18T03:30:33.661512+0000","last_clean":"2023-07-18T03:30:33.661512+0000","last_became_active":"2023-07-18T03:29:27.096660+0000","last_became_peered":"2023-07-18T03:29:27.096660+0000","last_unstale":"2023-07-18T03:30:33.661512+0000","last_undegraded":"2023-07-18T03:30:33.661512+0000","last_fullsized":"2023-07-18T03:30:33.661512+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:32:57.131034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749504+0000","last_change":"2023-07-18T03:29:27.107251+0000","last_active":"2023-07-18T03:30:33.749504+0000","last_peered":"2023-07-18T03:30:33.749504+0000","last_clean":"2023-07-18T03:30:33.749504+0000","last_became_active":"2023-07-18T03:29:27.106895+0000","last_became_peered":"2023-07-18T03:29:27.106895+0000","last_unstale":"2023-07-18T03:30:33.749504+0000","last_undegraded":"2023-07-18T03:30:33.749504+0000","last_fullsized":"2023-07-18T03:30:33.749504+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:27:10.110391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.658742+0000","last_change":"2023-07-18T03:29:27.103905+0000","last_active":"2023-07-18T03:30:33.658742+0000","last_peered":"2023-07-18T03:30:33.658742+0000","last_clean":"2023-07-18T03:30:33.658742+0000","last_became_active":"2023-07-18T03:29:27.103021+0000","last_became_peered":"2023-07-18T03:29:27.103021+0000","last_unstale":"2023-07-18T03:30:33.658742+0000","last_undegraded":"2023-07-18T03:30:33.658742+0000","last_fullsized":"2023-07-18T03:30:33.658742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:56:32.967945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655982+0000","last_change":"2023-07-18T03:29:27.103529+0000","last_active":"2023-07-18T03:30:33.655982+0000","last_peered":"2023-07-18T03:30:33.655982+0000","last_clean":"2023-07-18T03:30:33.655982+0000","last_became_active":"2023-07-18T03:29:27.103217+0000","last_became_peered":"2023-07-18T03:29:27.103217+0000","last_unstale":"2023-07-18T03:30:33.655982+0000","last_undegraded":"2023-07-18T03:30:33.655982+0000","last_fullsized":"2023-07-18T03:30:33.655982+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:36:22.276029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655161+0000","last_change":"2023-07-18T03:29:27.096640+0000","last_active":"2023-07-18T03:30:33.655161+0000","last_peered":"2023-07-18T03:30:33.655161+0000","last_clean":"2023-07-18T03:30:33.655161+0000","last_became_active":"2023-07-18T03:29:27.096347+0000","last_became_peered":"2023-07-18T03:29:27.096347+0000","last_unstale":"2023-07-18T03:30:33.655161+0000","last_undegraded":"2023-07-18T03:30:33.655161+0000","last_fullsized":"2023-07-18T03:30:33.655161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:51:47.250203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655071+0000","last_change":"2023-07-18T03:29:27.095910+0000","last_active":"2023-07-18T03:30:33.655071+0000","last_peered":"2023-07-18T03:30:33.655071+0000","last_clean":"2023-07-18T03:30:33.655071+0000","last_became_active":"2023-07-18T03:29:27.095556+0000","last_became_peered":"2023-07-18T03:29:27.095556+0000","last_unstale":"2023-07-18T03:30:33.655071+0000","last_undegraded":"2023-07-18T03:30:33.655071+0000","last_fullsized":"2023-07-18T03:30:33.655071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:17:31.377765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650364+0000","last_change":"2023-07-18T03:29:27.108220+0000","last_active":"2023-07-18T03:30:33.650364+0000","last_peered":"2023-07-18T03:30:33.650364+0000","last_clean":"2023-07-18T03:30:33.650364+0000","last_became_active":"2023-07-18T03:29:27.107869+0000","last_became_peered":"2023-07-18T03:29:27.107869+0000","last_unstale":"2023-07-18T03:30:33.650364+0000","last_undegraded":"2023-07-18T03:30:33.650364+0000","last_fullsized":"2023-07-18T03:30:33.650364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:57:32.649126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656345+0000","last_change":"2023-07-18T03:29:27.100634+0000","last_active":"2023-07-18T03:30:33.656345+0000","last_peered":"2023-07-18T03:30:33.656345+0000","last_clean":"2023-07-18T03:30:33.656345+0000","last_became_active":"2023-07-18T03:29:27.100316+0000","last_became_peered":"2023-07-18T03:29:27.100316+0000","last_unstale":"2023-07-18T03:30:33.656345+0000","last_undegraded":"2023-07-18T03:30:33.656345+0000","last_fullsized":"2023-07-18T03:30:33.656345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:24:14.323573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.652475+0000","last_change":"2023-07-18T03:29:27.100322+0000","last_active":"2023-07-18T03:30:33.652475+0000","last_peered":"2023-07-18T03:30:33.652475+0000","last_clean":"2023-07-18T03:30:33.652475+0000","last_became_active":"2023-07-18T03:29:27.099782+0000","last_became_peered":"2023-07-18T03:29:27.099782+0000","last_unstale":"2023-07-18T03:30:33.652475+0000","last_undegraded":"2023-07-18T03:30:33.652475+0000","last_fullsized":"2023-07-18T03:30:33.652475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:03:29.585791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.654114+0000","last_change":"2023-07-18T03:29:27.096639+0000","last_active":"2023-07-18T03:30:33.654114+0000","last_peered":"2023-07-18T03:30:33.654114+0000","last_clean":"2023-07-18T03:30:33.654114+0000","last_became_active":"2023-07-18T03:29:27.096348+0000","last_became_peered":"2023-07-18T03:29:27.096348+0000","last_unstale":"2023-07-18T03:30:33.654114+0000","last_undegraded":"2023-07-18T03:30:33.654114+0000","last_fullsized":"2023-07-18T03:30:33.654114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:57:04.900158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":103,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:43.760810+0000","last_change":"2023-07-18T03:29:26.287722+0000","last_active":"2023-07-18T03:30:43.760810+0000","last_peered":"2023-07-18T03:30:43.760810+0000","last_clean":"2023-07-18T03:30:43.760810+0000","last_became_active":"2023-07-18T03:29:26.287327+0000","last_became_peered":"2023-07-18T03:29:26.287327+0000","last_unstale":"2023-07-18T03:30:43.760810+0000","last_undegraded":"2023-07-18T03:30:43.760810+0000","last_fullsized":"2023-07-18T03:30:43.760810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:00:13.869444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661775+0000","last_change":"2023-07-18T03:29:27.092771+0000","last_active":"2023-07-18T03:30:33.661775+0000","last_peered":"2023-07-18T03:30:33.661775+0000","last_clean":"2023-07-18T03:30:33.661775+0000","last_became_active":"2023-07-18T03:29:27.092456+0000","last_became_peered":"2023-07-18T03:29:27.092456+0000","last_unstale":"2023-07-18T03:30:33.661775+0000","last_undegraded":"2023-07-18T03:30:33.661775+0000","last_fullsized":"2023-07-18T03:30:33.661775+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:36:58.677882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.662061+0000","last_change":"2023-07-18T03:29:26.289683+0000","last_active":"2023-07-18T03:30:33.662061+0000","last_peered":"2023-07-18T03:30:33.662061+0000","last_clean":"2023-07-18T03:30:33.662061+0000","last_became_active":"2023-07-18T03:29:26.289356+0000","last_became_peered":"2023-07-18T03:29:26.289356+0000","last_unstale":"2023-07-18T03:30:33.662061+0000","last_undegraded":"2023-07-18T03:30:33.662061+0000","last_fullsized":"2023-07-18T03:30:33.662061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:32:24.704699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655443+0000","last_change":"2023-07-18T03:29:27.098709+0000","last_active":"2023-07-18T03:30:33.655443+0000","last_peered":"2023-07-18T03:30:33.655443+0000","last_clean":"2023-07-18T03:30:33.655443+0000","last_became_active":"2023-07-18T03:29:27.098395+0000","last_became_peered":"2023-07-18T03:29:27.098395+0000","last_unstale":"2023-07-18T03:30:33.655443+0000","last_undegraded":"2023-07-18T03:30:33.655443+0000","last_fullsized":"2023-07-18T03:30:33.655443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:57.798041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.658004+0000","last_change":"2023-07-18T03:29:26.297119+0000","last_active":"2023-07-18T03:30:33.658004+0000","last_peered":"2023-07-18T03:30:33.658004+0000","last_clean":"2023-07-18T03:30:33.658004+0000","last_became_active":"2023-07-18T03:29:26.296649+0000","last_became_peered":"2023-07-18T03:29:26.296649+0000","last_unstale":"2023-07-18T03:30:33.658004+0000","last_undegraded":"2023-07-18T03:30:33.658004+0000","last_fullsized":"2023-07-18T03:30:33.658004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:19:40.529493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650567+0000","last_change":"2023-07-18T03:29:27.105511+0000","last_active":"2023-07-18T03:30:33.650567+0000","last_peered":"2023-07-18T03:30:33.650567+0000","last_clean":"2023-07-18T03:30:33.650567+0000","last_became_active":"2023-07-18T03:29:27.104946+0000","last_became_peered":"2023-07-18T03:29:27.104946+0000","last_unstale":"2023-07-18T03:30:33.650567+0000","last_undegraded":"2023-07-18T03:30:33.650567+0000","last_fullsized":"2023-07-18T03:30:33.650567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:27:54.324118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.657412+0000","last_change":"2023-07-18T03:29:26.290708+0000","last_active":"2023-07-18T03:30:33.657412+0000","last_peered":"2023-07-18T03:30:33.657412+0000","last_clean":"2023-07-18T03:30:33.657412+0000","last_became_active":"2023-07-18T03:29:26.290383+0000","last_became_peered":"2023-07-18T03:29:26.290383+0000","last_unstale":"2023-07-18T03:30:33.657412+0000","last_undegraded":"2023-07-18T03:30:33.657412+0000","last_fullsized":"2023-07-18T03:30:33.657412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:23:20.274883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660376+0000","last_change":"2023-07-18T03:29:27.096630+0000","last_active":"2023-07-18T03:30:33.660376+0000","last_peered":"2023-07-18T03:30:33.660376+0000","last_clean":"2023-07-18T03:30:33.660376+0000","last_became_active":"2023-07-18T03:29:27.096337+0000","last_became_peered":"2023-07-18T03:29:27.096337+0000","last_unstale":"2023-07-18T03:30:33.660376+0000","last_undegraded":"2023-07-18T03:30:33.660376+0000","last_fullsized":"2023-07-18T03:30:33.660376+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:01:51.889873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":97,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:39.709864+0000","last_change":"2023-07-18T03:29:26.291622+0000","last_active":"2023-07-18T03:30:39.709864+0000","last_peered":"2023-07-18T03:30:39.709864+0000","last_clean":"2023-07-18T03:30:39.709864+0000","last_became_active":"2023-07-18T03:29:26.291317+0000","last_became_peered":"2023-07-18T03:29:26.291317+0000","last_unstale":"2023-07-18T03:30:39.709864+0000","last_undegraded":"2023-07-18T03:30:39.709864+0000","last_fullsized":"2023-07-18T03:30:39.709864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:44:15.122562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660520+0000","last_change":"2023-07-18T03:30:33.660520+0000","last_active":"2023-07-18T03:30:33.660520+0000","last_peered":"2023-07-18T03:30:33.660520+0000","last_clean":"2023-07-18T03:30:33.660520+0000","last_became_active":"2023-07-18T03:30:33.660098+0000","last_became_peered":"2023-07-18T03:30:33.660098+0000","last_unstale":"2023-07-18T03:30:33.660520+0000","last_undegraded":"2023-07-18T03:30:33.660520+0000","last_fullsized":"2023-07-18T03:30:33.660520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:16:28.408741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660746+0000","last_change":"2023-07-18T03:29:26.290493+0000","last_active":"2023-07-18T03:30:33.660746+0000","last_peered":"2023-07-18T03:30:33.660746+0000","last_clean":"2023-07-18T03:30:33.660746+0000","last_became_active":"2023-07-18T03:29:26.290187+0000","last_became_peered":"2023-07-18T03:29:26.290187+0000","last_unstale":"2023-07-18T03:30:33.660746+0000","last_undegraded":"2023-07-18T03:30:33.660746+0000","last_fullsized":"2023-07-18T03:30:33.660746+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:58:16.632361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.659999+0000","last_change":"2023-07-18T03:29:27.095129+0000","last_active":"2023-07-18T03:30:33.659999+0000","last_peered":"2023-07-18T03:30:33.659999+0000","last_clean":"2023-07-18T03:30:33.659999+0000","last_became_active":"2023-07-18T03:29:27.094821+0000","last_became_peered":"2023-07-18T03:29:27.094821+0000","last_unstale":"2023-07-18T03:30:33.659999+0000","last_undegraded":"2023-07-18T03:30:33.659999+0000","last_fullsized":"2023-07-18T03:30:33.659999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:43:40.005624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655778+0000","last_change":"2023-07-18T03:29:26.285860+0000","last_active":"2023-07-18T03:30:33.655778+0000","last_peered":"2023-07-18T03:30:33.655778+0000","last_clean":"2023-07-18T03:30:33.655778+0000","last_became_active":"2023-07-18T03:29:26.285491+0000","last_became_peered":"2023-07-18T03:29:26.285491+0000","last_unstale":"2023-07-18T03:30:33.655778+0000","last_undegraded":"2023-07-18T03:30:33.655778+0000","last_fullsized":"2023-07-18T03:30:33.655778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:39:26.999366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660058+0000","last_change":"2023-07-18T03:29:27.093861+0000","last_active":"2023-07-18T03:30:33.660058+0000","last_peered":"2023-07-18T03:30:33.660058+0000","last_clean":"2023-07-18T03:30:33.660058+0000","last_became_active":"2023-07-18T03:29:27.093445+0000","last_became_peered":"2023-07-18T03:29:27.093445+0000","last_unstale":"2023-07-18T03:30:33.660058+0000","last_undegraded":"2023-07-18T03:30:33.660058+0000","last_fullsized":"2023-07-18T03:30:33.660058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:45:32.889832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.659732+0000","last_change":"2023-07-18T03:29:26.288108+0000","last_active":"2023-07-18T03:30:33.659732+0000","last_peered":"2023-07-18T03:30:33.659732+0000","last_clean":"2023-07-18T03:30:33.659732+0000","last_became_active":"2023-07-18T03:29:26.287531+0000","last_became_peered":"2023-07-18T03:29:26.287531+0000","last_unstale":"2023-07-18T03:30:33.659732+0000","last_undegraded":"2023-07-18T03:30:33.659732+0000","last_fullsized":"2023-07-18T03:30:33.659732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:52:58.383534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.652172+0000","last_change":"2023-07-18T03:29:27.092908+0000","last_active":"2023-07-18T03:30:33.652172+0000","last_peered":"2023-07-18T03:30:33.652172+0000","last_clean":"2023-07-18T03:30:33.652172+0000","last_became_active":"2023-07-18T03:29:27.092566+0000","last_became_peered":"2023-07-18T03:29:27.092566+0000","last_unstale":"2023-07-18T03:30:33.652172+0000","last_undegraded":"2023-07-18T03:30:33.652172+0000","last_fullsized":"2023-07-18T03:30:33.652172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:46:32.375687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":168,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:44.717016+0000","last_change":"2023-07-18T03:29:26.294251+0000","last_active":"2023-07-18T03:30:44.717016+0000","last_peered":"2023-07-18T03:30:44.717016+0000","last_clean":"2023-07-18T03:30:44.717016+0000","last_became_active":"2023-07-18T03:29:26.293847+0000","last_became_peered":"2023-07-18T03:29:26.293847+0000","last_unstale":"2023-07-18T03:30:44.717016+0000","last_undegraded":"2023-07-18T03:30:44.717016+0000","last_fullsized":"2023-07-18T03:30:44.717016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:29:39.028643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":145,"num_read_kb":145,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650892+0000","last_change":"2023-07-18T03:29:27.106551+0000","last_active":"2023-07-18T03:30:33.650892+0000","last_peered":"2023-07-18T03:30:33.650892+0000","last_clean":"2023-07-18T03:30:33.650892+0000","last_became_active":"2023-07-18T03:29:27.105187+0000","last_became_peered":"2023-07-18T03:29:27.105187+0000","last_unstale":"2023-07-18T03:30:33.650892+0000","last_undegraded":"2023-07-18T03:30:33.650892+0000","last_fullsized":"2023-07-18T03:30:33.650892+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:59:30.478453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749555+0000","last_change":"2023-07-18T03:29:26.293284+0000","last_active":"2023-07-18T03:30:33.749555+0000","last_peered":"2023-07-18T03:30:33.749555+0000","last_clean":"2023-07-18T03:30:33.749555+0000","last_became_active":"2023-07-18T03:29:26.292955+0000","last_became_peered":"2023-07-18T03:29:26.292955+0000","last_unstale":"2023-07-18T03:30:33.749555+0000","last_undegraded":"2023-07-18T03:30:33.749555+0000","last_fullsized":"2023-07-18T03:30:33.749555+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:14:25.817345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661734+0000","last_change":"2023-07-18T03:30:33.661734+0000","last_active":"2023-07-18T03:30:33.661734+0000","last_peered":"2023-07-18T03:30:33.661734+0000","last_clean":"2023-07-18T03:30:33.661734+0000","last_became_active":"2023-07-18T03:30:33.661363+0000","last_became_peered":"2023-07-18T03:30:33.661363+0000","last_unstale":"2023-07-18T03:30:33.661734+0000","last_undegraded":"2023-07-18T03:30:33.661734+0000","last_fullsized":"2023-07-18T03:30:33.661734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:21:26.542417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660448+0000","last_change":"2023-07-18T03:29:26.290810+0000","last_active":"2023-07-18T03:30:33.660448+0000","last_peered":"2023-07-18T03:30:33.660448+0000","last_clean":"2023-07-18T03:30:33.660448+0000","last_became_active":"2023-07-18T03:29:26.290448+0000","last_became_peered":"2023-07-18T03:29:26.290448+0000","last_unstale":"2023-07-18T03:30:33.660448+0000","last_undegraded":"2023-07-18T03:30:33.660448+0000","last_fullsized":"2023-07-18T03:30:33.660448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:37:32.462232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650723+0000","last_change":"2023-07-18T03:29:27.105801+0000","last_active":"2023-07-18T03:30:33.650723+0000","last_peered":"2023-07-18T03:30:33.650723+0000","last_clean":"2023-07-18T03:30:33.650723+0000","last_became_active":"2023-07-18T03:29:27.104715+0000","last_became_peered":"2023-07-18T03:29:27.104715+0000","last_unstale":"2023-07-18T03:30:33.650723+0000","last_undegraded":"2023-07-18T03:30:33.650723+0000","last_fullsized":"2023-07-18T03:30:33.650723+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:40:38.561885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'31","reported_seq":206,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:44.717360+0000","last_change":"2023-07-18T03:29:26.293306+0000","last_active":"2023-07-18T03:30:44.717360+0000","last_peered":"2023-07-18T03:30:44.717360+0000","last_clean":"2023-07-18T03:30:44.717360+0000","last_became_active":"2023-07-18T03:29:26.292946+0000","last_became_peered":"2023-07-18T03:29:26.292946+0000","last_unstale":"2023-07-18T03:30:44.717360+0000","last_undegraded":"2023-07-18T03:30:44.717360+0000","last_fullsized":"2023-07-18T03:30:44.717360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:24:06.931197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58845,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":157,"num_read_kb":135,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.657688+0000","last_change":"2023-07-18T03:29:27.103941+0000","last_active":"2023-07-18T03:30:33.657688+0000","last_peered":"2023-07-18T03:30:33.657688+0000","last_clean":"2023-07-18T03:30:33.657688+0000","last_became_active":"2023-07-18T03:29:27.103506+0000","last_became_peered":"2023-07-18T03:29:27.103506+0000","last_unstale":"2023-07-18T03:30:33.657688+0000","last_undegraded":"2023-07-18T03:30:33.657688+0000","last_fullsized":"2023-07-18T03:30:33.657688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:18:23.410060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.676819+0000","last_change":"2023-07-18T03:30:33.676417+0000","last_active":"2023-07-18T03:30:33.676819+0000","last_peered":"2023-07-18T03:30:33.676819+0000","last_clean":"2023-07-18T03:30:33.676819+0000","last_became_active":"2023-07-18T03:30:33.662637+0000","last_became_peered":"2023-07-18T03:30:33.662637+0000","last_unstale":"2023-07-18T03:30:33.676819+0000","last_undegraded":"2023-07-18T03:30:33.676819+0000","last_fullsized":"2023-07-18T03:30:33.676819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:15:40.468772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.750602+0000","last_change":"2023-07-18T03:30:33.750602+0000","last_active":"2023-07-18T03:30:33.750602+0000","last_peered":"2023-07-18T03:30:33.750602+0000","last_clean":"2023-07-18T03:30:33.750602+0000","last_became_active":"2023-07-18T03:30:33.750170+0000","last_became_peered":"2023-07-18T03:30:33.750170+0000","last_unstale":"2023-07-18T03:30:33.750602+0000","last_undegraded":"2023-07-18T03:30:33.750602+0000","last_fullsized":"2023-07-18T03:30:33.750602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:13:32.425846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.659763+0000","last_change":"2023-07-18T03:29:26.290487+0000","last_active":"2023-07-18T03:30:33.659763+0000","last_peered":"2023-07-18T03:30:33.659763+0000","last_clean":"2023-07-18T03:30:33.659763+0000","last_became_active":"2023-07-18T03:29:26.290115+0000","last_became_peered":"2023-07-18T03:29:26.290115+0000","last_unstale":"2023-07-18T03:30:33.659763+0000","last_undegraded":"2023-07-18T03:30:33.659763+0000","last_fullsized":"2023-07-18T03:30:33.659763+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:23:20.930476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749286+0000","last_change":"2023-07-18T03:29:27.108550+0000","last_active":"2023-07-18T03:30:33.749286+0000","last_peered":"2023-07-18T03:30:33.749286+0000","last_clean":"2023-07-18T03:30:33.749286+0000","last_became_active":"2023-07-18T03:29:27.108069+0000","last_became_peered":"2023-07-18T03:29:27.108069+0000","last_unstale":"2023-07-18T03:30:33.749286+0000","last_undegraded":"2023-07-18T03:30:33.749286+0000","last_fullsized":"2023-07-18T03:30:33.749286+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:37:05.273266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":92,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:39.754936+0000","last_change":"2023-07-18T03:29:26.290896+0000","last_active":"2023-07-18T03:30:39.754936+0000","last_peered":"2023-07-18T03:30:39.754936+0000","last_clean":"2023-07-18T03:30:39.754936+0000","last_became_active":"2023-07-18T03:29:26.290450+0000","last_became_peered":"2023-07-18T03:29:26.290450+0000","last_unstale":"2023-07-18T03:30:39.754936+0000","last_undegraded":"2023-07-18T03:30:39.754936+0000","last_fullsized":"2023-07-18T03:30:39.754936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:30:12.255317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.748869+0000","last_change":"2023-07-18T03:29:27.109184+0000","last_active":"2023-07-18T03:30:33.748869+0000","last_peered":"2023-07-18T03:30:33.748869+0000","last_clean":"2023-07-18T03:30:33.748869+0000","last_became_active":"2023-07-18T03:29:27.107993+0000","last_became_peered":"2023-07-18T03:29:27.107993+0000","last_unstale":"2023-07-18T03:30:33.748869+0000","last_undegraded":"2023-07-18T03:30:33.748869+0000","last_fullsized":"2023-07-18T03:30:33.748869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:09:48.362579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656895+0000","last_change":"2023-07-18T03:29:26.293671+0000","last_active":"2023-07-18T03:30:33.656895+0000","last_peered":"2023-07-18T03:30:33.656895+0000","last_clean":"2023-07-18T03:30:33.656895+0000","last_became_active":"2023-07-18T03:29:26.293205+0000","last_became_peered":"2023-07-18T03:29:26.293205+0000","last_unstale":"2023-07-18T03:30:33.656895+0000","last_undegraded":"2023-07-18T03:30:33.656895+0000","last_fullsized":"2023-07-18T03:30:33.656895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:23:48.250370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.663227+0000","last_change":"2023-07-18T03:30:33.659828+0000","last_active":"2023-07-18T03:30:33.663227+0000","last_peered":"2023-07-18T03:30:33.663227+0000","last_clean":"2023-07-18T03:30:33.663227+0000","last_became_active":"2023-07-18T03:30:33.659440+0000","last_became_peered":"2023-07-18T03:30:33.659440+0000","last_unstale":"2023-07-18T03:30:33.663227+0000","last_undegraded":"2023-07-18T03:30:33.663227+0000","last_fullsized":"2023-07-18T03:30:33.663227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:21:00.901622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.667448+0000","last_change":"2023-07-18T03:30:33.666902+0000","last_active":"2023-07-18T03:30:33.667448+0000","last_peered":"2023-07-18T03:30:33.667448+0000","last_clean":"2023-07-18T03:30:33.667448+0000","last_became_active":"2023-07-18T03:30:33.661326+0000","last_became_peered":"2023-07-18T03:30:33.661326+0000","last_unstale":"2023-07-18T03:30:33.667448+0000","last_undegraded":"2023-07-18T03:30:33.667448+0000","last_fullsized":"2023-07-18T03:30:33.667448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:12:15.691642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656030+0000","last_change":"2023-07-18T03:29:27.098479+0000","last_active":"2023-07-18T03:30:33.656030+0000","last_peered":"2023-07-18T03:30:33.656030+0000","last_clean":"2023-07-18T03:30:33.656030+0000","last_became_active":"2023-07-18T03:29:27.098153+0000","last_became_peered":"2023-07-18T03:29:27.098153+0000","last_unstale":"2023-07-18T03:30:33.656030+0000","last_undegraded":"2023-07-18T03:30:33.656030+0000","last_fullsized":"2023-07-18T03:30:33.656030+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:53:21.610571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.652236+0000","last_change":"2023-07-18T03:29:26.290953+0000","last_active":"2023-07-18T03:30:33.652236+0000","last_peered":"2023-07-18T03:30:33.652236+0000","last_clean":"2023-07-18T03:30:33.652236+0000","last_became_active":"2023-07-18T03:29:26.290633+0000","last_became_peered":"2023-07-18T03:29:26.290633+0000","last_unstale":"2023-07-18T03:30:33.652236+0000","last_undegraded":"2023-07-18T03:30:33.652236+0000","last_fullsized":"2023-07-18T03:30:33.652236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:43:09.103910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661197+0000","last_change":"2023-07-18T03:29:27.093651+0000","last_active":"2023-07-18T03:30:33.661197+0000","last_peered":"2023-07-18T03:30:33.661197+0000","last_clean":"2023-07-18T03:30:33.661197+0000","last_became_active":"2023-07-18T03:29:27.093307+0000","last_became_peered":"2023-07-18T03:29:27.093307+0000","last_unstale":"2023-07-18T03:30:33.661197+0000","last_undegraded":"2023-07-18T03:30:33.661197+0000","last_fullsized":"2023-07-18T03:30:33.661197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:10:55.332949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":92,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:40.757472+0000","last_change":"2023-07-18T03:29:26.287405+0000","last_active":"2023-07-18T03:30:40.757472+0000","last_peered":"2023-07-18T03:30:40.757472+0000","last_clean":"2023-07-18T03:30:40.757472+0000","last_became_active":"2023-07-18T03:29:26.287048+0000","last_became_peered":"2023-07-18T03:29:26.287048+0000","last_unstale":"2023-07-18T03:30:40.757472+0000","last_undegraded":"2023-07-18T03:30:40.757472+0000","last_fullsized":"2023-07-18T03:30:40.757472+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:46:08.045548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661021+0000","last_change":"2023-07-18T03:29:27.103218+0000","last_active":"2023-07-18T03:30:33.661021+0000","last_peered":"2023-07-18T03:30:33.661021+0000","last_clean":"2023-07-18T03:30:33.661021+0000","last_became_active":"2023-07-18T03:29:27.102917+0000","last_became_peered":"2023-07-18T03:29:27.102917+0000","last_unstale":"2023-07-18T03:30:33.661021+0000","last_undegraded":"2023-07-18T03:30:33.661021+0000","last_fullsized":"2023-07-18T03:30:33.661021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:15:09.570588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.659924+0000","last_change":"2023-07-18T03:29:27.098696+0000","last_active":"2023-07-18T03:30:33.659924+0000","last_peered":"2023-07-18T03:30:33.659924+0000","last_clean":"2023-07-18T03:30:33.659924+0000","last_became_active":"2023-07-18T03:29:27.098253+0000","last_became_peered":"2023-07-18T03:29:27.098253+0000","last_unstale":"2023-07-18T03:30:33.659924+0000","last_undegraded":"2023-07-18T03:30:33.659924+0000","last_fullsized":"2023-07-18T03:30:33.659924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:31:51.020473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:39.754302+0000","last_change":"2023-07-18T03:29:26.290308+0000","last_active":"2023-07-18T03:30:39.754302+0000","last_peered":"2023-07-18T03:30:39.754302+0000","last_clean":"2023-07-18T03:30:39.754302+0000","last_became_active":"2023-07-18T03:29:26.289937+0000","last_became_peered":"2023-07-18T03:29:26.289937+0000","last_unstale":"2023-07-18T03:30:39.754302+0000","last_undegraded":"2023-07-18T03:30:39.754302+0000","last_fullsized":"2023-07-18T03:30:39.754302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:48:40.209353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661087+0000","last_change":"2023-07-18T03:29:27.096469+0000","last_active":"2023-07-18T03:30:33.661087+0000","last_peered":"2023-07-18T03:30:33.661087+0000","last_clean":"2023-07-18T03:30:33.661087+0000","last_became_active":"2023-07-18T03:29:27.096171+0000","last_became_peered":"2023-07-18T03:29:27.096171+0000","last_unstale":"2023-07-18T03:30:33.661087+0000","last_undegraded":"2023-07-18T03:30:33.661087+0000","last_fullsized":"2023-07-18T03:30:33.661087+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:12:58.040917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.658966+0000","last_change":"2023-07-18T03:29:26.290935+0000","last_active":"2023-07-18T03:30:33.658966+0000","last_peered":"2023-07-18T03:30:33.658966+0000","last_clean":"2023-07-18T03:30:33.658966+0000","last_became_active":"2023-07-18T03:29:26.290595+0000","last_became_peered":"2023-07-18T03:29:26.290595+0000","last_unstale":"2023-07-18T03:30:33.658966+0000","last_undegraded":"2023-07-18T03:30:33.658966+0000","last_fullsized":"2023-07-18T03:30:33.658966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:48:44.101957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":335,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.748928+0000","last_change":"2023-07-18T03:28:28.122385+0000","last_active":"2023-07-18T03:30:33.748928+0000","last_peered":"2023-07-18T03:30:33.748928+0000","last_clean":"2023-07-18T03:30:33.748928+0000","last_became_active":"2023-07-18T03:28:28.121932+0000","last_became_peered":"2023-07-18T03:28:28.121932+0000","last_unstale":"2023-07-18T03:30:33.748928+0000","last_undegraded":"2023-07-18T03:30:33.748928+0000","last_fullsized":"2023-07-18T03:30:33.748928+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:25:38.267627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:25:38.267627+0000","last_clean_scrub_stamp":"2023-07-18T03:25:38.267627+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:04:59.247851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655491+0000","last_change":"2023-07-18T03:29:27.101582+0000","last_active":"2023-07-18T03:30:33.655491+0000","last_peered":"2023-07-18T03:30:33.655491+0000","last_clean":"2023-07-18T03:30:33.655491+0000","last_became_active":"2023-07-18T03:29:27.101226+0000","last_became_peered":"2023-07-18T03:29:27.101226+0000","last_unstale":"2023-07-18T03:30:33.655491+0000","last_undegraded":"2023-07-18T03:30:33.655491+0000","last_fullsized":"2023-07-18T03:30:33.655491+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:24:53.297066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749186+0000","last_change":"2023-07-18T03:29:26.291787+0000","last_active":"2023-07-18T03:30:33.749186+0000","last_peered":"2023-07-18T03:30:33.749186+0000","last_clean":"2023-07-18T03:30:33.749186+0000","last_became_active":"2023-07-18T03:29:26.291471+0000","last_became_peered":"2023-07-18T03:29:26.291471+0000","last_unstale":"2023-07-18T03:30:33.749186+0000","last_undegraded":"2023-07-18T03:30:33.749186+0000","last_fullsized":"2023-07-18T03:30:33.749186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:03:33.650313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.657936+0000","last_change":"2023-07-18T03:29:26.296936+0000","last_active":"2023-07-18T03:30:33.657936+0000","last_peered":"2023-07-18T03:30:33.657936+0000","last_clean":"2023-07-18T03:30:33.657936+0000","last_became_active":"2023-07-18T03:29:26.296241+0000","last_became_peered":"2023-07-18T03:29:26.296241+0000","last_unstale":"2023-07-18T03:30:33.657936+0000","last_undegraded":"2023-07-18T03:30:33.657936+0000","last_fullsized":"2023-07-18T03:30:33.657936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:51:59.499717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656549+0000","last_change":"2023-07-18T03:29:27.102715+0000","last_active":"2023-07-18T03:30:33.656549+0000","last_peered":"2023-07-18T03:30:33.656549+0000","last_clean":"2023-07-18T03:30:33.656549+0000","last_became_active":"2023-07-18T03:29:27.100914+0000","last_became_peered":"2023-07-18T03:29:27.100914+0000","last_unstale":"2023-07-18T03:30:33.656549+0000","last_undegraded":"2023-07-18T03:30:33.656549+0000","last_fullsized":"2023-07-18T03:30:33.656549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:09:20.580139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'9","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.652605+0000","last_change":"2023-07-18T03:29:26.290231+0000","last_active":"2023-07-18T03:30:33.652605+0000","last_peered":"2023-07-18T03:30:33.652605+0000","last_clean":"2023-07-18T03:30:33.652605+0000","last_became_active":"2023-07-18T03:29:26.289753+0000","last_became_peered":"2023-07-18T03:29:26.289753+0000","last_unstale":"2023-07-18T03:30:33.652605+0000","last_undegraded":"2023-07-18T03:30:33.652605+0000","last_fullsized":"2023-07-18T03:30:33.652605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:37:14.149936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661595+0000","last_change":"2023-07-18T03:29:27.103964+0000","last_active":"2023-07-18T03:30:33.661595+0000","last_peered":"2023-07-18T03:30:33.661595+0000","last_clean":"2023-07-18T03:30:33.661595+0000","last_became_active":"2023-07-18T03:29:27.103660+0000","last_became_peered":"2023-07-18T03:29:27.103660+0000","last_unstale":"2023-07-18T03:30:33.661595+0000","last_undegraded":"2023-07-18T03:30:33.661595+0000","last_fullsized":"2023-07-18T03:30:33.661595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:24:11.718523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.659213+0000","last_change":"2023-07-18T03:29:26.286436+0000","last_active":"2023-07-18T03:30:33.659213+0000","last_peered":"2023-07-18T03:30:33.659213+0000","last_clean":"2023-07-18T03:30:33.659213+0000","last_became_active":"2023-07-18T03:29:26.286070+0000","last_became_peered":"2023-07-18T03:29:26.286070+0000","last_unstale":"2023-07-18T03:30:33.659213+0000","last_undegraded":"2023-07-18T03:30:33.659213+0000","last_fullsized":"2023-07-18T03:30:33.659213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:49:30.399461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.750552+0000","last_change":"2023-07-18T03:30:33.750552+0000","last_active":"2023-07-18T03:30:33.750552+0000","last_peered":"2023-07-18T03:30:33.750552+0000","last_clean":"2023-07-18T03:30:33.750552+0000","last_became_active":"2023-07-18T03:30:33.750210+0000","last_became_peered":"2023-07-18T03:30:33.750210+0000","last_unstale":"2023-07-18T03:30:33.750552+0000","last_undegraded":"2023-07-18T03:30:33.750552+0000","last_fullsized":"2023-07-18T03:30:33.750552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:30:46.022642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650623+0000","last_change":"2023-07-18T03:29:26.289628+0000","last_active":"2023-07-18T03:30:33.650623+0000","last_peered":"2023-07-18T03:30:33.650623+0000","last_clean":"2023-07-18T03:30:33.650623+0000","last_became_active":"2023-07-18T03:29:26.289013+0000","last_became_peered":"2023-07-18T03:29:26.289013+0000","last_unstale":"2023-07-18T03:30:33.650623+0000","last_undegraded":"2023-07-18T03:30:33.650623+0000","last_fullsized":"2023-07-18T03:30:33.650623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:10:03.968656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656945+0000","last_change":"2023-07-18T03:29:27.090943+0000","last_active":"2023-07-18T03:30:33.656945+0000","last_peered":"2023-07-18T03:30:33.656945+0000","last_clean":"2023-07-18T03:30:33.656945+0000","last_became_active":"2023-07-18T03:29:27.090465+0000","last_became_peered":"2023-07-18T03:29:27.090465+0000","last_unstale":"2023-07-18T03:30:33.656945+0000","last_undegraded":"2023-07-18T03:30:33.656945+0000","last_fullsized":"2023-07-18T03:30:33.656945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:26:40.736484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660770+0000","last_change":"2023-07-18T03:29:26.288576+0000","last_active":"2023-07-18T03:30:33.660770+0000","last_peered":"2023-07-18T03:30:33.660770+0000","last_clean":"2023-07-18T03:30:33.660770+0000","last_became_active":"2023-07-18T03:29:26.287514+0000","last_became_peered":"2023-07-18T03:29:26.287514+0000","last_unstale":"2023-07-18T03:30:33.660770+0000","last_undegraded":"2023-07-18T03:30:33.660770+0000","last_fullsized":"2023-07-18T03:30:33.660770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:35:34.080524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656181+0000","last_change":"2023-07-18T03:29:27.101111+0000","last_active":"2023-07-18T03:30:33.656181+0000","last_peered":"2023-07-18T03:30:33.656181+0000","last_clean":"2023-07-18T03:30:33.656181+0000","last_became_active":"2023-07-18T03:29:27.100737+0000","last_became_peered":"2023-07-18T03:29:27.100737+0000","last_unstale":"2023-07-18T03:30:33.656181+0000","last_undegraded":"2023-07-18T03:30:33.656181+0000","last_fullsized":"2023-07-18T03:30:33.656181+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:17:22.437647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.654205+0000","last_change":"2023-07-18T03:29:26.296974+0000","last_active":"2023-07-18T03:30:33.654205+0000","last_peered":"2023-07-18T03:30:33.654205+0000","last_clean":"2023-07-18T03:30:33.654205+0000","last_became_active":"2023-07-18T03:29:26.296451+0000","last_became_peered":"2023-07-18T03:29:26.296451+0000","last_unstale":"2023-07-18T03:30:33.654205+0000","last_undegraded":"2023-07-18T03:30:33.654205+0000","last_fullsized":"2023-07-18T03:30:33.654205+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:48:52.975744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661117+0000","last_change":"2023-07-18T03:29:27.095857+0000","last_active":"2023-07-18T03:30:33.661117+0000","last_peered":"2023-07-18T03:30:33.661117+0000","last_clean":"2023-07-18T03:30:33.661117+0000","last_became_active":"2023-07-18T03:29:27.095553+0000","last_became_peered":"2023-07-18T03:29:27.095553+0000","last_unstale":"2023-07-18T03:30:33.661117+0000","last_undegraded":"2023-07-18T03:30:33.661117+0000","last_fullsized":"2023-07-18T03:30:33.661117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:22:26.560346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.657524+0000","last_change":"2023-07-18T03:29:26.295936+0000","last_active":"2023-07-18T03:30:33.657524+0000","last_peered":"2023-07-18T03:30:33.657524+0000","last_clean":"2023-07-18T03:30:33.657524+0000","last_became_active":"2023-07-18T03:29:26.295467+0000","last_became_peered":"2023-07-18T03:29:26.295467+0000","last_unstale":"2023-07-18T03:30:33.657524+0000","last_undegraded":"2023-07-18T03:30:33.657524+0000","last_fullsized":"2023-07-18T03:30:33.657524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:23:28.043840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749424+0000","last_change":"2023-07-18T03:29:27.108528+0000","last_active":"2023-07-18T03:30:33.749424+0000","last_peered":"2023-07-18T03:30:33.749424+0000","last_clean":"2023-07-18T03:30:33.749424+0000","last_became_active":"2023-07-18T03:29:27.108039+0000","last_became_peered":"2023-07-18T03:29:27.108039+0000","last_unstale":"2023-07-18T03:30:33.749424+0000","last_undegraded":"2023-07-18T03:30:33.749424+0000","last_fullsized":"2023-07-18T03:30:33.749424+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:13:41.092469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'5","reported_seq":94,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:41.764673+0000","last_change":"2023-07-18T03:29:26.294006+0000","last_active":"2023-07-18T03:30:41.764673+0000","last_peered":"2023-07-18T03:30:41.764673+0000","last_clean":"2023-07-18T03:30:41.764673+0000","last_became_active":"2023-07-18T03:29:26.293700+0000","last_became_peered":"2023-07-18T03:29:26.293700+0000","last_unstale":"2023-07-18T03:30:41.764673+0000","last_undegraded":"2023-07-18T03:30:41.764673+0000","last_fullsized":"2023-07-18T03:30:41.764673+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:41:32.418658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749567+0000","last_change":"2023-07-18T03:29:27.102764+0000","last_active":"2023-07-18T03:30:33.749567+0000","last_peered":"2023-07-18T03:30:33.749567+0000","last_clean":"2023-07-18T03:30:33.749567+0000","last_became_active":"2023-07-18T03:29:27.102200+0000","last_became_peered":"2023-07-18T03:29:27.102200+0000","last_unstale":"2023-07-18T03:30:33.749567+0000","last_undegraded":"2023-07-18T03:30:33.749567+0000","last_fullsized":"2023-07-18T03:30:33.749567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:49:22.828462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660299+0000","last_change":"2023-07-18T03:29:26.294271+0000","last_active":"2023-07-18T03:30:33.660299+0000","last_peered":"2023-07-18T03:30:33.660299+0000","last_clean":"2023-07-18T03:30:33.660299+0000","last_became_active":"2023-07-18T03:29:26.293938+0000","last_became_peered":"2023-07-18T03:29:26.293938+0000","last_unstale":"2023-07-18T03:30:33.660299+0000","last_undegraded":"2023-07-18T03:30:33.660299+0000","last_fullsized":"2023-07-18T03:30:33.660299+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:59:22.557498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749202+0000","last_change":"2023-07-18T03:29:27.106510+0000","last_active":"2023-07-18T03:30:33.749202+0000","last_peered":"2023-07-18T03:30:33.749202+0000","last_clean":"2023-07-18T03:30:33.749202+0000","last_became_active":"2023-07-18T03:29:27.106222+0000","last_became_peered":"2023-07-18T03:29:27.106222+0000","last_unstale":"2023-07-18T03:30:33.749202+0000","last_undegraded":"2023-07-18T03:30:33.749202+0000","last_fullsized":"2023-07-18T03:30:33.749202+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:08:13.558332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656074+0000","last_change":"2023-07-18T03:29:26.293583+0000","last_active":"2023-07-18T03:30:33.656074+0000","last_peered":"2023-07-18T03:30:33.656074+0000","last_clean":"2023-07-18T03:30:33.656074+0000","last_became_active":"2023-07-18T03:29:26.293076+0000","last_became_peered":"2023-07-18T03:29:26.293076+0000","last_unstale":"2023-07-18T03:30:33.656074+0000","last_undegraded":"2023-07-18T03:30:33.656074+0000","last_fullsized":"2023-07-18T03:30:33.656074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:58:56.281865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749636+0000","last_change":"2023-07-18T03:29:27.109175+0000","last_active":"2023-07-18T03:30:33.749636+0000","last_peered":"2023-07-18T03:30:33.749636+0000","last_clean":"2023-07-18T03:30:33.749636+0000","last_became_active":"2023-07-18T03:29:27.108669+0000","last_became_peered":"2023-07-18T03:29:27.108669+0000","last_unstale":"2023-07-18T03:30:33.749636+0000","last_undegraded":"2023-07-18T03:30:33.749636+0000","last_fullsized":"2023-07-18T03:30:33.749636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:34:18.667617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656335+0000","last_change":"2023-07-18T03:29:27.099770+0000","last_active":"2023-07-18T03:30:33.656335+0000","last_peered":"2023-07-18T03:30:33.656335+0000","last_clean":"2023-07-18T03:30:33.656335+0000","last_became_active":"2023-07-18T03:29:27.099487+0000","last_became_peered":"2023-07-18T03:29:27.099487+0000","last_unstale":"2023-07-18T03:30:33.656335+0000","last_undegraded":"2023-07-18T03:30:33.656335+0000","last_fullsized":"2023-07-18T03:30:33.656335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:13:59.713331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.664744+0000","last_change":"2023-07-18T03:30:33.664744+0000","last_active":"2023-07-18T03:30:33.664744+0000","last_peered":"2023-07-18T03:30:33.664744+0000","last_clean":"2023-07-18T03:30:33.664744+0000","last_became_active":"2023-07-18T03:30:33.664378+0000","last_became_peered":"2023-07-18T03:30:33.664378+0000","last_unstale":"2023-07-18T03:30:33.664744+0000","last_undegraded":"2023-07-18T03:30:33.664744+0000","last_fullsized":"2023-07-18T03:30:33.664744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:46:59.976803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.653582+0000","last_change":"2023-07-18T03:29:27.099196+0000","last_active":"2023-07-18T03:30:33.653582+0000","last_peered":"2023-07-18T03:30:33.653582+0000","last_clean":"2023-07-18T03:30:33.653582+0000","last_became_active":"2023-07-18T03:29:27.098903+0000","last_became_peered":"2023-07-18T03:29:27.098903+0000","last_unstale":"2023-07-18T03:30:33.653582+0000","last_undegraded":"2023-07-18T03:30:33.653582+0000","last_fullsized":"2023-07-18T03:30:33.653582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:43:55.170768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660224+0000","last_change":"2023-07-18T03:30:33.660224+0000","last_active":"2023-07-18T03:30:33.660224+0000","last_peered":"2023-07-18T03:30:33.660224+0000","last_clean":"2023-07-18T03:30:33.660224+0000","last_became_active":"2023-07-18T03:30:33.659847+0000","last_became_peered":"2023-07-18T03:30:33.659847+0000","last_unstale":"2023-07-18T03:30:33.660224+0000","last_undegraded":"2023-07-18T03:30:33.660224+0000","last_fullsized":"2023-07-18T03:30:33.660224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:08:47.876757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656416+0000","last_change":"2023-07-18T03:29:27.101610+0000","last_active":"2023-07-18T03:30:33.656416+0000","last_peered":"2023-07-18T03:30:33.656416+0000","last_clean":"2023-07-18T03:30:33.656416+0000","last_became_active":"2023-07-18T03:29:27.101332+0000","last_became_peered":"2023-07-18T03:29:27.101332+0000","last_unstale":"2023-07-18T03:30:33.656416+0000","last_undegraded":"2023-07-18T03:30:33.656416+0000","last_fullsized":"2023-07-18T03:30:33.656416+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:15:53.329517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656022+0000","last_change":"2023-07-18T03:29:27.093345+0000","last_active":"2023-07-18T03:30:33.656022+0000","last_peered":"2023-07-18T03:30:33.656022+0000","last_clean":"2023-07-18T03:30:33.656022+0000","last_became_active":"2023-07-18T03:29:27.093048+0000","last_became_peered":"2023-07-18T03:29:27.093048+0000","last_unstale":"2023-07-18T03:30:33.656022+0000","last_undegraded":"2023-07-18T03:30:33.656022+0000","last_fullsized":"2023-07-18T03:30:33.656022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:43:10.411014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.654001+0000","last_change":"2023-07-18T03:29:27.098311+0000","last_active":"2023-07-18T03:30:33.654001+0000","last_peered":"2023-07-18T03:30:33.654001+0000","last_clean":"2023-07-18T03:30:33.654001+0000","last_became_active":"2023-07-18T03:29:27.097980+0000","last_became_peered":"2023-07-18T03:29:27.097980+0000","last_unstale":"2023-07-18T03:30:33.654001+0000","last_undegraded":"2023-07-18T03:30:33.654001+0000","last_fullsized":"2023-07-18T03:30:33.654001+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:18:24.177425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.663242+0000","last_change":"2023-07-18T03:30:33.660269+0000","last_active":"2023-07-18T03:30:33.663242+0000","last_peered":"2023-07-18T03:30:33.663242+0000","last_clean":"2023-07-18T03:30:33.663242+0000","last_became_active":"2023-07-18T03:30:33.659999+0000","last_became_peered":"2023-07-18T03:30:33.659999+0000","last_unstale":"2023-07-18T03:30:33.663242+0000","last_undegraded":"2023-07-18T03:30:33.663242+0000","last_fullsized":"2023-07-18T03:30:33.663242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:26:07.126950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660153+0000","last_change":"2023-07-18T03:29:27.098541+0000","last_active":"2023-07-18T03:30:33.660153+0000","last_peered":"2023-07-18T03:30:33.660153+0000","last_clean":"2023-07-18T03:30:33.660153+0000","last_became_active":"2023-07-18T03:29:27.097824+0000","last_became_peered":"2023-07-18T03:29:27.097824+0000","last_unstale":"2023-07-18T03:30:33.660153+0000","last_undegraded":"2023-07-18T03:30:33.660153+0000","last_fullsized":"2023-07-18T03:30:33.660153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:57:42.609243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660576+0000","last_change":"2023-07-18T03:29:27.096983+0000","last_active":"2023-07-18T03:30:33.660576+0000","last_peered":"2023-07-18T03:30:33.660576+0000","last_clean":"2023-07-18T03:30:33.660576+0000","last_became_active":"2023-07-18T03:29:27.096632+0000","last_became_peered":"2023-07-18T03:29:27.096632+0000","last_unstale":"2023-07-18T03:30:33.660576+0000","last_undegraded":"2023-07-18T03:30:33.660576+0000","last_fullsized":"2023-07-18T03:30:33.660576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:28:14.181259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650674+0000","last_change":"2023-07-18T03:29:27.105236+0000","last_active":"2023-07-18T03:30:33.650674+0000","last_peered":"2023-07-18T03:30:33.650674+0000","last_clean":"2023-07-18T03:30:33.650674+0000","last_became_active":"2023-07-18T03:29:27.104906+0000","last_became_peered":"2023-07-18T03:29:27.104906+0000","last_unstale":"2023-07-18T03:30:33.650674+0000","last_undegraded":"2023-07-18T03:30:33.650674+0000","last_fullsized":"2023-07-18T03:30:33.650674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:14:25.383609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656158+0000","last_change":"2023-07-18T03:29:27.103461+0000","last_active":"2023-07-18T03:30:33.656158+0000","last_peered":"2023-07-18T03:30:33.656158+0000","last_clean":"2023-07-18T03:30:33.656158+0000","last_became_active":"2023-07-18T03:29:27.103162+0000","last_became_peered":"2023-07-18T03:29:27.103162+0000","last_unstale":"2023-07-18T03:30:33.656158+0000","last_undegraded":"2023-07-18T03:30:33.656158+0000","last_fullsized":"2023-07-18T03:30:33.656158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:36:23.614131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656048+0000","last_change":"2023-07-18T03:29:27.095300+0000","last_active":"2023-07-18T03:30:33.656048+0000","last_peered":"2023-07-18T03:30:33.656048+0000","last_clean":"2023-07-18T03:30:33.656048+0000","last_became_active":"2023-07-18T03:29:27.094985+0000","last_became_peered":"2023-07-18T03:29:27.094985+0000","last_unstale":"2023-07-18T03:30:33.656048+0000","last_undegraded":"2023-07-18T03:30:33.656048+0000","last_fullsized":"2023-07-18T03:30:33.656048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:37:22.084325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65465,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":745,"num_read_kb":449,"num_write":232,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":195759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514523,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":436016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":0.39500000000000002}]},{"osd":1,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.61499999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.434}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":4,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.63900000000000001}]},{"osd":5,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.56999999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.54400000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.623},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.71499999999999997}]},{"osd":8,"last update":"Tue Jul 18 03:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.754},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.70099999999999996}]},{"osd":9,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.78500000000000003}]},{"osd":10,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.77200000000000002}]}]},{"osd":10,"up_from":69,"seq":296352743456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1613824,"data_stored":1085806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53800000000000003,"15min":0.54600000000000004},"min":{"1min":0.35799999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70899999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.35899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78000000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41399999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66300000000000003,"15min":0.66800000000000004},"min":{"1min":0.42899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91500000000000004,"5min":1.302,"15min":1.302},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.71099999999999997},"min":{"1min":0.54600000000000004,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93999999999999995,"5min":1,"15min":1},"last":0.81499999999999995}]},{"osd":3,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.68000000000000005,"15min":0.68899999999999995},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.876,"5min":1.276,"15min":1.276},"last":0.495},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.72899999999999998,"15min":0.73399999999999999},"min":{"1min":0.436,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":0.48399999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65700000000000003,"15min":0.66100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.89300000000000002,"5min":1.119,"15min":1.119},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.33700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.85399999999999998,"5min":1.278,"15min":1.278},"last":0.46200000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.746,"15min":0.752},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.004,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.80400000000000005,"15min":0.81000000000000005},"min":{"1min":0.52400000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0229999999999999,"5min":1.222,"15min":1.222},"last":0.75}]},{"osd":6,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.82799999999999996,"15min":0.83599999999999997},"min":{"1min":0.503,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82699999999999996,"15min":0.83199999999999996},"min":{"1min":0.53600000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.54300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":8,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.86399999999999999,"15min":0.871},"min":{"1min":0.57099999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.107,"5min":1.466,"15min":1.466},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.88300000000000001,"15min":0.89600000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0509999999999999,"5min":1.518,"15min":1.518},"last":0.77600000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.87,"15min":0.875},"min":{"1min":0.59599999999999997,"5min":0.503,"15min":0.503},"max":{"1min":1.169,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.88500000000000001,"15min":0.89900000000000002},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.153,"5min":1.585,"15min":1.585},"last":0.626}]},{"osd":11,"last update":"Tue Jul 18 03:29:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.71499999999999997}]}]},{"osd":9,"up_from":64,"seq":274877906980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":435904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.47099999999999997,"15min":0.48199999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.58999999999999997,"5min":0.747,"15min":0.747},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.47099999999999997,"15min":0.48699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.57199999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.318}]},{"osd":1,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.69599999999999995,"15min":0.71199999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86799999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.61899999999999999,"15min":0.63500000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.746,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46500000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.67300000000000004,"15min":0.69099999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.71899999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.69899999999999995,"15min":0.71399999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81899999999999995,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.66200000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.50600000000000001,"15min":0.52200000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.81699999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.60099999999999998,"15min":0.622},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82499999999999996,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.34899999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.76600000000000001,"15min":0.79300000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.80800000000000005,"5min":1.355,"15min":1.355},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.69299999999999995,"15min":0.71099999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.77100000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.372}]},{"osd":5,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.79100000000000004,"15min":0.81100000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90000000000000002,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.622},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.77800000000000002,"15min":0.79800000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.871,"5min":1.302,"15min":1.302},"last":0.68100000000000005}]},{"osd":6,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.745,"15min":0.77700000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.75600000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.78600000000000003,"15min":0.82199999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85399999999999998,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.69399999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.78800000000000003,"15min":0.80600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89500000000000002,"5min":1.137,"15min":1.137},"last":0.435},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.81999999999999995,"15min":0.84499999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.91000000000000003,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.56499999999999995}]},{"osd":8,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.86799999999999999,"15min":0.89400000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92800000000000005,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.77700000000000002,"15min":0.79200000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.85599999999999998,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.51900000000000002}]},{"osd":10,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.80000000000000004,"15min":0.81899999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94599999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.84399999999999997,"15min":0.86499999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95799999999999996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.76100000000000001}]}]},{"osd":8,"up_from":56,"seq":240518168615,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":435929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.39500000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.747,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57999999999999996,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.49399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.41199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86599999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.77600000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.439,"15min":0.433},"min":{"1min":0.35099999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.499},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.45700000000000002,"15min":0.44700000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.47299999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57899999999999996,"15min":0.57299999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.64600000000000002}]},{"osd":4,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.63600000000000001,"15min":0.62},"min":{"1min":0.42999999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.80900000000000005}]},{"osd":5,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.373,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54500000000000004,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.52000000000000002,"5min":0.437,"15min":0.437},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69599999999999995,"15min":0.69399999999999995},"min":{"1min":0.49399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89200000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.82499999999999996}]},{"osd":7,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.5,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.47099999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.83999999999999997}]},{"osd":9,"last update":"Tue Jul 18 03:30:05 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.752,"15min":0.72999999999999998},"min":{"1min":0.56499999999999995,"5min":0.505,"15min":0.505},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.78300000000000003,"15min":0.77300000000000002},"min":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.91300000000000003}]},{"osd":10,"last update":"Tue Jul 18 03:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.81299999999999994,"15min":0.79800000000000004},"min":{"1min":0.69099999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.80900000000000005,"15min":0.79100000000000004},"min":{"1min":0.67200000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.145,"5min":1.208,"15min":1.208},"last":0.89900000000000002}]},{"osd":11,"last update":"Tue Jul 18 03:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.93200000000000005}]}]},{"osd":7,"up_from":49,"seq":210453397547,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1531904,"data_stored":1028661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57799999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.74199999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46600000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60799999999999998,"15min":0.59199999999999997},"min":{"1min":0.434,"5min":0.379,"15min":0.379},"max":{"1min":0.86599999999999999,"5min":0.872,"15min":0.872},"last":0.751},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63400000000000001,"15min":0.61399999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83499999999999996,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.78900000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.45600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.878,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.65400000000000003,"15min":0.65500000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55700000000000005}]},{"osd":4,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56899999999999995,"15min":0.55800000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.74399999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76500000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.498}]},{"osd":5,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67600000000000005,"15min":0.66200000000000003},"min":{"1min":0.499,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88600000000000001,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.45100000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.88300000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.74099999999999999,"15min":0.73299999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.76600000000000001,"15min":0.73299999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":6.1909999999999998,"5min":6.1909999999999998,"15min":6.1909999999999998},"last":0.44800000000000001}]},{"osd":8,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.80400000000000005,"15min":0.77600000000000002},"min":{"1min":0.51300000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":4.5880000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.76900000000000002,"15min":0.73299999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.6050000000000004,"5min":4.6050000000000004,"15min":4.6050000000000004},"last":0.84799999999999998}]},{"osd":9,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":0.86199999999999999,"15min":0.80700000000000005},"min":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":9.8439999999999994,"5min":9.8439999999999994,"15min":9.8439999999999994},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81899999999999995,"15min":0.81499999999999995},"min":{"1min":0.629,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.72199999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79500000000000004,"15min":0.79900000000000004},"min":{"1min":0.54100000000000004,"5min":0.432,"15min":0.432},"max":{"1min":1,"5min":1.171,"15min":1.171},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85199999999999998,"15min":0.86199999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0249999999999999,"5min":1.085,"15min":1.085},"last":0.83599999999999997}]},{"osd":11,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.874}]}]},{"osd":6,"up_from":44,"seq":188978561072,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":435579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.443,"15min":0.41399999999999998},"min":{"1min":0.27100000000000002,"5min":0.254,"15min":0.254},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.46100000000000002,"15min":0.45000000000000001},"min":{"1min":0.28999999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.2320000000000002,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":0.35699999999999998}]},{"osd":1,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.63900000000000001,"15min":0.58499999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":5.0700000000000003,"5min":5.0700000000000003,"15min":5.0700000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.64100000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86699999999999999,"5min":1.008,"15min":1.008},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.68999999999999995,"15min":0.63800000000000001},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":5.0300000000000002,"5min":5.0300000000000002,"15min":5.0300000000000002},"last":0.52500000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.55900000000000005,"15min":0.50900000000000001},"min":{"1min":0.32100000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":5.2089999999999996,"5min":5.2089999999999996,"15min":5.2089999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54500000000000004,"15min":0.53700000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.39000000000000001}]},{"osd":4,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.623,"15min":0.61399999999999999},"min":{"1min":0.44,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86599999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.73199999999999998,"15min":0.68700000000000006},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":5.1070000000000002,"5min":5.1070000000000002,"15min":5.1070000000000002},"last":0.73299999999999998}]},{"osd":5,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67800000000000005,"15min":0.67000000000000004},"min":{"1min":0.45300000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94599999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.73299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.83599999999999997,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.65700000000000003}]},{"osd":7,"last update":"Tue Jul 18 03:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.71799999999999997,"15min":0.72299999999999998},"min":{"1min":0.50900000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.91400000000000003,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.80300000000000005,"15min":0.78300000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.9699999999999998,"5min":4.9699999999999998,"15min":4.9699999999999998},"last":0.67000000000000004}]},{"osd":8,"last update":"Tue Jul 18 03:29:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.86399999999999999,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":5.1760000000000002,"5min":5.1760000000000002,"15min":5.1760000000000002},"last":0.627},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.77400000000000002,"15min":0.78000000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.74399999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.75700000000000001,"15min":0.76600000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.81999999999999995,"15min":0.79800000000000004},"min":{"1min":0.438,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":5.2409999999999997,"5min":5.2409999999999997,"15min":5.2409999999999997},"last":0.57199999999999995}]},{"osd":10,"last update":"Tue Jul 18 03:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.80400000000000005,"15min":0.82099999999999995},"min":{"1min":0.63,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.91500000000000004,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.022,"5min":0.84799999999999998,"15min":0.81899999999999995},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":5.141,"5min":5.141,"15min":5.141},"last":0.77400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":495013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.55800000000000005,"15min":0.49399999999999999},"min":{"1min":0.375,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.55900000000000005,"15min":0.51400000000000001},"min":{"1min":0.35399999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.48299999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58699999999999997,"15min":0.54800000000000004},"min":{"1min":0.50600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88400000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56699999999999995,"15min":0.52500000000000002},"min":{"1min":0.51200000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.86299999999999999}]},{"osd":3,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.626,"15min":0.56599999999999995},"min":{"1min":0.41599999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.4710000000000001,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":0.441},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.65200000000000002,"15min":0.59199999999999997},"min":{"1min":0.436,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":0.61299999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.49299999999999999,"15min":0.42299999999999999},"min":{"1min":0.34699999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.48099999999999998,"15min":0.432},"min":{"1min":0.374,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79500000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59799999999999998}]},{"osd":5,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.63600000000000001,"15min":0.58899999999999997},"min":{"1min":0.53000000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.66400000000000003,"15min":0.622},"min":{"1min":0.56799999999999995,"5min":0.433,"15min":0.433},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.92200000000000004}]},{"osd":6,"last update":"Tue Jul 18 03:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.66700000000000004,"15min":0.629},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.71499999999999997,"15min":0.67900000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.752}]},{"osd":7,"last update":"Tue Jul 18 03:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.67100000000000004,"15min":0.64000000000000001},"min":{"1min":0.53700000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.73099999999999998,"15min":0.68899999999999995},"min":{"1min":0.59199999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0369999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.83999999999999997}]},{"osd":8,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.81299999999999994,"15min":0.80600000000000005},"min":{"1min":0.51900000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.01},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.78900000000000003,"15min":0.77900000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.82799999999999996}]},{"osd":9,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.92400000000000004,"15min":0.91600000000000004},"min":{"1min":0.61299999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.875,"15min":0.86599999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":0.88100000000000001}]},{"osd":10,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.85399999999999998,"15min":0.84799999999999998},"min":{"1min":0.62,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.141,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.56699999999999995,"5min":0.439,"15min":0.439},"max":{"1min":1.159,"5min":1.528,"15min":1.528},"last":0.97299999999999998}]},{"osd":11,"last update":"Tue Jul 18 03:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.95399999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":436912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.60699999999999998,"15min":0.53800000000000003},"min":{"1min":0.32700000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":7.0199999999999996,"5min":7.0199999999999996,"15min":7.0199999999999996},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.50700000000000001},"min":{"1min":0.35299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.5}]},{"osd":2,"last update":"Tue Jul 18 03:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.53600000000000003},"min":{"1min":0.39500000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.58699999999999997,"15min":0.53500000000000003},"min":{"1min":0.41799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.60399999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.373},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":7.274,"5min":7.274,"15min":7.274},"last":0.42899999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58099999999999996,"15min":0.55500000000000005},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.379,"5min":0.378,"15min":0.378},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.52100000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.67500000000000004,"15min":0.63},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":7.1079999999999997,"5min":7.1079999999999997,"15min":7.1079999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.63200000000000001,"15min":0.623},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.57499999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56799999999999995,"15min":0.55100000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.621,"15min":0.57099999999999995},"min":{"1min":0.34799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.2130000000000001,"5min":7.2130000000000001,"15min":7.2130000000000001},"last":0.59399999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.47099999999999997}]},{"osd":8,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.81499999999999995,"15min":0.79200000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.1449999999999996,"5min":7.1449999999999996,"15min":7.1449999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.77000000000000002,"15min":0.79000000000000004},"min":{"1min":0.47999999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96799999999999997,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.64100000000000001}]},{"osd":9,"last update":"Tue Jul 18 03:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":0.85999999999999999,"15min":0.80800000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":12.891,"5min":12.891,"15min":12.891},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.83199999999999996,"15min":0.81100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":7.2439999999999998,"5min":7.2439999999999998,"15min":7.2439999999999998},"last":0.46400000000000002}]},{"osd":10,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.77300000000000002,"15min":0.78800000000000003},"min":{"1min":0.53600000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0680000000000001,"5min":1.177,"15min":1.177},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.83799999999999997,"15min":0.81599999999999995},"min":{"1min":0.5,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":7.0720000000000001,"5min":7.0720000000000001,"15min":7.0720000000000001},"last":0.65600000000000003}]},{"osd":11,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":7.1790000000000003,"5min":7.1790000000000003,"15min":7.1790000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.64800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":495507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.39800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.55500000000000005},"min":{"1min":0.36099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82199999999999995,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.53300000000000003}]},{"osd":1,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78400000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.70099999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61699999999999999,"15min":0.58599999999999997},"min":{"1min":0.48299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.503},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88600000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.65400000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.73399999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.41699999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.45100000000000001,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.88200000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.59299999999999997,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.61599999999999999,"15min":0.58399999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89900000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.81499999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.505,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.61699999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.378,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.94299999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.73299999999999998,"15min":0.69499999999999995},"min":{"1min":0.48599999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":5.8890000000000002,"5min":5.8890000000000002,"15min":5.8890000000000002},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.97399999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.70399999999999996,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.71499999999999997,"15min":0.69299999999999995},"min":{"1min":0.746,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":1.044}]},{"osd":11,"last update":"Tue Jul 18 03:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":1.0209999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":435458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.377,"15min":0.34599999999999997},"min":{"1min":0.26600000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.88500000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.372},"min":{"1min":0.28999999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.30099999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58999999999999997,"15min":0.53500000000000003},"min":{"1min":0.39500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3660000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.51300000000000001},"min":{"1min":0.39100000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.40100000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57799999999999996,"15min":0.55700000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.3080000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61899999999999999,"15min":0.56999999999999995},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3919999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.751}]},{"osd":4,"last update":"Tue Jul 18 03:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.58299999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.93600000000000005,"5min":1.754,"15min":1.754},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66200000000000003,"15min":0.60999999999999999},"min":{"1min":0.39900000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93300000000000005,"5min":1.744,"15min":1.744},"last":0.64200000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.50600000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.357,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.436,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3460000000000001,"5min":1.827,"15min":1.827},"last":0.71699999999999997}]},{"osd":6,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.95199999999999996,"5min":1.002,"15min":1.002},"last":0.435},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.47699999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.66700000000000004,"15min":0.67400000000000004},"min":{"1min":0.41499999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82799999999999996,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.48399999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95999999999999996,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.83999999999999997}]},{"osd":8,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.49399999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.82999999999999996,"15min":0.85199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92500000000000004,"5min":1.113,"15min":1.113},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.74399999999999999,"15min":0.76200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.995,"5min":0.997,"15min":0.997},"last":0.52500000000000002}]},{"osd":10,"last update":"Tue Jul 18 03:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.79100000000000004,"15min":0.81000000000000005},"min":{"1min":0.54000000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.90300000000000002,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.78200000000000003,"15min":0.80500000000000005},"min":{"1min":0.496,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91800000000000004,"5min":1.786,"15min":1.786},"last":0.499}]},{"osd":11,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.74099999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953527,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":435434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53200000000000003,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.54700000000000004,"15min":0.50700000000000001},"min":{"1min":0.48099999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89500000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.502}]},{"osd":1,"last update":"Tue Jul 18 03:30:23 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.42299999999999999,"15min":0.39400000000000002},"min":{"1min":0.28199999999999997,"5min":0.246,"15min":0.246},"max":{"1min":0.66000000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.436,"15min":0.39100000000000001},"min":{"1min":0.36599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67400000000000004,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.53300000000000003}]},{"osd":2,"last update":"Tue Jul 18 03:30:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.64300000000000002,"15min":0.58499999999999996},"min":{"1min":0.438,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60699999999999998,"15min":0.56799999999999995},"min":{"1min":0.46400000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79100000000000004,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.64400000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:30:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61399999999999999,"15min":0.55900000000000005},"min":{"1min":0.46500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.55200000000000005},"min":{"1min":0.40400000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.91200000000000003,"5min":1.038,"15min":1.038},"last":0.81399999999999995}]},{"osd":5,"last update":"Tue Jul 18 03:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63,"15min":0.61399999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89200000000000002,"5min":1.038,"15min":1.038},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.64300000000000002,"15min":0.61599999999999999},"min":{"1min":0.58599999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.75900000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.63700000000000001,"15min":0.60499999999999998},"min":{"1min":0.41599999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.97599999999999998,"5min":1.085,"15min":1.085},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.41699999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.78900000000000003}]},{"osd":7,"last update":"Tue Jul 18 03:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.65400000000000003,"15min":0.63400000000000001},"min":{"1min":0.51300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.99399999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.67000000000000004,"15min":0.64800000000000002},"min":{"1min":0.54200000000000004,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.999,"5min":1.121,"15min":1.121},"last":0.57499999999999996}]},{"osd":8,"last update":"Tue Jul 18 03:29:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71399999999999997,"15min":0.71499999999999997},"min":{"1min":0.53000000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91100000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.78200000000000003,"15min":0.77500000000000002},"min":{"1min":0.59999999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0920000000000001,"5min":1.208,"15min":1.208},"last":1.0149999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.78400000000000003,"15min":0.77000000000000002},"min":{"1min":0.69299999999999995,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.80700000000000005,"15min":0.79500000000000004},"min":{"1min":0.60899999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1379999999999999,"5min":1.163,"15min":1.163},"last":0.96899999999999997}]},{"osd":10,"last update":"Tue Jul 18 03:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.84799999999999998,"15min":0.83899999999999997},"min":{"1min":0.63300000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.073,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.82099999999999995,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.101,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.90800000000000003}]},{"osd":11,"last update":"Tue Jul 18 03:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.042}]}]},{"osd":5,"up_from":37,"seq":158913790004,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1544192,"data_stored":1028435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.78900000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.31900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.72399999999999998}]},{"osd":1,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.54800000000000004,"15min":0.51700000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.55800000000000005,"15min":0.52900000000000003},"min":{"1min":0.41399999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.63500000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.40799999999999997,"15min":0.38400000000000001},"min":{"1min":0.29099999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.43099999999999999,"15min":0.41499999999999998},"min":{"1min":0.29899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71299999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.46500000000000002}]},{"osd":3,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.55900000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75800000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.58099999999999996}]},{"osd":4,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.59699999999999998,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.59899999999999998,"15min":0.56599999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.59699999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:30:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.623},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82599999999999996,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.622}]},{"osd":7,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.496,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94199999999999995,"5min":1.633,"15min":1.633},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.49299999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89100000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.65500000000000003}]},{"osd":8,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85199999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94099999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.76500000000000001}]},{"osd":9,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.76100000000000001,"15min":0.75900000000000001},"min":{"1min":0.52800000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94799999999999995,"5min":1.024,"15min":1.024},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.754,"15min":0.75600000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93200000000000005,"5min":1.008,"15min":1.008},"last":0.69799999999999995}]},{"osd":10,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.80100000000000005,"15min":0.79500000000000004},"min":{"1min":0.69399999999999995,"5min":0.504,"15min":0.504},"max":{"1min":0.98599999999999999,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.069},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77700000000000002,"15min":0.77100000000000002},"min":{"1min":0.67600000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.97299999999999998,"5min":1.548,"15min":1.548},"last":0.73899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:30:47.814 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:30:47.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.975+0000 7fbf72787700 1 -- 172.21.15.40:0/1428426725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf6c100cf0 msgr2=0x7fbf6c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.975+0000 7fbf72787700 1 --2- 172.21.15.40:0/1428426725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf6c100cf0 0x7fbf6c101140 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009990 tx=0x7fbf60009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.975+0000 7fbf72787700 1 -- 172.21.15.40:0/1428426725 shutdown_connections 2023-07-18T03:30:47.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.975+0000 7fbf72787700 1 --2- 172.21.15.40:0/1428426725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf6c101680 0x7fbf6c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.975+0000 7fbf72787700 1 --2- 172.21.15.40:0/1428426725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf6c0ffb00 0x7fbf6c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.975+0000 7fbf72787700 1 --2- 172.21.15.40:0/1428426725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf6c100cf0 0x7fbf6c101140 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.975+0000 7fbf72787700 1 -- 172.21.15.40:0/1428426725 >> 172.21.15.40:0/1428426725 conn(0x7fbf6c0fb100 msgr2=0x7fbf6c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:47.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.975+0000 7fbf72787700 1 -- 172.21.15.40:0/1428426725 shutdown_connections 2023-07-18T03:30:47.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.975+0000 7fbf72787700 1 -- 172.21.15.40:0/1428426725 wait complete. 2023-07-18T03:30:47.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.975+0000 7fbf72787700 1 Processor -- start 2023-07-18T03:30:47.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf72787700 1 -- start start 2023-07-18T03:30:47.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf72787700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf6c0ffb00 0x7fbf6c1957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf72787700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf6c100cf0 0x7fbf6c195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf72787700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf6c101680 0x7fbf6c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf72787700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c19d320 con 0x7fbf6c0ffb00 2023-07-18T03:30:47.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf72787700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c19d460 con 0x7fbf6c101680 2023-07-18T03:30:47.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf72787700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c19d5f0 con 0x7fbf6c100cf0 2023-07-18T03:30:47.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf6bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf6c0ffb00 0x7fbf6c1957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf6bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf6c0ffb00 0x7fbf6c1957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57764/0 (socket says 172.21.15.40:57764) 2023-07-18T03:30:47.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf6bfff700 1 -- 172.21.15.40:0/2190467256 learned_addr learned my addr 172.21.15.40:0/2190467256 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:47.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf70d24700 1 --2- 172.21.15.40:0/2190467256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf6c101680 0x7fbf6c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf6b7fe700 1 --2- 172.21.15.40:0/2190467256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf6c100cf0 0x7fbf6c195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf6bfff700 1 -- 172.21.15.40:0/2190467256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf6c100cf0 msgr2=0x7fbf6c195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf6bfff700 1 --2- 172.21.15.40:0/2190467256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf6c100cf0 0x7fbf6c195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf6bfff700 1 -- 172.21.15.40:0/2190467256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf6c101680 msgr2=0x7fbf6c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:47.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf6bfff700 1 --2- 172.21.15.40:0/2190467256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf6c101680 0x7fbf6c19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:47.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf6bfff700 1 -- 172.21.15.40:0/2190467256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf600095f0 con 0x7fbf6c0ffb00 2023-07-18T03:30:47.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.976+0000 7fbf70d24700 1 --2- 172.21.15.40:0/2190467256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf6c101680 0x7fbf6c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:47.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.977+0000 7fbf6bfff700 1 --2- 172.21.15.40:0/2190467256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf6c0ffb00 0x7fbf6c1957a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00d980 tx=0x7fbf5c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:47.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.977+0000 7fbf697fa700 1 -- 172.21.15.40:0/2190467256 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c00de60 con 0x7fbf6c0ffb00 2023-07-18T03:30:47.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.977+0000 7fbf72787700 1 -- 172.21.15.40:0/2190467256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6c19d8d0 con 0x7fbf6c0ffb00 2023-07-18T03:30:47.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.977+0000 7fbf697fa700 1 -- 172.21.15.40:0/2190467256 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf5c0044e0 con 0x7fbf6c0ffb00 2023-07-18T03:30:47.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.977+0000 7fbf697fa700 1 -- 172.21.15.40:0/2190467256 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c003be0 con 0x7fbf6c0ffb00 2023-07-18T03:30:47.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.977+0000 7fbf72787700 1 -- 172.21.15.40:0/2190467256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6c19dde0 con 0x7fbf6c0ffb00 2023-07-18T03:30:47.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.978+0000 7fbf72787700 1 -- 172.21.15.40:0/2190467256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6c10b2d0 con 0x7fbf6c0ffb00 2023-07-18T03:30:47.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.979+0000 7fbf697fa700 1 -- 172.21.15.40:0/2190467256 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbf5c009bb0 con 0x7fbf6c0ffb00 2023-07-18T03:30:47.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.980+0000 7fbf697fa700 1 --2- 172.21.15.40:0/2190467256 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbf540a1220 0x7fbf540a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:47.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.980+0000 7fbf6b7fe700 1 --2- 172.21.15.40:0/2190467256 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbf540a1220 0x7fbf540a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:47.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.980+0000 7fbf697fa700 1 -- 172.21.15.40:0/2190467256 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbf5c0ccc60 con 0x7fbf6c0ffb00 2023-07-18T03:30:47.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.980+0000 7fbf6b7fe700 1 --2- 172.21.15.40:0/2190467256 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbf540a1220 0x7fbf540a36d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbf60006010 tx=0x7fbf6000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:47.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:47.983+0000 7fbf697fa700 1 -- 172.21.15.40:0/2190467256 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbf5c0961d0 con 0x7fbf6c0ffb00 2023-07-18T03:30:48.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:47 smithi134 ceph-mon[98216]: pgmap v290: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s; 7 B/s, 0 keys/s, 0 objects/s recovering 2023-07-18T03:30:48.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:47 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3356882302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:48.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:47 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2475316995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:48.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:47 smithi040 ceph-mon[96439]: pgmap v290: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s; 7 B/s, 0 keys/s, 0 objects/s recovering 2023-07-18T03:30:48.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:47 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3356882302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:48.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:47 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2475316995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:48.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:47 smithi064 ceph-mon[98059]: pgmap v290: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s; 7 B/s, 0 keys/s, 0 objects/s recovering 2023-07-18T03:30:48.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:47 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3356882302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:48.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:47 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2475316995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:48.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.177+0000 7fbf72787700 1 -- 172.21.15.40:0/2190467256 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbf6c060950 con 0x7fbf540a1220 2023-07-18T03:30:48.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.189+0000 7fbf697fa700 1 -- 172.21.15.40:0/2190467256 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343917 (secure 0 0 0) 0x7fbf6c060950 con 0x7fbf540a1220 2023-07-18T03:30:48.189 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:48.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.193+0000 7fbf72787700 1 -- 172.21.15.40:0/2190467256 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbf540a1220 msgr2=0x7fbf540a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:48.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.193+0000 7fbf72787700 1 --2- 172.21.15.40:0/2190467256 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbf540a1220 0x7fbf540a36d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbf60006010 tx=0x7fbf6000b540 comp rx=0 tx=0).stop 2023-07-18T03:30:48.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.193+0000 7fbf72787700 1 -- 172.21.15.40:0/2190467256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf6c0ffb00 msgr2=0x7fbf6c1957a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:48.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.193+0000 7fbf72787700 1 --2- 172.21.15.40:0/2190467256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf6c0ffb00 0x7fbf6c1957a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00d980 tx=0x7fbf5c00dd40 comp rx=0 tx=0).stop 2023-07-18T03:30:48.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.193+0000 7fbf72787700 1 -- 172.21.15.40:0/2190467256 shutdown_connections 2023-07-18T03:30:48.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.193+0000 7fbf72787700 1 --2- 172.21.15.40:0/2190467256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbf6c101680 0x7fbf6c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.193+0000 7fbf72787700 1 --2- 172.21.15.40:0/2190467256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf6c0ffb00 0x7fbf6c1957a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.193+0000 7fbf72787700 1 --2- 172.21.15.40:0/2190467256 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbf540a1220 0x7fbf540a36d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.193+0000 7fbf72787700 1 --2- 172.21.15.40:0/2190467256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf6c100cf0 0x7fbf6c195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.193+0000 7fbf72787700 1 -- 172.21.15.40:0/2190467256 >> 172.21.15.40:0/2190467256 conn(0x7fbf6c0fb100 msgr2=0x7fbf6c0fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:48.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.193+0000 7fbf72787700 1 -- 172.21.15.40:0/2190467256 shutdown_connections 2023-07-18T03:30:48.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.193+0000 7fbf72787700 1 -- 172.21.15.40:0/2190467256 wait complete. 2023-07-18T03:30:48.193 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-18T03:30:48.202 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2023-07-18T03:30:47.835920+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656397,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":892,"num_read_kb":564,"num_write":416,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180826,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336300,"kb_used_data":12876,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605524,"statfs":{"total":1151940427776,"available":1151596056576,"internally_reserved":0,"allocated":13185024,"data_stored":7189928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":636,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":57,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003857"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656547+0000","last_change":"2023-07-18T03:29:27.101623+0000","last_active":"2023-07-18T03:30:33.656547+0000","last_peered":"2023-07-18T03:30:33.656547+0000","last_clean":"2023-07-18T03:30:33.656547+0000","last_became_active":"2023-07-18T03:29:27.101297+0000","last_became_peered":"2023-07-18T03:29:27.101297+0000","last_unstale":"2023-07-18T03:30:33.656547+0000","last_undegraded":"2023-07-18T03:30:33.656547+0000","last_fullsized":"2023-07-18T03:30:33.656547+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:38:25.465606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655619+0000","last_change":"2023-07-18T03:29:27.104159+0000","last_active":"2023-07-18T03:30:33.655619+0000","last_peered":"2023-07-18T03:30:33.655619+0000","last_clean":"2023-07-18T03:30:33.655619+0000","last_became_active":"2023-07-18T03:29:27.103830+0000","last_became_peered":"2023-07-18T03:29:27.103830+0000","last_unstale":"2023-07-18T03:30:33.655619+0000","last_undegraded":"2023-07-18T03:30:33.655619+0000","last_fullsized":"2023-07-18T03:30:33.655619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:02:01.106624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655451+0000","last_change":"2023-07-18T03:29:27.098858+0000","last_active":"2023-07-18T03:30:33.655451+0000","last_peered":"2023-07-18T03:30:33.655451+0000","last_clean":"2023-07-18T03:30:33.655451+0000","last_became_active":"2023-07-18T03:29:27.098540+0000","last_became_peered":"2023-07-18T03:29:27.098540+0000","last_unstale":"2023-07-18T03:30:33.655451+0000","last_undegraded":"2023-07-18T03:30:33.655451+0000","last_fullsized":"2023-07-18T03:30:33.655451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:59:37.520654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.658876+0000","last_change":"2023-07-18T03:29:27.103106+0000","last_active":"2023-07-18T03:30:33.658876+0000","last_peered":"2023-07-18T03:30:33.658876+0000","last_clean":"2023-07-18T03:30:33.658876+0000","last_became_active":"2023-07-18T03:29:27.102780+0000","last_became_peered":"2023-07-18T03:29:27.102780+0000","last_unstale":"2023-07-18T03:30:33.658876+0000","last_undegraded":"2023-07-18T03:30:33.658876+0000","last_fullsized":"2023-07-18T03:30:33.658876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:50:45.417621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.657667+0000","last_change":"2023-07-18T03:29:27.095432+0000","last_active":"2023-07-18T03:30:33.657667+0000","last_peered":"2023-07-18T03:30:33.657667+0000","last_clean":"2023-07-18T03:30:33.657667+0000","last_became_active":"2023-07-18T03:29:27.095103+0000","last_became_peered":"2023-07-18T03:29:27.095103+0000","last_unstale":"2023-07-18T03:30:33.657667+0000","last_undegraded":"2023-07-18T03:30:33.657667+0000","last_fullsized":"2023-07-18T03:30:33.657667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:34:25.462492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660813+0000","last_change":"2023-07-18T03:29:27.091412+0000","last_active":"2023-07-18T03:30:33.660813+0000","last_peered":"2023-07-18T03:30:33.660813+0000","last_clean":"2023-07-18T03:30:33.660813+0000","last_became_active":"2023-07-18T03:29:27.091060+0000","last_became_peered":"2023-07-18T03:29:27.091060+0000","last_unstale":"2023-07-18T03:30:33.660813+0000","last_undegraded":"2023-07-18T03:30:33.660813+0000","last_fullsized":"2023-07-18T03:30:33.660813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:56:48.388976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.748981+0000","last_change":"2023-07-18T03:29:27.105223+0000","last_active":"2023-07-18T03:30:33.748981+0000","last_peered":"2023-07-18T03:30:33.748981+0000","last_clean":"2023-07-18T03:30:33.748981+0000","last_became_active":"2023-07-18T03:29:27.104926+0000","last_became_peered":"2023-07-18T03:29:27.104926+0000","last_unstale":"2023-07-18T03:30:33.748981+0000","last_undegraded":"2023-07-18T03:30:33.748981+0000","last_fullsized":"2023-07-18T03:30:33.748981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:53:06.895279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660183+0000","last_change":"2023-07-18T03:29:27.093722+0000","last_active":"2023-07-18T03:30:33.660183+0000","last_peered":"2023-07-18T03:30:33.660183+0000","last_clean":"2023-07-18T03:30:33.660183+0000","last_became_active":"2023-07-18T03:29:27.093103+0000","last_became_peered":"2023-07-18T03:29:27.093103+0000","last_unstale":"2023-07-18T03:30:33.660183+0000","last_undegraded":"2023-07-18T03:30:33.660183+0000","last_fullsized":"2023-07-18T03:30:33.660183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:57:08.943397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.651027+0000","last_change":"2023-07-18T03:29:27.094681+0000","last_active":"2023-07-18T03:30:33.651027+0000","last_peered":"2023-07-18T03:30:33.651027+0000","last_clean":"2023-07-18T03:30:33.651027+0000","last_became_active":"2023-07-18T03:29:27.094284+0000","last_became_peered":"2023-07-18T03:29:27.094284+0000","last_unstale":"2023-07-18T03:30:33.651027+0000","last_undegraded":"2023-07-18T03:30:33.651027+0000","last_fullsized":"2023-07-18T03:30:33.651027+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:25:26.164002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661512+0000","last_change":"2023-07-18T03:29:27.100759+0000","last_active":"2023-07-18T03:30:33.661512+0000","last_peered":"2023-07-18T03:30:33.661512+0000","last_clean":"2023-07-18T03:30:33.661512+0000","last_became_active":"2023-07-18T03:29:27.096660+0000","last_became_peered":"2023-07-18T03:29:27.096660+0000","last_unstale":"2023-07-18T03:30:33.661512+0000","last_undegraded":"2023-07-18T03:30:33.661512+0000","last_fullsized":"2023-07-18T03:30:33.661512+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:32:57.131034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749504+0000","last_change":"2023-07-18T03:29:27.107251+0000","last_active":"2023-07-18T03:30:33.749504+0000","last_peered":"2023-07-18T03:30:33.749504+0000","last_clean":"2023-07-18T03:30:33.749504+0000","last_became_active":"2023-07-18T03:29:27.106895+0000","last_became_peered":"2023-07-18T03:29:27.106895+0000","last_unstale":"2023-07-18T03:30:33.749504+0000","last_undegraded":"2023-07-18T03:30:33.749504+0000","last_fullsized":"2023-07-18T03:30:33.749504+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:27:10.110391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.658742+0000","last_change":"2023-07-18T03:29:27.103905+0000","last_active":"2023-07-18T03:30:33.658742+0000","last_peered":"2023-07-18T03:30:33.658742+0000","last_clean":"2023-07-18T03:30:33.658742+0000","last_became_active":"2023-07-18T03:29:27.103021+0000","last_became_peered":"2023-07-18T03:29:27.103021+0000","last_unstale":"2023-07-18T03:30:33.658742+0000","last_undegraded":"2023-07-18T03:30:33.658742+0000","last_fullsized":"2023-07-18T03:30:33.658742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:56:32.967945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655982+0000","last_change":"2023-07-18T03:29:27.103529+0000","last_active":"2023-07-18T03:30:33.655982+0000","last_peered":"2023-07-18T03:30:33.655982+0000","last_clean":"2023-07-18T03:30:33.655982+0000","last_became_active":"2023-07-18T03:29:27.103217+0000","last_became_peered":"2023-07-18T03:29:27.103217+0000","last_unstale":"2023-07-18T03:30:33.655982+0000","last_undegraded":"2023-07-18T03:30:33.655982+0000","last_fullsized":"2023-07-18T03:30:33.655982+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:36:22.276029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655161+0000","last_change":"2023-07-18T03:29:27.096640+0000","last_active":"2023-07-18T03:30:33.655161+0000","last_peered":"2023-07-18T03:30:33.655161+0000","last_clean":"2023-07-18T03:30:33.655161+0000","last_became_active":"2023-07-18T03:29:27.096347+0000","last_became_peered":"2023-07-18T03:29:27.096347+0000","last_unstale":"2023-07-18T03:30:33.655161+0000","last_undegraded":"2023-07-18T03:30:33.655161+0000","last_fullsized":"2023-07-18T03:30:33.655161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:51:47.250203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655071+0000","last_change":"2023-07-18T03:29:27.095910+0000","last_active":"2023-07-18T03:30:33.655071+0000","last_peered":"2023-07-18T03:30:33.655071+0000","last_clean":"2023-07-18T03:30:33.655071+0000","last_became_active":"2023-07-18T03:29:27.095556+0000","last_became_peered":"2023-07-18T03:29:27.095556+0000","last_unstale":"2023-07-18T03:30:33.655071+0000","last_undegraded":"2023-07-18T03:30:33.655071+0000","last_fullsized":"2023-07-18T03:30:33.655071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:17:31.377765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650364+0000","last_change":"2023-07-18T03:29:27.108220+0000","last_active":"2023-07-18T03:30:33.650364+0000","last_peered":"2023-07-18T03:30:33.650364+0000","last_clean":"2023-07-18T03:30:33.650364+0000","last_became_active":"2023-07-18T03:29:27.107869+0000","last_became_peered":"2023-07-18T03:29:27.107869+0000","last_unstale":"2023-07-18T03:30:33.650364+0000","last_undegraded":"2023-07-18T03:30:33.650364+0000","last_fullsized":"2023-07-18T03:30:33.650364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:57:32.649126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656345+0000","last_change":"2023-07-18T03:29:27.100634+0000","last_active":"2023-07-18T03:30:33.656345+0000","last_peered":"2023-07-18T03:30:33.656345+0000","last_clean":"2023-07-18T03:30:33.656345+0000","last_became_active":"2023-07-18T03:29:27.100316+0000","last_became_peered":"2023-07-18T03:29:27.100316+0000","last_unstale":"2023-07-18T03:30:33.656345+0000","last_undegraded":"2023-07-18T03:30:33.656345+0000","last_fullsized":"2023-07-18T03:30:33.656345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:24:14.323573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.652475+0000","last_change":"2023-07-18T03:29:27.100322+0000","last_active":"2023-07-18T03:30:33.652475+0000","last_peered":"2023-07-18T03:30:33.652475+0000","last_clean":"2023-07-18T03:30:33.652475+0000","last_became_active":"2023-07-18T03:29:27.099782+0000","last_became_peered":"2023-07-18T03:29:27.099782+0000","last_unstale":"2023-07-18T03:30:33.652475+0000","last_undegraded":"2023-07-18T03:30:33.652475+0000","last_fullsized":"2023-07-18T03:30:33.652475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:03:29.585791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.654114+0000","last_change":"2023-07-18T03:29:27.096639+0000","last_active":"2023-07-18T03:30:33.654114+0000","last_peered":"2023-07-18T03:30:33.654114+0000","last_clean":"2023-07-18T03:30:33.654114+0000","last_became_active":"2023-07-18T03:29:27.096348+0000","last_became_peered":"2023-07-18T03:29:27.096348+0000","last_unstale":"2023-07-18T03:30:33.654114+0000","last_undegraded":"2023-07-18T03:30:33.654114+0000","last_fullsized":"2023-07-18T03:30:33.654114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:57:04.900158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":103,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:43.760810+0000","last_change":"2023-07-18T03:29:26.287722+0000","last_active":"2023-07-18T03:30:43.760810+0000","last_peered":"2023-07-18T03:30:43.760810+0000","last_clean":"2023-07-18T03:30:43.760810+0000","last_became_active":"2023-07-18T03:29:26.287327+0000","last_became_peered":"2023-07-18T03:29:26.287327+0000","last_unstale":"2023-07-18T03:30:43.760810+0000","last_undegraded":"2023-07-18T03:30:43.760810+0000","last_fullsized":"2023-07-18T03:30:43.760810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:00:13.869444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661775+0000","last_change":"2023-07-18T03:29:27.092771+0000","last_active":"2023-07-18T03:30:33.661775+0000","last_peered":"2023-07-18T03:30:33.661775+0000","last_clean":"2023-07-18T03:30:33.661775+0000","last_became_active":"2023-07-18T03:29:27.092456+0000","last_became_peered":"2023-07-18T03:29:27.092456+0000","last_unstale":"2023-07-18T03:30:33.661775+0000","last_undegraded":"2023-07-18T03:30:33.661775+0000","last_fullsized":"2023-07-18T03:30:33.661775+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:36:58.677882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.662061+0000","last_change":"2023-07-18T03:29:26.289683+0000","last_active":"2023-07-18T03:30:33.662061+0000","last_peered":"2023-07-18T03:30:33.662061+0000","last_clean":"2023-07-18T03:30:33.662061+0000","last_became_active":"2023-07-18T03:29:26.289356+0000","last_became_peered":"2023-07-18T03:29:26.289356+0000","last_unstale":"2023-07-18T03:30:33.662061+0000","last_undegraded":"2023-07-18T03:30:33.662061+0000","last_fullsized":"2023-07-18T03:30:33.662061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:32:24.704699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655443+0000","last_change":"2023-07-18T03:29:27.098709+0000","last_active":"2023-07-18T03:30:33.655443+0000","last_peered":"2023-07-18T03:30:33.655443+0000","last_clean":"2023-07-18T03:30:33.655443+0000","last_became_active":"2023-07-18T03:29:27.098395+0000","last_became_peered":"2023-07-18T03:29:27.098395+0000","last_unstale":"2023-07-18T03:30:33.655443+0000","last_undegraded":"2023-07-18T03:30:33.655443+0000","last_fullsized":"2023-07-18T03:30:33.655443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:57.798041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.658004+0000","last_change":"2023-07-18T03:29:26.297119+0000","last_active":"2023-07-18T03:30:33.658004+0000","last_peered":"2023-07-18T03:30:33.658004+0000","last_clean":"2023-07-18T03:30:33.658004+0000","last_became_active":"2023-07-18T03:29:26.296649+0000","last_became_peered":"2023-07-18T03:29:26.296649+0000","last_unstale":"2023-07-18T03:30:33.658004+0000","last_undegraded":"2023-07-18T03:30:33.658004+0000","last_fullsized":"2023-07-18T03:30:33.658004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:19:40.529493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650567+0000","last_change":"2023-07-18T03:29:27.105511+0000","last_active":"2023-07-18T03:30:33.650567+0000","last_peered":"2023-07-18T03:30:33.650567+0000","last_clean":"2023-07-18T03:30:33.650567+0000","last_became_active":"2023-07-18T03:29:27.104946+0000","last_became_peered":"2023-07-18T03:29:27.104946+0000","last_unstale":"2023-07-18T03:30:33.650567+0000","last_undegraded":"2023-07-18T03:30:33.650567+0000","last_fullsized":"2023-07-18T03:30:33.650567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:27:54.324118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.657412+0000","last_change":"2023-07-18T03:29:26.290708+0000","last_active":"2023-07-18T03:30:33.657412+0000","last_peered":"2023-07-18T03:30:33.657412+0000","last_clean":"2023-07-18T03:30:33.657412+0000","last_became_active":"2023-07-18T03:29:26.290383+0000","last_became_peered":"2023-07-18T03:29:26.290383+0000","last_unstale":"2023-07-18T03:30:33.657412+0000","last_undegraded":"2023-07-18T03:30:33.657412+0000","last_fullsized":"2023-07-18T03:30:33.657412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:23:20.274883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660376+0000","last_change":"2023-07-18T03:29:27.096630+0000","last_active":"2023-07-18T03:30:33.660376+0000","last_peered":"2023-07-18T03:30:33.660376+0000","last_clean":"2023-07-18T03:30:33.660376+0000","last_became_active":"2023-07-18T03:29:27.096337+0000","last_became_peered":"2023-07-18T03:29:27.096337+0000","last_unstale":"2023-07-18T03:30:33.660376+0000","last_undegraded":"2023-07-18T03:30:33.660376+0000","last_fullsized":"2023-07-18T03:30:33.660376+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:01:51.889873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'7","reported_seq":104,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:44.718292+0000","last_change":"2023-07-18T03:29:26.291622+0000","last_active":"2023-07-18T03:30:44.718292+0000","last_peered":"2023-07-18T03:30:44.718292+0000","last_clean":"2023-07-18T03:30:44.718292+0000","last_became_active":"2023-07-18T03:29:26.291317+0000","last_became_peered":"2023-07-18T03:29:26.291317+0000","last_unstale":"2023-07-18T03:30:44.718292+0000","last_undegraded":"2023-07-18T03:30:44.718292+0000","last_fullsized":"2023-07-18T03:30:44.718292+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:44:15.122562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1282,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660520+0000","last_change":"2023-07-18T03:30:33.660520+0000","last_active":"2023-07-18T03:30:33.660520+0000","last_peered":"2023-07-18T03:30:33.660520+0000","last_clean":"2023-07-18T03:30:33.660520+0000","last_became_active":"2023-07-18T03:30:33.660098+0000","last_became_peered":"2023-07-18T03:30:33.660098+0000","last_unstale":"2023-07-18T03:30:33.660520+0000","last_undegraded":"2023-07-18T03:30:33.660520+0000","last_fullsized":"2023-07-18T03:30:33.660520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:16:28.408741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660746+0000","last_change":"2023-07-18T03:29:26.290493+0000","last_active":"2023-07-18T03:30:33.660746+0000","last_peered":"2023-07-18T03:30:33.660746+0000","last_clean":"2023-07-18T03:30:33.660746+0000","last_became_active":"2023-07-18T03:29:26.290187+0000","last_became_peered":"2023-07-18T03:29:26.290187+0000","last_unstale":"2023-07-18T03:30:33.660746+0000","last_undegraded":"2023-07-18T03:30:33.660746+0000","last_fullsized":"2023-07-18T03:30:33.660746+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:58:16.632361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.659999+0000","last_change":"2023-07-18T03:29:27.095129+0000","last_active":"2023-07-18T03:30:33.659999+0000","last_peered":"2023-07-18T03:30:33.659999+0000","last_clean":"2023-07-18T03:30:33.659999+0000","last_became_active":"2023-07-18T03:29:27.094821+0000","last_became_peered":"2023-07-18T03:29:27.094821+0000","last_unstale":"2023-07-18T03:30:33.659999+0000","last_undegraded":"2023-07-18T03:30:33.659999+0000","last_fullsized":"2023-07-18T03:30:33.659999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:43:40.005624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655778+0000","last_change":"2023-07-18T03:29:26.285860+0000","last_active":"2023-07-18T03:30:33.655778+0000","last_peered":"2023-07-18T03:30:33.655778+0000","last_clean":"2023-07-18T03:30:33.655778+0000","last_became_active":"2023-07-18T03:29:26.285491+0000","last_became_peered":"2023-07-18T03:29:26.285491+0000","last_unstale":"2023-07-18T03:30:33.655778+0000","last_undegraded":"2023-07-18T03:30:33.655778+0000","last_fullsized":"2023-07-18T03:30:33.655778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:39:26.999366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660058+0000","last_change":"2023-07-18T03:29:27.093861+0000","last_active":"2023-07-18T03:30:33.660058+0000","last_peered":"2023-07-18T03:30:33.660058+0000","last_clean":"2023-07-18T03:30:33.660058+0000","last_became_active":"2023-07-18T03:29:27.093445+0000","last_became_peered":"2023-07-18T03:29:27.093445+0000","last_unstale":"2023-07-18T03:30:33.660058+0000","last_undegraded":"2023-07-18T03:30:33.660058+0000","last_fullsized":"2023-07-18T03:30:33.660058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:45:32.889832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.659732+0000","last_change":"2023-07-18T03:29:26.288108+0000","last_active":"2023-07-18T03:30:33.659732+0000","last_peered":"2023-07-18T03:30:33.659732+0000","last_clean":"2023-07-18T03:30:33.659732+0000","last_became_active":"2023-07-18T03:29:26.287531+0000","last_became_peered":"2023-07-18T03:29:26.287531+0000","last_unstale":"2023-07-18T03:30:33.659732+0000","last_undegraded":"2023-07-18T03:30:33.659732+0000","last_fullsized":"2023-07-18T03:30:33.659732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:52:58.383534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.652172+0000","last_change":"2023-07-18T03:29:27.092908+0000","last_active":"2023-07-18T03:30:33.652172+0000","last_peered":"2023-07-18T03:30:33.652172+0000","last_clean":"2023-07-18T03:30:33.652172+0000","last_became_active":"2023-07-18T03:29:27.092566+0000","last_became_peered":"2023-07-18T03:29:27.092566+0000","last_unstale":"2023-07-18T03:30:33.652172+0000","last_undegraded":"2023-07-18T03:30:33.652172+0000","last_fullsized":"2023-07-18T03:30:33.652172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:46:32.375687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":168,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:44.717016+0000","last_change":"2023-07-18T03:29:26.294251+0000","last_active":"2023-07-18T03:30:44.717016+0000","last_peered":"2023-07-18T03:30:44.717016+0000","last_clean":"2023-07-18T03:30:44.717016+0000","last_became_active":"2023-07-18T03:29:26.293847+0000","last_became_peered":"2023-07-18T03:29:26.293847+0000","last_unstale":"2023-07-18T03:30:44.717016+0000","last_undegraded":"2023-07-18T03:30:44.717016+0000","last_fullsized":"2023-07-18T03:30:44.717016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:29:39.028643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":145,"num_read_kb":145,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650892+0000","last_change":"2023-07-18T03:29:27.106551+0000","last_active":"2023-07-18T03:30:33.650892+0000","last_peered":"2023-07-18T03:30:33.650892+0000","last_clean":"2023-07-18T03:30:33.650892+0000","last_became_active":"2023-07-18T03:29:27.105187+0000","last_became_peered":"2023-07-18T03:29:27.105187+0000","last_unstale":"2023-07-18T03:30:33.650892+0000","last_undegraded":"2023-07-18T03:30:33.650892+0000","last_fullsized":"2023-07-18T03:30:33.650892+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:59:30.478453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749555+0000","last_change":"2023-07-18T03:29:26.293284+0000","last_active":"2023-07-18T03:30:33.749555+0000","last_peered":"2023-07-18T03:30:33.749555+0000","last_clean":"2023-07-18T03:30:33.749555+0000","last_became_active":"2023-07-18T03:29:26.292955+0000","last_became_peered":"2023-07-18T03:29:26.292955+0000","last_unstale":"2023-07-18T03:30:33.749555+0000","last_undegraded":"2023-07-18T03:30:33.749555+0000","last_fullsized":"2023-07-18T03:30:33.749555+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:14:25.817345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661734+0000","last_change":"2023-07-18T03:30:33.661734+0000","last_active":"2023-07-18T03:30:33.661734+0000","last_peered":"2023-07-18T03:30:33.661734+0000","last_clean":"2023-07-18T03:30:33.661734+0000","last_became_active":"2023-07-18T03:30:33.661363+0000","last_became_peered":"2023-07-18T03:30:33.661363+0000","last_unstale":"2023-07-18T03:30:33.661734+0000","last_undegraded":"2023-07-18T03:30:33.661734+0000","last_fullsized":"2023-07-18T03:30:33.661734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:21:26.542417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660448+0000","last_change":"2023-07-18T03:29:26.290810+0000","last_active":"2023-07-18T03:30:33.660448+0000","last_peered":"2023-07-18T03:30:33.660448+0000","last_clean":"2023-07-18T03:30:33.660448+0000","last_became_active":"2023-07-18T03:29:26.290448+0000","last_became_peered":"2023-07-18T03:29:26.290448+0000","last_unstale":"2023-07-18T03:30:33.660448+0000","last_undegraded":"2023-07-18T03:30:33.660448+0000","last_fullsized":"2023-07-18T03:30:33.660448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:37:32.462232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650723+0000","last_change":"2023-07-18T03:29:27.105801+0000","last_active":"2023-07-18T03:30:33.650723+0000","last_peered":"2023-07-18T03:30:33.650723+0000","last_clean":"2023-07-18T03:30:33.650723+0000","last_became_active":"2023-07-18T03:29:27.104715+0000","last_became_peered":"2023-07-18T03:29:27.104715+0000","last_unstale":"2023-07-18T03:30:33.650723+0000","last_undegraded":"2023-07-18T03:30:33.650723+0000","last_fullsized":"2023-07-18T03:30:33.650723+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:40:38.561885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'31","reported_seq":206,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:44.717360+0000","last_change":"2023-07-18T03:29:26.293306+0000","last_active":"2023-07-18T03:30:44.717360+0000","last_peered":"2023-07-18T03:30:44.717360+0000","last_clean":"2023-07-18T03:30:44.717360+0000","last_became_active":"2023-07-18T03:29:26.292946+0000","last_became_peered":"2023-07-18T03:29:26.292946+0000","last_unstale":"2023-07-18T03:30:44.717360+0000","last_undegraded":"2023-07-18T03:30:44.717360+0000","last_fullsized":"2023-07-18T03:30:44.717360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:24:06.931197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58845,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":157,"num_read_kb":135,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.657688+0000","last_change":"2023-07-18T03:29:27.103941+0000","last_active":"2023-07-18T03:30:33.657688+0000","last_peered":"2023-07-18T03:30:33.657688+0000","last_clean":"2023-07-18T03:30:33.657688+0000","last_became_active":"2023-07-18T03:29:27.103506+0000","last_became_peered":"2023-07-18T03:29:27.103506+0000","last_unstale":"2023-07-18T03:30:33.657688+0000","last_undegraded":"2023-07-18T03:30:33.657688+0000","last_fullsized":"2023-07-18T03:30:33.657688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:18:23.410060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.676819+0000","last_change":"2023-07-18T03:30:33.676417+0000","last_active":"2023-07-18T03:30:33.676819+0000","last_peered":"2023-07-18T03:30:33.676819+0000","last_clean":"2023-07-18T03:30:33.676819+0000","last_became_active":"2023-07-18T03:30:33.662637+0000","last_became_peered":"2023-07-18T03:30:33.662637+0000","last_unstale":"2023-07-18T03:30:33.676819+0000","last_undegraded":"2023-07-18T03:30:33.676819+0000","last_fullsized":"2023-07-18T03:30:33.676819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:15:40.468772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.750602+0000","last_change":"2023-07-18T03:30:33.750602+0000","last_active":"2023-07-18T03:30:33.750602+0000","last_peered":"2023-07-18T03:30:33.750602+0000","last_clean":"2023-07-18T03:30:33.750602+0000","last_became_active":"2023-07-18T03:30:33.750170+0000","last_became_peered":"2023-07-18T03:30:33.750170+0000","last_unstale":"2023-07-18T03:30:33.750602+0000","last_undegraded":"2023-07-18T03:30:33.750602+0000","last_fullsized":"2023-07-18T03:30:33.750602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:13:32.425846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.659763+0000","last_change":"2023-07-18T03:29:26.290487+0000","last_active":"2023-07-18T03:30:33.659763+0000","last_peered":"2023-07-18T03:30:33.659763+0000","last_clean":"2023-07-18T03:30:33.659763+0000","last_became_active":"2023-07-18T03:29:26.290115+0000","last_became_peered":"2023-07-18T03:29:26.290115+0000","last_unstale":"2023-07-18T03:30:33.659763+0000","last_undegraded":"2023-07-18T03:30:33.659763+0000","last_fullsized":"2023-07-18T03:30:33.659763+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:23:20.930476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749286+0000","last_change":"2023-07-18T03:29:27.108550+0000","last_active":"2023-07-18T03:30:33.749286+0000","last_peered":"2023-07-18T03:30:33.749286+0000","last_clean":"2023-07-18T03:30:33.749286+0000","last_became_active":"2023-07-18T03:29:27.108069+0000","last_became_peered":"2023-07-18T03:29:27.108069+0000","last_unstale":"2023-07-18T03:30:33.749286+0000","last_undegraded":"2023-07-18T03:30:33.749286+0000","last_fullsized":"2023-07-18T03:30:33.749286+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:37:05.273266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":97,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:44.761547+0000","last_change":"2023-07-18T03:29:26.290896+0000","last_active":"2023-07-18T03:30:44.761547+0000","last_peered":"2023-07-18T03:30:44.761547+0000","last_clean":"2023-07-18T03:30:44.761547+0000","last_became_active":"2023-07-18T03:29:26.290450+0000","last_became_peered":"2023-07-18T03:29:26.290450+0000","last_unstale":"2023-07-18T03:30:44.761547+0000","last_undegraded":"2023-07-18T03:30:44.761547+0000","last_fullsized":"2023-07-18T03:30:44.761547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:30:12.255317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.748869+0000","last_change":"2023-07-18T03:29:27.109184+0000","last_active":"2023-07-18T03:30:33.748869+0000","last_peered":"2023-07-18T03:30:33.748869+0000","last_clean":"2023-07-18T03:30:33.748869+0000","last_became_active":"2023-07-18T03:29:27.107993+0000","last_became_peered":"2023-07-18T03:29:27.107993+0000","last_unstale":"2023-07-18T03:30:33.748869+0000","last_undegraded":"2023-07-18T03:30:33.748869+0000","last_fullsized":"2023-07-18T03:30:33.748869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:09:48.362579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656895+0000","last_change":"2023-07-18T03:29:26.293671+0000","last_active":"2023-07-18T03:30:33.656895+0000","last_peered":"2023-07-18T03:30:33.656895+0000","last_clean":"2023-07-18T03:30:33.656895+0000","last_became_active":"2023-07-18T03:29:26.293205+0000","last_became_peered":"2023-07-18T03:29:26.293205+0000","last_unstale":"2023-07-18T03:30:33.656895+0000","last_undegraded":"2023-07-18T03:30:33.656895+0000","last_fullsized":"2023-07-18T03:30:33.656895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:23:48.250370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.663227+0000","last_change":"2023-07-18T03:30:33.659828+0000","last_active":"2023-07-18T03:30:33.663227+0000","last_peered":"2023-07-18T03:30:33.663227+0000","last_clean":"2023-07-18T03:30:33.663227+0000","last_became_active":"2023-07-18T03:30:33.659440+0000","last_became_peered":"2023-07-18T03:30:33.659440+0000","last_unstale":"2023-07-18T03:30:33.663227+0000","last_undegraded":"2023-07-18T03:30:33.663227+0000","last_fullsized":"2023-07-18T03:30:33.663227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:21:00.901622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.667448+0000","last_change":"2023-07-18T03:30:33.666902+0000","last_active":"2023-07-18T03:30:33.667448+0000","last_peered":"2023-07-18T03:30:33.667448+0000","last_clean":"2023-07-18T03:30:33.667448+0000","last_became_active":"2023-07-18T03:30:33.661326+0000","last_became_peered":"2023-07-18T03:30:33.661326+0000","last_unstale":"2023-07-18T03:30:33.667448+0000","last_undegraded":"2023-07-18T03:30:33.667448+0000","last_fullsized":"2023-07-18T03:30:33.667448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:12:15.691642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656030+0000","last_change":"2023-07-18T03:29:27.098479+0000","last_active":"2023-07-18T03:30:33.656030+0000","last_peered":"2023-07-18T03:30:33.656030+0000","last_clean":"2023-07-18T03:30:33.656030+0000","last_became_active":"2023-07-18T03:29:27.098153+0000","last_became_peered":"2023-07-18T03:29:27.098153+0000","last_unstale":"2023-07-18T03:30:33.656030+0000","last_undegraded":"2023-07-18T03:30:33.656030+0000","last_fullsized":"2023-07-18T03:30:33.656030+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:53:21.610571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.652236+0000","last_change":"2023-07-18T03:29:26.290953+0000","last_active":"2023-07-18T03:30:33.652236+0000","last_peered":"2023-07-18T03:30:33.652236+0000","last_clean":"2023-07-18T03:30:33.652236+0000","last_became_active":"2023-07-18T03:29:26.290633+0000","last_became_peered":"2023-07-18T03:29:26.290633+0000","last_unstale":"2023-07-18T03:30:33.652236+0000","last_undegraded":"2023-07-18T03:30:33.652236+0000","last_fullsized":"2023-07-18T03:30:33.652236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:43:09.103910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661197+0000","last_change":"2023-07-18T03:29:27.093651+0000","last_active":"2023-07-18T03:30:33.661197+0000","last_peered":"2023-07-18T03:30:33.661197+0000","last_clean":"2023-07-18T03:30:33.661197+0000","last_became_active":"2023-07-18T03:29:27.093307+0000","last_became_peered":"2023-07-18T03:29:27.093307+0000","last_unstale":"2023-07-18T03:30:33.661197+0000","last_undegraded":"2023-07-18T03:30:33.661197+0000","last_fullsized":"2023-07-18T03:30:33.661197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:10:55.332949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":97,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:45.763511+0000","last_change":"2023-07-18T03:29:26.287405+0000","last_active":"2023-07-18T03:30:45.763511+0000","last_peered":"2023-07-18T03:30:45.763511+0000","last_clean":"2023-07-18T03:30:45.763511+0000","last_became_active":"2023-07-18T03:29:26.287048+0000","last_became_peered":"2023-07-18T03:29:26.287048+0000","last_unstale":"2023-07-18T03:30:45.763511+0000","last_undegraded":"2023-07-18T03:30:45.763511+0000","last_fullsized":"2023-07-18T03:30:45.763511+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:46:08.045548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661021+0000","last_change":"2023-07-18T03:29:27.103218+0000","last_active":"2023-07-18T03:30:33.661021+0000","last_peered":"2023-07-18T03:30:33.661021+0000","last_clean":"2023-07-18T03:30:33.661021+0000","last_became_active":"2023-07-18T03:29:27.102917+0000","last_became_peered":"2023-07-18T03:29:27.102917+0000","last_unstale":"2023-07-18T03:30:33.661021+0000","last_undegraded":"2023-07-18T03:30:33.661021+0000","last_fullsized":"2023-07-18T03:30:33.661021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:15:09.570588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.659924+0000","last_change":"2023-07-18T03:29:27.098696+0000","last_active":"2023-07-18T03:30:33.659924+0000","last_peered":"2023-07-18T03:30:33.659924+0000","last_clean":"2023-07-18T03:30:33.659924+0000","last_became_active":"2023-07-18T03:29:27.098253+0000","last_became_peered":"2023-07-18T03:29:27.098253+0000","last_unstale":"2023-07-18T03:30:33.659924+0000","last_undegraded":"2023-07-18T03:30:33.659924+0000","last_fullsized":"2023-07-18T03:30:33.659924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:31:51.020473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":96,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:44.760939+0000","last_change":"2023-07-18T03:29:26.290308+0000","last_active":"2023-07-18T03:30:44.760939+0000","last_peered":"2023-07-18T03:30:44.760939+0000","last_clean":"2023-07-18T03:30:44.760939+0000","last_became_active":"2023-07-18T03:29:26.289937+0000","last_became_peered":"2023-07-18T03:29:26.289937+0000","last_unstale":"2023-07-18T03:30:44.760939+0000","last_undegraded":"2023-07-18T03:30:44.760939+0000","last_fullsized":"2023-07-18T03:30:44.760939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:48:40.209353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661087+0000","last_change":"2023-07-18T03:29:27.096469+0000","last_active":"2023-07-18T03:30:33.661087+0000","last_peered":"2023-07-18T03:30:33.661087+0000","last_clean":"2023-07-18T03:30:33.661087+0000","last_became_active":"2023-07-18T03:29:27.096171+0000","last_became_peered":"2023-07-18T03:29:27.096171+0000","last_unstale":"2023-07-18T03:30:33.661087+0000","last_undegraded":"2023-07-18T03:30:33.661087+0000","last_fullsized":"2023-07-18T03:30:33.661087+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:12:58.040917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.658966+0000","last_change":"2023-07-18T03:29:26.290935+0000","last_active":"2023-07-18T03:30:33.658966+0000","last_peered":"2023-07-18T03:30:33.658966+0000","last_clean":"2023-07-18T03:30:33.658966+0000","last_became_active":"2023-07-18T03:29:26.290595+0000","last_became_peered":"2023-07-18T03:29:26.290595+0000","last_unstale":"2023-07-18T03:30:33.658966+0000","last_undegraded":"2023-07-18T03:30:33.658966+0000","last_fullsized":"2023-07-18T03:30:33.658966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:48:44.101957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":335,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.748928+0000","last_change":"2023-07-18T03:28:28.122385+0000","last_active":"2023-07-18T03:30:33.748928+0000","last_peered":"2023-07-18T03:30:33.748928+0000","last_clean":"2023-07-18T03:30:33.748928+0000","last_became_active":"2023-07-18T03:28:28.121932+0000","last_became_peered":"2023-07-18T03:28:28.121932+0000","last_unstale":"2023-07-18T03:30:33.748928+0000","last_undegraded":"2023-07-18T03:30:33.748928+0000","last_fullsized":"2023-07-18T03:30:33.748928+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:25:38.267627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:25:38.267627+0000","last_clean_scrub_stamp":"2023-07-18T03:25:38.267627+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:04:59.247851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655491+0000","last_change":"2023-07-18T03:29:27.101582+0000","last_active":"2023-07-18T03:30:33.655491+0000","last_peered":"2023-07-18T03:30:33.655491+0000","last_clean":"2023-07-18T03:30:33.655491+0000","last_became_active":"2023-07-18T03:29:27.101226+0000","last_became_peered":"2023-07-18T03:29:27.101226+0000","last_unstale":"2023-07-18T03:30:33.655491+0000","last_undegraded":"2023-07-18T03:30:33.655491+0000","last_fullsized":"2023-07-18T03:30:33.655491+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:24:53.297066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749186+0000","last_change":"2023-07-18T03:29:26.291787+0000","last_active":"2023-07-18T03:30:33.749186+0000","last_peered":"2023-07-18T03:30:33.749186+0000","last_clean":"2023-07-18T03:30:33.749186+0000","last_became_active":"2023-07-18T03:29:26.291471+0000","last_became_peered":"2023-07-18T03:29:26.291471+0000","last_unstale":"2023-07-18T03:30:33.749186+0000","last_undegraded":"2023-07-18T03:30:33.749186+0000","last_fullsized":"2023-07-18T03:30:33.749186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:03:33.650313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.657936+0000","last_change":"2023-07-18T03:29:26.296936+0000","last_active":"2023-07-18T03:30:33.657936+0000","last_peered":"2023-07-18T03:30:33.657936+0000","last_clean":"2023-07-18T03:30:33.657936+0000","last_became_active":"2023-07-18T03:29:26.296241+0000","last_became_peered":"2023-07-18T03:29:26.296241+0000","last_unstale":"2023-07-18T03:30:33.657936+0000","last_undegraded":"2023-07-18T03:30:33.657936+0000","last_fullsized":"2023-07-18T03:30:33.657936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:51:59.499717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656549+0000","last_change":"2023-07-18T03:29:27.102715+0000","last_active":"2023-07-18T03:30:33.656549+0000","last_peered":"2023-07-18T03:30:33.656549+0000","last_clean":"2023-07-18T03:30:33.656549+0000","last_became_active":"2023-07-18T03:29:27.100914+0000","last_became_peered":"2023-07-18T03:29:27.100914+0000","last_unstale":"2023-07-18T03:30:33.656549+0000","last_undegraded":"2023-07-18T03:30:33.656549+0000","last_fullsized":"2023-07-18T03:30:33.656549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:09:20.580139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'9","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.652605+0000","last_change":"2023-07-18T03:29:26.290231+0000","last_active":"2023-07-18T03:30:33.652605+0000","last_peered":"2023-07-18T03:30:33.652605+0000","last_clean":"2023-07-18T03:30:33.652605+0000","last_became_active":"2023-07-18T03:29:26.289753+0000","last_became_peered":"2023-07-18T03:29:26.289753+0000","last_unstale":"2023-07-18T03:30:33.652605+0000","last_undegraded":"2023-07-18T03:30:33.652605+0000","last_fullsized":"2023-07-18T03:30:33.652605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:37:14.149936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661595+0000","last_change":"2023-07-18T03:29:27.103964+0000","last_active":"2023-07-18T03:30:33.661595+0000","last_peered":"2023-07-18T03:30:33.661595+0000","last_clean":"2023-07-18T03:30:33.661595+0000","last_became_active":"2023-07-18T03:29:27.103660+0000","last_became_peered":"2023-07-18T03:29:27.103660+0000","last_unstale":"2023-07-18T03:30:33.661595+0000","last_undegraded":"2023-07-18T03:30:33.661595+0000","last_fullsized":"2023-07-18T03:30:33.661595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:24:11.718523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.659213+0000","last_change":"2023-07-18T03:29:26.286436+0000","last_active":"2023-07-18T03:30:33.659213+0000","last_peered":"2023-07-18T03:30:33.659213+0000","last_clean":"2023-07-18T03:30:33.659213+0000","last_became_active":"2023-07-18T03:29:26.286070+0000","last_became_peered":"2023-07-18T03:29:26.286070+0000","last_unstale":"2023-07-18T03:30:33.659213+0000","last_undegraded":"2023-07-18T03:30:33.659213+0000","last_fullsized":"2023-07-18T03:30:33.659213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:49:30.399461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.750552+0000","last_change":"2023-07-18T03:30:33.750552+0000","last_active":"2023-07-18T03:30:33.750552+0000","last_peered":"2023-07-18T03:30:33.750552+0000","last_clean":"2023-07-18T03:30:33.750552+0000","last_became_active":"2023-07-18T03:30:33.750210+0000","last_became_peered":"2023-07-18T03:30:33.750210+0000","last_unstale":"2023-07-18T03:30:33.750552+0000","last_undegraded":"2023-07-18T03:30:33.750552+0000","last_fullsized":"2023-07-18T03:30:33.750552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:30:46.022642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650623+0000","last_change":"2023-07-18T03:29:26.289628+0000","last_active":"2023-07-18T03:30:33.650623+0000","last_peered":"2023-07-18T03:30:33.650623+0000","last_clean":"2023-07-18T03:30:33.650623+0000","last_became_active":"2023-07-18T03:29:26.289013+0000","last_became_peered":"2023-07-18T03:29:26.289013+0000","last_unstale":"2023-07-18T03:30:33.650623+0000","last_undegraded":"2023-07-18T03:30:33.650623+0000","last_fullsized":"2023-07-18T03:30:33.650623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:10:03.968656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656945+0000","last_change":"2023-07-18T03:29:27.090943+0000","last_active":"2023-07-18T03:30:33.656945+0000","last_peered":"2023-07-18T03:30:33.656945+0000","last_clean":"2023-07-18T03:30:33.656945+0000","last_became_active":"2023-07-18T03:29:27.090465+0000","last_became_peered":"2023-07-18T03:29:27.090465+0000","last_unstale":"2023-07-18T03:30:33.656945+0000","last_undegraded":"2023-07-18T03:30:33.656945+0000","last_fullsized":"2023-07-18T03:30:33.656945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:26:40.736484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660770+0000","last_change":"2023-07-18T03:29:26.288576+0000","last_active":"2023-07-18T03:30:33.660770+0000","last_peered":"2023-07-18T03:30:33.660770+0000","last_clean":"2023-07-18T03:30:33.660770+0000","last_became_active":"2023-07-18T03:29:26.287514+0000","last_became_peered":"2023-07-18T03:29:26.287514+0000","last_unstale":"2023-07-18T03:30:33.660770+0000","last_undegraded":"2023-07-18T03:30:33.660770+0000","last_fullsized":"2023-07-18T03:30:33.660770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:35:34.080524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656181+0000","last_change":"2023-07-18T03:29:27.101111+0000","last_active":"2023-07-18T03:30:33.656181+0000","last_peered":"2023-07-18T03:30:33.656181+0000","last_clean":"2023-07-18T03:30:33.656181+0000","last_became_active":"2023-07-18T03:29:27.100737+0000","last_became_peered":"2023-07-18T03:29:27.100737+0000","last_unstale":"2023-07-18T03:30:33.656181+0000","last_undegraded":"2023-07-18T03:30:33.656181+0000","last_fullsized":"2023-07-18T03:30:33.656181+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:17:22.437647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.654205+0000","last_change":"2023-07-18T03:29:26.296974+0000","last_active":"2023-07-18T03:30:33.654205+0000","last_peered":"2023-07-18T03:30:33.654205+0000","last_clean":"2023-07-18T03:30:33.654205+0000","last_became_active":"2023-07-18T03:29:26.296451+0000","last_became_peered":"2023-07-18T03:29:26.296451+0000","last_unstale":"2023-07-18T03:30:33.654205+0000","last_undegraded":"2023-07-18T03:30:33.654205+0000","last_fullsized":"2023-07-18T03:30:33.654205+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:48:52.975744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661117+0000","last_change":"2023-07-18T03:29:27.095857+0000","last_active":"2023-07-18T03:30:33.661117+0000","last_peered":"2023-07-18T03:30:33.661117+0000","last_clean":"2023-07-18T03:30:33.661117+0000","last_became_active":"2023-07-18T03:29:27.095553+0000","last_became_peered":"2023-07-18T03:29:27.095553+0000","last_unstale":"2023-07-18T03:30:33.661117+0000","last_undegraded":"2023-07-18T03:30:33.661117+0000","last_fullsized":"2023-07-18T03:30:33.661117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:22:26.560346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.657524+0000","last_change":"2023-07-18T03:29:26.295936+0000","last_active":"2023-07-18T03:30:33.657524+0000","last_peered":"2023-07-18T03:30:33.657524+0000","last_clean":"2023-07-18T03:30:33.657524+0000","last_became_active":"2023-07-18T03:29:26.295467+0000","last_became_peered":"2023-07-18T03:29:26.295467+0000","last_unstale":"2023-07-18T03:30:33.657524+0000","last_undegraded":"2023-07-18T03:30:33.657524+0000","last_fullsized":"2023-07-18T03:30:33.657524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:23:28.043840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749424+0000","last_change":"2023-07-18T03:29:27.108528+0000","last_active":"2023-07-18T03:30:33.749424+0000","last_peered":"2023-07-18T03:30:33.749424+0000","last_clean":"2023-07-18T03:30:33.749424+0000","last_became_active":"2023-07-18T03:29:27.108039+0000","last_became_peered":"2023-07-18T03:29:27.108039+0000","last_unstale":"2023-07-18T03:30:33.749424+0000","last_undegraded":"2023-07-18T03:30:33.749424+0000","last_fullsized":"2023-07-18T03:30:33.749424+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:13:41.092469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'5","reported_seq":94,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:41.764673+0000","last_change":"2023-07-18T03:29:26.294006+0000","last_active":"2023-07-18T03:30:41.764673+0000","last_peered":"2023-07-18T03:30:41.764673+0000","last_clean":"2023-07-18T03:30:41.764673+0000","last_became_active":"2023-07-18T03:29:26.293700+0000","last_became_peered":"2023-07-18T03:29:26.293700+0000","last_unstale":"2023-07-18T03:30:41.764673+0000","last_undegraded":"2023-07-18T03:30:41.764673+0000","last_fullsized":"2023-07-18T03:30:41.764673+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:41:32.418658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749567+0000","last_change":"2023-07-18T03:29:27.102764+0000","last_active":"2023-07-18T03:30:33.749567+0000","last_peered":"2023-07-18T03:30:33.749567+0000","last_clean":"2023-07-18T03:30:33.749567+0000","last_became_active":"2023-07-18T03:29:27.102200+0000","last_became_peered":"2023-07-18T03:29:27.102200+0000","last_unstale":"2023-07-18T03:30:33.749567+0000","last_undegraded":"2023-07-18T03:30:33.749567+0000","last_fullsized":"2023-07-18T03:30:33.749567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:49:22.828462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660299+0000","last_change":"2023-07-18T03:29:26.294271+0000","last_active":"2023-07-18T03:30:33.660299+0000","last_peered":"2023-07-18T03:30:33.660299+0000","last_clean":"2023-07-18T03:30:33.660299+0000","last_became_active":"2023-07-18T03:29:26.293938+0000","last_became_peered":"2023-07-18T03:29:26.293938+0000","last_unstale":"2023-07-18T03:30:33.660299+0000","last_undegraded":"2023-07-18T03:30:33.660299+0000","last_fullsized":"2023-07-18T03:30:33.660299+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:59:22.557498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749202+0000","last_change":"2023-07-18T03:29:27.106510+0000","last_active":"2023-07-18T03:30:33.749202+0000","last_peered":"2023-07-18T03:30:33.749202+0000","last_clean":"2023-07-18T03:30:33.749202+0000","last_became_active":"2023-07-18T03:29:27.106222+0000","last_became_peered":"2023-07-18T03:29:27.106222+0000","last_unstale":"2023-07-18T03:30:33.749202+0000","last_undegraded":"2023-07-18T03:30:33.749202+0000","last_fullsized":"2023-07-18T03:30:33.749202+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:08:13.558332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656074+0000","last_change":"2023-07-18T03:29:26.293583+0000","last_active":"2023-07-18T03:30:33.656074+0000","last_peered":"2023-07-18T03:30:33.656074+0000","last_clean":"2023-07-18T03:30:33.656074+0000","last_became_active":"2023-07-18T03:29:26.293076+0000","last_became_peered":"2023-07-18T03:29:26.293076+0000","last_unstale":"2023-07-18T03:30:33.656074+0000","last_undegraded":"2023-07-18T03:30:33.656074+0000","last_fullsized":"2023-07-18T03:30:33.656074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:58:56.281865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749636+0000","last_change":"2023-07-18T03:29:27.109175+0000","last_active":"2023-07-18T03:30:33.749636+0000","last_peered":"2023-07-18T03:30:33.749636+0000","last_clean":"2023-07-18T03:30:33.749636+0000","last_became_active":"2023-07-18T03:29:27.108669+0000","last_became_peered":"2023-07-18T03:29:27.108669+0000","last_unstale":"2023-07-18T03:30:33.749636+0000","last_undegraded":"2023-07-18T03:30:33.749636+0000","last_fullsized":"2023-07-18T03:30:33.749636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:34:18.667617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656335+0000","last_change":"2023-07-18T03:29:27.099770+0000","last_active":"2023-07-18T03:30:33.656335+0000","last_peered":"2023-07-18T03:30:33.656335+0000","last_clean":"2023-07-18T03:30:33.656335+0000","last_became_active":"2023-07-18T03:29:27.099487+0000","last_became_peered":"2023-07-18T03:29:27.099487+0000","last_unstale":"2023-07-18T03:30:33.656335+0000","last_undegraded":"2023-07-18T03:30:33.656335+0000","last_fullsized":"2023-07-18T03:30:33.656335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:13:59.713331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.664744+0000","last_change":"2023-07-18T03:30:33.664744+0000","last_active":"2023-07-18T03:30:33.664744+0000","last_peered":"2023-07-18T03:30:33.664744+0000","last_clean":"2023-07-18T03:30:33.664744+0000","last_became_active":"2023-07-18T03:30:33.664378+0000","last_became_peered":"2023-07-18T03:30:33.664378+0000","last_unstale":"2023-07-18T03:30:33.664744+0000","last_undegraded":"2023-07-18T03:30:33.664744+0000","last_fullsized":"2023-07-18T03:30:33.664744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:46:59.976803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.653582+0000","last_change":"2023-07-18T03:29:27.099196+0000","last_active":"2023-07-18T03:30:33.653582+0000","last_peered":"2023-07-18T03:30:33.653582+0000","last_clean":"2023-07-18T03:30:33.653582+0000","last_became_active":"2023-07-18T03:29:27.098903+0000","last_became_peered":"2023-07-18T03:29:27.098903+0000","last_unstale":"2023-07-18T03:30:33.653582+0000","last_undegraded":"2023-07-18T03:30:33.653582+0000","last_fullsized":"2023-07-18T03:30:33.653582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:43:55.170768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660224+0000","last_change":"2023-07-18T03:30:33.660224+0000","last_active":"2023-07-18T03:30:33.660224+0000","last_peered":"2023-07-18T03:30:33.660224+0000","last_clean":"2023-07-18T03:30:33.660224+0000","last_became_active":"2023-07-18T03:30:33.659847+0000","last_became_peered":"2023-07-18T03:30:33.659847+0000","last_unstale":"2023-07-18T03:30:33.660224+0000","last_undegraded":"2023-07-18T03:30:33.660224+0000","last_fullsized":"2023-07-18T03:30:33.660224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:08:47.876757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656416+0000","last_change":"2023-07-18T03:29:27.101610+0000","last_active":"2023-07-18T03:30:33.656416+0000","last_peered":"2023-07-18T03:30:33.656416+0000","last_clean":"2023-07-18T03:30:33.656416+0000","last_became_active":"2023-07-18T03:29:27.101332+0000","last_became_peered":"2023-07-18T03:29:27.101332+0000","last_unstale":"2023-07-18T03:30:33.656416+0000","last_undegraded":"2023-07-18T03:30:33.656416+0000","last_fullsized":"2023-07-18T03:30:33.656416+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:15:53.329517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656022+0000","last_change":"2023-07-18T03:29:27.093345+0000","last_active":"2023-07-18T03:30:33.656022+0000","last_peered":"2023-07-18T03:30:33.656022+0000","last_clean":"2023-07-18T03:30:33.656022+0000","last_became_active":"2023-07-18T03:29:27.093048+0000","last_became_peered":"2023-07-18T03:29:27.093048+0000","last_unstale":"2023-07-18T03:30:33.656022+0000","last_undegraded":"2023-07-18T03:30:33.656022+0000","last_fullsized":"2023-07-18T03:30:33.656022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:43:10.411014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.654001+0000","last_change":"2023-07-18T03:29:27.098311+0000","last_active":"2023-07-18T03:30:33.654001+0000","last_peered":"2023-07-18T03:30:33.654001+0000","last_clean":"2023-07-18T03:30:33.654001+0000","last_became_active":"2023-07-18T03:29:27.097980+0000","last_became_peered":"2023-07-18T03:29:27.097980+0000","last_unstale":"2023-07-18T03:30:33.654001+0000","last_undegraded":"2023-07-18T03:30:33.654001+0000","last_fullsized":"2023-07-18T03:30:33.654001+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:18:24.177425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.663242+0000","last_change":"2023-07-18T03:30:33.660269+0000","last_active":"2023-07-18T03:30:33.663242+0000","last_peered":"2023-07-18T03:30:33.663242+0000","last_clean":"2023-07-18T03:30:33.663242+0000","last_became_active":"2023-07-18T03:30:33.659999+0000","last_became_peered":"2023-07-18T03:30:33.659999+0000","last_unstale":"2023-07-18T03:30:33.663242+0000","last_undegraded":"2023-07-18T03:30:33.663242+0000","last_fullsized":"2023-07-18T03:30:33.663242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:26:07.126950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660153+0000","last_change":"2023-07-18T03:29:27.098541+0000","last_active":"2023-07-18T03:30:33.660153+0000","last_peered":"2023-07-18T03:30:33.660153+0000","last_clean":"2023-07-18T03:30:33.660153+0000","last_became_active":"2023-07-18T03:29:27.097824+0000","last_became_peered":"2023-07-18T03:29:27.097824+0000","last_unstale":"2023-07-18T03:30:33.660153+0000","last_undegraded":"2023-07-18T03:30:33.660153+0000","last_fullsized":"2023-07-18T03:30:33.660153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:57:42.609243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660576+0000","last_change":"2023-07-18T03:29:27.096983+0000","last_active":"2023-07-18T03:30:33.660576+0000","last_peered":"2023-07-18T03:30:33.660576+0000","last_clean":"2023-07-18T03:30:33.660576+0000","last_became_active":"2023-07-18T03:29:27.096632+0000","last_became_peered":"2023-07-18T03:29:27.096632+0000","last_unstale":"2023-07-18T03:30:33.660576+0000","last_undegraded":"2023-07-18T03:30:33.660576+0000","last_fullsized":"2023-07-18T03:30:33.660576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:28:14.181259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650674+0000","last_change":"2023-07-18T03:29:27.105236+0000","last_active":"2023-07-18T03:30:33.650674+0000","last_peered":"2023-07-18T03:30:33.650674+0000","last_clean":"2023-07-18T03:30:33.650674+0000","last_became_active":"2023-07-18T03:29:27.104906+0000","last_became_peered":"2023-07-18T03:29:27.104906+0000","last_unstale":"2023-07-18T03:30:33.650674+0000","last_undegraded":"2023-07-18T03:30:33.650674+0000","last_fullsized":"2023-07-18T03:30:33.650674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:14:25.383609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656158+0000","last_change":"2023-07-18T03:29:27.103461+0000","last_active":"2023-07-18T03:30:33.656158+0000","last_peered":"2023-07-18T03:30:33.656158+0000","last_clean":"2023-07-18T03:30:33.656158+0000","last_became_active":"2023-07-18T03:29:27.103162+0000","last_became_peered":"2023-07-18T03:29:27.103162+0000","last_unstale":"2023-07-18T03:30:33.656158+0000","last_undegraded":"2023-07-18T03:30:33.656158+0000","last_fullsized":"2023-07-18T03:30:33.656158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:36:23.614131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656048+0000","last_change":"2023-07-18T03:29:27.095300+0000","last_active":"2023-07-18T03:30:33.656048+0000","last_peered":"2023-07-18T03:30:33.656048+0000","last_clean":"2023-07-18T03:30:33.656048+0000","last_became_active":"2023-07-18T03:29:27.094985+0000","last_became_peered":"2023-07-18T03:29:27.094985+0000","last_unstale":"2023-07-18T03:30:33.656048+0000","last_undegraded":"2023-07-18T03:30:33.656048+0000","last_fullsized":"2023-07-18T03:30:33.656048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:37:22.084325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65783,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":766,"num_read_kb":455,"num_write":233,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":201033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514524,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":436016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":0.52900000000000003}]},{"osd":1,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.60999999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.66500000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.70999999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.54400000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.69599999999999995}]},{"osd":8,"last update":"Tue Jul 18 03:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.755},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.73699999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.872}]},{"osd":10,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.77400000000000002}]}]},{"osd":10,"up_from":69,"seq":296352743456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1613824,"data_stored":1085806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53800000000000003,"15min":0.54600000000000004},"min":{"1min":0.35799999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70899999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.35899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78000000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41399999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66300000000000003,"15min":0.66800000000000004},"min":{"1min":0.42899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91500000000000004,"5min":1.302,"15min":1.302},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.71099999999999997},"min":{"1min":0.54600000000000004,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93999999999999995,"5min":1,"15min":1},"last":0.81499999999999995}]},{"osd":3,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.68000000000000005,"15min":0.68899999999999995},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.876,"5min":1.276,"15min":1.276},"last":0.495},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.72899999999999998,"15min":0.73399999999999999},"min":{"1min":0.436,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":0.48399999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65700000000000003,"15min":0.66100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.89300000000000002,"5min":1.119,"15min":1.119},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.33700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.85399999999999998,"5min":1.278,"15min":1.278},"last":0.46200000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.746,"15min":0.752},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.004,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.80400000000000005,"15min":0.81000000000000005},"min":{"1min":0.52400000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0229999999999999,"5min":1.222,"15min":1.222},"last":0.75}]},{"osd":6,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.82799999999999996,"15min":0.83599999999999997},"min":{"1min":0.503,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82699999999999996,"15min":0.83199999999999996},"min":{"1min":0.53600000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.54300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":8,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.86399999999999999,"15min":0.871},"min":{"1min":0.57099999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.107,"5min":1.466,"15min":1.466},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.88300000000000001,"15min":0.89600000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0509999999999999,"5min":1.518,"15min":1.518},"last":0.77600000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.87,"15min":0.875},"min":{"1min":0.59599999999999997,"5min":0.503,"15min":0.503},"max":{"1min":1.169,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.88500000000000001,"15min":0.89900000000000002},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.153,"5min":1.585,"15min":1.585},"last":0.626}]},{"osd":11,"last update":"Tue Jul 18 03:29:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.71499999999999997}]}]},{"osd":9,"up_from":64,"seq":274877906980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":435904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.47099999999999997,"15min":0.48199999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.58999999999999997,"5min":0.747,"15min":0.747},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.47099999999999997,"15min":0.48699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.57199999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.318}]},{"osd":1,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.69599999999999995,"15min":0.71199999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86799999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.61899999999999999,"15min":0.63500000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.746,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46500000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.67300000000000004,"15min":0.69099999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.71899999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.69899999999999995,"15min":0.71399999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81899999999999995,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.66200000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.50600000000000001,"15min":0.52200000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.81699999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.60099999999999998,"15min":0.622},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82499999999999996,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.34899999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.76600000000000001,"15min":0.79300000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.80800000000000005,"5min":1.355,"15min":1.355},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.69299999999999995,"15min":0.71099999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.77100000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.372}]},{"osd":5,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.79100000000000004,"15min":0.81100000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90000000000000002,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.622},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.77800000000000002,"15min":0.79800000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.871,"5min":1.302,"15min":1.302},"last":0.68100000000000005}]},{"osd":6,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.745,"15min":0.77700000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.75600000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.78600000000000003,"15min":0.82199999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85399999999999998,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.69399999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.78800000000000003,"15min":0.80600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89500000000000002,"5min":1.137,"15min":1.137},"last":0.435},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.81999999999999995,"15min":0.84499999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.91000000000000003,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.56499999999999995}]},{"osd":8,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.86799999999999999,"15min":0.89400000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92800000000000005,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.77700000000000002,"15min":0.79200000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.85599999999999998,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.51900000000000002}]},{"osd":10,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.80000000000000004,"15min":0.81899999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94599999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.84399999999999997,"15min":0.86499999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95799999999999996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.76100000000000001}]}]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":435929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:30:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.57499999999999996,"15min":0.55000000000000004},"min":{"1min":0.438,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82799999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59299999999999997}]},{"osd":1,"last update":"Tue Jul 18 03:30:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.61899999999999999,"15min":0.59099999999999997},"min":{"1min":0.435,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86199999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.63600000000000001,"15min":0.61399999999999999},"min":{"1min":0.53200000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.875,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.73099999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:30:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.45900000000000002,"15min":0.439},"min":{"1min":0.376,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73099999999999998,"5min":0.755,"15min":0.755},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.48799999999999999,"15min":0.45700000000000002},"min":{"1min":0.35399999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.66000000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:30:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85599999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.64800000000000002,"15min":0.59599999999999997},"min":{"1min":0.47299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":6.1420000000000003,"5min":6.1420000000000003,"15min":6.1420000000000003},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:30:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.65200000000000002,"15min":0.626},"min":{"1min":0.55200000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.66800000000000004,"15min":0.63100000000000001},"min":{"1min":0.56200000000000006,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.81499999999999995}]},{"osd":5,"last update":"Tue Jul 18 03:30:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.59199999999999997,"15min":0.56899999999999995},"min":{"1min":0.42999999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.57699999999999996,"15min":0.54900000000000004},"min":{"1min":0.53700000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91500000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69499999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:30:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70399999999999996,"15min":0.69199999999999995},"min":{"1min":0.45200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.95599999999999996,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.755},{"interface":"front","average":{"1min":1.054,"5min":0.76900000000000002,"15min":0.71799999999999997},"min":{"1min":0.52400000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":5.9740000000000002,"5min":5.9740000000000002,"15min":5.9740000000000002},"last":0.78600000000000003}]},{"osd":7,"last update":"Tue Jul 18 03:30:46 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.755,"15min":0.72399999999999998},"min":{"1min":0.60399999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.099,"5min":1.153,"15min":1.153},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.72699999999999998,"15min":0.69599999999999995},"min":{"1min":0.64300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.048,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.89200000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:30:05 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.752,"15min":0.72999999999999998},"min":{"1min":0.56499999999999995,"5min":0.505,"15min":0.505},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.78300000000000003,"15min":0.77300000000000002},"min":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.85599999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.81299999999999994,"15min":0.79800000000000004},"min":{"1min":0.69099999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.80900000000000005,"15min":0.79100000000000004},"min":{"1min":0.67200000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.145,"5min":1.208,"15min":1.208},"last":0.96599999999999997}]},{"osd":11,"last update":"Tue Jul 18 03:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.872}]}]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1531904,"data_stored":1028979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57799999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.74199999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.38400000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60799999999999998,"15min":0.59199999999999997},"min":{"1min":0.434,"5min":0.379,"15min":0.379},"max":{"1min":0.86599999999999999,"5min":0.872,"15min":0.872},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63400000000000001,"15min":0.61399999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83499999999999996,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56200000000000006}]},{"osd":3,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.45600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.878,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.501},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.65400000000000003,"15min":0.65500000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56899999999999995,"15min":0.55800000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.74399999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76500000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.36199999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67600000000000005,"15min":0.66200000000000003},"min":{"1min":0.499,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88600000000000001,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.45100000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.64900000000000002}]},{"osd":6,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.74099999999999999,"15min":0.73299999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.433},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.76600000000000001,"15min":0.73299999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":6.1909999999999998,"5min":6.1909999999999998,"15min":6.1909999999999998},"last":0.46100000000000002}]},{"osd":8,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.80400000000000005,"15min":0.77600000000000002},"min":{"1min":0.51300000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":4.5880000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.76900000000000002,"15min":0.73299999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.6050000000000004,"5min":4.6050000000000004,"15min":4.6050000000000004},"last":0.62}]},{"osd":9,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":0.86199999999999999,"15min":0.80700000000000005},"min":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":9.8439999999999994,"5min":9.8439999999999994,"15min":9.8439999999999994},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81899999999999995,"15min":0.81499999999999995},"min":{"1min":0.629,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.69799999999999995}]},{"osd":10,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79500000000000004,"15min":0.79900000000000004},"min":{"1min":0.54100000000000004,"5min":0.432,"15min":0.432},"max":{"1min":1,"5min":1.171,"15min":1.171},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85199999999999998,"15min":0.86199999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0249999999999999,"5min":1.085,"15min":1.085},"last":0.74099999999999999}]},{"osd":11,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.75900000000000001}]}]},{"osd":6,"up_from":44,"seq":188978561073,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":435579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.443,"15min":0.41399999999999998},"min":{"1min":0.27100000000000002,"5min":0.254,"15min":0.254},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.46100000000000002,"15min":0.45000000000000001},"min":{"1min":0.28999999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.2320000000000002,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":0.375}]},{"osd":1,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.63900000000000001,"15min":0.58499999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":5.0700000000000003,"5min":5.0700000000000003,"15min":5.0700000000000003},"last":0.313},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.41699999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86699999999999999,"5min":1.008,"15min":1.008},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.68999999999999995,"15min":0.63800000000000001},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":5.0300000000000002,"5min":5.0300000000000002,"15min":5.0300000000000002},"last":0.65500000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.55900000000000005,"15min":0.50900000000000001},"min":{"1min":0.32100000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":5.2089999999999996,"5min":5.2089999999999996,"15min":5.2089999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54500000000000004,"15min":0.53700000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.497}]},{"osd":4,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.623,"15min":0.61399999999999999},"min":{"1min":0.44,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86599999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.436},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.73199999999999998,"15min":0.68700000000000006},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":5.1070000000000002,"5min":5.1070000000000002,"15min":5.1070000000000002},"last":0.48099999999999998}]},{"osd":5,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67800000000000005,"15min":0.67000000000000004},"min":{"1min":0.45300000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94599999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.73299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.83599999999999997,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.71799999999999997,"15min":0.72299999999999998},"min":{"1min":0.50900000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.91400000000000003,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.80300000000000005,"15min":0.78300000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.9699999999999998,"5min":4.9699999999999998,"15min":4.9699999999999998},"last":0.57899999999999996}]},{"osd":8,"last update":"Tue Jul 18 03:29:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.86399999999999999,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":5.1760000000000002,"5min":5.1760000000000002,"15min":5.1760000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.77400000000000002,"15min":0.78000000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.67400000000000004}]},{"osd":9,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.75700000000000001,"15min":0.76600000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.81999999999999995,"15min":0.79800000000000004},"min":{"1min":0.438,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":5.2409999999999997,"5min":5.2409999999999997,"15min":5.2409999999999997},"last":0.60699999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.80400000000000005,"15min":0.82099999999999995},"min":{"1min":0.63,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.91500000000000004,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.022,"5min":0.84799999999999998,"15min":0.81899999999999995},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":5.141,"5min":5.141,"15min":5.141},"last":0.63300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":499651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.55800000000000005,"15min":0.49399999999999999},"min":{"1min":0.375,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.55900000000000005,"15min":0.51400000000000001},"min":{"1min":0.35399999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.58799999999999997}]},{"osd":2,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58699999999999997,"15min":0.54800000000000004},"min":{"1min":0.50600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88400000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56699999999999995,"15min":0.52500000000000002},"min":{"1min":0.51200000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51600000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.626,"15min":0.56599999999999995},"min":{"1min":0.41599999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.4710000000000001,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.65200000000000002,"15min":0.59199999999999997},"min":{"1min":0.436,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":0.78900000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.49299999999999999,"15min":0.42299999999999999},"min":{"1min":0.34699999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.48099999999999998,"15min":0.432},"min":{"1min":0.374,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79500000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.624}]},{"osd":5,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.63600000000000001,"15min":0.58899999999999997},"min":{"1min":0.53000000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.66400000000000003,"15min":0.622},"min":{"1min":0.56799999999999995,"5min":0.433,"15min":0.433},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.73299999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.66700000000000004,"15min":0.629},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.71499999999999997,"15min":0.67900000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue Jul 18 03:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.67100000000000004,"15min":0.64000000000000001},"min":{"1min":0.53700000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.73099999999999998,"15min":0.68899999999999995},"min":{"1min":0.59199999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0369999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.88700000000000001}]},{"osd":8,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.81299999999999994,"15min":0.80600000000000005},"min":{"1min":0.51900000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.02},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.78900000000000003,"15min":0.77900000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.86299999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.92400000000000004,"15min":0.91600000000000004},"min":{"1min":0.61299999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.875,"15min":0.86599999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":0.93000000000000005}]},{"osd":10,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.85399999999999998,"15min":0.84799999999999998},"min":{"1min":0.62,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.141,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.56699999999999995,"5min":0.439,"15min":0.439},"max":{"1min":1.159,"5min":1.528,"15min":1.528},"last":1.0669999999999999}]},{"osd":11,"last update":"Tue Jul 18 03:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":1.0449999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":436912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.60699999999999998,"15min":0.53800000000000003},"min":{"1min":0.32700000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":7.0199999999999996,"5min":7.0199999999999996,"15min":7.0199999999999996},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.50700000000000001},"min":{"1min":0.35299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.5}]},{"osd":2,"last update":"Tue Jul 18 03:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.53600000000000003},"min":{"1min":0.39500000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.58699999999999997,"15min":0.53500000000000003},"min":{"1min":0.41799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.60399999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.373},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":7.274,"5min":7.274,"15min":7.274},"last":0.42899999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58099999999999996,"15min":0.55500000000000005},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.379,"5min":0.378,"15min":0.378},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.52100000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.67500000000000004,"15min":0.63},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":7.1079999999999997,"5min":7.1079999999999997,"15min":7.1079999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.63200000000000001,"15min":0.623},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.57499999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56799999999999995,"15min":0.55100000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.621,"15min":0.57099999999999995},"min":{"1min":0.34799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.2130000000000001,"5min":7.2130000000000001,"15min":7.2130000000000001},"last":0.59399999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.47099999999999997}]},{"osd":8,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.81499999999999995,"15min":0.79200000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.1449999999999996,"5min":7.1449999999999996,"15min":7.1449999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.77000000000000002,"15min":0.79000000000000004},"min":{"1min":0.47999999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96799999999999997,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.64100000000000001}]},{"osd":9,"last update":"Tue Jul 18 03:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":0.85999999999999999,"15min":0.80800000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":12.891,"5min":12.891,"15min":12.891},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.83199999999999996,"15min":0.81100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":7.2439999999999998,"5min":7.2439999999999998,"15min":7.2439999999999998},"last":0.46400000000000002}]},{"osd":10,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.77300000000000002,"15min":0.78800000000000003},"min":{"1min":0.53600000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0680000000000001,"5min":1.177,"15min":1.177},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.83799999999999997,"15min":0.81599999999999995},"min":{"1min":0.5,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":7.0720000000000001,"5min":7.0720000000000001,"15min":7.0720000000000001},"last":0.65600000000000003}]},{"osd":11,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":7.1790000000000003,"5min":7.1790000000000003,"15min":7.1790000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.64800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":495507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.39800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.55500000000000005},"min":{"1min":0.36099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82199999999999995,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.53300000000000003}]},{"osd":1,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78400000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.70099999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61699999999999999,"15min":0.58599999999999997},"min":{"1min":0.48299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.503},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88600000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.65400000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.73399999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.41699999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.45100000000000001,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.88200000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.59299999999999997,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.61599999999999999,"15min":0.58399999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89900000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.81499999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.505,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.61699999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.378,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.94299999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.73299999999999998,"15min":0.69499999999999995},"min":{"1min":0.48599999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":5.8890000000000002,"5min":5.8890000000000002,"15min":5.8890000000000002},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.97399999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.70399999999999996,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.71499999999999997,"15min":0.69299999999999995},"min":{"1min":0.746,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":1.044}]},{"osd":11,"last update":"Tue Jul 18 03:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":1.0209999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":435458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.377,"15min":0.34599999999999997},"min":{"1min":0.26600000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.88500000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.372},"min":{"1min":0.28999999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.30099999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58999999999999997,"15min":0.53500000000000003},"min":{"1min":0.39500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3660000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.51300000000000001},"min":{"1min":0.39100000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.40100000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57799999999999996,"15min":0.55700000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.3080000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61899999999999999,"15min":0.56999999999999995},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3919999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.751}]},{"osd":4,"last update":"Tue Jul 18 03:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.58299999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.93600000000000005,"5min":1.754,"15min":1.754},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66200000000000003,"15min":0.60999999999999999},"min":{"1min":0.39900000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93300000000000005,"5min":1.744,"15min":1.744},"last":0.64200000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.50600000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.357,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.436,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3460000000000001,"5min":1.827,"15min":1.827},"last":0.71699999999999997}]},{"osd":6,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.95199999999999996,"5min":1.002,"15min":1.002},"last":0.435},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.47699999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.66700000000000004,"15min":0.67400000000000004},"min":{"1min":0.41499999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82799999999999996,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.48399999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95999999999999996,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.83999999999999997}]},{"osd":8,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.49399999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.82999999999999996,"15min":0.85199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92500000000000004,"5min":1.113,"15min":1.113},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.74399999999999999,"15min":0.76200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.995,"5min":0.997,"15min":0.997},"last":0.52500000000000002}]},{"osd":10,"last update":"Tue Jul 18 03:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.79100000000000004,"15min":0.81000000000000005},"min":{"1min":0.54000000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.90300000000000002,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.78200000000000003,"15min":0.80500000000000005},"min":{"1min":0.496,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91800000000000004,"5min":1.786,"15min":1.786},"last":0.499}]},{"osd":11,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.74099999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953528,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":435434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53200000000000003,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.54700000000000004,"15min":0.50700000000000001},"min":{"1min":0.48099999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89500000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.502}]},{"osd":1,"last update":"Tue Jul 18 03:30:23 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.42299999999999999,"15min":0.39400000000000002},"min":{"1min":0.28199999999999997,"5min":0.246,"15min":0.246},"max":{"1min":0.66000000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.436,"15min":0.39100000000000001},"min":{"1min":0.36599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67400000000000004,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.49299999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:30:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.64300000000000002,"15min":0.58499999999999996},"min":{"1min":0.438,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60699999999999998,"15min":0.56799999999999995},"min":{"1min":0.46400000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79100000000000004,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56499999999999995}]},{"osd":3,"last update":"Tue Jul 18 03:30:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61399999999999999,"15min":0.55900000000000005},"min":{"1min":0.46500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.55200000000000005},"min":{"1min":0.40400000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.91200000000000003,"5min":1.038,"15min":1.038},"last":0.52200000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63,"15min":0.61399999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89200000000000002,"5min":1.038,"15min":1.038},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.64300000000000002,"15min":0.61599999999999999},"min":{"1min":0.58599999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.69099999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.63700000000000001,"15min":0.60499999999999998},"min":{"1min":0.41599999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.97599999999999998,"5min":1.085,"15min":1.085},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.41699999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.629}]},{"osd":7,"last update":"Tue Jul 18 03:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.65400000000000003,"15min":0.63400000000000001},"min":{"1min":0.51300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.99399999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.67000000000000004,"15min":0.64800000000000002},"min":{"1min":0.54200000000000004,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.999,"5min":1.121,"15min":1.121},"last":0.81299999999999994}]},{"osd":8,"last update":"Tue Jul 18 03:29:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71399999999999997,"15min":0.71499999999999997},"min":{"1min":0.53000000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91100000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.78200000000000003,"15min":0.77500000000000002},"min":{"1min":0.59999999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0920000000000001,"5min":1.208,"15min":1.208},"last":0.82899999999999996}]},{"osd":9,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.78400000000000003,"15min":0.77000000000000002},"min":{"1min":0.69299999999999995,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.879,"5min":0.80700000000000005,"15min":0.79500000000000004},"min":{"1min":0.60899999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1379999999999999,"5min":1.163,"15min":1.163},"last":0.84899999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.84799999999999998,"15min":0.83899999999999997},"min":{"1min":0.63300000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.073,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.82099999999999995,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.101,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.876}]},{"osd":11,"last update":"Tue Jul 18 03:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.86099999999999999}]}]},{"osd":5,"up_from":37,"seq":158913790005,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1544192,"data_stored":1028753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.78900000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.624},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.31900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.47399999999999998}]},{"osd":1,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.54800000000000004,"15min":0.51700000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.55800000000000005,"15min":0.52900000000000003},"min":{"1min":0.41399999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.50900000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.40799999999999997,"15min":0.38400000000000001},"min":{"1min":0.29099999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.43099999999999999,"15min":0.41499999999999998},"min":{"1min":0.29899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71299999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.442}]},{"osd":3,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.55900000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75800000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.60599999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.59699999999999998,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.59899999999999998,"15min":0.56599999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.68999999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:30:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.623},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82599999999999996,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.63800000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.496,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94199999999999995,"5min":1.633,"15min":1.633},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.49299999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89100000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.70399999999999996}]},{"osd":8,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85199999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94099999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.754}]},{"osd":9,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.76100000000000001,"15min":0.75900000000000001},"min":{"1min":0.52800000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94799999999999995,"5min":1.024,"15min":1.024},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.754,"15min":0.75600000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93200000000000005,"5min":1.008,"15min":1.008},"last":0.74099999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.80100000000000005,"15min":0.79500000000000004},"min":{"1min":0.69399999999999995,"5min":0.504,"15min":0.504},"max":{"1min":0.98599999999999999,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77700000000000002,"15min":0.77100000000000002},"min":{"1min":0.67600000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.97299999999999998,"5min":1.548,"15min":1.548},"last":0.77000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:30:48.209 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-07-18T03:30:48.210 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-18T03:30:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.372+0000 7f279f901700 1 -- 172.21.15.40:0/3926253606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27980fe9d0 msgr2=0x7f27980fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.372+0000 7f279f901700 1 --2- 172.21.15.40:0/3926253606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27980fe9d0 0x7f27980fede0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f279000b0a0 tx=0x7f279000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.372+0000 7f279f901700 1 -- 172.21.15.40:0/3926253606 shutdown_connections 2023-07-18T03:30:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.372+0000 7f279f901700 1 --2- 172.21.15.40:0/3926253606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2798100550 0x7f2798106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.372+0000 7f279f901700 1 --2- 172.21.15.40:0/3926253606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27980fe9d0 0x7f27980fede0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.372+0000 7f279f901700 1 --2- 172.21.15.40:0/3926253606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27980ffbc0 0x7f2798100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.372+0000 7f279f901700 1 -- 172.21.15.40:0/3926253606 >> 172.21.15.40:0/3926253606 conn(0x7f27980f9fd0 msgr2=0x7f27980fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.372+0000 7f279f901700 1 -- 172.21.15.40:0/3926253606 shutdown_connections 2023-07-18T03:30:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.372+0000 7f279f901700 1 -- 172.21.15.40:0/3926253606 wait complete. 2023-07-18T03:30:48.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279f901700 1 Processor -- start 2023-07-18T03:30:48.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279f901700 1 -- start start 2023-07-18T03:30:48.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279f901700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27980fe9d0 0x7f2798194890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:48.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279f901700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27980ffbc0 0x7f2798194dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:48.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279f901700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2798100550 0x7f279819be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:48.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279f901700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279819c410 con 0x7f27980fe9d0 2023-07-18T03:30:48.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279f901700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279819c550 con 0x7f27980ffbc0 2023-07-18T03:30:48.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279f901700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279819c6e0 con 0x7f2798100550 2023-07-18T03:30:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279d69d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27980fe9d0 0x7f2798194890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279d69d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27980fe9d0 0x7f2798194890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57778/0 (socket says 172.21.15.40:57778) 2023-07-18T03:30:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279d69d700 1 -- 172.21.15.40:0/1008953100 learned_addr learned my addr 172.21.15.40:0/1008953100 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279ce9c700 1 --2- 172.21.15.40:0/1008953100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27980ffbc0 0x7f2798194dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279de9e700 1 --2- 172.21.15.40:0/1008953100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2798100550 0x7f279819be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279d69d700 1 -- 172.21.15.40:0/1008953100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2798100550 msgr2=0x7f279819be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:48.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279d69d700 1 --2- 172.21.15.40:0/1008953100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2798100550 0x7f279819be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279d69d700 1 -- 172.21.15.40:0/1008953100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27980ffbc0 msgr2=0x7f2798194dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279d69d700 1 --2- 172.21.15.40:0/1008953100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27980ffbc0 0x7f2798194dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.373+0000 7f279d69d700 1 -- 172.21.15.40:0/1008953100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2790009d00 con 0x7f27980fe9d0 2023-07-18T03:30:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.374+0000 7f279ce9c700 1 --2- 172.21.15.40:0/1008953100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27980ffbc0 0x7f2798194dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.374+0000 7f279d69d700 1 --2- 172.21.15.40:0/1008953100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27980fe9d0 0x7f2798194890 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f279000b070 tx=0x7f2790004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.374+0000 7f278e7fc700 1 -- 172.21.15.40:0/1008953100 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000e040 con 0x7f27980fe9d0 2023-07-18T03:30:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.374+0000 7f278e7fc700 1 -- 172.21.15.40:0/1008953100 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2790009440 con 0x7f27980fe9d0 2023-07-18T03:30:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.374+0000 7f279f901700 1 -- 172.21.15.40:0/1008953100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f279819c960 con 0x7f27980fe9d0 2023-07-18T03:30:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.374+0000 7f278e7fc700 1 -- 172.21.15.40:0/1008953100 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2790004020 con 0x7f27980fe9d0 2023-07-18T03:30:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.374+0000 7f279f901700 1 -- 172.21.15.40:0/1008953100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f279819ce30 con 0x7f27980fe9d0 2023-07-18T03:30:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.375+0000 7f279f901700 1 -- 172.21.15.40:0/1008953100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2798105f70 con 0x7f27980fe9d0 2023-07-18T03:30:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.376+0000 7f278e7fc700 1 -- 172.21.15.40:0/1008953100 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f279001d720 con 0x7f27980fe9d0 2023-07-18T03:30:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.376+0000 7f278e7fc700 1 --2- 172.21.15.40:0/1008953100 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f27840a12a0 0x7f27840a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:48.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.377+0000 7f279ce9c700 1 --2- 172.21.15.40:0/1008953100 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f27840a12a0 0x7f27840a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:48.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.377+0000 7f278e7fc700 1 -- 172.21.15.40:0/1008953100 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f27900ce9d0 con 0x7f27980fe9d0 2023-07-18T03:30:48.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.377+0000 7f279ce9c700 1 --2- 172.21.15.40:0/1008953100 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f27840a12a0 0x7f27840a3750 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2788005950 tx=0x7f27880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:48.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.380+0000 7f278e7fc700 1 -- 172.21.15.40:0/1008953100 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2790098050 con 0x7f27980fe9d0 2023-07-18T03:30:48.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.573+0000 7f279f901700 1 -- 172.21.15.40:0/1008953100 --> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2798060950 con 0x7f27840a12a0 2023-07-18T03:30:48.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.585+0000 7f278e7fc700 1 -- 172.21.15.40:0/1008953100 <== mgr.14152 v2:172.21.15.40:6800/3612011194 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343917 (secure 0 0 0) 0x7f2798060950 con 0x7f27840a12a0 2023-07-18T03:30:48.585 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:48.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.588+0000 7f279f901700 1 -- 172.21.15.40:0/1008953100 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f27840a12a0 msgr2=0x7f27840a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:48.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.588+0000 7f279f901700 1 --2- 172.21.15.40:0/1008953100 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f27840a12a0 0x7f27840a3750 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2788005950 tx=0x7f27880058e0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.588+0000 7f279f901700 1 -- 172.21.15.40:0/1008953100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27980fe9d0 msgr2=0x7f2798194890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:48.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.588+0000 7f279f901700 1 --2- 172.21.15.40:0/1008953100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27980fe9d0 0x7f2798194890 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f279000b070 tx=0x7f2790004db0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.589+0000 7f279f901700 1 -- 172.21.15.40:0/1008953100 shutdown_connections 2023-07-18T03:30:48.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.589+0000 7f279f901700 1 --2- 172.21.15.40:0/1008953100 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f27840a12a0 0x7f27840a3750 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.589+0000 7f279f901700 1 --2- 172.21.15.40:0/1008953100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2798100550 0x7f279819be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.589+0000 7f279f901700 1 --2- 172.21.15.40:0/1008953100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27980fe9d0 0x7f2798194890 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.589+0000 7f279f901700 1 --2- 172.21.15.40:0/1008953100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27980ffbc0 0x7f2798194dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.589+0000 7f279f901700 1 -- 172.21.15.40:0/1008953100 >> 172.21.15.40:0/1008953100 conn(0x7f27980f9fd0 msgr2=0x7f27981030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:48.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.589+0000 7f279f901700 1 -- 172.21.15.40:0/1008953100 shutdown_connections 2023-07-18T03:30:48.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.589+0000 7f279f901700 1 -- 172.21.15.40:0/1008953100 wait complete. 2023-07-18T03:30:48.588 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-07-18T03:30:48.597 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2023-07-18T03:30:47.835920+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656397,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":892,"num_read_kb":564,"num_write":416,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1180826,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336300,"kb_used_data":12876,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605524,"statfs":{"total":1151940427776,"available":1151596056576,"internally_reserved":0,"allocated":13185024,"data_stored":7189928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":636,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":57,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003857"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656547+0000","last_change":"2023-07-18T03:29:27.101623+0000","last_active":"2023-07-18T03:30:33.656547+0000","last_peered":"2023-07-18T03:30:33.656547+0000","last_clean":"2023-07-18T03:30:33.656547+0000","last_became_active":"2023-07-18T03:29:27.101297+0000","last_became_peered":"2023-07-18T03:29:27.101297+0000","last_unstale":"2023-07-18T03:30:33.656547+0000","last_undegraded":"2023-07-18T03:30:33.656547+0000","last_fullsized":"2023-07-18T03:30:33.656547+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:38:25.465606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655619+0000","last_change":"2023-07-18T03:29:27.104159+0000","last_active":"2023-07-18T03:30:33.655619+0000","last_peered":"2023-07-18T03:30:33.655619+0000","last_clean":"2023-07-18T03:30:33.655619+0000","last_became_active":"2023-07-18T03:29:27.103830+0000","last_became_peered":"2023-07-18T03:29:27.103830+0000","last_unstale":"2023-07-18T03:30:33.655619+0000","last_undegraded":"2023-07-18T03:30:33.655619+0000","last_fullsized":"2023-07-18T03:30:33.655619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:02:01.106624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655451+0000","last_change":"2023-07-18T03:29:27.098858+0000","last_active":"2023-07-18T03:30:33.655451+0000","last_peered":"2023-07-18T03:30:33.655451+0000","last_clean":"2023-07-18T03:30:33.655451+0000","last_became_active":"2023-07-18T03:29:27.098540+0000","last_became_peered":"2023-07-18T03:29:27.098540+0000","last_unstale":"2023-07-18T03:30:33.655451+0000","last_undegraded":"2023-07-18T03:30:33.655451+0000","last_fullsized":"2023-07-18T03:30:33.655451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:59:37.520654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.658876+0000","last_change":"2023-07-18T03:29:27.103106+0000","last_active":"2023-07-18T03:30:33.658876+0000","last_peered":"2023-07-18T03:30:33.658876+0000","last_clean":"2023-07-18T03:30:33.658876+0000","last_became_active":"2023-07-18T03:29:27.102780+0000","last_became_peered":"2023-07-18T03:29:27.102780+0000","last_unstale":"2023-07-18T03:30:33.658876+0000","last_undegraded":"2023-07-18T03:30:33.658876+0000","last_fullsized":"2023-07-18T03:30:33.658876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:50:45.417621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.657667+0000","last_change":"2023-07-18T03:29:27.095432+0000","last_active":"2023-07-18T03:30:33.657667+0000","last_peered":"2023-07-18T03:30:33.657667+0000","last_clean":"2023-07-18T03:30:33.657667+0000","last_became_active":"2023-07-18T03:29:27.095103+0000","last_became_peered":"2023-07-18T03:29:27.095103+0000","last_unstale":"2023-07-18T03:30:33.657667+0000","last_undegraded":"2023-07-18T03:30:33.657667+0000","last_fullsized":"2023-07-18T03:30:33.657667+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:34:25.462492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660813+0000","last_change":"2023-07-18T03:29:27.091412+0000","last_active":"2023-07-18T03:30:33.660813+0000","last_peered":"2023-07-18T03:30:33.660813+0000","last_clean":"2023-07-18T03:30:33.660813+0000","last_became_active":"2023-07-18T03:29:27.091060+0000","last_became_peered":"2023-07-18T03:29:27.091060+0000","last_unstale":"2023-07-18T03:30:33.660813+0000","last_undegraded":"2023-07-18T03:30:33.660813+0000","last_fullsized":"2023-07-18T03:30:33.660813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:56:48.388976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.748981+0000","last_change":"2023-07-18T03:29:27.105223+0000","last_active":"2023-07-18T03:30:33.748981+0000","last_peered":"2023-07-18T03:30:33.748981+0000","last_clean":"2023-07-18T03:30:33.748981+0000","last_became_active":"2023-07-18T03:29:27.104926+0000","last_became_peered":"2023-07-18T03:29:27.104926+0000","last_unstale":"2023-07-18T03:30:33.748981+0000","last_undegraded":"2023-07-18T03:30:33.748981+0000","last_fullsized":"2023-07-18T03:30:33.748981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:53:06.895279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660183+0000","last_change":"2023-07-18T03:29:27.093722+0000","last_active":"2023-07-18T03:30:33.660183+0000","last_peered":"2023-07-18T03:30:33.660183+0000","last_clean":"2023-07-18T03:30:33.660183+0000","last_became_active":"2023-07-18T03:29:27.093103+0000","last_became_peered":"2023-07-18T03:29:27.093103+0000","last_unstale":"2023-07-18T03:30:33.660183+0000","last_undegraded":"2023-07-18T03:30:33.660183+0000","last_fullsized":"2023-07-18T03:30:33.660183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:57:08.943397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.651027+0000","last_change":"2023-07-18T03:29:27.094681+0000","last_active":"2023-07-18T03:30:33.651027+0000","last_peered":"2023-07-18T03:30:33.651027+0000","last_clean":"2023-07-18T03:30:33.651027+0000","last_became_active":"2023-07-18T03:29:27.094284+0000","last_became_peered":"2023-07-18T03:29:27.094284+0000","last_unstale":"2023-07-18T03:30:33.651027+0000","last_undegraded":"2023-07-18T03:30:33.651027+0000","last_fullsized":"2023-07-18T03:30:33.651027+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:25:26.164002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661512+0000","last_change":"2023-07-18T03:29:27.100759+0000","last_active":"2023-07-18T03:30:33.661512+0000","last_peered":"2023-07-18T03:30:33.661512+0000","last_clean":"2023-07-18T03:30:33.661512+0000","last_became_active":"2023-07-18T03:29:27.096660+0000","last_became_peered":"2023-07-18T03:29:27.096660+0000","last_unstale":"2023-07-18T03:30:33.661512+0000","last_undegraded":"2023-07-18T03:30:33.661512+0000","last_fullsized":"2023-07-18T03:30:33.661512+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:32:57.131034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749504+0000","last_change":"2023-07-18T03:29:27.107251+0000","last_active":"2023-07-18T03:30:33.749504+0000","last_peered":"2023-07-18T03:30:33.749504+0000","last_clean":"2023-07-18T03:30:33.749504+0000","last_became_active":"2023-07-18T03:29:27.106895+0000","last_became_peered":"2023-07-18T03:29:27.106895+0000","last_unstale":"2023-07-18T03:30:33.749504+0000","last_undegraded":"2023-07-18T03:30:33.749504+0000","last_fullsized":"2023-07-18T03:30:33.749504+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:27:10.110391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.658742+0000","last_change":"2023-07-18T03:29:27.103905+0000","last_active":"2023-07-18T03:30:33.658742+0000","last_peered":"2023-07-18T03:30:33.658742+0000","last_clean":"2023-07-18T03:30:33.658742+0000","last_became_active":"2023-07-18T03:29:27.103021+0000","last_became_peered":"2023-07-18T03:29:27.103021+0000","last_unstale":"2023-07-18T03:30:33.658742+0000","last_undegraded":"2023-07-18T03:30:33.658742+0000","last_fullsized":"2023-07-18T03:30:33.658742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:56:32.967945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655982+0000","last_change":"2023-07-18T03:29:27.103529+0000","last_active":"2023-07-18T03:30:33.655982+0000","last_peered":"2023-07-18T03:30:33.655982+0000","last_clean":"2023-07-18T03:30:33.655982+0000","last_became_active":"2023-07-18T03:29:27.103217+0000","last_became_peered":"2023-07-18T03:29:27.103217+0000","last_unstale":"2023-07-18T03:30:33.655982+0000","last_undegraded":"2023-07-18T03:30:33.655982+0000","last_fullsized":"2023-07-18T03:30:33.655982+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:36:22.276029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655161+0000","last_change":"2023-07-18T03:29:27.096640+0000","last_active":"2023-07-18T03:30:33.655161+0000","last_peered":"2023-07-18T03:30:33.655161+0000","last_clean":"2023-07-18T03:30:33.655161+0000","last_became_active":"2023-07-18T03:29:27.096347+0000","last_became_peered":"2023-07-18T03:29:27.096347+0000","last_unstale":"2023-07-18T03:30:33.655161+0000","last_undegraded":"2023-07-18T03:30:33.655161+0000","last_fullsized":"2023-07-18T03:30:33.655161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:51:47.250203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655071+0000","last_change":"2023-07-18T03:29:27.095910+0000","last_active":"2023-07-18T03:30:33.655071+0000","last_peered":"2023-07-18T03:30:33.655071+0000","last_clean":"2023-07-18T03:30:33.655071+0000","last_became_active":"2023-07-18T03:29:27.095556+0000","last_became_peered":"2023-07-18T03:29:27.095556+0000","last_unstale":"2023-07-18T03:30:33.655071+0000","last_undegraded":"2023-07-18T03:30:33.655071+0000","last_fullsized":"2023-07-18T03:30:33.655071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:17:31.377765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650364+0000","last_change":"2023-07-18T03:29:27.108220+0000","last_active":"2023-07-18T03:30:33.650364+0000","last_peered":"2023-07-18T03:30:33.650364+0000","last_clean":"2023-07-18T03:30:33.650364+0000","last_became_active":"2023-07-18T03:29:27.107869+0000","last_became_peered":"2023-07-18T03:29:27.107869+0000","last_unstale":"2023-07-18T03:30:33.650364+0000","last_undegraded":"2023-07-18T03:30:33.650364+0000","last_fullsized":"2023-07-18T03:30:33.650364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:57:32.649126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656345+0000","last_change":"2023-07-18T03:29:27.100634+0000","last_active":"2023-07-18T03:30:33.656345+0000","last_peered":"2023-07-18T03:30:33.656345+0000","last_clean":"2023-07-18T03:30:33.656345+0000","last_became_active":"2023-07-18T03:29:27.100316+0000","last_became_peered":"2023-07-18T03:29:27.100316+0000","last_unstale":"2023-07-18T03:30:33.656345+0000","last_undegraded":"2023-07-18T03:30:33.656345+0000","last_fullsized":"2023-07-18T03:30:33.656345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:24:14.323573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.652475+0000","last_change":"2023-07-18T03:29:27.100322+0000","last_active":"2023-07-18T03:30:33.652475+0000","last_peered":"2023-07-18T03:30:33.652475+0000","last_clean":"2023-07-18T03:30:33.652475+0000","last_became_active":"2023-07-18T03:29:27.099782+0000","last_became_peered":"2023-07-18T03:29:27.099782+0000","last_unstale":"2023-07-18T03:30:33.652475+0000","last_undegraded":"2023-07-18T03:30:33.652475+0000","last_fullsized":"2023-07-18T03:30:33.652475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:03:29.585791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.654114+0000","last_change":"2023-07-18T03:29:27.096639+0000","last_active":"2023-07-18T03:30:33.654114+0000","last_peered":"2023-07-18T03:30:33.654114+0000","last_clean":"2023-07-18T03:30:33.654114+0000","last_became_active":"2023-07-18T03:29:27.096348+0000","last_became_peered":"2023-07-18T03:29:27.096348+0000","last_unstale":"2023-07-18T03:30:33.654114+0000","last_undegraded":"2023-07-18T03:30:33.654114+0000","last_fullsized":"2023-07-18T03:30:33.654114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:57:04.900158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":103,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:43.760810+0000","last_change":"2023-07-18T03:29:26.287722+0000","last_active":"2023-07-18T03:30:43.760810+0000","last_peered":"2023-07-18T03:30:43.760810+0000","last_clean":"2023-07-18T03:30:43.760810+0000","last_became_active":"2023-07-18T03:29:26.287327+0000","last_became_peered":"2023-07-18T03:29:26.287327+0000","last_unstale":"2023-07-18T03:30:43.760810+0000","last_undegraded":"2023-07-18T03:30:43.760810+0000","last_fullsized":"2023-07-18T03:30:43.760810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:00:13.869444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661775+0000","last_change":"2023-07-18T03:29:27.092771+0000","last_active":"2023-07-18T03:30:33.661775+0000","last_peered":"2023-07-18T03:30:33.661775+0000","last_clean":"2023-07-18T03:30:33.661775+0000","last_became_active":"2023-07-18T03:29:27.092456+0000","last_became_peered":"2023-07-18T03:29:27.092456+0000","last_unstale":"2023-07-18T03:30:33.661775+0000","last_undegraded":"2023-07-18T03:30:33.661775+0000","last_fullsized":"2023-07-18T03:30:33.661775+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:36:58.677882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.662061+0000","last_change":"2023-07-18T03:29:26.289683+0000","last_active":"2023-07-18T03:30:33.662061+0000","last_peered":"2023-07-18T03:30:33.662061+0000","last_clean":"2023-07-18T03:30:33.662061+0000","last_became_active":"2023-07-18T03:29:26.289356+0000","last_became_peered":"2023-07-18T03:29:26.289356+0000","last_unstale":"2023-07-18T03:30:33.662061+0000","last_undegraded":"2023-07-18T03:30:33.662061+0000","last_fullsized":"2023-07-18T03:30:33.662061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:32:24.704699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655443+0000","last_change":"2023-07-18T03:29:27.098709+0000","last_active":"2023-07-18T03:30:33.655443+0000","last_peered":"2023-07-18T03:30:33.655443+0000","last_clean":"2023-07-18T03:30:33.655443+0000","last_became_active":"2023-07-18T03:29:27.098395+0000","last_became_peered":"2023-07-18T03:29:27.098395+0000","last_unstale":"2023-07-18T03:30:33.655443+0000","last_undegraded":"2023-07-18T03:30:33.655443+0000","last_fullsized":"2023-07-18T03:30:33.655443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:21:57.798041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.658004+0000","last_change":"2023-07-18T03:29:26.297119+0000","last_active":"2023-07-18T03:30:33.658004+0000","last_peered":"2023-07-18T03:30:33.658004+0000","last_clean":"2023-07-18T03:30:33.658004+0000","last_became_active":"2023-07-18T03:29:26.296649+0000","last_became_peered":"2023-07-18T03:29:26.296649+0000","last_unstale":"2023-07-18T03:30:33.658004+0000","last_undegraded":"2023-07-18T03:30:33.658004+0000","last_fullsized":"2023-07-18T03:30:33.658004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:19:40.529493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650567+0000","last_change":"2023-07-18T03:29:27.105511+0000","last_active":"2023-07-18T03:30:33.650567+0000","last_peered":"2023-07-18T03:30:33.650567+0000","last_clean":"2023-07-18T03:30:33.650567+0000","last_became_active":"2023-07-18T03:29:27.104946+0000","last_became_peered":"2023-07-18T03:29:27.104946+0000","last_unstale":"2023-07-18T03:30:33.650567+0000","last_undegraded":"2023-07-18T03:30:33.650567+0000","last_fullsized":"2023-07-18T03:30:33.650567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:27:54.324118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.657412+0000","last_change":"2023-07-18T03:29:26.290708+0000","last_active":"2023-07-18T03:30:33.657412+0000","last_peered":"2023-07-18T03:30:33.657412+0000","last_clean":"2023-07-18T03:30:33.657412+0000","last_became_active":"2023-07-18T03:29:26.290383+0000","last_became_peered":"2023-07-18T03:29:26.290383+0000","last_unstale":"2023-07-18T03:30:33.657412+0000","last_undegraded":"2023-07-18T03:30:33.657412+0000","last_fullsized":"2023-07-18T03:30:33.657412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:23:20.274883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660376+0000","last_change":"2023-07-18T03:29:27.096630+0000","last_active":"2023-07-18T03:30:33.660376+0000","last_peered":"2023-07-18T03:30:33.660376+0000","last_clean":"2023-07-18T03:30:33.660376+0000","last_became_active":"2023-07-18T03:29:27.096337+0000","last_became_peered":"2023-07-18T03:29:27.096337+0000","last_unstale":"2023-07-18T03:30:33.660376+0000","last_undegraded":"2023-07-18T03:30:33.660376+0000","last_fullsized":"2023-07-18T03:30:33.660376+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:01:51.889873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'7","reported_seq":104,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:44.718292+0000","last_change":"2023-07-18T03:29:26.291622+0000","last_active":"2023-07-18T03:30:44.718292+0000","last_peered":"2023-07-18T03:30:44.718292+0000","last_clean":"2023-07-18T03:30:44.718292+0000","last_became_active":"2023-07-18T03:29:26.291317+0000","last_became_peered":"2023-07-18T03:29:26.291317+0000","last_unstale":"2023-07-18T03:30:44.718292+0000","last_undegraded":"2023-07-18T03:30:44.718292+0000","last_fullsized":"2023-07-18T03:30:44.718292+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:44:15.122562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1282,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660520+0000","last_change":"2023-07-18T03:30:33.660520+0000","last_active":"2023-07-18T03:30:33.660520+0000","last_peered":"2023-07-18T03:30:33.660520+0000","last_clean":"2023-07-18T03:30:33.660520+0000","last_became_active":"2023-07-18T03:30:33.660098+0000","last_became_peered":"2023-07-18T03:30:33.660098+0000","last_unstale":"2023-07-18T03:30:33.660520+0000","last_undegraded":"2023-07-18T03:30:33.660520+0000","last_fullsized":"2023-07-18T03:30:33.660520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:16:28.408741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660746+0000","last_change":"2023-07-18T03:29:26.290493+0000","last_active":"2023-07-18T03:30:33.660746+0000","last_peered":"2023-07-18T03:30:33.660746+0000","last_clean":"2023-07-18T03:30:33.660746+0000","last_became_active":"2023-07-18T03:29:26.290187+0000","last_became_peered":"2023-07-18T03:29:26.290187+0000","last_unstale":"2023-07-18T03:30:33.660746+0000","last_undegraded":"2023-07-18T03:30:33.660746+0000","last_fullsized":"2023-07-18T03:30:33.660746+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:58:16.632361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.659999+0000","last_change":"2023-07-18T03:29:27.095129+0000","last_active":"2023-07-18T03:30:33.659999+0000","last_peered":"2023-07-18T03:30:33.659999+0000","last_clean":"2023-07-18T03:30:33.659999+0000","last_became_active":"2023-07-18T03:29:27.094821+0000","last_became_peered":"2023-07-18T03:29:27.094821+0000","last_unstale":"2023-07-18T03:30:33.659999+0000","last_undegraded":"2023-07-18T03:30:33.659999+0000","last_fullsized":"2023-07-18T03:30:33.659999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:43:40.005624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655778+0000","last_change":"2023-07-18T03:29:26.285860+0000","last_active":"2023-07-18T03:30:33.655778+0000","last_peered":"2023-07-18T03:30:33.655778+0000","last_clean":"2023-07-18T03:30:33.655778+0000","last_became_active":"2023-07-18T03:29:26.285491+0000","last_became_peered":"2023-07-18T03:29:26.285491+0000","last_unstale":"2023-07-18T03:30:33.655778+0000","last_undegraded":"2023-07-18T03:30:33.655778+0000","last_fullsized":"2023-07-18T03:30:33.655778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:39:26.999366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660058+0000","last_change":"2023-07-18T03:29:27.093861+0000","last_active":"2023-07-18T03:30:33.660058+0000","last_peered":"2023-07-18T03:30:33.660058+0000","last_clean":"2023-07-18T03:30:33.660058+0000","last_became_active":"2023-07-18T03:29:27.093445+0000","last_became_peered":"2023-07-18T03:29:27.093445+0000","last_unstale":"2023-07-18T03:30:33.660058+0000","last_undegraded":"2023-07-18T03:30:33.660058+0000","last_fullsized":"2023-07-18T03:30:33.660058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:45:32.889832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.659732+0000","last_change":"2023-07-18T03:29:26.288108+0000","last_active":"2023-07-18T03:30:33.659732+0000","last_peered":"2023-07-18T03:30:33.659732+0000","last_clean":"2023-07-18T03:30:33.659732+0000","last_became_active":"2023-07-18T03:29:26.287531+0000","last_became_peered":"2023-07-18T03:29:26.287531+0000","last_unstale":"2023-07-18T03:30:33.659732+0000","last_undegraded":"2023-07-18T03:30:33.659732+0000","last_fullsized":"2023-07-18T03:30:33.659732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:52:58.383534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.652172+0000","last_change":"2023-07-18T03:29:27.092908+0000","last_active":"2023-07-18T03:30:33.652172+0000","last_peered":"2023-07-18T03:30:33.652172+0000","last_clean":"2023-07-18T03:30:33.652172+0000","last_became_active":"2023-07-18T03:29:27.092566+0000","last_became_peered":"2023-07-18T03:29:27.092566+0000","last_unstale":"2023-07-18T03:30:33.652172+0000","last_undegraded":"2023-07-18T03:30:33.652172+0000","last_fullsized":"2023-07-18T03:30:33.652172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:46:32.375687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":168,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:44.717016+0000","last_change":"2023-07-18T03:29:26.294251+0000","last_active":"2023-07-18T03:30:44.717016+0000","last_peered":"2023-07-18T03:30:44.717016+0000","last_clean":"2023-07-18T03:30:44.717016+0000","last_became_active":"2023-07-18T03:29:26.293847+0000","last_became_peered":"2023-07-18T03:29:26.293847+0000","last_unstale":"2023-07-18T03:30:44.717016+0000","last_undegraded":"2023-07-18T03:30:44.717016+0000","last_fullsized":"2023-07-18T03:30:44.717016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:29:39.028643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":145,"num_read_kb":145,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650892+0000","last_change":"2023-07-18T03:29:27.106551+0000","last_active":"2023-07-18T03:30:33.650892+0000","last_peered":"2023-07-18T03:30:33.650892+0000","last_clean":"2023-07-18T03:30:33.650892+0000","last_became_active":"2023-07-18T03:29:27.105187+0000","last_became_peered":"2023-07-18T03:29:27.105187+0000","last_unstale":"2023-07-18T03:30:33.650892+0000","last_undegraded":"2023-07-18T03:30:33.650892+0000","last_fullsized":"2023-07-18T03:30:33.650892+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:59:30.478453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749555+0000","last_change":"2023-07-18T03:29:26.293284+0000","last_active":"2023-07-18T03:30:33.749555+0000","last_peered":"2023-07-18T03:30:33.749555+0000","last_clean":"2023-07-18T03:30:33.749555+0000","last_became_active":"2023-07-18T03:29:26.292955+0000","last_became_peered":"2023-07-18T03:29:26.292955+0000","last_unstale":"2023-07-18T03:30:33.749555+0000","last_undegraded":"2023-07-18T03:30:33.749555+0000","last_fullsized":"2023-07-18T03:30:33.749555+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:14:25.817345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661734+0000","last_change":"2023-07-18T03:30:33.661734+0000","last_active":"2023-07-18T03:30:33.661734+0000","last_peered":"2023-07-18T03:30:33.661734+0000","last_clean":"2023-07-18T03:30:33.661734+0000","last_became_active":"2023-07-18T03:30:33.661363+0000","last_became_peered":"2023-07-18T03:30:33.661363+0000","last_unstale":"2023-07-18T03:30:33.661734+0000","last_undegraded":"2023-07-18T03:30:33.661734+0000","last_fullsized":"2023-07-18T03:30:33.661734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:21:26.542417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660448+0000","last_change":"2023-07-18T03:29:26.290810+0000","last_active":"2023-07-18T03:30:33.660448+0000","last_peered":"2023-07-18T03:30:33.660448+0000","last_clean":"2023-07-18T03:30:33.660448+0000","last_became_active":"2023-07-18T03:29:26.290448+0000","last_became_peered":"2023-07-18T03:29:26.290448+0000","last_unstale":"2023-07-18T03:30:33.660448+0000","last_undegraded":"2023-07-18T03:30:33.660448+0000","last_fullsized":"2023-07-18T03:30:33.660448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:37:32.462232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650723+0000","last_change":"2023-07-18T03:29:27.105801+0000","last_active":"2023-07-18T03:30:33.650723+0000","last_peered":"2023-07-18T03:30:33.650723+0000","last_clean":"2023-07-18T03:30:33.650723+0000","last_became_active":"2023-07-18T03:29:27.104715+0000","last_became_peered":"2023-07-18T03:29:27.104715+0000","last_unstale":"2023-07-18T03:30:33.650723+0000","last_undegraded":"2023-07-18T03:30:33.650723+0000","last_fullsized":"2023-07-18T03:30:33.650723+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:40:38.561885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'31","reported_seq":206,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:44.717360+0000","last_change":"2023-07-18T03:29:26.293306+0000","last_active":"2023-07-18T03:30:44.717360+0000","last_peered":"2023-07-18T03:30:44.717360+0000","last_clean":"2023-07-18T03:30:44.717360+0000","last_became_active":"2023-07-18T03:29:26.292946+0000","last_became_peered":"2023-07-18T03:29:26.292946+0000","last_unstale":"2023-07-18T03:30:44.717360+0000","last_undegraded":"2023-07-18T03:30:44.717360+0000","last_fullsized":"2023-07-18T03:30:44.717360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:24:06.931197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58845,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":157,"num_read_kb":135,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.657688+0000","last_change":"2023-07-18T03:29:27.103941+0000","last_active":"2023-07-18T03:30:33.657688+0000","last_peered":"2023-07-18T03:30:33.657688+0000","last_clean":"2023-07-18T03:30:33.657688+0000","last_became_active":"2023-07-18T03:29:27.103506+0000","last_became_peered":"2023-07-18T03:29:27.103506+0000","last_unstale":"2023-07-18T03:30:33.657688+0000","last_undegraded":"2023-07-18T03:30:33.657688+0000","last_fullsized":"2023-07-18T03:30:33.657688+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:18:23.410060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.676819+0000","last_change":"2023-07-18T03:30:33.676417+0000","last_active":"2023-07-18T03:30:33.676819+0000","last_peered":"2023-07-18T03:30:33.676819+0000","last_clean":"2023-07-18T03:30:33.676819+0000","last_became_active":"2023-07-18T03:30:33.662637+0000","last_became_peered":"2023-07-18T03:30:33.662637+0000","last_unstale":"2023-07-18T03:30:33.676819+0000","last_undegraded":"2023-07-18T03:30:33.676819+0000","last_fullsized":"2023-07-18T03:30:33.676819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:15:40.468772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.750602+0000","last_change":"2023-07-18T03:30:33.750602+0000","last_active":"2023-07-18T03:30:33.750602+0000","last_peered":"2023-07-18T03:30:33.750602+0000","last_clean":"2023-07-18T03:30:33.750602+0000","last_became_active":"2023-07-18T03:30:33.750170+0000","last_became_peered":"2023-07-18T03:30:33.750170+0000","last_unstale":"2023-07-18T03:30:33.750602+0000","last_undegraded":"2023-07-18T03:30:33.750602+0000","last_fullsized":"2023-07-18T03:30:33.750602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:13:32.425846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.659763+0000","last_change":"2023-07-18T03:29:26.290487+0000","last_active":"2023-07-18T03:30:33.659763+0000","last_peered":"2023-07-18T03:30:33.659763+0000","last_clean":"2023-07-18T03:30:33.659763+0000","last_became_active":"2023-07-18T03:29:26.290115+0000","last_became_peered":"2023-07-18T03:29:26.290115+0000","last_unstale":"2023-07-18T03:30:33.659763+0000","last_undegraded":"2023-07-18T03:30:33.659763+0000","last_fullsized":"2023-07-18T03:30:33.659763+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:23:20.930476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749286+0000","last_change":"2023-07-18T03:29:27.108550+0000","last_active":"2023-07-18T03:30:33.749286+0000","last_peered":"2023-07-18T03:30:33.749286+0000","last_clean":"2023-07-18T03:30:33.749286+0000","last_became_active":"2023-07-18T03:29:27.108069+0000","last_became_peered":"2023-07-18T03:29:27.108069+0000","last_unstale":"2023-07-18T03:30:33.749286+0000","last_undegraded":"2023-07-18T03:30:33.749286+0000","last_fullsized":"2023-07-18T03:30:33.749286+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:37:05.273266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":97,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:44.761547+0000","last_change":"2023-07-18T03:29:26.290896+0000","last_active":"2023-07-18T03:30:44.761547+0000","last_peered":"2023-07-18T03:30:44.761547+0000","last_clean":"2023-07-18T03:30:44.761547+0000","last_became_active":"2023-07-18T03:29:26.290450+0000","last_became_peered":"2023-07-18T03:29:26.290450+0000","last_unstale":"2023-07-18T03:30:44.761547+0000","last_undegraded":"2023-07-18T03:30:44.761547+0000","last_fullsized":"2023-07-18T03:30:44.761547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:30:12.255317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.748869+0000","last_change":"2023-07-18T03:29:27.109184+0000","last_active":"2023-07-18T03:30:33.748869+0000","last_peered":"2023-07-18T03:30:33.748869+0000","last_clean":"2023-07-18T03:30:33.748869+0000","last_became_active":"2023-07-18T03:29:27.107993+0000","last_became_peered":"2023-07-18T03:29:27.107993+0000","last_unstale":"2023-07-18T03:30:33.748869+0000","last_undegraded":"2023-07-18T03:30:33.748869+0000","last_fullsized":"2023-07-18T03:30:33.748869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T11:09:48.362579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656895+0000","last_change":"2023-07-18T03:29:26.293671+0000","last_active":"2023-07-18T03:30:33.656895+0000","last_peered":"2023-07-18T03:30:33.656895+0000","last_clean":"2023-07-18T03:30:33.656895+0000","last_became_active":"2023-07-18T03:29:26.293205+0000","last_became_peered":"2023-07-18T03:29:26.293205+0000","last_unstale":"2023-07-18T03:30:33.656895+0000","last_undegraded":"2023-07-18T03:30:33.656895+0000","last_fullsized":"2023-07-18T03:30:33.656895+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:23:48.250370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.663227+0000","last_change":"2023-07-18T03:30:33.659828+0000","last_active":"2023-07-18T03:30:33.663227+0000","last_peered":"2023-07-18T03:30:33.663227+0000","last_clean":"2023-07-18T03:30:33.663227+0000","last_became_active":"2023-07-18T03:30:33.659440+0000","last_became_peered":"2023-07-18T03:30:33.659440+0000","last_unstale":"2023-07-18T03:30:33.663227+0000","last_undegraded":"2023-07-18T03:30:33.663227+0000","last_fullsized":"2023-07-18T03:30:33.663227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:21:00.901622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.667448+0000","last_change":"2023-07-18T03:30:33.666902+0000","last_active":"2023-07-18T03:30:33.667448+0000","last_peered":"2023-07-18T03:30:33.667448+0000","last_clean":"2023-07-18T03:30:33.667448+0000","last_became_active":"2023-07-18T03:30:33.661326+0000","last_became_peered":"2023-07-18T03:30:33.661326+0000","last_unstale":"2023-07-18T03:30:33.667448+0000","last_undegraded":"2023-07-18T03:30:33.667448+0000","last_fullsized":"2023-07-18T03:30:33.667448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:12:15.691642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656030+0000","last_change":"2023-07-18T03:29:27.098479+0000","last_active":"2023-07-18T03:30:33.656030+0000","last_peered":"2023-07-18T03:30:33.656030+0000","last_clean":"2023-07-18T03:30:33.656030+0000","last_became_active":"2023-07-18T03:29:27.098153+0000","last_became_peered":"2023-07-18T03:29:27.098153+0000","last_unstale":"2023-07-18T03:30:33.656030+0000","last_undegraded":"2023-07-18T03:30:33.656030+0000","last_fullsized":"2023-07-18T03:30:33.656030+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:53:21.610571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.652236+0000","last_change":"2023-07-18T03:29:26.290953+0000","last_active":"2023-07-18T03:30:33.652236+0000","last_peered":"2023-07-18T03:30:33.652236+0000","last_clean":"2023-07-18T03:30:33.652236+0000","last_became_active":"2023-07-18T03:29:26.290633+0000","last_became_peered":"2023-07-18T03:29:26.290633+0000","last_unstale":"2023-07-18T03:30:33.652236+0000","last_undegraded":"2023-07-18T03:30:33.652236+0000","last_fullsized":"2023-07-18T03:30:33.652236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:43:09.103910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661197+0000","last_change":"2023-07-18T03:29:27.093651+0000","last_active":"2023-07-18T03:30:33.661197+0000","last_peered":"2023-07-18T03:30:33.661197+0000","last_clean":"2023-07-18T03:30:33.661197+0000","last_became_active":"2023-07-18T03:29:27.093307+0000","last_became_peered":"2023-07-18T03:29:27.093307+0000","last_unstale":"2023-07-18T03:30:33.661197+0000","last_undegraded":"2023-07-18T03:30:33.661197+0000","last_fullsized":"2023-07-18T03:30:33.661197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:10:55.332949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":97,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:45.763511+0000","last_change":"2023-07-18T03:29:26.287405+0000","last_active":"2023-07-18T03:30:45.763511+0000","last_peered":"2023-07-18T03:30:45.763511+0000","last_clean":"2023-07-18T03:30:45.763511+0000","last_became_active":"2023-07-18T03:29:26.287048+0000","last_became_peered":"2023-07-18T03:29:26.287048+0000","last_unstale":"2023-07-18T03:30:45.763511+0000","last_undegraded":"2023-07-18T03:30:45.763511+0000","last_fullsized":"2023-07-18T03:30:45.763511+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:46:08.045548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661021+0000","last_change":"2023-07-18T03:29:27.103218+0000","last_active":"2023-07-18T03:30:33.661021+0000","last_peered":"2023-07-18T03:30:33.661021+0000","last_clean":"2023-07-18T03:30:33.661021+0000","last_became_active":"2023-07-18T03:29:27.102917+0000","last_became_peered":"2023-07-18T03:29:27.102917+0000","last_unstale":"2023-07-18T03:30:33.661021+0000","last_undegraded":"2023-07-18T03:30:33.661021+0000","last_fullsized":"2023-07-18T03:30:33.661021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:15:09.570588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.659924+0000","last_change":"2023-07-18T03:29:27.098696+0000","last_active":"2023-07-18T03:30:33.659924+0000","last_peered":"2023-07-18T03:30:33.659924+0000","last_clean":"2023-07-18T03:30:33.659924+0000","last_became_active":"2023-07-18T03:29:27.098253+0000","last_became_peered":"2023-07-18T03:29:27.098253+0000","last_unstale":"2023-07-18T03:30:33.659924+0000","last_undegraded":"2023-07-18T03:30:33.659924+0000","last_fullsized":"2023-07-18T03:30:33.659924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:31:51.020473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":96,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:44.760939+0000","last_change":"2023-07-18T03:29:26.290308+0000","last_active":"2023-07-18T03:30:44.760939+0000","last_peered":"2023-07-18T03:30:44.760939+0000","last_clean":"2023-07-18T03:30:44.760939+0000","last_became_active":"2023-07-18T03:29:26.289937+0000","last_became_peered":"2023-07-18T03:29:26.289937+0000","last_unstale":"2023-07-18T03:30:44.760939+0000","last_undegraded":"2023-07-18T03:30:44.760939+0000","last_fullsized":"2023-07-18T03:30:44.760939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:48:40.209353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661087+0000","last_change":"2023-07-18T03:29:27.096469+0000","last_active":"2023-07-18T03:30:33.661087+0000","last_peered":"2023-07-18T03:30:33.661087+0000","last_clean":"2023-07-18T03:30:33.661087+0000","last_became_active":"2023-07-18T03:29:27.096171+0000","last_became_peered":"2023-07-18T03:29:27.096171+0000","last_unstale":"2023-07-18T03:30:33.661087+0000","last_undegraded":"2023-07-18T03:30:33.661087+0000","last_fullsized":"2023-07-18T03:30:33.661087+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:12:58.040917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.658966+0000","last_change":"2023-07-18T03:29:26.290935+0000","last_active":"2023-07-18T03:30:33.658966+0000","last_peered":"2023-07-18T03:30:33.658966+0000","last_clean":"2023-07-18T03:30:33.658966+0000","last_became_active":"2023-07-18T03:29:26.290595+0000","last_became_peered":"2023-07-18T03:29:26.290595+0000","last_unstale":"2023-07-18T03:30:33.658966+0000","last_undegraded":"2023-07-18T03:30:33.658966+0000","last_fullsized":"2023-07-18T03:30:33.658966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:48:44.101957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":335,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.748928+0000","last_change":"2023-07-18T03:28:28.122385+0000","last_active":"2023-07-18T03:30:33.748928+0000","last_peered":"2023-07-18T03:30:33.748928+0000","last_clean":"2023-07-18T03:30:33.748928+0000","last_became_active":"2023-07-18T03:28:28.121932+0000","last_became_peered":"2023-07-18T03:28:28.121932+0000","last_unstale":"2023-07-18T03:30:33.748928+0000","last_undegraded":"2023-07-18T03:30:33.748928+0000","last_fullsized":"2023-07-18T03:30:33.748928+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:25:38.267627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:25:38.267627+0000","last_clean_scrub_stamp":"2023-07-18T03:25:38.267627+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:04:59.247851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.655491+0000","last_change":"2023-07-18T03:29:27.101582+0000","last_active":"2023-07-18T03:30:33.655491+0000","last_peered":"2023-07-18T03:30:33.655491+0000","last_clean":"2023-07-18T03:30:33.655491+0000","last_became_active":"2023-07-18T03:29:27.101226+0000","last_became_peered":"2023-07-18T03:29:27.101226+0000","last_unstale":"2023-07-18T03:30:33.655491+0000","last_undegraded":"2023-07-18T03:30:33.655491+0000","last_fullsized":"2023-07-18T03:30:33.655491+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:24:53.297066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749186+0000","last_change":"2023-07-18T03:29:26.291787+0000","last_active":"2023-07-18T03:30:33.749186+0000","last_peered":"2023-07-18T03:30:33.749186+0000","last_clean":"2023-07-18T03:30:33.749186+0000","last_became_active":"2023-07-18T03:29:26.291471+0000","last_became_peered":"2023-07-18T03:29:26.291471+0000","last_unstale":"2023-07-18T03:30:33.749186+0000","last_undegraded":"2023-07-18T03:30:33.749186+0000","last_fullsized":"2023-07-18T03:30:33.749186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:03:33.650313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.657936+0000","last_change":"2023-07-18T03:29:26.296936+0000","last_active":"2023-07-18T03:30:33.657936+0000","last_peered":"2023-07-18T03:30:33.657936+0000","last_clean":"2023-07-18T03:30:33.657936+0000","last_became_active":"2023-07-18T03:29:26.296241+0000","last_became_peered":"2023-07-18T03:29:26.296241+0000","last_unstale":"2023-07-18T03:30:33.657936+0000","last_undegraded":"2023-07-18T03:30:33.657936+0000","last_fullsized":"2023-07-18T03:30:33.657936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:51:59.499717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656549+0000","last_change":"2023-07-18T03:29:27.102715+0000","last_active":"2023-07-18T03:30:33.656549+0000","last_peered":"2023-07-18T03:30:33.656549+0000","last_clean":"2023-07-18T03:30:33.656549+0000","last_became_active":"2023-07-18T03:29:27.100914+0000","last_became_peered":"2023-07-18T03:29:27.100914+0000","last_unstale":"2023-07-18T03:30:33.656549+0000","last_undegraded":"2023-07-18T03:30:33.656549+0000","last_fullsized":"2023-07-18T03:30:33.656549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:09:20.580139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'9","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.652605+0000","last_change":"2023-07-18T03:29:26.290231+0000","last_active":"2023-07-18T03:30:33.652605+0000","last_peered":"2023-07-18T03:30:33.652605+0000","last_clean":"2023-07-18T03:30:33.652605+0000","last_became_active":"2023-07-18T03:29:26.289753+0000","last_became_peered":"2023-07-18T03:29:26.289753+0000","last_unstale":"2023-07-18T03:30:33.652605+0000","last_undegraded":"2023-07-18T03:30:33.652605+0000","last_fullsized":"2023-07-18T03:30:33.652605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:37:14.149936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661595+0000","last_change":"2023-07-18T03:29:27.103964+0000","last_active":"2023-07-18T03:30:33.661595+0000","last_peered":"2023-07-18T03:30:33.661595+0000","last_clean":"2023-07-18T03:30:33.661595+0000","last_became_active":"2023-07-18T03:29:27.103660+0000","last_became_peered":"2023-07-18T03:29:27.103660+0000","last_unstale":"2023-07-18T03:30:33.661595+0000","last_undegraded":"2023-07-18T03:30:33.661595+0000","last_fullsized":"2023-07-18T03:30:33.661595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:24:11.718523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.659213+0000","last_change":"2023-07-18T03:29:26.286436+0000","last_active":"2023-07-18T03:30:33.659213+0000","last_peered":"2023-07-18T03:30:33.659213+0000","last_clean":"2023-07-18T03:30:33.659213+0000","last_became_active":"2023-07-18T03:29:26.286070+0000","last_became_peered":"2023-07-18T03:29:26.286070+0000","last_unstale":"2023-07-18T03:30:33.659213+0000","last_undegraded":"2023-07-18T03:30:33.659213+0000","last_fullsized":"2023-07-18T03:30:33.659213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:49:30.399461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.750552+0000","last_change":"2023-07-18T03:30:33.750552+0000","last_active":"2023-07-18T03:30:33.750552+0000","last_peered":"2023-07-18T03:30:33.750552+0000","last_clean":"2023-07-18T03:30:33.750552+0000","last_became_active":"2023-07-18T03:30:33.750210+0000","last_became_peered":"2023-07-18T03:30:33.750210+0000","last_unstale":"2023-07-18T03:30:33.750552+0000","last_undegraded":"2023-07-18T03:30:33.750552+0000","last_fullsized":"2023-07-18T03:30:33.750552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:30:46.022642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650623+0000","last_change":"2023-07-18T03:29:26.289628+0000","last_active":"2023-07-18T03:30:33.650623+0000","last_peered":"2023-07-18T03:30:33.650623+0000","last_clean":"2023-07-18T03:30:33.650623+0000","last_became_active":"2023-07-18T03:29:26.289013+0000","last_became_peered":"2023-07-18T03:29:26.289013+0000","last_unstale":"2023-07-18T03:30:33.650623+0000","last_undegraded":"2023-07-18T03:30:33.650623+0000","last_fullsized":"2023-07-18T03:30:33.650623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:10:03.968656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656945+0000","last_change":"2023-07-18T03:29:27.090943+0000","last_active":"2023-07-18T03:30:33.656945+0000","last_peered":"2023-07-18T03:30:33.656945+0000","last_clean":"2023-07-18T03:30:33.656945+0000","last_became_active":"2023-07-18T03:29:27.090465+0000","last_became_peered":"2023-07-18T03:29:27.090465+0000","last_unstale":"2023-07-18T03:30:33.656945+0000","last_undegraded":"2023-07-18T03:30:33.656945+0000","last_fullsized":"2023-07-18T03:30:33.656945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:26:40.736484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660770+0000","last_change":"2023-07-18T03:29:26.288576+0000","last_active":"2023-07-18T03:30:33.660770+0000","last_peered":"2023-07-18T03:30:33.660770+0000","last_clean":"2023-07-18T03:30:33.660770+0000","last_became_active":"2023-07-18T03:29:26.287514+0000","last_became_peered":"2023-07-18T03:29:26.287514+0000","last_unstale":"2023-07-18T03:30:33.660770+0000","last_undegraded":"2023-07-18T03:30:33.660770+0000","last_fullsized":"2023-07-18T03:30:33.660770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:35:34.080524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656181+0000","last_change":"2023-07-18T03:29:27.101111+0000","last_active":"2023-07-18T03:30:33.656181+0000","last_peered":"2023-07-18T03:30:33.656181+0000","last_clean":"2023-07-18T03:30:33.656181+0000","last_became_active":"2023-07-18T03:29:27.100737+0000","last_became_peered":"2023-07-18T03:29:27.100737+0000","last_unstale":"2023-07-18T03:30:33.656181+0000","last_undegraded":"2023-07-18T03:30:33.656181+0000","last_fullsized":"2023-07-18T03:30:33.656181+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:17:22.437647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.654205+0000","last_change":"2023-07-18T03:29:26.296974+0000","last_active":"2023-07-18T03:30:33.654205+0000","last_peered":"2023-07-18T03:30:33.654205+0000","last_clean":"2023-07-18T03:30:33.654205+0000","last_became_active":"2023-07-18T03:29:26.296451+0000","last_became_peered":"2023-07-18T03:29:26.296451+0000","last_unstale":"2023-07-18T03:30:33.654205+0000","last_undegraded":"2023-07-18T03:30:33.654205+0000","last_fullsized":"2023-07-18T03:30:33.654205+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T03:48:52.975744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.661117+0000","last_change":"2023-07-18T03:29:27.095857+0000","last_active":"2023-07-18T03:30:33.661117+0000","last_peered":"2023-07-18T03:30:33.661117+0000","last_clean":"2023-07-18T03:30:33.661117+0000","last_became_active":"2023-07-18T03:29:27.095553+0000","last_became_peered":"2023-07-18T03:29:27.095553+0000","last_unstale":"2023-07-18T03:30:33.661117+0000","last_undegraded":"2023-07-18T03:30:33.661117+0000","last_fullsized":"2023-07-18T03:30:33.661117+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:22:26.560346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.657524+0000","last_change":"2023-07-18T03:29:26.295936+0000","last_active":"2023-07-18T03:30:33.657524+0000","last_peered":"2023-07-18T03:30:33.657524+0000","last_clean":"2023-07-18T03:30:33.657524+0000","last_became_active":"2023-07-18T03:29:26.295467+0000","last_became_peered":"2023-07-18T03:29:26.295467+0000","last_unstale":"2023-07-18T03:30:33.657524+0000","last_undegraded":"2023-07-18T03:30:33.657524+0000","last_fullsized":"2023-07-18T03:30:33.657524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:23:28.043840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749424+0000","last_change":"2023-07-18T03:29:27.108528+0000","last_active":"2023-07-18T03:30:33.749424+0000","last_peered":"2023-07-18T03:30:33.749424+0000","last_clean":"2023-07-18T03:30:33.749424+0000","last_became_active":"2023-07-18T03:29:27.108039+0000","last_became_peered":"2023-07-18T03:29:27.108039+0000","last_unstale":"2023-07-18T03:30:33.749424+0000","last_undegraded":"2023-07-18T03:30:33.749424+0000","last_fullsized":"2023-07-18T03:30:33.749424+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:13:41.092469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'5","reported_seq":94,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:41.764673+0000","last_change":"2023-07-18T03:29:26.294006+0000","last_active":"2023-07-18T03:30:41.764673+0000","last_peered":"2023-07-18T03:30:41.764673+0000","last_clean":"2023-07-18T03:30:41.764673+0000","last_became_active":"2023-07-18T03:29:26.293700+0000","last_became_peered":"2023-07-18T03:29:26.293700+0000","last_unstale":"2023-07-18T03:30:41.764673+0000","last_undegraded":"2023-07-18T03:30:41.764673+0000","last_fullsized":"2023-07-18T03:30:41.764673+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T09:41:32.418658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749567+0000","last_change":"2023-07-18T03:29:27.102764+0000","last_active":"2023-07-18T03:30:33.749567+0000","last_peered":"2023-07-18T03:30:33.749567+0000","last_clean":"2023-07-18T03:30:33.749567+0000","last_became_active":"2023-07-18T03:29:27.102200+0000","last_became_peered":"2023-07-18T03:29:27.102200+0000","last_unstale":"2023-07-18T03:30:33.749567+0000","last_undegraded":"2023-07-18T03:30:33.749567+0000","last_fullsized":"2023-07-18T03:30:33.749567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T14:49:22.828462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660299+0000","last_change":"2023-07-18T03:29:26.294271+0000","last_active":"2023-07-18T03:30:33.660299+0000","last_peered":"2023-07-18T03:30:33.660299+0000","last_clean":"2023-07-18T03:30:33.660299+0000","last_became_active":"2023-07-18T03:29:26.293938+0000","last_became_peered":"2023-07-18T03:29:26.293938+0000","last_unstale":"2023-07-18T03:30:33.660299+0000","last_undegraded":"2023-07-18T03:30:33.660299+0000","last_fullsized":"2023-07-18T03:30:33.660299+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T12:59:22.557498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749202+0000","last_change":"2023-07-18T03:29:27.106510+0000","last_active":"2023-07-18T03:30:33.749202+0000","last_peered":"2023-07-18T03:30:33.749202+0000","last_clean":"2023-07-18T03:30:33.749202+0000","last_became_active":"2023-07-18T03:29:27.106222+0000","last_became_peered":"2023-07-18T03:29:27.106222+0000","last_unstale":"2023-07-18T03:30:33.749202+0000","last_undegraded":"2023-07-18T03:30:33.749202+0000","last_fullsized":"2023-07-18T03:30:33.749202+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T07:08:13.558332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656074+0000","last_change":"2023-07-18T03:29:26.293583+0000","last_active":"2023-07-18T03:30:33.656074+0000","last_peered":"2023-07-18T03:30:33.656074+0000","last_clean":"2023-07-18T03:30:33.656074+0000","last_became_active":"2023-07-18T03:29:26.293076+0000","last_became_peered":"2023-07-18T03:29:26.293076+0000","last_unstale":"2023-07-18T03:30:33.656074+0000","last_undegraded":"2023-07-18T03:30:33.656074+0000","last_fullsized":"2023-07-18T03:30:33.656074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:25.264073+0000","last_clean_scrub_stamp":"2023-07-18T03:29:25.264073+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:58:56.281865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.749636+0000","last_change":"2023-07-18T03:29:27.109175+0000","last_active":"2023-07-18T03:30:33.749636+0000","last_peered":"2023-07-18T03:30:33.749636+0000","last_clean":"2023-07-18T03:30:33.749636+0000","last_became_active":"2023-07-18T03:29:27.108669+0000","last_became_peered":"2023-07-18T03:29:27.108669+0000","last_unstale":"2023-07-18T03:30:33.749636+0000","last_undegraded":"2023-07-18T03:30:33.749636+0000","last_fullsized":"2023-07-18T03:30:33.749636+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:34:18.667617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656335+0000","last_change":"2023-07-18T03:29:27.099770+0000","last_active":"2023-07-18T03:30:33.656335+0000","last_peered":"2023-07-18T03:30:33.656335+0000","last_clean":"2023-07-18T03:30:33.656335+0000","last_became_active":"2023-07-18T03:29:27.099487+0000","last_became_peered":"2023-07-18T03:29:27.099487+0000","last_unstale":"2023-07-18T03:30:33.656335+0000","last_undegraded":"2023-07-18T03:30:33.656335+0000","last_fullsized":"2023-07-18T03:30:33.656335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T15:13:59.713331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.664744+0000","last_change":"2023-07-18T03:30:33.664744+0000","last_active":"2023-07-18T03:30:33.664744+0000","last_peered":"2023-07-18T03:30:33.664744+0000","last_clean":"2023-07-18T03:30:33.664744+0000","last_became_active":"2023-07-18T03:30:33.664378+0000","last_became_peered":"2023-07-18T03:30:33.664378+0000","last_unstale":"2023-07-18T03:30:33.664744+0000","last_undegraded":"2023-07-18T03:30:33.664744+0000","last_fullsized":"2023-07-18T03:30:33.664744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:46:59.976803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.653582+0000","last_change":"2023-07-18T03:29:27.099196+0000","last_active":"2023-07-18T03:30:33.653582+0000","last_peered":"2023-07-18T03:30:33.653582+0000","last_clean":"2023-07-18T03:30:33.653582+0000","last_became_active":"2023-07-18T03:29:27.098903+0000","last_became_peered":"2023-07-18T03:29:27.098903+0000","last_unstale":"2023-07-18T03:30:33.653582+0000","last_undegraded":"2023-07-18T03:30:33.653582+0000","last_fullsized":"2023-07-18T03:30:33.653582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:43:55.170768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660224+0000","last_change":"2023-07-18T03:30:33.660224+0000","last_active":"2023-07-18T03:30:33.660224+0000","last_peered":"2023-07-18T03:30:33.660224+0000","last_clean":"2023-07-18T03:30:33.660224+0000","last_became_active":"2023-07-18T03:30:33.659847+0000","last_became_peered":"2023-07-18T03:30:33.659847+0000","last_unstale":"2023-07-18T03:30:33.660224+0000","last_undegraded":"2023-07-18T03:30:33.660224+0000","last_fullsized":"2023-07-18T03:30:33.660224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:08:47.876757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656416+0000","last_change":"2023-07-18T03:29:27.101610+0000","last_active":"2023-07-18T03:30:33.656416+0000","last_peered":"2023-07-18T03:30:33.656416+0000","last_clean":"2023-07-18T03:30:33.656416+0000","last_became_active":"2023-07-18T03:29:27.101332+0000","last_became_peered":"2023-07-18T03:29:27.101332+0000","last_unstale":"2023-07-18T03:30:33.656416+0000","last_undegraded":"2023-07-18T03:30:33.656416+0000","last_fullsized":"2023-07-18T03:30:33.656416+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T06:15:53.329517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656022+0000","last_change":"2023-07-18T03:29:27.093345+0000","last_active":"2023-07-18T03:30:33.656022+0000","last_peered":"2023-07-18T03:30:33.656022+0000","last_clean":"2023-07-18T03:30:33.656022+0000","last_became_active":"2023-07-18T03:29:27.093048+0000","last_became_peered":"2023-07-18T03:29:27.093048+0000","last_unstale":"2023-07-18T03:30:33.656022+0000","last_undegraded":"2023-07-18T03:30:33.656022+0000","last_fullsized":"2023-07-18T03:30:33.656022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T05:43:10.411014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.654001+0000","last_change":"2023-07-18T03:29:27.098311+0000","last_active":"2023-07-18T03:30:33.654001+0000","last_peered":"2023-07-18T03:30:33.654001+0000","last_clean":"2023-07-18T03:30:33.654001+0000","last_became_active":"2023-07-18T03:29:27.097980+0000","last_became_peered":"2023-07-18T03:29:27.097980+0000","last_unstale":"2023-07-18T03:30:33.654001+0000","last_undegraded":"2023-07-18T03:30:33.654001+0000","last_fullsized":"2023-07-18T03:30:33.654001+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:18:24.177425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.663242+0000","last_change":"2023-07-18T03:30:33.660269+0000","last_active":"2023-07-18T03:30:33.663242+0000","last_peered":"2023-07-18T03:30:33.663242+0000","last_clean":"2023-07-18T03:30:33.663242+0000","last_became_active":"2023-07-18T03:30:33.659999+0000","last_became_peered":"2023-07-18T03:30:33.659999+0000","last_unstale":"2023-07-18T03:30:33.663242+0000","last_undegraded":"2023-07-18T03:30:33.663242+0000","last_fullsized":"2023-07-18T03:30:33.663242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:26:07.126950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660153+0000","last_change":"2023-07-18T03:29:27.098541+0000","last_active":"2023-07-18T03:30:33.660153+0000","last_peered":"2023-07-18T03:30:33.660153+0000","last_clean":"2023-07-18T03:30:33.660153+0000","last_became_active":"2023-07-18T03:29:27.097824+0000","last_became_peered":"2023-07-18T03:29:27.097824+0000","last_unstale":"2023-07-18T03:30:33.660153+0000","last_undegraded":"2023-07-18T03:30:33.660153+0000","last_fullsized":"2023-07-18T03:30:33.660153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T10:57:42.609243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.660576+0000","last_change":"2023-07-18T03:29:27.096983+0000","last_active":"2023-07-18T03:30:33.660576+0000","last_peered":"2023-07-18T03:30:33.660576+0000","last_clean":"2023-07-18T03:30:33.660576+0000","last_became_active":"2023-07-18T03:29:27.096632+0000","last_became_peered":"2023-07-18T03:29:27.096632+0000","last_unstale":"2023-07-18T03:30:33.660576+0000","last_undegraded":"2023-07-18T03:30:33.660576+0000","last_fullsized":"2023-07-18T03:30:33.660576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:28:14.181259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.650674+0000","last_change":"2023-07-18T03:29:27.105236+0000","last_active":"2023-07-18T03:30:33.650674+0000","last_peered":"2023-07-18T03:30:33.650674+0000","last_clean":"2023-07-18T03:30:33.650674+0000","last_became_active":"2023-07-18T03:29:27.104906+0000","last_became_peered":"2023-07-18T03:29:27.104906+0000","last_unstale":"2023-07-18T03:30:33.650674+0000","last_undegraded":"2023-07-18T03:30:33.650674+0000","last_fullsized":"2023-07-18T03:30:33.650674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T04:14:25.383609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656158+0000","last_change":"2023-07-18T03:29:27.103461+0000","last_active":"2023-07-18T03:30:33.656158+0000","last_peered":"2023-07-18T03:30:33.656158+0000","last_clean":"2023-07-18T03:30:33.656158+0000","last_became_active":"2023-07-18T03:29:27.103162+0000","last_became_peered":"2023-07-18T03:29:27.103162+0000","last_unstale":"2023-07-18T03:30:33.656158+0000","last_undegraded":"2023-07-18T03:30:33.656158+0000","last_fullsized":"2023-07-18T03:30:33.656158+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T08:36:23.614131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-18T03:30:33.656048+0000","last_change":"2023-07-18T03:29:27.095300+0000","last_active":"2023-07-18T03:30:33.656048+0000","last_peered":"2023-07-18T03:30:33.656048+0000","last_clean":"2023-07-18T03:30:33.656048+0000","last_became_active":"2023-07-18T03:29:27.094985+0000","last_became_peered":"2023-07-18T03:29:27.094985+0000","last_unstale":"2023-07-18T03:30:33.656048+0000","last_undegraded":"2023-07-18T03:30:33.656048+0000","last_fullsized":"2023-07-18T03:30:33.656048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-18T03:29:26.266282+0000","last_clean_scrub_stamp":"2023-07-18T03:29:26.266282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-19T13:37:22.084325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65783,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":766,"num_read_kb":455,"num_write":233,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":201033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514524,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":436016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":0.52900000000000003}]},{"osd":1,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.60999999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.66500000000000004}]},{"osd":5,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.70999999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.54400000000000004}]},{"osd":7,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.69599999999999995}]},{"osd":8,"last update":"Tue Jul 18 03:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.755},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.73699999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.872}]},{"osd":10,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.77400000000000002}]}]},{"osd":10,"up_from":69,"seq":296352743456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1613824,"data_stored":1085806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53800000000000003,"15min":0.54600000000000004},"min":{"1min":0.35799999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70899999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.35899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78000000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41399999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66300000000000003,"15min":0.66800000000000004},"min":{"1min":0.42899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91500000000000004,"5min":1.302,"15min":1.302},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.71099999999999997},"min":{"1min":0.54600000000000004,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93999999999999995,"5min":1,"15min":1},"last":0.81499999999999995}]},{"osd":3,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.68000000000000005,"15min":0.68899999999999995},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.876,"5min":1.276,"15min":1.276},"last":0.495},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.72899999999999998,"15min":0.73399999999999999},"min":{"1min":0.436,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":0.48399999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.65700000000000003,"15min":0.66100000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.89300000000000002,"5min":1.119,"15min":1.119},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61299999999999999,"15min":0.61699999999999999},"min":{"1min":0.33700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.85399999999999998,"5min":1.278,"15min":1.278},"last":0.46200000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.746,"15min":0.752},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.004,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.80400000000000005,"15min":0.81000000000000005},"min":{"1min":0.52400000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0229999999999999,"5min":1.222,"15min":1.222},"last":0.75}]},{"osd":6,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.82799999999999996,"15min":0.83599999999999997},"min":{"1min":0.503,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.82699999999999996,"15min":0.83199999999999996},"min":{"1min":0.53600000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.54300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":8,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.86399999999999999,"15min":0.871},"min":{"1min":0.57099999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.107,"5min":1.466,"15min":1.466},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.88300000000000001,"15min":0.89600000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0509999999999999,"5min":1.518,"15min":1.518},"last":0.77600000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.87,"15min":0.875},"min":{"1min":0.59599999999999997,"5min":0.503,"15min":0.503},"max":{"1min":1.169,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.88500000000000001,"15min":0.89900000000000002},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.153,"5min":1.585,"15min":1.585},"last":0.626}]},{"osd":11,"last update":"Tue Jul 18 03:29:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.71499999999999997}]}]},{"osd":9,"up_from":64,"seq":274877906980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":435904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.47099999999999997,"15min":0.48199999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.58999999999999997,"5min":0.747,"15min":0.747},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.47099999999999997,"15min":0.48699999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.57199999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.318}]},{"osd":1,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.69599999999999995,"15min":0.71199999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86799999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.61899999999999999,"15min":0.63500000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.746,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46500000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.67300000000000004,"15min":0.69099999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.71899999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.69899999999999995,"15min":0.71399999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81899999999999995,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.66200000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.50600000000000001,"15min":0.52200000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.81699999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.60099999999999998,"15min":0.622},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.82499999999999996,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.34899999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.76600000000000001,"15min":0.79300000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.80800000000000005,"5min":1.355,"15min":1.355},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.69299999999999995,"15min":0.71099999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.77100000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.372}]},{"osd":5,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.79100000000000004,"15min":0.81100000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.90000000000000002,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.622},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.77800000000000002,"15min":0.79800000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.871,"5min":1.302,"15min":1.302},"last":0.68100000000000005}]},{"osd":6,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.745,"15min":0.77700000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.75600000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.78600000000000003,"15min":0.82199999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85399999999999998,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.69399999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.78800000000000003,"15min":0.80600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89500000000000002,"5min":1.137,"15min":1.137},"last":0.435},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.81999999999999995,"15min":0.84499999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.91000000000000003,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.56499999999999995}]},{"osd":8,"last update":"Tue Jul 18 03:30:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.86799999999999999,"15min":0.89400000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92800000000000005,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.77700000000000002,"15min":0.79200000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.85599999999999998,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.51900000000000002}]},{"osd":10,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.80000000000000004,"15min":0.81899999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94599999999999995,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.84399999999999997,"15min":0.86499999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95799999999999996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.76100000000000001}]}]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":435929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:30:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.57499999999999996,"15min":0.55000000000000004},"min":{"1min":0.438,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82799999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59299999999999997}]},{"osd":1,"last update":"Tue Jul 18 03:30:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.61899999999999999,"15min":0.59099999999999997},"min":{"1min":0.435,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86199999999999999,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.63600000000000001,"15min":0.61399999999999999},"min":{"1min":0.53200000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.875,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.73099999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:30:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.45900000000000002,"15min":0.439},"min":{"1min":0.376,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.73099999999999998,"5min":0.755,"15min":0.755},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.48799999999999999,"15min":0.45700000000000002},"min":{"1min":0.35399999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.66000000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:30:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85599999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.64800000000000002,"15min":0.59599999999999997},"min":{"1min":0.47299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":6.1420000000000003,"5min":6.1420000000000003,"15min":6.1420000000000003},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Jul 18 03:30:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.65200000000000002,"15min":0.626},"min":{"1min":0.55200000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.66800000000000004,"15min":0.63100000000000001},"min":{"1min":0.56200000000000006,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.81499999999999995}]},{"osd":5,"last update":"Tue Jul 18 03:30:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.59199999999999997,"15min":0.56899999999999995},"min":{"1min":0.42999999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.57699999999999996,"15min":0.54900000000000004},"min":{"1min":0.53700000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91500000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69499999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:30:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70399999999999996,"15min":0.69199999999999995},"min":{"1min":0.45200000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.95599999999999996,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.755},{"interface":"front","average":{"1min":1.054,"5min":0.76900000000000002,"15min":0.71799999999999997},"min":{"1min":0.52400000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":5.9740000000000002,"5min":5.9740000000000002,"15min":5.9740000000000002},"last":0.78600000000000003}]},{"osd":7,"last update":"Tue Jul 18 03:30:46 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.755,"15min":0.72399999999999998},"min":{"1min":0.60399999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.099,"5min":1.153,"15min":1.153},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.72699999999999998,"15min":0.69599999999999995},"min":{"1min":0.64300000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.048,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.89200000000000002}]},{"osd":9,"last update":"Tue Jul 18 03:30:05 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.752,"15min":0.72999999999999998},"min":{"1min":0.56499999999999995,"5min":0.505,"15min":0.505},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.78300000000000003,"15min":0.77300000000000002},"min":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.85599999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.81299999999999994,"15min":0.79800000000000004},"min":{"1min":0.69099999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.80900000000000005,"15min":0.79100000000000004},"min":{"1min":0.67200000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.145,"5min":1.208,"15min":1.208},"last":0.96599999999999997}]},{"osd":11,"last update":"Tue Jul 18 03:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.872}]}]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1531904,"data_stored":1028979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:28:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57799999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.74199999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.38400000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60799999999999998,"15min":0.59199999999999997},"min":{"1min":0.434,"5min":0.379,"15min":0.379},"max":{"1min":0.86599999999999999,"5min":0.872,"15min":0.872},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63400000000000001,"15min":0.61399999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83499999999999996,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56200000000000006}]},{"osd":3,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.45600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.878,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.501},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.65400000000000003,"15min":0.65500000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56899999999999995,"15min":0.55800000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.74399999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76500000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.36199999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67600000000000005,"15min":0.66200000000000003},"min":{"1min":0.499,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88600000000000001,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.45100000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.64900000000000002}]},{"osd":6,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.74099999999999999,"15min":0.73299999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.433},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.76600000000000001,"15min":0.73299999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":6.1909999999999998,"5min":6.1909999999999998,"15min":6.1909999999999998},"last":0.46100000000000002}]},{"osd":8,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.80400000000000005,"15min":0.77600000000000002},"min":{"1min":0.51300000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":4.5880000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.76900000000000002,"15min":0.73299999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":4.6050000000000004,"5min":4.6050000000000004,"15min":4.6050000000000004},"last":0.62}]},{"osd":9,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":0.86199999999999999,"15min":0.80700000000000005},"min":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":9.8439999999999994,"5min":9.8439999999999994,"15min":9.8439999999999994},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.81899999999999995,"15min":0.81499999999999995},"min":{"1min":0.629,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.69799999999999995}]},{"osd":10,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79500000000000004,"15min":0.79900000000000004},"min":{"1min":0.54100000000000004,"5min":0.432,"15min":0.432},"max":{"1min":1,"5min":1.171,"15min":1.171},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.85199999999999998,"15min":0.86199999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0249999999999999,"5min":1.085,"15min":1.085},"last":0.74099999999999999}]},{"osd":11,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.75900000000000001}]}]},{"osd":6,"up_from":44,"seq":188978561073,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":435579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.443,"15min":0.41399999999999998},"min":{"1min":0.27100000000000002,"5min":0.254,"15min":0.254},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.46100000000000002,"15min":0.45000000000000001},"min":{"1min":0.28999999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.2320000000000002,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":0.375}]},{"osd":1,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.63900000000000001,"15min":0.58499999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":5.0700000000000003,"5min":5.0700000000000003,"15min":5.0700000000000003},"last":0.313},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.41699999999999998}]},{"osd":2,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86699999999999999,"5min":1.008,"15min":1.008},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.68999999999999995,"15min":0.63800000000000001},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":5.0300000000000002,"5min":5.0300000000000002,"15min":5.0300000000000002},"last":0.65500000000000003}]},{"osd":3,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.55900000000000005,"15min":0.50900000000000001},"min":{"1min":0.32100000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":5.2089999999999996,"5min":5.2089999999999996,"15min":5.2089999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54500000000000004,"15min":0.53700000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"last":0.497}]},{"osd":4,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.623,"15min":0.61399999999999999},"min":{"1min":0.44,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86599999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.436},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.73199999999999998,"15min":0.68700000000000006},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":5.1070000000000002,"5min":5.1070000000000002,"15min":5.1070000000000002},"last":0.48099999999999998}]},{"osd":5,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67800000000000005,"15min":0.67000000000000004},"min":{"1min":0.45300000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94599999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.73299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.83599999999999997,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Tue Jul 18 03:30:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.71799999999999997,"15min":0.72299999999999998},"min":{"1min":0.50900000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.91400000000000003,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.80300000000000005,"15min":0.78300000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.9699999999999998,"5min":4.9699999999999998,"15min":4.9699999999999998},"last":0.57899999999999996}]},{"osd":8,"last update":"Tue Jul 18 03:29:51 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.86399999999999999,"15min":0.84499999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":5.1760000000000002,"5min":5.1760000000000002,"15min":5.1760000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.77400000000000002,"15min":0.78000000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.67400000000000004}]},{"osd":9,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.75700000000000001,"15min":0.76600000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.81999999999999995,"15min":0.79800000000000004},"min":{"1min":0.438,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":5.2409999999999997,"5min":5.2409999999999997,"15min":5.2409999999999997},"last":0.60699999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.80400000000000005,"15min":0.82099999999999995},"min":{"1min":0.63,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.91500000000000004,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.022,"5min":0.84799999999999998,"15min":0.81899999999999995},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":5.141,"5min":5.141,"15min":5.141},"last":0.63300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":499651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.55800000000000005,"15min":0.49399999999999999},"min":{"1min":0.375,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.55900000000000005,"15min":0.51400000000000001},"min":{"1min":0.35399999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.58799999999999997}]},{"osd":2,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58699999999999997,"15min":0.54800000000000004},"min":{"1min":0.50600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88400000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56699999999999995,"15min":0.52500000000000002},"min":{"1min":0.51200000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51600000000000001}]},{"osd":3,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.626,"15min":0.56599999999999995},"min":{"1min":0.41599999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":2.4710000000000001,"5min":2.4710000000000001,"15min":2.4710000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.65200000000000002,"15min":0.59199999999999997},"min":{"1min":0.436,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":0.78900000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.49299999999999999,"15min":0.42299999999999999},"min":{"1min":0.34699999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.48099999999999998,"15min":0.432},"min":{"1min":0.374,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79500000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.624}]},{"osd":5,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.63600000000000001,"15min":0.58899999999999997},"min":{"1min":0.53000000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.66400000000000003,"15min":0.622},"min":{"1min":0.56799999999999995,"5min":0.433,"15min":0.433},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.73299999999999998}]},{"osd":6,"last update":"Tue Jul 18 03:30:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.66700000000000004,"15min":0.629},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.71499999999999997,"15min":0.67900000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue Jul 18 03:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.67100000000000004,"15min":0.64000000000000001},"min":{"1min":0.53700000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.73099999999999998,"15min":0.68899999999999995},"min":{"1min":0.59199999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0369999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.88700000000000001}]},{"osd":8,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.81299999999999994,"15min":0.80600000000000005},"min":{"1min":0.51900000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.02},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.78900000000000003,"15min":0.77900000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.86299999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.92400000000000004,"15min":0.91600000000000004},"min":{"1min":0.61299999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.875,"15min":0.86599999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":0.93000000000000005}]},{"osd":10,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.85399999999999998,"15min":0.84799999999999998},"min":{"1min":0.62,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.141,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.56699999999999995,"5min":0.439,"15min":0.439},"max":{"1min":1.159,"5min":1.528,"15min":1.528},"last":1.0669999999999999}]},{"osd":11,"last update":"Tue Jul 18 03:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":1.0449999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":436912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.60699999999999998,"15min":0.53800000000000003},"min":{"1min":0.32700000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":7.0199999999999996,"5min":7.0199999999999996,"15min":7.0199999999999996},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.50700000000000001},"min":{"1min":0.35299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.5}]},{"osd":2,"last update":"Tue Jul 18 03:30:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.53600000000000003},"min":{"1min":0.39500000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.58699999999999997,"15min":0.53500000000000003},"min":{"1min":0.41799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.60399999999999998}]},{"osd":3,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.373},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":7.274,"5min":7.274,"15min":7.274},"last":0.42899999999999999}]},{"osd":4,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58099999999999996,"15min":0.55500000000000005},"min":{"1min":0.40600000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.379,"5min":0.378,"15min":0.378},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.52100000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.67500000000000004,"15min":0.63},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":7.1079999999999997,"5min":7.1079999999999997,"15min":7.1079999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.63200000000000001,"15min":0.623},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84699999999999998,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.57499999999999996}]},{"osd":6,"last update":"Tue Jul 18 03:30:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56799999999999995,"15min":0.55100000000000005},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.621,"15min":0.57099999999999995},"min":{"1min":0.34799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":7.2130000000000001,"5min":7.2130000000000001,"15min":7.2130000000000001},"last":0.59399999999999997}]},{"osd":7,"last update":"Tue Jul 18 03:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.47099999999999997}]},{"osd":8,"last update":"Tue Jul 18 03:29:45 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.81499999999999995,"15min":0.79200000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":7.1449999999999996,"5min":7.1449999999999996,"15min":7.1449999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.77000000000000002,"15min":0.79000000000000004},"min":{"1min":0.47999999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96799999999999997,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.64100000000000001}]},{"osd":9,"last update":"Tue Jul 18 03:30:06 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":0.85999999999999999,"15min":0.80800000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":12.891,"5min":12.891,"15min":12.891},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.83199999999999996,"15min":0.81100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":7.2439999999999998,"5min":7.2439999999999998,"15min":7.2439999999999998},"last":0.46400000000000002}]},{"osd":10,"last update":"Tue Jul 18 03:30:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.77300000000000002,"15min":0.78800000000000003},"min":{"1min":0.53600000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0680000000000001,"5min":1.177,"15min":1.177},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.83799999999999997,"15min":0.81599999999999995},"min":{"1min":0.5,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":7.0720000000000001,"5min":7.0720000000000001,"15min":7.0720000000000001},"last":0.65600000000000003}]},{"osd":11,"last update":"Tue Jul 18 03:29:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":7.1790000000000003,"5min":7.1790000000000003,"15min":7.1790000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.64800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":495507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.39800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.55500000000000005},"min":{"1min":0.36099999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82199999999999995,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.53300000000000003}]},{"osd":1,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.34899999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78400000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.70099999999999996}]},{"osd":3,"last update":"Tue Jul 18 03:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.61699999999999999,"15min":0.58599999999999997},"min":{"1min":0.48299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.503},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88600000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.65400000000000003}]},{"osd":4,"last update":"Tue Jul 18 03:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.73399999999999999}]},{"osd":5,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.41699999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.45100000000000001,"15min":0.42799999999999999},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.88200000000000001}]},{"osd":6,"last update":"Tue Jul 18 03:30:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.59299999999999997,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.61599999999999999,"15min":0.58399999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.89900000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.81499999999999995}]},{"osd":7,"last update":"Tue Jul 18 03:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.433,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.505,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.61699999999999999}]},{"osd":8,"last update":"Tue Jul 18 03:29:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.378,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.94299999999999995}]},{"osd":9,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.73299999999999998,"15min":0.69499999999999995},"min":{"1min":0.48599999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":5.8890000000000002,"5min":5.8890000000000002,"15min":5.8890000000000002},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.97399999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.70399999999999996,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.71499999999999997,"15min":0.69299999999999995},"min":{"1min":0.746,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":1.044}]},{"osd":11,"last update":"Tue Jul 18 03:30:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":1.0209999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":937984,"data_stored":435458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.377,"15min":0.34599999999999997},"min":{"1min":0.26600000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.88500000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.372},"min":{"1min":0.28999999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.30099999999999999}]},{"osd":1,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58999999999999997,"15min":0.53500000000000003},"min":{"1min":0.39500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.3660000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.51300000000000001},"min":{"1min":0.39100000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.40100000000000002}]},{"osd":2,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57799999999999996,"15min":0.55700000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.3080000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61899999999999999,"15min":0.56999999999999995},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3919999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.751}]},{"osd":4,"last update":"Tue Jul 18 03:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.58299999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.93600000000000005,"5min":1.754,"15min":1.754},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66200000000000003,"15min":0.60999999999999999},"min":{"1min":0.39900000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93300000000000005,"5min":1.744,"15min":1.744},"last":0.64200000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:29:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.50600000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.357,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.436,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3460000000000001,"5min":1.827,"15min":1.827},"last":0.71699999999999997}]},{"osd":6,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.95199999999999996,"5min":1.002,"15min":1.002},"last":0.435},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58799999999999997,"15min":0.58299999999999996},"min":{"1min":0.38700000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.47699999999999998}]},{"osd":7,"last update":"Tue Jul 18 03:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.66700000000000004,"15min":0.67400000000000004},"min":{"1min":0.41499999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82799999999999996,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.48399999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95999999999999996,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.83999999999999997}]},{"osd":8,"last update":"Tue Jul 18 03:28:42 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.49399999999999999}]},{"osd":9,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.82999999999999996,"15min":0.85199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92500000000000004,"5min":1.113,"15min":1.113},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.74399999999999999,"15min":0.76200000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.995,"5min":0.997,"15min":0.997},"last":0.52500000000000002}]},{"osd":10,"last update":"Tue Jul 18 03:30:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.79100000000000004,"15min":0.81000000000000005},"min":{"1min":0.54000000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.90300000000000002,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.78200000000000003,"15min":0.80500000000000005},"min":{"1min":0.496,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91800000000000004,"5min":1.786,"15min":1.786},"last":0.499}]},{"osd":11,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.74099999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953528,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":435434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53200000000000003,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.54700000000000004,"15min":0.50700000000000001},"min":{"1min":0.48099999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.89500000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.502}]},{"osd":1,"last update":"Tue Jul 18 03:30:23 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.42299999999999999,"15min":0.39400000000000002},"min":{"1min":0.28199999999999997,"5min":0.246,"15min":0.246},"max":{"1min":0.66000000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.436,"15min":0.39100000000000001},"min":{"1min":0.36599999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.67400000000000004,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.49299999999999999}]},{"osd":2,"last update":"Tue Jul 18 03:30:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.64300000000000002,"15min":0.58499999999999996},"min":{"1min":0.438,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60699999999999998,"15min":0.56799999999999995},"min":{"1min":0.46400000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79100000000000004,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56499999999999995}]},{"osd":3,"last update":"Tue Jul 18 03:30:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61399999999999999,"15min":0.55900000000000005},"min":{"1min":0.46500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.55200000000000005},"min":{"1min":0.40400000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.91200000000000003,"5min":1.038,"15min":1.038},"last":0.52200000000000002}]},{"osd":5,"last update":"Tue Jul 18 03:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63,"15min":0.61399999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89200000000000002,"5min":1.038,"15min":1.038},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.64300000000000002,"15min":0.61599999999999999},"min":{"1min":0.58599999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96399999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.69099999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.63700000000000001,"15min":0.60499999999999998},"min":{"1min":0.41599999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.97599999999999998,"5min":1.085,"15min":1.085},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.41699999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.629}]},{"osd":7,"last update":"Tue Jul 18 03:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.65400000000000003,"15min":0.63400000000000001},"min":{"1min":0.51300000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.99399999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.67000000000000004,"15min":0.64800000000000002},"min":{"1min":0.54200000000000004,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.999,"5min":1.121,"15min":1.121},"last":0.81299999999999994}]},{"osd":8,"last update":"Tue Jul 18 03:29:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71399999999999997,"15min":0.71499999999999997},"min":{"1min":0.53000000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91100000000000003,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.78200000000000003,"15min":0.77500000000000002},"min":{"1min":0.59999999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0920000000000001,"5min":1.208,"15min":1.208},"last":0.82899999999999996}]},{"osd":9,"last update":"Tue Jul 18 03:30:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.78400000000000003,"15min":0.77000000000000002},"min":{"1min":0.69299999999999995,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.879,"5min":0.80700000000000005,"15min":0.79500000000000004},"min":{"1min":0.60899999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1379999999999999,"5min":1.163,"15min":1.163},"last":0.84899999999999998}]},{"osd":10,"last update":"Tue Jul 18 03:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.84799999999999998,"15min":0.83899999999999997},"min":{"1min":0.63300000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.073,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.82099999999999995,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.101,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.876}]},{"osd":11,"last update":"Tue Jul 18 03:30:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.86099999999999999}]}]},{"osd":5,"up_from":37,"seq":158913790005,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1544192,"data_stored":1028753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.78900000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.624},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.31900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.47399999999999998}]},{"osd":1,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.54800000000000004,"15min":0.51700000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.55800000000000005,"15min":0.52900000000000003},"min":{"1min":0.41399999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.50900000000000001}]},{"osd":2,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.40799999999999997,"15min":0.38400000000000001},"min":{"1min":0.29099999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.43099999999999999,"15min":0.41499999999999998},"min":{"1min":0.29899999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71299999999999997,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.442}]},{"osd":3,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.55900000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75800000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.60599999999999998}]},{"osd":4,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.59699999999999998,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.59899999999999998,"15min":0.56599999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.68999999999999995}]},{"osd":6,"last update":"Tue Jul 18 03:30:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.623},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82599999999999996,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.63800000000000001}]},{"osd":7,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.496,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94199999999999995,"5min":1.633,"15min":1.633},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.49299999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89100000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.70399999999999996}]},{"osd":8,"last update":"Tue Jul 18 03:29:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85199999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94099999999999995,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.754}]},{"osd":9,"last update":"Tue Jul 18 03:30:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.76100000000000001,"15min":0.75900000000000001},"min":{"1min":0.52800000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.94799999999999995,"5min":1.024,"15min":1.024},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.754,"15min":0.75600000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93200000000000005,"5min":1.008,"15min":1.008},"last":0.74099999999999999}]},{"osd":10,"last update":"Tue Jul 18 03:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.80100000000000005,"15min":0.79500000000000004},"min":{"1min":0.69399999999999995,"5min":0.504,"15min":0.504},"max":{"1min":0.98599999999999999,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77700000000000002,"15min":0.77100000000000002},"min":{"1min":0.67600000000000005,"5min":0.499,"15min":0.499},"max":{"1min":0.97299999999999998,"5min":1.548,"15min":1.548},"last":0.77000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-18T03:30:48.605 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:30:48.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.768+0000 7ff3306f1700 1 -- 172.21.15.40:0/3555484737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff328102e90 msgr2=0x7ff328105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:48.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.768+0000 7ff3306f1700 1 --2- 172.21.15.40:0/3555484737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff328102e90 0x7ff328105270 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff324009990 tx=0x7ff324009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.768+0000 7ff3306f1700 1 -- 172.21.15.40:0/3555484737 shutdown_connections 2023-07-18T03:30:48.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.768+0000 7ff3306f1700 1 --2- 172.21.15.40:0/3555484737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3281057b0 0x7ff328107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.768+0000 7ff3306f1700 1 --2- 172.21.15.40:0/3555484737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff328100570 0x7ff328102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.768+0000 7ff3306f1700 1 --2- 172.21.15.40:0/3555484737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff328102e90 0x7ff328105270 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.768+0000 7ff3306f1700 1 -- 172.21.15.40:0/3555484737 >> 172.21.15.40:0/3555484737 conn(0x7ff3280f9fd0 msgr2=0x7ff3280fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:48.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.768+0000 7ff3306f1700 1 -- 172.21.15.40:0/3555484737 shutdown_connections 2023-07-18T03:30:48.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.768+0000 7ff3306f1700 1 -- 172.21.15.40:0/3555484737 wait complete. 2023-07-18T03:30:48.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.769+0000 7ff3306f1700 1 Processor -- start 2023-07-18T03:30:48.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.769+0000 7ff3306f1700 1 -- start start 2023-07-18T03:30:48.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.769+0000 7ff3306f1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff328100570 0x7ff328197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:48.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.769+0000 7ff3306f1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff328102e90 0x7ff328197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:48.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.769+0000 7ff3306f1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3281057b0 0x7ff32819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:48.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.769+0000 7ff3306f1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32819f500 con 0x7ff328102e90 2023-07-18T03:30:48.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.769+0000 7ff3306f1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32819f640 con 0x7ff328100570 2023-07-18T03:30:48.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.769+0000 7ff3306f1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32819f7d0 con 0x7ff3281057b0 2023-07-18T03:30:48.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.769+0000 7ff32dc8c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff328102e90 0x7ff328197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:48.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.769+0000 7ff32dc8c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff328102e90 0x7ff328197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57794/0 (socket says 172.21.15.40:57794) 2023-07-18T03:30:48.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.769+0000 7ff32dc8c700 1 -- 172.21.15.40:0/4136758195 learned_addr learned my addr 172.21.15.40:0/4136758195 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:48.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.769+0000 7ff32e48d700 1 --2- 172.21.15.40:0/4136758195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff328100570 0x7ff328197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:48.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.769+0000 7ff32ec8e700 1 --2- 172.21.15.40:0/4136758195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3281057b0 0x7ff32819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:48.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.770+0000 7ff32dc8c700 1 -- 172.21.15.40:0/4136758195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3281057b0 msgr2=0x7ff32819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:48.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.770+0000 7ff32dc8c700 1 --2- 172.21.15.40:0/4136758195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3281057b0 0x7ff32819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.770+0000 7ff32dc8c700 1 -- 172.21.15.40:0/4136758195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff328100570 msgr2=0x7ff328197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:48.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.770+0000 7ff32dc8c700 1 --2- 172.21.15.40:0/4136758195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff328100570 0x7ff328197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.770+0000 7ff32dc8c700 1 -- 172.21.15.40:0/4136758195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3240095f0 con 0x7ff328102e90 2023-07-18T03:30:48.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.770+0000 7ff32dc8c700 1 --2- 172.21.15.40:0/4136758195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff328102e90 0x7ff328197ec0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff3240059d0 tx=0x7ff3240047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:48.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.770+0000 7ff31b7fe700 1 -- 172.21.15.40:0/4136758195 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32401d070 con 0x7ff328102e90 2023-07-18T03:30:48.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.770+0000 7ff31b7fe700 1 -- 172.21.15.40:0/4136758195 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff32400fb90 con 0x7ff328102e90 2023-07-18T03:30:48.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.770+0000 7ff3306f1700 1 -- 172.21.15.40:0/4136758195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff32819fa50 con 0x7ff328102e90 2023-07-18T03:30:48.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.770+0000 7ff31b7fe700 1 -- 172.21.15.40:0/4136758195 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff324017650 con 0x7ff328102e90 2023-07-18T03:30:48.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.770+0000 7ff3306f1700 1 -- 172.21.15.40:0/4136758195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32819fed0 con 0x7ff328102e90 2023-07-18T03:30:48.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.771+0000 7ff3306f1700 1 -- 172.21.15.40:0/4136758195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff328107370 con 0x7ff328102e90 2023-07-18T03:30:48.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.773+0000 7ff31b7fe700 1 -- 172.21.15.40:0/4136758195 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff32400fd00 con 0x7ff328102e90 2023-07-18T03:30:48.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.773+0000 7ff31b7fe700 1 --2- 172.21.15.40:0/4136758195 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff3140a12a0 0x7ff3140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:48.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.773+0000 7ff31b7fe700 1 -- 172.21.15.40:0/4136758195 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff3240ce5c0 con 0x7ff328102e90 2023-07-18T03:30:48.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.773+0000 7ff32e48d700 1 --2- 172.21.15.40:0/4136758195 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff3140a12a0 0x7ff3140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:48.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.774+0000 7ff32e48d700 1 --2- 172.21.15.40:0/4136758195 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff3140a12a0 0x7ff3140a3750 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff31c005950 tx=0x7ff31c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:48.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.777+0000 7ff31b7fe700 1 -- 172.21.15.40:0/4136758195 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff324097c20 con 0x7ff328102e90 2023-07-18T03:30:48.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.975+0000 7ff3306f1700 1 -- 172.21.15.40:0/4136758195 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff32802c940 con 0x7ff328102e90 2023-07-18T03:30:48.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.977+0000 7ff31b7fe700 1 -- 172.21.15.40:0/4136758195 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22599 (secure 0 0 0) 0x7ff32409b240 con 0x7ff328102e90 2023-07-18T03:30:48.974 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:48.974 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":83,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:30:33.646627+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:30:48.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.979+0000 7ff3306f1700 1 -- 172.21.15.40:0/4136758195 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff3140a12a0 msgr2=0x7ff3140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:48.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.979+0000 7ff3306f1700 1 --2- 172.21.15.40:0/4136758195 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff3140a12a0 0x7ff3140a3750 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff31c005950 tx=0x7ff31c00a300 comp rx=0 tx=0).stop 2023-07-18T03:30:48.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.979+0000 7ff3306f1700 1 -- 172.21.15.40:0/4136758195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff328102e90 msgr2=0x7ff328197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:48.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.979+0000 7ff3306f1700 1 --2- 172.21.15.40:0/4136758195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff328102e90 0x7ff328197ec0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff3240059d0 tx=0x7ff3240047a0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.979+0000 7ff3306f1700 1 -- 172.21.15.40:0/4136758195 shutdown_connections 2023-07-18T03:30:48.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.979+0000 7ff3306f1700 1 --2- 172.21.15.40:0/4136758195 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff3140a12a0 0x7ff3140a3750 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.979+0000 7ff3306f1700 1 --2- 172.21.15.40:0/4136758195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3281057b0 0x7ff32819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.979+0000 7ff3306f1700 1 --2- 172.21.15.40:0/4136758195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff328100570 0x7ff328197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.979+0000 7ff3306f1700 1 --2- 172.21.15.40:0/4136758195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff328102e90 0x7ff328197ec0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:48.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.979+0000 7ff3306f1700 1 -- 172.21.15.40:0/4136758195 >> 172.21.15.40:0/4136758195 conn(0x7ff3280f9fd0 msgr2=0x7ff3280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:48.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.979+0000 7ff3306f1700 1 -- 172.21.15.40:0/4136758195 shutdown_connections 2023-07-18T03:30:48.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:48.979+0000 7ff3306f1700 1 -- 172.21.15.40:0/4136758195 wait complete. 2023-07-18T03:30:48.987 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:48 smithi040 ceph-mon[96439]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:30:48.988 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:25:38.267262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:29:24.727790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:29:26.078507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:30:48.989 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:30:49.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:48 smithi134 ceph-mon[98216]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:30:49.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:48 smithi064 ceph-mon[98059]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:30:49.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.121+0000 7f869ba28700 1 -- 172.21.15.40:0/2999347233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8694100560 msgr2=0x7f8694106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:49.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.121+0000 7f869ba28700 1 --2- 172.21.15.40:0/2999347233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8694100560 0x7f8694106e20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f868c00b0a0 tx=0x7f868c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.121+0000 7f869ba28700 1 -- 172.21.15.40:0/2999347233 shutdown_connections 2023-07-18T03:30:49.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.121+0000 7f869ba28700 1 --2- 172.21.15.40:0/2999347233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8694100560 0x7f8694106e20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.121+0000 7f869ba28700 1 --2- 172.21.15.40:0/2999347233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86940fe9e0 0x7f86940fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.121+0000 7f869ba28700 1 --2- 172.21.15.40:0/2999347233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86940ffbd0 0x7f8694100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.121+0000 7f869ba28700 1 -- 172.21.15.40:0/2999347233 >> 172.21.15.40:0/2999347233 conn(0x7f86940fa000 msgr2=0x7f86940fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:49.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.121+0000 7f869ba28700 1 -- 172.21.15.40:0/2999347233 shutdown_connections 2023-07-18T03:30:49.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.121+0000 7f869ba28700 1 -- 172.21.15.40:0/2999347233 wait complete. 2023-07-18T03:30:49.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f869ba28700 1 Processor -- start 2023-07-18T03:30:49.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f869ba28700 1 -- start start 2023-07-18T03:30:49.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f869ba28700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86940fe9e0 0x7f8694102920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:49.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f869ba28700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86940ffbd0 0x7f8694100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:49.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f869ba28700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8694100560 0x7f86941014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:49.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f869ba28700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8694101c90 con 0x7f8694100560 2023-07-18T03:30:49.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f869ba28700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8694101e00 con 0x7f86940fe9e0 2023-07-18T03:30:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f869ba28700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869419f9e0 con 0x7f86940ffbd0 2023-07-18T03:30:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f8699fc5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8694100560 0x7f86941014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f8699fc5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8694100560 0x7f86941014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57810/0 (socket says 172.21.15.40:57810) 2023-07-18T03:30:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f8699fc5700 1 -- 172.21.15.40:0/116152792 learned_addr learned my addr 172.21.15.40:0/116152792 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f86997c4700 1 --2- 172.21.15.40:0/116152792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86940fe9e0 0x7f8694102920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f8699fc5700 1 -- 172.21.15.40:0/116152792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86940ffbd0 msgr2=0x7f8694100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f8698fc3700 1 --2- 172.21.15.40:0/116152792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86940ffbd0 0x7f8694100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f8699fc5700 1 --2- 172.21.15.40:0/116152792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86940ffbd0 0x7f8694100fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f8699fc5700 1 -- 172.21.15.40:0/116152792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86940fe9e0 msgr2=0x7f8694102920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:49.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f8699fc5700 1 --2- 172.21.15.40:0/116152792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86940fe9e0 0x7f8694102920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f8699fc5700 1 -- 172.21.15.40:0/116152792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868c009d00 con 0x7f8694100560 2023-07-18T03:30:49.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f86997c4700 1 --2- 172.21.15.40:0/116152792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86940fe9e0 0x7f8694102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:49.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f8698fc3700 1 --2- 172.21.15.40:0/116152792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86940ffbd0 0x7f8694100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:49.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.122+0000 7f8699fc5700 1 --2- 172.21.15.40:0/116152792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8694100560 0x7f86941014f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f868c00b830 tx=0x7f868c004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:49.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.123+0000 7f86867fc700 1 -- 172.21.15.40:0/116152792 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c00e040 con 0x7f8694100560 2023-07-18T03:30:49.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.123+0000 7f86867fc700 1 -- 172.21.15.40:0/116152792 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f868c0094c0 con 0x7f8694100560 2023-07-18T03:30:49.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.123+0000 7f869ba28700 1 -- 172.21.15.40:0/116152792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869419fc10 con 0x7f8694100560 2023-07-18T03:30:49.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.123+0000 7f86867fc700 1 -- 172.21.15.40:0/116152792 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c01bc70 con 0x7f8694100560 2023-07-18T03:30:49.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.123+0000 7f869ba28700 1 -- 172.21.15.40:0/116152792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86941a0030 con 0x7f8694100560 2023-07-18T03:30:49.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.124+0000 7f869ba28700 1 -- 172.21.15.40:0/116152792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f869410a1c0 con 0x7f8694100560 2023-07-18T03:30:49.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.125+0000 7f86867fc700 1 -- 172.21.15.40:0/116152792 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f868c01d890 con 0x7f8694100560 2023-07-18T03:30:49.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.125+0000 7f86867fc700 1 --2- 172.21.15.40:0/116152792 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f86800a11e0 0x7f86800a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:49.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.126+0000 7f86997c4700 1 --2- 172.21.15.40:0/116152792 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f86800a11e0 0x7f86800a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:49.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.126+0000 7f86867fc700 1 -- 172.21.15.40:0/116152792 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f868c0cf0e0 con 0x7f8694100560 2023-07-18T03:30:49.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.126+0000 7f86997c4700 1 --2- 172.21.15.40:0/116152792 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f86800a11e0 0x7f86800a3690 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f86940ffa30 tx=0x7f868800b670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:49.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.129+0000 7f86867fc700 1 -- 172.21.15.40:0/116152792 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f868c098800 con 0x7f8694100560 2023-07-18T03:30:49.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.328+0000 7f869ba28700 1 -- 172.21.15.40:0/116152792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86941a05e0 con 0x7f8694100560 2023-07-18T03:30:49.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.329+0000 7f86867fc700 1 -- 172.21.15.40:0/116152792 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f868c0210a0 con 0x7f8694100560 2023-07-18T03:30:49.326 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-07-18T03:30:49.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.331+0000 7f869ba28700 1 -- 172.21.15.40:0/116152792 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f86800a11e0 msgr2=0x7f86800a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:49.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.331+0000 7f869ba28700 1 --2- 172.21.15.40:0/116152792 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f86800a11e0 0x7f86800a3690 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f86940ffa30 tx=0x7f868800b670 comp rx=0 tx=0).stop 2023-07-18T03:30:49.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.331+0000 7f869ba28700 1 -- 172.21.15.40:0/116152792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8694100560 msgr2=0x7f86941014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:49.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.331+0000 7f869ba28700 1 --2- 172.21.15.40:0/116152792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8694100560 0x7f86941014f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f868c00b830 tx=0x7f868c004d30 comp rx=0 tx=0).stop 2023-07-18T03:30:49.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.331+0000 7f869ba28700 1 -- 172.21.15.40:0/116152792 shutdown_connections 2023-07-18T03:30:49.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.331+0000 7f869ba28700 1 --2- 172.21.15.40:0/116152792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8694100560 0x7f86941014f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.331+0000 7f869ba28700 1 --2- 172.21.15.40:0/116152792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86940fe9e0 0x7f8694102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.331+0000 7f869ba28700 1 --2- 172.21.15.40:0/116152792 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f86800a11e0 0x7f86800a3690 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.331+0000 7f869ba28700 1 --2- 172.21.15.40:0/116152792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86940ffbd0 0x7f8694100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.331+0000 7f869ba28700 1 -- 172.21.15.40:0/116152792 >> 172.21.15.40:0/116152792 conn(0x7f86940fa000 msgr2=0x7f86940676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:49.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.331+0000 7f869ba28700 1 -- 172.21.15.40:0/116152792 shutdown_connections 2023-07-18T03:30:49.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.331+0000 7f869ba28700 1 -- 172.21.15.40:0/116152792 wait complete. 2023-07-18T03:30:49.340 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:30:49.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.473+0000 7fafa27c7700 1 -- 172.21.15.40:0/2079880324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf9c100cd0 msgr2=0x7faf9c101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:49.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.473+0000 7fafa27c7700 1 --2- 172.21.15.40:0/2079880324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf9c100cd0 0x7faf9c101120 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7faf90009990 tx=0x7faf90009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.473+0000 7fafa27c7700 1 -- 172.21.15.40:0/2079880324 shutdown_connections 2023-07-18T03:30:49.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.473+0000 7fafa27c7700 1 --2- 172.21.15.40:0/2079880324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf9c101660 0x7faf9c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.473+0000 7fafa27c7700 1 --2- 172.21.15.40:0/2079880324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf9c0ffae0 0x7faf9c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.473+0000 7fafa27c7700 1 --2- 172.21.15.40:0/2079880324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf9c100cd0 0x7faf9c101120 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.473+0000 7fafa27c7700 1 -- 172.21.15.40:0/2079880324 >> 172.21.15.40:0/2079880324 conn(0x7faf9c0fb120 msgr2=0x7faf9c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:49.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.473+0000 7fafa27c7700 1 -- 172.21.15.40:0/2079880324 shutdown_connections 2023-07-18T03:30:49.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.473+0000 7fafa27c7700 1 -- 172.21.15.40:0/2079880324 wait complete. 2023-07-18T03:30:49.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7fafa27c7700 1 Processor -- start 2023-07-18T03:30:49.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7fafa27c7700 1 -- start start 2023-07-18T03:30:49.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7fafa27c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf9c0ffae0 0x7faf9c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:49.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7fafa27c7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf9c100cd0 0x7faf9c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:49.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7fafa27c7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf9c101660 0x7faf9c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:49.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7fafa27c7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0735e0 con 0x7faf9c100cd0 2023-07-18T03:30:49.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7fafa27c7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c073750 con 0x7faf9c0ffae0 2023-07-18T03:30:49.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7fafa27c7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c19f840 con 0x7faf9c101660 2023-07-18T03:30:49.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7faf9b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf9c100cd0 0x7faf9c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:49.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7faf9b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf9c100cd0 0x7faf9c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57822/0 (socket says 172.21.15.40:57822) 2023-07-18T03:30:49.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7faf9b7fe700 1 -- 172.21.15.40:0/1738384089 learned_addr learned my addr 172.21.15.40:0/1738384089 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:49.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7fafa0d64700 1 --2- 172.21.15.40:0/1738384089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf9c101660 0x7faf9c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:49.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7faf9bfff700 1 --2- 172.21.15.40:0/1738384089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf9c0ffae0 0x7faf9c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:49.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7faf9b7fe700 1 -- 172.21.15.40:0/1738384089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf9c101660 msgr2=0x7faf9c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:49.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7faf9b7fe700 1 --2- 172.21.15.40:0/1738384089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf9c101660 0x7faf9c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7faf9b7fe700 1 -- 172.21.15.40:0/1738384089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf9c0ffae0 msgr2=0x7faf9c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:49.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7faf9b7fe700 1 --2- 172.21.15.40:0/1738384089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf9c0ffae0 0x7faf9c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7faf9b7fe700 1 -- 172.21.15.40:0/1738384089 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf900095f0 con 0x7faf9c100cd0 2023-07-18T03:30:49.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.474+0000 7fafa0d64700 1 --2- 172.21.15.40:0/1738384089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf9c101660 0x7faf9c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:49.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.475+0000 7faf9b7fe700 1 --2- 172.21.15.40:0/1738384089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf9c100cd0 0x7faf9c072900 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7faf90000c00 tx=0x7faf900047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:49.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.475+0000 7faf997fa700 1 -- 172.21.15.40:0/1738384089 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9001d070 con 0x7faf9c100cd0 2023-07-18T03:30:49.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.475+0000 7faf997fa700 1 -- 172.21.15.40:0/1738384089 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf90022470 con 0x7faf9c100cd0 2023-07-18T03:30:49.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.475+0000 7fafa27c7700 1 -- 172.21.15.40:0/1738384089 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9c19fac0 con 0x7faf9c100cd0 2023-07-18T03:30:49.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.475+0000 7faf997fa700 1 -- 172.21.15.40:0/1738384089 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf90017650 con 0x7faf9c100cd0 2023-07-18T03:30:49.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.475+0000 7fafa27c7700 1 -- 172.21.15.40:0/1738384089 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9c19fee0 con 0x7faf9c100cd0 2023-07-18T03:30:49.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.476+0000 7fafa27c7700 1 -- 172.21.15.40:0/1738384089 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9c106a30 con 0x7faf9c100cd0 2023-07-18T03:30:49.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.478+0000 7faf997fa700 1 -- 172.21.15.40:0/1738384089 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7faf900225e0 con 0x7faf9c100cd0 2023-07-18T03:30:49.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.478+0000 7faf997fa700 1 --2- 172.21.15.40:0/1738384089 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faf840a11e0 0x7faf840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:49.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.478+0000 7faf9bfff700 1 --2- 172.21.15.40:0/1738384089 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faf840a11e0 0x7faf840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:49.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.478+0000 7faf997fa700 1 -- 172.21.15.40:0/1738384089 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7faf900ce470 con 0x7faf9c100cd0 2023-07-18T03:30:49.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.479+0000 7faf9bfff700 1 --2- 172.21.15.40:0/1738384089 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faf840a11e0 0x7faf840a3690 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7faf9c100b30 tx=0x7faf8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:49.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.481+0000 7faf997fa700 1 -- 172.21.15.40:0/1738384089 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7faf90097ad0 con 0x7faf9c100cd0 2023-07-18T03:30:49.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.682+0000 7fafa27c7700 1 -- 172.21.15.40:0/1738384089 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faf9c02caa0 con 0x7faf9c100cd0 2023-07-18T03:30:49.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.683+0000 7faf997fa700 1 -- 172.21.15.40:0/1738384089 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7faf9009b0f0 con 0x7faf9c100cd0 2023-07-18T03:30:49.680 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-07-18T03:30:49.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.685+0000 7fafa27c7700 1 -- 172.21.15.40:0/1738384089 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faf840a11e0 msgr2=0x7faf840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:49.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.685+0000 7fafa27c7700 1 --2- 172.21.15.40:0/1738384089 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faf840a11e0 0x7faf840a3690 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7faf9c100b30 tx=0x7faf8c009260 comp rx=0 tx=0).stop 2023-07-18T03:30:49.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.685+0000 7fafa27c7700 1 -- 172.21.15.40:0/1738384089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf9c100cd0 msgr2=0x7faf9c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:49.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.685+0000 7fafa27c7700 1 --2- 172.21.15.40:0/1738384089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf9c100cd0 0x7faf9c072900 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7faf90000c00 tx=0x7faf900047a0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.685+0000 7fafa27c7700 1 -- 172.21.15.40:0/1738384089 shutdown_connections 2023-07-18T03:30:49.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.685+0000 7fafa27c7700 1 --2- 172.21.15.40:0/1738384089 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faf840a11e0 0x7faf840a3690 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.685+0000 7fafa27c7700 1 --2- 172.21.15.40:0/1738384089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf9c101660 0x7faf9c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.685+0000 7fafa27c7700 1 --2- 172.21.15.40:0/1738384089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf9c0ffae0 0x7faf9c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.685+0000 7fafa27c7700 1 --2- 172.21.15.40:0/1738384089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf9c100cd0 0x7faf9c072900 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.685+0000 7fafa27c7700 1 -- 172.21.15.40:0/1738384089 >> 172.21.15.40:0/1738384089 conn(0x7faf9c0fb120 msgr2=0x7faf9c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.685+0000 7fafa27c7700 1 -- 172.21.15.40:0/1738384089 shutdown_connections 2023-07-18T03:30:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.685+0000 7fafa27c7700 1 -- 172.21.15.40:0/1738384089 wait complete. 2023-07-18T03:30:49.694 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:30:49.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.828+0000 7ff317b6b700 1 -- 172.21.15.40:0/4164076858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3100ffbd0 msgr2=0x7ff310100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:49.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.828+0000 7ff317b6b700 1 --2- 172.21.15.40:0/4164076858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3100ffbd0 0x7ff310100020 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff300009990 tx=0x7ff300009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.828+0000 7ff317b6b700 1 -- 172.21.15.40:0/4164076858 shutdown_connections 2023-07-18T03:30:49.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.828+0000 7ff317b6b700 1 --2- 172.21.15.40:0/4164076858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff310100560 0x7ff310106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.828+0000 7ff317b6b700 1 --2- 172.21.15.40:0/4164076858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3100fe9e0 0x7ff3100fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.828+0000 7ff317b6b700 1 --2- 172.21.15.40:0/4164076858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3100ffbd0 0x7ff310100020 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.828+0000 7ff317b6b700 1 -- 172.21.15.40:0/4164076858 >> 172.21.15.40:0/4164076858 conn(0x7ff3100fa000 msgr2=0x7ff3100fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:49.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.828+0000 7ff317b6b700 1 -- 172.21.15.40:0/4164076858 shutdown_connections 2023-07-18T03:30:49.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.828+0000 7ff317b6b700 1 -- 172.21.15.40:0/4164076858 wait complete. 2023-07-18T03:30:49.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.828+0000 7ff317b6b700 1 Processor -- start 2023-07-18T03:30:49.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff317b6b700 1 -- start start 2023-07-18T03:30:49.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff317b6b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3100fe9e0 0x7ff3101947b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:49.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff317b6b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3100ffbd0 0x7ff310194cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:49.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff317b6b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff310100560 0x7ff31019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:49.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff317b6b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31019c330 con 0x7ff3100fe9e0 2023-07-18T03:30:49.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff317b6b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31019c470 con 0x7ff310100560 2023-07-18T03:30:49.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff317b6b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31019c600 con 0x7ff3100ffbd0 2023-07-18T03:30:49.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff315907700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3100fe9e0 0x7ff3101947b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:49.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff315907700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3100fe9e0 0x7ff3101947b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57834/0 (socket says 172.21.15.40:57834) 2023-07-18T03:30:49.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff315907700 1 -- 172.21.15.40:0/1288257816 learned_addr learned my addr 172.21.15.40:0/1288257816 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:49.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff315106700 1 --2- 172.21.15.40:0/1288257816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3100ffbd0 0x7ff310194cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:49.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff316108700 1 --2- 172.21.15.40:0/1288257816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff310100560 0x7ff31019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:49.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff315907700 1 -- 172.21.15.40:0/1288257816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3100ffbd0 msgr2=0x7ff310194cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:49.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff315907700 1 --2- 172.21.15.40:0/1288257816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3100ffbd0 0x7ff310194cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff315907700 1 -- 172.21.15.40:0/1288257816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff310100560 msgr2=0x7ff31019bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:49.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff315907700 1 --2- 172.21.15.40:0/1288257816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff310100560 0x7ff31019bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:49.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff315907700 1 -- 172.21.15.40:0/1288257816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3000095f0 con 0x7ff3100fe9e0 2023-07-18T03:30:49.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.829+0000 7ff315907700 1 --2- 172.21.15.40:0/1288257816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3100fe9e0 0x7ff3101947b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00d980 tx=0x7ff30c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:49.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.830+0000 7ff306ffd700 1 -- 172.21.15.40:0/1288257816 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c004020 con 0x7ff3100fe9e0 2023-07-18T03:30:49.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.830+0000 7ff306ffd700 1 -- 172.21.15.40:0/1288257816 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff30c00f690 con 0x7ff3100fe9e0 2023-07-18T03:30:49.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.830+0000 7ff306ffd700 1 -- 172.21.15.40:0/1288257816 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c003a60 con 0x7ff3100fe9e0 2023-07-18T03:30:49.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.830+0000 7ff317b6b700 1 -- 172.21.15.40:0/1288257816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31019c8e0 con 0x7ff3100fe9e0 2023-07-18T03:30:49.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.830+0000 7ff317b6b700 1 -- 172.21.15.40:0/1288257816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31019cdf0 con 0x7ff3100fe9e0 2023-07-18T03:30:49.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.831+0000 7ff317b6b700 1 -- 172.21.15.40:0/1288257816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff310106020 con 0x7ff3100fe9e0 2023-07-18T03:30:49.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.832+0000 7ff306ffd700 1 -- 172.21.15.40:0/1288257816 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff30c00fb00 con 0x7ff3100fe9e0 2023-07-18T03:30:49.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.832+0000 7ff306ffd700 1 --2- 172.21.15.40:0/1288257816 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff2fc0a1260 0x7ff2fc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:49.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.832+0000 7ff306ffd700 1 -- 172.21.15.40:0/1288257816 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff30c0cc970 con 0x7ff3100fe9e0 2023-07-18T03:30:49.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.832+0000 7ff315106700 1 --2- 172.21.15.40:0/1288257816 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff2fc0a1260 0x7ff2fc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:49.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.833+0000 7ff315106700 1 --2- 172.21.15.40:0/1288257816 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff2fc0a1260 0x7ff2fc0a3710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff30000b5c0 tx=0x7ff300005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:49.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:49.836+0000 7ff306ffd700 1 -- 172.21.15.40:0/1288257816 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff30c095fc0 con 0x7ff3100fe9e0 2023-07-18T03:30:50.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.035+0000 7ff317b6b700 1 -- 172.21.15.40:0/1288257816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff31002ca00 con 0x7ff3100fe9e0 2023-07-18T03:30:50.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.035+0000 7ff306ffd700 1 -- 172.21.15.40:0/1288257816 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff30c0995e0 con 0x7ff3100fe9e0 2023-07-18T03:30:50.032 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-07-18T03:30:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.037+0000 7ff317b6b700 1 -- 172.21.15.40:0/1288257816 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff2fc0a1260 msgr2=0x7ff2fc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.037+0000 7ff317b6b700 1 --2- 172.21.15.40:0/1288257816 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff2fc0a1260 0x7ff2fc0a3710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff30000b5c0 tx=0x7ff300005960 comp rx=0 tx=0).stop 2023-07-18T03:30:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.037+0000 7ff317b6b700 1 -- 172.21.15.40:0/1288257816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3100fe9e0 msgr2=0x7ff3101947b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.037+0000 7ff317b6b700 1 --2- 172.21.15.40:0/1288257816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3100fe9e0 0x7ff3101947b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00d980 tx=0x7ff30c00dc90 comp rx=0 tx=0).stop 2023-07-18T03:30:50.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.037+0000 7ff317b6b700 1 -- 172.21.15.40:0/1288257816 shutdown_connections 2023-07-18T03:30:50.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.037+0000 7ff317b6b700 1 --2- 172.21.15.40:0/1288257816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff310100560 0x7ff31019bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.037+0000 7ff317b6b700 1 --2- 172.21.15.40:0/1288257816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3100fe9e0 0x7ff3101947b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.037+0000 7ff317b6b700 1 --2- 172.21.15.40:0/1288257816 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff2fc0a1260 0x7ff2fc0a3710 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.037+0000 7ff317b6b700 1 --2- 172.21.15.40:0/1288257816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3100ffbd0 0x7ff310194cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.037+0000 7ff317b6b700 1 -- 172.21.15.40:0/1288257816 >> 172.21.15.40:0/1288257816 conn(0x7ff3100fa000 msgr2=0x7ff310103160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:50.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.037+0000 7ff317b6b700 1 -- 172.21.15.40:0/1288257816 shutdown_connections 2023-07-18T03:30:50.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.038+0000 7ff317b6b700 1 -- 172.21.15.40:0/1288257816 wait complete. 2023-07-18T03:30:50.045 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:49 smithi040 ceph-mon[96439]: pgmap v291: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s; 7 B/s, 0 keys/s, 0 objects/s recovering 2023-07-18T03:30:50.046 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:49 smithi040 ceph-mon[96439]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:30:50.046 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:49 smithi040 ceph-mon[96439]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:30:50.046 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:49 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/4136758195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:50.046 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:49 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/116152792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:50.046 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:49 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1738384089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:50.047 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:30:50.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:49 smithi134 ceph-mon[98216]: pgmap v291: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s; 7 B/s, 0 keys/s, 0 objects/s recovering 2023-07-18T03:30:50.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:49 smithi134 ceph-mon[98216]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:30:50.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:49 smithi134 ceph-mon[98216]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:30:50.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:49 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/4136758195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:50.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:49 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/116152792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:50.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:49 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1738384089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:50.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:49 smithi064 ceph-mon[98059]: pgmap v291: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s; 7 B/s, 0 keys/s, 0 objects/s recovering 2023-07-18T03:30:50.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:49 smithi064 ceph-mon[98059]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:30:50.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:49 smithi064 ceph-mon[98059]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-18T03:30:50.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:49 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/4136758195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:50.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:49 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/116152792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:30:50.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:49 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1738384089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:30:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.177+0000 7efd996bd700 1 -- 172.21.15.40:0/2541606307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101660 msgr2=0x7efd94107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.177+0000 7efd996bd700 1 --2- 172.21.15.40:0/2541606307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101660 0x7efd94107f20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00b0a0 tx=0x7efd8c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.177+0000 7efd996bd700 1 -- 172.21.15.40:0/2541606307 shutdown_connections 2023-07-18T03:30:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.177+0000 7efd996bd700 1 --2- 172.21.15.40:0/2541606307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101660 0x7efd94107f20 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.177+0000 7efd996bd700 1 --2- 172.21.15.40:0/2541606307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd940ffae0 0x7efd940ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.177+0000 7efd996bd700 1 --2- 172.21.15.40:0/2541606307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd94100cd0 0x7efd94101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.177+0000 7efd996bd700 1 -- 172.21.15.40:0/2541606307 >> 172.21.15.40:0/2541606307 conn(0x7efd940fb100 msgr2=0x7efd940fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.177+0000 7efd996bd700 1 -- 172.21.15.40:0/2541606307 shutdown_connections 2023-07-18T03:30:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.177+0000 7efd996bd700 1 -- 172.21.15.40:0/2541606307 wait complete. 2023-07-18T03:30:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.178+0000 7efd996bd700 1 Processor -- start 2023-07-18T03:30:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.178+0000 7efd996bd700 1 -- start start 2023-07-18T03:30:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.178+0000 7efd996bd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd940ffae0 0x7efd941958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.178+0000 7efd996bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd94100cd0 0x7efd94195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.178+0000 7efd996bd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101660 0x7efd9419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.178+0000 7efd996bd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9419d430 con 0x7efd94101660 2023-07-18T03:30:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.178+0000 7efd996bd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9419d570 con 0x7efd94100cd0 2023-07-18T03:30:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.178+0000 7efd996bd700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9419d700 con 0x7efd940ffae0 2023-07-18T03:30:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.179+0000 7efd937fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101660 0x7efd9419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.179+0000 7efd937fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101660 0x7efd9419ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57852/0 (socket says 172.21.15.40:57852) 2023-07-18T03:30:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.179+0000 7efd937fe700 1 -- 172.21.15.40:0/3378275731 learned_addr learned my addr 172.21.15.40:0/3378275731 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.179+0000 7efd8a7fc700 1 --2- 172.21.15.40:0/3378275731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd94100cd0 0x7efd94195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.179+0000 7efd92ffd700 1 --2- 172.21.15.40:0/3378275731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd940ffae0 0x7efd941958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.179+0000 7efd937fe700 1 -- 172.21.15.40:0/3378275731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd940ffae0 msgr2=0x7efd941958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.179+0000 7efd937fe700 1 --2- 172.21.15.40:0/3378275731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd940ffae0 0x7efd941958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.179+0000 7efd937fe700 1 -- 172.21.15.40:0/3378275731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd94100cd0 msgr2=0x7efd94195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.179+0000 7efd937fe700 1 --2- 172.21.15.40:0/3378275731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd94100cd0 0x7efd94195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.179+0000 7efd937fe700 1 -- 172.21.15.40:0/3378275731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c009d00 con 0x7efd94101660 2023-07-18T03:30:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.179+0000 7efd937fe700 1 --2- 172.21.15.40:0/3378275731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101660 0x7efd9419ce60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00b830 tx=0x7efd8c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.179+0000 7efd90ff9700 1 -- 172.21.15.40:0/3378275731 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c00e040 con 0x7efd94101660 2023-07-18T03:30:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.179+0000 7efd90ff9700 1 -- 172.21.15.40:0/3378275731 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd8c01db50 con 0x7efd94101660 2023-07-18T03:30:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.179+0000 7efd90ff9700 1 -- 172.21.15.40:0/3378275731 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c01bb20 con 0x7efd94101660 2023-07-18T03:30:50.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.179+0000 7efd996bd700 1 -- 172.21.15.40:0/3378275731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9419d980 con 0x7efd94101660 2023-07-18T03:30:50.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.179+0000 7efd996bd700 1 -- 172.21.15.40:0/3378275731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9419de00 con 0x7efd94101660 2023-07-18T03:30:50.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.180+0000 7efd996bd700 1 -- 172.21.15.40:0/3378275731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9410b2c0 con 0x7efd94101660 2023-07-18T03:30:50.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.181+0000 7efd90ff9700 1 -- 172.21.15.40:0/3378275731 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7efd8c01dcc0 con 0x7efd94101660 2023-07-18T03:30:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.182+0000 7efd90ff9700 1 --2- 172.21.15.40:0/3378275731 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efd800a12a0 0x7efd800a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.182+0000 7efd92ffd700 1 --2- 172.21.15.40:0/3378275731 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efd800a12a0 0x7efd800a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.182+0000 7efd90ff9700 1 -- 172.21.15.40:0/3378275731 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7efd8c0cec10 con 0x7efd94101660 2023-07-18T03:30:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.183+0000 7efd92ffd700 1 --2- 172.21.15.40:0/3378275731 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efd800a12a0 0x7efd800a3750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efd94100b30 tx=0x7efd8400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.185+0000 7efd90ff9700 1 -- 172.21.15.40:0/3378275731 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7efd8c098290 con 0x7efd94101660 2023-07-18T03:30:50.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.429+0000 7efd996bd700 1 -- 172.21.15.40:0/3378275731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd94106920 con 0x7efd94101660 2023-07-18T03:30:50.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.430+0000 7efd90ff9700 1 -- 172.21.15.40:0/3378275731 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10888 (secure 0 0 0) 0x7efd8c09b8b0 con 0x7efd94101660 2023-07-18T03:30:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:50.427 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:29:26.718789+0000","modified":"2023-07-18T03:29:34.650353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24523,"mds_2":24482,"mds_3":24452,"mds_4":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14601":{"gid":14601,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1818167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1818167554},{"type":"v1","addr":"172.21.15.40:6839","nonce":1818167554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14613":{"gid":14613,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/3953074960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":3953074960},{"type":"v1","addr":"172.21.15.40:6841","nonce":3953074960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3509655945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3509655945},{"type":"v1","addr":"172.21.15.134:6837","nonce":3509655945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/3007039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3007039043},{"type":"v1","addr":"172.21.15.64:6839","nonce":3007039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:30:50.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.432+0000 7efd996bd700 1 -- 172.21.15.40:0/3378275731 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efd800a12a0 msgr2=0x7efd800a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.432+0000 7efd996bd700 1 --2- 172.21.15.40:0/3378275731 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efd800a12a0 0x7efd800a3750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efd94100b30 tx=0x7efd8400b410 comp rx=0 tx=0).stop 2023-07-18T03:30:50.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.432+0000 7efd996bd700 1 -- 172.21.15.40:0/3378275731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101660 msgr2=0x7efd9419ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.432+0000 7efd996bd700 1 --2- 172.21.15.40:0/3378275731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101660 0x7efd9419ce60 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00b830 tx=0x7efd8c004c80 comp rx=0 tx=0).stop 2023-07-18T03:30:50.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.432+0000 7efd996bd700 1 -- 172.21.15.40:0/3378275731 shutdown_connections 2023-07-18T03:30:50.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.432+0000 7efd996bd700 1 --2- 172.21.15.40:0/3378275731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd94101660 0x7efd9419ce60 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.432+0000 7efd996bd700 1 --2- 172.21.15.40:0/3378275731 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efd800a12a0 0x7efd800a3750 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.432+0000 7efd996bd700 1 --2- 172.21.15.40:0/3378275731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd940ffae0 0x7efd941958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.432+0000 7efd996bd700 1 --2- 172.21.15.40:0/3378275731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd94100cd0 0x7efd94195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.432+0000 7efd996bd700 1 -- 172.21.15.40:0/3378275731 >> 172.21.15.40:0/3378275731 conn(0x7efd940fb100 msgr2=0x7efd940fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:50.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.432+0000 7efd996bd700 1 -- 172.21.15.40:0/3378275731 shutdown_connections 2023-07-18T03:30:50.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.432+0000 7efd996bd700 1 -- 172.21.15.40:0/3378275731 wait complete. 2023-07-18T03:30:50.430 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 27 2023-07-18T03:30:50.441 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:30:50.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.570+0000 7f79ce332700 1 -- 172.21.15.40:0/3578377371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c8100590 msgr2=0x7f79c8106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.570+0000 7f79ce332700 1 --2- 172.21.15.40:0/3578377371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c8100590 0x7f79c8106e50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f79c000b0a0 tx=0x7f79c000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.570+0000 7f79ce332700 1 -- 172.21.15.40:0/3578377371 shutdown_connections 2023-07-18T03:30:50.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.570+0000 7f79ce332700 1 --2- 172.21.15.40:0/3578377371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c8100590 0x7f79c8106e50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.570+0000 7f79ce332700 1 --2- 172.21.15.40:0/3578377371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79c80fea10 0x7f79c80fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.570+0000 7f79ce332700 1 --2- 172.21.15.40:0/3578377371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79c80ffc00 0x7f79c8100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.570+0000 7f79ce332700 1 -- 172.21.15.40:0/3578377371 >> 172.21.15.40:0/3578377371 conn(0x7f79c80fa010 msgr2=0x7f79c80fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:50.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.570+0000 7f79ce332700 1 -- 172.21.15.40:0/3578377371 shutdown_connections 2023-07-18T03:30:50.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.570+0000 7f79ce332700 1 -- 172.21.15.40:0/3578377371 wait complete. 2023-07-18T03:30:50.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.570+0000 7f79ce332700 1 Processor -- start 2023-07-18T03:30:50.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79ce332700 1 -- start start 2023-07-18T03:30:50.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79ce332700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79c80fea10 0x7f79c81979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79ce332700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79c80ffc00 0x7f79c8197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79ce332700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c8100590 0x7f79c819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79ce332700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c819f570 con 0x7f79c8100590 2023-07-18T03:30:50.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79ce332700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c819f6b0 con 0x7f79c80ffc00 2023-07-18T03:30:50.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79ce332700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c819f840 con 0x7f79c80fea10 2023-07-18T03:30:50.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79cc8cf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c8100590 0x7f79c819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79cc8cf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c8100590 0x7f79c819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57866/0 (socket says 172.21.15.40:57866) 2023-07-18T03:30:50.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79cc8cf700 1 -- 172.21.15.40:0/130209875 learned_addr learned my addr 172.21.15.40:0/130209875 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:50.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79c77fe700 1 --2- 172.21.15.40:0/130209875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79c80ffc00 0x7f79c8197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79c7fff700 1 --2- 172.21.15.40:0/130209875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79c80fea10 0x7f79c81979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79cc8cf700 1 -- 172.21.15.40:0/130209875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79c80fea10 msgr2=0x7f79c81979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79cc8cf700 1 --2- 172.21.15.40:0/130209875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79c80fea10 0x7f79c81979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79cc8cf700 1 -- 172.21.15.40:0/130209875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79c80ffc00 msgr2=0x7f79c8197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79cc8cf700 1 --2- 172.21.15.40:0/130209875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79c80ffc00 0x7f79c8197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79cc8cf700 1 -- 172.21.15.40:0/130209875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c0009d00 con 0x7f79c8100590 2023-07-18T03:30:50.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79c7fff700 1 --2- 172.21.15.40:0/130209875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79c80fea10 0x7f79c81979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:50.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.571+0000 7f79cc8cf700 1 --2- 172.21.15.40:0/130209875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c8100590 0x7f79c819efa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f79c000b830 tx=0x7f79c0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:50.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.572+0000 7f79c57fa700 1 -- 172.21.15.40:0/130209875 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c000e040 con 0x7f79c8100590 2023-07-18T03:30:50.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.572+0000 7f79c57fa700 1 -- 172.21.15.40:0/130209875 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79c001db50 con 0x7f79c8100590 2023-07-18T03:30:50.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.572+0000 7f79ce332700 1 -- 172.21.15.40:0/130209875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c819fac0 con 0x7f79c8100590 2023-07-18T03:30:50.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.572+0000 7f79c57fa700 1 -- 172.21.15.40:0/130209875 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c001bbd0 con 0x7f79c8100590 2023-07-18T03:30:50.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.572+0000 7f79ce332700 1 -- 172.21.15.40:0/130209875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c819ff40 con 0x7f79c8100590 2023-07-18T03:30:50.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.573+0000 7f79ce332700 1 -- 172.21.15.40:0/130209875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c810a1f0 con 0x7f79c8100590 2023-07-18T03:30:50.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.574+0000 7f79c57fa700 1 -- 172.21.15.40:0/130209875 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f79c001dcc0 con 0x7f79c8100590 2023-07-18T03:30:50.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.574+0000 7f79c57fa700 1 --2- 172.21.15.40:0/130209875 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f79b00a1220 0x7f79b00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.574+0000 7f79c57fa700 1 -- 172.21.15.40:0/130209875 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f79c00ced10 con 0x7f79c8100590 2023-07-18T03:30:50.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.574+0000 7f79c7fff700 1 --2- 172.21.15.40:0/130209875 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f79b00a1220 0x7f79b00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.575+0000 7f79c7fff700 1 --2- 172.21.15.40:0/130209875 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f79b00a1220 0x7f79b00a36d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f79b8005fd0 tx=0x7f79b8005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:50.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.578+0000 7f79c57fa700 1 -- 172.21.15.40:0/130209875 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f79c0098370 con 0x7f79c8100590 2023-07-18T03:30:50.765 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:50 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1288257816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:50.765 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:50 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3378275731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:50.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.781+0000 7f79ce332700 1 -- 172.21.15.40:0/130209875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79c810a3e0 con 0x7f79c8100590 2023-07-18T03:30:50.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.782+0000 7f79c57fa700 1 -- 172.21.15.40:0/130209875 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22599 (secure 0 0 0) 0x7f79c009b990 con 0x7f79c8100590 2023-07-18T03:30:50.779 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:30:50.779 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":83,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:30:33.646627+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:30:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.784+0000 7f79ce332700 1 -- 172.21.15.40:0/130209875 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f79b00a1220 msgr2=0x7f79b00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.784+0000 7f79ce332700 1 --2- 172.21.15.40:0/130209875 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f79b00a1220 0x7f79b00a36d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f79b8005fd0 tx=0x7f79b8005f00 comp rx=0 tx=0).stop 2023-07-18T03:30:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.784+0000 7f79ce332700 1 -- 172.21.15.40:0/130209875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c8100590 msgr2=0x7f79c819efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.784+0000 7f79ce332700 1 --2- 172.21.15.40:0/130209875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c8100590 0x7f79c819efa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f79c000b830 tx=0x7f79c0004c80 comp rx=0 tx=0).stop 2023-07-18T03:30:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.784+0000 7f79ce332700 1 -- 172.21.15.40:0/130209875 shutdown_connections 2023-07-18T03:30:50.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.784+0000 7f79ce332700 1 --2- 172.21.15.40:0/130209875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c8100590 0x7f79c819efa0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.784+0000 7f79ce332700 1 --2- 172.21.15.40:0/130209875 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f79b00a1220 0x7f79b00a36d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.784+0000 7f79ce332700 1 --2- 172.21.15.40:0/130209875 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f79c80fea10 0x7f79c81979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.784+0000 7f79ce332700 1 --2- 172.21.15.40:0/130209875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79c80ffc00 0x7f79c8197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.784+0000 7f79ce332700 1 -- 172.21.15.40:0/130209875 >> 172.21.15.40:0/130209875 conn(0x7f79c80fa010 msgr2=0x7f79c80fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.784+0000 7f79ce332700 1 -- 172.21.15.40:0/130209875 shutdown_connections 2023-07-18T03:30:50.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.784+0000 7f79ce332700 1 -- 172.21.15.40:0/130209875 wait complete. 2023-07-18T03:30:50.794 DEBUG:tasks.fwd_scrub:Yielding 2023-07-18T03:30:50.794 INFO:teuthology.run_tasks:Running task background_exec... 2023-07-18T03:30:50.804 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi040.front.sepia.ceph.com 2023-07-18T03:30:50.804 DEBUG:teuthology.orchestra.run.smithi040:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-07-18T03:30:50.805 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-07-18T03:30:50.805 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:30:50.805 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:30:50.807 INFO:teuthology.run_tasks:Running task workunit... 2023-07-18T03:30:50.827 INFO:tasks.workunit:Pulling workunits from ref 81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:30:50.828 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-07-18T03:30:50.828 INFO:tasks.workunit:timeout=3h 2023-07-18T03:30:50.829 INFO:tasks.workunit:cleanup=True 2023-07-18T03:30:50.829 DEBUG:teuthology.orchestra.run.smithi040:> stat -- /home/ubuntu/cephtest/mnt.0 2023-07-18T03:30:50.848 INFO:teuthology.orchestra.run.smithi040.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-07-18T03:30:50.849 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-07-18T03:30:50.849 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2023-07-18T03:30:50.849 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-18T03:30:50.850 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:cephfs_t:s0 2023-07-18T03:30:50.850 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-07-18 03:29:33.854824296 +0000 2023-07-18T03:30:50.850 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-07-18 03:29:33.854824296 +0000 2023-07-18T03:30:50.850 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-07-18 03:30:43.868468861 +0000 2023-07-18T03:30:50.851 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2023-07-18 03:29:33.854824296 +0000 2023-07-18T03:30:50.851 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-07-18T03:30:50.851 DEBUG:teuthology.orchestra.run.smithi040:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-07-18T03:30:50.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.951+0000 7f89653fc700 1 -- 172.21.15.40:0/905498340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89600ffae0 msgr2=0x7f89600ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.951+0000 7f89653fc700 1 --2- 172.21.15.40:0/905498340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89600ffae0 0x7f89600ffef0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8950009990 tx=0x7f8950009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.952+0000 7f89653fc700 1 -- 172.21.15.40:0/905498340 shutdown_connections 2023-07-18T03:30:50.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.952+0000 7f89653fc700 1 --2- 172.21.15.40:0/905498340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8960101660 0x7f8960107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.952+0000 7f89653fc700 1 --2- 172.21.15.40:0/905498340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89600ffae0 0x7f89600ffef0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.952+0000 7f89653fc700 1 --2- 172.21.15.40:0/905498340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8960100cd0 0x7f8960101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.952+0000 7f89653fc700 1 -- 172.21.15.40:0/905498340 >> 172.21.15.40:0/905498340 conn(0x7f89600fb120 msgr2=0x7f89600fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:50.950 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-18T03:30:50.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.952+0000 7f89653fc700 1 -- 172.21.15.40:0/905498340 shutdown_connections 2023-07-18T03:30:50.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.952+0000 7f89653fc700 1 -- 172.21.15.40:0/905498340 wait complete. 2023-07-18T03:30:50.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f89653fc700 1 Processor -- start 2023-07-18T03:30:50.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f89653fc700 1 -- start start 2023-07-18T03:30:50.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f89653fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89600ffae0 0x7f89600742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f89653fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960100cd0 0x7f8960072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f89653fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8960101660 0x7f8960072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f89653fc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89600735e0 con 0x7f8960100cd0 2023-07-18T03:30:50.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f89653fc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8960073750 con 0x7f89600ffae0 2023-07-18T03:30:50.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f89653fc700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896019f880 con 0x7f8960101660 2023-07-18T03:30:50.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f895e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960100cd0 0x7f8960072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f895e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960100cd0 0x7f8960072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57880/0 (socket says 172.21.15.40:57880) 2023-07-18T03:30:50.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f895e7fc700 1 -- 172.21.15.40:0/2549411892 learned_addr learned my addr 172.21.15.40:0/2549411892 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:50.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f895effd700 1 --2- 172.21.15.40:0/2549411892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89600ffae0 0x7f89600742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f895f7fe700 1 --2- 172.21.15.40:0/2549411892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8960101660 0x7f8960072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f895e7fc700 1 -- 172.21.15.40:0/2549411892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8960101660 msgr2=0x7f8960072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f895e7fc700 1 --2- 172.21.15.40:0/2549411892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8960101660 0x7f8960072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f895e7fc700 1 -- 172.21.15.40:0/2549411892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89600ffae0 msgr2=0x7f89600742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f895e7fc700 1 --2- 172.21.15.40:0/2549411892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89600ffae0 0x7f89600742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f895e7fc700 1 -- 172.21.15.40:0/2549411892 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89500095f0 con 0x7f8960100cd0 2023-07-18T03:30:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.953+0000 7f895f7fe700 1 --2- 172.21.15.40:0/2549411892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8960101660 0x7f8960072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.954+0000 7f895e7fc700 1 --2- 172.21.15.40:0/2549411892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960100cd0 0x7f8960072900 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f895400d980 tx=0x7f895400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.954+0000 7f8947fff700 1 -- 172.21.15.40:0/2549411892 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954004220 con 0x7f8960100cd0 2023-07-18T03:30:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.954+0000 7f89653fc700 1 -- 172.21.15.40:0/2549411892 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f896019fb60 con 0x7f8960100cd0 2023-07-18T03:30:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.954+0000 7f8947fff700 1 -- 172.21.15.40:0/2549411892 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8954007640 con 0x7f8960100cd0 2023-07-18T03:30:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.954+0000 7f8947fff700 1 -- 172.21.15.40:0/2549411892 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954005610 con 0x7f8960100cd0 2023-07-18T03:30:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.954+0000 7f89653fc700 1 -- 172.21.15.40:0/2549411892 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89601a0070 con 0x7f8960100cd0 2023-07-18T03:30:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.956+0000 7f8947fff700 1 -- 172.21.15.40:0/2549411892 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8954004750 con 0x7f8960100cd0 2023-07-18T03:30:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.957+0000 7f8947fff700 1 --2- 172.21.15.40:0/2549411892 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89480a11e0 0x7f89480a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.957+0000 7f895effd700 1 --2- 172.21.15.40:0/2549411892 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89480a11e0 0x7f89480a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.957+0000 7f8947fff700 1 -- 172.21.15.40:0/2549411892 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f89540cc9c0 con 0x7f8960100cd0 2023-07-18T03:30:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.958+0000 7f895effd700 1 --2- 172.21.15.40:0/2549411892 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89480a11e0 0x7f89480a3690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f895000b5c0 tx=0x7f8950005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.958+0000 7f8971582b80 1 Processor -- start 2023-07-18T03:30:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.958+0000 7f8971582b80 1 -- start start 2023-07-18T03:30:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.958+0000 7f8971582b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698550 0x560335698960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.958+0000 7f8971582b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335698ea0 0x560335699760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.958+0000 7f8971582b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335699cf0 0x56033569a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.958+0000 7f8971582b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033559dd30 con 0x560335699cf0 2023-07-18T03:30:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.958+0000 7f8971582b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603355f0ac0 con 0x560335698550 2023-07-18T03:30:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.958+0000 7f8971582b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603355e9f50 con 0x560335698ea0 2023-07-18T03:30:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.958+0000 7f895f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335699cf0 0x56033569a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.958+0000 7f895f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335699cf0 0x56033569a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57890/0 (socket says 172.21.15.40:57890) 2023-07-18T03:30:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.958+0000 7f895f7fe700 1 -- 172.21.15.40:0/1527825704 learned_addr learned my addr 172.21.15.40:0/1527825704 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.958+0000 7f895effd700 1 --2- 172.21.15.40:0/1527825704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698550 0x560335698960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.958+0000 7f895e7fc700 1 --2- 172.21.15.40:0/1527825704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335698ea0 0x560335699760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.958+0000 7f895f7fe700 1 -- 172.21.15.40:0/1527825704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335698ea0 msgr2=0x560335699760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.959+0000 7f895f7fe700 1 --2- 172.21.15.40:0/1527825704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335698ea0 0x560335699760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.959+0000 7f895f7fe700 1 -- 172.21.15.40:0/1527825704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698550 msgr2=0x560335698960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:50.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.959+0000 7f895f7fe700 1 --2- 172.21.15.40:0/1527825704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698550 0x560335698960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.959+0000 7f895f7fe700 1 -- 172.21.15.40:0/1527825704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560335417e60 con 0x560335699cf0 2023-07-18T03:30:50.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.959+0000 7f895effd700 1 --2- 172.21.15.40:0/1527825704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698550 0x560335698960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:50.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.959+0000 7f895f7fe700 1 --2- 172.21.15.40:0/1527825704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335699cf0 0x56033569a160 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8958009ca0 tx=0x7f8958007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f087c146ff28fc62 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:50.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.959+0000 7f89457fa700 1 -- 172.21.15.40:0/1527825704 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958014400 con 0x560335699cf0 2023-07-18T03:30:50.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.959+0000 7f89457fa700 1 -- 172.21.15.40:0/1527825704 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8958014a20 con 0x560335699cf0 2023-07-18T03:30:50.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.959+0000 7f89457fa700 1 -- 172.21.15.40:0/1527825704 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958013a20 con 0x560335699cf0 2023-07-18T03:30:50.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.960+0000 7f8971582b80 1 -- 172.21.15.40:0/1527825704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335699cf0 msgr2=0x56033569a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.960+0000 7f8971582b80 1 --2- 172.21.15.40:0/1527825704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335699cf0 0x56033569a160 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8958009ca0 tx=0x7f8958007500 comp rx=0 tx=0).stop 2023-07-18T03:30:50.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.960+0000 7f8971582b80 1 -- 172.21.15.40:0/1527825704 shutdown_connections 2023-07-18T03:30:50.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.960+0000 7f8971582b80 1 --2- 172.21.15.40:0/1527825704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335699cf0 0x56033569a160 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.960+0000 7f8971582b80 1 --2- 172.21.15.40:0/1527825704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698550 0x560335698960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.960+0000 7f8971582b80 1 --2- 172.21.15.40:0/1527825704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335698ea0 0x560335699760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.960+0000 7f8971582b80 1 -- 172.21.15.40:0/1527825704 >> 172.21.15.40:0/1527825704 conn(0x56033568de20 msgr2=0x560335693250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:50.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.960+0000 7f8971582b80 1 -- 172.21.15.40:0/1527825704 shutdown_connections 2023-07-18T03:30:50.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.960+0000 7f8971582b80 1 -- 172.21.15.40:0/1527825704 wait complete. 2023-07-18T03:30:50.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.960+0000 7f8971582b80 1 -- start start 2023-07-18T03:30:50.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.960+0000 7f8971582b80 1 Processor -- start 2023-07-18T03:30:50.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.960+0000 7f8971582b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698550 0x5603356a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.960+0000 7f8971582b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698ea0 0x5603356a2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.960+0000 7f8971582b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335699cf0 0x5603356a9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f8971582b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560335589b40 con 0x560335698550 2023-07-18T03:30:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f8971582b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560335589840 con 0x560335698ea0 2023-07-18T03:30:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f8971582b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56033546d9a0 con 0x560335699cf0 2023-07-18T03:30:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f8971582b80 10 client.0 _ll_register_callbacks cb 0x5603355877e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f895effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698550 0x5603356a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f895effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698550 0x5603356a21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57892/0 (socket says 172.21.15.40:57892) 2023-07-18T03:30:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f895effd700 1 -- 172.21.15.40:0/2136088324 learned_addr learned my addr 172.21.15.40:0/2136088324 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f895f7fe700 1 --2- 172.21.15.40:0/2136088324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335699cf0 0x5603356a9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f895e7fc700 1 --2- 172.21.15.40:0/2136088324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698ea0 0x5603356a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f895effd700 1 -- 172.21.15.40:0/2136088324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335699cf0 msgr2=0x5603356a9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f895effd700 1 --2- 172.21.15.40:0/2136088324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335699cf0 0x5603356a9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f895effd700 1 -- 172.21.15.40:0/2136088324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698ea0 msgr2=0x5603356a2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f895effd700 1 --2- 172.21.15.40:0/2136088324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698ea0 0x5603356a2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f895effd700 1 -- 172.21.15.40:0/2136088324 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8960108850 con 0x560335698550 2023-07-18T03:30:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f895e7fc700 1 --2- 172.21.15.40:0/2136088324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698ea0 0x5603356a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f895effd700 1 --2- 172.21.15.40:0/2136088324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698550 0x5603356a21f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8960100b30 tx=0x7f8950009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf4a389e469a177 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f8942ff5700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.961+0000 7f8942ff5700 1 -- 172.21.15.40:0/2136088324 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950003f00 con 0x560335698550 2023-07-18T03:30:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.962+0000 7f8971582b80 1 -- 172.21.15.40:0/2136088324 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560335543570 con 0x560335698550 2023-07-18T03:30:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.962+0000 7f8942ff5700 1 -- 172.21.15.40:0/2136088324 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8950004060 con 0x560335698550 2023-07-18T03:30:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.962+0000 7f8942ff5700 1 -- 172.21.15.40:0/2136088324 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8950004220 con 0x560335698550 2023-07-18T03:30:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.962+0000 7f8971582b80 10 client.15429 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.962+0000 7f8971582b80 1 -- 172.21.15.40:0/2136088324 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f8958009900 con 0x560335698550 2023-07-18T03:30:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.962+0000 7f8942ff5700 1 -- 172.21.15.40:0/2136088324 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f895003f020 con 0x560335698550 2023-07-18T03:30:50.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.962+0000 7f8942ff5700 1 -- 172.21.15.40:0/2136088324 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f895002f070 con 0x560335698550 2023-07-18T03:30:50.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.962+0000 7f8971582b80 10 client.15429 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:50.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.962+0000 7f8971582b80 10 client.15429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:30:50.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.962+0000 7f8971582b80 20 client.15429 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:50.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.962+0000 7f8971582b80 1 --2- 172.21.15.40:0/2136088324 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5603356ae010 0x5603356b0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.962+0000 7f8971582b80 4 client.15429 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:30:50.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.962+0000 7f8971582b80 1 -- 172.21.15.40:0/2136088324 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603355e2f30 con 0x5603356ae010 2023-07-18T03:30:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.962+0000 7f895e7fc700 1 --2- 172.21.15.40:0/2136088324 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5603356ae010 0x5603356b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.963+0000 7f8942ff5700 10 client.15429 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:30:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.963+0000 7f895e7fc700 1 --2- 172.21.15.40:0/2136088324 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5603356ae010 0x5603356b0420 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59f790e6b0b77aa8 server_cookie=737bdbbb43403567 in_seq=0 out_seq=0 2023-07-18T03:30:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.964+0000 7f8942ff5700 1 -- 172.21.15.40:0/2136088324 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5603355e2f30 con 0x5603356ae010 2023-07-18T03:30:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.964+0000 7f8942ff5700 10 client.15429 handle_command_reply: tid=0 2023-07-18T03:30:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.964+0000 7f8971582b80 1 client.15429 shutdown 2023-07-18T03:30:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.964+0000 7f8971582b80 1 -- 172.21.15.40:0/2136088324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698550 msgr2=0x5603356a21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.964+0000 7f8971582b80 1 --2- 172.21.15.40:0/2136088324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698550 0x5603356a21f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8960100b30 tx=0x7f8950009ca0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.964+0000 7f8971582b80 1 -- 172.21.15.40:0/2136088324 shutdown_connections 2023-07-18T03:30:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.964+0000 7f8971582b80 1 --2- 172.21.15.40:0/2136088324 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5603356ae010 0x5603356b0420 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.964+0000 7f8971582b80 1 --2- 172.21.15.40:0/2136088324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335699cf0 0x5603356a9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.964+0000 7f8971582b80 1 --2- 172.21.15.40:0/2136088324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698550 0x5603356a21f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.964+0000 7f8971582b80 1 --2- 172.21.15.40:0/2136088324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698ea0 0x5603356a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.964+0000 7f8942ff5700 0 client.15429 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:30:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.964+0000 7f8971582b80 1 -- 172.21.15.40:0/2136088324 >> 172.21.15.40:0/2136088324 conn(0x56033568de20 msgr2=0x560335692930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:50.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.964+0000 7f8971582b80 1 -- 172.21.15.40:0/2136088324 shutdown_connections 2023-07-18T03:30:50.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.964+0000 7f8971582b80 1 -- 172.21.15.40:0/2136088324 wait complete. 2023-07-18T03:30:50.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.964+0000 7f8971582b80 20 client.15429 trim_cache size 0 max 16384 2023-07-18T03:30:50.973 INFO:tasks.workunit.client.0.smithi040.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-07-18T03:30:50.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.981+0000 7f8971582b80 1 Processor -- start 2023-07-18T03:30:50.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.981+0000 7f8971582b80 1 -- start start 2023-07-18T03:30:50.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f8971582b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698550 0x560335692930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f8971582b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335698ea0 0x5603356a9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f8971582b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335699cf0 0x5603356a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f8971582b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603356a1dc0 con 0x560335698550 2023-07-18T03:30:50.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f8971582b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560335699270 con 0x560335699cf0 2023-07-18T03:30:50.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f8971582b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560335694db0 con 0x560335698ea0 2023-07-18T03:30:50.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f895effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698550 0x560335692930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f895effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698550 0x560335692930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57902/0 (socket says 172.21.15.40:57902) 2023-07-18T03:30:50.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f895effd700 1 -- 172.21.15.40:0/3682062624 learned_addr learned my addr 172.21.15.40:0/3682062624 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:50.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f895f7fe700 1 --2- 172.21.15.40:0/3682062624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335699cf0 0x5603356a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f895e7fc700 1 --2- 172.21.15.40:0/3682062624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335698ea0 0x5603356a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f895effd700 1 -- 172.21.15.40:0/3682062624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335698ea0 msgr2=0x5603356a9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f895effd700 1 --2- 172.21.15.40:0/3682062624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335698ea0 0x5603356a9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f895effd700 1 -- 172.21.15.40:0/3682062624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335699cf0 msgr2=0x5603356a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f895effd700 1 --2- 172.21.15.40:0/3682062624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335699cf0 0x5603356a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f895effd700 1 -- 172.21.15.40:0/3682062624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56033569a390 con 0x560335698550 2023-07-18T03:30:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f895effd700 1 --2- 172.21.15.40:0/3682062624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698550 0x560335692930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8950009990 tx=0x7f8950005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3c53ddf25c9077 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f8945ffb700 1 -- 172.21.15.40:0/3682062624 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950004100 con 0x560335698550 2023-07-18T03:30:50.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f8945ffb700 1 -- 172.21.15.40:0/3682062624 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8950028070 con 0x560335698550 2023-07-18T03:30:50.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.982+0000 7f8945ffb700 1 -- 172.21.15.40:0/3682062624 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950044be0 con 0x560335698550 2023-07-18T03:30:50.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.983+0000 7f8971582b80 1 -- 172.21.15.40:0/3682062624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698550 msgr2=0x560335692930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.983+0000 7f8971582b80 1 --2- 172.21.15.40:0/3682062624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698550 0x560335692930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8950009990 tx=0x7f8950005a50 comp rx=0 tx=0).stop 2023-07-18T03:30:50.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.983+0000 7f8971582b80 1 -- 172.21.15.40:0/3682062624 shutdown_connections 2023-07-18T03:30:50.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.983+0000 7f8971582b80 1 --2- 172.21.15.40:0/3682062624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335699cf0 0x5603356a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.983+0000 7f8971582b80 1 --2- 172.21.15.40:0/3682062624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698550 0x560335692930 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.983+0000 7f8971582b80 1 --2- 172.21.15.40:0/3682062624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335698ea0 0x5603356a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.984+0000 7f8971582b80 1 -- 172.21.15.40:0/3682062624 >> 172.21.15.40:0/3682062624 conn(0x56033568de20 msgr2=0x5603356a0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.984+0000 7f8971582b80 1 -- 172.21.15.40:0/3682062624 shutdown_connections 2023-07-18T03:30:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.984+0000 7f8971582b80 1 -- 172.21.15.40:0/3682062624 wait complete. 2023-07-18T03:30:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.984+0000 7f8971582b80 1 -- start start 2023-07-18T03:30:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.984+0000 7f8971582b80 1 Processor -- start 2023-07-18T03:30:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.984+0000 7f8971582b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698550 0x5603356a9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.984+0000 7f8971582b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698ea0 0x5603356a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.984+0000 7f8971582b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335699cf0 0x5603356a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.984+0000 7f8971582b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603356a6b00 con 0x560335698ea0 2023-07-18T03:30:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.984+0000 7f8971582b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560335701ec0 con 0x560335698550 2023-07-18T03:30:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.984+0000 7f8971582b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560335702000 con 0x560335699cf0 2023-07-18T03:30:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.984+0000 7f8971582b80 10 client.0 _ll_register_callbacks cb 0x5603355877e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f895e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698ea0 0x5603356a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f895e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698ea0 0x5603356a5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57908/0 (socket says 172.21.15.40:57908) 2023-07-18T03:30:50.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f895e7fc700 1 -- 172.21.15.40:0/606762100 learned_addr learned my addr 172.21.15.40:0/606762100 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:50.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f895effd700 1 --2- 172.21.15.40:0/606762100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698550 0x5603356a9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f895f7fe700 1 --2- 172.21.15.40:0/606762100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335699cf0 0x5603356a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f895e7fc700 1 -- 172.21.15.40:0/606762100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335699cf0 msgr2=0x5603356a64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:50.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f895e7fc700 1 --2- 172.21.15.40:0/606762100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335699cf0 0x5603356a64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f895e7fc700 1 -- 172.21.15.40:0/606762100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698550 msgr2=0x5603356a9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:50.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f895e7fc700 1 --2- 172.21.15.40:0/606762100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698550 0x5603356a9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f895e7fc700 1 -- 172.21.15.40:0/606762100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8958009900 con 0x560335698ea0 2023-07-18T03:30:50.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f895effd700 1 --2- 172.21.15.40:0/606762100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698550 0x5603356a9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:30:50.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f895e7fc700 1 --2- 172.21.15.40:0/606762100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698ea0 0x5603356a5fa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8954099950 tx=0x7f89540a2170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988f099a4b453e31 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:50.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f8942ff5700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:50.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f8942ff5700 1 -- 172.21.15.40:0/606762100 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89540980b0 con 0x560335698ea0 2023-07-18T03:30:50.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f8942ff5700 1 -- 172.21.15.40:0/606762100 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f89540b21d0 con 0x560335698ea0 2023-07-18T03:30:50.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f8971582b80 1 -- 172.21.15.40:0/606762100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8958014a20 con 0x560335698ea0 2023-07-18T03:30:50.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f8942ff5700 1 -- 172.21.15.40:0/606762100 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89540b2390 con 0x560335698ea0 2023-07-18T03:30:50.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f8971582b80 10 client.15435 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:50.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.985+0000 7f8971582b80 1 -- 172.21.15.40:0/606762100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5603356934f0 con 0x560335698ea0 2023-07-18T03:30:50.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.986+0000 7f8942ff5700 1 -- 172.21.15.40:0/606762100 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8958014a20 con 0x560335698ea0 2023-07-18T03:30:50.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.986+0000 7f8942ff5700 1 -- 172.21.15.40:0/606762100 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8954095a40 con 0x560335698ea0 2023-07-18T03:30:50.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.986+0000 7f8971582b80 10 client.15435 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:50.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.986+0000 7f8971582b80 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:30:50.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.986+0000 7f8971582b80 20 client.15435 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:50.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.986+0000 7f8971582b80 1 --2- 172.21.15.40:0/606762100 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5603357037c0 0x560335705bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:50.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.986+0000 7f8971582b80 4 client.15435 mds_command: new command op to 24527 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:30:50.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.986+0000 7f8971582b80 1 -- 172.21.15.40:0/606762100 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5603355a50d0 con 0x5603357037c0 2023-07-18T03:30:50.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.986+0000 7f895effd700 1 --2- 172.21.15.40:0/606762100 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5603357037c0 0x560335705bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:50.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.986+0000 7f895effd700 1 --2- 172.21.15.40:0/606762100 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5603357037c0 0x560335705bd0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dc89c0c03d908bc server_cookie=c7b587dd1f02ba39 in_seq=0 out_seq=0 2023-07-18T03:30:50.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.986+0000 7f8942ff5700 10 client.15435 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:30:50.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.988+0000 7f8942ff5700 1 -- 172.21.15.40:0/606762100 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5603355a50d0 con 0x5603357037c0 2023-07-18T03:30:50.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.988+0000 7f8942ff5700 10 client.15435 handle_command_reply: tid=0 2023-07-18T03:30:50.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.988+0000 7f8971582b80 1 client.15435 shutdown 2023-07-18T03:30:50.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.988+0000 7f8971582b80 1 -- 172.21.15.40:0/606762100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698ea0 msgr2=0x5603356a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.988+0000 7f8971582b80 1 --2- 172.21.15.40:0/606762100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698ea0 0x5603356a5fa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8954099950 tx=0x7f89540a2170 comp rx=0 tx=0).stop 2023-07-18T03:30:50.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.988+0000 7f8971582b80 1 -- 172.21.15.40:0/606762100 shutdown_connections 2023-07-18T03:30:50.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.988+0000 7f8971582b80 1 --2- 172.21.15.40:0/606762100 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5603357037c0 0x560335705bd0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.988+0000 7f8971582b80 1 --2- 172.21.15.40:0/606762100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560335699cf0 0x5603356a64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.988+0000 7f8971582b80 1 --2- 172.21.15.40:0/606762100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560335698550 0x5603356a9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.988+0000 7f8971582b80 1 --2- 172.21.15.40:0/606762100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560335698ea0 0x5603356a5fa0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.988+0000 7f8942ff5700 0 client.15435 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:30:50.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.988+0000 7f8971582b80 1 -- 172.21.15.40:0/606762100 >> 172.21.15.40:0/606762100 conn(0x56033568de20 msgr2=0x5603356a00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:50.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.988+0000 7f8971582b80 1 -- 172.21.15.40:0/606762100 shutdown_connections 2023-07-18T03:30:50.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.989+0000 7f8971582b80 1 -- 172.21.15.40:0/606762100 wait complete. 2023-07-18T03:30:50.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.989+0000 7f8971582b80 20 client.15435 trim_cache size 0 max 16384 2023-07-18T03:30:50.993 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:30:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: "return_code": 0, 2023-07-18T03:30:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_tag": "de1afeab-0f2c-4cf8-812c-cc777f575e97", 2023-07-18T03:30:50.994 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": "asynchronous" 2023-07-18T03:30:50.994 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:30:50.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.989+0000 7f89653fc700 1 -- 172.21.15.40:0/2549411892 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89480a11e0 msgr2=0x7f89480a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.989+0000 7f89653fc700 1 --2- 172.21.15.40:0/2549411892 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89480a11e0 0x7f89480a3690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f895000b5c0 tx=0x7f8950005960 comp rx=0 tx=0).stop 2023-07-18T03:30:50.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.989+0000 7f89653fc700 1 -- 172.21.15.40:0/2549411892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960100cd0 msgr2=0x7f8960072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:50.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.989+0000 7f89653fc700 1 --2- 172.21.15.40:0/2549411892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960100cd0 0x7f8960072900 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f895400d980 tx=0x7f895400dc90 comp rx=0 tx=0).stop 2023-07-18T03:30:50.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.989+0000 7f89653fc700 1 -- 172.21.15.40:0/2549411892 shutdown_connections 2023-07-18T03:30:50.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.989+0000 7f89653fc700 1 --2- 172.21.15.40:0/2549411892 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89480a11e0 0x7f89480a3690 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.989+0000 7f89653fc700 1 --2- 172.21.15.40:0/2549411892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8960101660 0x7f8960072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.989+0000 7f89653fc700 1 --2- 172.21.15.40:0/2549411892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89600ffae0 0x7f89600742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.989+0000 7f89653fc700 1 --2- 172.21.15.40:0/2549411892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8960100cd0 0x7f8960072900 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:50.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.989+0000 7f89653fc700 1 -- 172.21.15.40:0/2549411892 >> 172.21.15.40:0/2549411892 conn(0x7f89600fb120 msgr2=0x7f89600fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:50.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.989+0000 7f89653fc700 1 -- 172.21.15.40:0/2549411892 shutdown_connections 2023-07-18T03:30:50.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:50.989+0000 7f89653fc700 1 -- 172.21.15.40:0/2549411892 wait complete. 2023-07-18T03:30:50.998 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:30:51.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:50 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1288257816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:51.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:50 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3378275731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:51.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:50 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1288257816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:30:51.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:50 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3378275731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:30:51.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.170+0000 7fc087bb4700 1 -- 172.21.15.40:0/2567318000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0801057a0 msgr2=0x7fc080107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:51.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.170+0000 7fc087bb4700 1 --2- 172.21.15.40:0/2567318000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0801057a0 0x7fc080107c00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc07800b0a0 tx=0x7fc07800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.170+0000 7fc087bb4700 1 -- 172.21.15.40:0/2567318000 shutdown_connections 2023-07-18T03:30:51.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.170+0000 7fc087bb4700 1 --2- 172.21.15.40:0/2567318000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0801057a0 0x7fc080107c00 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.170+0000 7fc087bb4700 1 --2- 172.21.15.40:0/2567318000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc080100560 0x7fc080102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.170+0000 7fc087bb4700 1 --2- 172.21.15.40:0/2567318000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc080102e80 0x7fc080105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.170+0000 7fc087bb4700 1 -- 172.21.15.40:0/2567318000 >> 172.21.15.40:0/2567318000 conn(0x7fc0800fa000 msgr2=0x7fc0800fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:51.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.170+0000 7fc087bb4700 1 -- 172.21.15.40:0/2567318000 shutdown_connections 2023-07-18T03:30:51.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.170+0000 7fc087bb4700 1 -- 172.21.15.40:0/2567318000 wait complete. 2023-07-18T03:30:51.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.171+0000 7fc087bb4700 1 Processor -- start 2023-07-18T03:30:51.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.171+0000 7fc087bb4700 1 -- start start 2023-07-18T03:30:51.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.171+0000 7fc087bb4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc080100560 0x7fc080195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.171+0000 7fc087bb4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc080102e80 0x7fc080195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.171+0000 7fc087bb4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0801057a0 0x7fc08019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.171+0000 7fc087bb4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08019d490 con 0x7fc080100560 2023-07-18T03:30:51.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.171+0000 7fc087bb4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08019d5d0 con 0x7fc080102e80 2023-07-18T03:30:51.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.171+0000 7fc087bb4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08019d760 con 0x7fc0801057a0 2023-07-18T03:30:51.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.171+0000 7fc085950700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc080100560 0x7fc080195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:51.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.171+0000 7fc085950700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc080100560 0x7fc080195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57920/0 (socket says 172.21.15.40:57920) 2023-07-18T03:30:51.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.171+0000 7fc085950700 1 -- 172.21.15.40:0/2744207213 learned_addr learned my addr 172.21.15.40:0/2744207213 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:51.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.171+0000 7fc08514f700 1 --2- 172.21.15.40:0/2744207213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc080102e80 0x7fc080195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:51.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.171+0000 7fc086151700 1 --2- 172.21.15.40:0/2744207213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0801057a0 0x7fc08019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:51.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.172+0000 7fc085950700 1 -- 172.21.15.40:0/2744207213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0801057a0 msgr2=0x7fc08019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:51.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.172+0000 7fc085950700 1 --2- 172.21.15.40:0/2744207213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0801057a0 0x7fc08019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.172+0000 7fc085950700 1 -- 172.21.15.40:0/2744207213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc080102e80 msgr2=0x7fc080195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:51.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.172+0000 7fc085950700 1 --2- 172.21.15.40:0/2744207213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc080102e80 0x7fc080195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.172+0000 7fc085950700 1 -- 172.21.15.40:0/2744207213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078009d00 con 0x7fc080100560 2023-07-18T03:30:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.172+0000 7fc085950700 1 --2- 172.21.15.40:0/2744207213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc080100560 0x7fc080195910 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00e9a0 tx=0x7fc07c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.172+0000 7fc072ffd700 1 -- 172.21.15.40:0/2744207213 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c009790 con 0x7fc080100560 2023-07-18T03:30:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.172+0000 7fc072ffd700 1 -- 172.21.15.40:0/2744207213 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07c010430 con 0x7fc080100560 2023-07-18T03:30:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.172+0000 7fc072ffd700 1 -- 172.21.15.40:0/2744207213 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c018680 con 0x7fc080100560 2023-07-18T03:30:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.172+0000 7fc087bb4700 1 -- 172.21.15.40:0/2744207213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08019da40 con 0x7fc080100560 2023-07-18T03:30:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.172+0000 7fc087bb4700 1 -- 172.21.15.40:0/2744207213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08019df50 con 0x7fc080100560 2023-07-18T03:30:51.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.174+0000 7fc072ffd700 1 -- 172.21.15.40:0/2744207213 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc07c0105a0 con 0x7fc080100560 2023-07-18T03:30:51.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.175+0000 7fc072ffd700 1 --2- 172.21.15.40:0/2744207213 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc06c0a1260 0x7fc06c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.175+0000 7fc072ffd700 1 -- 172.21.15.40:0/2744207213 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc07c014070 con 0x7fc080100560 2023-07-18T03:30:51.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.175+0000 7fc08514f700 1 --2- 172.21.15.40:0/2744207213 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc06c0a1260 0x7fc06c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:51.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.175+0000 7fc08514f700 1 --2- 172.21.15.40:0/2744207213 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc06c0a1260 0x7fc06c0a3710 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc074009c60 tx=0x7fc074009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:51.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.175+0000 7fc093d3ab80 1 Processor -- start 2023-07-18T03:30:51.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc093d3ab80 1 -- start start 2023-07-18T03:30:51.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc093d3ab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc093d3ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224c9230 0x55ce224c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc093d3ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224ca080 0x55ce224ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc093d3ab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce223c9cb0 con 0x55ce224c88e0 2023-07-18T03:30:51.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc093d3ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce2241ca40 con 0x55ce224c9230 2023-07-18T03:30:51.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc093d3ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce22415ed0 con 0x55ce224ca080 2023-07-18T03:30:51.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc085950700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:51.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc085950700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57928/0 (socket says 172.21.15.40:57928) 2023-07-18T03:30:51.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc085950700 1 -- 172.21.15.40:0/3411563233 learned_addr learned my addr 172.21.15.40:0/3411563233 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:51.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc08514f700 1 --2- 172.21.15.40:0/3411563233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224c9230 0x55ce224c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:51.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc085950700 1 -- 172.21.15.40:0/3411563233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224ca080 msgr2=0x55ce224ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:51.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc086151700 1 --2- 172.21.15.40:0/3411563233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224ca080 0x55ce224ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:51.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc085950700 1 --2- 172.21.15.40:0/3411563233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224ca080 0x55ce224ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc085950700 1 -- 172.21.15.40:0/3411563233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224c9230 msgr2=0x55ce224c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:51.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc085950700 1 --2- 172.21.15.40:0/3411563233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224c9230 0x55ce224c9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc085950700 1 -- 172.21.15.40:0/3411563233 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce22243de0 con 0x55ce224c88e0 2023-07-18T03:30:51.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc08514f700 1 --2- 172.21.15.40:0/3411563233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224c9230 0x55ce224c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:51.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.176+0000 7fc085950700 1 --2- 172.21.15.40:0/3411563233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224c8cf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0963f0 tx=0x7fc07c09df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8d6b8d0aeba13d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:51.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.177+0000 7fc06bfff700 1 -- 172.21.15.40:0/3411563233 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c0b1070 con 0x55ce224c88e0 2023-07-18T03:30:51.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.177+0000 7fc06bfff700 1 -- 172.21.15.40:0/3411563233 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07c098290 con 0x55ce224c88e0 2023-07-18T03:30:51.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.177+0000 7fc06bfff700 1 -- 172.21.15.40:0/3411563233 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c0a1eb0 con 0x55ce224c88e0 2023-07-18T03:30:51.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.177+0000 7fc093d3ab80 1 -- 172.21.15.40:0/3411563233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 msgr2=0x55ce224c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:51.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.177+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/3411563233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224c8cf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0963f0 tx=0x7fc07c09df20 comp rx=0 tx=0).stop 2023-07-18T03:30:51.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.177+0000 7fc093d3ab80 1 -- 172.21.15.40:0/3411563233 shutdown_connections 2023-07-18T03:30:51.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.177+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/3411563233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224ca080 0x55ce224ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.177+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/3411563233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224c8cf0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.177+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/3411563233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224c9230 0x55ce224c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.177+0000 7fc093d3ab80 1 -- 172.21.15.40:0/3411563233 >> 172.21.15.40:0/3411563233 conn(0x55ce224b9da0 msgr2=0x55ce224bf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:51.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.177+0000 7fc093d3ab80 1 -- 172.21.15.40:0/3411563233 shutdown_connections 2023-07-18T03:30:51.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.177+0000 7fc093d3ab80 1 -- 172.21.15.40:0/3411563233 wait complete. 2023-07-18T03:30:51.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.178+0000 7fc093d3ab80 1 -- start start 2023-07-18T03:30:51.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.178+0000 7fc093d3ab80 1 Processor -- start 2023-07-18T03:30:51.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.178+0000 7fc093d3ab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224d2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.178+0000 7fc093d3ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224c9230 0x55ce224d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc093d3ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224ca080 0x55ce224d9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc093d3ab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce223b5ac0 con 0x55ce224c88e0 2023-07-18T03:30:51.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc093d3ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce223b57c0 con 0x55ce224ca080 2023-07-18T03:30:51.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc093d3ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce22299920 con 0x55ce224c9230 2023-07-18T03:30:51.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc085950700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224d2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:51.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc093d3ab80 10 client.0 _ll_register_callbacks cb 0x55ce223b3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:51.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc085950700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224d2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57934/0 (socket says 172.21.15.40:57934) 2023-07-18T03:30:51.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc085950700 1 -- 172.21.15.40:0/1313357152 learned_addr learned my addr 172.21.15.40:0/1313357152 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:51.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc08514f700 1 --2- 172.21.15.40:0/1313357152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224c9230 0x55ce224d2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:51.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc085950700 1 -- 172.21.15.40:0/1313357152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224c9230 msgr2=0x55ce224d2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:51.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc085950700 1 --2- 172.21.15.40:0/1313357152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224c9230 0x55ce224d2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc085950700 1 -- 172.21.15.40:0/1313357152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224ca080 msgr2=0x55ce224d9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:51.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc085950700 1 --2- 172.21.15.40:0/1313357152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224ca080 0x55ce224d9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc085950700 1 -- 172.21.15.40:0/1313357152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08019df50 con 0x55ce224c88e0 2023-07-18T03:30:51.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc08514f700 1 --2- 172.21.15.40:0/1313357152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224c9230 0x55ce224d2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:30:51.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc085950700 1 --2- 172.21.15.40:0/1313357152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224d2640 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0a2500 tx=0x7fc07c0ca560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694196a37d0ce9f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:51.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc0697fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:51.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc0697fa700 1 -- 172.21.15.40:0/1313357152 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c0b1070 con 0x55ce224c88e0 2023-07-18T03:30:51.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc0697fa700 1 -- 172.21.15.40:0/1313357152 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc07c0b81c0 con 0x55ce224c88e0 2023-07-18T03:30:51.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc093d3ab80 1 -- 172.21.15.40:0/1313357152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce2236f4f0 con 0x55ce224c88e0 2023-07-18T03:30:51.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc0697fa700 1 -- 172.21.15.40:0/1313357152 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc07c0109c0 con 0x55ce224c88e0 2023-07-18T03:30:51.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc093d3ab80 10 client.15447 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:51.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.179+0000 7fc093d3ab80 1 -- 172.21.15.40:0/1313357152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ce2222cc60 con 0x55ce224c88e0 2023-07-18T03:30:51.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.180+0000 7fc0697fa700 1 -- 172.21.15.40:0/1313357152 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ce2236f4f0 con 0x55ce224c88e0 2023-07-18T03:30:51.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.180+0000 7fc0697fa700 1 -- 172.21.15.40:0/1313357152 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc07c098e50 con 0x55ce224c88e0 2023-07-18T03:30:51.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.180+0000 7fc093d3ab80 10 client.15447 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:51.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.180+0000 7fc093d3ab80 10 client.15447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:30:51.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.180+0000 7fc093d3ab80 20 client.15447 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:51.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.180+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/1313357152 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ce224ddf20 0x55ce224e0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.180+0000 7fc093d3ab80 4 client.15447 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:30:51.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.180+0000 7fc093d3ab80 1 -- 172.21.15.40:0/1313357152 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce2240eeb0 con 0x55ce224ddf20 2023-07-18T03:30:51.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.180+0000 7fc08514f700 1 --2- 172.21.15.40:0/1313357152 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ce224ddf20 0x55ce224e0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:51.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.181+0000 7fc08514f700 1 --2- 172.21.15.40:0/1313357152 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ce224ddf20 0x55ce224e0330 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac1e097a1f8ae901 server_cookie=7287802ef1c4d84f in_seq=0 out_seq=0 2023-07-18T03:30:51.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.181+0000 7fc0697fa700 10 client.15447 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:30:51.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.181+0000 7fc0697fa700 1 -- 172.21.15.40:0/1313357152 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ce2240eeb0 con 0x55ce224ddf20 2023-07-18T03:30:51.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.181+0000 7fc0697fa700 10 client.15447 handle_command_reply: tid=0 2023-07-18T03:30:51.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.181+0000 7fc093d3ab80 1 client.15447 shutdown 2023-07-18T03:30:51.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.182+0000 7fc093d3ab80 1 -- 172.21.15.40:0/1313357152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 msgr2=0x55ce224d2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:51.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.182+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/1313357152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224d2640 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0a2500 tx=0x7fc07c0ca560 comp rx=0 tx=0).stop 2023-07-18T03:30:51.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.182+0000 7fc093d3ab80 1 -- 172.21.15.40:0/1313357152 shutdown_connections 2023-07-18T03:30:51.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.182+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/1313357152 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ce224ddf20 0x55ce224e0330 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.182+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/1313357152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224ca080 0x55ce224d9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.182+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/1313357152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224d2640 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.182+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/1313357152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224c9230 0x55ce224d2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.182+0000 7fc0697fa700 0 client.15447 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:30:51.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.182+0000 7fc093d3ab80 1 -- 172.21.15.40:0/1313357152 >> 172.21.15.40:0/1313357152 conn(0x55ce224b9da0 msgr2=0x55ce224be8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:51.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.182+0000 7fc093d3ab80 1 -- 172.21.15.40:0/1313357152 shutdown_connections 2023-07-18T03:30:51.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.182+0000 7fc093d3ab80 1 -- 172.21.15.40:0/1313357152 wait complete. 2023-07-18T03:30:51.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.182+0000 7fc093d3ab80 20 client.15447 trim_cache size 0 max 16384 2023-07-18T03:30:51.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.198+0000 7fc093d3ab80 1 Processor -- start 2023-07-18T03:30:51.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc093d3ab80 1 -- start start 2023-07-18T03:30:51.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc093d3ab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc093d3ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224c9230 0x55ce224d9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc093d3ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224ca080 0x55ce224d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc093d3ab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce224d1af0 con 0x55ce224c88e0 2023-07-18T03:30:51.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc093d3ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce224b9970 con 0x55ce224ca080 2023-07-18T03:30:51.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc093d3ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce224cdb80 con 0x55ce224c9230 2023-07-18T03:30:51.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc085950700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:51.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc085950700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224d1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57942/0 (socket says 172.21.15.40:57942) 2023-07-18T03:30:51.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc085950700 1 -- 172.21.15.40:0/880363063 learned_addr learned my addr 172.21.15.40:0/880363063 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:51.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc085950700 1 -- 172.21.15.40:0/880363063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224c9230 msgr2=0x55ce224d9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:51.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc085950700 1 --2- 172.21.15.40:0/880363063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224c9230 0x55ce224d9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc085950700 1 -- 172.21.15.40:0/880363063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224ca080 msgr2=0x55ce224d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:51.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc085950700 1 --2- 172.21.15.40:0/880363063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224ca080 0x55ce224d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc085950700 1 -- 172.21.15.40:0/880363063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce2226a250 con 0x55ce224c88e0 2023-07-18T03:30:51.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc085950700 1 --2- 172.21.15.40:0/880363063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224d1260 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0ca530 tx=0x7fc07c09df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51b77a9683062c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:51.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc070dac700 1 -- 172.21.15.40:0/880363063 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c0b1020 con 0x55ce224c88e0 2023-07-18T03:30:51.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc070dac700 1 -- 172.21.15.40:0/880363063 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07c0109c0 con 0x55ce224c88e0 2023-07-18T03:30:51.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.199+0000 7fc070dac700 1 -- 172.21.15.40:0/880363063 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c0ca330 con 0x55ce224c88e0 2023-07-18T03:30:51.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.200+0000 7fc093d3ab80 1 -- 172.21.15.40:0/880363063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 msgr2=0x55ce224d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:51.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.200+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/880363063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224d1260 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0ca530 tx=0x7fc07c09df20 comp rx=0 tx=0).stop 2023-07-18T03:30:51.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.200+0000 7fc093d3ab80 1 -- 172.21.15.40:0/880363063 shutdown_connections 2023-07-18T03:30:51.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.200+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/880363063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224ca080 0x55ce224d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.200+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/880363063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c88e0 0x55ce224d1260 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.200+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/880363063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224c9230 0x55ce224d9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.200+0000 7fc093d3ab80 1 -- 172.21.15.40:0/880363063 >> 172.21.15.40:0/880363063 conn(0x55ce224b9da0 msgr2=0x55ce224e4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:51.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.200+0000 7fc093d3ab80 1 -- 172.21.15.40:0/880363063 shutdown_connections 2023-07-18T03:30:51.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.200+0000 7fc093d3ab80 1 -- 172.21.15.40:0/880363063 wait complete. 2023-07-18T03:30:51.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.200+0000 7fc093d3ab80 1 -- start start 2023-07-18T03:30:51.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.201+0000 7fc093d3ab80 1 Processor -- start 2023-07-18T03:30:51.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.201+0000 7fc093d3ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224c88e0 0x55ce224d9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.201+0000 7fc093d3ab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c9230 0x55ce224d9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.201+0000 7fc093d3ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224ca080 0x55ce224d7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.201+0000 7fc093d3ab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce224d8000 con 0x55ce224c9230 2023-07-18T03:30:51.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.201+0000 7fc093d3ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce224d8170 con 0x55ce224c88e0 2023-07-18T03:30:51.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.201+0000 7fc093d3ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce224d8330 con 0x55ce224ca080 2023-07-18T03:30:51.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.201+0000 7fc093d3ab80 10 client.0 _ll_register_callbacks cb 0x7fc07c099f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:30:51.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.201+0000 7fc08514f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c9230 0x55ce224d9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:51.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.201+0000 7fc08514f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c9230 0x55ce224d9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57950/0 (socket says 172.21.15.40:57950) 2023-07-18T03:30:51.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.201+0000 7fc08514f700 1 -- 172.21.15.40:0/2289804776 learned_addr learned my addr 172.21.15.40:0/2289804776 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:30:51.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.201+0000 7fc086151700 1 --2- 172.21.15.40:0/2289804776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224ca080 0x55ce224d7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:51.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.201+0000 7fc085950700 1 --2- 172.21.15.40:0/2289804776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224c88e0 0x55ce224d9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:51.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.201+0000 7fc08514f700 1 -- 172.21.15.40:0/2289804776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224ca080 msgr2=0x55ce224d7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:51.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc08514f700 1 --2- 172.21.15.40:0/2289804776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224ca080 0x55ce224d7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc08514f700 1 -- 172.21.15.40:0/2289804776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224c88e0 msgr2=0x55ce224d9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:30:51.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc08514f700 1 --2- 172.21.15.40:0/2289804776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224c88e0 0x55ce224d9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc08514f700 1 -- 172.21.15.40:0/2289804776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce2222cc60 con 0x55ce224c9230 2023-07-18T03:30:51.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc085950700 1 --2- 172.21.15.40:0/2289804776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224c88e0 0x55ce224d9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:30:51.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc08514f700 1 --2- 172.21.15.40:0/2289804776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c9230 0x55ce224d9960 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc074013010 tx=0x7fc074004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c74bfadd7f79a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:30:51.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc0697fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:30:51.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc0697fa700 1 -- 172.21.15.40:0/2289804776 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074021040 con 0x55ce224c9230 2023-07-18T03:30:51.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc0697fa700 1 -- 172.21.15.40:0/2289804776 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc074004340 con 0x55ce224c9230 2023-07-18T03:30:51.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc093d3ab80 1 -- 172.21.15.40:0/2289804776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc07c098290 con 0x55ce224c9230 2023-07-18T03:30:51.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc0697fa700 1 -- 172.21.15.40:0/2289804776 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc074004500 con 0x55ce224c9230 2023-07-18T03:30:51.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc093d3ab80 10 client.15453 fetch_fsmap learned FSMap version 27 2023-07-18T03:30:51.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc093d3ab80 1 -- 172.21.15.40:0/2289804776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ce222422b0 con 0x55ce224c9230 2023-07-18T03:30:51.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc0697fa700 1 -- 172.21.15.40:0/2289804776 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07c098290 con 0x55ce224c9230 2023-07-18T03:30:51.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc0697fa700 1 -- 172.21.15.40:0/2289804776 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc07401c930 con 0x55ce224c9230 2023-07-18T03:30:51.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc093d3ab80 10 client.15453 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:30:51.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc093d3ab80 10 client.15453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:30:51.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.202+0000 7fc093d3ab80 20 client.15453 populate_metadata read hostname 'smithi040' 2023-07-18T03:30:51.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.203+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/2289804776 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ce22533160 0x55ce22535570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:30:51.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.203+0000 7fc093d3ab80 4 client.15453 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:30:51.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.203+0000 7fc093d3ab80 1 -- 172.21.15.40:0/2289804776 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ce223d1050 con 0x55ce22533160 2023-07-18T03:30:51.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.203+0000 7fc085950700 1 --2- 172.21.15.40:0/2289804776 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ce22533160 0x55ce22535570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:30:51.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.203+0000 7fc085950700 1 --2- 172.21.15.40:0/2289804776 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ce22533160 0x55ce22535570 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a3ae86e79f98110 server_cookie=68e96479c1e2eb7a in_seq=0 out_seq=0 2023-07-18T03:30:51.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.203+0000 7fc0697fa700 10 client.15453 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:30:51.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.203+0000 7fc0697fa700 1 -- 172.21.15.40:0/2289804776 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ce223d1050 con 0x55ce22533160 2023-07-18T03:30:51.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.203+0000 7fc0697fa700 10 client.15453 handle_command_reply: tid=0 2023-07-18T03:30:51.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.203+0000 7fc093d3ab80 1 client.15453 shutdown 2023-07-18T03:30:51.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc093d3ab80 1 -- 172.21.15.40:0/2289804776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c9230 msgr2=0x55ce224d9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:51.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/2289804776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c9230 0x55ce224d9960 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc074013010 tx=0x7fc074004750 comp rx=0 tx=0).stop 2023-07-18T03:30:51.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc093d3ab80 1 -- 172.21.15.40:0/2289804776 shutdown_connections 2023-07-18T03:30:51.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/2289804776 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ce22533160 0x55ce22535570 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/2289804776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce224ca080 0x55ce224d7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/2289804776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ce224c88e0 0x55ce224d9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc093d3ab80 1 --2- 172.21.15.40:0/2289804776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ce224c9230 0x55ce224d9960 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc0697fa700 0 client.15453 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:30:51.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc093d3ab80 1 -- 172.21.15.40:0/2289804776 >> 172.21.15.40:0/2289804776 conn(0x55ce224b9da0 msgr2=0x55ce224d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:51.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc093d3ab80 1 -- 172.21.15.40:0/2289804776 shutdown_connections 2023-07-18T03:30:51.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc093d3ab80 1 -- 172.21.15.40:0/2289804776 wait complete. 2023-07-18T03:30:51.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc093d3ab80 20 client.15453 trim_cache size 0 max 16384 2023-07-18T03:30:51.210 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:30:51.210 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:30:51.210 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:30:51.210 INFO:teuthology.orchestra.run.smithi040.stdout: "de1afeab-0f2c-4cf8-812c-cc777f575e97": { 2023-07-18T03:30:51.210 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:30:51.210 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "de1afeab-0f2c-4cf8-812c-cc777f575e97", 2023-07-18T03:30:51.211 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:30:51.211 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:30:51.211 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:30:51.211 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:30:51.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc087bb4700 1 -- 172.21.15.40:0/2744207213 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc06c0a1260 msgr2=0x7fc06c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:51.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc087bb4700 1 --2- 172.21.15.40:0/2744207213 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc06c0a1260 0x7fc06c0a3710 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc074009c60 tx=0x7fc074009260 comp rx=0 tx=0).stop 2023-07-18T03:30:51.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc087bb4700 1 -- 172.21.15.40:0/2744207213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc080100560 msgr2=0x7fc080195910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:30:51.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc087bb4700 1 --2- 172.21.15.40:0/2744207213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc080100560 0x7fc080195910 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00e9a0 tx=0x7fc07c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc087bb4700 1 -- 172.21.15.40:0/2744207213 shutdown_connections 2023-07-18T03:30:51.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc087bb4700 1 --2- 172.21.15.40:0/2744207213 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc06c0a1260 0x7fc06c0a3710 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc087bb4700 1 --2- 172.21.15.40:0/2744207213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0801057a0 0x7fc08019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc087bb4700 1 --2- 172.21.15.40:0/2744207213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc080100560 0x7fc080195910 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc087bb4700 1 --2- 172.21.15.40:0/2744207213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc080102e80 0x7fc080195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:30:51.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc087bb4700 1 -- 172.21.15.40:0/2744207213 >> 172.21.15.40:0/2744207213 conn(0x7fc0800fa000 msgr2=0x7fc0801011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:30:51.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc087bb4700 1 -- 172.21.15.40:0/2744207213 shutdown_connections 2023-07-18T03:30:51.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:30:51.204+0000 7fc087bb4700 1 -- 172.21.15.40:0/2744207213 wait complete. 2023-07-18T03:30:51.214 INFO:tasks.cephfs.filesystem:scrub status for tag:de1afeab-0f2c-4cf8-812c-cc777f575e97 - {'path': '/', 'tag': 'de1afeab-0f2c-4cf8-812c-cc777f575e97', 'options': 'recursive,force'} 2023-07-18T03:30:52.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:51 smithi134 ceph-mon[98216]: pgmap v292: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-18T03:30:52.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:51 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/130209875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:52.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:51 smithi134 ceph-mon[98216]: scrub queued for path: / 2023-07-18T03:30:52.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:51 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:30:52.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:51 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:30:52.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:51 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:30:52.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:51 smithi040 ceph-mon[96439]: pgmap v292: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-18T03:30:52.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:51 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/130209875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:52.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:51 smithi040 ceph-mon[96439]: scrub queued for path: / 2023-07-18T03:30:52.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:51 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:30:52.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:51 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:30:52.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:51 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:30:52.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:51 smithi064 ceph-mon[98059]: pgmap v292: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-18T03:30:52.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:51 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/130209875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:30:52.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:51 smithi064 ceph-mon[98059]: scrub queued for path: / 2023-07-18T03:30:52.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:51 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:30:52.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:51 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:30:52.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:51 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:30:54.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:53 smithi134 ceph-mon[98216]: pgmap v293: 97 pgs: 97 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1023 B/s wr, 12 op/s 2023-07-18T03:30:54.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:53 smithi040 ceph-mon[96439]: pgmap v293: 97 pgs: 97 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1023 B/s wr, 12 op/s 2023-07-18T03:30:54.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:53 smithi064 ceph-mon[98059]: pgmap v293: 97 pgs: 97 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1023 B/s wr, 12 op/s 2023-07-18T03:30:56.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:55 smithi134 ceph-mon[98216]: pgmap v294: 97 pgs: 97 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 10 op/s 2023-07-18T03:30:56.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:55 smithi134 ceph-mon[98216]: scrub summary: idle 2023-07-18T03:30:56.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:55 smithi040 ceph-mon[96439]: pgmap v294: 97 pgs: 97 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 10 op/s 2023-07-18T03:30:56.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:55 smithi040 ceph-mon[96439]: scrub summary: idle 2023-07-18T03:30:56.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:55 smithi064 ceph-mon[98059]: pgmap v294: 97 pgs: 97 active+clean; 646 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 10 op/s 2023-07-18T03:30:56.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:55 smithi064 ceph-mon[98059]: scrub summary: idle 2023-07-18T03:30:58.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:57 smithi134 ceph-mon[98216]: pgmap v295: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-07-18T03:30:58.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:57 smithi040 ceph-mon[96439]: pgmap v295: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-07-18T03:30:58.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:57 smithi064 ceph-mon[98059]: pgmap v295: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-07-18T03:31:00.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:30:59 smithi134 ceph-mon[98216]: pgmap v296: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-07-18T03:31:00.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:30:59 smithi040 ceph-mon[96439]: pgmap v296: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-07-18T03:31:00.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:30:59 smithi064 ceph-mon[98059]: pgmap v296: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-07-18T03:31:02.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:01 smithi134 ceph-mon[98216]: pgmap v297: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-07-18T03:31:02.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:01 smithi040 ceph-mon[96439]: pgmap v297: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-07-18T03:31:02.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:01 smithi064 ceph-mon[98059]: pgmap v297: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-07-18T03:31:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.981+0000 7f25e849b700 1 -- 172.21.15.40:0/1567011063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e00fea20 msgr2=0x7f25e00fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.981+0000 7f25e849b700 1 --2- 172.21.15.40:0/1567011063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e00fea20 0x7f25e00fee30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f25d000b0a0 tx=0x7f25d000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:31:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.981+0000 7f25e849b700 1 -- 172.21.15.40:0/1567011063 shutdown_connections 2023-07-18T03:31:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.981+0000 7f25e849b700 1 --2- 172.21.15.40:0/1567011063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25e01005a0 0x7f25e0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.981+0000 7f25e849b700 1 --2- 172.21.15.40:0/1567011063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e00fea20 0x7f25e00fee30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.981+0000 7f25e849b700 1 --2- 172.21.15.40:0/1567011063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25e00ffc10 0x7f25e0100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.981+0000 7f25e849b700 1 -- 172.21.15.40:0/1567011063 >> 172.21.15.40:0/1567011063 conn(0x7f25e00fa020 msgr2=0x7f25e00fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.981+0000 7f25e849b700 1 -- 172.21.15.40:0/1567011063 shutdown_connections 2023-07-18T03:31:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.981+0000 7f25e849b700 1 -- 172.21.15.40:0/1567011063 wait complete. 2023-07-18T03:31:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.982+0000 7f25e849b700 1 Processor -- start 2023-07-18T03:31:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.982+0000 7f25e849b700 1 -- start start 2023-07-18T03:31:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.982+0000 7f25e849b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25e00fea20 0x7f25e0100ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.982+0000 7f25e849b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25e00ffc10 0x7f25e0101530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.982+0000 7f25e849b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e01005a0 0x7f25e0101a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.982+0000 7f25e849b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e019f9f0 con 0x7f25e01005a0 2023-07-18T03:31:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.982+0000 7f25e849b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e019fb30 con 0x7f25e00ffc10 2023-07-18T03:31:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.982+0000 7f25e849b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e019fc70 con 0x7f25e00fea20 2023-07-18T03:31:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.982+0000 7f25e6a38700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e01005a0 0x7f25e0101a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.982+0000 7f25e6a38700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e01005a0 0x7f25e0101a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57968/0 (socket says 172.21.15.40:57968) 2023-07-18T03:31:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.982+0000 7f25e6a38700 1 -- 172.21.15.40:0/1208651046 learned_addr learned my addr 172.21.15.40:0/1208651046 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.982+0000 7f25e6237700 1 --2- 172.21.15.40:0/1208651046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25e00fea20 0x7f25e0100ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.982+0000 7f25e5a36700 1 --2- 172.21.15.40:0/1208651046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25e00ffc10 0x7f25e0101530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.983+0000 7f25e6a38700 1 -- 172.21.15.40:0/1208651046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25e00fea20 msgr2=0x7f25e0100ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.983+0000 7f25e6a38700 1 --2- 172.21.15.40:0/1208651046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25e00fea20 0x7f25e0100ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.983+0000 7f25e6a38700 1 -- 172.21.15.40:0/1208651046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25e00ffc10 msgr2=0x7f25e0101530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.983+0000 7f25e6a38700 1 --2- 172.21.15.40:0/1208651046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25e00ffc10 0x7f25e0101530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.983+0000 7f25e6a38700 1 -- 172.21.15.40:0/1208651046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d0009d00 con 0x7f25e01005a0 2023-07-18T03:31:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.983+0000 7f25e6a38700 1 --2- 172.21.15.40:0/1208651046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e01005a0 0x7f25e0101a70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f25d800c900 tx=0x7f25d800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.983+0000 7f25df7fe700 1 -- 172.21.15.40:0/1208651046 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d80078d0 con 0x7f25e01005a0 2023-07-18T03:31:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.983+0000 7f25df7fe700 1 -- 172.21.15.40:0/1208651046 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25d800f450 con 0x7f25e01005a0 2023-07-18T03:31:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.983+0000 7f25e849b700 1 -- 172.21.15.40:0/1208651046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e019ff00 con 0x7f25e01005a0 2023-07-18T03:31:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.983+0000 7f25df7fe700 1 -- 172.21.15.40:0/1208651046 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8018660 con 0x7f25e01005a0 2023-07-18T03:31:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.983+0000 7f25e849b700 1 -- 172.21.15.40:0/1208651046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e01a0330 con 0x7f25e01005a0 2023-07-18T03:31:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.986+0000 7f25df7fe700 1 -- 172.21.15.40:0/1208651046 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f25d800f5c0 con 0x7f25e01005a0 2023-07-18T03:31:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.987+0000 7f25df7fe700 1 --2- 172.21.15.40:0/1208651046 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f25c80a1150 0x7f25c80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.987+0000 7f25df7fe700 1 -- 172.21.15.40:0/1208651046 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f25d80cc7c0 con 0x7f25e01005a0 2023-07-18T03:31:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.987+0000 7f25e6237700 1 --2- 172.21.15.40:0/1208651046 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f25c80a1150 0x7f25c80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.987+0000 7f25f4621b80 1 Processor -- start 2023-07-18T03:31:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.987+0000 7f25e6237700 1 --2- 172.21.15.40:0/1208651046 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f25c80a1150 0x7f25c80a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009fd0 tx=0x7f25d000b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.987+0000 7f25f4621b80 1 -- start start 2023-07-18T03:31:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25f4621b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e9995d26b0 0x55e999776300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25f4621b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e999776840 0x55e9997771a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25f4621b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9997776e0 0x55e999777b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25f4621b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99967bc50 con 0x55e9995d26b0 2023-07-18T03:31:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25f4621b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9996ce9e0 con 0x55e999776840 2023-07-18T03:31:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25f4621b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9996c7e70 con 0x55e9997776e0 2023-07-18T03:31:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25e6237700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e9995d26b0 0x55e999776300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25e6237700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e9995d26b0 0x55e999776300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57972/0 (socket says 172.21.15.40:57972) 2023-07-18T03:31:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25e6237700 1 -- 172.21.15.40:0/565296084 learned_addr learned my addr 172.21.15.40:0/565296084 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25e5a36700 1 --2- 172.21.15.40:0/565296084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e999776840 0x55e9997771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25e6a38700 1 --2- 172.21.15.40:0/565296084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9997776e0 0x55e999777b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25e6237700 1 -- 172.21.15.40:0/565296084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9997776e0 msgr2=0x55e999777b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25e6237700 1 --2- 172.21.15.40:0/565296084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9997776e0 0x55e999777b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25e6237700 1 -- 172.21.15.40:0/565296084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e999776840 msgr2=0x55e9997771a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25e6237700 1 --2- 172.21.15.40:0/565296084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e999776840 0x55e9997771a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25e6237700 1 -- 172.21.15.40:0/565296084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9994f5d80 con 0x55e9995d26b0 2023-07-18T03:31:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25e6a38700 1 --2- 172.21.15.40:0/565296084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9997776e0 0x55e999777b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:31:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25e5a36700 1 --2- 172.21.15.40:0/565296084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e999776840 0x55e9997771a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25e6237700 1 --2- 172.21.15.40:0/565296084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e9995d26b0 0x55e999776300 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f25d0015000 tx=0x7f25d001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cac6bf6854d98a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.988+0000 7f25dcdac700 1 -- 172.21.15.40:0/565296084 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d000e040 con 0x55e9995d26b0 2023-07-18T03:31:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.989+0000 7f25dcdac700 1 -- 172.21.15.40:0/565296084 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25d001cb00 con 0x55e9995d26b0 2023-07-18T03:31:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.989+0000 7f25dcdac700 1 -- 172.21.15.40:0/565296084 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0005760 con 0x55e9995d26b0 2023-07-18T03:31:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.989+0000 7f25f4621b80 1 -- 172.21.15.40:0/565296084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e9995d26b0 msgr2=0x55e999776300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.989+0000 7f25f4621b80 1 --2- 172.21.15.40:0/565296084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e9995d26b0 0x55e999776300 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f25d0015000 tx=0x7f25d001c000 comp rx=0 tx=0).stop 2023-07-18T03:31:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.989+0000 7f25f4621b80 1 -- 172.21.15.40:0/565296084 shutdown_connections 2023-07-18T03:31:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.989+0000 7f25f4621b80 1 --2- 172.21.15.40:0/565296084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9997776e0 0x55e999777b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.989+0000 7f25f4621b80 1 --2- 172.21.15.40:0/565296084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e9995d26b0 0x55e999776300 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.989+0000 7f25f4621b80 1 --2- 172.21.15.40:0/565296084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e999776840 0x55e9997771a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.989+0000 7f25f4621b80 1 -- 172.21.15.40:0/565296084 >> 172.21.15.40:0/565296084 conn(0x55e99976bbe0 msgr2=0x55e999770cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.989+0000 7f25f4621b80 1 -- 172.21.15.40:0/565296084 shutdown_connections 2023-07-18T03:31:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.989+0000 7f25f4621b80 1 -- 172.21.15.40:0/565296084 wait complete. 2023-07-18T03:31:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.990+0000 7f25f4621b80 1 -- start start 2023-07-18T03:31:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.990+0000 7f25f4621b80 1 Processor -- start 2023-07-18T03:31:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.990+0000 7f25f4621b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9995d26b0 0x55e99977f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.990+0000 7f25f4621b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e999776840 0x55e99977fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.990+0000 7f25f4621b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9997776e0 0x55e999786c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.990+0000 7f25f4621b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e999667a60 con 0x55e999776840 2023-07-18T03:31:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.990+0000 7f25f4621b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e999667760 con 0x55e9997776e0 2023-07-18T03:31:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.990+0000 7f25f4621b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99954b8c0 con 0x55e9995d26b0 2023-07-18T03:31:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.990+0000 7f25f4621b80 10 client.0 _ll_register_callbacks cb 0x55e999665700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.990+0000 7f25e5a36700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e999776840 0x55e99977fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.990+0000 7f25e5a36700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e999776840 0x55e99977fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57980/0 (socket says 172.21.15.40:57980) 2023-07-18T03:31:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.990+0000 7f25e6237700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9995d26b0 0x55e99977f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.990+0000 7f25e5a36700 1 -- 172.21.15.40:0/4146372706 learned_addr learned my addr 172.21.15.40:0/4146372706 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.990+0000 7f25e6a38700 1 --2- 172.21.15.40:0/4146372706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9997776e0 0x55e999786c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.991+0000 7f25e6237700 1 -- 172.21.15.40:0/4146372706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9997776e0 msgr2=0x55e999786c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.991+0000 7f25e6237700 1 --2- 172.21.15.40:0/4146372706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9997776e0 0x55e999786c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.991+0000 7f25e6237700 1 -- 172.21.15.40:0/4146372706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e999776840 msgr2=0x55e99977fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.991+0000 7f25e6237700 1 --2- 172.21.15.40:0/4146372706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e999776840 0x55e99977fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.991+0000 7f25e6237700 1 -- 172.21.15.40:0/4146372706 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e0107790 con 0x55e9995d26b0 2023-07-18T03:31:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.991+0000 7f25e5a36700 1 --2- 172.21.15.40:0/4146372706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e999776840 0x55e99977fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.991+0000 7f25e6237700 1 --2- 172.21.15.40:0/4146372706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9995d26b0 0x55e99977f6e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f25d000b0a0 tx=0x7f25d001c4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2760422b23afcd1e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.991+0000 7f25c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:31:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.991+0000 7f25c5ffb700 1 -- 172.21.15.40:0/4146372706 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d000e040 con 0x55e9995d26b0 2023-07-18T03:31:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.991+0000 7f25f4621b80 1 -- 172.21.15.40:0/4146372706 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e999621490 con 0x55e9995d26b0 2023-07-18T03:31:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.991+0000 7f25c5ffb700 1 -- 172.21.15.40:0/4146372706 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f25d0023040 con 0x55e9995d26b0 2023-07-18T03:31:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.991+0000 7f25c5ffb700 1 -- 172.21.15.40:0/4146372706 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25d00054c0 con 0x55e9995d26b0 2023-07-18T03:31:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.991+0000 7f25f4621b80 10 client.25156 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.991+0000 7f25f4621b80 1 -- 172.21.15.40:0/4146372706 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e9994dec00 con 0x55e9995d26b0 2023-07-18T03:31:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.992+0000 7f25c5ffb700 1 -- 172.21.15.40:0/4146372706 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e999621490 con 0x55e9995d26b0 2023-07-18T03:31:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.992+0000 7f25c5ffb700 1 -- 172.21.15.40:0/4146372706 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f25d0044bd0 con 0x55e9995d26b0 2023-07-18T03:31:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.992+0000 7f25f4621b80 10 client.25156 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.992+0000 7f25f4621b80 10 client.25156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:03.999 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.992+0000 7f25f4621b80 20 client.25156 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.992+0000 7f25f4621b80 1 --2- 172.21.15.40:0/4146372706 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55e99978af00 0x55e99978d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.992+0000 7f25f4621b80 4 client.25156 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.992+0000 7f25f4621b80 1 -- 172.21.15.40:0/4146372706 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9996c0e50 con 0x55e99978af00 2023-07-18T03:31:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.992+0000 7f25e5a36700 1 --2- 172.21.15.40:0/4146372706 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55e99978af00 0x55e99978d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.993+0000 7f25e5a36700 1 --2- 172.21.15.40:0/4146372706 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55e99978af00 0x55e99978d2e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21ccb20b0447be26 server_cookie=f7719ed0ab20ab5e in_seq=0 out_seq=0 2023-07-18T03:31:04.000 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.993+0000 7f25c5ffb700 10 client.25156 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.993+0000 7f25c5ffb700 1 -- 172.21.15.40:0/4146372706 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e9996c0e50 con 0x55e99978af00 2023-07-18T03:31:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.993+0000 7f25c5ffb700 10 client.25156 handle_command_reply: tid=0 2023-07-18T03:31:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.993+0000 7f25f4621b80 1 client.25156 shutdown 2023-07-18T03:31:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.993+0000 7f25f4621b80 1 -- 172.21.15.40:0/4146372706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9995d26b0 msgr2=0x55e99977f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.993+0000 7f25f4621b80 1 --2- 172.21.15.40:0/4146372706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9995d26b0 0x55e99977f6e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f25d000b0a0 tx=0x7f25d001c4c0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.993+0000 7f25f4621b80 1 -- 172.21.15.40:0/4146372706 shutdown_connections 2023-07-18T03:31:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.993+0000 7f25f4621b80 1 --2- 172.21.15.40:0/4146372706 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55e99978af00 0x55e99978d2e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.993+0000 7f25f4621b80 1 --2- 172.21.15.40:0/4146372706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9997776e0 0x55e999786c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.993+0000 7f25c5ffb700 0 client.25156 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.993+0000 7f25f4621b80 1 --2- 172.21.15.40:0/4146372706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9995d26b0 0x55e99977f6e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.993+0000 7f25f4621b80 1 --2- 172.21.15.40:0/4146372706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e999776840 0x55e99977fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.993+0000 7f25f4621b80 1 -- 172.21.15.40:0/4146372706 >> 172.21.15.40:0/4146372706 conn(0x55e99976bbe0 msgr2=0x55e9997704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.994+0000 7f25f4621b80 1 -- 172.21.15.40:0/4146372706 shutdown_connections 2023-07-18T03:31:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.994+0000 7f25f4621b80 1 -- 172.21.15.40:0/4146372706 wait complete. 2023-07-18T03:31:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:03.994+0000 7f25f4621b80 20 client.25156 trim_cache size 0 max 16384 2023-07-18T03:31:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.008+0000 7f25f4621b80 1 Processor -- start 2023-07-18T03:31:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.008+0000 7f25f4621b80 1 -- start start 2023-07-18T03:31:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25f4621b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e9995d26b0 0x55e99977a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25f4621b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e999776840 0x55e99977aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25f4621b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9997776e0 0x55e999786c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25f4621b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e999768e40 con 0x55e9995d26b0 2023-07-18T03:31:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25f4621b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99976ba80 con 0x55e999776840 2023-07-18T03:31:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25f4621b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9996c0e50 con 0x55e9997776e0 2023-07-18T03:31:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25e6237700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e9995d26b0 0x55e99977a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25e6237700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e9995d26b0 0x55e99977a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57986/0 (socket says 172.21.15.40:57986) 2023-07-18T03:31:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25e6237700 1 -- 172.21.15.40:0/714177333 learned_addr learned my addr 172.21.15.40:0/714177333 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25e6a38700 1 --2- 172.21.15.40:0/714177333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9997776e0 0x55e999786c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25e5a36700 1 --2- 172.21.15.40:0/714177333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e999776840 0x55e99977aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25e6237700 1 -- 172.21.15.40:0/714177333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9997776e0 msgr2=0x55e999786c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25e6237700 1 --2- 172.21.15.40:0/714177333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9997776e0 0x55e999786c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25e6237700 1 -- 172.21.15.40:0/714177333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e999776840 msgr2=0x55e99977aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25e6237700 1 --2- 172.21.15.40:0/714177333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e999776840 0x55e99977aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25e6237700 1 -- 172.21.15.40:0/714177333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e99951c1f0 con 0x55e9995d26b0 2023-07-18T03:31:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25e5a36700 1 --2- 172.21.15.40:0/714177333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e999776840 0x55e99977aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25e6a38700 1 --2- 172.21.15.40:0/714177333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9997776e0 0x55e999786c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25e6237700 1 --2- 172.21.15.40:0/714177333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e9995d26b0 0x55e99977a590 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f25d000d010 tx=0x7f25d000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ec5560ba046cf4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25dd5ad700 1 -- 172.21.15.40:0/714177333 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d00305f0 con 0x55e9995d26b0 2023-07-18T03:31:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25dd5ad700 1 -- 172.21.15.40:0/714177333 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25d0030cf0 con 0x55e9995d26b0 2023-07-18T03:31:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.009+0000 7f25dd5ad700 1 -- 172.21.15.40:0/714177333 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0044430 con 0x55e9995d26b0 2023-07-18T03:31:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.010+0000 7f25f4621b80 1 -- 172.21.15.40:0/714177333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e9995d26b0 msgr2=0x55e99977a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.010+0000 7f25f4621b80 1 --2- 172.21.15.40:0/714177333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e9995d26b0 0x55e99977a590 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f25d000d010 tx=0x7f25d000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.010+0000 7f25f4621b80 1 -- 172.21.15.40:0/714177333 shutdown_connections 2023-07-18T03:31:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.010+0000 7f25f4621b80 1 --2- 172.21.15.40:0/714177333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9997776e0 0x55e999786c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.010+0000 7f25f4621b80 1 --2- 172.21.15.40:0/714177333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e9995d26b0 0x55e99977a590 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.010+0000 7f25f4621b80 1 --2- 172.21.15.40:0/714177333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e999776840 0x55e99977aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.010+0000 7f25f4621b80 1 -- 172.21.15.40:0/714177333 >> 172.21.15.40:0/714177333 conn(0x55e99976bbe0 msgr2=0x55e999791c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.010+0000 7f25f4621b80 1 -- 172.21.15.40:0/714177333 shutdown_connections 2023-07-18T03:31:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.010+0000 7f25f4621b80 1 -- 172.21.15.40:0/714177333 wait complete. 2023-07-18T03:31:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.010+0000 7f25f4621b80 1 -- start start 2023-07-18T03:31:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.010+0000 7f25f4621b80 1 Processor -- start 2023-07-18T03:31:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25f4621b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9995d26b0 0x55e9997827c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25f4621b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e999776840 0x55e999782d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25f4621b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9997776e0 0x55e999785d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25f4621b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e999786260 con 0x55e999776840 2023-07-18T03:31:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25f4621b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9997863d0 con 0x55e9997776e0 2023-07-18T03:31:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25f4621b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e999786590 con 0x55e9995d26b0 2023-07-18T03:31:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25f4621b80 10 client.0 _ll_register_callbacks cb 0x7f25d0009230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25e5a36700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e999776840 0x55e999782d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25e5a36700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e999776840 0x55e999782d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57994/0 (socket says 172.21.15.40:57994) 2023-07-18T03:31:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25e5a36700 1 -- 172.21.15.40:0/4040373728 learned_addr learned my addr 172.21.15.40:0/4040373728 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25e6237700 1 --2- 172.21.15.40:0/4040373728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9995d26b0 0x55e9997827c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25e6a38700 1 --2- 172.21.15.40:0/4040373728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9997776e0 0x55e999785d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25e5a36700 1 -- 172.21.15.40:0/4040373728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9995d26b0 msgr2=0x55e9997827c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25e5a36700 1 --2- 172.21.15.40:0/4040373728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9995d26b0 0x55e9997827c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25e5a36700 1 -- 172.21.15.40:0/4040373728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9997776e0 msgr2=0x55e999785d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25e5a36700 1 --2- 172.21.15.40:0/4040373728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9997776e0 0x55e999785d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25e5a36700 1 -- 172.21.15.40:0/4040373728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9994dec00 con 0x55e999776840 2023-07-18T03:31:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25e5a36700 1 --2- 172.21.15.40:0/4040373728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e999776840 0x55e999782d00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f25d4000c00 tx=0x7f25d401eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b37e7acb9b845a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.011+0000 7f25c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.012+0000 7f25c5ffb700 1 -- 172.21.15.40:0/4040373728 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4017040 con 0x55e999776840 2023-07-18T03:31:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.012+0000 7f25c5ffb700 1 -- 172.21.15.40:0/4040373728 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f25d4012a40 con 0x55e999776840 2023-07-18T03:31:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.012+0000 7f25f4621b80 1 -- 172.21.15.40:0/4040373728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25d001cb00 con 0x55e999776840 2023-07-18T03:31:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.012+0000 7f25c5ffb700 1 -- 172.21.15.40:0/4040373728 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25d401e5e0 con 0x55e999776840 2023-07-18T03:31:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.012+0000 7f25f4621b80 10 client.15471 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.012+0000 7f25f4621b80 1 -- 172.21.15.40:0/4040373728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e9994f4250 con 0x55e999776840 2023-07-18T03:31:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.012+0000 7f25c5ffb700 1 -- 172.21.15.40:0/4040373728 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25d001cb00 con 0x55e999776840 2023-07-18T03:31:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.012+0000 7f25c5ffb700 1 -- 172.21.15.40:0/4040373728 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f25d401e040 con 0x55e999776840 2023-07-18T03:31:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.013+0000 7f25f4621b80 10 client.15471 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.013+0000 7f25f4621b80 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.013+0000 7f25f4621b80 20 client.15471 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.013+0000 7f25f4621b80 1 --2- 172.21.15.40:0/4040373728 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55e9997dfa20 0x55e9997e1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.013+0000 7f25f4621b80 4 client.15471 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:31:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.013+0000 7f25f4621b80 1 -- 172.21.15.40:0/4040373728 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e999682ff0 con 0x55e9997dfa20 2023-07-18T03:31:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.013+0000 7f25e6237700 1 --2- 172.21.15.40:0/4040373728 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55e9997dfa20 0x55e9997e1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.013+0000 7f25c5ffb700 10 client.15471 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.013+0000 7f25e6237700 1 --2- 172.21.15.40:0/4040373728 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55e9997dfa20 0x55e9997e1e00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7828dbdda4902f1 server_cookie=520f70e59f3daf5 in_seq=0 out_seq=0 2023-07-18T03:31:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.025+0000 7f25c5ffb700 1 -- 172.21.15.40:0/4040373728 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e999682ff0 con 0x55e9997dfa20 2023-07-18T03:31:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.025+0000 7f25c5ffb700 10 client.15471 handle_command_reply: tid=0 2023-07-18T03:31:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.026+0000 7f25f4621b80 1 client.15471 shutdown 2023-07-18T03:31:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.026+0000 7f25f4621b80 1 -- 172.21.15.40:0/4040373728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e999776840 msgr2=0x55e999782d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.026+0000 7f25f4621b80 1 --2- 172.21.15.40:0/4040373728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e999776840 0x55e999782d00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f25d4000c00 tx=0x7f25d401eee0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.026+0000 7f25f4621b80 1 -- 172.21.15.40:0/4040373728 shutdown_connections 2023-07-18T03:31:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.026+0000 7f25f4621b80 1 --2- 172.21.15.40:0/4040373728 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55e9997dfa20 0x55e9997e1e00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.026+0000 7f25f4621b80 1 --2- 172.21.15.40:0/4040373728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9997776e0 0x55e999785d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.026+0000 7f25f4621b80 1 --2- 172.21.15.40:0/4040373728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e9995d26b0 0x55e9997827c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.026+0000 7f25f4621b80 1 --2- 172.21.15.40:0/4040373728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55e999776840 0x55e999782d00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.026+0000 7f25c5ffb700 0 client.15471 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.026+0000 7f25f4621b80 1 -- 172.21.15.40:0/4040373728 >> 172.21.15.40:0/4040373728 conn(0x55e99976bbe0 msgr2=0x55e99977a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.026+0000 7f25f4621b80 1 -- 172.21.15.40:0/4040373728 shutdown_connections 2023-07-18T03:31:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.026+0000 7f25f4621b80 1 -- 172.21.15.40:0/4040373728 wait complete. 2023-07-18T03:31:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.026+0000 7f25f4621b80 20 client.15471 trim_cache size 0 max 16384 2023-07-18T03:31:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:31:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:31:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:31:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:31:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.026+0000 7f25e849b700 1 -- 172.21.15.40:0/1208651046 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f25c80a1150 msgr2=0x7f25c80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.026+0000 7f25e849b700 1 --2- 172.21.15.40:0/1208651046 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f25c80a1150 0x7f25c80a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009fd0 tx=0x7f25d000b7b0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.026+0000 7f25e849b700 1 -- 172.21.15.40:0/1208651046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e01005a0 msgr2=0x7f25e0101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.026+0000 7f25e849b700 1 --2- 172.21.15.40:0/1208651046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e01005a0 0x7f25e0101a70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f25d800c900 tx=0x7f25d800cc10 comp rx=0 tx=0).stop 2023-07-18T03:31:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.027+0000 7f25e849b700 1 -- 172.21.15.40:0/1208651046 shutdown_connections 2023-07-18T03:31:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.027+0000 7f25e849b700 1 --2- 172.21.15.40:0/1208651046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25e01005a0 0x7f25e0101a70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.027+0000 7f25e849b700 1 --2- 172.21.15.40:0/1208651046 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f25c80a1150 0x7f25c80a3600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.027+0000 7f25e849b700 1 --2- 172.21.15.40:0/1208651046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25e00fea20 0x7f25e0100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.027+0000 7f25e849b700 1 --2- 172.21.15.40:0/1208651046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25e00ffc10 0x7f25e0101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.027+0000 7f25e849b700 1 -- 172.21.15.40:0/1208651046 >> 172.21.15.40:0/1208651046 conn(0x7f25e00fa020 msgr2=0x7f25e0068500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.027+0000 7f25e849b700 1 -- 172.21.15.40:0/1208651046 shutdown_connections 2023-07-18T03:31:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:04.027+0000 7f25e849b700 1 -- 172.21.15.40:0/1208651046 wait complete. 2023-07-18T03:31:04.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:03 smithi134 ceph-mon[98216]: pgmap v298: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2023-07-18T03:31:04.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:03 smithi134 ceph-mon[98216]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-07-18T03:31:04.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:03 smithi040 ceph-mon[96439]: pgmap v298: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2023-07-18T03:31:04.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:03 smithi040 ceph-mon[96439]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-07-18T03:31:04.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:03 smithi064 ceph-mon[98059]: pgmap v298: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2023-07-18T03:31:04.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:03 smithi064 ceph-mon[98059]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-07-18T03:31:06.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:05 smithi134 ceph-mon[98216]: pgmap v299: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 9 op/s 2023-07-18T03:31:06.084 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:05 smithi040 ceph-mon[96439]: pgmap v299: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 9 op/s 2023-07-18T03:31:06.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:05 smithi064 ceph-mon[98059]: pgmap v299: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 9 op/s 2023-07-18T03:31:08.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:07 smithi134 ceph-mon[98216]: pgmap v300: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-07-18T03:31:08.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:07 smithi040 ceph-mon[96439]: pgmap v300: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-07-18T03:31:08.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:07 smithi064 ceph-mon[98059]: pgmap v300: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-07-18T03:31:10.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:09 smithi134 ceph-mon[98216]: pgmap v301: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2023-07-18T03:31:10.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:09 smithi040 ceph-mon[96439]: pgmap v301: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2023-07-18T03:31:10.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:09 smithi064 ceph-mon[98059]: pgmap v301: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2023-07-18T03:31:12.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:11 smithi134 ceph-mon[98216]: pgmap v302: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2023-07-18T03:31:12.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:11 smithi040 ceph-mon[96439]: pgmap v302: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2023-07-18T03:31:12.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:11 smithi064 ceph-mon[98059]: pgmap v302: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2023-07-18T03:31:14.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:13 smithi134 ceph-mon[98216]: pgmap v303: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.9 KiB/s wr, 16 op/s 2023-07-18T03:31:14.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:13 smithi040 ceph-mon[96439]: pgmap v303: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.9 KiB/s wr, 16 op/s 2023-07-18T03:31:14.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:13 smithi064 ceph-mon[98059]: pgmap v303: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.9 KiB/s wr, 16 op/s 2023-07-18T03:31:16.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:15 smithi134 ceph-mon[98216]: pgmap v304: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2023-07-18T03:31:16.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:15 smithi040 ceph-mon[96439]: pgmap v304: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2023-07-18T03:31:16.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:15 smithi064 ceph-mon[98059]: pgmap v304: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2023-07-18T03:31:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.141+0000 7fe387846700 1 -- 172.21.15.40:0/49944525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3800fea00 msgr2=0x7fe3800fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.141+0000 7fe387846700 1 --2- 172.21.15.40:0/49944525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3800fea00 0x7fe3800fee10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009990 tx=0x7fe37c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.141+0000 7fe387846700 1 -- 172.21.15.40:0/49944525 shutdown_connections 2023-07-18T03:31:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.141+0000 7fe387846700 1 --2- 172.21.15.40:0/49944525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe380100640 0x7fe380106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.141+0000 7fe387846700 1 --2- 172.21.15.40:0/49944525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3800fea00 0x7fe3800fee10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.141+0000 7fe387846700 1 --2- 172.21.15.40:0/49944525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3800ffc90 0x7fe380100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.141+0000 7fe387846700 1 -- 172.21.15.40:0/49944525 >> 172.21.15.40:0/49944525 conn(0x7fe3800fa000 msgr2=0x7fe3800fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.141+0000 7fe387846700 1 -- 172.21.15.40:0/49944525 shutdown_connections 2023-07-18T03:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.141+0000 7fe387846700 1 -- 172.21.15.40:0/49944525 wait complete. 2023-07-18T03:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe387846700 1 Processor -- start 2023-07-18T03:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe387846700 1 -- start start 2023-07-18T03:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe387846700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3800fea00 0x7fe380101070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe387846700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3800ffc90 0x7fe3801015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe387846700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe380100640 0x7fe380101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe387846700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38019f8f0 con 0x7fe380100640 2023-07-18T03:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe387846700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38019fa30 con 0x7fe3800ffc90 2023-07-18T03:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe387846700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38019fb70 con 0x7fe3800fea00 2023-07-18T03:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe385de3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe380100640 0x7fe380101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe385de3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe380100640 0x7fe380101af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58010/0 (socket says 172.21.15.40:58010) 2023-07-18T03:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe385de3700 1 -- 172.21.15.40:0/2322790705 learned_addr learned my addr 172.21.15.40:0/2322790705 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe384de1700 1 --2- 172.21.15.40:0/2322790705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3800ffc90 0x7fe3801015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe3855e2700 1 --2- 172.21.15.40:0/2322790705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3800fea00 0x7fe380101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe385de3700 1 -- 172.21.15.40:0/2322790705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3800fea00 msgr2=0x7fe380101070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe385de3700 1 --2- 172.21.15.40:0/2322790705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3800fea00 0x7fe380101070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe385de3700 1 -- 172.21.15.40:0/2322790705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3800ffc90 msgr2=0x7fe3801015b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe385de3700 1 --2- 172.21.15.40:0/2322790705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3800ffc90 0x7fe3801015b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.142+0000 7fe385de3700 1 -- 172.21.15.40:0/2322790705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c0095f0 con 0x7fe380100640 2023-07-18T03:31:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.143+0000 7fe385de3700 1 --2- 172.21.15.40:0/2322790705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe380100640 0x7fe380101af0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe37800ada0 tx=0x7fe37800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.143+0000 7fe3767fc700 1 -- 172.21.15.40:0/2322790705 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37800f070 con 0x7fe380100640 2023-07-18T03:31:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.143+0000 7fe387846700 1 -- 172.21.15.40:0/2322790705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38019fe00 con 0x7fe380100640 2023-07-18T03:31:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.143+0000 7fe3767fc700 1 -- 172.21.15.40:0/2322790705 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe378007b30 con 0x7fe380100640 2023-07-18T03:31:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.143+0000 7fe3767fc700 1 -- 172.21.15.40:0/2322790705 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378013a50 con 0x7fe380100640 2023-07-18T03:31:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.143+0000 7fe387846700 1 -- 172.21.15.40:0/2322790705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3801a0290 con 0x7fe380100640 2023-07-18T03:31:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.145+0000 7fe3767fc700 1 -- 172.21.15.40:0/2322790705 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe37800c8d0 con 0x7fe380100640 2023-07-18T03:31:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.146+0000 7fe3767fc700 1 --2- 172.21.15.40:0/2322790705 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe36c0a1150 0x7fe36c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.146+0000 7fe3855e2700 1 --2- 172.21.15.40:0/2322790705 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe36c0a1150 0x7fe36c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.146+0000 7fe3767fc700 1 -- 172.21.15.40:0/2322790705 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe3780cc020 con 0x7fe380100640 2023-07-18T03:31:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.147+0000 7fe3855e2700 1 --2- 172.21.15.40:0/2322790705 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe36c0a1150 0x7fe36c0a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe380102980 tx=0x7fe37c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.147+0000 7fe3939ccb80 1 Processor -- start 2023-07-18T03:31:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.147+0000 7fe3939ccb80 1 -- start start 2023-07-18T03:31:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.147+0000 7fe3939ccb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a6094b6b0 0x561a60aef300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.147+0000 7fe3939ccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a60aef840 0x561a60af01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.147+0000 7fe3939ccb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a60af06e0 0x561a60af0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.147+0000 7fe3939ccb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a609f4c50 con 0x561a6094b6b0 2023-07-18T03:31:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.147+0000 7fe3939ccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a60a479e0 con 0x561a60aef840 2023-07-18T03:31:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.147+0000 7fe3939ccb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a60a40e70 con 0x561a60af06e0 2023-07-18T03:31:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.148+0000 7fe3855e2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a6094b6b0 0x561a60aef300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.148+0000 7fe3855e2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a6094b6b0 0x561a60aef300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58014/0 (socket says 172.21.15.40:58014) 2023-07-18T03:31:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.148+0000 7fe3855e2700 1 -- 172.21.15.40:0/247344020 learned_addr learned my addr 172.21.15.40:0/247344020 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.148+0000 7fe384de1700 1 --2- 172.21.15.40:0/247344020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a60aef840 0x561a60af01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.148+0000 7fe385de3700 1 --2- 172.21.15.40:0/247344020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a60af06e0 0x561a60af0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.148+0000 7fe3855e2700 1 -- 172.21.15.40:0/247344020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a60af06e0 msgr2=0x561a60af0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.148+0000 7fe3855e2700 1 --2- 172.21.15.40:0/247344020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a60af06e0 0x561a60af0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.148+0000 7fe3855e2700 1 -- 172.21.15.40:0/247344020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a60aef840 msgr2=0x561a60af01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.148+0000 7fe3855e2700 1 --2- 172.21.15.40:0/247344020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a60aef840 0x561a60af01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.148+0000 7fe3855e2700 1 -- 172.21.15.40:0/247344020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a6086ed80 con 0x561a6094b6b0 2023-07-18T03:31:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.148+0000 7fe384de1700 1 --2- 172.21.15.40:0/247344020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a60aef840 0x561a60af01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.148+0000 7fe3855e2700 1 --2- 172.21.15.40:0/247344020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a6094b6b0 0x561a60aef300 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe37c015000 tx=0x7fe37c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3579ce24c9b33fd9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.148+0000 7fe36b7fe700 1 -- 172.21.15.40:0/247344020 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c02a070 con 0x561a6094b6b0 2023-07-18T03:31:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.148+0000 7fe36b7fe700 1 -- 172.21.15.40:0/247344020 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe37c01e040 con 0x561a6094b6b0 2023-07-18T03:31:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.148+0000 7fe36b7fe700 1 -- 172.21.15.40:0/247344020 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c023a40 con 0x561a6094b6b0 2023-07-18T03:31:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.149+0000 7fe3939ccb80 1 -- 172.21.15.40:0/247344020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a6094b6b0 msgr2=0x561a60aef300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.149+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/247344020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a6094b6b0 0x561a60aef300 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe37c015000 tx=0x7fe37c020000 comp rx=0 tx=0).stop 2023-07-18T03:31:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.149+0000 7fe3939ccb80 1 -- 172.21.15.40:0/247344020 shutdown_connections 2023-07-18T03:31:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.149+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/247344020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a60af06e0 0x561a60af0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.149+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/247344020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a6094b6b0 0x561a60aef300 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.149+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/247344020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a60aef840 0x561a60af01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.149+0000 7fe3939ccb80 1 -- 172.21.15.40:0/247344020 >> 172.21.15.40:0/247344020 conn(0x561a60ae4be0 msgr2=0x561a60ae9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.149+0000 7fe3939ccb80 1 -- 172.21.15.40:0/247344020 shutdown_connections 2023-07-18T03:31:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.149+0000 7fe3939ccb80 1 -- 172.21.15.40:0/247344020 wait complete. 2023-07-18T03:31:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.149+0000 7fe3939ccb80 1 -- start start 2023-07-18T03:31:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe3939ccb80 1 Processor -- start 2023-07-18T03:31:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe3939ccb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6094b6b0 0x561a60af87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe3939ccb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60aef840 0x561a60af8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe3939ccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a60af06e0 0x561a60affd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe3939ccb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a609e0a60 con 0x561a60aef840 2023-07-18T03:31:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe3939ccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a609e0760 con 0x561a60af06e0 2023-07-18T03:31:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe3939ccb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a608c48c0 con 0x561a6094b6b0 2023-07-18T03:31:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe3939ccb80 10 client.0 _ll_register_callbacks cb 0x561a609de700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe384de1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60aef840 0x561a60af8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe384de1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60aef840 0x561a60af8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58022/0 (socket says 172.21.15.40:58022) 2023-07-18T03:31:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe384de1700 1 -- 172.21.15.40:0/744404242 learned_addr learned my addr 172.21.15.40:0/744404242 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe385de3700 1 --2- 172.21.15.40:0/744404242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a60af06e0 0x561a60affd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe384de1700 1 -- 172.21.15.40:0/744404242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6094b6b0 msgr2=0x561a60af87b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:31:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe384de1700 1 --2- 172.21.15.40:0/744404242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6094b6b0 0x561a60af87b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe384de1700 1 -- 172.21.15.40:0/744404242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a60af06e0 msgr2=0x561a60affd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe384de1700 1 --2- 172.21.15.40:0/744404242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a60af06e0 0x561a60affd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe384de1700 1 -- 172.21.15.40:0/744404242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3700078a0 con 0x561a60aef840 2023-07-18T03:31:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.150+0000 7fe385de3700 1 --2- 172.21.15.40:0/744404242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a60af06e0 0x561a60affd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.151+0000 7fe384de1700 1 --2- 172.21.15.40:0/744404242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60aef840 0x561a60af8cf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe3700116f0 tx=0x7fe370011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a57abc25a50bc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.151+0000 7fe368ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.151+0000 7fe368ff9700 1 -- 172.21.15.40:0/744404242 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370016070 con 0x561a60aef840 2023-07-18T03:31:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.151+0000 7fe368ff9700 1 -- 172.21.15.40:0/744404242 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe370004320 con 0x561a60aef840 2023-07-18T03:31:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.151+0000 7fe3939ccb80 1 -- 172.21.15.40:0/744404242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a6099a490 con 0x561a60aef840 2023-07-18T03:31:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.151+0000 7fe368ff9700 1 -- 172.21.15.40:0/744404242 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3700044e0 con 0x561a60aef840 2023-07-18T03:31:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.151+0000 7fe3939ccb80 10 client.15483 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.151+0000 7fe3939ccb80 1 -- 172.21.15.40:0/744404242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fe380107750 con 0x561a60aef840 2023-07-18T03:31:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.151+0000 7fe368ff9700 1 -- 172.21.15.40:0/744404242 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561a6099a490 con 0x561a60aef840 2023-07-18T03:31:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.151+0000 7fe368ff9700 1 -- 172.21.15.40:0/744404242 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe370024050 con 0x561a60aef840 2023-07-18T03:31:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.151+0000 7fe3939ccb80 10 client.15483 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.151+0000 7fe3939ccb80 10 client.15483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.152+0000 7fe3939ccb80 20 client.15483 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.152+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/744404242 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561a60b045c0 0x561a60b069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.152+0000 7fe3939ccb80 4 client.15483 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.152+0000 7fe3939ccb80 1 -- 172.21.15.40:0/744404242 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a60a39e50 con 0x561a60b045c0 2023-07-18T03:31:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.152+0000 7fe3855e2700 1 --2- 172.21.15.40:0/744404242 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561a60b045c0 0x561a60b069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.152+0000 7fe368ff9700 10 client.15483 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.152+0000 7fe3855e2700 1 --2- 172.21.15.40:0/744404242 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561a60b045c0 0x561a60b069d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0535d26a0c76947 server_cookie=91dad41123c86111 in_seq=0 out_seq=0 2023-07-18T03:31:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.153+0000 7fe368ff9700 1 -- 172.21.15.40:0/744404242 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561a60a39e50 con 0x561a60b045c0 2023-07-18T03:31:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.153+0000 7fe368ff9700 10 client.15483 handle_command_reply: tid=0 2023-07-18T03:31:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.153+0000 7fe3939ccb80 1 client.15483 shutdown 2023-07-18T03:31:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.153+0000 7fe3939ccb80 1 -- 172.21.15.40:0/744404242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60aef840 msgr2=0x561a60af8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.153+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/744404242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60aef840 0x561a60af8cf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe3700116f0 tx=0x7fe370011ab0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.153+0000 7fe3939ccb80 1 -- 172.21.15.40:0/744404242 shutdown_connections 2023-07-18T03:31:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.153+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/744404242 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561a60b045c0 0x561a60b069d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.153+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/744404242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a60af06e0 0x561a60affd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.153+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/744404242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a6094b6b0 0x561a60af87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.153+0000 7fe368ff9700 0 client.15483 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.153+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/744404242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60aef840 0x561a60af8cf0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.153+0000 7fe3939ccb80 1 -- 172.21.15.40:0/744404242 >> 172.21.15.40:0/744404242 conn(0x561a60ae4be0 msgr2=0x561a60ae94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.153+0000 7fe3939ccb80 1 -- 172.21.15.40:0/744404242 shutdown_connections 2023-07-18T03:31:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.153+0000 7fe3939ccb80 1 -- 172.21.15.40:0/744404242 wait complete. 2023-07-18T03:31:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.154+0000 7fe3939ccb80 20 client.15483 trim_cache size 0 max 16384 2023-07-18T03:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.168+0000 7fe3939ccb80 1 Processor -- start 2023-07-18T03:31:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.168+0000 7fe3939ccb80 1 -- start start 2023-07-18T03:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.168+0000 7fe3939ccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a6094b6b0 0x561a60af0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.168+0000 7fe3939ccb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a60aef840 0x561a60affcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.168+0000 7fe3939ccb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60af06e0 0x561a60af3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.168+0000 7fe3939ccb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a60af0ff0 con 0x561a60af06e0 2023-07-18T03:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.168+0000 7fe3939ccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a60af9930 con 0x561a6094b6b0 2023-07-18T03:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.168+0000 7fe3939ccb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a60af9430 con 0x561a60aef840 2023-07-18T03:31:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.169+0000 7fe385de3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60af06e0 0x561a60af3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.169+0000 7fe385de3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60af06e0 0x561a60af3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58032/0 (socket says 172.21.15.40:58032) 2023-07-18T03:31:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.169+0000 7fe385de3700 1 -- 172.21.15.40:0/637567072 learned_addr learned my addr 172.21.15.40:0/637567072 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.169+0000 7fe3855e2700 1 --2- 172.21.15.40:0/637567072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a6094b6b0 0x561a60af0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.169+0000 7fe384de1700 1 --2- 172.21.15.40:0/637567072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a60aef840 0x561a60affcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.169+0000 7fe385de3700 1 -- 172.21.15.40:0/637567072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a60aef840 msgr2=0x561a60affcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.169+0000 7fe385de3700 1 --2- 172.21.15.40:0/637567072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a60aef840 0x561a60affcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.169+0000 7fe385de3700 1 -- 172.21.15.40:0/637567072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a6094b6b0 msgr2=0x561a60af0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.169+0000 7fe385de3700 1 --2- 172.21.15.40:0/637567072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a6094b6b0 0x561a60af0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.169+0000 7fe385de3700 1 -- 172.21.15.40:0/637567072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a60857c00 con 0x561a60af06e0 2023-07-18T03:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.169+0000 7fe3855e2700 1 --2- 172.21.15.40:0/637567072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a6094b6b0 0x561a60af0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.169+0000 7fe385de3700 1 --2- 172.21.15.40:0/637567072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60af06e0 0x561a60af3530 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe378000f80 tx=0x7fe3780957e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0af79983e17803 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.169+0000 7fe36bfff700 1 -- 172.21.15.40:0/637567072 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378097840 con 0x561a60af06e0 2023-07-18T03:31:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.169+0000 7fe36bfff700 1 -- 172.21.15.40:0/637567072 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3780840b0 con 0x561a60af06e0 2023-07-18T03:31:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.169+0000 7fe36bfff700 1 -- 172.21.15.40:0/637567072 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37809f110 con 0x561a60af06e0 2023-07-18T03:31:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.170+0000 7fe3939ccb80 1 -- 172.21.15.40:0/637567072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60af06e0 msgr2=0x561a60af3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.170+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/637567072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60af06e0 0x561a60af3530 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe378000f80 tx=0x7fe3780957e0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.170+0000 7fe3939ccb80 1 -- 172.21.15.40:0/637567072 shutdown_connections 2023-07-18T03:31:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.170+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/637567072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60af06e0 0x561a60af3530 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.170+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/637567072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a6094b6b0 0x561a60af0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.172 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.170+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/637567072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a60aef840 0x561a60affcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.172 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.170+0000 7fe3939ccb80 1 -- 172.21.15.40:0/637567072 >> 172.21.15.40:0/637567072 conn(0x561a60ae4be0 msgr2=0x561a60b0b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:17.172 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.170+0000 7fe3939ccb80 1 -- 172.21.15.40:0/637567072 shutdown_connections 2023-07-18T03:31:17.172 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.170+0000 7fe3939ccb80 1 -- 172.21.15.40:0/637567072 wait complete. 2023-07-18T03:31:17.172 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.170+0000 7fe3939ccb80 1 -- start start 2023-07-18T03:31:17.172 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.170+0000 7fe3939ccb80 1 Processor -- start 2023-07-18T03:31:17.172 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.171+0000 7fe3939ccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a6094b6b0 0x561a60aff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.171+0000 7fe3939ccb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a60aef840 0x561a60affb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.171+0000 7fe3939ccb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60af06e0 0x561a60afc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.171+0000 7fe3939ccb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a60afc6c0 con 0x561a60af06e0 2023-07-18T03:31:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.171+0000 7fe3939ccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a60afc830 con 0x561a6094b6b0 2023-07-18T03:31:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.171+0000 7fe3939ccb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a60afc9f0 con 0x561a60aef840 2023-07-18T03:31:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.171+0000 7fe3939ccb80 10 client.0 _ll_register_callbacks cb 0x7fe370007cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.171+0000 7fe385de3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60af06e0 0x561a60afc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.171+0000 7fe385de3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60af06e0 0x561a60afc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58038/0 (socket says 172.21.15.40:58038) 2023-07-18T03:31:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.171+0000 7fe385de3700 1 -- 172.21.15.40:0/334286461 learned_addr learned my addr 172.21.15.40:0/334286461 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.171+0000 7fe384de1700 1 --2- 172.21.15.40:0/334286461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a60aef840 0x561a60affb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.171+0000 7fe3855e2700 1 --2- 172.21.15.40:0/334286461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a6094b6b0 0x561a60aff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.171+0000 7fe385de3700 1 -- 172.21.15.40:0/334286461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a60aef840 msgr2=0x561a60affb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.171+0000 7fe385de3700 1 --2- 172.21.15.40:0/334286461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a60aef840 0x561a60affb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.171+0000 7fe385de3700 1 -- 172.21.15.40:0/334286461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a6094b6b0 msgr2=0x561a60aff5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.171+0000 7fe385de3700 1 --2- 172.21.15.40:0/334286461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a6094b6b0 0x561a60aff5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.171+0000 7fe385de3700 1 -- 172.21.15.40:0/334286461 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a6086ed80 con 0x561a60af06e0 2023-07-18T03:31:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.172+0000 7fe385de3700 1 --2- 172.21.15.40:0/334286461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60af06e0 0x561a60afc070 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe3780c9e70 tx=0x7fe378084220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4754ef4614ff4c00 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.172+0000 7fe368ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.172+0000 7fe368ff9700 1 -- 172.21.15.40:0/334286461 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378097840 con 0x561a60af06e0 2023-07-18T03:31:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.172+0000 7fe368ff9700 1 -- 172.21.15.40:0/334286461 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe37809d190 con 0x561a60af06e0 2023-07-18T03:31:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.172+0000 7fe3939ccb80 1 -- 172.21.15.40:0/334286461 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe37c01e040 con 0x561a60af06e0 2023-07-18T03:31:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.172+0000 7fe368ff9700 1 -- 172.21.15.40:0/334286461 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe37809f110 con 0x561a60af06e0 2023-07-18T03:31:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.172+0000 7fe3939ccb80 10 client.15489 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.172+0000 7fe3939ccb80 1 -- 172.21.15.40:0/334286461 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fe3801a0290 con 0x561a60af06e0 2023-07-18T03:31:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.172+0000 7fe368ff9700 1 -- 172.21.15.40:0/334286461 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe37c01e040 con 0x561a60af06e0 2023-07-18T03:31:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.172+0000 7fe368ff9700 1 -- 172.21.15.40:0/334286461 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe378098990 con 0x561a60af06e0 2023-07-18T03:31:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.172+0000 7fe3939ccb80 10 client.15489 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.172+0000 7fe3939ccb80 10 client.15489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.172+0000 7fe3939ccb80 20 client.15489 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.172+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/334286461 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561a60b596a0 0x561a60b5bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.172+0000 7fe3939ccb80 4 client.15489 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:31:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.172+0000 7fe3939ccb80 1 -- 172.21.15.40:0/334286461 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561a609fbff0 con 0x561a60b596a0 2023-07-18T03:31:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.173+0000 7fe3855e2700 1 --2- 172.21.15.40:0/334286461 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561a60b596a0 0x561a60b5bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.173+0000 7fe368ff9700 10 client.15489 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.173+0000 7fe3855e2700 1 --2- 172.21.15.40:0/334286461 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561a60b596a0 0x561a60b5bab0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f21f5d6889e14433 server_cookie=21d876400d159484 in_seq=0 out_seq=0 2023-07-18T03:31:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.177+0000 7fe368ff9700 1 -- 172.21.15.40:0/334286461 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561a609fbff0 con 0x561a60b596a0 2023-07-18T03:31:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.177+0000 7fe368ff9700 10 client.15489 handle_command_reply: tid=0 2023-07-18T03:31:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.177+0000 7fe3939ccb80 1 client.15489 shutdown 2023-07-18T03:31:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.177+0000 7fe3939ccb80 1 -- 172.21.15.40:0/334286461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60af06e0 msgr2=0x561a60afc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.177+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/334286461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60af06e0 0x561a60afc070 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe3780c9e70 tx=0x7fe378084220 comp rx=0 tx=0).stop 2023-07-18T03:31:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.177+0000 7fe3939ccb80 1 -- 172.21.15.40:0/334286461 shutdown_connections 2023-07-18T03:31:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.177+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/334286461 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561a60b596a0 0x561a60b5bab0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.177+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/334286461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a60af06e0 0x561a60afc070 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.177+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/334286461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a6094b6b0 0x561a60aff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.177+0000 7fe368ff9700 0 client.15489 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:17.179 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.177+0000 7fe3939ccb80 1 --2- 172.21.15.40:0/334286461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a60aef840 0x561a60affb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.177+0000 7fe3939ccb80 1 -- 172.21.15.40:0/334286461 >> 172.21.15.40:0/334286461 conn(0x561a60ae4be0 msgr2=0x561a60af3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.177+0000 7fe3939ccb80 1 -- 172.21.15.40:0/334286461 shutdown_connections 2023-07-18T03:31:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.177+0000 7fe3939ccb80 1 -- 172.21.15.40:0/334286461 wait complete. 2023-07-18T03:31:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.177+0000 7fe3939ccb80 20 client.15489 trim_cache size 0 max 16384 2023-07-18T03:31:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:31:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:31:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:31:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:31:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.178+0000 7fe387846700 1 -- 172.21.15.40:0/2322790705 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe36c0a1150 msgr2=0x7fe36c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.178+0000 7fe387846700 1 --2- 172.21.15.40:0/2322790705 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe36c0a1150 0x7fe36c0a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe380102980 tx=0x7fe37c005fd0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.178+0000 7fe387846700 1 -- 172.21.15.40:0/2322790705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe380100640 msgr2=0x7fe380101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.178+0000 7fe387846700 1 --2- 172.21.15.40:0/2322790705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe380100640 0x7fe380101af0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe37800ada0 tx=0x7fe37800c330 comp rx=0 tx=0).stop 2023-07-18T03:31:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.178+0000 7fe387846700 1 -- 172.21.15.40:0/2322790705 shutdown_connections 2023-07-18T03:31:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.178+0000 7fe387846700 1 --2- 172.21.15.40:0/2322790705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe380100640 0x7fe380101af0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.178+0000 7fe387846700 1 --2- 172.21.15.40:0/2322790705 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe36c0a1150 0x7fe36c0a3600 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.178+0000 7fe387846700 1 --2- 172.21.15.40:0/2322790705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3800fea00 0x7fe380101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.178+0000 7fe387846700 1 --2- 172.21.15.40:0/2322790705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3800ffc90 0x7fe3801015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.178+0000 7fe387846700 1 -- 172.21.15.40:0/2322790705 >> 172.21.15.40:0/2322790705 conn(0x7fe3800fa000 msgr2=0x7fe380067900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.178+0000 7fe387846700 1 -- 172.21.15.40:0/2322790705 shutdown_connections 2023-07-18T03:31:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:17.178+0000 7fe387846700 1 -- 172.21.15.40:0/2322790705 wait complete. 2023-07-18T03:31:18.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:17 smithi134 ceph-mon[98216]: pgmap v305: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2023-07-18T03:31:18.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:17 smithi040 ceph-mon[96439]: pgmap v305: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2023-07-18T03:31:18.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:17 smithi064 ceph-mon[98059]: pgmap v305: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2023-07-18T03:31:20.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:19 smithi134 ceph-mon[98216]: pgmap v306: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 12 op/s 2023-07-18T03:31:20.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:19 smithi040 ceph-mon[96439]: pgmap v306: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 12 op/s 2023-07-18T03:31:20.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:19 smithi064 ceph-mon[98059]: pgmap v306: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 12 op/s 2023-07-18T03:31:21.215 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:31:21.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.351+0000 7fd1d4b4f700 1 -- 172.21.15.40:0/3151528898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d00fe3d0 msgr2=0x7fd1d00fe7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.351+0000 7fd1d4b4f700 1 --2- 172.21.15.40:0/3151528898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d00fe3d0 0x7fd1d00fe7e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0009990 tx=0x7fd1c0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.351+0000 7fd1d4b4f700 1 -- 172.21.15.40:0/3151528898 shutdown_connections 2023-07-18T03:31:21.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.351+0000 7fd1d4b4f700 1 --2- 172.21.15.40:0/3151528898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d00ff6d0 0x7fd1d00ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.351+0000 7fd1d4b4f700 1 --2- 172.21.15.40:0/3151528898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d00fe3d0 0x7fd1d00fe7e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.351+0000 7fd1d4b4f700 1 --2- 172.21.15.40:0/3151528898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1d00fed20 0x7fd1d00ff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.351+0000 7fd1d4b4f700 1 -- 172.21.15.40:0/3151528898 >> 172.21.15.40:0/3151528898 conn(0x7fd1d00fa000 msgr2=0x7fd1d00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:21.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.351+0000 7fd1d4b4f700 1 -- 172.21.15.40:0/3151528898 shutdown_connections 2023-07-18T03:31:21.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.351+0000 7fd1d4b4f700 1 -- 172.21.15.40:0/3151528898 wait complete. 2023-07-18T03:31:21.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.352+0000 7fd1d4b4f700 1 Processor -- start 2023-07-18T03:31:21.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.352+0000 7fd1d4b4f700 1 -- start start 2023-07-18T03:31:21.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.352+0000 7fd1d4b4f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d00fe3d0 0x7fd1d0197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.352+0000 7fd1d4b4f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d00fed20 0x7fd1d0197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.352+0000 7fd1d4b4f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1d00ff6d0 0x7fd1d019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.352+0000 7fd1d4b4f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d019f5d0 con 0x7fd1d00fed20 2023-07-18T03:31:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.352+0000 7fd1d4b4f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d019f710 con 0x7fd1d00fe3d0 2023-07-18T03:31:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.352+0000 7fd1d4b4f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d019f8a0 con 0x7fd1d00ff6d0 2023-07-18T03:31:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.353+0000 7fd1cdd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d00fed20 0x7fd1d0197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.353+0000 7fd1cdd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d00fed20 0x7fd1d0197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58050/0 (socket says 172.21.15.40:58050) 2023-07-18T03:31:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.353+0000 7fd1ce59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d00fe3d0 0x7fd1d0197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.353+0000 7fd1cdd9b700 1 -- 172.21.15.40:0/2391077103 learned_addr learned my addr 172.21.15.40:0/2391077103 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.353+0000 7fd1ced9d700 1 --2- 172.21.15.40:0/2391077103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1d00ff6d0 0x7fd1d019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.353+0000 7fd1cdd9b700 1 -- 172.21.15.40:0/2391077103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1d00ff6d0 msgr2=0x7fd1d019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.353+0000 7fd1cdd9b700 1 --2- 172.21.15.40:0/2391077103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1d00ff6d0 0x7fd1d019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.353+0000 7fd1cdd9b700 1 -- 172.21.15.40:0/2391077103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d00fe3d0 msgr2=0x7fd1d0197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.353+0000 7fd1cdd9b700 1 --2- 172.21.15.40:0/2391077103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d00fe3d0 0x7fd1d0197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.353+0000 7fd1cdd9b700 1 -- 172.21.15.40:0/2391077103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c00095f0 con 0x7fd1d00fed20 2023-07-18T03:31:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.353+0000 7fd1ced9d700 1 --2- 172.21.15.40:0/2391077103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1d00ff6d0 0x7fd1d019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.353+0000 7fd1cdd9b700 1 --2- 172.21.15.40:0/2391077103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d00fed20 0x7fd1d0197f90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400d980 tx=0x7fd1c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.353+0000 7fd1bf7fe700 1 -- 172.21.15.40:0/2391077103 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4004220 con 0x7fd1d00fed20 2023-07-18T03:31:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.353+0000 7fd1bf7fe700 1 -- 172.21.15.40:0/2391077103 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1c4007640 con 0x7fd1d00fed20 2023-07-18T03:31:21.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.353+0000 7fd1d4b4f700 1 -- 172.21.15.40:0/2391077103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d019fb80 con 0x7fd1d00fed20 2023-07-18T03:31:21.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.353+0000 7fd1bf7fe700 1 -- 172.21.15.40:0/2391077103 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4005610 con 0x7fd1d00fed20 2023-07-18T03:31:21.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.353+0000 7fd1d4b4f700 1 -- 172.21.15.40:0/2391077103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d01a0090 con 0x7fd1d00fed20 2023-07-18T03:31:21.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.358+0000 7fd1e0cd5b80 1 Processor -- start 2023-07-18T03:31:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.358+0000 7fd1e0cd5b80 1 -- start start 2023-07-18T03:31:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.359+0000 7fd1e0cd5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936c4d0 0x558e8936c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.359+0000 7fd1e0cd5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936ce20 0x558e8936d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.359+0000 7fd1e0cd5b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936dc70 0x558e8936e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.359+0000 7fd1e0cd5b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e89271cb0 con 0x558e8936dc70 2023-07-18T03:31:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.359+0000 7fd1e0cd5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e892c4a40 con 0x558e8936ce20 2023-07-18T03:31:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.359+0000 7fd1e0cd5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e892bded0 con 0x558e8936c4d0 2023-07-18T03:31:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.359+0000 7fd1ced9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936c4d0 0x558e8936c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.359+0000 7fd1ced9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936c4d0 0x558e8936c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:47082/0 (socket says 172.21.15.40:47082) 2023-07-18T03:31:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.359+0000 7fd1ced9d700 1 -- 172.21.15.40:0/4192313643 learned_addr learned my addr 172.21.15.40:0/4192313643 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.359+0000 7fd1ce59c700 1 --2- 172.21.15.40:0/4192313643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936ce20 0x558e8936d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.359+0000 7fd1ced9d700 1 -- 172.21.15.40:0/4192313643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936ce20 msgr2=0x558e8936d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.359+0000 7fd1ced9d700 1 --2- 172.21.15.40:0/4192313643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936ce20 0x558e8936d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.359+0000 7fd1ced9d700 1 -- 172.21.15.40:0/4192313643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936dc70 msgr2=0x558e8936e0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:31:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.359+0000 7fd1ced9d700 1 --2- 172.21.15.40:0/4192313643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936dc70 0x558e8936e0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.359+0000 7fd1ced9d700 1 -- 172.21.15.40:0/4192313643 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e890ebde0 con 0x558e8936c4d0 2023-07-18T03:31:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.359+0000 7fd1bf7fe700 1 -- 172.21.15.40:0/2391077103 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd1c4004750 con 0x7fd1d00fed20 2023-07-18T03:31:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.360+0000 7fd1ced9d700 1 --2- 172.21.15.40:0/4192313643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936c4d0 0x558e8936c8e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009dd0 tx=0x7fd1c80077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98e959dd1c72baec server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.360+0000 7fd1bcff9700 1 -- 172.21.15.40:0/4192313643 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8018070 con 0x558e8936c4d0 2023-07-18T03:31:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.360+0000 7fd1bcff9700 1 -- 172.21.15.40:0/4192313643 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1c800f050 con 0x558e8936c4d0 2023-07-18T03:31:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.360+0000 7fd1bcff9700 1 -- 172.21.15.40:0/4192313643 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8005a40 con 0x558e8936c4d0 2023-07-18T03:31:21.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.360+0000 7fd1bf7fe700 1 --2- 172.21.15.40:0/2391077103 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd1b80a1150 0x7fd1b80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.360+0000 7fd1ce59c700 1 --2- 172.21.15.40:0/2391077103 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd1b80a1150 0x7fd1b80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.360+0000 7fd1bf7fe700 1 -- 172.21.15.40:0/2391077103 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd1c40cf170 con 0x7fd1d00fed20 2023-07-18T03:31:21.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.361+0000 7fd1ce59c700 1 --2- 172.21.15.40:0/2391077103 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd1b80a1150 0x7fd1b80a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000b5c0 tx=0x7fd1c0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:21.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.361+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/4192313643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936c4d0 msgr2=0x558e8936c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.361+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/4192313643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936c4d0 0x558e8936c8e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009dd0 tx=0x7fd1c80077e0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.361+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/4192313643 shutdown_connections 2023-07-18T03:31:21.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.361+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/4192313643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936dc70 0x558e8936e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.361+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/4192313643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936c4d0 0x558e8936c8e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.361+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/4192313643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936ce20 0x558e8936d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.361+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/4192313643 >> 172.21.15.40:0/4192313643 conn(0x558e89361da0 msgr2=0x558e893671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:21.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.361+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/4192313643 shutdown_connections 2023-07-18T03:31:21.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.361+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/4192313643 wait complete. 2023-07-18T03:31:21.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.361+0000 7fd1e0cd5b80 1 -- start start 2023-07-18T03:31:21.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.361+0000 7fd1e0cd5b80 1 Processor -- start 2023-07-18T03:31:21.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1e0cd5b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936c4d0 0x558e89376100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1e0cd5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936ce20 0x558e89376640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1e0cd5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936dc70 0x558e8937d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1e0cd5b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8925dac0 con 0x558e8936c4d0 2023-07-18T03:31:21.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1e0cd5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e8925d7c0 con 0x558e8936ce20 2023-07-18T03:31:21.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1e0cd5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e89141920 con 0x558e8936dc70 2023-07-18T03:31:21.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1e0cd5b80 10 client.0 _ll_register_callbacks cb 0x558e8925b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:21.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1ce59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936c4d0 0x558e89376100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1ce59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936c4d0 0x558e89376100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58060/0 (socket says 172.21.15.40:58060) 2023-07-18T03:31:21.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1ce59c700 1 -- 172.21.15.40:0/3591475820 learned_addr learned my addr 172.21.15.40:0/3591475820 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:21.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1ced9d700 1 --2- 172.21.15.40:0/3591475820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936dc70 0x558e8937d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1cdd9b700 1 --2- 172.21.15.40:0/3591475820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936ce20 0x558e89376640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1ce59c700 1 -- 172.21.15.40:0/3591475820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936dc70 msgr2=0x558e8937d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:21.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1ce59c700 1 --2- 172.21.15.40:0/3591475820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936dc70 0x558e8937d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1ce59c700 1 -- 172.21.15.40:0/3591475820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936ce20 msgr2=0x558e89376640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:21.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1ce59c700 1 --2- 172.21.15.40:0/3591475820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936ce20 0x558e89376640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1ce59c700 1 -- 172.21.15.40:0/3591475820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c8009a00 con 0x558e8936c4d0 2023-07-18T03:31:21.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1ce59c700 1 --2- 172.21.15.40:0/3591475820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936c4d0 0x558e89376100 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0015000 tx=0x7fd1c0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1513bf0132735235 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:21.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.362+0000 7fd1b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:21.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.363+0000 7fd1b5ffb700 1 -- 172.21.15.40:0/3591475820 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0004180 con 0x558e8936c4d0 2023-07-18T03:31:21.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.363+0000 7fd1b5ffb700 1 -- 172.21.15.40:0/3591475820 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd1c00056c0 con 0x558e8936c4d0 2023-07-18T03:31:21.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.363+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/3591475820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e892174f0 con 0x558e8936c4d0 2023-07-18T03:31:21.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.363+0000 7fd1b5ffb700 1 -- 172.21.15.40:0/3591475820 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1c000bea0 con 0x558e8936c4d0 2023-07-18T03:31:21.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.363+0000 7fd1e0cd5b80 10 client.15498 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:21.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.363+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/3591475820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558e890d4c60 con 0x558e8936c4d0 2023-07-18T03:31:21.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.363+0000 7fd1b5ffb700 1 -- 172.21.15.40:0/3591475820 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1c0040080 con 0x558e8936c4d0 2023-07-18T03:31:21.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.363+0000 7fd1b5ffb700 1 -- 172.21.15.40:0/3591475820 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd1c003a030 con 0x558e8936c4d0 2023-07-18T03:31:21.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.363+0000 7fd1e0cd5b80 10 client.15498 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:21.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.363+0000 7fd1e0cd5b80 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:21.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.363+0000 7fd1e0cd5b80 20 client.15498 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:21.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.363+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/3591475820 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x558e89381d70 0x558e89384180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.363+0000 7fd1e0cd5b80 4 client.15498 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:21.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.363+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/3591475820 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e892b6eb0 con 0x558e89381d70 2023-07-18T03:31:21.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.363+0000 7fd1cdd9b700 1 --2- 172.21.15.40:0/3591475820 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x558e89381d70 0x558e89384180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.364+0000 7fd1b5ffb700 10 client.15498 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:21.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.364+0000 7fd1cdd9b700 1 --2- 172.21.15.40:0/3591475820 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x558e89381d70 0x558e89384180 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0ff06656a0130b5 server_cookie=61f9d2f4292b8346 in_seq=0 out_seq=0 2023-07-18T03:31:21.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.365+0000 7fd1b5ffb700 1 -- 172.21.15.40:0/3591475820 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558e892b6eb0 con 0x558e89381d70 2023-07-18T03:31:21.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.365+0000 7fd1b5ffb700 10 client.15498 handle_command_reply: tid=0 2023-07-18T03:31:21.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.365+0000 7fd1e0cd5b80 1 client.15498 shutdown 2023-07-18T03:31:21.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.365+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/3591475820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936c4d0 msgr2=0x558e89376100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.365+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/3591475820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936c4d0 0x558e89376100 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0015000 tx=0x7fd1c0009310 comp rx=0 tx=0).stop 2023-07-18T03:31:21.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.365+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/3591475820 shutdown_connections 2023-07-18T03:31:21.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.365+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/3591475820 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x558e89381d70 0x558e89384180 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.365+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/3591475820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936dc70 0x558e8937d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.365+0000 7fd1b5ffb700 0 client.15498 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:21.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.365+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/3591475820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936c4d0 0x558e89376100 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.365+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/3591475820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936ce20 0x558e89376640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.365+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/3591475820 >> 172.21.15.40:0/3591475820 conn(0x558e89361da0 msgr2=0x558e893668b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:21.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.365+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/3591475820 shutdown_connections 2023-07-18T03:31:21.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.365+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/3591475820 wait complete. 2023-07-18T03:31:21.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.365+0000 7fd1e0cd5b80 20 client.15498 trim_cache size 0 max 16384 2023-07-18T03:31:21.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.395+0000 7fd1e0cd5b80 1 Processor -- start 2023-07-18T03:31:21.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.395+0000 7fd1e0cd5b80 1 -- start start 2023-07-18T03:31:21.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.395+0000 7fd1e0cd5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936c4d0 0x558e89374e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.395+0000 7fd1e0cd5b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936ce20 0x558e8937d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.395+0000 7fd1e0cd5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936dc70 0x558e893668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.395+0000 7fd1e0cd5b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e89381ae0 con 0x558e8936ce20 2023-07-18T03:31:21.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.395+0000 7fd1e0cd5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e89384150 con 0x558e8936dc70 2023-07-18T03:31:21.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.395+0000 7fd1e0cd5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e893842c0 con 0x558e8936c4d0 2023-07-18T03:31:21.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.395+0000 7fd1cdd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936ce20 0x558e8937d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.395+0000 7fd1cdd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936ce20 0x558e8937d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58070/0 (socket says 172.21.15.40:58070) 2023-07-18T03:31:21.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.395+0000 7fd1cdd9b700 1 -- 172.21.15.40:0/3373091096 learned_addr learned my addr 172.21.15.40:0/3373091096 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:21.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.395+0000 7fd1ce59c700 1 --2- 172.21.15.40:0/3373091096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936c4d0 0x558e89374e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.395+0000 7fd1ced9d700 1 --2- 172.21.15.40:0/3373091096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936dc70 0x558e893668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.396+0000 7fd1cdd9b700 1 -- 172.21.15.40:0/3373091096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936c4d0 msgr2=0x558e89374e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:21.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.396+0000 7fd1cdd9b700 1 --2- 172.21.15.40:0/3373091096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936c4d0 0x558e89374e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.396+0000 7fd1cdd9b700 1 -- 172.21.15.40:0/3373091096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936dc70 msgr2=0x558e893668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:21.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.396+0000 7fd1cdd9b700 1 --2- 172.21.15.40:0/3373091096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936dc70 0x558e893668b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.396+0000 7fd1cdd9b700 1 -- 172.21.15.40:0/3373091096 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e89112250 con 0x558e8936ce20 2023-07-18T03:31:21.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.396+0000 7fd1ced9d700 1 --2- 172.21.15.40:0/3373091096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936dc70 0x558e893668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:31:21.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.396+0000 7fd1cdd9b700 1 --2- 172.21.15.40:0/3373091096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936ce20 0x558e8937d640 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4098e40 tx=0x7fd1c4005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf5aec3b7754d9d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:21.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.396+0000 7fd1bd7fa700 1 -- 172.21.15.40:0/3373091096 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c409cee0 con 0x558e8936ce20 2023-07-18T03:31:21.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.396+0000 7fd1bd7fa700 1 -- 172.21.15.40:0/3373091096 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1c4014050 con 0x558e8936ce20 2023-07-18T03:31:21.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.396+0000 7fd1bd7fa700 1 -- 172.21.15.40:0/3373091096 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c40983a0 con 0x558e8936ce20 2023-07-18T03:31:21.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/3373091096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936ce20 msgr2=0x558e8937d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/3373091096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936ce20 0x558e8937d640 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4098e40 tx=0x7fd1c4005770 comp rx=0 tx=0).stop 2023-07-18T03:31:21.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/3373091096 shutdown_connections 2023-07-18T03:31:21.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/3373091096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936dc70 0x558e893668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/3373091096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936c4d0 0x558e89374e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/3373091096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936ce20 0x558e8937d640 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/3373091096 >> 172.21.15.40:0/3373091096 conn(0x558e89361da0 msgr2=0x558e89388ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/3373091096 shutdown_connections 2023-07-18T03:31:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/3373091096 wait complete. 2023-07-18T03:31:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 1 -- start start 2023-07-18T03:31:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 1 Processor -- start 2023-07-18T03:31:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936c4d0 0x558e8937d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936ce20 0x558e8937a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936dc70 0x558e8937a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e893d5c20 con 0x558e8936ce20 2023-07-18T03:31:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e893d5d60 con 0x558e8936dc70 2023-07-18T03:31:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e893d5ed0 con 0x558e8936c4d0 2023-07-18T03:31:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1e0cd5b80 10 client.0 _ll_register_callbacks cb 0x7fd1c000ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:21.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1cdd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936ce20 0x558e8937a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.397+0000 7fd1cdd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936ce20 0x558e8937a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58074/0 (socket says 172.21.15.40:58074) 2023-07-18T03:31:21.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.398+0000 7fd1cdd9b700 1 -- 172.21.15.40:0/160243546 learned_addr learned my addr 172.21.15.40:0/160243546 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:21.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.398+0000 7fd1ce59c700 1 --2- 172.21.15.40:0/160243546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936c4d0 0x558e8937d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.398+0000 7fd1cdd9b700 1 -- 172.21.15.40:0/160243546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936c4d0 msgr2=0x558e8937d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:21.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.398+0000 7fd1cdd9b700 1 --2- 172.21.15.40:0/160243546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936c4d0 0x558e8937d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.398+0000 7fd1cdd9b700 1 -- 172.21.15.40:0/160243546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936dc70 msgr2=0x558e8937a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:21.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.398+0000 7fd1cdd9b700 1 --2- 172.21.15.40:0/160243546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936dc70 0x558e8937a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.398+0000 7fd1cdd9b700 1 -- 172.21.15.40:0/160243546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d01a0090 con 0x558e8936ce20 2023-07-18T03:31:21.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.398+0000 7fd1cdd9b700 1 --2- 172.21.15.40:0/160243546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936ce20 0x558e8937a0b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4000c00 tx=0x7fd1c4097300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2752703b8d438f55 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:21.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.398+0000 7fd1b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:21.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.398+0000 7fd1b5ffb700 1 -- 172.21.15.40:0/160243546 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c40b72c0 con 0x558e8936ce20 2023-07-18T03:31:21.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.398+0000 7fd1b5ffb700 1 -- 172.21.15.40:0/160243546 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd1c409cee0 con 0x558e8936ce20 2023-07-18T03:31:21.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.398+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/160243546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1c800f050 con 0x558e8936ce20 2023-07-18T03:31:21.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.398+0000 7fd1b5ffb700 1 -- 172.21.15.40:0/160243546 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1c40983a0 con 0x558e8936ce20 2023-07-18T03:31:21.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.398+0000 7fd1e0cd5b80 10 client.15504 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:21.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.398+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/160243546 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558e890ea2b0 con 0x558e8936ce20 2023-07-18T03:31:21.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.399+0000 7fd1b5ffb700 1 -- 172.21.15.40:0/160243546 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1c407b080 con 0x558e8936ce20 2023-07-18T03:31:21.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.399+0000 7fd1b5ffb700 1 -- 172.21.15.40:0/160243546 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd1c40acc90 con 0x558e8936ce20 2023-07-18T03:31:21.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.399+0000 7fd1e0cd5b80 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:21.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.399+0000 7fd1e0cd5b80 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:21.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.399+0000 7fd1e0cd5b80 20 client.15504 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:21.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.399+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/160243546 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x558e893d7450 0x558e893d9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.399+0000 7fd1e0cd5b80 4 client.15504 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:31:21.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.399+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/160243546 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558e89279050 con 0x558e893d7450 2023-07-18T03:31:21.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.399+0000 7fd1ce59c700 1 --2- 172.21.15.40:0/160243546 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x558e893d7450 0x558e893d9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.399+0000 7fd1b5ffb700 10 client.15504 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:21.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.399+0000 7fd1ce59c700 1 --2- 172.21.15.40:0/160243546 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x558e893d7450 0x558e893d9860 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc6d4e9f66d72713 server_cookie=aebee6a033354aaf in_seq=0 out_seq=0 2023-07-18T03:31:21.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1b5ffb700 1 -- 172.21.15.40:0/160243546 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558e89279050 con 0x558e893d7450 2023-07-18T03:31:21.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1b5ffb700 10 client.15504 handle_command_reply: tid=0 2023-07-18T03:31:21.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1e0cd5b80 1 client.15504 shutdown 2023-07-18T03:31:21.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/160243546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936ce20 msgr2=0x558e8937a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/160243546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936ce20 0x558e8937a0b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4000c00 tx=0x7fd1c4097300 comp rx=0 tx=0).stop 2023-07-18T03:31:21.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/160243546 shutdown_connections 2023-07-18T03:31:21.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/160243546 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x558e893d7450 0x558e893d9860 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/160243546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e8936dc70 0x558e8937a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1b5ffb700 0 client.15504 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:21.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/160243546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e8936c4d0 0x558e8937d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1e0cd5b80 1 --2- 172.21.15.40:0/160243546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558e8936ce20 0x558e8937a0b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/160243546 >> 172.21.15.40:0/160243546 conn(0x558e89361da0 msgr2=0x558e893668b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:21.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/160243546 shutdown_connections 2023-07-18T03:31:21.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1e0cd5b80 1 -- 172.21.15.40:0/160243546 wait complete. 2023-07-18T03:31:21.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1e0cd5b80 20 client.15504 trim_cache size 0 max 16384 2023-07-18T03:31:21.409 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:31:21.410 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "no active scrubs running", 2023-07-18T03:31:21.410 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": {} 2023-07-18T03:31:21.410 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:31:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1d4b4f700 1 -- 172.21.15.40:0/2391077103 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd1b80a1150 msgr2=0x7fd1b80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1d4b4f700 1 --2- 172.21.15.40:0/2391077103 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd1b80a1150 0x7fd1b80a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000b5c0 tx=0x7fd1c0005960 comp rx=0 tx=0).stop 2023-07-18T03:31:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1d4b4f700 1 -- 172.21.15.40:0/2391077103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d00fed20 msgr2=0x7fd1d0197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1d4b4f700 1 --2- 172.21.15.40:0/2391077103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d00fed20 0x7fd1d0197f90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400d980 tx=0x7fd1c400dc90 comp rx=0 tx=0).stop 2023-07-18T03:31:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.400+0000 7fd1d4b4f700 1 -- 172.21.15.40:0/2391077103 shutdown_connections 2023-07-18T03:31:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.401+0000 7fd1d4b4f700 1 --2- 172.21.15.40:0/2391077103 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd1b80a1150 0x7fd1b80a3600 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.401+0000 7fd1d4b4f700 1 --2- 172.21.15.40:0/2391077103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1d00ff6d0 0x7fd1d019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.401+0000 7fd1d4b4f700 1 --2- 172.21.15.40:0/2391077103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1d00fe3d0 0x7fd1d0197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.401+0000 7fd1d4b4f700 1 --2- 172.21.15.40:0/2391077103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1d00fed20 0x7fd1d0197f90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.401+0000 7fd1d4b4f700 1 -- 172.21.15.40:0/2391077103 >> 172.21.15.40:0/2391077103 conn(0x7fd1d00fa000 msgr2=0x7fd1d00fbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:21.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.401+0000 7fd1d4b4f700 1 -- 172.21.15.40:0/2391077103 shutdown_connections 2023-07-18T03:31:21.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.401+0000 7fd1d4b4f700 1 -- 172.21.15.40:0/2391077103 wait complete. 2023-07-18T03:31:21.413 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:31:21.413 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:31:21.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.543+0000 7f986cdef700 1 -- 172.21.15.40:0/1428776620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98680ffbc0 msgr2=0x7f9868100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.543+0000 7f986cdef700 1 --2- 172.21.15.40:0/1428776620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98680ffbc0 0x7f9868100010 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f985c009990 tx=0x7f985c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.544+0000 7f986cdef700 1 -- 172.21.15.40:0/1428776620 shutdown_connections 2023-07-18T03:31:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.544+0000 7f986cdef700 1 --2- 172.21.15.40:0/1428776620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9868100550 0x7f9868106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.544+0000 7f986cdef700 1 --2- 172.21.15.40:0/1428776620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98680fe9d0 0x7f98680fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.544+0000 7f986cdef700 1 --2- 172.21.15.40:0/1428776620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98680ffbc0 0x7f9868100010 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.544+0000 7f986cdef700 1 -- 172.21.15.40:0/1428776620 >> 172.21.15.40:0/1428776620 conn(0x7f98680f9fd0 msgr2=0x7f98680fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:21.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.544+0000 7f986cdef700 1 -- 172.21.15.40:0/1428776620 shutdown_connections 2023-07-18T03:31:21.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.544+0000 7f986cdef700 1 -- 172.21.15.40:0/1428776620 wait complete. 2023-07-18T03:31:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.544+0000 7f986cdef700 1 Processor -- start 2023-07-18T03:31:21.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.544+0000 7f986cdef700 1 -- start start 2023-07-18T03:31:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.545+0000 7f986cdef700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98680fe9d0 0x7f9868068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.545+0000 7f986cdef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98680ffbc0 0x7f986819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.545+0000 7f986cdef700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9868100550 0x7f986819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.545+0000 7f986cdef700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986819d360 con 0x7f98680ffbc0 2023-07-18T03:31:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.545+0000 7f986cdef700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986819d4d0 con 0x7f9868100550 2023-07-18T03:31:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.545+0000 7f986cdef700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986819d690 con 0x7f98680fe9d0 2023-07-18T03:31:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.545+0000 7f9865d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98680ffbc0 0x7f986819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.545+0000 7f9865d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98680ffbc0 0x7f986819a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58088/0 (socket says 172.21.15.40:58088) 2023-07-18T03:31:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.545+0000 7f9865d9b700 1 -- 172.21.15.40:0/1979102791 learned_addr learned my addr 172.21.15.40:0/1979102791 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.545+0000 7f986659c700 1 --2- 172.21.15.40:0/1979102791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98680fe9d0 0x7f9868068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.545+0000 7f9866d9d700 1 --2- 172.21.15.40:0/1979102791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9868100550 0x7f986819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.545+0000 7f9865d9b700 1 -- 172.21.15.40:0/1979102791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98680fe9d0 msgr2=0x7f9868068200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.545+0000 7f9865d9b700 1 --2- 172.21.15.40:0/1979102791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98680fe9d0 0x7f9868068200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.545+0000 7f9865d9b700 1 -- 172.21.15.40:0/1979102791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9868100550 msgr2=0x7f986819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.545+0000 7f9865d9b700 1 --2- 172.21.15.40:0/1979102791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9868100550 0x7f986819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.545+0000 7f9865d9b700 1 -- 172.21.15.40:0/1979102791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c0095f0 con 0x7f98680ffbc0 2023-07-18T03:31:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.545+0000 7f9865d9b700 1 --2- 172.21.15.40:0/1979102791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98680ffbc0 0x7f986819a540 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f985c00c010 tx=0x7f985c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.546+0000 7f98577fe700 1 -- 172.21.15.40:0/1979102791 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c01d070 con 0x7f98680ffbc0 2023-07-18T03:31:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.546+0000 7f98577fe700 1 -- 172.21.15.40:0/1979102791 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f985c005030 con 0x7f98680ffbc0 2023-07-18T03:31:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.546+0000 7f986cdef700 1 -- 172.21.15.40:0/1979102791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986819d910 con 0x7f98680ffbc0 2023-07-18T03:31:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.546+0000 7f98577fe700 1 -- 172.21.15.40:0/1979102791 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c004b90 con 0x7f98680ffbc0 2023-07-18T03:31:21.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.546+0000 7f986cdef700 1 -- 172.21.15.40:0/1979102791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986819dcd0 con 0x7f98680ffbc0 2023-07-18T03:31:21.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.547+0000 7f986cdef700 1 -- 172.21.15.40:0/1979102791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9868105f60 con 0x7f98680ffbc0 2023-07-18T03:31:21.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.548+0000 7f98577fe700 1 -- 172.21.15.40:0/1979102791 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f985c004d70 con 0x7f98680ffbc0 2023-07-18T03:31:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.548+0000 7f98577fe700 1 --2- 172.21.15.40:0/1979102791 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f98500a11a0 0x7f98500a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.549+0000 7f986659c700 1 --2- 172.21.15.40:0/1979102791 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f98500a11a0 0x7f98500a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.549+0000 7f98577fe700 1 -- 172.21.15.40:0/1979102791 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f985c0cdb80 con 0x7f98680ffbc0 2023-07-18T03:31:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.549+0000 7f986659c700 1 --2- 172.21.15.40:0/1979102791 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f98500a11a0 0x7f98500a3650 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f98680ffa20 tx=0x7f9858009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.557+0000 7f98577fe700 1 -- 172.21.15.40:0/1979102791 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f985c05a790 con 0x7f98680ffbc0 2023-07-18T03:31:21.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.817+0000 7f986cdef700 1 -- 172.21.15.40:0/1979102791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f986802c850 con 0x7f98680ffbc0 2023-07-18T03:31:21.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.818+0000 7f98577fe700 1 -- 172.21.15.40:0/1979102791 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10888 (secure 0 0 0) 0x7f985c09add0 con 0x7f98680ffbc0 2023-07-18T03:31:21.817 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:31:21.817 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:29:26.718789+0000","modified":"2023-07-18T03:29:34.650353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24523,"mds_2":24482,"mds_3":24452,"mds_4":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14601":{"gid":14601,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1818167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1818167554},{"type":"v1","addr":"172.21.15.40:6839","nonce":1818167554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14613":{"gid":14613,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/3953074960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":3953074960},{"type":"v1","addr":"172.21.15.40:6841","nonce":3953074960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3509655945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3509655945},{"type":"v1","addr":"172.21.15.134:6837","nonce":3509655945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/3007039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3007039043},{"type":"v1","addr":"172.21.15.64:6839","nonce":3007039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:31:21.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.821+0000 7f986cdef700 1 -- 172.21.15.40:0/1979102791 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f98500a11a0 msgr2=0x7f98500a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.821+0000 7f986cdef700 1 --2- 172.21.15.40:0/1979102791 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f98500a11a0 0x7f98500a3650 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f98680ffa20 tx=0x7f9858009260 comp rx=0 tx=0).stop 2023-07-18T03:31:21.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.821+0000 7f986cdef700 1 -- 172.21.15.40:0/1979102791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98680ffbc0 msgr2=0x7f986819a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.821+0000 7f986cdef700 1 --2- 172.21.15.40:0/1979102791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98680ffbc0 0x7f986819a540 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f985c00c010 tx=0x7f985c004730 comp rx=0 tx=0).stop 2023-07-18T03:31:21.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.821+0000 7f986cdef700 1 -- 172.21.15.40:0/1979102791 shutdown_connections 2023-07-18T03:31:21.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.821+0000 7f986cdef700 1 --2- 172.21.15.40:0/1979102791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9868100550 0x7f986819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.821+0000 7f986cdef700 1 --2- 172.21.15.40:0/1979102791 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f98500a11a0 0x7f98500a3650 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.821+0000 7f986cdef700 1 --2- 172.21.15.40:0/1979102791 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f98680fe9d0 0x7f9868068200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.821+0000 7f986cdef700 1 --2- 172.21.15.40:0/1979102791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98680ffbc0 0x7f986819a540 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.821+0000 7f986cdef700 1 -- 172.21.15.40:0/1979102791 >> 172.21.15.40:0/1979102791 conn(0x7f98680f9fd0 msgr2=0x7f98681030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:21.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.821+0000 7f986cdef700 1 -- 172.21.15.40:0/1979102791 shutdown_connections 2023-07-18T03:31:21.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.821+0000 7f986cdef700 1 -- 172.21.15.40:0/1979102791 wait complete. 2023-07-18T03:31:21.822 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 27 2023-07-18T03:31:21.832 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-18T03:31:21.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.965+0000 7fdc7b550700 1 -- 172.21.15.40:0/3208078839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc74105870 msgr2=0x7fdc74107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.966+0000 7fdc7b550700 1 --2- 172.21.15.40:0/3208078839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc74105870 0x7fdc74107c70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00b0a0 tx=0x7fdc6c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.966+0000 7fdc7b550700 1 -- 172.21.15.40:0/3208078839 shutdown_connections 2023-07-18T03:31:21.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.966+0000 7fdc7b550700 1 --2- 172.21.15.40:0/3208078839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc74105870 0x7fdc74107c70 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.966+0000 7fdc7b550700 1 --2- 172.21.15.40:0/3208078839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc74100630 0x7fdc74102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.966+0000 7fdc7b550700 1 --2- 172.21.15.40:0/3208078839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc74102f50 0x7fdc74105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.966+0000 7fdc7b550700 1 -- 172.21.15.40:0/3208078839 >> 172.21.15.40:0/3208078839 conn(0x7fdc740fa070 msgr2=0x7fdc740fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:21.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.966+0000 7fdc7b550700 1 -- 172.21.15.40:0/3208078839 shutdown_connections 2023-07-18T03:31:21.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.966+0000 7fdc7b550700 1 -- 172.21.15.40:0/3208078839 wait complete. 2023-07-18T03:31:21.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.966+0000 7fdc7b550700 1 Processor -- start 2023-07-18T03:31:21.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.966+0000 7fdc7b550700 1 -- start start 2023-07-18T03:31:21.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc7b550700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc74100630 0x7fdc741958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc7b550700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc74102f50 0x7fdc74195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc7b550700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc74105870 0x7fdc7419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc7b550700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7419d460 con 0x7fdc74105870 2023-07-18T03:31:21.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc7b550700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7419d5a0 con 0x7fdc74100630 2023-07-18T03:31:21.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc7b550700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7419d730 con 0x7fdc74102f50 2023-07-18T03:31:21.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc79aed700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc74105870 0x7fdc7419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc79aed700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc74105870 0x7fdc7419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58104/0 (socket says 172.21.15.40:58104) 2023-07-18T03:31:21.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc79aed700 1 -- 172.21.15.40:0/935408003 learned_addr learned my addr 172.21.15.40:0/935408003 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:21.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc78aeb700 1 --2- 172.21.15.40:0/935408003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc74102f50 0x7fdc74195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc792ec700 1 --2- 172.21.15.40:0/935408003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc74100630 0x7fdc741958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc79aed700 1 -- 172.21.15.40:0/935408003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc74102f50 msgr2=0x7fdc74195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc79aed700 1 --2- 172.21.15.40:0/935408003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc74102f50 0x7fdc74195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc79aed700 1 -- 172.21.15.40:0/935408003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc74100630 msgr2=0x7fdc741958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc79aed700 1 --2- 172.21.15.40:0/935408003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc74100630 0x7fdc741958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc79aed700 1 -- 172.21.15.40:0/935408003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c009d00 con 0x7fdc74105870 2023-07-18T03:31:21.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc79aed700 1 --2- 172.21.15.40:0/935408003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc74105870 0x7fdc7419ce90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00b780 tx=0x7fdc6c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:21.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.967+0000 7fdc6a7fc700 1 -- 172.21.15.40:0/935408003 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c00e040 con 0x7fdc74105870 2023-07-18T03:31:21.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.968+0000 7fdc7b550700 1 -- 172.21.15.40:0/935408003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc740fe4e0 con 0x7fdc74105870 2023-07-18T03:31:21.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.968+0000 7fdc6a7fc700 1 -- 172.21.15.40:0/935408003 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc6c004c50 con 0x7fdc74105870 2023-07-18T03:31:21.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.968+0000 7fdc6a7fc700 1 -- 172.21.15.40:0/935408003 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c005280 con 0x7fdc74105870 2023-07-18T03:31:21.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.968+0000 7fdc7b550700 1 -- 172.21.15.40:0/935408003 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc740fe9f0 con 0x7fdc74105870 2023-07-18T03:31:21.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.970+0000 7fdc6a7fc700 1 -- 172.21.15.40:0/935408003 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdc6c01d470 con 0x7fdc74105870 2023-07-18T03:31:21.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.971+0000 7fdc6a7fc700 1 --2- 172.21.15.40:0/935408003 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdc600a1150 0x7fdc600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.971+0000 7fdc792ec700 1 --2- 172.21.15.40:0/935408003 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdc600a1150 0x7fdc600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.971+0000 7fdc6a7fc700 1 -- 172.21.15.40:0/935408003 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fdc6c0cdec0 con 0x7fdc74105870 2023-07-18T03:31:21.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.971+0000 7fdc792ec700 1 --2- 172.21.15.40:0/935408003 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdc600a1150 0x7fdc600a3600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009c60 tx=0x7fdc70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:21.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.972+0000 7fdc876d6b80 1 Processor -- start 2023-07-18T03:31:21.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.972+0000 7fdc876d6b80 1 -- start start 2023-07-18T03:31:21.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.972+0000 7fdc876d6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0a4d0 0x55a828b0a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.972+0000 7fdc876d6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b0b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.972+0000 7fdc876d6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0bc70 0x55a828b0c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.972+0000 7fdc876d6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a828a0fcb0 con 0x55a828b0ae20 2023-07-18T03:31:21.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.972+0000 7fdc876d6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a828a62a40 con 0x55a828b0a4d0 2023-07-18T03:31:21.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.972+0000 7fdc876d6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a828a5bed0 con 0x55a828b0bc70 2023-07-18T03:31:21.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.972+0000 7fdc78aeb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b0b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.972+0000 7fdc78aeb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b0b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58110/0 (socket says 172.21.15.40:58110) 2023-07-18T03:31:21.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.972+0000 7fdc78aeb700 1 -- 172.21.15.40:0/1868301213 learned_addr learned my addr 172.21.15.40:0/1868301213 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:21.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.972+0000 7fdc792ec700 1 --2- 172.21.15.40:0/1868301213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0a4d0 0x55a828b0a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.973+0000 7fdc79aed700 1 --2- 172.21.15.40:0/1868301213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0bc70 0x55a828b0c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.973+0000 7fdc78aeb700 1 -- 172.21.15.40:0/1868301213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0bc70 msgr2=0x55a828b0c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:21.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.973+0000 7fdc78aeb700 1 --2- 172.21.15.40:0/1868301213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0bc70 0x55a828b0c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.973+0000 7fdc78aeb700 1 -- 172.21.15.40:0/1868301213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0a4d0 msgr2=0x55a828b0a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:21.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.973+0000 7fdc78aeb700 1 --2- 172.21.15.40:0/1868301213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0a4d0 0x55a828b0a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.973+0000 7fdc78aeb700 1 -- 172.21.15.40:0/1868301213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a828889de0 con 0x55a828b0ae20 2023-07-18T03:31:21.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.973+0000 7fdc78aeb700 1 --2- 172.21.15.40:0/1868301213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b0b6e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009d80 tx=0x7fdc64011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b0f643fb2c57a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:21.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.973+0000 7fdc5f7fe700 1 -- 172.21.15.40:0/1868301213 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc64004230 con 0x55a828b0ae20 2023-07-18T03:31:21.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.973+0000 7fdc5f7fe700 1 -- 172.21.15.40:0/1868301213 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc64004390 con 0x55a828b0ae20 2023-07-18T03:31:21.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.973+0000 7fdc5f7fe700 1 -- 172.21.15.40:0/1868301213 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc64019860 con 0x55a828b0ae20 2023-07-18T03:31:21.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.974+0000 7fdc876d6b80 1 -- 172.21.15.40:0/1868301213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 msgr2=0x55a828b0b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.974+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/1868301213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b0b6e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009d80 tx=0x7fdc64011720 comp rx=0 tx=0).stop 2023-07-18T03:31:21.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.974+0000 7fdc876d6b80 1 -- 172.21.15.40:0/1868301213 shutdown_connections 2023-07-18T03:31:21.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.974+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/1868301213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0bc70 0x55a828b0c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.974+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/1868301213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0a4d0 0x55a828b0a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.974+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/1868301213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b0b6e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.974+0000 7fdc876d6b80 1 -- 172.21.15.40:0/1868301213 >> 172.21.15.40:0/1868301213 conn(0x55a828affda0 msgr2=0x55a828b051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:21.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.974+0000 7fdc876d6b80 1 -- 172.21.15.40:0/1868301213 shutdown_connections 2023-07-18T03:31:21.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.974+0000 7fdc876d6b80 1 -- 172.21.15.40:0/1868301213 wait complete. 2023-07-18T03:31:21.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.974+0000 7fdc876d6b80 1 -- start start 2023-07-18T03:31:21.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.974+0000 7fdc876d6b80 1 Processor -- start 2023-07-18T03:31:21.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.975+0000 7fdc876d6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0a4d0 0x55a828b14110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.975+0000 7fdc876d6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b14650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.975+0000 7fdc876d6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0bc70 0x55a828b1b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.975+0000 7fdc876d6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8289fbac0 con 0x55a828b0ae20 2023-07-18T03:31:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.975+0000 7fdc876d6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8289fb7c0 con 0x55a828b0bc70 2023-07-18T03:31:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.975+0000 7fdc876d6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8288df920 con 0x55a828b0a4d0 2023-07-18T03:31:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.975+0000 7fdc876d6b80 10 client.0 _ll_register_callbacks cb 0x55a8289f9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.975+0000 7fdc78aeb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b14650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.975+0000 7fdc78aeb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b14650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58116/0 (socket says 172.21.15.40:58116) 2023-07-18T03:31:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.975+0000 7fdc78aeb700 1 -- 172.21.15.40:0/3947676802 learned_addr learned my addr 172.21.15.40:0/3947676802 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.975+0000 7fdc792ec700 1 --2- 172.21.15.40:0/3947676802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0a4d0 0x55a828b14110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.975+0000 7fdc79aed700 1 --2- 172.21.15.40:0/3947676802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0bc70 0x55a828b1b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.975+0000 7fdc78aeb700 1 -- 172.21.15.40:0/3947676802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0a4d0 msgr2=0x55a828b14110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:21.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.975+0000 7fdc78aeb700 1 --2- 172.21.15.40:0/3947676802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0a4d0 0x55a828b14110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.975+0000 7fdc78aeb700 1 -- 172.21.15.40:0/3947676802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0bc70 msgr2=0x55a828b1b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:21.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.975+0000 7fdc78aeb700 1 --2- 172.21.15.40:0/3947676802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0bc70 0x55a828b1b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.975+0000 7fdc78aeb700 1 -- 172.21.15.40:0/3947676802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc640099e0 con 0x55a828b0ae20 2023-07-18T03:31:21.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.976+0000 7fdc78aeb700 1 --2- 172.21.15.40:0/3947676802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b14650 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdc640095f0 tx=0x7fdc640039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd31d0914e242187 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:21.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.976+0000 7fdc5cff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:21.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.976+0000 7fdc5cff9700 1 -- 172.21.15.40:0/3947676802 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc64005450 con 0x55a828b0ae20 2023-07-18T03:31:21.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.976+0000 7fdc5cff9700 1 -- 172.21.15.40:0/3947676802 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fdc64004500 con 0x55a828b0ae20 2023-07-18T03:31:21.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.976+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3947676802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8289b54f0 con 0x55a828b0ae20 2023-07-18T03:31:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.976+0000 7fdc5cff9700 1 -- 172.21.15.40:0/3947676802 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc64003da0 con 0x55a828b0ae20 2023-07-18T03:31:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.976+0000 7fdc876d6b80 10 client.15522 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.976+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3947676802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a828872c60 con 0x55a828b0ae20 2023-07-18T03:31:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.976+0000 7fdc5cff9700 1 -- 172.21.15.40:0/3947676802 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a8289b54f0 con 0x55a828b0ae20 2023-07-18T03:31:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.976+0000 7fdc5cff9700 1 -- 172.21.15.40:0/3947676802 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdc64021040 con 0x55a828b0ae20 2023-07-18T03:31:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.976+0000 7fdc876d6b80 10 client.15522 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.976+0000 7fdc876d6b80 10 client.15522 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2023-07-18T03:31:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.977+0000 7fdc876d6b80 20 client.15522 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:21.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.977+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/3947676802 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55a828b1fe20 0x55a828b22230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:21.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.977+0000 7fdc876d6b80 4 client.15522 mds_command: new command op to 24442 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:21.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.977+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3947676802 --> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a828a54eb0 con 0x55a828b1fe20 2023-07-18T03:31:21.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.977+0000 7fdc792ec700 1 --2- 172.21.15.40:0/3947676802 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55a828b1fe20 0x55a828b22230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:21.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.977+0000 7fdc5cff9700 10 client.15522 ms_handle_connect on v2:172.21.15.134:6832/2345264306 2023-07-18T03:31:21.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.977+0000 7fdc792ec700 1 --2- 172.21.15.40:0/3947676802 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55a828b1fe20 0x55a828b22230 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c705d5a28405451c server_cookie=3bfed1cbbd04c9cf in_seq=0 out_seq=0 2023-07-18T03:31:21.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.978+0000 7fdc5cff9700 1 -- 172.21.15.40:0/3947676802 <== mds.4 v2:172.21.15.134:6832/2345264306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a828a54eb0 con 0x55a828b1fe20 2023-07-18T03:31:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.978+0000 7fdc5cff9700 10 client.15522 handle_command_reply: tid=0 2023-07-18T03:31:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.978+0000 7fdc876d6b80 1 client.15522 shutdown 2023-07-18T03:31:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.978+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3947676802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 msgr2=0x55a828b14650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.978+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/3947676802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b14650 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdc640095f0 tx=0x7fdc640039c0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.978+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3947676802 shutdown_connections 2023-07-18T03:31:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.978+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/3947676802 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55a828b1fe20 0x55a828b22230 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.978+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/3947676802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0bc70 0x55a828b1b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.978+0000 7fdc5cff9700 0 client.15522 ms_handle_reset on v2:172.21.15.134:6832/2345264306 2023-07-18T03:31:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.978+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/3947676802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0a4d0 0x55a828b14110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.978+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/3947676802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b14650 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.978+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3947676802 >> 172.21.15.40:0/3947676802 conn(0x55a828affda0 msgr2=0x55a828b048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.979+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3947676802 shutdown_connections 2023-07-18T03:31:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.979+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3947676802 wait complete. 2023-07-18T03:31:21.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:21.979+0000 7fdc876d6b80 20 client.15522 trim_cache size 0 max 16384 2023-07-18T03:31:22.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.005+0000 7fdc876d6b80 1 Processor -- start 2023-07-18T03:31:22.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.005+0000 7fdc876d6b80 1 -- start start 2023-07-18T03:31:22.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.005+0000 7fdc876d6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0a4d0 0x55a828b06270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.005+0000 7fdc876d6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0ae20 0x55a828b1b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.005+0000 7fdc876d6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0bc70 0x55a828b12080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.005+0000 7fdc876d6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a828b060e0 con 0x55a828b0a4d0 2023-07-18T03:31:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.005+0000 7fdc876d6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a828b14dc0 con 0x55a828b0bc70 2023-07-18T03:31:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.005+0000 7fdc876d6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a828b11e30 con 0x55a828b0ae20 2023-07-18T03:31:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.005+0000 7fdc792ec700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0a4d0 0x55a828b06270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.006+0000 7fdc792ec700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0a4d0 0x55a828b06270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58122/0 (socket says 172.21.15.40:58122) 2023-07-18T03:31:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.006+0000 7fdc792ec700 1 -- 172.21.15.40:0/3500940427 learned_addr learned my addr 172.21.15.40:0/3500940427 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.006+0000 7fdc78aeb700 1 --2- 172.21.15.40:0/3500940427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0ae20 0x55a828b1b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.006+0000 7fdc79aed700 1 --2- 172.21.15.40:0/3500940427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0bc70 0x55a828b12080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.006+0000 7fdc792ec700 1 -- 172.21.15.40:0/3500940427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0ae20 msgr2=0x55a828b1b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.006+0000 7fdc792ec700 1 --2- 172.21.15.40:0/3500940427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0ae20 0x55a828b1b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.006+0000 7fdc792ec700 1 -- 172.21.15.40:0/3500940427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0bc70 msgr2=0x55a828b12080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.006+0000 7fdc792ec700 1 --2- 172.21.15.40:0/3500940427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0bc70 0x55a828b12080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.006+0000 7fdc792ec700 1 -- 172.21.15.40:0/3500940427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8288b0250 con 0x55a828b0a4d0 2023-07-18T03:31:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.006+0000 7fdc792ec700 1 --2- 172.21.15.40:0/3500940427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0a4d0 0x55a828b06270 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdc70011ab0 tx=0x7fdc70014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34007d0a48213d2d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.006+0000 7fdc5ffff700 1 -- 172.21.15.40:0/3500940427 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70022070 con 0x55a828b0a4d0 2023-07-18T03:31:22.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.006+0000 7fdc5ffff700 1 -- 172.21.15.40:0/3500940427 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc70023040 con 0x55a828b0a4d0 2023-07-18T03:31:22.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.006+0000 7fdc5ffff700 1 -- 172.21.15.40:0/3500940427 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70029650 con 0x55a828b0a4d0 2023-07-18T03:31:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.007+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3500940427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0a4d0 msgr2=0x55a828b06270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.007+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/3500940427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0a4d0 0x55a828b06270 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdc70011ab0 tx=0x7fdc70014000 comp rx=0 tx=0).stop 2023-07-18T03:31:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.007+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3500940427 shutdown_connections 2023-07-18T03:31:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.007+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/3500940427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0bc70 0x55a828b12080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.007+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/3500940427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0a4d0 0x55a828b06270 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.007+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/3500940427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0ae20 0x55a828b1b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.007+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3500940427 >> 172.21.15.40:0/3500940427 conn(0x55a828affda0 msgr2=0x55a828b12e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.007+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3500940427 shutdown_connections 2023-07-18T03:31:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.007+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3500940427 wait complete. 2023-07-18T03:31:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.007+0000 7fdc876d6b80 1 -- start start 2023-07-18T03:31:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.007+0000 7fdc876d6b80 1 Processor -- start 2023-07-18T03:31:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.008+0000 7fdc876d6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0a4d0 0x55a828b17630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.008+0000 7fdc876d6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b17b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.008+0000 7fdc876d6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0bc70 0x55a828b1ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.008+0000 7fdc876d6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a828b1b0d0 con 0x55a828b0ae20 2023-07-18T03:31:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.008+0000 7fdc876d6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a828b1b240 con 0x55a828b0bc70 2023-07-18T03:31:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.008+0000 7fdc876d6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a828b1b400 con 0x55a828b0a4d0 2023-07-18T03:31:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.008+0000 7fdc876d6b80 10 client.0 _ll_register_callbacks cb 0x7fdc700096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.008+0000 7fdc78aeb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b17b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.008+0000 7fdc78aeb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b17b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58130/0 (socket says 172.21.15.40:58130) 2023-07-18T03:31:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.008+0000 7fdc78aeb700 1 -- 172.21.15.40:0/3855420933 learned_addr learned my addr 172.21.15.40:0/3855420933 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.008+0000 7fdc792ec700 1 --2- 172.21.15.40:0/3855420933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0a4d0 0x55a828b17630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.008+0000 7fdc79aed700 1 --2- 172.21.15.40:0/3855420933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0bc70 0x55a828b1ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.008+0000 7fdc78aeb700 1 -- 172.21.15.40:0/3855420933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0a4d0 msgr2=0x55a828b17630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.008+0000 7fdc78aeb700 1 --2- 172.21.15.40:0/3855420933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0a4d0 0x55a828b17630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.008+0000 7fdc78aeb700 1 -- 172.21.15.40:0/3855420933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0bc70 msgr2=0x55a828b1ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.008+0000 7fdc78aeb700 1 --2- 172.21.15.40:0/3855420933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0bc70 0x55a828b1ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.008+0000 7fdc78aeb700 1 -- 172.21.15.40:0/3855420933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc70017070 con 0x55a828b0ae20 2023-07-18T03:31:22.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.009+0000 7fdc78aeb700 1 --2- 172.21.15.40:0/3855420933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b17b70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdc64000c00 tx=0x7fdc64005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3294095ddf96865a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.009+0000 7fdc5cff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:22.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.009+0000 7fdc5cff9700 1 -- 172.21.15.40:0/3855420933 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc64003ce0 con 0x55a828b0ae20 2023-07-18T03:31:22.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.009+0000 7fdc5cff9700 1 -- 172.21.15.40:0/3855420933 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fdc64019890 con 0x55a828b0ae20 2023-07-18T03:31:22.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.009+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3855420933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc64004390 con 0x55a828b0ae20 2023-07-18T03:31:22.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.009+0000 7fdc5cff9700 1 -- 172.21.15.40:0/3855420933 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc64003e40 con 0x55a828b0ae20 2023-07-18T03:31:22.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.009+0000 7fdc876d6b80 10 client.15528 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:22.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.009+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3855420933 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a8288882b0 con 0x55a828b0ae20 2023-07-18T03:31:22.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.009+0000 7fdc5cff9700 1 -- 172.21.15.40:0/3855420933 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc64046080 con 0x55a828b0ae20 2023-07-18T03:31:22.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.009+0000 7fdc5cff9700 1 -- 172.21.15.40:0/3855420933 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdc64040030 con 0x55a828b0ae20 2023-07-18T03:31:22.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.009+0000 7fdc876d6b80 10 client.15528 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:22.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.009+0000 7fdc876d6b80 10 client.15528 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2023-07-18T03:31:22.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.009+0000 7fdc876d6b80 20 client.15528 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:22.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.009+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/3855420933 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55a828b74960 0x55a828b76d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.009+0000 7fdc876d6b80 4 client.15528 mds_command: new command op to 24442 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:31:22.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.009+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3855420933 --> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a828a17050 con 0x55a828b74960 2023-07-18T03:31:22.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.010+0000 7fdc792ec700 1 --2- 172.21.15.40:0/3855420933 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55a828b74960 0x55a828b76d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.010+0000 7fdc5cff9700 10 client.15528 ms_handle_connect on v2:172.21.15.134:6832/2345264306 2023-07-18T03:31:22.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.010+0000 7fdc792ec700 1 --2- 172.21.15.40:0/3855420933 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55a828b74960 0x55a828b76d70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8685603a22cdd44b server_cookie=d6d5c76c7edb97b8 in_seq=0 out_seq=0 2023-07-18T03:31:22.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.010+0000 7fdc5cff9700 1 -- 172.21.15.40:0/3855420933 <== mds.4 v2:172.21.15.134:6832/2345264306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a828a17050 con 0x55a828b74960 2023-07-18T03:31:22.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.010+0000 7fdc5cff9700 10 client.15528 handle_command_reply: tid=0 2023-07-18T03:31:22.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.010+0000 7fdc876d6b80 1 client.15528 shutdown 2023-07-18T03:31:22.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.010+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3855420933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 msgr2=0x55a828b17b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.010+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/3855420933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b17b70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdc64000c00 tx=0x7fdc64005e30 comp rx=0 tx=0).stop 2023-07-18T03:31:22.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.010+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3855420933 shutdown_connections 2023-07-18T03:31:22.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.010+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/3855420933 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55a828b74960 0x55a828b76d70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.010+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/3855420933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a828b0bc70 0x55a828b1ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.010+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/3855420933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a828b0a4d0 0x55a828b17630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.010+0000 7fdc876d6b80 1 --2- 172.21.15.40:0/3855420933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a828b0ae20 0x55a828b17b70 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.010+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3855420933 >> 172.21.15.40:0/3855420933 conn(0x55a828affda0 msgr2=0x55a828b12080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.010+0000 7fdc5cff9700 0 client.15528 ms_handle_reset on v2:172.21.15.134:6832/2345264306 2023-07-18T03:31:22.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.011+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3855420933 shutdown_connections 2023-07-18T03:31:22.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.011+0000 7fdc876d6b80 1 -- 172.21.15.40:0/3855420933 wait complete. 2023-07-18T03:31:22.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.011+0000 7fdc876d6b80 20 client.15528 trim_cache size 0 max 16384 2023-07-18T03:31:22.018 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:31:22.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.011+0000 7fdc7b550700 1 -- 172.21.15.40:0/935408003 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdc600a1150 msgr2=0x7fdc600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.011+0000 7fdc7b550700 1 --2- 172.21.15.40:0/935408003 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdc600a1150 0x7fdc600a3600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009c60 tx=0x7fdc70009260 comp rx=0 tx=0).stop 2023-07-18T03:31:22.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.011+0000 7fdc7b550700 1 -- 172.21.15.40:0/935408003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc74105870 msgr2=0x7fdc7419ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.011+0000 7fdc7b550700 1 --2- 172.21.15.40:0/935408003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc74105870 0x7fdc7419ce90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00b780 tx=0x7fdc6c00bab0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.011+0000 7fdc7b550700 1 -- 172.21.15.40:0/935408003 shutdown_connections 2023-07-18T03:31:22.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.011+0000 7fdc7b550700 1 --2- 172.21.15.40:0/935408003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc74105870 0x7fdc7419ce90 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.011+0000 7fdc7b550700 1 --2- 172.21.15.40:0/935408003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc74100630 0x7fdc741958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.011+0000 7fdc7b550700 1 --2- 172.21.15.40:0/935408003 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdc600a1150 0x7fdc600a3600 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.011+0000 7fdc7b550700 1 --2- 172.21.15.40:0/935408003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc74102f50 0x7fdc74195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.011+0000 7fdc7b550700 1 -- 172.21.15.40:0/935408003 >> 172.21.15.40:0/935408003 conn(0x7fdc740fa070 msgr2=0x7fdc74101290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.011+0000 7fdc7b550700 1 -- 172.21.15.40:0/935408003 shutdown_connections 2023-07-18T03:31:22.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.011+0000 7fdc7b550700 1 -- 172.21.15.40:0/935408003 wait complete. 2023-07-18T03:31:22.021 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-18T03:31:22.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:21 smithi134 ceph-mon[98216]: pgmap v307: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-18T03:31:22.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:21 smithi040 ceph-mon[96439]: pgmap v307: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-18T03:31:22.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:21 smithi064 ceph-mon[98059]: pgmap v307: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-07-18T03:31:22.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.156+0000 7ff4556e3700 1 -- 172.21.15.40:0/1517110288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4501005c0 msgr2=0x7ff4501029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.156+0000 7ff4556e3700 1 --2- 172.21.15.40:0/1517110288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4501005c0 0x7ff4501029a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff440009990 tx=0x7ff440009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.156+0000 7ff4556e3700 1 -- 172.21.15.40:0/1517110288 shutdown_connections 2023-07-18T03:31:22.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.156+0000 7ff4556e3700 1 --2- 172.21.15.40:0/1517110288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff450105800 0x7ff450107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.156+0000 7ff4556e3700 1 --2- 172.21.15.40:0/1517110288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4501005c0 0x7ff4501029a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.156+0000 7ff4556e3700 1 --2- 172.21.15.40:0/1517110288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff450102ee0 0x7ff4501052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.156+0000 7ff4556e3700 1 -- 172.21.15.40:0/1517110288 >> 172.21.15.40:0/1517110288 conn(0x7ff4500fa000 msgr2=0x7ff4500fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.156+0000 7ff4556e3700 1 -- 172.21.15.40:0/1517110288 shutdown_connections 2023-07-18T03:31:22.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.156+0000 7ff4556e3700 1 -- 172.21.15.40:0/1517110288 wait complete. 2023-07-18T03:31:22.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.157+0000 7ff4556e3700 1 Processor -- start 2023-07-18T03:31:22.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.157+0000 7ff4556e3700 1 -- start start 2023-07-18T03:31:22.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.157+0000 7ff4556e3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4501005c0 0x7ff4500fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.157+0000 7ff4556e3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff450102ee0 0x7ff4500fe570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.157+0000 7ff4556e3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff450105800 0x7ff4500feab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.157+0000 7ff4556e3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4500ff250 con 0x7ff4501005c0 2023-07-18T03:31:22.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.157+0000 7ff4556e3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4500ff3c0 con 0x7ff450102ee0 2023-07-18T03:31:22.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.157+0000 7ff4556e3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45019f870 con 0x7ff450105800 2023-07-18T03:31:22.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.157+0000 7ff44effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4501005c0 0x7ff4500fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.157+0000 7ff44effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4501005c0 0x7ff4500fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58142/0 (socket says 172.21.15.40:58142) 2023-07-18T03:31:22.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.157+0000 7ff44effd700 1 -- 172.21.15.40:0/1461153025 learned_addr learned my addr 172.21.15.40:0/1461153025 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.157+0000 7ff44e7fc700 1 --2- 172.21.15.40:0/1461153025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff450102ee0 0x7ff4500fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.158+0000 7ff44f7fe700 1 --2- 172.21.15.40:0/1461153025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff450105800 0x7ff4500feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.158+0000 7ff44effd700 1 -- 172.21.15.40:0/1461153025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff450105800 msgr2=0x7ff4500feab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.158+0000 7ff44effd700 1 --2- 172.21.15.40:0/1461153025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff450105800 0x7ff4500feab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.158+0000 7ff44effd700 1 -- 172.21.15.40:0/1461153025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff450102ee0 msgr2=0x7ff4500fe570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.158+0000 7ff44effd700 1 --2- 172.21.15.40:0/1461153025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff450102ee0 0x7ff4500fe570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.158+0000 7ff44effd700 1 -- 172.21.15.40:0/1461153025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4400095f0 con 0x7ff4501005c0 2023-07-18T03:31:22.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.158+0000 7ff44effd700 1 --2- 172.21.15.40:0/1461153025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4501005c0 0x7ff4500fff30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff44000f690 tx=0x7ff44000f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.158+0000 7ff437fff700 1 -- 172.21.15.40:0/1461153025 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44001d070 con 0x7ff4501005c0 2023-07-18T03:31:22.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.158+0000 7ff437fff700 1 -- 172.21.15.40:0/1461153025 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff440003960 con 0x7ff4501005c0 2023-07-18T03:31:22.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.158+0000 7ff4556e3700 1 -- 172.21.15.40:0/1461153025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45019faf0 con 0x7ff4501005c0 2023-07-18T03:31:22.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.158+0000 7ff437fff700 1 -- 172.21.15.40:0/1461153025 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff440005630 con 0x7ff4501005c0 2023-07-18T03:31:22.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.158+0000 7ff4556e3700 1 -- 172.21.15.40:0/1461153025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45019ffa0 con 0x7ff4501005c0 2023-07-18T03:31:22.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.160+0000 7ff437fff700 1 -- 172.21.15.40:0/1461153025 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff440003ad0 con 0x7ff4501005c0 2023-07-18T03:31:22.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.161+0000 7ff437fff700 1 --2- 172.21.15.40:0/1461153025 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4380a1150 0x7ff4380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.161+0000 7ff44e7fc700 1 --2- 172.21.15.40:0/1461153025 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4380a1150 0x7ff4380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.161+0000 7ff437fff700 1 -- 172.21.15.40:0/1461153025 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff4400cdba0 con 0x7ff4501005c0 2023-07-18T03:31:22.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.162+0000 7ff44e7fc700 1 --2- 172.21.15.40:0/1461153025 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4380a1150 0x7ff4380a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff444005950 tx=0x7ff44400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.162+0000 7ff461869b80 1 Processor -- start 2023-07-18T03:31:22.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.162+0000 7ff461869b80 1 -- start start 2023-07-18T03:31:22.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.162+0000 7ff461869b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6f4d0 0x559043f6f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.162+0000 7ff461869b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6fe20 0x559043f706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.162+0000 7ff461869b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f70c70 0x559043f710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.162+0000 7ff461869b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x559043e74cb0 con 0x559043f70c70 2023-07-18T03:31:22.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.163+0000 7ff461869b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559043ec7a40 con 0x559043f6f4d0 2023-07-18T03:31:22.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.163+0000 7ff461869b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559043ec0ed0 con 0x559043f6fe20 2023-07-18T03:31:22.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.163+0000 7ff44f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f70c70 0x559043f710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.163+0000 7ff44f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f70c70 0x559043f710e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58154/0 (socket says 172.21.15.40:58154) 2023-07-18T03:31:22.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.163+0000 7ff44f7fe700 1 -- 172.21.15.40:0/500008921 learned_addr learned my addr 172.21.15.40:0/500008921 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.163+0000 7ff44effd700 1 --2- 172.21.15.40:0/500008921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6f4d0 0x559043f6f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.163+0000 7ff44e7fc700 1 --2- 172.21.15.40:0/500008921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6fe20 0x559043f706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.163+0000 7ff44effd700 1 -- 172.21.15.40:0/500008921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6fe20 msgr2=0x559043f706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.163+0000 7ff44effd700 1 --2- 172.21.15.40:0/500008921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6fe20 0x559043f706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.163+0000 7ff44effd700 1 -- 172.21.15.40:0/500008921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f70c70 msgr2=0x559043f710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.163+0000 7ff44effd700 1 --2- 172.21.15.40:0/500008921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f70c70 0x559043f710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.163+0000 7ff44effd700 1 -- 172.21.15.40:0/500008921 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559043ceede0 con 0x559043f6f4d0 2023-07-18T03:31:22.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.163+0000 7ff44f7fe700 1 --2- 172.21.15.40:0/500008921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f70c70 0x559043f710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:22.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.163+0000 7ff44effd700 1 --2- 172.21.15.40:0/500008921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6f4d0 0x559043f6f8e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff440009540 tx=0x7ff440009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55aca6e19d4f0029 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.163+0000 7ff4357fa700 1 -- 172.21.15.40:0/500008921 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44009b020 con 0x559043f6f4d0 2023-07-18T03:31:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.164+0000 7ff4357fa700 1 -- 172.21.15.40:0/500008921 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff44009f050 con 0x559043f6f4d0 2023-07-18T03:31:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.164+0000 7ff4357fa700 1 -- 172.21.15.40:0/500008921 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44009a510 con 0x559043f6f4d0 2023-07-18T03:31:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.164+0000 7ff461869b80 1 -- 172.21.15.40:0/500008921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6f4d0 msgr2=0x559043f6f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.164+0000 7ff461869b80 1 --2- 172.21.15.40:0/500008921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6f4d0 0x559043f6f8e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff440009540 tx=0x7ff440009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.164+0000 7ff461869b80 1 -- 172.21.15.40:0/500008921 shutdown_connections 2023-07-18T03:31:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.164+0000 7ff461869b80 1 --2- 172.21.15.40:0/500008921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f70c70 0x559043f710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.164+0000 7ff461869b80 1 --2- 172.21.15.40:0/500008921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6f4d0 0x559043f6f8e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.164+0000 7ff461869b80 1 --2- 172.21.15.40:0/500008921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6fe20 0x559043f706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.164+0000 7ff461869b80 1 -- 172.21.15.40:0/500008921 >> 172.21.15.40:0/500008921 conn(0x559043f64da0 msgr2=0x559043f6a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.164+0000 7ff461869b80 1 -- 172.21.15.40:0/500008921 shutdown_connections 2023-07-18T03:31:22.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.164+0000 7ff461869b80 1 -- 172.21.15.40:0/500008921 wait complete. 2023-07-18T03:31:22.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.165+0000 7ff461869b80 1 -- start start 2023-07-18T03:31:22.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.165+0000 7ff461869b80 1 Processor -- start 2023-07-18T03:31:22.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.165+0000 7ff461869b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6f4d0 0x559043f78da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.165+0000 7ff461869b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6fe20 0x559043f792e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.165+0000 7ff461869b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f70c70 0x559043f80320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.165+0000 7ff461869b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x559043e60ac0 con 0x559043f70c70 2023-07-18T03:31:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.165+0000 7ff461869b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559043e607c0 con 0x559043f6fe20 2023-07-18T03:31:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.165+0000 7ff461869b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559043d44920 con 0x559043f6f4d0 2023-07-18T03:31:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.165+0000 7ff461869b80 10 client.0 _ll_register_callbacks cb 0x559043e5e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.165+0000 7ff44e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6fe20 0x559043f792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.165+0000 7ff44e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6fe20 0x559043f792e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:41484/0 (socket says 172.21.15.40:41484) 2023-07-18T03:31:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.165+0000 7ff44e7fc700 1 -- 172.21.15.40:0/208285723 learned_addr learned my addr 172.21.15.40:0/208285723 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.165+0000 7ff44f7fe700 1 --2- 172.21.15.40:0/208285723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f70c70 0x559043f80320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.165+0000 7ff44effd700 1 --2- 172.21.15.40:0/208285723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6f4d0 0x559043f78da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.166+0000 7ff44e7fc700 1 -- 172.21.15.40:0/208285723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6f4d0 msgr2=0x559043f78da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.166+0000 7ff44e7fc700 1 --2- 172.21.15.40:0/208285723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6f4d0 0x559043f78da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.166+0000 7ff44e7fc700 1 -- 172.21.15.40:0/208285723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f70c70 msgr2=0x559043f80320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.166+0000 7ff44e7fc700 1 --2- 172.21.15.40:0/208285723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f70c70 0x559043f80320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.166+0000 7ff44e7fc700 1 -- 172.21.15.40:0/208285723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4480099a0 con 0x559043f6fe20 2023-07-18T03:31:22.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.166+0000 7ff44f7fe700 1 --2- 172.21.15.40:0/208285723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f70c70 0x559043f80320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.166+0000 7ff44e7fc700 1 --2- 172.21.15.40:0/208285723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6fe20 0x559043f792e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff4440116f0 tx=0x7ff4440047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aed57bf44826b2b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.166+0000 7ff432ff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:31:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.166+0000 7ff432ff5700 1 -- 172.21.15.40:0/208285723 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444010ee0 con 0x559043f6fe20 2023-07-18T03:31:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.166+0000 7ff461869b80 1 -- 172.21.15.40:0/208285723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559043e1a4f0 con 0x559043f6fe20 2023-07-18T03:31:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.166+0000 7ff432ff5700 1 -- 172.21.15.40:0/208285723 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff444017070 con 0x559043f6fe20 2023-07-18T03:31:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.166+0000 7ff432ff5700 1 -- 172.21.15.40:0/208285723 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff444028470 con 0x559043f6fe20 2023-07-18T03:31:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.166+0000 7ff461869b80 10 client.25247 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.166+0000 7ff461869b80 1 -- 172.21.15.40:0/208285723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7ff45019ffa0 con 0x559043f6fe20 2023-07-18T03:31:22.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.167+0000 7ff432ff5700 1 -- 172.21.15.40:0/208285723 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559043e1a4f0 con 0x559043f6fe20 2023-07-18T03:31:22.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.167+0000 7ff432ff5700 1 -- 172.21.15.40:0/208285723 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff444031050 con 0x559043f6fe20 2023-07-18T03:31:22.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.167+0000 7ff461869b80 10 client.25247 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:22.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.167+0000 7ff461869b80 10 client.25247 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-18T03:31:22.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.167+0000 7ff461869b80 20 client.25247 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:22.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.167+0000 7ff461869b80 1 --2- 172.21.15.40:0/208285723 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x559043f84bd0 0x559043f86fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.167+0000 7ff461869b80 4 client.25247 mds_command: new command op to 24452 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.167+0000 7ff461869b80 1 -- 172.21.15.40:0/208285723 --> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559043eb9eb0 con 0x559043f84bd0 2023-07-18T03:31:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.167+0000 7ff44effd700 1 --2- 172.21.15.40:0/208285723 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x559043f84bd0 0x559043f86fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.168+0000 7ff432ff5700 10 client.25247 ms_handle_connect on v2:172.21.15.64:6834/1864495009 2023-07-18T03:31:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.168+0000 7ff44effd700 1 --2- 172.21.15.40:0/208285723 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x559043f84bd0 0x559043f86fe0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c3d662e6d097e37f server_cookie=6c12db0de38d3326 in_seq=0 out_seq=0 2023-07-18T03:31:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.168+0000 7ff432ff5700 1 -- 172.21.15.40:0/208285723 <== mds.3 v2:172.21.15.64:6834/1864495009 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559043eb9eb0 con 0x559043f84bd0 2023-07-18T03:31:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.168+0000 7ff432ff5700 10 client.25247 handle_command_reply: tid=0 2023-07-18T03:31:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.168+0000 7ff461869b80 1 client.25247 shutdown 2023-07-18T03:31:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.168+0000 7ff461869b80 1 -- 172.21.15.40:0/208285723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6fe20 msgr2=0x559043f792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.168+0000 7ff461869b80 1 --2- 172.21.15.40:0/208285723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6fe20 0x559043f792e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff4440116f0 tx=0x7ff4440047c0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.168+0000 7ff461869b80 1 -- 172.21.15.40:0/208285723 shutdown_connections 2023-07-18T03:31:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.168+0000 7ff461869b80 1 --2- 172.21.15.40:0/208285723 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x559043f84bd0 0x559043f86fe0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.168+0000 7ff461869b80 1 --2- 172.21.15.40:0/208285723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f70c70 0x559043f80320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.169+0000 7ff461869b80 1 --2- 172.21.15.40:0/208285723 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6f4d0 0x559043f78da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.169+0000 7ff461869b80 1 --2- 172.21.15.40:0/208285723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6fe20 0x559043f792e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.168+0000 7ff432ff5700 0 client.25247 ms_handle_reset on v2:172.21.15.64:6834/1864495009 2023-07-18T03:31:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.169+0000 7ff461869b80 1 -- 172.21.15.40:0/208285723 >> 172.21.15.40:0/208285723 conn(0x559043f64da0 msgr2=0x559043f698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.169+0000 7ff461869b80 1 -- 172.21.15.40:0/208285723 shutdown_connections 2023-07-18T03:31:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.169+0000 7ff461869b80 1 -- 172.21.15.40:0/208285723 wait complete. 2023-07-18T03:31:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.169+0000 7ff461869b80 20 client.25247 trim_cache size 0 max 16384 2023-07-18T03:31:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.182+0000 7ff461869b80 1 Processor -- start 2023-07-18T03:31:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.182+0000 7ff461869b80 1 -- start start 2023-07-18T03:31:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.182+0000 7ff461869b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f64da0 0x559043f78da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.182+0000 7ff461869b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f6f4d0 0x559043f792e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.182+0000 7ff461869b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6fe20 0x559043f80240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.182+0000 7ff461869b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x559043f79cd0 con 0x559043f6f4d0 2023-07-18T03:31:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.182+0000 7ff461869b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559043f78280 con 0x559043f64da0 2023-07-18T03:31:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.182+0000 7ff461869b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559043f64970 con 0x559043f6fe20 2023-07-18T03:31:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.182+0000 7ff44e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f6f4d0 0x559043f792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.182+0000 7ff44e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f6f4d0 0x559043f792e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58166/0 (socket says 172.21.15.40:58166) 2023-07-18T03:31:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.182+0000 7ff44effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f64da0 0x559043f78da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.182+0000 7ff44e7fc700 1 -- 172.21.15.40:0/601267979 learned_addr learned my addr 172.21.15.40:0/601267979 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.182+0000 7ff44f7fe700 1 --2- 172.21.15.40:0/601267979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6fe20 0x559043f80240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.183+0000 7ff44e7fc700 1 -- 172.21.15.40:0/601267979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6fe20 msgr2=0x559043f80240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.183+0000 7ff44e7fc700 1 --2- 172.21.15.40:0/601267979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6fe20 0x559043f80240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.183+0000 7ff44e7fc700 1 -- 172.21.15.40:0/601267979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f64da0 msgr2=0x559043f78da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.183+0000 7ff44e7fc700 1 --2- 172.21.15.40:0/601267979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f64da0 0x559043f78da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.183+0000 7ff44e7fc700 1 -- 172.21.15.40:0/601267979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559043cd7c60 con 0x559043f6f4d0 2023-07-18T03:31:22.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.183+0000 7ff44effd700 1 --2- 172.21.15.40:0/601267979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f64da0 0x559043f78da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:22.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.183+0000 7ff44e7fc700 1 --2- 172.21.15.40:0/601267979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f6f4d0 0x559043f792e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff444000c00 tx=0x7ff44401fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e42ac639d7ecaaa server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.183+0000 7ff435ffb700 1 -- 172.21.15.40:0/601267979 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444038070 con 0x559043f6f4d0 2023-07-18T03:31:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.183+0000 7ff435ffb700 1 -- 172.21.15.40:0/601267979 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff444015070 con 0x559043f6f4d0 2023-07-18T03:31:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.183+0000 7ff435ffb700 1 -- 172.21.15.40:0/601267979 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444028b90 con 0x559043f6f4d0 2023-07-18T03:31:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.184+0000 7ff461869b80 1 -- 172.21.15.40:0/601267979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f6f4d0 msgr2=0x559043f792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.184+0000 7ff461869b80 1 --2- 172.21.15.40:0/601267979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f6f4d0 0x559043f792e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff444000c00 tx=0x7ff44401fa20 comp rx=0 tx=0).stop 2023-07-18T03:31:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.184+0000 7ff461869b80 1 -- 172.21.15.40:0/601267979 shutdown_connections 2023-07-18T03:31:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.184+0000 7ff461869b80 1 --2- 172.21.15.40:0/601267979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6fe20 0x559043f80240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.184+0000 7ff461869b80 1 --2- 172.21.15.40:0/601267979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f64da0 0x559043f78da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.184+0000 7ff461869b80 1 --2- 172.21.15.40:0/601267979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f6f4d0 0x559043f792e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.184+0000 7ff461869b80 1 -- 172.21.15.40:0/601267979 >> 172.21.15.40:0/601267979 conn(0x559043f77e50 msgr2=0x559043f76fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.184+0000 7ff461869b80 1 -- 172.21.15.40:0/601267979 shutdown_connections 2023-07-18T03:31:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.184+0000 7ff461869b80 1 -- 172.21.15.40:0/601267979 wait complete. 2023-07-18T03:31:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.184+0000 7ff461869b80 1 -- start start 2023-07-18T03:31:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.184+0000 7ff461869b80 1 Processor -- start 2023-07-18T03:31:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.184+0000 7ff461869b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f64da0 0x559043f7fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff461869b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6f4d0 0x559043f80170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff461869b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6fe20 0x559043f7c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff461869b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x559043f7cca0 con 0x559043f64da0 2023-07-18T03:31:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff461869b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559043f7cde0 con 0x559043f6fe20 2023-07-18T03:31:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff461869b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559043f7cfa0 con 0x559043f6f4d0 2023-07-18T03:31:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff461869b80 10 client.0 _ll_register_callbacks cb 0x7ff444010d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff44effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f64da0 0x559043f7fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff44effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f64da0 0x559043f7fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58170/0 (socket says 172.21.15.40:58170) 2023-07-18T03:31:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff44effd700 1 -- 172.21.15.40:0/2667703305 learned_addr learned my addr 172.21.15.40:0/2667703305 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff44e7fc700 1 --2- 172.21.15.40:0/2667703305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6f4d0 0x559043f80170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff44f7fe700 1 --2- 172.21.15.40:0/2667703305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6fe20 0x559043f7c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff44effd700 1 -- 172.21.15.40:0/2667703305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6f4d0 msgr2=0x559043f80170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff44effd700 1 --2- 172.21.15.40:0/2667703305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6f4d0 0x559043f80170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff44effd700 1 -- 172.21.15.40:0/2667703305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6fe20 msgr2=0x559043f7c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff44effd700 1 --2- 172.21.15.40:0/2667703305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6fe20 0x559043f7c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff44effd700 1 -- 172.21.15.40:0/2667703305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559043ceede0 con 0x559043f64da0 2023-07-18T03:31:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff44effd700 1 --2- 172.21.15.40:0/2667703305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f64da0 0x559043f7fc30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff4400cb6d0 tx=0x7ff440097490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70de33ff54d88260 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff432ff5700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff432ff5700 1 -- 172.21.15.40:0/2667703305 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff440099440 con 0x559043f64da0 2023-07-18T03:31:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff432ff5700 1 -- 172.21.15.40:0/2667703305 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff44009b020 con 0x559043f64da0 2023-07-18T03:31:22.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.185+0000 7ff461869b80 1 -- 172.21.15.40:0/2667703305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff44009f050 con 0x559043f64da0 2023-07-18T03:31:22.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.186+0000 7ff432ff5700 1 -- 172.21.15.40:0/2667703305 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4400a33e0 con 0x559043f64da0 2023-07-18T03:31:22.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.186+0000 7ff461869b80 10 client.15546 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:22.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.186+0000 7ff461869b80 1 -- 172.21.15.40:0/2667703305 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7ff45019ffa0 con 0x559043f64da0 2023-07-18T03:31:22.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.186+0000 7ff432ff5700 1 -- 172.21.15.40:0/2667703305 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff44009f050 con 0x559043f64da0 2023-07-18T03:31:22.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.186+0000 7ff432ff5700 1 -- 172.21.15.40:0/2667703305 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff4400a0070 con 0x559043f64da0 2023-07-18T03:31:22.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.186+0000 7ff461869b80 10 client.15546 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:22.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.186+0000 7ff461869b80 10 client.15546 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-18T03:31:22.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.186+0000 7ff461869b80 20 client.15546 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:22.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.186+0000 7ff461869b80 1 --2- 172.21.15.40:0/2667703305 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x559043f70c70 0x559043fdbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.186+0000 7ff461869b80 4 client.15546 mds_command: new command op to 24452 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:31:22.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.186+0000 7ff461869b80 1 -- 172.21.15.40:0/2667703305 --> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559043e7c050 con 0x559043f70c70 2023-07-18T03:31:22.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.186+0000 7ff44e7fc700 1 --2- 172.21.15.40:0/2667703305 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x559043f70c70 0x559043fdbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.187+0000 7ff432ff5700 10 client.15546 ms_handle_connect on v2:172.21.15.64:6834/1864495009 2023-07-18T03:31:22.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.187+0000 7ff44e7fc700 1 --2- 172.21.15.40:0/2667703305 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x559043f70c70 0x559043fdbb40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6f251b901dd9341 server_cookie=e5e888637306153a in_seq=0 out_seq=0 2023-07-18T03:31:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.187+0000 7ff432ff5700 1 -- 172.21.15.40:0/2667703305 <== mds.3 v2:172.21.15.64:6834/1864495009 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559043e7c050 con 0x559043f70c70 2023-07-18T03:31:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.187+0000 7ff432ff5700 10 client.15546 handle_command_reply: tid=0 2023-07-18T03:31:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.187+0000 7ff461869b80 1 client.15546 shutdown 2023-07-18T03:31:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.187+0000 7ff461869b80 1 -- 172.21.15.40:0/2667703305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f64da0 msgr2=0x559043f7fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.187+0000 7ff461869b80 1 --2- 172.21.15.40:0/2667703305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f64da0 0x559043f7fc30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff4400cb6d0 tx=0x7ff440097490 comp rx=0 tx=0).stop 2023-07-18T03:31:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.187+0000 7ff461869b80 1 -- 172.21.15.40:0/2667703305 shutdown_connections 2023-07-18T03:31:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.187+0000 7ff461869b80 1 --2- 172.21.15.40:0/2667703305 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x559043f70c70 0x559043fdbb40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.187+0000 7ff461869b80 1 --2- 172.21.15.40:0/2667703305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559043f6fe20 0x559043f7c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.187+0000 7ff432ff5700 0 client.15546 ms_handle_reset on v2:172.21.15.64:6834/1864495009 2023-07-18T03:31:22.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.187+0000 7ff461869b80 1 --2- 172.21.15.40:0/2667703305 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559043f64da0 0x559043f7fc30 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.187+0000 7ff461869b80 1 --2- 172.21.15.40:0/2667703305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559043f6f4d0 0x559043f80170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.187+0000 7ff461869b80 1 -- 172.21.15.40:0/2667703305 >> 172.21.15.40:0/2667703305 conn(0x559043f77e50 msgr2=0x559043f73ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.187+0000 7ff461869b80 1 -- 172.21.15.40:0/2667703305 shutdown_connections 2023-07-18T03:31:22.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.187+0000 7ff461869b80 1 -- 172.21.15.40:0/2667703305 wait complete. 2023-07-18T03:31:22.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.187+0000 7ff461869b80 20 client.15546 trim_cache size 0 max 16384 2023-07-18T03:31:22.195 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:31:22.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.188+0000 7ff4556e3700 1 -- 172.21.15.40:0/1461153025 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4380a1150 msgr2=0x7ff4380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.188+0000 7ff4556e3700 1 --2- 172.21.15.40:0/1461153025 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4380a1150 0x7ff4380a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff444005950 tx=0x7ff44400b410 comp rx=0 tx=0).stop 2023-07-18T03:31:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.188+0000 7ff4556e3700 1 -- 172.21.15.40:0/1461153025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4501005c0 msgr2=0x7ff4500fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.188+0000 7ff4556e3700 1 --2- 172.21.15.40:0/1461153025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4501005c0 0x7ff4500fff30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff44000f690 tx=0x7ff44000f6c0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.188+0000 7ff4556e3700 1 -- 172.21.15.40:0/1461153025 shutdown_connections 2023-07-18T03:31:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.188+0000 7ff4556e3700 1 --2- 172.21.15.40:0/1461153025 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4380a1150 0x7ff4380a3600 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.188+0000 7ff4556e3700 1 --2- 172.21.15.40:0/1461153025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff450105800 0x7ff4500feab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.188+0000 7ff4556e3700 1 --2- 172.21.15.40:0/1461153025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4501005c0 0x7ff4500fff30 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.188+0000 7ff4556e3700 1 --2- 172.21.15.40:0/1461153025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff450102ee0 0x7ff4500fe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.188+0000 7ff4556e3700 1 -- 172.21.15.40:0/1461153025 >> 172.21.15.40:0/1461153025 conn(0x7ff4500fa000 msgr2=0x7ff450101220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.188+0000 7ff4556e3700 1 -- 172.21.15.40:0/1461153025 shutdown_connections 2023-07-18T03:31:22.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.188+0000 7ff4556e3700 1 -- 172.21.15.40:0/1461153025 wait complete. 2023-07-18T03:31:22.198 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:31:22.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.332+0000 7f4dd3ab1700 1 -- 172.21.15.40:0/917926282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc105820 msgr2=0x7f4dcc107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.332+0000 7f4dd3ab1700 1 --2- 172.21.15.40:0/917926282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc105820 0x7f4dcc107c00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400b0a0 tx=0x7f4dc400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.332+0000 7f4dd3ab1700 1 -- 172.21.15.40:0/917926282 shutdown_connections 2023-07-18T03:31:22.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.332+0000 7f4dd3ab1700 1 --2- 172.21.15.40:0/917926282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc105820 0x7f4dcc107c00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.332+0000 7f4dd3ab1700 1 --2- 172.21.15.40:0/917926282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dcc1005e0 0x7f4dcc1029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.332+0000 7f4dd3ab1700 1 --2- 172.21.15.40:0/917926282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dcc102f00 0x7f4dcc1052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.332+0000 7f4dd3ab1700 1 -- 172.21.15.40:0/917926282 >> 172.21.15.40:0/917926282 conn(0x7f4dcc0fa000 msgr2=0x7f4dcc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.332+0000 7f4dd3ab1700 1 -- 172.21.15.40:0/917926282 shutdown_connections 2023-07-18T03:31:22.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.332+0000 7f4dd3ab1700 1 -- 172.21.15.40:0/917926282 wait complete. 2023-07-18T03:31:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.333+0000 7f4dd3ab1700 1 Processor -- start 2023-07-18T03:31:22.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.333+0000 7f4dd3ab1700 1 -- start start 2023-07-18T03:31:22.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.333+0000 7f4dd3ab1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dcc1005e0 0x7f4dcc0fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.333+0000 7f4dd3ab1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dcc102f00 0x7f4dcc0fe590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.333+0000 7f4dd3ab1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc105820 0x7f4dcc0fead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.333+0000 7f4dd3ab1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc0ff270 con 0x7f4dcc105820 2023-07-18T03:31:22.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.333+0000 7f4dd3ab1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc0ff3e0 con 0x7f4dcc1005e0 2023-07-18T03:31:22.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.333+0000 7f4dd3ab1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc19f980 con 0x7f4dcc102f00 2023-07-18T03:31:22.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.333+0000 7f4dd204e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc105820 0x7f4dcc0fead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.333+0000 7f4dd204e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc105820 0x7f4dcc0fead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58188/0 (socket says 172.21.15.40:58188) 2023-07-18T03:31:22.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.333+0000 7f4dd204e700 1 -- 172.21.15.40:0/3806761963 learned_addr learned my addr 172.21.15.40:0/3806761963 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.333+0000 7f4dd104c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dcc102f00 0x7f4dcc0fe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.333+0000 7f4dd184d700 1 --2- 172.21.15.40:0/3806761963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dcc1005e0 0x7f4dcc0fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.334+0000 7f4dd204e700 1 -- 172.21.15.40:0/3806761963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dcc102f00 msgr2=0x7f4dcc0fe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.334+0000 7f4dd204e700 1 --2- 172.21.15.40:0/3806761963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dcc102f00 0x7f4dcc0fe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.334+0000 7f4dd204e700 1 -- 172.21.15.40:0/3806761963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dcc1005e0 msgr2=0x7f4dcc0fff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.334+0000 7f4dd204e700 1 --2- 172.21.15.40:0/3806761963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dcc1005e0 0x7f4dcc0fff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.334+0000 7f4dd204e700 1 -- 172.21.15.40:0/3806761963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc4009d00 con 0x7f4dcc105820 2023-07-18T03:31:22.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.334+0000 7f4dd204e700 1 --2- 172.21.15.40:0/3806761963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc105820 0x7f4dcc0fead0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4dc40062a0 tx=0x7f4dc4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.334+0000 7f4dc2ffd700 1 -- 172.21.15.40:0/3806761963 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc400e040 con 0x7f4dcc105820 2023-07-18T03:31:22.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.334+0000 7f4dd3ab1700 1 -- 172.21.15.40:0/3806761963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dcc19fc00 con 0x7f4dcc105820 2023-07-18T03:31:22.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.334+0000 7f4dc2ffd700 1 -- 172.21.15.40:0/3806761963 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4dc400cec0 con 0x7f4dcc105820 2023-07-18T03:31:22.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.334+0000 7f4dc2ffd700 1 -- 172.21.15.40:0/3806761963 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc401baf0 con 0x7f4dcc105820 2023-07-18T03:31:22.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.334+0000 7f4dd3ab1700 1 -- 172.21.15.40:0/3806761963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dcc1a00d0 con 0x7f4dcc105820 2023-07-18T03:31:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.336+0000 7f4dc2ffd700 1 -- 172.21.15.40:0/3806761963 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4dc401bc50 con 0x7f4dcc105820 2023-07-18T03:31:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.337+0000 7f4dc2ffd700 1 --2- 172.21.15.40:0/3806761963 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4db80a1150 0x7f4db80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.337+0000 7f4dd184d700 1 --2- 172.21.15.40:0/3806761963 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4db80a1150 0x7f4db80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.337+0000 7f4dc2ffd700 1 -- 172.21.15.40:0/3806761963 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4dc40ce440 con 0x7f4dcc105820 2023-07-18T03:31:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.338+0000 7f4dd184d700 1 --2- 172.21.15.40:0/3806761963 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4db80a1150 0x7f4db80a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8005950 tx=0x7f4dc800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.338+0000 7f4ddfc37b80 1 Processor -- start 2023-07-18T03:31:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.338+0000 7f4ddfc37b80 1 -- start start 2023-07-18T03:31:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.338+0000 7f4ddfc37b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61d4d0 0x559b5f61d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.338+0000 7f4ddfc37b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61de20 0x559b5f61e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.338+0000 7f4ddfc37b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61ec70 0x559b5f61f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.338+0000 7f4ddfc37b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5f522cb0 con 0x559b5f61de20 2023-07-18T03:31:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.338+0000 7f4ddfc37b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5f575a40 con 0x559b5f61d4d0 2023-07-18T03:31:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.338+0000 7f4ddfc37b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5f56eed0 con 0x559b5f61ec70 2023-07-18T03:31:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.339+0000 7f4dd104c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61de20 0x559b5f61e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.339+0000 7f4dd104c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61de20 0x559b5f61e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58194/0 (socket says 172.21.15.40:58194) 2023-07-18T03:31:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.339+0000 7f4dd104c700 1 -- 172.21.15.40:0/2767653349 learned_addr learned my addr 172.21.15.40:0/2767653349 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.339+0000 7f4dd204e700 1 --2- 172.21.15.40:0/2767653349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61ec70 0x559b5f61f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.339+0000 7f4dd184d700 1 --2- 172.21.15.40:0/2767653349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61d4d0 0x559b5f61d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.339+0000 7f4dd104c700 1 -- 172.21.15.40:0/2767653349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61ec70 msgr2=0x559b5f61f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.339+0000 7f4dd104c700 1 --2- 172.21.15.40:0/2767653349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61ec70 0x559b5f61f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.339+0000 7f4dd104c700 1 -- 172.21.15.40:0/2767653349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61d4d0 msgr2=0x559b5f61d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.339+0000 7f4dd104c700 1 --2- 172.21.15.40:0/2767653349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61d4d0 0x559b5f61d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.339+0000 7f4dd104c700 1 -- 172.21.15.40:0/2767653349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b5f39cde0 con 0x559b5f61de20 2023-07-18T03:31:22.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.339+0000 7f4dd184d700 1 --2- 172.21.15.40:0/2767653349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61d4d0 0x559b5f61d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:31:22.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.339+0000 7f4dd104c700 1 --2- 172.21.15.40:0/2767653349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61de20 0x559b5f61e6e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00bd70 tx=0x7f4dbc0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da5a5c615b7416a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.339+0000 7f4db7fff700 1 -- 172.21.15.40:0/2767653349 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc010ec0 con 0x559b5f61de20 2023-07-18T03:31:22.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.339+0000 7f4db7fff700 1 -- 172.21.15.40:0/2767653349 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4dbc015070 con 0x559b5f61de20 2023-07-18T03:31:22.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.339+0000 7f4db7fff700 1 -- 172.21.15.40:0/2767653349 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc011800 con 0x559b5f61de20 2023-07-18T03:31:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.340+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/2767653349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61de20 msgr2=0x559b5f61e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.340+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/2767653349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61de20 0x559b5f61e6e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00bd70 tx=0x7f4dbc0106b0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.340+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/2767653349 shutdown_connections 2023-07-18T03:31:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.340+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/2767653349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61ec70 0x559b5f61f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.340+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/2767653349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61d4d0 0x559b5f61d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.340+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/2767653349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61de20 0x559b5f61e6e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.340+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/2767653349 >> 172.21.15.40:0/2767653349 conn(0x559b5f612da0 msgr2=0x559b5f6181d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.340+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/2767653349 shutdown_connections 2023-07-18T03:31:22.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.340+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/2767653349 wait complete. 2023-07-18T03:31:22.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.340+0000 7f4ddfc37b80 1 -- start start 2023-07-18T03:31:22.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.341+0000 7f4ddfc37b80 1 Processor -- start 2023-07-18T03:31:22.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.341+0000 7f4ddfc37b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61d4d0 0x559b5f627170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.341+0000 7f4ddfc37b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61de20 0x559b5f6276b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.341+0000 7f4ddfc37b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61ec70 0x559b5f62e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.341+0000 7f4ddfc37b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5f50eac0 con 0x559b5f61d4d0 2023-07-18T03:31:22.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.341+0000 7f4ddfc37b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5f50e7c0 con 0x559b5f61de20 2023-07-18T03:31:22.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.341+0000 7f4ddfc37b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5f3f2920 con 0x559b5f61ec70 2023-07-18T03:31:22.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.341+0000 7f4ddfc37b80 10 client.0 _ll_register_callbacks cb 0x559b5f50c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:22.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.341+0000 7f4dd184d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61d4d0 0x559b5f627170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.341+0000 7f4dd184d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61d4d0 0x559b5f627170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58198/0 (socket says 172.21.15.40:58198) 2023-07-18T03:31:22.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.341+0000 7f4dd184d700 1 -- 172.21.15.40:0/1968884386 learned_addr learned my addr 172.21.15.40:0/1968884386 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.341+0000 7f4dd204e700 1 --2- 172.21.15.40:0/1968884386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61ec70 0x559b5f62e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.341+0000 7f4dd104c700 1 --2- 172.21.15.40:0/1968884386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61de20 0x559b5f6276b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.342+0000 7f4dd184d700 1 -- 172.21.15.40:0/1968884386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61ec70 msgr2=0x559b5f62e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.342+0000 7f4dd184d700 1 --2- 172.21.15.40:0/1968884386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61ec70 0x559b5f62e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.342+0000 7f4dd184d700 1 -- 172.21.15.40:0/1968884386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61de20 msgr2=0x559b5f6276b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.342+0000 7f4dd184d700 1 --2- 172.21.15.40:0/1968884386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61de20 0x559b5f6276b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.342+0000 7f4dd184d700 1 -- 172.21.15.40:0/1968884386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dbc00b9a0 con 0x559b5f61d4d0 2023-07-18T03:31:22.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.342+0000 7f4dd184d700 1 --2- 172.21.15.40:0/1968884386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61d4d0 0x559b5f627170 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80116c0 tx=0x7f4dc80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6184f9fa3c5cdc8a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.342+0000 7f4db57fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:22.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.342+0000 7f4db57fa700 1 -- 172.21.15.40:0/1968884386 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc801e830 con 0x559b5f61d4d0 2023-07-18T03:31:22.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.342+0000 7f4db57fa700 1 -- 172.21.15.40:0/1968884386 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4dc8017070 con 0x559b5f61d4d0 2023-07-18T03:31:22.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.342+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/1968884386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b5f4c84f0 con 0x559b5f61d4d0 2023-07-18T03:31:22.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.342+0000 7f4db57fa700 1 -- 172.21.15.40:0/1968884386 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dc801e990 con 0x559b5f61d4d0 2023-07-18T03:31:22.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.342+0000 7f4ddfc37b80 10 client.15558 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:22.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.342+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/1968884386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559b5f385c60 con 0x559b5f61d4d0 2023-07-18T03:31:22.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.342+0000 7f4db57fa700 1 -- 172.21.15.40:0/1968884386 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559b5f4c84f0 con 0x559b5f61d4d0 2023-07-18T03:31:22.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.342+0000 7f4db57fa700 1 -- 172.21.15.40:0/1968884386 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4dc8027ba0 con 0x559b5f61d4d0 2023-07-18T03:31:22.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.343+0000 7f4ddfc37b80 10 client.15558 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:22.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.343+0000 7f4ddfc37b80 10 client.15558 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-18T03:31:22.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.343+0000 7f4ddfc37b80 20 client.15558 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:22.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.343+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/1968884386 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x559b5f632ef0 0x559b5f635300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.343+0000 7f4ddfc37b80 4 client.15558 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:22.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.343+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/1968884386 --> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b5f567eb0 con 0x559b5f632ef0 2023-07-18T03:31:22.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.343+0000 7f4dd104c700 1 --2- 172.21.15.40:0/1968884386 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x559b5f632ef0 0x559b5f635300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.344+0000 7f4dd104c700 1 --2- 172.21.15.40:0/1968884386 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x559b5f632ef0 0x559b5f635300 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8cfcc27f189991df server_cookie=2df2130927b7ce07 in_seq=0 out_seq=0 2023-07-18T03:31:22.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.344+0000 7f4db57fa700 10 client.15558 ms_handle_connect on v2:172.21.15.64:6836/3077792296 2023-07-18T03:31:22.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.344+0000 7f4db57fa700 1 -- 172.21.15.40:0/1968884386 <== mds.2 v2:172.21.15.64:6836/3077792296 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559b5f567eb0 con 0x559b5f632ef0 2023-07-18T03:31:22.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.344+0000 7f4db57fa700 10 client.15558 handle_command_reply: tid=0 2023-07-18T03:31:22.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.344+0000 7f4ddfc37b80 1 client.15558 shutdown 2023-07-18T03:31:22.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.344+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/1968884386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61d4d0 msgr2=0x559b5f627170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.344+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/1968884386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61d4d0 0x559b5f627170 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80116c0 tx=0x7f4dc80047c0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.344+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/1968884386 shutdown_connections 2023-07-18T03:31:22.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.344+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/1968884386 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x559b5f632ef0 0x559b5f635300 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.345+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/1968884386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61ec70 0x559b5f62e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.345+0000 7f4db57fa700 0 client.15558 ms_handle_reset on v2:172.21.15.64:6836/3077792296 2023-07-18T03:31:22.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.345+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/1968884386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61d4d0 0x559b5f627170 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.345+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/1968884386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61de20 0x559b5f6276b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.345+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/1968884386 >> 172.21.15.40:0/1968884386 conn(0x559b5f612da0 msgr2=0x559b5f6178b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.345+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/1968884386 shutdown_connections 2023-07-18T03:31:22.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.345+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/1968884386 wait complete. 2023-07-18T03:31:22.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.345+0000 7f4ddfc37b80 20 client.15558 trim_cache size 0 max 16384 2023-07-18T03:31:22.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4ddfc37b80 1 Processor -- start 2023-07-18T03:31:22.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4ddfc37b80 1 -- start start 2023-07-18T03:31:22.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4ddfc37b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61d4d0 0x559b5f61d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4ddfc37b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61de20 0x559b5f62e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4ddfc37b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61ec70 0x559b5f626820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4ddfc37b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5f6352d0 con 0x559b5f61d4d0 2023-07-18T03:31:22.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4ddfc37b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5f635440 con 0x559b5f61ec70 2023-07-18T03:31:22.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4ddfc37b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5f625e50 con 0x559b5f61de20 2023-07-18T03:31:22.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4dd184d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61d4d0 0x559b5f61d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4dd184d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61d4d0 0x559b5f61d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58206/0 (socket says 172.21.15.40:58206) 2023-07-18T03:31:22.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4dd184d700 1 -- 172.21.15.40:0/2726929836 learned_addr learned my addr 172.21.15.40:0/2726929836 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4dd204e700 1 --2- 172.21.15.40:0/2726929836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61ec70 0x559b5f626820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4dd104c700 1 --2- 172.21.15.40:0/2726929836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61de20 0x559b5f62e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4dd184d700 1 -- 172.21.15.40:0/2726929836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61de20 msgr2=0x559b5f62e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4dd184d700 1 --2- 172.21.15.40:0/2726929836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61de20 0x559b5f62e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4dd184d700 1 -- 172.21.15.40:0/2726929836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61ec70 msgr2=0x559b5f626820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4dd184d700 1 --2- 172.21.15.40:0/2726929836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61ec70 0x559b5f626820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.358+0000 7f4dd184d700 1 -- 172.21.15.40:0/2726929836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b5f3c3250 con 0x559b5f61d4d0 2023-07-18T03:31:22.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.359+0000 7f4dd184d700 1 --2- 172.21.15.40:0/2726929836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61d4d0 0x559b5f61d8a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8010ac0 tx=0x7f4dc8004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2cc407c140ccc06 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.359+0000 7f4dc0dac700 1 -- 172.21.15.40:0/2726929836 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc801ba40 con 0x559b5f61d4d0 2023-07-18T03:31:22.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.359+0000 7f4dc0dac700 1 -- 172.21.15.40:0/2726929836 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4dc8015070 con 0x559b5f61d4d0 2023-07-18T03:31:22.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.359+0000 7f4dc0dac700 1 -- 172.21.15.40:0/2726929836 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc801eaf0 con 0x559b5f61d4d0 2023-07-18T03:31:22.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.360+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/2726929836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61d4d0 msgr2=0x559b5f61d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.360+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/2726929836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61d4d0 0x559b5f61d8a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8010ac0 tx=0x7f4dc8004690 comp rx=0 tx=0).stop 2023-07-18T03:31:22.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.360+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/2726929836 shutdown_connections 2023-07-18T03:31:22.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.360+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/2726929836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61ec70 0x559b5f626820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.360+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/2726929836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61d4d0 0x559b5f61d8a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.360+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/2726929836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61de20 0x559b5f62e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.360+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/2726929836 >> 172.21.15.40:0/2726929836 conn(0x559b5f612da0 msgr2=0x559b5f639c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.360+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/2726929836 shutdown_connections 2023-07-18T03:31:22.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.360+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/2726929836 wait complete. 2023-07-18T03:31:22.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.360+0000 7f4ddfc37b80 1 -- start start 2023-07-18T03:31:22.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.360+0000 7f4ddfc37b80 1 Processor -- start 2023-07-18T03:31:22.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.360+0000 7f4ddfc37b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61d4d0 0x559b5f62a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4ddfc37b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61de20 0x559b5f62ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4ddfc37b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61ec70 0x559b5f62dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4ddfc37b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5f62e1c0 con 0x559b5f61ec70 2023-07-18T03:31:22.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4ddfc37b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5f62e330 con 0x559b5f61de20 2023-07-18T03:31:22.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4ddfc37b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5f62e4f0 con 0x559b5f61d4d0 2023-07-18T03:31:22.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4ddfc37b80 10 client.0 _ll_register_callbacks cb 0x7f4dc8010d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:22.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4dd204e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61ec70 0x559b5f62dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4dd204e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61ec70 0x559b5f62dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58214/0 (socket says 172.21.15.40:58214) 2023-07-18T03:31:22.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4dd204e700 1 -- 172.21.15.40:0/939132713 learned_addr learned my addr 172.21.15.40:0/939132713 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4dd184d700 1 --2- 172.21.15.40:0/939132713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61d4d0 0x559b5f62a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4dd104c700 1 --2- 172.21.15.40:0/939132713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61de20 0x559b5f62ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.363 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:31:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4dd204e700 1 -- 172.21.15.40:0/939132713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61d4d0 msgr2=0x559b5f62a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4dd204e700 1 --2- 172.21.15.40:0/939132713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61d4d0 0x559b5f62a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4dd204e700 1 -- 172.21.15.40:0/939132713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61de20 msgr2=0x559b5f62ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4dd204e700 1 --2- 172.21.15.40:0/939132713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61de20 0x559b5f62ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4dd204e700 1 -- 172.21.15.40:0/939132713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b5f385c60 con 0x559b5f61ec70 2023-07-18T03:31:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4dd104c700 1 --2- 172.21.15.40:0/939132713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61de20 0x559b5f62ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:31:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4dd204e700 1 --2- 172.21.15.40:0/939132713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61ec70 0x559b5f62dc80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4007ab0 tx=0x7f4dc401b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904ce4d38f54bf32 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4db57fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4db57fa700 1 -- 172.21.15.40:0/939132713 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc40a3c80 con 0x559b5f61ec70 2023-07-18T03:31:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4db57fa700 1 -- 172.21.15.40:0/939132713 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4dc4011c60 con 0x559b5f61ec70 2023-07-18T03:31:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.361+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/939132713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4dbc015070 con 0x559b5f61ec70 2023-07-18T03:31:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.362+0000 7f4db57fa700 1 -- 172.21.15.40:0/939132713 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dc4011e20 con 0x559b5f61ec70 2023-07-18T03:31:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.362+0000 7f4ddfc37b80 10 client.15564 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.362+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/939132713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559b5f39b2b0 con 0x559b5f61ec70 2023-07-18T03:31:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.362+0000 7f4db57fa700 1 -- 172.21.15.40:0/939132713 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4dbc015070 con 0x559b5f61ec70 2023-07-18T03:31:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.362+0000 7f4db57fa700 1 -- 172.21.15.40:0/939132713 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4dc409adb0 con 0x559b5f61ec70 2023-07-18T03:31:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.362+0000 7f4ddfc37b80 10 client.15564 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.362+0000 7f4ddfc37b80 10 client.15564 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-18T03:31:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.362+0000 7f4ddfc37b80 20 client.15564 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.362+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/939132713 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x559b5f687cc0 0x559b5f68a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.362+0000 7f4ddfc37b80 4 client.15564 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:31:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.362+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/939132713 --> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559b5f52a050 con 0x559b5f687cc0 2023-07-18T03:31:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.362+0000 7f4dd184d700 1 --2- 172.21.15.40:0/939132713 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x559b5f687cc0 0x559b5f68a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.363+0000 7f4db57fa700 10 client.15564 ms_handle_connect on v2:172.21.15.64:6836/3077792296 2023-07-18T03:31:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.363+0000 7f4dd184d700 1 --2- 172.21.15.40:0/939132713 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x559b5f687cc0 0x559b5f68a0d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10fb78bb1778ea20 server_cookie=d5da57dd8c172a60 in_seq=0 out_seq=0 2023-07-18T03:31:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.363+0000 7f4db57fa700 1 -- 172.21.15.40:0/939132713 <== mds.2 v2:172.21.15.64:6836/3077792296 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559b5f52a050 con 0x559b5f687cc0 2023-07-18T03:31:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.363+0000 7f4db57fa700 10 client.15564 handle_command_reply: tid=0 2023-07-18T03:31:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.363+0000 7f4ddfc37b80 1 client.15564 shutdown 2023-07-18T03:31:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.363+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/939132713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61ec70 msgr2=0x559b5f62dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.363+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/939132713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61ec70 0x559b5f62dc80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4007ab0 tx=0x7f4dc401b550 comp rx=0 tx=0).stop 2023-07-18T03:31:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.363+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/939132713 shutdown_connections 2023-07-18T03:31:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.363+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/939132713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559b5f61ec70 0x559b5f62dc80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.363+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/939132713 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x559b5f687cc0 0x559b5f68a0d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.363+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/939132713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559b5f61d4d0 0x559b5f62a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.363+0000 7f4ddfc37b80 1 --2- 172.21.15.40:0/939132713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b5f61de20 0x559b5f62ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.363+0000 7f4db57fa700 0 client.15564 ms_handle_reset on v2:172.21.15.64:6836/3077792296 2023-07-18T03:31:22.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.363+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/939132713 >> 172.21.15.40:0/939132713 conn(0x559b5f612da0 msgr2=0x559b5f6250a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.363+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/939132713 shutdown_connections 2023-07-18T03:31:22.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.363+0000 7f4ddfc37b80 1 -- 172.21.15.40:0/939132713 wait complete. 2023-07-18T03:31:22.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.363+0000 7f4ddfc37b80 20 client.15564 trim_cache size 0 max 16384 2023-07-18T03:31:22.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.364+0000 7f4dd3ab1700 1 -- 172.21.15.40:0/3806761963 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4db80a1150 msgr2=0x7f4db80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.364+0000 7f4dd3ab1700 1 --2- 172.21.15.40:0/3806761963 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4db80a1150 0x7f4db80a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8005950 tx=0x7f4dc800b410 comp rx=0 tx=0).stop 2023-07-18T03:31:22.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.364+0000 7f4dd3ab1700 1 -- 172.21.15.40:0/3806761963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc105820 msgr2=0x7f4dcc0fead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.364+0000 7f4dd3ab1700 1 --2- 172.21.15.40:0/3806761963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc105820 0x7f4dcc0fead0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4dc40062a0 tx=0x7f4dc4004d00 comp rx=0 tx=0).stop 2023-07-18T03:31:22.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.364+0000 7f4dd3ab1700 1 -- 172.21.15.40:0/3806761963 shutdown_connections 2023-07-18T03:31:22.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.364+0000 7f4dd3ab1700 1 --2- 172.21.15.40:0/3806761963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dcc105820 0x7f4dcc0fead0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.364+0000 7f4dd3ab1700 1 --2- 172.21.15.40:0/3806761963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4dcc1005e0 0x7f4dcc0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.364+0000 7f4dd3ab1700 1 --2- 172.21.15.40:0/3806761963 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4db80a1150 0x7f4db80a3600 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.364+0000 7f4dd3ab1700 1 --2- 172.21.15.40:0/3806761963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dcc102f00 0x7f4dcc0fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.364+0000 7f4dd3ab1700 1 -- 172.21.15.40:0/3806761963 >> 172.21.15.40:0/3806761963 conn(0x7f4dcc0fa000 msgr2=0x7f4dcc101240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.364+0000 7f4dd3ab1700 1 -- 172.21.15.40:0/3806761963 shutdown_connections 2023-07-18T03:31:22.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.364+0000 7f4dd3ab1700 1 -- 172.21.15.40:0/3806761963 wait complete. 2023-07-18T03:31:22.374 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:31:22.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.515+0000 7fa233c39700 1 -- 172.21.15.40:0/4067551579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa22c0ffbd0 msgr2=0x7fa22c100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.515+0000 7fa233c39700 1 --2- 172.21.15.40:0/4067551579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa22c0ffbd0 0x7fa22c100020 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009990 tx=0x7fa21c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.516+0000 7fa233c39700 1 -- 172.21.15.40:0/4067551579 shutdown_connections 2023-07-18T03:31:22.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.516+0000 7fa233c39700 1 --2- 172.21.15.40:0/4067551579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa22c100560 0x7fa22c106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.516+0000 7fa233c39700 1 --2- 172.21.15.40:0/4067551579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa22c0fe9e0 0x7fa22c0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.516+0000 7fa233c39700 1 --2- 172.21.15.40:0/4067551579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa22c0ffbd0 0x7fa22c100020 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.516+0000 7fa233c39700 1 -- 172.21.15.40:0/4067551579 >> 172.21.15.40:0/4067551579 conn(0x7fa22c0fa000 msgr2=0x7fa22c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.516+0000 7fa233c39700 1 -- 172.21.15.40:0/4067551579 shutdown_connections 2023-07-18T03:31:22.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.516+0000 7fa233c39700 1 -- 172.21.15.40:0/4067551579 wait complete. 2023-07-18T03:31:22.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.516+0000 7fa233c39700 1 Processor -- start 2023-07-18T03:31:22.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.517+0000 7fa233c39700 1 -- start start 2023-07-18T03:31:22.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.517+0000 7fa233c39700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa22c0fe9e0 0x7fa22c194770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.517+0000 7fa233c39700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa22c0ffbd0 0x7fa22c194cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.517+0000 7fa233c39700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa22c100560 0x7fa22c19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.517+0000 7fa233c39700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c19c2f0 con 0x7fa22c100560 2023-07-18T03:31:22.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.517+0000 7fa233c39700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c19c430 con 0x7fa22c0ffbd0 2023-07-18T03:31:22.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.517+0000 7fa233c39700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c19c5c0 con 0x7fa22c0fe9e0 2023-07-18T03:31:22.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.517+0000 7fa2321d6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa22c100560 0x7fa22c19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.517+0000 7fa2321d6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa22c100560 0x7fa22c19bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58228/0 (socket says 172.21.15.40:58228) 2023-07-18T03:31:22.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.517+0000 7fa2321d6700 1 -- 172.21.15.40:0/800560138 learned_addr learned my addr 172.21.15.40:0/800560138 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.517+0000 7fa2311d4700 1 --2- 172.21.15.40:0/800560138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa22c0ffbd0 0x7fa22c194cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.517+0000 7fa2319d5700 1 --2- 172.21.15.40:0/800560138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa22c0fe9e0 0x7fa22c194770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.517+0000 7fa2321d6700 1 -- 172.21.15.40:0/800560138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa22c0fe9e0 msgr2=0x7fa22c194770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.517+0000 7fa2321d6700 1 --2- 172.21.15.40:0/800560138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa22c0fe9e0 0x7fa22c194770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.517+0000 7fa2321d6700 1 -- 172.21.15.40:0/800560138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa22c0ffbd0 msgr2=0x7fa22c194cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.517+0000 7fa2321d6700 1 --2- 172.21.15.40:0/800560138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa22c0ffbd0 0x7fa22c194cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.517+0000 7fa2321d6700 1 -- 172.21.15.40:0/800560138 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c0095f0 con 0x7fa22c100560 2023-07-18T03:31:22.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.518+0000 7fa2321d6700 1 --2- 172.21.15.40:0/800560138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa22c100560 0x7fa22c19bd20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa224007ee0 tx=0x7fa22400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.518+0000 7fa222ffd700 1 -- 172.21.15.40:0/800560138 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224017070 con 0x7fa22c100560 2023-07-18T03:31:22.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.518+0000 7fa222ffd700 1 -- 172.21.15.40:0/800560138 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa22400f040 con 0x7fa22c100560 2023-07-18T03:31:22.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.518+0000 7fa233c39700 1 -- 172.21.15.40:0/800560138 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22c19c8a0 con 0x7fa22c100560 2023-07-18T03:31:22.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.518+0000 7fa222ffd700 1 -- 172.21.15.40:0/800560138 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224005740 con 0x7fa22c100560 2023-07-18T03:31:22.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.518+0000 7fa233c39700 1 -- 172.21.15.40:0/800560138 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22c19cdb0 con 0x7fa22c100560 2023-07-18T03:31:22.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.520+0000 7fa222ffd700 1 -- 172.21.15.40:0/800560138 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa22400dc60 con 0x7fa22c100560 2023-07-18T03:31:22.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.521+0000 7fa222ffd700 1 --2- 172.21.15.40:0/800560138 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa2180a11d0 0x7fa2180a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.521+0000 7fa222ffd700 1 -- 172.21.15.40:0/800560138 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa2240cc0e0 con 0x7fa22c100560 2023-07-18T03:31:22.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.521+0000 7fa2319d5700 1 --2- 172.21.15.40:0/800560138 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa2180a11d0 0x7fa2180a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.521+0000 7fa2319d5700 1 --2- 172.21.15.40:0/800560138 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa2180a11d0 0x7fa2180a3680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa228005950 tx=0x7fa2280058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.522+0000 7fa23fdbfb80 1 Processor -- start 2023-07-18T03:31:22.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.522+0000 7fa23fdbfb80 1 -- start start 2023-07-18T03:31:22.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.522+0000 7fa23fdbfb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dc0b90 0x563bb0dbf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.522+0000 7fa23fdbfb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dbf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.522+0000 7fa23fdbfb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dbfc80 0x563bb0dcb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.522+0000 7fa23fdbfb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0cc4cb0 con 0x563bb0dc0fa0 2023-07-18T03:31:22.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.522+0000 7fa23fdbfb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0d17a40 con 0x563bb0dc0b90 2023-07-18T03:31:22.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.522+0000 7fa23fdbfb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0d10ed0 con 0x563bb0dbfc80 2023-07-18T03:31:22.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.522+0000 7fa2311d4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dbf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.522+0000 7fa2319d5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dc0b90 0x563bb0dbf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.522+0000 7fa2311d4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dbf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58234/0 (socket says 172.21.15.40:58234) 2023-07-18T03:31:22.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.522+0000 7fa2311d4700 1 -- 172.21.15.40:0/1445439907 learned_addr learned my addr 172.21.15.40:0/1445439907 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.522+0000 7fa2319d5700 1 --2- 172.21.15.40:0/1445439907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dc0b90 0x563bb0dbf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:41558/0 (socket says 172.21.15.40:41558) 2023-07-18T03:31:22.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.523+0000 7fa2321d6700 1 --2- 172.21.15.40:0/1445439907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dbfc80 0x563bb0dcb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.523+0000 7fa2311d4700 1 -- 172.21.15.40:0/1445439907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dbfc80 msgr2=0x563bb0dcb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.523+0000 7fa2311d4700 1 --2- 172.21.15.40:0/1445439907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dbfc80 0x563bb0dcb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.523+0000 7fa2311d4700 1 -- 172.21.15.40:0/1445439907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dc0b90 msgr2=0x563bb0dbf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.523+0000 7fa2311d4700 1 --2- 172.21.15.40:0/1445439907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dc0b90 0x563bb0dbf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.523+0000 7fa2311d4700 1 -- 172.21.15.40:0/1445439907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bb0b3ede0 con 0x563bb0dc0fa0 2023-07-18T03:31:22.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.523+0000 7fa2311d4700 1 --2- 172.21.15.40:0/1445439907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dbf740 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa21c015000 tx=0x7fa21c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b6f43033d7db24 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.523+0000 7fa217fff700 1 -- 172.21.15.40:0/1445439907 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c01d070 con 0x563bb0dc0fa0 2023-07-18T03:31:22.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.523+0000 7fa217fff700 1 -- 172.21.15.40:0/1445439907 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa21c01e040 con 0x563bb0dc0fa0 2023-07-18T03:31:22.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.523+0000 7fa217fff700 1 -- 172.21.15.40:0/1445439907 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c004560 con 0x563bb0dc0fa0 2023-07-18T03:31:22.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.524+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/1445439907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 msgr2=0x563bb0dbf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.524+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/1445439907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dbf740 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa21c015000 tx=0x7fa21c009f90 comp rx=0 tx=0).stop 2023-07-18T03:31:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.524+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/1445439907 shutdown_connections 2023-07-18T03:31:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.524+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/1445439907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dbfc80 0x563bb0dcb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.524+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/1445439907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dc0b90 0x563bb0dbf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.524+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/1445439907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dbf740 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.524+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/1445439907 >> 172.21.15.40:0/1445439907 conn(0x563bb0db4da0 msgr2=0x563bb0dba1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.524+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/1445439907 shutdown_connections 2023-07-18T03:31:22.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.524+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/1445439907 wait complete. 2023-07-18T03:31:22.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.524+0000 7fa23fdbfb80 1 -- start start 2023-07-18T03:31:22.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.525+0000 7fa23fdbfb80 1 Processor -- start 2023-07-18T03:31:22.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.525+0000 7fa23fdbfb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dbfc80 0x563bb0dcd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.525+0000 7fa23fdbfb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dc0b90 0x563bb0dcda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.525+0000 7fa23fdbfb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dc0fa0 0x563bb0dd4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.525+0000 7fa23fdbfb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0cb0ac0 con 0x563bb0dbfc80 2023-07-18T03:31:22.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.525+0000 7fa23fdbfb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0cb07c0 con 0x563bb0dc0b90 2023-07-18T03:31:22.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.525+0000 7fa23fdbfb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0b94920 con 0x563bb0dc0fa0 2023-07-18T03:31:22.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.525+0000 7fa2319d5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dbfc80 0x563bb0dcd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.525+0000 7fa23fdbfb80 10 client.0 _ll_register_callbacks cb 0x563bb0cae760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:22.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.525+0000 7fa2319d5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dbfc80 0x563bb0dcd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58238/0 (socket says 172.21.15.40:58238) 2023-07-18T03:31:22.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.525+0000 7fa2319d5700 1 -- 172.21.15.40:0/137051027 learned_addr learned my addr 172.21.15.40:0/137051027 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.525+0000 7fa2311d4700 1 --2- 172.21.15.40:0/137051027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dc0b90 0x563bb0dcda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.525+0000 7fa2321d6700 1 --2- 172.21.15.40:0/137051027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dc0fa0 0x563bb0dd4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.526+0000 7fa2319d5700 1 -- 172.21.15.40:0/137051027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dc0fa0 msgr2=0x563bb0dd4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.526+0000 7fa2319d5700 1 --2- 172.21.15.40:0/137051027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dc0fa0 0x563bb0dd4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.526+0000 7fa2319d5700 1 -- 172.21.15.40:0/137051027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dc0b90 msgr2=0x563bb0dcda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.526+0000 7fa2319d5700 1 --2- 172.21.15.40:0/137051027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dc0b90 0x563bb0dcda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.526+0000 7fa2319d5700 1 -- 172.21.15.40:0/137051027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22c107750 con 0x563bb0dbfc80 2023-07-18T03:31:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.526+0000 7fa2311d4700 1 --2- 172.21.15.40:0/137051027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dc0b90 0x563bb0dcda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.526+0000 7fa2321d6700 1 --2- 172.21.15.40:0/137051027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dc0fa0 0x563bb0dd4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.526+0000 7fa2319d5700 1 --2- 172.21.15.40:0/137051027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dbfc80 0x563bb0dcd520 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa22800ff30 tx=0x7fa228004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600588229644ac36 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.526+0000 7fa2157fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:22.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.526+0000 7fa2157fa700 1 -- 172.21.15.40:0/137051027 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228028670 con 0x563bb0dbfc80 2023-07-18T03:31:22.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.526+0000 7fa2157fa700 1 -- 172.21.15.40:0/137051027 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa2280287d0 con 0x563bb0dbfc80 2023-07-18T03:31:22.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.526+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/137051027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563bb0c6a4f0 con 0x563bb0dbfc80 2023-07-18T03:31:22.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.526+0000 7fa2157fa700 1 -- 172.21.15.40:0/137051027 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa228011e70 con 0x563bb0dbfc80 2023-07-18T03:31:22.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.526+0000 7fa23fdbfb80 10 client.15576 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:22.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.526+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/137051027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563bb0b27c60 con 0x563bb0dbfc80 2023-07-18T03:31:22.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.526+0000 7fa2157fa700 1 -- 172.21.15.40:0/137051027 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563bb0c6a4f0 con 0x563bb0dbfc80 2023-07-18T03:31:22.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.527+0000 7fa2157fa700 1 -- 172.21.15.40:0/137051027 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa228031050 con 0x563bb0dbfc80 2023-07-18T03:31:22.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.527+0000 7fa23fdbfb80 10 client.15576 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:22.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.527+0000 7fa23fdbfb80 10 client.15576 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-18T03:31:22.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.527+0000 7fa23fdbfb80 20 client.15576 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:22.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.527+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/137051027 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x563bb0dd9400 0x563bb0ddb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.527+0000 7fa23fdbfb80 4 client.15576 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:22.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.527+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/137051027 --> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563bb0d09eb0 con 0x563bb0dd9400 2023-07-18T03:31:22.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.527+0000 7fa2311d4700 1 --2- 172.21.15.40:0/137051027 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x563bb0dd9400 0x563bb0ddb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.527+0000 7fa2157fa700 10 client.15576 ms_handle_connect on v2:172.21.15.134:6838/3192254101 2023-07-18T03:31:22.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.527+0000 7fa2311d4700 1 --2- 172.21.15.40:0/137051027 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x563bb0dd9400 0x563bb0ddb810 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bab121841b5d4a7e server_cookie=f50a0108f36fb516 in_seq=0 out_seq=0 2023-07-18T03:31:22.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.528+0000 7fa2157fa700 1 -- 172.21.15.40:0/137051027 <== mds.1 v2:172.21.15.134:6838/3192254101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563bb0d09eb0 con 0x563bb0dd9400 2023-07-18T03:31:22.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.528+0000 7fa2157fa700 10 client.15576 handle_command_reply: tid=0 2023-07-18T03:31:22.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.528+0000 7fa23fdbfb80 1 client.15576 shutdown 2023-07-18T03:31:22.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.528+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/137051027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dbfc80 msgr2=0x563bb0dcd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.528+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/137051027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dbfc80 0x563bb0dcd520 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa22800ff30 tx=0x7fa228004750 comp rx=0 tx=0).stop 2023-07-18T03:31:22.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.528+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/137051027 shutdown_connections 2023-07-18T03:31:22.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.528+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/137051027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dc0fa0 0x563bb0dd4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.528+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/137051027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dbfc80 0x563bb0dcd520 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.528+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/137051027 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x563bb0dd9400 0x563bb0ddb810 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.528+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/137051027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dc0b90 0x563bb0dcda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.528+0000 7fa2157fa700 0 client.15576 ms_handle_reset on v2:172.21.15.134:6838/3192254101 2023-07-18T03:31:22.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.528+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/137051027 >> 172.21.15.40:0/137051027 conn(0x563bb0db4da0 msgr2=0x563bb0dbf410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.528+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/137051027 shutdown_connections 2023-07-18T03:31:22.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.528+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/137051027 wait complete. 2023-07-18T03:31:22.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.529+0000 7fa23fdbfb80 20 client.15576 trim_cache size 0 max 16384 2023-07-18T03:31:22.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.545+0000 7fa23fdbfb80 1 Processor -- start 2023-07-18T03:31:22.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.545+0000 7fa23fdbfb80 1 -- start start 2023-07-18T03:31:22.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.545+0000 7fa23fdbfb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dbfc80 0x563bb0dcc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.545+0000 7fa23fdbfb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dc0b90 0x563bb0dd4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.545+0000 7fa23fdbfb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dc7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.545+0000 7fa23fdbfb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0dd9050 con 0x563bb0dc0fa0 2023-07-18T03:31:22.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.545+0000 7fa23fdbfb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0de0510 con 0x563bb0dc0b90 2023-07-18T03:31:22.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.545+0000 7fa23fdbfb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0de0680 con 0x563bb0dbfc80 2023-07-18T03:31:22.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.545+0000 7fa2321d6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dc7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.545+0000 7fa2321d6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dc7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58250/0 (socket says 172.21.15.40:58250) 2023-07-18T03:31:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.545+0000 7fa2321d6700 1 -- 172.21.15.40:0/1528188547 learned_addr learned my addr 172.21.15.40:0/1528188547 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.545+0000 7fa2311d4700 1 --2- 172.21.15.40:0/1528188547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dc0b90 0x563bb0dd4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.545+0000 7fa2319d5700 1 --2- 172.21.15.40:0/1528188547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dbfc80 0x563bb0dcc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.545+0000 7fa2321d6700 1 -- 172.21.15.40:0/1528188547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dbfc80 msgr2=0x563bb0dcc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.545+0000 7fa2321d6700 1 --2- 172.21.15.40:0/1528188547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dbfc80 0x563bb0dcc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.546+0000 7fa2321d6700 1 -- 172.21.15.40:0/1528188547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dc0b90 msgr2=0x563bb0dd4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.546+0000 7fa2321d6700 1 --2- 172.21.15.40:0/1528188547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dc0b90 0x563bb0dd4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.546+0000 7fa2321d6700 1 -- 172.21.15.40:0/1528188547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bb0b65250 con 0x563bb0dc0fa0 2023-07-18T03:31:22.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.546+0000 7fa2321d6700 1 --2- 172.21.15.40:0/1528188547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dc7fa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa2240c9bc0 tx=0x7fa224084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550e0e5d476b5d5f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.546+0000 7fa220dac700 1 -- 172.21.15.40:0/1528188547 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22409d500 con 0x563bb0dc0fa0 2023-07-18T03:31:22.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.546+0000 7fa220dac700 1 -- 172.21.15.40:0/1528188547 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2240a1920 con 0x563bb0dc0fa0 2023-07-18T03:31:22.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.546+0000 7fa220dac700 1 -- 172.21.15.40:0/1528188547 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2240841d0 con 0x563bb0dc0fa0 2023-07-18T03:31:22.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.547+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/1528188547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 msgr2=0x563bb0dc7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.547+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/1528188547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dc7fa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa2240c9bc0 tx=0x7fa224084040 comp rx=0 tx=0).stop 2023-07-18T03:31:22.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.547+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/1528188547 shutdown_connections 2023-07-18T03:31:22.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.547+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/1528188547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dc7fa0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.547+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/1528188547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dbfc80 0x563bb0dcc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.547+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/1528188547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dc0b90 0x563bb0dd4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.547+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/1528188547 >> 172.21.15.40:0/1528188547 conn(0x563bb0db4da0 msgr2=0x563bb0dbf410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.547+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/1528188547 shutdown_connections 2023-07-18T03:31:22.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.547+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/1528188547 wait complete. 2023-07-18T03:31:22.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.547+0000 7fa23fdbfb80 1 -- start start 2023-07-18T03:31:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.547+0000 7fa23fdbfb80 1 Processor -- start 2023-07-18T03:31:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa23fdbfb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dbfc80 0x563bb0dd2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa23fdbfb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dc0b90 0x563bb0dd3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa23fdbfb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dd4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa23fdbfb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0dd4550 con 0x563bb0dc0fa0 2023-07-18T03:31:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa23fdbfb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0dd46c0 con 0x563bb0dbfc80 2023-07-18T03:31:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa23fdbfb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bb0dd4880 con 0x563bb0dc0b90 2023-07-18T03:31:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa23fdbfb80 10 client.0 _ll_register_callbacks cb 0x563bb0cae760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:22.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa2321d6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dd4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa2321d6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dd4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58256/0 (socket says 172.21.15.40:58256) 2023-07-18T03:31:22.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa2311d4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dc0b90 0x563bb0dd3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa2321d6700 1 -- 172.21.15.40:0/3977612528 learned_addr learned my addr 172.21.15.40:0/3977612528 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa2319d5700 1 --2- 172.21.15.40:0/3977612528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dbfc80 0x563bb0dd2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa2321d6700 1 -- 172.21.15.40:0/3977612528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dc0b90 msgr2=0x563bb0dd3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa2321d6700 1 --2- 172.21.15.40:0/3977612528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dc0b90 0x563bb0dd3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa2321d6700 1 -- 172.21.15.40:0/3977612528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dbfc80 msgr2=0x563bb0dd2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa2321d6700 1 --2- 172.21.15.40:0/3977612528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dbfc80 0x563bb0dd2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa2321d6700 1 -- 172.21.15.40:0/3977612528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22c19cdb0 con 0x563bb0dc0fa0 2023-07-18T03:31:22.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa2321d6700 1 --2- 172.21.15.40:0/3977612528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dd4010 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa2240959d0 tx=0x7fa224099640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6152eca19112c48 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa2157fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:22.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa2157fa700 1 -- 172.21.15.40:0/3977612528 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2240841b0 con 0x563bb0dc0fa0 2023-07-18T03:31:22.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa2157fa700 1 -- 172.21.15.40:0/3977612528 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa22409d500 con 0x563bb0dc0fa0 2023-07-18T03:31:22.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.548+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/3977612528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa21c01e040 con 0x563bb0dc0fa0 2023-07-18T03:31:22.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.549+0000 7fa2157fa700 1 -- 172.21.15.40:0/3977612528 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2240058a0 con 0x563bb0dc0fa0 2023-07-18T03:31:22.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.549+0000 7fa23fdbfb80 10 client.15582 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:22.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.549+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/3977612528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563bb0b3d2b0 con 0x563bb0dc0fa0 2023-07-18T03:31:22.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.549+0000 7fa2157fa700 1 -- 172.21.15.40:0/3977612528 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa21c01e040 con 0x563bb0dc0fa0 2023-07-18T03:31:22.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.549+0000 7fa2157fa700 1 -- 172.21.15.40:0/3977612528 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa2240997a0 con 0x563bb0dc0fa0 2023-07-18T03:31:22.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.549+0000 7fa23fdbfb80 10 client.15582 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:22.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.549+0000 7fa23fdbfb80 10 client.15582 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-18T03:31:22.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.549+0000 7fa23fdbfb80 20 client.15582 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:22.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.549+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/3977612528 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x563bb0e2dfa0 0x563bb0e303b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.549+0000 7fa23fdbfb80 4 client.15582 mds_command: new command op to 24523 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:31:22.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.549+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/3977612528 --> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563bb0ccc050 con 0x563bb0e2dfa0 2023-07-18T03:31:22.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.549+0000 7fa2319d5700 1 --2- 172.21.15.40:0/3977612528 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x563bb0e2dfa0 0x563bb0e303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.550+0000 7fa2157fa700 10 client.15582 ms_handle_connect on v2:172.21.15.134:6838/3192254101 2023-07-18T03:31:22.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.550+0000 7fa2319d5700 1 --2- 172.21.15.40:0/3977612528 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x563bb0e2dfa0 0x563bb0e303b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c26390f9240c1b0 server_cookie=98b5434465d3e77c in_seq=0 out_seq=0 2023-07-18T03:31:22.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.550+0000 7fa2157fa700 1 -- 172.21.15.40:0/3977612528 <== mds.1 v2:172.21.15.134:6838/3192254101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563bb0ccc050 con 0x563bb0e2dfa0 2023-07-18T03:31:22.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.550+0000 7fa2157fa700 10 client.15582 handle_command_reply: tid=0 2023-07-18T03:31:22.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.550+0000 7fa23fdbfb80 1 client.15582 shutdown 2023-07-18T03:31:22.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.550+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/3977612528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 msgr2=0x563bb0dd4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.550+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/3977612528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dd4010 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa2240959d0 tx=0x7fa224099640 comp rx=0 tx=0).stop 2023-07-18T03:31:22.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.550+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/3977612528 shutdown_connections 2023-07-18T03:31:22.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.550+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/3977612528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563bb0dc0fa0 0x563bb0dd4010 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.550+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/3977612528 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x563bb0e2dfa0 0x563bb0e303b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.550+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/3977612528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb0dbfc80 0x563bb0dd2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.550+0000 7fa2157fa700 0 client.15582 ms_handle_reset on v2:172.21.15.134:6838/3192254101 2023-07-18T03:31:22.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.550+0000 7fa23fdbfb80 1 --2- 172.21.15.40:0/3977612528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563bb0dc0b90 0x563bb0dd3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.550+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/3977612528 >> 172.21.15.40:0/3977612528 conn(0x563bb0db4da0 msgr2=0x563bb0dc7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.550+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/3977612528 shutdown_connections 2023-07-18T03:31:22.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.550+0000 7fa23fdbfb80 1 -- 172.21.15.40:0/3977612528 wait complete. 2023-07-18T03:31:22.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.551+0000 7fa23fdbfb80 20 client.15582 trim_cache size 0 max 16384 2023-07-18T03:31:22.558 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:31:22.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.551+0000 7fa233c39700 1 -- 172.21.15.40:0/800560138 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa2180a11d0 msgr2=0x7fa2180a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.551+0000 7fa233c39700 1 --2- 172.21.15.40:0/800560138 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa2180a11d0 0x7fa2180a3680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa228005950 tx=0x7fa2280058e0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.551+0000 7fa233c39700 1 -- 172.21.15.40:0/800560138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa22c100560 msgr2=0x7fa22c19bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.551+0000 7fa233c39700 1 --2- 172.21.15.40:0/800560138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa22c100560 0x7fa22c19bd20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa224007ee0 tx=0x7fa22400d3b0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.551+0000 7fa233c39700 1 -- 172.21.15.40:0/800560138 shutdown_connections 2023-07-18T03:31:22.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.551+0000 7fa233c39700 1 --2- 172.21.15.40:0/800560138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa22c100560 0x7fa22c19bd20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.551+0000 7fa233c39700 1 --2- 172.21.15.40:0/800560138 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa2180a11d0 0x7fa2180a3680 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.551+0000 7fa233c39700 1 --2- 172.21.15.40:0/800560138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa22c0fe9e0 0x7fa22c194770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.551+0000 7fa233c39700 1 --2- 172.21.15.40:0/800560138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa22c0ffbd0 0x7fa22c194cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.551+0000 7fa233c39700 1 -- 172.21.15.40:0/800560138 >> 172.21.15.40:0/800560138 conn(0x7fa22c0fa000 msgr2=0x7fa22c103100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.551+0000 7fa233c39700 1 -- 172.21.15.40:0/800560138 shutdown_connections 2023-07-18T03:31:22.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.551+0000 7fa233c39700 1 -- 172.21.15.40:0/800560138 wait complete. 2023-07-18T03:31:22.561 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:31:22.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.701+0000 7f28a1feb700 1 -- 172.21.15.40:0/1804594812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f289c102ee0 msgr2=0x7f289c1052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.701+0000 7f28a1feb700 1 --2- 172.21.15.40:0/1804594812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f289c102ee0 0x7f289c1052c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2890009990 tx=0x7f2890009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.701+0000 7f28a1feb700 1 -- 172.21.15.40:0/1804594812 shutdown_connections 2023-07-18T03:31:22.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.701+0000 7f28a1feb700 1 --2- 172.21.15.40:0/1804594812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c105800 0x7f289c107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.701+0000 7f28a1feb700 1 --2- 172.21.15.40:0/1804594812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c1005c0 0x7f289c1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.701+0000 7f28a1feb700 1 --2- 172.21.15.40:0/1804594812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f289c102ee0 0x7f289c1052c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.701+0000 7f28a1feb700 1 -- 172.21.15.40:0/1804594812 >> 172.21.15.40:0/1804594812 conn(0x7f289c0fa000 msgr2=0x7f289c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.701+0000 7f28a1feb700 1 -- 172.21.15.40:0/1804594812 shutdown_connections 2023-07-18T03:31:22.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.701+0000 7f28a1feb700 1 -- 172.21.15.40:0/1804594812 wait complete. 2023-07-18T03:31:22.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.702+0000 7f28a1feb700 1 Processor -- start 2023-07-18T03:31:22.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.702+0000 7f28a1feb700 1 -- start start 2023-07-18T03:31:22.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.702+0000 7f28a1feb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c1005c0 0x7f289c0fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.702+0000 7f28a1feb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f289c102ee0 0x7f289c0fe570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.702+0000 7f28a1feb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c105800 0x7f289c0feab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.702+0000 7f28a1feb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c0ff250 con 0x7f289c102ee0 2023-07-18T03:31:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.702+0000 7f28a1feb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c0ff3c0 con 0x7f289c1005c0 2023-07-18T03:31:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.702+0000 7f28a1feb700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c19f890 con 0x7f289c105800 2023-07-18T03:31:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.702+0000 7f289affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f289c102ee0 0x7f289c0fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.702+0000 7f289affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f289c102ee0 0x7f289c0fe570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58268/0 (socket says 172.21.15.40:58268) 2023-07-18T03:31:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.702+0000 7f289affd700 1 -- 172.21.15.40:0/2235717034 learned_addr learned my addr 172.21.15.40:0/2235717034 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.703+0000 7f289b7fe700 1 --2- 172.21.15.40:0/2235717034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c1005c0 0x7f289c0fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.703+0000 7f289bfff700 1 --2- 172.21.15.40:0/2235717034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c105800 0x7f289c0feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.703+0000 7f289affd700 1 -- 172.21.15.40:0/2235717034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c105800 msgr2=0x7f289c0feab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.703+0000 7f289affd700 1 --2- 172.21.15.40:0/2235717034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c105800 0x7f289c0feab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.703+0000 7f289affd700 1 -- 172.21.15.40:0/2235717034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c1005c0 msgr2=0x7f289c0fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.703+0000 7f289affd700 1 --2- 172.21.15.40:0/2235717034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c1005c0 0x7f289c0fff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.703+0000 7f289affd700 1 -- 172.21.15.40:0/2235717034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28900095f0 con 0x7f289c102ee0 2023-07-18T03:31:22.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.703+0000 7f289bfff700 1 --2- 172.21.15.40:0/2235717034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c105800 0x7f289c0feab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:22.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.703+0000 7f289affd700 1 --2- 172.21.15.40:0/2235717034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f289c102ee0 0x7f289c0fe570 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f28900098e0 tx=0x7f2890004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.703+0000 7f2898ff9700 1 -- 172.21.15.40:0/2235717034 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289001d070 con 0x7f289c102ee0 2023-07-18T03:31:22.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.703+0000 7f2898ff9700 1 -- 172.21.15.40:0/2235717034 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f289000fc70 con 0x7f289c102ee0 2023-07-18T03:31:22.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.703+0000 7f28a1feb700 1 -- 172.21.15.40:0/2235717034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289c19fb10 con 0x7f289c102ee0 2023-07-18T03:31:22.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.703+0000 7f2898ff9700 1 -- 172.21.15.40:0/2235717034 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2890017650 con 0x7f289c102ee0 2023-07-18T03:31:22.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.703+0000 7f28a1feb700 1 -- 172.21.15.40:0/2235717034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289c19ff30 con 0x7f289c102ee0 2023-07-18T03:31:22.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.706+0000 7f2898ff9700 1 -- 172.21.15.40:0/2235717034 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2890021410 con 0x7f289c102ee0 2023-07-18T03:31:22.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.707+0000 7f2898ff9700 1 --2- 172.21.15.40:0/2235717034 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f28840a1150 0x7f28840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.707+0000 7f2898ff9700 1 -- 172.21.15.40:0/2235717034 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f28900cde00 con 0x7f289c102ee0 2023-07-18T03:31:22.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.707+0000 7f289b7fe700 1 --2- 172.21.15.40:0/2235717034 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f28840a1150 0x7f28840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.707+0000 7f28ae171b80 1 Processor -- start 2023-07-18T03:31:22.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.707+0000 7f289b7fe700 1 --2- 172.21.15.40:0/2235717034 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f28840a1150 0x7f28840a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f288c009c60 tx=0x7f288c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.707+0000 7f28ae171b80 1 -- start start 2023-07-18T03:31:22.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.707+0000 7f28ae171b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6cca4d0 0x5631c6cca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.707+0000 7f28ae171b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6ccae20 0x5631c6ccb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.707+0000 7f28ae171b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccbc70 0x5631c6ccc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.708+0000 7f28ae171b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c6bcfcb0 con 0x5631c6ccbc70 2023-07-18T03:31:22.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.708+0000 7f28ae171b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c6c22a40 con 0x5631c6ccae20 2023-07-18T03:31:22.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.708+0000 7f28ae171b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c6c1bed0 con 0x5631c6cca4d0 2023-07-18T03:31:22.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.708+0000 7f289bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccbc70 0x5631c6ccc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.708+0000 7f289bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccbc70 0x5631c6ccc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58278/0 (socket says 172.21.15.40:58278) 2023-07-18T03:31:22.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.708+0000 7f289bfff700 1 -- 172.21.15.40:0/1799524650 learned_addr learned my addr 172.21.15.40:0/1799524650 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.708+0000 7f289affd700 1 --2- 172.21.15.40:0/1799524650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6ccae20 0x5631c6ccb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.708+0000 7f289b7fe700 1 --2- 172.21.15.40:0/1799524650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6cca4d0 0x5631c6cca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.708+0000 7f289bfff700 1 -- 172.21.15.40:0/1799524650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6cca4d0 msgr2=0x5631c6cca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.708+0000 7f289bfff700 1 --2- 172.21.15.40:0/1799524650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6cca4d0 0x5631c6cca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.708+0000 7f289bfff700 1 -- 172.21.15.40:0/1799524650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6ccae20 msgr2=0x5631c6ccb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.708+0000 7f289bfff700 1 --2- 172.21.15.40:0/1799524650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6ccae20 0x5631c6ccb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.708+0000 7f289bfff700 1 -- 172.21.15.40:0/1799524650 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631c6a49de0 con 0x5631c6ccbc70 2023-07-18T03:31:22.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.708+0000 7f289bfff700 1 --2- 172.21.15.40:0/1799524650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccbc70 0x5631c6ccc0e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2894009e00 tx=0x7f28940077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c879d3a8ca89c655 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.708+0000 7f2881ffb700 1 -- 172.21.15.40:0/1799524650 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2894014610 con 0x5631c6ccbc70 2023-07-18T03:31:22.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.708+0000 7f2881ffb700 1 -- 172.21.15.40:0/1799524650 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f289400f050 con 0x5631c6ccbc70 2023-07-18T03:31:22.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.708+0000 7f2881ffb700 1 -- 172.21.15.40:0/1799524650 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2894013940 con 0x5631c6ccbc70 2023-07-18T03:31:22.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.709+0000 7f28ae171b80 1 -- 172.21.15.40:0/1799524650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccbc70 msgr2=0x5631c6ccc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.709+0000 7f28ae171b80 1 --2- 172.21.15.40:0/1799524650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccbc70 0x5631c6ccc0e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2894009e00 tx=0x7f28940077e0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.709+0000 7f28ae171b80 1 -- 172.21.15.40:0/1799524650 shutdown_connections 2023-07-18T03:31:22.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.709+0000 7f28ae171b80 1 --2- 172.21.15.40:0/1799524650 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccbc70 0x5631c6ccc0e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.709+0000 7f28ae171b80 1 --2- 172.21.15.40:0/1799524650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6cca4d0 0x5631c6cca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.709+0000 7f28ae171b80 1 --2- 172.21.15.40:0/1799524650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6ccae20 0x5631c6ccb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.709+0000 7f28ae171b80 1 -- 172.21.15.40:0/1799524650 >> 172.21.15.40:0/1799524650 conn(0x5631c6cbfda0 msgr2=0x5631c6cc51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.709+0000 7f28ae171b80 1 -- 172.21.15.40:0/1799524650 shutdown_connections 2023-07-18T03:31:22.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.709+0000 7f28ae171b80 1 -- 172.21.15.40:0/1799524650 wait complete. 2023-07-18T03:31:22.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.709+0000 7f28ae171b80 1 -- start start 2023-07-18T03:31:22.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.710+0000 7f28ae171b80 1 Processor -- start 2023-07-18T03:31:22.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.710+0000 7f28ae171b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6cca4d0 0x5631c6cd40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.710+0000 7f28ae171b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccae20 0x5631c6cd4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.710+0000 7f28ae171b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6ccbc70 0x5631c6cdb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.710+0000 7f28ae171b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c6bbbac0 con 0x5631c6ccae20 2023-07-18T03:31:22.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.710+0000 7f28ae171b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c6bbb7c0 con 0x5631c6cca4d0 2023-07-18T03:31:22.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.710+0000 7f28ae171b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c6a9f920 con 0x5631c6ccbc70 2023-07-18T03:31:22.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.710+0000 7f28ae171b80 10 client.0 _ll_register_callbacks cb 0x5631c6bb9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:22.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.710+0000 7f289affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccae20 0x5631c6cd4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.710+0000 7f289affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccae20 0x5631c6cd4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58282/0 (socket says 172.21.15.40:58282) 2023-07-18T03:31:22.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.710+0000 7f289affd700 1 -- 172.21.15.40:0/2588896944 learned_addr learned my addr 172.21.15.40:0/2588896944 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.710+0000 7f289bfff700 1 --2- 172.21.15.40:0/2588896944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6ccbc70 0x5631c6cdb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.711+0000 7f289affd700 1 -- 172.21.15.40:0/2588896944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6ccbc70 msgr2=0x5631c6cdb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.711+0000 7f289affd700 1 --2- 172.21.15.40:0/2588896944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6ccbc70 0x5631c6cdb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.711+0000 7f289affd700 1 -- 172.21.15.40:0/2588896944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6cca4d0 msgr2=0x5631c6cd40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.711+0000 7f289affd700 1 --2- 172.21.15.40:0/2588896944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6cca4d0 0x5631c6cd40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.711+0000 7f289affd700 1 -- 172.21.15.40:0/2588896944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2894009a60 con 0x5631c6ccae20 2023-07-18T03:31:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.711+0000 7f289affd700 1 --2- 172.21.15.40:0/2588896944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccae20 0x5631c6cd4610 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f28900cb8e0 tx=0x7f2890009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143c4494308426d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.711+0000 7f287affd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.711+0000 7f287affd700 1 -- 172.21.15.40:0/2588896944 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289009adc0 con 0x5631c6ccae20 2023-07-18T03:31:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.711+0000 7f287affd700 1 -- 172.21.15.40:0/2588896944 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f289000fde0 con 0x5631c6ccae20 2023-07-18T03:31:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.711+0000 7f28ae171b80 1 -- 172.21.15.40:0/2588896944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631c6b754f0 con 0x5631c6ccae20 2023-07-18T03:31:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.711+0000 7f287affd700 1 -- 172.21.15.40:0/2588896944 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f289009b280 con 0x5631c6ccae20 2023-07-18T03:31:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.711+0000 7f28ae171b80 10 client.15594 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:22.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.711+0000 7f28ae171b80 1 -- 172.21.15.40:0/2588896944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5631c6a32c60 con 0x5631c6ccae20 2023-07-18T03:31:22.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.711+0000 7f287affd700 1 -- 172.21.15.40:0/2588896944 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5631c6b754f0 con 0x5631c6ccae20 2023-07-18T03:31:22.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.712+0000 7f287affd700 1 -- 172.21.15.40:0/2588896944 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2890084ad0 con 0x5631c6ccae20 2023-07-18T03:31:22.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.712+0000 7f28ae171b80 10 client.15594 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:22.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.712+0000 7f28ae171b80 10 client.15594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.712+0000 7f28ae171b80 20 client.15594 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.712+0000 7f28ae171b80 1 --2- 172.21.15.40:0/2588896944 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5631c6cdfb30 0x5631c6ce1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.712+0000 7f28ae171b80 4 client.15594 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.712+0000 7f28ae171b80 1 -- 172.21.15.40:0/2588896944 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631c6c14eb0 con 0x5631c6cdfb30 2023-07-18T03:31:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.713+0000 7f289b7fe700 1 --2- 172.21.15.40:0/2588896944 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5631c6cdfb30 0x5631c6ce1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.713+0000 7f287affd700 10 client.15594 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.713+0000 7f289b7fe700 1 --2- 172.21.15.40:0/2588896944 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5631c6cdfb30 0x5631c6ce1f40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a61111673cd9744e server_cookie=860d9ee2383fe860 in_seq=0 out_seq=0 2023-07-18T03:31:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.714+0000 7f287affd700 1 -- 172.21.15.40:0/2588896944 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5631c6c14eb0 con 0x5631c6cdfb30 2023-07-18T03:31:22.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.714+0000 7f287affd700 10 client.15594 handle_command_reply: tid=0 2023-07-18T03:31:22.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.714+0000 7f28ae171b80 1 client.15594 shutdown 2023-07-18T03:31:22.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.714+0000 7f28ae171b80 1 -- 172.21.15.40:0/2588896944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccae20 msgr2=0x5631c6cd4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.714+0000 7f28ae171b80 1 --2- 172.21.15.40:0/2588896944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccae20 0x5631c6cd4610 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f28900cb8e0 tx=0x7f2890009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.714+0000 7f28ae171b80 1 -- 172.21.15.40:0/2588896944 shutdown_connections 2023-07-18T03:31:22.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.714+0000 7f28ae171b80 1 --2- 172.21.15.40:0/2588896944 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5631c6cdfb30 0x5631c6ce1f40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.714+0000 7f28ae171b80 1 --2- 172.21.15.40:0/2588896944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6ccbc70 0x5631c6cdb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.714+0000 7f28ae171b80 1 --2- 172.21.15.40:0/2588896944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6cca4d0 0x5631c6cd40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.714+0000 7f28ae171b80 1 --2- 172.21.15.40:0/2588896944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccae20 0x5631c6cd4610 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.714+0000 7f28ae171b80 1 -- 172.21.15.40:0/2588896944 >> 172.21.15.40:0/2588896944 conn(0x5631c6cbfda0 msgr2=0x5631c6cc48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.714+0000 7f287affd700 0 client.15594 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:22.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.714+0000 7f28ae171b80 1 -- 172.21.15.40:0/2588896944 shutdown_connections 2023-07-18T03:31:22.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.714+0000 7f28ae171b80 1 -- 172.21.15.40:0/2588896944 wait complete. 2023-07-18T03:31:22.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.715+0000 7f28ae171b80 20 client.15594 trim_cache size 0 max 16384 2023-07-18T03:31:22.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.728+0000 7f28ae171b80 1 Processor -- start 2023-07-18T03:31:22.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.728+0000 7f28ae171b80 1 -- start start 2023-07-18T03:31:22.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.728+0000 7f28ae171b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6cca4d0 0x5631c6cc1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.728+0000 7f28ae171b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccae20 0x5631c6cdb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.728+0000 7f28ae171b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6ccbc70 0x5631c6ce1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.728+0000 7f28ae171b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c6cd3580 con 0x5631c6ccae20 2023-07-18T03:31:22.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.728+0000 7f28ae171b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c6cdf950 con 0x5631c6cca4d0 2023-07-18T03:31:22.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.728+0000 7f28ae171b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c6ccc400 con 0x5631c6ccbc70 2023-07-18T03:31:22.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.728+0000 7f289affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccae20 0x5631c6cdb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.728+0000 7f289affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccae20 0x5631c6cdb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58290/0 (socket says 172.21.15.40:58290) 2023-07-18T03:31:22.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.728+0000 7f289affd700 1 -- 172.21.15.40:0/4128284743 learned_addr learned my addr 172.21.15.40:0/4128284743 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.728+0000 7f289b7fe700 1 --2- 172.21.15.40:0/4128284743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6cca4d0 0x5631c6cc1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.729+0000 7f289bfff700 1 --2- 172.21.15.40:0/4128284743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6ccbc70 0x5631c6ce1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.729+0000 7f289affd700 1 -- 172.21.15.40:0/4128284743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6ccbc70 msgr2=0x5631c6ce1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.729+0000 7f289affd700 1 --2- 172.21.15.40:0/4128284743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6ccbc70 0x5631c6ce1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.729+0000 7f289affd700 1 -- 172.21.15.40:0/4128284743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6cca4d0 msgr2=0x5631c6cc1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.729+0000 7f289affd700 1 --2- 172.21.15.40:0/4128284743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6cca4d0 0x5631c6cc1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.729+0000 7f289affd700 1 -- 172.21.15.40:0/4128284743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631c6a70250 con 0x5631c6ccae20 2023-07-18T03:31:22.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.729+0000 7f289affd700 1 --2- 172.21.15.40:0/4128284743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccae20 0x5631c6cdb610 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2890097780 tx=0x7f289009aad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d68bacd77df847f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.729+0000 7f28827fc700 1 -- 172.21.15.40:0/4128284743 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289000fde0 con 0x5631c6ccae20 2023-07-18T03:31:22.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.729+0000 7f28827fc700 1 -- 172.21.15.40:0/4128284743 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f289009b280 con 0x5631c6ccae20 2023-07-18T03:31:22.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.729+0000 7f28827fc700 1 -- 172.21.15.40:0/4128284743 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289009a5a0 con 0x5631c6ccae20 2023-07-18T03:31:22.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.730+0000 7f28ae171b80 1 -- 172.21.15.40:0/4128284743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccae20 msgr2=0x5631c6cdb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.730+0000 7f28ae171b80 1 --2- 172.21.15.40:0/4128284743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccae20 0x5631c6cdb610 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2890097780 tx=0x7f289009aad0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.730+0000 7f28ae171b80 1 -- 172.21.15.40:0/4128284743 shutdown_connections 2023-07-18T03:31:22.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.730+0000 7f28ae171b80 1 --2- 172.21.15.40:0/4128284743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6ccbc70 0x5631c6ce1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.730+0000 7f28ae171b80 1 --2- 172.21.15.40:0/4128284743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6cca4d0 0x5631c6cc1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.730+0000 7f28ae171b80 1 --2- 172.21.15.40:0/4128284743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6ccae20 0x5631c6cdb610 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.730+0000 7f28ae171b80 1 -- 172.21.15.40:0/4128284743 >> 172.21.15.40:0/4128284743 conn(0x5631c6cbfda0 msgr2=0x5631c6ce68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.730+0000 7f28ae171b80 1 -- 172.21.15.40:0/4128284743 shutdown_connections 2023-07-18T03:31:22.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.730+0000 7f28ae171b80 1 -- 172.21.15.40:0/4128284743 wait complete. 2023-07-18T03:31:22.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.730+0000 7f28ae171b80 1 -- start start 2023-07-18T03:31:22.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f28ae171b80 1 Processor -- start 2023-07-18T03:31:22.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f28ae171b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6cca4d0 0x5631c6cd7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f28ae171b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6ccae20 0x5631c6cd7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f28ae171b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6ccbc70 0x5631c6cdaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f28ae171b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c6cdaff0 con 0x5631c6cca4d0 2023-07-18T03:31:22.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f28ae171b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c6cdb160 con 0x5631c6ccbc70 2023-07-18T03:31:22.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f28ae171b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631c6cdb320 con 0x5631c6ccae20 2023-07-18T03:31:22.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f28ae171b80 10 client.0 _ll_register_callbacks cb 0x7f289000b2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:22.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f289b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6cca4d0 0x5631c6cd7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f289b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6cca4d0 0x5631c6cd7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58294/0 (socket says 172.21.15.40:58294) 2023-07-18T03:31:22.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f289b7fe700 1 -- 172.21.15.40:0/2748974655 learned_addr learned my addr 172.21.15.40:0/2748974655 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:22.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f289bfff700 1 --2- 172.21.15.40:0/2748974655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6ccbc70 0x5631c6cdaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f289affd700 1 --2- 172.21.15.40:0/2748974655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6ccae20 0x5631c6cd7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f289b7fe700 1 -- 172.21.15.40:0/2748974655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6ccae20 msgr2=0x5631c6cd7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f289b7fe700 1 --2- 172.21.15.40:0/2748974655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6ccae20 0x5631c6cd7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f289b7fe700 1 -- 172.21.15.40:0/2748974655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6ccbc70 msgr2=0x5631c6cdaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:22.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f289b7fe700 1 --2- 172.21.15.40:0/2748974655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6ccbc70 0x5631c6cdaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f289b7fe700 1 -- 172.21.15.40:0/2748974655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631c6a32c60 con 0x5631c6cca4d0 2023-07-18T03:31:22.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f289b7fe700 1 --2- 172.21.15.40:0/2748974655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6cca4d0 0x5631c6cd7550 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x5631c6ce2840 tx=0x7f288c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440a2d6be0306ff6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:22.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.731+0000 7f287affd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:22.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.732+0000 7f287affd700 1 -- 172.21.15.40:0/2748974655 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c023070 con 0x5631c6cca4d0 2023-07-18T03:31:22.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.732+0000 7f287affd700 1 -- 172.21.15.40:0/2748974655 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f288c0255c0 con 0x5631c6cca4d0 2023-07-18T03:31:22.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.732+0000 7f28ae171b80 1 -- 172.21.15.40:0/2748974655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f289400f050 con 0x5631c6cca4d0 2023-07-18T03:31:22.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.732+0000 7f287affd700 1 -- 172.21.15.40:0/2748974655 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f288c025860 con 0x5631c6cca4d0 2023-07-18T03:31:22.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.732+0000 7f28ae171b80 10 client.15600 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:22.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.732+0000 7f28ae171b80 1 -- 172.21.15.40:0/2748974655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5631c6a482b0 con 0x5631c6cca4d0 2023-07-18T03:31:22.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.732+0000 7f287affd700 1 -- 172.21.15.40:0/2748974655 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f289400f050 con 0x5631c6cca4d0 2023-07-18T03:31:22.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.732+0000 7f287affd700 1 -- 172.21.15.40:0/2748974655 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f288c038050 con 0x5631c6cca4d0 2023-07-18T03:31:22.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.732+0000 7f28ae171b80 10 client.15600 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.732+0000 7f28ae171b80 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.732+0000 7f28ae171b80 20 client.15600 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.732+0000 7f28ae171b80 1 --2- 172.21.15.40:0/2748974655 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5631c6d34b30 0x5631c6d36f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.732+0000 7f28ae171b80 4 client.15600 mds_command: new command op to 24527 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:31:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.732+0000 7f28ae171b80 1 -- 172.21.15.40:0/2748974655 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5631c6bd7050 con 0x5631c6d34b30 2023-07-18T03:31:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.733+0000 7f289affd700 1 --2- 172.21.15.40:0/2748974655 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5631c6d34b30 0x5631c6d36f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:22.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.733+0000 7f287affd700 10 client.15600 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:22.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.733+0000 7f289affd700 1 --2- 172.21.15.40:0/2748974655 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5631c6d34b30 0x5631c6d36f40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c47c0575c2a80e7f server_cookie=13a6f6349e405e7c in_seq=0 out_seq=0 2023-07-18T03:31:22.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.733+0000 7f287affd700 1 -- 172.21.15.40:0/2748974655 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5631c6bd7050 con 0x5631c6d34b30 2023-07-18T03:31:22.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.733+0000 7f287affd700 10 client.15600 handle_command_reply: tid=0 2023-07-18T03:31:22.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.733+0000 7f28ae171b80 1 client.15600 shutdown 2023-07-18T03:31:22.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.733+0000 7f28ae171b80 1 -- 172.21.15.40:0/2748974655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6cca4d0 msgr2=0x5631c6cd7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.733+0000 7f28ae171b80 1 --2- 172.21.15.40:0/2748974655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6cca4d0 0x5631c6cd7550 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x5631c6ce2840 tx=0x7f288c0047c0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.733+0000 7f28ae171b80 1 -- 172.21.15.40:0/2748974655 shutdown_connections 2023-07-18T03:31:22.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.733+0000 7f28ae171b80 1 --2- 172.21.15.40:0/2748974655 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5631c6d34b30 0x5631c6d36f40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.733+0000 7f28ae171b80 1 --2- 172.21.15.40:0/2748974655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631c6ccbc70 0x5631c6cdaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.733+0000 7f28ae171b80 1 --2- 172.21.15.40:0/2748974655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631c6cca4d0 0x5631c6cd7550 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.733+0000 7f287affd700 0 client.15600 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:22.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.733+0000 7f28ae171b80 1 --2- 172.21.15.40:0/2748974655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631c6ccae20 0x5631c6cd7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.733+0000 7f28ae171b80 1 -- 172.21.15.40:0/2748974655 >> 172.21.15.40:0/2748974655 conn(0x5631c6cbfda0 msgr2=0x5631c6cc1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.734+0000 7f28ae171b80 1 -- 172.21.15.40:0/2748974655 shutdown_connections 2023-07-18T03:31:22.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.734+0000 7f28ae171b80 1 -- 172.21.15.40:0/2748974655 wait complete. 2023-07-18T03:31:22.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.734+0000 7f28ae171b80 20 client.15600 trim_cache size 0 max 16384 2023-07-18T03:31:22.741 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:31:22.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.734+0000 7f28a1feb700 1 -- 172.21.15.40:0/2235717034 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f28840a1150 msgr2=0x7f28840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.734+0000 7f28a1feb700 1 --2- 172.21.15.40:0/2235717034 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f28840a1150 0x7f28840a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f288c009c60 tx=0x7f288c009260 comp rx=0 tx=0).stop 2023-07-18T03:31:22.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.734+0000 7f28a1feb700 1 -- 172.21.15.40:0/2235717034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f289c102ee0 msgr2=0x7f289c0fe570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:22.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.734+0000 7f28a1feb700 1 --2- 172.21.15.40:0/2235717034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f289c102ee0 0x7f289c0fe570 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f28900098e0 tx=0x7f2890004730 comp rx=0 tx=0).stop 2023-07-18T03:31:22.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.734+0000 7f28a1feb700 1 -- 172.21.15.40:0/2235717034 shutdown_connections 2023-07-18T03:31:22.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.734+0000 7f28a1feb700 1 --2- 172.21.15.40:0/2235717034 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f28840a1150 0x7f28840a3600 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.734+0000 7f28a1feb700 1 --2- 172.21.15.40:0/2235717034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f289c105800 0x7f289c0feab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.734+0000 7f28a1feb700 1 --2- 172.21.15.40:0/2235717034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f289c1005c0 0x7f289c0fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.734+0000 7f28a1feb700 1 --2- 172.21.15.40:0/2235717034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f289c102ee0 0x7f289c0fe570 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:22.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.734+0000 7f28a1feb700 1 -- 172.21.15.40:0/2235717034 >> 172.21.15.40:0/2235717034 conn(0x7f289c0fa000 msgr2=0x7f289c101240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:22.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.734+0000 7f28a1feb700 1 -- 172.21.15.40:0/2235717034 shutdown_connections 2023-07-18T03:31:22.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:22.734+0000 7f28a1feb700 1 -- 172.21.15.40:0/2235717034 wait complete. 2023-07-18T03:31:23.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:22 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1979102791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:31:23.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:22 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1979102791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:31:23.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:22 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1979102791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:31:23.745 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:31:23.745 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:31:23.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.881+0000 7f89bd8a9700 1 -- 172.21.15.40:0/1731956914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b8102f20 msgr2=0x7f89b8105300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:23.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.881+0000 7f89bd8a9700 1 --2- 172.21.15.40:0/1731956914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b8102f20 0x7f89b8105300 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009990 tx=0x7f89ac009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.881+0000 7f89bd8a9700 1 -- 172.21.15.40:0/1731956914 shutdown_connections 2023-07-18T03:31:23.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.881+0000 7f89bd8a9700 1 --2- 172.21.15.40:0/1731956914 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b8105840 0x7f89b8107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.881+0000 7f89bd8a9700 1 --2- 172.21.15.40:0/1731956914 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b8100600 0x7f89b81029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.881+0000 7f89bd8a9700 1 --2- 172.21.15.40:0/1731956914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b8102f20 0x7f89b8105300 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.881+0000 7f89bd8a9700 1 -- 172.21.15.40:0/1731956914 >> 172.21.15.40:0/1731956914 conn(0x7f89b80fa020 msgr2=0x7f89b80fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:23.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.881+0000 7f89bd8a9700 1 -- 172.21.15.40:0/1731956914 shutdown_connections 2023-07-18T03:31:23.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.881+0000 7f89bd8a9700 1 -- 172.21.15.40:0/1731956914 wait complete. 2023-07-18T03:31:23.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.882+0000 7f89bd8a9700 1 Processor -- start 2023-07-18T03:31:23.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.882+0000 7f89bd8a9700 1 -- start start 2023-07-18T03:31:23.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89bd8a9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b8100600 0x7f89b81958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89bd8a9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b8102f20 0x7f89b8195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89bd8a9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b8105840 0x7f89b819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89bd8a9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b819d3e0 con 0x7f89b8105840 2023-07-18T03:31:23.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89bd8a9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b819d520 con 0x7f89b8100600 2023-07-18T03:31:23.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89bd8a9700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b819d6b0 con 0x7f89b8102f20 2023-07-18T03:31:23.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89b77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b8105840 0x7f89b819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:23.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89b77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b8105840 0x7f89b819ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58312/0 (socket says 172.21.15.40:58312) 2023-07-18T03:31:23.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89b77fe700 1 -- 172.21.15.40:0/852931955 learned_addr learned my addr 172.21.15.40:0/852931955 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:23.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89b6ffd700 1 --2- 172.21.15.40:0/852931955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b8100600 0x7f89b81958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:23.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89b67fc700 1 --2- 172.21.15.40:0/852931955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b8102f20 0x7f89b8195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:23.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89b77fe700 1 -- 172.21.15.40:0/852931955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b8102f20 msgr2=0x7f89b8195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:23.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89b77fe700 1 --2- 172.21.15.40:0/852931955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b8102f20 0x7f89b8195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89b77fe700 1 -- 172.21.15.40:0/852931955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b8100600 msgr2=0x7f89b81958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:23.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89b77fe700 1 --2- 172.21.15.40:0/852931955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b8100600 0x7f89b81958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89b77fe700 1 -- 172.21.15.40:0/852931955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac0095f0 con 0x7f89b8105840 2023-07-18T03:31:23.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89b67fc700 1 --2- 172.21.15.40:0/852931955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b8102f20 0x7f89b8195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:23.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.883+0000 7f89b77fe700 1 --2- 172.21.15.40:0/852931955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b8105840 0x7f89b819ce10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f89b0007ee0 tx=0x7f89b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:23.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.884+0000 7f89bc8a7700 1 -- 172.21.15.40:0/852931955 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b000ddd0 con 0x7f89b8105840 2023-07-18T03:31:23.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.884+0000 7f89bd8a9700 1 -- 172.21.15.40:0/852931955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b819d990 con 0x7f89b8105840 2023-07-18T03:31:23.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.884+0000 7f89bc8a7700 1 -- 172.21.15.40:0/852931955 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b000f040 con 0x7f89b8105840 2023-07-18T03:31:23.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.884+0000 7f89bc8a7700 1 -- 172.21.15.40:0/852931955 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b00048e0 con 0x7f89b8105840 2023-07-18T03:31:23.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.884+0000 7f89bd8a9700 1 -- 172.21.15.40:0/852931955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b819dea0 con 0x7f89b8105840 2023-07-18T03:31:23.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.886+0000 7f89bc8a7700 1 -- 172.21.15.40:0/852931955 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f89b0003a00 con 0x7f89b8105840 2023-07-18T03:31:23.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.886+0000 7f89bc8a7700 1 --2- 172.21.15.40:0/852931955 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89a00a1150 0x7f89a00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.887+0000 7f89bc8a7700 1 -- 172.21.15.40:0/852931955 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f89b00cc1e0 con 0x7f89b8105840 2023-07-18T03:31:23.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.887+0000 7f89b6ffd700 1 --2- 172.21.15.40:0/852931955 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89a00a1150 0x7f89a00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:23.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.887+0000 7f89b6ffd700 1 --2- 172.21.15.40:0/852931955 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89a00a1150 0x7f89a00a3600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f89a8005950 tx=0x7f89a800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:23.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.887+0000 7f89c9a2fb80 1 Processor -- start 2023-07-18T03:31:23.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89c9a2fb80 1 -- start start 2023-07-18T03:31:23.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89c9a2fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b764619550 0x55b764619960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89c9a2fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b764619ea0 0x55b76461a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89c9a2fb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b76461acf0 0x55b76461b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89c9a2fb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76451ed30 con 0x55b76461acf0 2023-07-18T03:31:23.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89c9a2fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b764571ac0 con 0x55b764619ea0 2023-07-18T03:31:23.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89c9a2fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76456af50 con 0x55b764619550 2023-07-18T03:31:23.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89b77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b76461acf0 0x55b76461b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:23.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89b77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b76461acf0 0x55b76461b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58320/0 (socket says 172.21.15.40:58320) 2023-07-18T03:31:23.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89b77fe700 1 -- 172.21.15.40:0/3541194664 learned_addr learned my addr 172.21.15.40:0/3541194664 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:23.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89b67fc700 1 --2- 172.21.15.40:0/3541194664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b764619ea0 0x55b76461a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:23.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89b6ffd700 1 --2- 172.21.15.40:0/3541194664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b764619550 0x55b764619960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:23.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89b77fe700 1 -- 172.21.15.40:0/3541194664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b764619550 msgr2=0x55b764619960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:23.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89b77fe700 1 --2- 172.21.15.40:0/3541194664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b764619550 0x55b764619960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89b77fe700 1 -- 172.21.15.40:0/3541194664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b764619ea0 msgr2=0x55b76461a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:23.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89b77fe700 1 --2- 172.21.15.40:0/3541194664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b764619ea0 0x55b76461a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89b77fe700 1 -- 172.21.15.40:0/3541194664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b764398e60 con 0x55b76461acf0 2023-07-18T03:31:23.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.888+0000 7f89b77fe700 1 --2- 172.21.15.40:0/3541194664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b76461acf0 0x55b76461b160 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f89b0096040 tx=0x7f89b0003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d6e8cbc9400706 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:23.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.889+0000 7f899dffb700 1 -- 172.21.15.40:0/3541194664 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b00995d0 con 0x55b76461acf0 2023-07-18T03:31:23.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.889+0000 7f899dffb700 1 -- 172.21.15.40:0/3541194664 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b009d600 con 0x55b76461acf0 2023-07-18T03:31:23.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.889+0000 7f899dffb700 1 -- 172.21.15.40:0/3541194664 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b00840b0 con 0x55b76461acf0 2023-07-18T03:31:23.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.889+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3541194664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b76461acf0 msgr2=0x55b76461b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:23.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.889+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3541194664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b76461acf0 0x55b76461b160 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f89b0096040 tx=0x7f89b0003f50 comp rx=0 tx=0).stop 2023-07-18T03:31:23.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.889+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3541194664 shutdown_connections 2023-07-18T03:31:23.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.889+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3541194664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b76461acf0 0x55b76461b160 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.889+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3541194664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b764619550 0x55b764619960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.889+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3541194664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b764619ea0 0x55b76461a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.889+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3541194664 >> 172.21.15.40:0/3541194664 conn(0x55b76460ee20 msgr2=0x55b764614250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.890+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3541194664 shutdown_connections 2023-07-18T03:31:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.890+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3541194664 wait complete. 2023-07-18T03:31:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.890+0000 7f89c9a2fb80 1 -- start start 2023-07-18T03:31:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.890+0000 7f89c9a2fb80 1 Processor -- start 2023-07-18T03:31:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.890+0000 7f89c9a2fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b764619550 0x55b7646231a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.890+0000 7f89c9a2fb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619ea0 0x55b7646236e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.890+0000 7f89c9a2fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b76461acf0 0x55b76462a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.890+0000 7f89c9a2fb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76450ab40 con 0x55b764619ea0 2023-07-18T03:31:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.890+0000 7f89c9a2fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b76450a840 con 0x55b764619550 2023-07-18T03:31:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.890+0000 7f89c9a2fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7643ee9a0 con 0x55b76461acf0 2023-07-18T03:31:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.890+0000 7f89c9a2fb80 10 client.0 _ll_register_callbacks cb 0x55b7645087e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f89b67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619ea0 0x55b7646236e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f89b67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619ea0 0x55b7646236e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58324/0 (socket says 172.21.15.40:58324) 2023-07-18T03:31:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f89b67fc700 1 -- 172.21.15.40:0/3823028704 learned_addr learned my addr 172.21.15.40:0/3823028704 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f89b6ffd700 1 --2- 172.21.15.40:0/3823028704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b764619550 0x55b7646231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f89b67fc700 1 -- 172.21.15.40:0/3823028704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b76461acf0 msgr2=0x55b76462a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f89b67fc700 1 --2- 172.21.15.40:0/3823028704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b76461acf0 0x55b76462a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f89b67fc700 1 -- 172.21.15.40:0/3823028704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b764619550 msgr2=0x55b7646231a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f89b67fc700 1 --2- 172.21.15.40:0/3823028704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b764619550 0x55b7646231a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f89b67fc700 1 -- 172.21.15.40:0/3823028704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b819dea0 con 0x55b764619ea0 2023-07-18T03:31:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f89b6ffd700 1 --2- 172.21.15.40:0/3823028704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b764619550 0x55b7646231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:31:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f89b67fc700 1 --2- 172.21.15.40:0/3823028704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619ea0 0x55b7646236e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00bc40 tx=0x7f89ac009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84479dcae2748f43 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f8996ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:23.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f8996ffd700 1 -- 172.21.15.40:0/3823028704 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac003ca0 con 0x55b764619ea0 2023-07-18T03:31:23.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f8996ffd700 1 -- 172.21.15.40:0/3823028704 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f89ac003e00 con 0x55b764619ea0 2023-07-18T03:31:23.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3823028704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7644c4570 con 0x55b764619ea0 2023-07-18T03:31:23.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f8996ffd700 1 -- 172.21.15.40:0/3823028704 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89ac0050d0 con 0x55b764619ea0 2023-07-18T03:31:23.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f89c9a2fb80 10 client.15612 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:23.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.891+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3823028704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b764381ce0 con 0x55b764619ea0 2023-07-18T03:31:23.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.892+0000 7f8996ffd700 1 -- 172.21.15.40:0/3823028704 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b7644c4570 con 0x55b764619ea0 2023-07-18T03:31:23.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.892+0000 7f8996ffd700 1 -- 172.21.15.40:0/3823028704 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f89ac02d020 con 0x55b764619ea0 2023-07-18T03:31:23.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.892+0000 7f89c9a2fb80 10 client.15612 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:23.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.892+0000 7f89c9a2fb80 10 client.15612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:23.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.892+0000 7f89c9a2fb80 20 client.15612 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:23.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.892+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3823028704 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55b76462ef70 0x55b764631380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.892+0000 7f89c9a2fb80 4 client.15612 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:23.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.892+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3823028704 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b764563f30 con 0x55b76462ef70 2023-07-18T03:31:23.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.892+0000 7f89b6ffd700 1 --2- 172.21.15.40:0/3823028704 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55b76462ef70 0x55b764631380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:23.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.893+0000 7f89b6ffd700 1 --2- 172.21.15.40:0/3823028704 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55b76462ef70 0x55b764631380 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a75ab2d3cbde9b server_cookie=d3663ec66db12236 in_seq=0 out_seq=0 2023-07-18T03:31:23.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.893+0000 7f8996ffd700 10 client.15612 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:23.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.893+0000 7f8996ffd700 1 -- 172.21.15.40:0/3823028704 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b764563f30 con 0x55b76462ef70 2023-07-18T03:31:23.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.893+0000 7f8996ffd700 10 client.15612 handle_command_reply: tid=0 2023-07-18T03:31:23.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.893+0000 7f89c9a2fb80 1 client.15612 shutdown 2023-07-18T03:31:23.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.893+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3823028704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619ea0 msgr2=0x55b7646236e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:23.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.893+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3823028704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619ea0 0x55b7646236e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00bc40 tx=0x7f89ac009f90 comp rx=0 tx=0).stop 2023-07-18T03:31:23.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.894+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3823028704 shutdown_connections 2023-07-18T03:31:23.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.894+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3823028704 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55b76462ef70 0x55b764631380 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.894+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3823028704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b76461acf0 0x55b76462a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.894+0000 7f8996ffd700 0 client.15612 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:23.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.894+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3823028704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b764619550 0x55b7646231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.894+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3823028704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619ea0 0x55b7646236e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.894+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3823028704 >> 172.21.15.40:0/3823028704 conn(0x55b76460ee20 msgr2=0x55b764613930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:23.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.894+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3823028704 shutdown_connections 2023-07-18T03:31:23.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.894+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3823028704 wait complete. 2023-07-18T03:31:23.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.894+0000 7f89c9a2fb80 20 client.15612 trim_cache size 0 max 16384 2023-07-18T03:31:23.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.911+0000 7f89c9a2fb80 1 Processor -- start 2023-07-18T03:31:23.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.911+0000 7f89c9a2fb80 1 -- start start 2023-07-18T03:31:23.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.911+0000 7f89c9a2fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b76460ee20 0x55b76461dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.911+0000 7f89c9a2fb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619550 0x55b76462a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.911+0000 7f89c9a2fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b764619ea0 0x55b76461e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.911+0000 7f89c9a2fb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b764623eb0 con 0x55b764619550 2023-07-18T03:31:23.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.911+0000 7f89c9a2fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b764620e30 con 0x55b76460ee20 2023-07-18T03:31:23.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.911+0000 7f89c9a2fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7646242f0 con 0x55b764619ea0 2023-07-18T03:31:23.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.911+0000 7f89b67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619550 0x55b76462a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:23.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.911+0000 7f89b67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619550 0x55b76462a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58332/0 (socket says 172.21.15.40:58332) 2023-07-18T03:31:23.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.911+0000 7f89b67fc700 1 -- 172.21.15.40:0/3440135103 learned_addr learned my addr 172.21.15.40:0/3440135103 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:23.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.911+0000 7f89b6ffd700 1 --2- 172.21.15.40:0/3440135103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b76460ee20 0x55b76461dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:23.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.912+0000 7f89b77fe700 1 --2- 172.21.15.40:0/3440135103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b764619ea0 0x55b76461e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:23.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.912+0000 7f89b67fc700 1 -- 172.21.15.40:0/3440135103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b764619ea0 msgr2=0x55b76461e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:23.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.912+0000 7f89b67fc700 1 --2- 172.21.15.40:0/3440135103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b764619ea0 0x55b76461e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.912+0000 7f89b67fc700 1 -- 172.21.15.40:0/3440135103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b76460ee20 msgr2=0x55b76461dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:23.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.912+0000 7f89b67fc700 1 --2- 172.21.15.40:0/3440135103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b76460ee20 0x55b76461dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.912+0000 7f89b67fc700 1 -- 172.21.15.40:0/3440135103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7643bf2d0 con 0x55b764619550 2023-07-18T03:31:23.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.912+0000 7f89b67fc700 1 --2- 172.21.15.40:0/3440135103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619550 0x55b76462a6e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00fe20 tx=0x7f89ac0254c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1f74e5d9eb33a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:23.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.912+0000 7f899e7fc700 1 -- 172.21.15.40:0/3440135103 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac025610 con 0x55b764619550 2023-07-18T03:31:23.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.912+0000 7f899e7fc700 1 -- 172.21.15.40:0/3440135103 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89ac020040 con 0x55b764619550 2023-07-18T03:31:23.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.912+0000 7f899e7fc700 1 -- 172.21.15.40:0/3440135103 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac025a30 con 0x55b764619550 2023-07-18T03:31:23.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.913+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3440135103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619550 msgr2=0x55b76462a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:23.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.913+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3440135103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619550 0x55b76462a6e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00fe20 tx=0x7f89ac0254c0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.913+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3440135103 shutdown_connections 2023-07-18T03:31:23.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.913+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3440135103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b764619ea0 0x55b76461e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.913+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3440135103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b76460ee20 0x55b76461dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.913+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3440135103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619550 0x55b76462a6e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.913+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3440135103 >> 172.21.15.40:0/3440135103 conn(0x55b764610680 msgr2=0x55b764621080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:23.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.913+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3440135103 shutdown_connections 2023-07-18T03:31:23.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.913+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3440135103 wait complete. 2023-07-18T03:31:23.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.913+0000 7f89c9a2fb80 1 -- start start 2023-07-18T03:31:23.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.914+0000 7f89c9a2fb80 1 Processor -- start 2023-07-18T03:31:23.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.914+0000 7f89c9a2fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b76460ee20 0x55b76462a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.914+0000 7f89c9a2fb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619550 0x55b764627050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.914+0000 7f89c9a2fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b764619ea0 0x55b764627590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.914+0000 7f89c9a2fb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b764627ca0 con 0x55b764619550 2023-07-18T03:31:23.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.914+0000 7f89c9a2fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b764682e40 con 0x55b76460ee20 2023-07-18T03:31:23.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.914+0000 7f89c9a2fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b764683000 con 0x55b764619ea0 2023-07-18T03:31:23.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.914+0000 7f89c9a2fb80 10 client.0 _ll_register_callbacks cb 0x7f89ac009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:23.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.914+0000 7f89b67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619550 0x55b764627050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:23.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.914+0000 7f89b67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619550 0x55b764627050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58338/0 (socket says 172.21.15.40:58338) 2023-07-18T03:31:23.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.914+0000 7f89b67fc700 1 -- 172.21.15.40:0/3403976925 learned_addr learned my addr 172.21.15.40:0/3403976925 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:23.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.914+0000 7f89b6ffd700 1 --2- 172.21.15.40:0/3403976925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b76460ee20 0x55b76462a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:23.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.914+0000 7f89b77fe700 1 --2- 172.21.15.40:0/3403976925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b764619ea0 0x55b764627590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:23.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.914+0000 7f89b67fc700 1 -- 172.21.15.40:0/3403976925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b764619ea0 msgr2=0x55b764627590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:23.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.914+0000 7f89b67fc700 1 --2- 172.21.15.40:0/3403976925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b764619ea0 0x55b764627590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.915+0000 7f89b67fc700 1 -- 172.21.15.40:0/3403976925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b76460ee20 msgr2=0x55b76462a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:23.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.915+0000 7f89b67fc700 1 --2- 172.21.15.40:0/3403976925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b76460ee20 0x55b76462a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.915+0000 7f89b67fc700 1 -- 172.21.15.40:0/3403976925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b764381ce0 con 0x55b764619550 2023-07-18T03:31:23.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.915+0000 7f89b6ffd700 1 --2- 172.21.15.40:0/3403976925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b76460ee20 0x55b76462a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:23.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.915+0000 7f89b67fc700 1 --2- 172.21.15.40:0/3403976925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619550 0x55b764627050 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f89ac000c00 tx=0x7f89ac010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0cb968828e4d408 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:23.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.915+0000 7f8996ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:23.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.915+0000 7f8996ffd700 1 -- 172.21.15.40:0/3403976925 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac003aa0 con 0x55b764619550 2023-07-18T03:31:23.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.915+0000 7f8996ffd700 1 -- 172.21.15.40:0/3403976925 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f89ac034020 con 0x55b764619550 2023-07-18T03:31:23.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.915+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3403976925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89b009d600 con 0x55b764619550 2023-07-18T03:31:23.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.915+0000 7f8996ffd700 1 -- 172.21.15.40:0/3403976925 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89ac003c00 con 0x55b764619550 2023-07-18T03:31:23.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.915+0000 7f89c9a2fb80 10 client.15618 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:23.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.915+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3403976925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b764397330 con 0x55b764619550 2023-07-18T03:31:23.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.915+0000 7f8996ffd700 1 -- 172.21.15.40:0/3403976925 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b009d600 con 0x55b764619550 2023-07-18T03:31:23.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.915+0000 7f8996ffd700 1 -- 172.21.15.40:0/3403976925 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f89ac02d070 con 0x55b764619550 2023-07-18T03:31:23.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.916+0000 7f89c9a2fb80 10 client.15618 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:23.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.916+0000 7f89c9a2fb80 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:23.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.916+0000 7f89c9a2fb80 20 client.15618 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:23.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.916+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3403976925 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55b76461acf0 0x55b7646865c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:23.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.916+0000 7f89c9a2fb80 4 client.15618 mds_command: new command op to 24527 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:31:23.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.916+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3403976925 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b7645260d0 con 0x55b76461acf0 2023-07-18T03:31:23.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.916+0000 7f89b6ffd700 1 --2- 172.21.15.40:0/3403976925 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55b76461acf0 0x55b7646865c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:23.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.916+0000 7f8996ffd700 10 client.15618 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:23.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.916+0000 7f89b6ffd700 1 --2- 172.21.15.40:0/3403976925 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55b76461acf0 0x55b7646865c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb9b05c48ec70c85 server_cookie=b46349657344ee10 in_seq=0 out_seq=0 2023-07-18T03:31:23.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.918+0000 7f8996ffd700 1 -- 172.21.15.40:0/3403976925 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b7645260d0 con 0x55b76461acf0 2023-07-18T03:31:23.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.918+0000 7f8996ffd700 10 client.15618 handle_command_reply: tid=0 2023-07-18T03:31:23.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.918+0000 7f89c9a2fb80 1 client.15618 shutdown 2023-07-18T03:31:23.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.918+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3403976925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619550 msgr2=0x55b764627050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:23.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.918+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3403976925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619550 0x55b764627050 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f89ac000c00 tx=0x7f89ac010f30 comp rx=0 tx=0).stop 2023-07-18T03:31:23.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.918+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3403976925 shutdown_connections 2023-07-18T03:31:23.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.918+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3403976925 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55b76461acf0 0x55b7646865c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.918+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3403976925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b764619ea0 0x55b764627590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.918+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3403976925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b76460ee20 0x55b76462a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.918+0000 7f89c9a2fb80 1 --2- 172.21.15.40:0/3403976925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b764619550 0x55b764627050 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.918+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3403976925 >> 172.21.15.40:0/3403976925 conn(0x55b764610680 msgr2=0x55b76461dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:23.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.918+0000 7f8996ffd700 0 client.15618 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:23.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.918+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3403976925 shutdown_connections 2023-07-18T03:31:23.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.918+0000 7f89c9a2fb80 1 -- 172.21.15.40:0/3403976925 wait complete. 2023-07-18T03:31:23.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.918+0000 7f89c9a2fb80 20 client.15618 trim_cache size 0 max 16384 2023-07-18T03:31:23.925 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:31:23.925 INFO:teuthology.orchestra.run.smithi040.stdout: "return_code": 0, 2023-07-18T03:31:23.925 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_tag": "e8358fb5-0e1d-4fb8-934e-d28d96d4f336", 2023-07-18T03:31:23.925 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": "asynchronous" 2023-07-18T03:31:23.926 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:31:23.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.919+0000 7f89bd8a9700 1 -- 172.21.15.40:0/852931955 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89a00a1150 msgr2=0x7f89a00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:23.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.919+0000 7f89bd8a9700 1 --2- 172.21.15.40:0/852931955 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89a00a1150 0x7f89a00a3600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f89a8005950 tx=0x7f89a800a300 comp rx=0 tx=0).stop 2023-07-18T03:31:23.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.919+0000 7f89bd8a9700 1 -- 172.21.15.40:0/852931955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b8105840 msgr2=0x7f89b819ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:23.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.919+0000 7f89bd8a9700 1 --2- 172.21.15.40:0/852931955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b8105840 0x7f89b819ce10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f89b0007ee0 tx=0x7f89b000d3b0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.919+0000 7f89bd8a9700 1 -- 172.21.15.40:0/852931955 shutdown_connections 2023-07-18T03:31:23.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.919+0000 7f89bd8a9700 1 --2- 172.21.15.40:0/852931955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89b8105840 0x7f89b819ce10 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.919+0000 7f89bd8a9700 1 --2- 172.21.15.40:0/852931955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b8100600 0x7f89b81958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.919+0000 7f89bd8a9700 1 --2- 172.21.15.40:0/852931955 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f89a00a1150 0x7f89a00a3600 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.919+0000 7f89bd8a9700 1 --2- 172.21.15.40:0/852931955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b8102f20 0x7f89b8195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:23.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.919+0000 7f89bd8a9700 1 -- 172.21.15.40:0/852931955 >> 172.21.15.40:0/852931955 conn(0x7f89b80fa020 msgr2=0x7f89b8101280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:23.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.919+0000 7f89bd8a9700 1 -- 172.21.15.40:0/852931955 shutdown_connections 2023-07-18T03:31:23.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:23.919+0000 7f89bd8a9700 1 -- 172.21.15.40:0/852931955 wait complete. 2023-07-18T03:31:23.929 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:31:24.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.061+0000 7fc0c3b26700 1 -- 172.21.15.40:0/3391685066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc0ffb50 msgr2=0x7fc0bc0fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:24.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.061+0000 7fc0c3b26700 1 --2- 172.21.15.40:0/3391685066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc0ffb50 0x7fc0bc0fff60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009990 tx=0x7fc0b8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.061+0000 7fc0c3b26700 1 -- 172.21.15.40:0/3391685066 shutdown_connections 2023-07-18T03:31:24.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.061+0000 7fc0c3b26700 1 --2- 172.21.15.40:0/3391685066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0bc1016d0 0x7fc0bc107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.061+0000 7fc0c3b26700 1 --2- 172.21.15.40:0/3391685066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc0ffb50 0x7fc0bc0fff60 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.061+0000 7fc0c3b26700 1 --2- 172.21.15.40:0/3391685066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0bc100d40 0x7fc0bc101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.061+0000 7fc0c3b26700 1 -- 172.21.15.40:0/3391685066 >> 172.21.15.40:0/3391685066 conn(0x7fc0bc0fb170 msgr2=0x7fc0bc0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:24.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.061+0000 7fc0c3b26700 1 -- 172.21.15.40:0/3391685066 shutdown_connections 2023-07-18T03:31:24.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.061+0000 7fc0c3b26700 1 -- 172.21.15.40:0/3391685066 wait complete. 2023-07-18T03:31:24.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c3b26700 1 Processor -- start 2023-07-18T03:31:24.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c3b26700 1 -- start start 2023-07-18T03:31:24.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c3b26700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc0ffb50 0x7fc0bc195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c3b26700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0bc100d40 0x7fc0bc195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c3b26700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0bc1016d0 0x7fc0bc19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c3b26700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc19d3d0 con 0x7fc0bc0ffb50 2023-07-18T03:31:24.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c3b26700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc19d510 con 0x7fc0bc100d40 2023-07-18T03:31:24.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c3b26700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc19d6a0 con 0x7fc0bc1016d0 2023-07-18T03:31:24.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c18c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc0ffb50 0x7fc0bc195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:24.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c18c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc0ffb50 0x7fc0bc195850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58348/0 (socket says 172.21.15.40:58348) 2023-07-18T03:31:24.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c18c2700 1 -- 172.21.15.40:0/3891662506 learned_addr learned my addr 172.21.15.40:0/3891662506 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:24.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c10c1700 1 --2- 172.21.15.40:0/3891662506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0bc100d40 0x7fc0bc195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:24.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c20c3700 1 --2- 172.21.15.40:0/3891662506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0bc1016d0 0x7fc0bc19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:24.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c18c2700 1 -- 172.21.15.40:0/3891662506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0bc1016d0 msgr2=0x7fc0bc19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:24.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c18c2700 1 --2- 172.21.15.40:0/3891662506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0bc1016d0 0x7fc0bc19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c18c2700 1 -- 172.21.15.40:0/3891662506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0bc100d40 msgr2=0x7fc0bc195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:24.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c18c2700 1 --2- 172.21.15.40:0/3891662506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0bc100d40 0x7fc0bc195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.062+0000 7fc0c18c2700 1 -- 172.21.15.40:0/3891662506 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b80095f0 con 0x7fc0bc0ffb50 2023-07-18T03:31:24.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.063+0000 7fc0c18c2700 1 --2- 172.21.15.40:0/3891662506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc0ffb50 0x7fc0bc195850 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80098e0 tx=0x7fc0b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:24.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.063+0000 7fc0b2ffd700 1 -- 172.21.15.40:0/3891662506 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b801d070 con 0x7fc0bc0ffb50 2023-07-18T03:31:24.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.063+0000 7fc0c3b26700 1 -- 172.21.15.40:0/3891662506 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0bc19d920 con 0x7fc0bc0ffb50 2023-07-18T03:31:24.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.063+0000 7fc0b2ffd700 1 -- 172.21.15.40:0/3891662506 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0b8005030 con 0x7fc0bc0ffb50 2023-07-18T03:31:24.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.063+0000 7fc0b2ffd700 1 -- 172.21.15.40:0/3891662506 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8004b50 con 0x7fc0bc0ffb50 2023-07-18T03:31:24.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.063+0000 7fc0c3b26700 1 -- 172.21.15.40:0/3891662506 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0bc19dce0 con 0x7fc0bc0ffb50 2023-07-18T03:31:24.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.065+0000 7fc0b2ffd700 1 -- 172.21.15.40:0/3891662506 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc0b8004d30 con 0x7fc0bc0ffb50 2023-07-18T03:31:24.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.066+0000 7fc0b2ffd700 1 --2- 172.21.15.40:0/3891662506 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc0a80a1150 0x7fc0a80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.066+0000 7fc0c10c1700 1 --2- 172.21.15.40:0/3891662506 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc0a80a1150 0x7fc0a80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:24.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.066+0000 7fc0b2ffd700 1 -- 172.21.15.40:0/3891662506 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc0b80cdb90 con 0x7fc0bc0ffb50 2023-07-18T03:31:24.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.067+0000 7fc0cfcacb80 1 Processor -- start 2023-07-18T03:31:24.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.067+0000 7fc0c10c1700 1 --2- 172.21.15.40:0/3891662506 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc0a80a1150 0x7fc0a80a3600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009c60 tx=0x7fc0ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:24.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.067+0000 7fc0cfcacb80 1 -- start start 2023-07-18T03:31:24.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.067+0000 7fc0cfcacb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9c98e0 0x5590ee9c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.067+0000 7fc0cfcacb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9ca230 0x5590ee9caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.067+0000 7fc0cfcacb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9cb080 0x5590ee9cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.067+0000 7fc0cfcacb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ee8cacb0 con 0x5590ee9c98e0 2023-07-18T03:31:24.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.067+0000 7fc0cfcacb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ee91da40 con 0x5590ee9ca230 2023-07-18T03:31:24.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.067+0000 7fc0cfcacb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ee916ed0 con 0x5590ee9cb080 2023-07-18T03:31:24.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.067+0000 7fc0c18c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9c98e0 0x5590ee9c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:24.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.067+0000 7fc0c18c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9c98e0 0x5590ee9c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58356/0 (socket says 172.21.15.40:58356) 2023-07-18T03:31:24.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.067+0000 7fc0c18c2700 1 -- 172.21.15.40:0/577320115 learned_addr learned my addr 172.21.15.40:0/577320115 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:24.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.067+0000 7fc0c10c1700 1 --2- 172.21.15.40:0/577320115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9ca230 0x5590ee9caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:24.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.068+0000 7fc0c20c3700 1 --2- 172.21.15.40:0/577320115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9cb080 0x5590ee9cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:24.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.068+0000 7fc0c18c2700 1 -- 172.21.15.40:0/577320115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9cb080 msgr2=0x5590ee9cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:24.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.068+0000 7fc0c18c2700 1 --2- 172.21.15.40:0/577320115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9cb080 0x5590ee9cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.068+0000 7fc0c18c2700 1 -- 172.21.15.40:0/577320115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9ca230 msgr2=0x5590ee9caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.068+0000 7fc0c18c2700 1 --2- 172.21.15.40:0/577320115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9ca230 0x5590ee9caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.068+0000 7fc0c18c2700 1 -- 172.21.15.40:0/577320115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590ee744de0 con 0x5590ee9c98e0 2023-07-18T03:31:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.068+0000 7fc0c18c2700 1 --2- 172.21.15.40:0/577320115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9c98e0 0x5590ee9c9cf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80a2ea0 tx=0x7fc0b809b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa41e5640fa8ece server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.068+0000 7fc0a7fff700 1 -- 172.21.15.40:0/577320115 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80a33d0 con 0x5590ee9c98e0 2023-07-18T03:31:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.068+0000 7fc0a7fff700 1 -- 172.21.15.40:0/577320115 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0b80051a0 con 0x5590ee9c98e0 2023-07-18T03:31:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.068+0000 7fc0a7fff700 1 -- 172.21.15.40:0/577320115 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80a2f40 con 0x5590ee9c98e0 2023-07-18T03:31:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.069+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/577320115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9c98e0 msgr2=0x5590ee9c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.069+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/577320115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9c98e0 0x5590ee9c9cf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80a2ea0 tx=0x7fc0b809b090 comp rx=0 tx=0).stop 2023-07-18T03:31:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.069+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/577320115 shutdown_connections 2023-07-18T03:31:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.069+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/577320115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9cb080 0x5590ee9cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.069+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/577320115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9c98e0 0x5590ee9c9cf0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.069+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/577320115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9ca230 0x5590ee9caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.069+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/577320115 >> 172.21.15.40:0/577320115 conn(0x5590ee9bada0 msgr2=0x5590ee9c01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.069+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/577320115 shutdown_connections 2023-07-18T03:31:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.069+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/577320115 wait complete. 2023-07-18T03:31:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.069+0000 7fc0cfcacb80 1 -- start start 2023-07-18T03:31:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.069+0000 7fc0cfcacb80 1 Processor -- start 2023-07-18T03:31:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0cfcacb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9c98e0 0x5590ee9d3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0cfcacb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9ca230 0x5590ee9d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0cfcacb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9cb080 0x5590ee9da780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0cfcacb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ee8b6ac0 con 0x5590ee9c98e0 2023-07-18T03:31:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0cfcacb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ee8b67c0 con 0x5590ee9cb080 2023-07-18T03:31:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0cfcacb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ee79a920 con 0x5590ee9ca230 2023-07-18T03:31:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0cfcacb80 10 client.0 _ll_register_callbacks cb 0x5590ee8b4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0c18c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9c98e0 0x5590ee9d3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0c18c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9c98e0 0x5590ee9d3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58362/0 (socket says 172.21.15.40:58362) 2023-07-18T03:31:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0c18c2700 1 -- 172.21.15.40:0/4135568455 learned_addr learned my addr 172.21.15.40:0/4135568455 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0c10c1700 1 --2- 172.21.15.40:0/4135568455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9ca230 0x5590ee9d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0c18c2700 1 -- 172.21.15.40:0/4135568455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9ca230 msgr2=0x5590ee9d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0c18c2700 1 --2- 172.21.15.40:0/4135568455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9ca230 0x5590ee9d3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0c18c2700 1 -- 172.21.15.40:0/4135568455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9cb080 msgr2=0x5590ee9da780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:24.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0c18c2700 1 --2- 172.21.15.40:0/4135568455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9cb080 0x5590ee9da780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0c18c2700 1 -- 172.21.15.40:0/4135568455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0bc19dce0 con 0x5590ee9c98e0 2023-07-18T03:31:24.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0c18c2700 1 --2- 172.21.15.40:0/4135568455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9c98e0 0x5590ee9d3200 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80974c0 tx=0x7fc0b80a34f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6af5e666c02754c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:24.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.070+0000 7fc0a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.071+0000 7fc0a57fa700 1 -- 172.21.15.40:0/4135568455 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b809b3b0 con 0x5590ee9c98e0 2023-07-18T03:31:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.071+0000 7fc0a57fa700 1 -- 172.21.15.40:0/4135568455 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc0b80a3150 con 0x5590ee9c98e0 2023-07-18T03:31:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.071+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/4135568455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590ee8704f0 con 0x5590ee9c98e0 2023-07-18T03:31:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.071+0000 7fc0a57fa700 1 -- 172.21.15.40:0/4135568455 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0b80b92c0 con 0x5590ee9c98e0 2023-07-18T03:31:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.071+0000 7fc0cfcacb80 10 client.15630 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.071+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/4135568455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5590ee72dc60 con 0x5590ee9c98e0 2023-07-18T03:31:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.071+0000 7fc0a57fa700 1 -- 172.21.15.40:0/4135568455 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5590ee8704f0 con 0x5590ee9c98e0 2023-07-18T03:31:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.071+0000 7fc0a57fa700 1 -- 172.21.15.40:0/4135568455 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc0b8099f70 con 0x5590ee9c98e0 2023-07-18T03:31:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.071+0000 7fc0cfcacb80 10 client.15630 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.071+0000 7fc0cfcacb80 10 client.15630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.071+0000 7fc0cfcacb80 20 client.15630 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.071+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/4135568455 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5590ee9de960 0x5590ee9e0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.071+0000 7fc0cfcacb80 4 client.15630 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.071+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/4135568455 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590ee90feb0 con 0x5590ee9de960 2023-07-18T03:31:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.071+0000 7fc0c10c1700 1 --2- 172.21.15.40:0/4135568455 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5590ee9de960 0x5590ee9e0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.072+0000 7fc0a57fa700 10 client.15630 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.072+0000 7fc0c10c1700 1 --2- 172.21.15.40:0/4135568455 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5590ee9de960 0x5590ee9e0d40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db8027e587796a7b server_cookie=b033ed02814d8c27 in_seq=0 out_seq=0 2023-07-18T03:31:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.073+0000 7fc0a57fa700 1 -- 172.21.15.40:0/4135568455 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5590ee90feb0 con 0x5590ee9de960 2023-07-18T03:31:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.073+0000 7fc0a57fa700 10 client.15630 handle_command_reply: tid=0 2023-07-18T03:31:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.073+0000 7fc0cfcacb80 1 client.15630 shutdown 2023-07-18T03:31:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.073+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/4135568455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9c98e0 msgr2=0x5590ee9d3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:24.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.073+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/4135568455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9c98e0 0x5590ee9d3200 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80974c0 tx=0x7fc0b80a34f0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.073+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/4135568455 shutdown_connections 2023-07-18T03:31:24.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.073+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/4135568455 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5590ee9de960 0x5590ee9e0d40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.073+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/4135568455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9cb080 0x5590ee9da780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.073+0000 7fc0a57fa700 0 client.15630 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:24.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.073+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/4135568455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9c98e0 0x5590ee9d3200 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.073+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/4135568455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9ca230 0x5590ee9d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.073+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/4135568455 >> 172.21.15.40:0/4135568455 conn(0x5590ee9bada0 msgr2=0x5590ee9bf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:24.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.073+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/4135568455 shutdown_connections 2023-07-18T03:31:24.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.073+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/4135568455 wait complete. 2023-07-18T03:31:24.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.073+0000 7fc0cfcacb80 20 client.15630 trim_cache size 0 max 16384 2023-07-18T03:31:24.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:23 smithi040 ceph-mon[96439]: pgmap v308: 97 pgs: 97 active+clean; 653 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-07-18T03:31:24.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.093+0000 7fc0cfcacb80 1 Processor -- start 2023-07-18T03:31:24.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.093+0000 7fc0cfcacb80 1 -- start start 2023-07-18T03:31:24.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.093+0000 7fc0cfcacb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9c98e0 0x5590ee9cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.093+0000 7fc0cfcacb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9ca230 0x5590ee9da740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.093+0000 7fc0cfcacb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9cb080 0x5590ee9ce480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.093+0000 7fc0cfcacb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ee9d26b0 con 0x5590ee9cb080 2023-07-18T03:31:24.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.093+0000 7fc0cfcacb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ee9ba970 con 0x5590ee9c98e0 2023-07-18T03:31:24.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.093+0000 7fc0cfcacb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ee9c9f00 con 0x5590ee9ca230 2023-07-18T03:31:24.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.094+0000 7fc0c18c2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9c98e0 0x5590ee9cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:24.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.094+0000 7fc0c18c2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9c98e0 0x5590ee9cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:41696/0 (socket says 172.21.15.40:41696) 2023-07-18T03:31:24.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.094+0000 7fc0c18c2700 1 -- 172.21.15.40:0/3362553389 learned_addr learned my addr 172.21.15.40:0/3362553389 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:24.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.094+0000 7fc0c20c3700 1 --2- 172.21.15.40:0/3362553389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9cb080 0x5590ee9ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:24.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.094+0000 7fc0c10c1700 1 --2- 172.21.15.40:0/3362553389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9ca230 0x5590ee9da740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:24.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.094+0000 7fc0c18c2700 1 -- 172.21.15.40:0/3362553389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9ca230 msgr2=0x5590ee9da740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:24.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.094+0000 7fc0c18c2700 1 --2- 172.21.15.40:0/3362553389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9ca230 0x5590ee9da740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.094+0000 7fc0c18c2700 1 -- 172.21.15.40:0/3362553389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9cb080 msgr2=0x5590ee9ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:24.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.094+0000 7fc0c18c2700 1 --2- 172.21.15.40:0/3362553389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9cb080 0x5590ee9ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.094+0000 7fc0c18c2700 1 -- 172.21.15.40:0/3362553389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590ee76b250 con 0x5590ee9c98e0 2023-07-18T03:31:24.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.094+0000 7fc0c20c3700 1 --2- 172.21.15.40:0/3362553389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9cb080 0x5590ee9ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:24.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.094+0000 7fc0c18c2700 1 --2- 172.21.15.40:0/3362553389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9c98e0 0x5590ee9cdf40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8099360 tx=0x7fc0b809a500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39be025bec22811f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:24.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.094+0000 7fc0b0dac700 1 -- 172.21.15.40:0/3362553389 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8099460 con 0x5590ee9c98e0 2023-07-18T03:31:24.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.094+0000 7fc0b0dac700 1 -- 172.21.15.40:0/3362553389 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0b80a3150 con 0x5590ee9c98e0 2023-07-18T03:31:24.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.095+0000 7fc0b0dac700 1 -- 172.21.15.40:0/3362553389 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b809a1a0 con 0x5590ee9c98e0 2023-07-18T03:31:24.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.095+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/3362553389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9c98e0 msgr2=0x5590ee9cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:24.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.095+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/3362553389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9c98e0 0x5590ee9cdf40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8099360 tx=0x7fc0b809a500 comp rx=0 tx=0).stop 2023-07-18T03:31:24.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.095+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/3362553389 shutdown_connections 2023-07-18T03:31:24.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.095+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/3362553389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9cb080 0x5590ee9ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.095+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/3362553389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9c98e0 0x5590ee9cdf40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.095+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/3362553389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9ca230 0x5590ee9da740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.095+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/3362553389 >> 172.21.15.40:0/3362553389 conn(0x5590ee9bada0 msgr2=0x5590ee9e5660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:24.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.095+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/3362553389 shutdown_connections 2023-07-18T03:31:24.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.095+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/3362553389 wait complete. 2023-07-18T03:31:24.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.096+0000 7fc0cfcacb80 1 -- start start 2023-07-18T03:31:24.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.096+0000 7fc0cfcacb80 1 Processor -- start 2023-07-18T03:31:24.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.096+0000 7fc0cfcacb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9c98e0 0x5590ee9da120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.096+0000 7fc0cfcacb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9ca230 0x5590ee9da660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.096+0000 7fc0cfcacb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9cb080 0x5590ee9d6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.096+0000 7fc0cfcacb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ee9d7490 con 0x5590ee9ca230 2023-07-18T03:31:24.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.096+0000 7fc0cfcacb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ee9d7600 con 0x5590ee9c98e0 2023-07-18T03:31:24.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.096+0000 7fc0cfcacb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ee9d77c0 con 0x5590ee9cb080 2023-07-18T03:31:24.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.096+0000 7fc0cfcacb80 10 client.0 _ll_register_callbacks cb 0x7fc0ac0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:24.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.096+0000 7fc0c10c1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9ca230 0x5590ee9da660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:24.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.096+0000 7fc0c10c1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9ca230 0x5590ee9da660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58378/0 (socket says 172.21.15.40:58378) 2023-07-18T03:31:24.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.096+0000 7fc0c10c1700 1 -- 172.21.15.40:0/3492946434 learned_addr learned my addr 172.21.15.40:0/3492946434 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:24.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.096+0000 7fc0c18c2700 1 --2- 172.21.15.40:0/3492946434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9c98e0 0x5590ee9da120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:24.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.097+0000 7fc0c20c3700 1 --2- 172.21.15.40:0/3492946434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9cb080 0x5590ee9d6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:24.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.097+0000 7fc0c10c1700 1 -- 172.21.15.40:0/3492946434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9cb080 msgr2=0x5590ee9d6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:24.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.097+0000 7fc0c10c1700 1 --2- 172.21.15.40:0/3492946434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9cb080 0x5590ee9d6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.097+0000 7fc0c10c1700 1 -- 172.21.15.40:0/3492946434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9c98e0 msgr2=0x5590ee9da120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:24.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.097+0000 7fc0c10c1700 1 --2- 172.21.15.40:0/3492946434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9c98e0 0x5590ee9da120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.097+0000 7fc0c10c1700 1 -- 172.21.15.40:0/3492946434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b4003d00 con 0x5590ee9ca230 2023-07-18T03:31:24.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.097+0000 7fc0c20c3700 1 --2- 172.21.15.40:0/3492946434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9cb080 0x5590ee9d6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:31:24.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.097+0000 7fc0c10c1700 1 --2- 172.21.15.40:0/3492946434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9ca230 0x5590ee9da660 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x5590ee9e1610 tx=0x7fc0ac0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b56386305c9607f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:24.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.097+0000 7fc0a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:24.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.097+0000 7fc0a57fa700 1 -- 172.21.15.40:0/3492946434 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac02a240 con 0x5590ee9ca230 2023-07-18T03:31:24.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.097+0000 7fc0a57fa700 1 -- 172.21.15.40:0/3492946434 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc0ac02a3a0 con 0x5590ee9ca230 2023-07-18T03:31:24.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.097+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/3492946434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0b80051a0 con 0x5590ee9ca230 2023-07-18T03:31:24.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.097+0000 7fc0a57fa700 1 -- 172.21.15.40:0/3492946434 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0ac02ae50 con 0x5590ee9ca230 2023-07-18T03:31:24.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.097+0000 7fc0cfcacb80 10 client.15636 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:24.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.097+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/3492946434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fc0b80ca070 con 0x5590ee9ca230 2023-07-18T03:31:24.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.097+0000 7fc0a57fa700 1 -- 172.21.15.40:0/3492946434 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0b80051a0 con 0x5590ee9ca230 2023-07-18T03:31:24.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.098+0000 7fc0a57fa700 1 -- 172.21.15.40:0/3492946434 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc0ac03b020 con 0x5590ee9ca230 2023-07-18T03:31:24.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.098+0000 7fc0cfcacb80 10 client.15636 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:24.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.098+0000 7fc0cfcacb80 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:24.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.098+0000 7fc0cfcacb80 20 client.15636 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:24.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.098+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/3492946434 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5590ee9de960 0x5590eea35f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:24.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.098+0000 7fc0cfcacb80 4 client.15636 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:31:24.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.098+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/3492946434 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5590ee8d2050 con 0x5590ee9de960 2023-07-18T03:31:24.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.098+0000 7fc0c18c2700 1 --2- 172.21.15.40:0/3492946434 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5590ee9de960 0x5590eea35f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:24.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.098+0000 7fc0a57fa700 10 client.15636 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:24.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.098+0000 7fc0c18c2700 1 --2- 172.21.15.40:0/3492946434 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5590ee9de960 0x5590eea35f40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8abccd8b37c3e508 server_cookie=240d9274ddde9b74 in_seq=0 out_seq=0 2023-07-18T03:31:24.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.098+0000 7fc0a57fa700 1 -- 172.21.15.40:0/3492946434 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5590ee8d2050 con 0x5590ee9de960 2023-07-18T03:31:24.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.099+0000 7fc0a57fa700 10 client.15636 handle_command_reply: tid=0 2023-07-18T03:31:24.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.099+0000 7fc0cfcacb80 1 client.15636 shutdown 2023-07-18T03:31:24.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.099+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/3492946434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9ca230 msgr2=0x5590ee9da660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:24.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.099+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/3492946434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9ca230 0x5590ee9da660 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x5590ee9e1610 tx=0x7fc0ac0047c0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.099+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/3492946434 shutdown_connections 2023-07-18T03:31:24.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.099+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/3492946434 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5590ee9de960 0x5590eea35f40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.099+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/3492946434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5590ee9cb080 0x5590ee9d6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.099+0000 7fc0a57fa700 0 client.15636 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.099+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/3492946434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590ee9c98e0 0x5590ee9da120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.099+0000 7fc0cfcacb80 1 --2- 172.21.15.40:0/3492946434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5590ee9ca230 0x5590ee9da660 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.099+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/3492946434 >> 172.21.15.40:0/3492946434 conn(0x5590ee9bada0 msgr2=0x5590ee9cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.099+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/3492946434 shutdown_connections 2023-07-18T03:31:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.099+0000 7fc0cfcacb80 1 -- 172.21.15.40:0/3492946434 wait complete. 2023-07-18T03:31:24.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.099+0000 7fc0cfcacb80 20 client.15636 trim_cache size 0 max 16384 2023-07-18T03:31:24.107 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:31:24.107 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:31:24.107 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:31:24.107 INFO:teuthology.orchestra.run.smithi040.stdout: "e8358fb5-0e1d-4fb8-934e-d28d96d4f336": { 2023-07-18T03:31:24.107 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:31:24.107 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "e8358fb5-0e1d-4fb8-934e-d28d96d4f336", 2023-07-18T03:31:24.107 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:31:24.108 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:31:24.108 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:31:24.108 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:31:24.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.100+0000 7fc0c3b26700 1 -- 172.21.15.40:0/3891662506 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc0a80a1150 msgr2=0x7fc0a80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:24.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.100+0000 7fc0c3b26700 1 --2- 172.21.15.40:0/3891662506 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc0a80a1150 0x7fc0a80a3600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009c60 tx=0x7fc0ac009260 comp rx=0 tx=0).stop 2023-07-18T03:31:24.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.100+0000 7fc0c3b26700 1 -- 172.21.15.40:0/3891662506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc0ffb50 msgr2=0x7fc0bc195850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:24.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.100+0000 7fc0c3b26700 1 --2- 172.21.15.40:0/3891662506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc0ffb50 0x7fc0bc195850 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80098e0 tx=0x7fc0b8004730 comp rx=0 tx=0).stop 2023-07-18T03:31:24.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.100+0000 7fc0c3b26700 1 -- 172.21.15.40:0/3891662506 shutdown_connections 2023-07-18T03:31:24.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.100+0000 7fc0c3b26700 1 --2- 172.21.15.40:0/3891662506 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc0a80a1150 0x7fc0a80a3600 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.100+0000 7fc0c3b26700 1 --2- 172.21.15.40:0/3891662506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0bc1016d0 0x7fc0bc19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.100+0000 7fc0c3b26700 1 --2- 172.21.15.40:0/3891662506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc0ffb50 0x7fc0bc195850 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.100+0000 7fc0c3b26700 1 --2- 172.21.15.40:0/3891662506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0bc100d40 0x7fc0bc195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:24.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.100+0000 7fc0c3b26700 1 -- 172.21.15.40:0/3891662506 >> 172.21.15.40:0/3891662506 conn(0x7fc0bc0fb170 msgr2=0x7fc0bc104250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:24.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.100+0000 7fc0c3b26700 1 -- 172.21.15.40:0/3891662506 shutdown_connections 2023-07-18T03:31:24.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:24.100+0000 7fc0c3b26700 1 -- 172.21.15.40:0/3891662506 wait complete. 2023-07-18T03:31:24.110 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:23 smithi064 ceph-mon[98059]: pgmap v308: 97 pgs: 97 active+clean; 653 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-07-18T03:31:24.113 INFO:tasks.cephfs.filesystem:scrub status for tag:e8358fb5-0e1d-4fb8-934e-d28d96d4f336 - {'path': '/', 'tag': 'e8358fb5-0e1d-4fb8-934e-d28d96d4f336', 'options': 'recursive,force'} 2023-07-18T03:31:24.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:23 smithi134 ceph-mon[98216]: pgmap v308: 97 pgs: 97 active+clean; 653 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-07-18T03:31:25.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:24 smithi040 ceph-mon[96439]: scrub queued for path: / 2023-07-18T03:31:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:24 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:31:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:24 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:31:25.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:24 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:31:25.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:24 smithi064 ceph-mon[98059]: scrub queued for path: / 2023-07-18T03:31:25.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:24 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:31:25.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:24 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:31:25.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:24 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:31:25.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:24 smithi134 ceph-mon[98216]: scrub queued for path: / 2023-07-18T03:31:25.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:24 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:31:25.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:24 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:31:25.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:24 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:31:26.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:25 smithi040 ceph-mon[96439]: pgmap v309: 97 pgs: 97 active+clean; 653 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-18T03:31:26.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:25 smithi040 ceph-mon[96439]: scrub summary: idle 2023-07-18T03:31:26.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:25 smithi064 ceph-mon[98059]: pgmap v309: 97 pgs: 97 active+clean; 653 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-18T03:31:26.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:25 smithi064 ceph-mon[98059]: scrub summary: idle 2023-07-18T03:31:26.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:25 smithi134 ceph-mon[98216]: pgmap v309: 97 pgs: 97 active+clean; 653 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-18T03:31:26.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:25 smithi134 ceph-mon[98216]: scrub summary: idle 2023-07-18T03:31:28.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:27 smithi040 ceph-mon[96439]: pgmap v310: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-07-18T03:31:28.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:27 smithi064 ceph-mon[98059]: pgmap v310: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-07-18T03:31:28.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:27 smithi134 ceph-mon[98216]: pgmap v310: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-07-18T03:31:30.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:29 smithi064 ceph-mon[98059]: pgmap v311: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-07-18T03:31:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.305+0000 7f4f163ad700 1 -- 172.21.15.40:0/4230459428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f100fea00 msgr2=0x7f4f100fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.305+0000 7f4f163ad700 1 --2- 172.21.15.40:0/4230459428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f100fea00 0x7f4f100fee10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009990 tx=0x7f4f00009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.306+0000 7f4f163ad700 1 -- 172.21.15.40:0/4230459428 shutdown_connections 2023-07-18T03:31:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.306+0000 7f4f163ad700 1 --2- 172.21.15.40:0/4230459428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f10100640 0x7f4f10106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.306+0000 7f4f163ad700 1 --2- 172.21.15.40:0/4230459428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f100fea00 0x7f4f100fee10 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.306+0000 7f4f163ad700 1 --2- 172.21.15.40:0/4230459428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f100ffc90 0x7f4f10100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.306+0000 7f4f163ad700 1 -- 172.21.15.40:0/4230459428 >> 172.21.15.40:0/4230459428 conn(0x7f4f100fa000 msgr2=0x7f4f100fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.306+0000 7f4f163ad700 1 -- 172.21.15.40:0/4230459428 shutdown_connections 2023-07-18T03:31:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.306+0000 7f4f163ad700 1 -- 172.21.15.40:0/4230459428 wait complete. 2023-07-18T03:31:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.306+0000 7f4f163ad700 1 Processor -- start 2023-07-18T03:31:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.306+0000 7f4f163ad700 1 -- start start 2023-07-18T03:31:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.307+0000 7f4f163ad700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f100fea00 0x7f4f10194740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.307+0000 7f4f163ad700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f100ffc90 0x7f4f10194c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.307+0000 7f4f163ad700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f10100640 0x7f4f10195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.307+0000 7f4f163ad700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f10196160 con 0x7f4f10100640 2023-07-18T03:31:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.307+0000 7f4f163ad700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f101962d0 con 0x7f4f100ffc90 2023-07-18T03:31:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.307+0000 7f4f163ad700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f10196490 con 0x7f4f100fea00 2023-07-18T03:31:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.307+0000 7f4f0f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f100ffc90 0x7f4f10194c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.307+0000 7f4f1494a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f10100640 0x7f4f10195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.307+0000 7f4f0f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f100ffc90 0x7f4f10194c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:41718/0 (socket says 172.21.15.40:41718) 2023-07-18T03:31:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.307+0000 7f4f1494a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f10100640 0x7f4f10195b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58394/0 (socket says 172.21.15.40:58394) 2023-07-18T03:31:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.307+0000 7f4f0f7fe700 1 -- 172.21.15.40:0/1702688550 learned_addr learned my addr 172.21.15.40:0/1702688550 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.307+0000 7f4f0ffff700 1 --2- 172.21.15.40:0/1702688550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f100fea00 0x7f4f10194740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.308+0000 7f4f0f7fe700 1 -- 172.21.15.40:0/1702688550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f100fea00 msgr2=0x7f4f10194740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.308+0000 7f4f0f7fe700 1 --2- 172.21.15.40:0/1702688550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f100fea00 0x7f4f10194740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.308+0000 7f4f0f7fe700 1 -- 172.21.15.40:0/1702688550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f10100640 msgr2=0x7f4f10195b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.308+0000 7f4f0f7fe700 1 --2- 172.21.15.40:0/1702688550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f10100640 0x7f4f10195b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.308+0000 7f4f0f7fe700 1 -- 172.21.15.40:0/1702688550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f000095f0 con 0x7f4f100ffc90 2023-07-18T03:31:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.308+0000 7f4f0f7fe700 1 --2- 172.21.15.40:0/1702688550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f100ffc90 0x7f4f10194c80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400c9b0 tx=0x7f4f0400ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.308+0000 7f4f0d7fa700 1 -- 172.21.15.40:0/1702688550 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f04007b00 con 0x7f4f100ffc90 2023-07-18T03:31:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.308+0000 7f4f163ad700 1 -- 172.21.15.40:0/1702688550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f10196770 con 0x7f4f100ffc90 2023-07-18T03:31:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.308+0000 7f4f0d7fa700 1 -- 172.21.15.40:0/1702688550 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f0400ce50 con 0x7f4f100ffc90 2023-07-18T03:31:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.308+0000 7f4f0d7fa700 1 -- 172.21.15.40:0/1702688550 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f04018660 con 0x7f4f100ffc90 2023-07-18T03:31:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.308+0000 7f4f163ad700 1 -- 172.21.15.40:0/1702688550 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1019cf30 con 0x7f4f100ffc90 2023-07-18T03:31:30.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:29 smithi134 ceph-mon[98216]: pgmap v311: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-07-18T03:31:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.311+0000 7f4f0d7fa700 1 -- 172.21.15.40:0/1702688550 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4f0400f450 con 0x7f4f100ffc90 2023-07-18T03:31:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.311+0000 7f4f0d7fa700 1 --2- 172.21.15.40:0/1702688550 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4ef80a1150 0x7f4ef80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.312+0000 7f4f0ffff700 1 --2- 172.21.15.40:0/1702688550 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4ef80a1150 0x7f4ef80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.312+0000 7f4f0d7fa700 1 -- 172.21.15.40:0/1702688550 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4f040cc870 con 0x7f4f100ffc90 2023-07-18T03:31:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.312+0000 7f4f22533b80 1 Processor -- start 2023-07-18T03:31:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.312+0000 7f4f22533b80 1 -- start start 2023-07-18T03:31:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.312+0000 7f4f0ffff700 1 --2- 172.21.15.40:0/1702688550 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4ef80a1150 0x7f4ef80a3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000b5c0 tx=0x7f4f00005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.312+0000 7f4f22533b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 0x559422d77490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.312+0000 7f4f22533b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 0x559422d76040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.312+0000 7f4f22533b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422d77e40 0x559422d765e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.312+0000 7f4f22533b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x559422c7bc50 con 0x559422bd26b0 2023-07-18T03:31:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.312+0000 7f4f22533b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559422cce9e0 con 0x559422d779d0 2023-07-18T03:31:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.312+0000 7f4f22533b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559422cc7e70 con 0x559422d77e40 2023-07-18T03:31:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.313+0000 7f4f0ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 0x559422d77490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.313+0000 7f4f0ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 0x559422d77490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58398/0 (socket says 172.21.15.40:58398) 2023-07-18T03:31:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.313+0000 7f4f0ffff700 1 -- 172.21.15.40:0/3083659984 learned_addr learned my addr 172.21.15.40:0/3083659984 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.313+0000 7f4f0f7fe700 1 --2- 172.21.15.40:0/3083659984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 0x559422d76040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.313+0000 7f4f1494a700 1 --2- 172.21.15.40:0/3083659984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422d77e40 0x559422d765e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.313+0000 7f4f0ffff700 1 -- 172.21.15.40:0/3083659984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422d77e40 msgr2=0x559422d765e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.313+0000 7f4f0ffff700 1 --2- 172.21.15.40:0/3083659984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422d77e40 0x559422d765e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.313+0000 7f4f0ffff700 1 -- 172.21.15.40:0/3083659984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 msgr2=0x559422d76040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.313+0000 7f4f0ffff700 1 --2- 172.21.15.40:0/3083659984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 0x559422d76040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.313+0000 7f4f0ffff700 1 -- 172.21.15.40:0/3083659984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559422af5d80 con 0x559422bd26b0 2023-07-18T03:31:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.313+0000 7f4f0ffff700 1 --2- 172.21.15.40:0/3083659984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 0x559422d77490 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009990 tx=0x7f4f00020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ded0cc4540f94d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:30.323 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.313+0000 7f4ef67fc700 1 -- 172.21.15.40:0/3083659984 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0002a070 con 0x559422bd26b0 2023-07-18T03:31:30.323 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.313+0000 7f4ef67fc700 1 -- 172.21.15.40:0/3083659984 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f0001e040 con 0x559422bd26b0 2023-07-18T03:31:30.323 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.313+0000 7f4ef67fc700 1 -- 172.21.15.40:0/3083659984 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f00023aa0 con 0x559422bd26b0 2023-07-18T03:31:30.323 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.314+0000 7f4f22533b80 1 -- 172.21.15.40:0/3083659984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 msgr2=0x559422d77490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:30.323 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.314+0000 7f4f22533b80 1 --2- 172.21.15.40:0/3083659984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 0x559422d77490 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009990 tx=0x7f4f00020000 comp rx=0 tx=0).stop 2023-07-18T03:31:30.323 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.314+0000 7f4f22533b80 1 -- 172.21.15.40:0/3083659984 shutdown_connections 2023-07-18T03:31:30.323 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.314+0000 7f4f22533b80 1 --2- 172.21.15.40:0/3083659984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422d77e40 0x559422d765e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.323 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.314+0000 7f4f22533b80 1 --2- 172.21.15.40:0/3083659984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 0x559422d77490 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.314+0000 7f4f22533b80 1 --2- 172.21.15.40:0/3083659984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 0x559422d76040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.314+0000 7f4f22533b80 1 -- 172.21.15.40:0/3083659984 >> 172.21.15.40:0/3083659984 conn(0x559422d6bbe0 msgr2=0x559422d70cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.314+0000 7f4f22533b80 1 -- 172.21.15.40:0/3083659984 shutdown_connections 2023-07-18T03:31:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.314+0000 7f4f22533b80 1 -- 172.21.15.40:0/3083659984 wait complete. 2023-07-18T03:31:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.315+0000 7f4f22533b80 1 -- start start 2023-07-18T03:31:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.315+0000 7f4f22533b80 1 Processor -- start 2023-07-18T03:31:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.315+0000 7f4f22533b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 0x559422d83bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.315+0000 7f4f22533b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 0x559422d84110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.315+0000 7f4f22533b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422d77e40 0x559422d8b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.315+0000 7f4f22533b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x559422c67a60 con 0x559422bd26b0 2023-07-18T03:31:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.315+0000 7f4f22533b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559422c67760 con 0x559422d779d0 2023-07-18T03:31:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.315+0000 7f4f22533b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559422b4b8c0 con 0x559422d77e40 2023-07-18T03:31:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.315+0000 7f4f22533b80 10 client.0 _ll_register_callbacks cb 0x559422c65700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.315+0000 7f4f1494a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422d77e40 0x559422d8b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.315+0000 7f4f0f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 0x559422d84110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.315+0000 7f4f1494a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422d77e40 0x559422d8b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:47434/0 (socket says 172.21.15.40:47434) 2023-07-18T03:31:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.315+0000 7f4f0f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 0x559422d84110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:41734/0 (socket says 172.21.15.40:41734) 2023-07-18T03:31:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.315+0000 7f4f1494a700 1 -- 172.21.15.40:0/990019501 learned_addr learned my addr 172.21.15.40:0/990019501 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.316+0000 7f4f0f7fe700 1 -- 172.21.15.40:0/990019501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422d77e40 msgr2=0x559422d8b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.316+0000 7f4f0f7fe700 1 --2- 172.21.15.40:0/990019501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422d77e40 0x559422d8b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.316+0000 7f4f0f7fe700 1 -- 172.21.15.40:0/990019501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 msgr2=0x559422d83bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.316+0000 7f4f0f7fe700 1 --2- 172.21.15.40:0/990019501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 0x559422d83bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.316+0000 7f4f0f7fe700 1 -- 172.21.15.40:0/990019501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f10107750 con 0x559422d779d0 2023-07-18T03:31:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.316+0000 7f4f0f7fe700 1 --2- 172.21.15.40:0/990019501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 0x559422d84110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4f04099960 tx=0x7f4f04007e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e680f0c29c7cde4c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.316+0000 7f4eef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:31:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.316+0000 7f4eef7fe700 1 -- 172.21.15.40:0/990019501 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0409d040 con 0x559422d779d0 2023-07-18T03:31:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.316+0000 7f4f22533b80 1 -- 172.21.15.40:0/990019501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559422c21490 con 0x559422d779d0 2023-07-18T03:31:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.316+0000 7f4eef7fe700 1 -- 172.21.15.40:0/990019501 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4f040b1070 con 0x559422d779d0 2023-07-18T03:31:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.317+0000 7f4eef7fe700 1 -- 172.21.15.40:0/990019501 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f0409dc90 con 0x559422d779d0 2023-07-18T03:31:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.317+0000 7f4f22533b80 10 client.25337 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.317+0000 7f4f22533b80 1 -- 172.21.15.40:0/990019501 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559422adec00 con 0x559422d779d0 2023-07-18T03:31:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.317+0000 7f4eef7fe700 1 -- 172.21.15.40:0/990019501 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559422c21490 con 0x559422d779d0 2023-07-18T03:31:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.317+0000 7f4eef7fe700 1 -- 172.21.15.40:0/990019501 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4f04099150 con 0x559422d779d0 2023-07-18T03:31:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.317+0000 7f4f22533b80 10 client.25337 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.317+0000 7f4f22533b80 10 client.25337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.317+0000 7f4f22533b80 20 client.25337 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.317+0000 7f4f22533b80 1 --2- 172.21.15.40:0/990019501 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559422d8f920 0x559422d91d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.317+0000 7f4f22533b80 4 client.25337 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.317+0000 7f4f22533b80 1 -- 172.21.15.40:0/990019501 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559422cc0e50 con 0x559422d8f920 2023-07-18T03:31:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.317+0000 7f4f0ffff700 1 --2- 172.21.15.40:0/990019501 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559422d8f920 0x559422d91d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.318+0000 7f4eef7fe700 10 client.25337 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.318+0000 7f4f0ffff700 1 --2- 172.21.15.40:0/990019501 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559422d8f920 0x559422d91d30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df4c4df397f5da12 server_cookie=4b248185e6d65669 in_seq=0 out_seq=0 2023-07-18T03:31:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.318+0000 7f4eef7fe700 1 -- 172.21.15.40:0/990019501 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559422cc0e50 con 0x559422d8f920 2023-07-18T03:31:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.319+0000 7f4eef7fe700 10 client.25337 handle_command_reply: tid=0 2023-07-18T03:31:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.319+0000 7f4f22533b80 1 client.25337 shutdown 2023-07-18T03:31:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.319+0000 7f4f22533b80 1 -- 172.21.15.40:0/990019501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 msgr2=0x559422d84110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.319+0000 7f4f22533b80 1 --2- 172.21.15.40:0/990019501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 0x559422d84110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4f04099960 tx=0x7f4f04007e60 comp rx=0 tx=0).stop 2023-07-18T03:31:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.319+0000 7f4f22533b80 1 -- 172.21.15.40:0/990019501 shutdown_connections 2023-07-18T03:31:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.319+0000 7f4f22533b80 1 --2- 172.21.15.40:0/990019501 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559422d8f920 0x559422d91d30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.319+0000 7f4f22533b80 1 --2- 172.21.15.40:0/990019501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422d77e40 0x559422d8b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.319+0000 7f4eef7fe700 0 client.25337 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.319+0000 7f4f22533b80 1 --2- 172.21.15.40:0/990019501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 0x559422d83bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.319+0000 7f4f22533b80 1 --2- 172.21.15.40:0/990019501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 0x559422d84110 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.319+0000 7f4f22533b80 1 -- 172.21.15.40:0/990019501 >> 172.21.15.40:0/990019501 conn(0x559422d6bbe0 msgr2=0x559422d704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.319+0000 7f4f22533b80 1 -- 172.21.15.40:0/990019501 shutdown_connections 2023-07-18T03:31:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.319+0000 7f4f22533b80 1 -- 172.21.15.40:0/990019501 wait complete. 2023-07-18T03:31:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.319+0000 7f4f22533b80 20 client.25337 trim_cache size 0 max 16384 2023-07-18T03:31:30.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:29 smithi040 ceph-mon[96439]: pgmap v311: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-07-18T03:31:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.346+0000 7f4f22533b80 1 Processor -- start 2023-07-18T03:31:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.346+0000 7f4f22533b80 1 -- start start 2023-07-18T03:31:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.346+0000 7f4f22533b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422bd26b0 0x559422d966b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.346+0000 7f4f22533b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422d779d0 0x559422d76250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.346+0000 7f4f22533b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d77e40 0x559422d76790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.346+0000 7f4f22533b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x559422d837a0 con 0x559422d779d0 2023-07-18T03:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.346+0000 7f4f22533b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559422d71b80 con 0x559422d77e40 2023-07-18T03:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.346+0000 7f4f22533b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559422d8f4c0 con 0x559422bd26b0 2023-07-18T03:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.347+0000 7f4f0f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422d779d0 0x559422d76250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.347+0000 7f4f0f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422d779d0 0x559422d76250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58414/0 (socket says 172.21.15.40:58414) 2023-07-18T03:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.347+0000 7f4f0f7fe700 1 -- 172.21.15.40:0/1023099442 learned_addr learned my addr 172.21.15.40:0/1023099442 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.347+0000 7f4f1494a700 1 --2- 172.21.15.40:0/1023099442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d77e40 0x559422d76790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.347+0000 7f4f0ffff700 1 --2- 172.21.15.40:0/1023099442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422bd26b0 0x559422d966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.347+0000 7f4f1494a700 1 -- 172.21.15.40:0/1023099442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422bd26b0 msgr2=0x559422d966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.347+0000 7f4f1494a700 1 --2- 172.21.15.40:0/1023099442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422bd26b0 0x559422d966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.347+0000 7f4f1494a700 1 -- 172.21.15.40:0/1023099442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422d779d0 msgr2=0x559422d76250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.347+0000 7f4f1494a700 1 --2- 172.21.15.40:0/1023099442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422d779d0 0x559422d76250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.347+0000 7f4f1494a700 1 -- 172.21.15.40:0/1023099442 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559422b1c1f0 con 0x559422d77e40 2023-07-18T03:31:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.347+0000 7f4f0f7fe700 1 --2- 172.21.15.40:0/1023099442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422d779d0 0x559422d76250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.347+0000 7f4f1494a700 1 --2- 172.21.15.40:0/1023099442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d77e40 0x559422d76790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009f20 tx=0x7f4f08009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=738ac166e7155b89 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.347+0000 7f4ef6ffd700 1 -- 172.21.15.40:0/1023099442 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f08007da0 con 0x559422d77e40 2023-07-18T03:31:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.347+0000 7f4ef6ffd700 1 -- 172.21.15.40:0/1023099442 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f0800f040 con 0x559422d77e40 2023-07-18T03:31:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.347+0000 7f4ef6ffd700 1 -- 172.21.15.40:0/1023099442 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0801f780 con 0x559422d77e40 2023-07-18T03:31:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.348+0000 7f4f22533b80 1 -- 172.21.15.40:0/1023099442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d77e40 msgr2=0x559422d76790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.348+0000 7f4f22533b80 1 --2- 172.21.15.40:0/1023099442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d77e40 0x559422d76790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009f20 tx=0x7f4f08009f50 comp rx=0 tx=0).stop 2023-07-18T03:31:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.348+0000 7f4f22533b80 1 -- 172.21.15.40:0/1023099442 shutdown_connections 2023-07-18T03:31:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.348+0000 7f4f22533b80 1 --2- 172.21.15.40:0/1023099442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d77e40 0x559422d76790 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.348+0000 7f4f22533b80 1 --2- 172.21.15.40:0/1023099442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422bd26b0 0x559422d966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.348+0000 7f4f22533b80 1 --2- 172.21.15.40:0/1023099442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422d779d0 0x559422d76250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.348+0000 7f4f22533b80 1 -- 172.21.15.40:0/1023099442 >> 172.21.15.40:0/1023099442 conn(0x559422d6bbe0 msgr2=0x559422d70c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.348+0000 7f4f22533b80 1 -- 172.21.15.40:0/1023099442 shutdown_connections 2023-07-18T03:31:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.348+0000 7f4f22533b80 1 -- 172.21.15.40:0/1023099442 wait complete. 2023-07-18T03:31:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.349+0000 7f4f22533b80 1 -- start start 2023-07-18T03:31:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.349+0000 7f4f22533b80 1 Processor -- start 2023-07-18T03:31:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.349+0000 7f4f22533b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 0x559422d7b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.349+0000 7f4f22533b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 0x559422d7be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.349+0000 7f4f22533b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422d77e40 0x559422d8b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.349+0000 7f4f22533b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x559422d7a2b0 con 0x559422bd26b0 2023-07-18T03:31:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.349+0000 7f4f22533b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559422d7a420 con 0x559422d779d0 2023-07-18T03:31:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.349+0000 7f4f22533b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559422d7a5e0 con 0x559422d77e40 2023-07-18T03:31:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.349+0000 7f4f22533b80 10 client.0 _ll_register_callbacks cb 0x7f4f040997e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.349+0000 7f4f0ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 0x559422d7b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.350+0000 7f4f0ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 0x559422d7b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58418/0 (socket says 172.21.15.40:58418) 2023-07-18T03:31:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.350+0000 7f4f0ffff700 1 -- 172.21.15.40:0/425867839 learned_addr learned my addr 172.21.15.40:0/425867839 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.350+0000 7f4f0f7fe700 1 --2- 172.21.15.40:0/425867839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 0x559422d7be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.350+0000 7f4f1494a700 1 --2- 172.21.15.40:0/425867839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422d77e40 0x559422d8b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.350+0000 7f4f0ffff700 1 -- 172.21.15.40:0/425867839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422d77e40 msgr2=0x559422d8b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.350+0000 7f4f0ffff700 1 --2- 172.21.15.40:0/425867839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422d77e40 0x559422d8b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.350+0000 7f4f0ffff700 1 -- 172.21.15.40:0/425867839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 msgr2=0x559422d7be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.350+0000 7f4f0ffff700 1 --2- 172.21.15.40:0/425867839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 0x559422d7be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.350+0000 7f4f0ffff700 1 -- 172.21.15.40:0/425867839 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559422adec00 con 0x559422bd26b0 2023-07-18T03:31:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.350+0000 7f4f0f7fe700 1 --2- 172.21.15.40:0/425867839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 0x559422d7be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.350+0000 7f4f0ffff700 1 --2- 172.21.15.40:0/425867839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 0x559422d7b890 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4f00030040 tx=0x7f4f0000b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a2c4fe8bdd9219 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.350+0000 7f4eef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.350+0000 7f4eef7fe700 1 -- 172.21.15.40:0/425867839 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0002a040 con 0x559422bd26b0 2023-07-18T03:31:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.350+0000 7f4f22533b80 1 -- 172.21.15.40:0/425867839 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f0001e040 con 0x559422bd26b0 2023-07-18T03:31:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.350+0000 7f4eef7fe700 1 -- 172.21.15.40:0/425867839 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4f00037b70 con 0x559422bd26b0 2023-07-18T03:31:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.351+0000 7f4eef7fe700 1 -- 172.21.15.40:0/425867839 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f00037d30 con 0x559422bd26b0 2023-07-18T03:31:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.351+0000 7f4f22533b80 10 client.15651 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.351+0000 7f4f22533b80 1 -- 172.21.15.40:0/425867839 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f4f08005e70 con 0x559422bd26b0 2023-07-18T03:31:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.351+0000 7f4eef7fe700 1 -- 172.21.15.40:0/425867839 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f00046020 con 0x559422bd26b0 2023-07-18T03:31:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.351+0000 7f4eef7fe700 1 -- 172.21.15.40:0/425867839 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4f00023930 con 0x559422bd26b0 2023-07-18T03:31:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.351+0000 7f4f22533b80 10 client.15651 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.351+0000 7f4f22533b80 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.351+0000 7f4f22533b80 20 client.15651 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.351+0000 7f4f22533b80 1 --2- 172.21.15.40:0/425867839 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559422ddffe0 0x559422de23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.351+0000 7f4f22533b80 4 client.15651 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:31:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.351+0000 7f4f22533b80 1 -- 172.21.15.40:0/425867839 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559422c82ff0 con 0x559422ddffe0 2023-07-18T03:31:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.351+0000 7f4f0f7fe700 1 --2- 172.21.15.40:0/425867839 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559422ddffe0 0x559422de23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.352+0000 7f4eef7fe700 10 client.15651 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.352+0000 7f4f0f7fe700 1 --2- 172.21.15.40:0/425867839 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559422ddffe0 0x559422de23f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da604bfde6bc61e9 server_cookie=49f861157b7b6330 in_seq=0 out_seq=0 2023-07-18T03:31:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.360+0000 7f4eef7fe700 1 -- 172.21.15.40:0/425867839 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559422c82ff0 con 0x559422ddffe0 2023-07-18T03:31:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.360+0000 7f4eef7fe700 10 client.15651 handle_command_reply: tid=0 2023-07-18T03:31:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.360+0000 7f4f22533b80 1 client.15651 shutdown 2023-07-18T03:31:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.360+0000 7f4f22533b80 1 -- 172.21.15.40:0/425867839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 msgr2=0x559422d7b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.360+0000 7f4f22533b80 1 --2- 172.21.15.40:0/425867839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 0x559422d7b890 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4f00030040 tx=0x7f4f0000b7d0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.360+0000 7f4f22533b80 1 -- 172.21.15.40:0/425867839 shutdown_connections 2023-07-18T03:31:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.360+0000 7f4f22533b80 1 --2- 172.21.15.40:0/425867839 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559422ddffe0 0x559422de23f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.360+0000 7f4f22533b80 1 --2- 172.21.15.40:0/425867839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559422d77e40 0x559422d8b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.360+0000 7f4f22533b80 1 --2- 172.21.15.40:0/425867839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559422bd26b0 0x559422d7b890 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.360+0000 7f4f22533b80 1 --2- 172.21.15.40:0/425867839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559422d779d0 0x559422d7be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.360+0000 7f4f22533b80 1 -- 172.21.15.40:0/425867839 >> 172.21.15.40:0/425867839 conn(0x559422d6bbe0 msgr2=0x559422d7e960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.360+0000 7f4eef7fe700 0 client.15651 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.360+0000 7f4f22533b80 1 -- 172.21.15.40:0/425867839 shutdown_connections 2023-07-18T03:31:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.360+0000 7f4f22533b80 1 -- 172.21.15.40:0/425867839 wait complete. 2023-07-18T03:31:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.360+0000 7f4f22533b80 20 client.15651 trim_cache size 0 max 16384 2023-07-18T03:31:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:31:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:31:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:31:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:31:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.361+0000 7f4f163ad700 1 -- 172.21.15.40:0/1702688550 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4ef80a1150 msgr2=0x7f4ef80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.361+0000 7f4f163ad700 1 --2- 172.21.15.40:0/1702688550 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4ef80a1150 0x7f4ef80a3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000b5c0 tx=0x7f4f00005960 comp rx=0 tx=0).stop 2023-07-18T03:31:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.361+0000 7f4f163ad700 1 -- 172.21.15.40:0/1702688550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f100ffc90 msgr2=0x7f4f10194c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.361+0000 7f4f163ad700 1 --2- 172.21.15.40:0/1702688550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f100ffc90 0x7f4f10194c80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400c9b0 tx=0x7f4f0400ccc0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.361+0000 7f4f163ad700 1 -- 172.21.15.40:0/1702688550 shutdown_connections 2023-07-18T03:31:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.361+0000 7f4f163ad700 1 --2- 172.21.15.40:0/1702688550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f10100640 0x7f4f10195b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.361+0000 7f4f163ad700 1 --2- 172.21.15.40:0/1702688550 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f4ef80a1150 0x7f4ef80a3600 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.361+0000 7f4f163ad700 1 --2- 172.21.15.40:0/1702688550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f100fea00 0x7f4f10194740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.361+0000 7f4f163ad700 1 --2- 172.21.15.40:0/1702688550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f100ffc90 0x7f4f10194c80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.361+0000 7f4f163ad700 1 -- 172.21.15.40:0/1702688550 >> 172.21.15.40:0/1702688550 conn(0x7f4f100fa000 msgr2=0x7f4f100fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.361+0000 7f4f163ad700 1 -- 172.21.15.40:0/1702688550 shutdown_connections 2023-07-18T03:31:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:30.361+0000 7f4f163ad700 1 -- 172.21.15.40:0/1702688550 wait complete. 2023-07-18T03:31:32.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:31 smithi064 ceph-mon[98059]: pgmap v312: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-07-18T03:31:32.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:31 smithi134 ceph-mon[98216]: pgmap v312: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-07-18T03:31:32.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:31 smithi040 ceph-mon[96439]: pgmap v312: 97 pgs: 97 active+clean; 663 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-07-18T03:31:33.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:32 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:31:33.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:32 smithi064 ceph-mon[98059]: pgmap v313: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2023-07-18T03:31:33.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:32 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:31:33.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:32 smithi134 ceph-mon[98216]: pgmap v313: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2023-07-18T03:31:33.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:32 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:31:33.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:32 smithi040 ceph-mon[96439]: pgmap v313: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2023-07-18T03:31:35.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:35 smithi134 ceph-mon[98216]: pgmap v314: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-07-18T03:31:35.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:35.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:35.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:35.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:35.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:35.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:35 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:35.831 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:35 smithi040 ceph-mon[96439]: pgmap v314: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-07-18T03:31:35.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:35.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:35.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:35.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:35.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:35.832 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:35 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:35.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:35 smithi064 ceph-mon[98059]: pgmap v314: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-07-18T03:31:35.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:35.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:35.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:35.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:35.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:35.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:35 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:36.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:36 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:31:36.817 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:36 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:31:36.818 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:36 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:36.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:36 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:31:36.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:36 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:31:36.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:36 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:36.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:36 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:31:36.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:36 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:31:36.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:36 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:31:38.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:37 smithi134 ceph-mon[98216]: pgmap v315: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-07-18T03:31:38.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:37 smithi040 ceph-mon[96439]: pgmap v315: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-07-18T03:31:38.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:37 smithi064 ceph-mon[98059]: pgmap v315: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-07-18T03:31:40.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:39 smithi134 ceph-mon[98216]: pgmap v316: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 12 op/s 2023-07-18T03:31:40.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:39 smithi040 ceph-mon[96439]: pgmap v316: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 12 op/s 2023-07-18T03:31:40.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:39 smithi064 ceph-mon[98059]: pgmap v316: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 12 op/s 2023-07-18T03:31:42.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:41 smithi134 ceph-mon[98216]: pgmap v317: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 10 op/s 2023-07-18T03:31:42.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:41 smithi040 ceph-mon[96439]: pgmap v317: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 10 op/s 2023-07-18T03:31:42.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:41 smithi064 ceph-mon[98059]: pgmap v317: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 10 op/s 2023-07-18T03:31:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.535+0000 7f7a6ef3d700 1 -- 172.21.15.40:0/3447261670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680fec00 msgr2=0x7f7a680ff070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.535+0000 7f7a6ef3d700 1 --2- 172.21.15.40:0/3447261670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680fec00 0x7f7a680ff070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7a540098f0 tx=0x7f7a54009c00 comp rx=0 tx=0).stop 2023-07-18T03:31:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.535+0000 7f7a6ef3d700 1 -- 172.21.15.40:0/3447261670 shutdown_connections 2023-07-18T03:31:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.535+0000 7f7a6ef3d700 1 --2- 172.21.15.40:0/3447261670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a680ff5b0 0x7f7a68106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.535+0000 7f7a6ef3d700 1 --2- 172.21.15.40:0/3447261670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a680fffb0 0x7f7a681003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.535+0000 7f7a6ef3d700 1 --2- 172.21.15.40:0/3447261670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680fec00 0x7f7a680ff070 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.535+0000 7f7a6ef3d700 1 -- 172.21.15.40:0/3447261670 >> 172.21.15.40:0/3447261670 conn(0x7f7a680f9780 msgr2=0x7f7a680fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.535+0000 7f7a6ef3d700 1 -- 172.21.15.40:0/3447261670 shutdown_connections 2023-07-18T03:31:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.535+0000 7f7a6ef3d700 1 -- 172.21.15.40:0/3447261670 wait complete. 2023-07-18T03:31:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.536+0000 7f7a6ef3d700 1 Processor -- start 2023-07-18T03:31:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.536+0000 7f7a6ef3d700 1 -- start start 2023-07-18T03:31:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.536+0000 7f7a6ef3d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a680fec00 0x7f7a681985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.536+0000 7f7a6ef3d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680ff5b0 0x7f7a6819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.536+0000 7f7a6ef3d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a680fffb0 0x7f7a6819d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.536+0000 7f7a6ef3d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6819d620 con 0x7f7a680fffb0 2023-07-18T03:31:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.536+0000 7f7a6ef3d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6819d760 con 0x7f7a680ff5b0 2023-07-18T03:31:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.536+0000 7f7a6ef3d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a68194390 con 0x7f7a680fec00 2023-07-18T03:31:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.537+0000 7f7a6d4da700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a680fffb0 0x7f7a6819d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.537+0000 7f7a6d4da700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a680fffb0 0x7f7a6819d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58434/0 (socket says 172.21.15.40:58434) 2023-07-18T03:31:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.537+0000 7f7a6d4da700 1 -- 172.21.15.40:0/3353727483 learned_addr learned my addr 172.21.15.40:0/3353727483 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.537+0000 7f7a6ccd9700 1 --2- 172.21.15.40:0/3353727483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a680fec00 0x7f7a681985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.537+0000 7f7a67fff700 1 --2- 172.21.15.40:0/3353727483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680ff5b0 0x7f7a6819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.537+0000 7f7a6d4da700 1 -- 172.21.15.40:0/3353727483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a680fec00 msgr2=0x7f7a681985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.537+0000 7f7a6d4da700 1 --2- 172.21.15.40:0/3353727483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a680fec00 0x7f7a681985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.537+0000 7f7a6d4da700 1 -- 172.21.15.40:0/3353727483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680ff5b0 msgr2=0x7f7a6819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.537+0000 7f7a6d4da700 1 --2- 172.21.15.40:0/3353727483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680ff5b0 0x7f7a6819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.537+0000 7f7a6d4da700 1 -- 172.21.15.40:0/3353727483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a54009520 con 0x7f7a680fffb0 2023-07-18T03:31:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.537+0000 7f7a67fff700 1 --2- 172.21.15.40:0/3353727483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680ff5b0 0x7f7a6819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:31:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.537+0000 7f7a6d4da700 1 --2- 172.21.15.40:0/3353727483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a680fffb0 0x7f7a6819d050 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000ef30 tx=0x7f7a6000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.537+0000 7f7a65ffb700 1 -- 172.21.15.40:0/3353727483 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60018070 con 0x7f7a680fffb0 2023-07-18T03:31:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.538+0000 7f7a65ffb700 1 -- 172.21.15.40:0/3353727483 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a6000f040 con 0x7f7a680fffb0 2023-07-18T03:31:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.538+0000 7f7a6ef3d700 1 -- 172.21.15.40:0/3353727483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a68194610 con 0x7f7a680fffb0 2023-07-18T03:31:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.538+0000 7f7a65ffb700 1 -- 172.21.15.40:0/3353727483 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6000cda0 con 0x7f7a680fffb0 2023-07-18T03:31:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.538+0000 7f7a6ef3d700 1 -- 172.21.15.40:0/3353727483 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a68194ad0 con 0x7f7a680fffb0 2023-07-18T03:31:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.541+0000 7f7a65ffb700 1 -- 172.21.15.40:0/3353727483 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7a60003a00 con 0x7f7a680fffb0 2023-07-18T03:31:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.542+0000 7f7a65ffb700 1 --2- 172.21.15.40:0/3353727483 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7a580a1150 0x7f7a580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.542+0000 7f7a6ccd9700 1 --2- 172.21.15.40:0/3353727483 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7a580a1150 0x7f7a580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:43.539 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.542+0000 7f7a65ffb700 1 -- 172.21.15.40:0/3353727483 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7a600cc560 con 0x7f7a680fffb0 2023-07-18T03:31:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.543+0000 7f7a6ccd9700 1 --2- 172.21.15.40:0/3353727483 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7a580a1150 0x7f7a580a3600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7a680fe5d0 tx=0x7f7a5c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.543+0000 7f7a7b0c3b80 1 Processor -- start 2023-07-18T03:31:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.543+0000 7f7a7b0c3b80 1 -- start start 2023-07-18T03:31:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.543+0000 7f7a7b0c3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c9394a6b0 0x560c93af6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.543+0000 7f7a7b0c3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c93af7050 0x560c93af79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.543+0000 7f7a7b0c3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c93af7ef0 0x560c93af8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.543+0000 7f7a7b0c3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c939f3c50 con 0x560c93af7050 2023-07-18T03:31:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.543+0000 7f7a7b0c3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c93a469e0 con 0x560c9394a6b0 2023-07-18T03:31:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.543+0000 7f7a7b0c3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c93a3fe70 con 0x560c93af7ef0 2023-07-18T03:31:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.543+0000 7f7a6ccd9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c9394a6b0 0x560c93af6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.543+0000 7f7a6ccd9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c9394a6b0 0x560c93af6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:41766/0 (socket says 172.21.15.40:41766) 2023-07-18T03:31:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.543+0000 7f7a6ccd9700 1 -- 172.21.15.40:0/163711366 learned_addr learned my addr 172.21.15.40:0/163711366 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.544+0000 7f7a6d4da700 1 --2- 172.21.15.40:0/163711366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c93af7ef0 0x560c93af8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.544+0000 7f7a6ccd9700 1 -- 172.21.15.40:0/163711366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c93af7ef0 msgr2=0x560c93af8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.544+0000 7f7a6ccd9700 1 --2- 172.21.15.40:0/163711366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c93af7ef0 0x560c93af8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.544+0000 7f7a6ccd9700 1 -- 172.21.15.40:0/163711366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c93af7050 msgr2=0x560c93af79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.544+0000 7f7a6ccd9700 1 --2- 172.21.15.40:0/163711366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c93af7050 0x560c93af79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.544+0000 7f7a6ccd9700 1 -- 172.21.15.40:0/163711366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c9386dd80 con 0x560c9394a6b0 2023-07-18T03:31:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.544+0000 7f7a6ccd9700 1 --2- 172.21.15.40:0/163711366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c9394a6b0 0x560c93af6b10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00e460 tx=0x7f7a5c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e3a44f20e206642 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.544+0000 7f7a52ffd700 1 -- 172.21.15.40:0/163711366 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c004830 con 0x560c9394a6b0 2023-07-18T03:31:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.544+0000 7f7a52ffd700 1 -- 172.21.15.40:0/163711366 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a5c021070 con 0x560c9394a6b0 2023-07-18T03:31:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.544+0000 7f7a52ffd700 1 -- 172.21.15.40:0/163711366 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0266e0 con 0x560c9394a6b0 2023-07-18T03:31:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.545+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/163711366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c9394a6b0 msgr2=0x560c93af6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.545+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/163711366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c9394a6b0 0x560c93af6b10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00e460 tx=0x7f7a5c014000 comp rx=0 tx=0).stop 2023-07-18T03:31:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.545+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/163711366 shutdown_connections 2023-07-18T03:31:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.545+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/163711366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c93af7ef0 0x560c93af8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.545+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/163711366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c9394a6b0 0x560c93af6b10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.545+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/163711366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c93af7050 0x560c93af79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.545+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/163711366 >> 172.21.15.40:0/163711366 conn(0x560c93ae3be0 msgr2=0x560c93ae8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.545+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/163711366 shutdown_connections 2023-07-18T03:31:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.545+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/163711366 wait complete. 2023-07-18T03:31:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.546+0000 7f7a7b0c3b80 1 -- start start 2023-07-18T03:31:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.546+0000 7f7a7b0c3b80 1 Processor -- start 2023-07-18T03:31:43.549 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.546+0000 7f7a7b0c3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c9394a6b0 0x560c93b000e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.550 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.546+0000 7f7a7b0c3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c93af7050 0x560c93b00620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.550 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.546+0000 7f7a7b0c3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c93af7ef0 0x560c93b07660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.550 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.546+0000 7f7a7b0c3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c939dfa60 con 0x560c9394a6b0 2023-07-18T03:31:43.550 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.546+0000 7f7a7b0c3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c939df760 con 0x560c93af7ef0 2023-07-18T03:31:43.550 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.546+0000 7f7a7b0c3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c938c38c0 con 0x560c93af7050 2023-07-18T03:31:43.550 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.546+0000 7f7a7b0c3b80 10 client.0 _ll_register_callbacks cb 0x560c939dd700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:43.550 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.547+0000 7f7a6ccd9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c9394a6b0 0x560c93b000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.547+0000 7f7a6ccd9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c9394a6b0 0x560c93b000e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58446/0 (socket says 172.21.15.40:58446) 2023-07-18T03:31:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.547+0000 7f7a6ccd9700 1 -- 172.21.15.40:0/1927078354 learned_addr learned my addr 172.21.15.40:0/1927078354 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.547+0000 7f7a67fff700 1 --2- 172.21.15.40:0/1927078354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c93af7050 0x560c93b00620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.547+0000 7f7a6ccd9700 1 -- 172.21.15.40:0/1927078354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c93af7050 msgr2=0x560c93b00620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.547+0000 7f7a6ccd9700 1 --2- 172.21.15.40:0/1927078354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c93af7050 0x560c93b00620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.547+0000 7f7a6ccd9700 1 -- 172.21.15.40:0/1927078354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c93af7ef0 msgr2=0x560c93b07660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.547+0000 7f7a6ccd9700 1 --2- 172.21.15.40:0/1927078354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c93af7ef0 0x560c93b07660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.547+0000 7f7a6ccd9700 1 -- 172.21.15.40:0/1927078354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5c017070 con 0x560c9394a6b0 2023-07-18T03:31:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.547+0000 7f7a67fff700 1 --2- 172.21.15.40:0/1927078354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c93af7050 0x560c93b00620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:31:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.547+0000 7f7a6ccd9700 1 --2- 172.21.15.40:0/1927078354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c9394a6b0 0x560c93b000e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0093f0 tx=0x7f7a5c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae21310351afb551 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.547+0000 7f7a47fff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.548+0000 7f7a47fff700 1 -- 172.21.15.40:0/1927078354 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c004c20 con 0x560c9394a6b0 2023-07-18T03:31:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.548+0000 7f7a47fff700 1 -- 172.21.15.40:0/1927078354 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7a5c004d80 con 0x560c9394a6b0 2023-07-18T03:31:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.548+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/1927078354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c93999490 con 0x560c9394a6b0 2023-07-18T03:31:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.548+0000 7f7a47fff700 1 -- 172.21.15.40:0/1927078354 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a5c004f40 con 0x560c9394a6b0 2023-07-18T03:31:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.548+0000 7f7a7b0c3b80 10 client.15660 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.548+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/1927078354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560c93856c00 con 0x560c9394a6b0 2023-07-18T03:31:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.548+0000 7f7a47fff700 1 -- 172.21.15.40:0/1927078354 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560c93999490 con 0x560c9394a6b0 2023-07-18T03:31:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.548+0000 7f7a47fff700 1 -- 172.21.15.40:0/1927078354 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7a5c038020 con 0x560c9394a6b0 2023-07-18T03:31:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.548+0000 7f7a7b0c3b80 10 client.15660 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.548+0000 7f7a7b0c3b80 10 client.15660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.548+0000 7f7a7b0c3b80 20 client.15660 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.548+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/1927078354 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560c93b0bda0 0x560c93b0e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.548+0000 7f7a7b0c3b80 4 client.15660 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.548+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/1927078354 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c93a38e50 con 0x560c93b0bda0 2023-07-18T03:31:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.549+0000 7f7a67fff700 1 --2- 172.21.15.40:0/1927078354 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560c93b0bda0 0x560c93b0e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.549+0000 7f7a47fff700 10 client.15660 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.549+0000 7f7a67fff700 1 --2- 172.21.15.40:0/1927078354 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560c93b0bda0 0x560c93b0e1b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a9b6a459850a24f server_cookie=447704b931d3554d in_seq=0 out_seq=0 2023-07-18T03:31:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.550+0000 7f7a47fff700 1 -- 172.21.15.40:0/1927078354 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560c93a38e50 con 0x560c93b0bda0 2023-07-18T03:31:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.550+0000 7f7a47fff700 10 client.15660 handle_command_reply: tid=0 2023-07-18T03:31:43.555 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.550+0000 7f7a7b0c3b80 1 client.15660 shutdown 2023-07-18T03:31:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.550+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/1927078354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c9394a6b0 msgr2=0x560c93b000e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.550+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/1927078354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c9394a6b0 0x560c93b000e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0093f0 tx=0x7f7a5c00ee70 comp rx=0 tx=0).stop 2023-07-18T03:31:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.550+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/1927078354 shutdown_connections 2023-07-18T03:31:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.550+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/1927078354 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560c93b0bda0 0x560c93b0e1b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.550+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/1927078354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c93af7ef0 0x560c93b07660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.550+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/1927078354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c9394a6b0 0x560c93b000e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.556 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.550+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/1927078354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c93af7050 0x560c93b00620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.550+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/1927078354 >> 172.21.15.40:0/1927078354 conn(0x560c93ae3be0 msgr2=0x560c93ae84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.550+0000 7f7a47fff700 0 client.15660 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.551+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/1927078354 shutdown_connections 2023-07-18T03:31:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.551+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/1927078354 wait complete. 2023-07-18T03:31:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.551+0000 7f7a7b0c3b80 20 client.15660 trim_cache size 0 max 16384 2023-07-18T03:31:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.580+0000 7f7a7b0c3b80 1 Processor -- start 2023-07-18T03:31:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.580+0000 7f7a7b0c3b80 1 -- start start 2023-07-18T03:31:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.580+0000 7f7a7b0c3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c9394a6b0 0x560c93af82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.580+0000 7f7a7b0c3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c93af7050 0x560c93b07620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.580+0000 7f7a7b0c3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c93af7ef0 0x560c93b12b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.580+0000 7f7a7b0c3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c93ae0bb0 con 0x560c93af7050 2023-07-18T03:31:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.580+0000 7f7a7b0c3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c93aea7a0 con 0x560c93af7ef0 2023-07-18T03:31:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.580+0000 7f7a7b0c3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c93ae9900 con 0x560c9394a6b0 2023-07-18T03:31:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.580+0000 7f7a6d4da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c93af7ef0 0x560c93b12b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.580+0000 7f7a6d4da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c93af7ef0 0x560c93b12b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:41782/0 (socket says 172.21.15.40:41782) 2023-07-18T03:31:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.581+0000 7f7a6ccd9700 1 --2- 172.21.15.40:0/2796943548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c9394a6b0 0x560c93af82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.581+0000 7f7a6d4da700 1 -- 172.21.15.40:0/2796943548 learned_addr learned my addr 172.21.15.40:0/2796943548 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.581+0000 7f7a67fff700 1 --2- 172.21.15.40:0/2796943548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c93af7050 0x560c93b07620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.581+0000 7f7a6ccd9700 1 -- 172.21.15.40:0/2796943548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c93af7ef0 msgr2=0x560c93b12b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.581+0000 7f7a6ccd9700 1 --2- 172.21.15.40:0/2796943548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c93af7ef0 0x560c93b12b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.581+0000 7f7a6ccd9700 1 -- 172.21.15.40:0/2796943548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c93af7050 msgr2=0x560c93b07620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.581+0000 7f7a6ccd9700 1 --2- 172.21.15.40:0/2796943548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c93af7050 0x560c93b07620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.581+0000 7f7a6ccd9700 1 -- 172.21.15.40:0/2796943548 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c938941f0 con 0x560c9394a6b0 2023-07-18T03:31:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.581+0000 7f7a6d4da700 1 --2- 172.21.15.40:0/2796943548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c93af7ef0 0x560c93b12b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.581+0000 7f7a6ccd9700 1 --2- 172.21.15.40:0/2796943548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c9394a6b0 0x560c93af82c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00eb00 tx=0x7f7a5c030670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eccfd930d03ca036 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.581+0000 7f7a537fe700 1 -- 172.21.15.40:0/2796943548 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c03f070 con 0x560c9394a6b0 2023-07-18T03:31:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.581+0000 7f7a537fe700 1 -- 172.21.15.40:0/2796943548 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a5c030e40 con 0x560c9394a6b0 2023-07-18T03:31:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.581+0000 7f7a537fe700 1 -- 172.21.15.40:0/2796943548 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0038e0 con 0x560c9394a6b0 2023-07-18T03:31:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.582+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/2796943548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c9394a6b0 msgr2=0x560c93af82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.582+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/2796943548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c9394a6b0 0x560c93af82c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00eb00 tx=0x7f7a5c030670 comp rx=0 tx=0).stop 2023-07-18T03:31:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.582+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/2796943548 shutdown_connections 2023-07-18T03:31:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.582+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/2796943548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c93af7ef0 0x560c93b12b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.582+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/2796943548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c9394a6b0 0x560c93af82c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.582+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/2796943548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c93af7050 0x560c93b07620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.582+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/2796943548 >> 172.21.15.40:0/2796943548 conn(0x560c93ae3be0 msgr2=0x560c93ae8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.583+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/2796943548 shutdown_connections 2023-07-18T03:31:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.583+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/2796943548 wait complete. 2023-07-18T03:31:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.583+0000 7f7a7b0c3b80 1 -- start start 2023-07-18T03:31:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.583+0000 7f7a7b0c3b80 1 Processor -- start 2023-07-18T03:31:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.584+0000 7f7a7b0c3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c9394a6b0 0x560c93b03850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.584+0000 7f7a7b0c3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c93af7050 0x560c93b03d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.584+0000 7f7a7b0c3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c93af7ef0 0x560c93b066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.584+0000 7f7a7b0c3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c93b06bf0 con 0x560c93af7ef0 2023-07-18T03:31:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.584+0000 7f7a7b0c3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c93b06d30 con 0x560c93af7050 2023-07-18T03:31:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.584+0000 7f7a7b0c3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c93b06ef0 con 0x560c9394a6b0 2023-07-18T03:31:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.584+0000 7f7a7b0c3b80 10 client.0 _ll_register_callbacks cb 0x7f7a5c00e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.584+0000 7f7a6ccd9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c9394a6b0 0x560c93b03850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.584+0000 7f7a6ccd9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c9394a6b0 0x560c93b03850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:47488/0 (socket says 172.21.15.40:47488) 2023-07-18T03:31:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.584+0000 7f7a6ccd9700 1 -- 172.21.15.40:0/3705350093 learned_addr learned my addr 172.21.15.40:0/3705350093 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.584+0000 7f7a6d4da700 1 --2- 172.21.15.40:0/3705350093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c93af7ef0 0x560c93b066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.584+0000 7f7a6ccd9700 1 -- 172.21.15.40:0/3705350093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c93af7050 msgr2=0x560c93b03d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:31:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.584+0000 7f7a6ccd9700 1 --2- 172.21.15.40:0/3705350093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c93af7050 0x560c93b03d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.584+0000 7f7a6ccd9700 1 -- 172.21.15.40:0/3705350093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c93af7ef0 msgr2=0x560c93b066b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.584+0000 7f7a6ccd9700 1 --2- 172.21.15.40:0/3705350093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c93af7ef0 0x560c93b066b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.584+0000 7f7a6ccd9700 1 -- 172.21.15.40:0/3705350093 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a68194ad0 con 0x560c9394a6b0 2023-07-18T03:31:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.585+0000 7f7a6ccd9700 1 --2- 172.21.15.40:0/3705350093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c9394a6b0 0x560c93b03850 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c004590 tx=0x7f7a5c03a650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=827b41964a3308be server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.585+0000 7f7a47fff700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:31:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.585+0000 7f7a47fff700 1 -- 172.21.15.40:0/3705350093 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c03f070 con 0x560c9394a6b0 2023-07-18T03:31:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.585+0000 7f7a47fff700 1 -- 172.21.15.40:0/3705350093 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7a5c030c60 con 0x560c9394a6b0 2023-07-18T03:31:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.585+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/3705350093 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a5c021070 con 0x560c9394a6b0 2023-07-18T03:31:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.585+0000 7f7a47fff700 1 -- 172.21.15.40:0/3705350093 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a5c03aa50 con 0x560c9394a6b0 2023-07-18T03:31:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.585+0000 7f7a7b0c3b80 10 client.25345 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.585+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/3705350093 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560c93856c00 con 0x560c9394a6b0 2023-07-18T03:31:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.585+0000 7f7a47fff700 1 -- 172.21.15.40:0/3705350093 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a5c021070 con 0x560c9394a6b0 2023-07-18T03:31:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.585+0000 7f7a47fff700 1 -- 172.21.15.40:0/3705350093 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7a5c04c020 con 0x560c9394a6b0 2023-07-18T03:31:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.585+0000 7f7a7b0c3b80 10 client.25345 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.585+0000 7f7a7b0c3b80 10 client.25345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.585+0000 7f7a7b0c3b80 20 client.25345 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.585+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/3705350093 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560c93b60510 0x560c93b628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.586+0000 7f7a7b0c3b80 4 client.25345 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:31:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.586+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/3705350093 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560c939faff0 con 0x560c93b60510 2023-07-18T03:31:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.586+0000 7f7a67fff700 1 --2- 172.21.15.40:0/3705350093 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560c93b60510 0x560c93b628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.586+0000 7f7a47fff700 10 client.25345 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.586+0000 7f7a67fff700 1 --2- 172.21.15.40:0/3705350093 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560c93b60510 0x560c93b628f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39fc9e79081e78c1 server_cookie=757928a8f4d5fc98 in_seq=0 out_seq=0 2023-07-18T03:31:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.590+0000 7f7a47fff700 1 -- 172.21.15.40:0/3705350093 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560c939faff0 con 0x560c93b60510 2023-07-18T03:31:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.590+0000 7f7a47fff700 10 client.25345 handle_command_reply: tid=0 2023-07-18T03:31:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.590+0000 7f7a7b0c3b80 1 client.25345 shutdown 2023-07-18T03:31:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.591+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/3705350093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c9394a6b0 msgr2=0x560c93b03850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.591+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/3705350093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c9394a6b0 0x560c93b03850 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c004590 tx=0x7f7a5c03a650 comp rx=0 tx=0).stop 2023-07-18T03:31:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.591+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/3705350093 shutdown_connections 2023-07-18T03:31:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.591+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/3705350093 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560c93b60510 0x560c93b628f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.591+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/3705350093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560c93af7ef0 0x560c93b066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.591+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/3705350093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c9394a6b0 0x560c93b03850 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.591+0000 7f7a7b0c3b80 1 --2- 172.21.15.40:0/3705350093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c93af7050 0x560c93b03d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.591+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/3705350093 >> 172.21.15.40:0/3705350093 conn(0x560c93ae3be0 msgr2=0x560c93afedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.591+0000 7f7a47fff700 0 client.25345 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.591+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/3705350093 shutdown_connections 2023-07-18T03:31:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.591+0000 7f7a7b0c3b80 1 -- 172.21.15.40:0/3705350093 wait complete. 2023-07-18T03:31:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.591+0000 7f7a7b0c3b80 20 client.25345 trim_cache size 0 max 16384 2023-07-18T03:31:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:31:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:31:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:31:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:31:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.592+0000 7f7a6ef3d700 1 -- 172.21.15.40:0/3353727483 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7a580a1150 msgr2=0x7f7a580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.592+0000 7f7a6ef3d700 1 --2- 172.21.15.40:0/3353727483 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7a580a1150 0x7f7a580a3600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7a680fe5d0 tx=0x7f7a5c009310 comp rx=0 tx=0).stop 2023-07-18T03:31:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.592+0000 7f7a6ef3d700 1 -- 172.21.15.40:0/3353727483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a680fffb0 msgr2=0x7f7a6819d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.592+0000 7f7a6ef3d700 1 --2- 172.21.15.40:0/3353727483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a680fffb0 0x7f7a6819d050 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000ef30 tx=0x7f7a6000c450 comp rx=0 tx=0).stop 2023-07-18T03:31:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.596+0000 7f7a6ef3d700 1 -- 172.21.15.40:0/3353727483 shutdown_connections 2023-07-18T03:31:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.596+0000 7f7a6ef3d700 1 --2- 172.21.15.40:0/3353727483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a680fffb0 0x7f7a6819d050 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.596+0000 7f7a6ef3d700 1 --2- 172.21.15.40:0/3353727483 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7a580a1150 0x7f7a580a3600 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.596+0000 7f7a6ef3d700 1 --2- 172.21.15.40:0/3353727483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a680fec00 0x7f7a681985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.596+0000 7f7a6ef3d700 1 --2- 172.21.15.40:0/3353727483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a680ff5b0 0x7f7a6819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.596+0000 7f7a6ef3d700 1 -- 172.21.15.40:0/3353727483 >> 172.21.15.40:0/3353727483 conn(0x7f7a680f9780 msgr2=0x7f7a68103130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.596+0000 7f7a6ef3d700 1 -- 172.21.15.40:0/3353727483 shutdown_connections 2023-07-18T03:31:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:43.596+0000 7f7a6ef3d700 1 -- 172.21.15.40:0/3353727483 wait complete. 2023-07-18T03:31:44.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:43 smithi134 ceph-mon[98216]: pgmap v318: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 13 op/s 2023-07-18T03:31:44.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:43 smithi040 ceph-mon[96439]: pgmap v318: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 13 op/s 2023-07-18T03:31:44.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:43 smithi064 ceph-mon[98059]: pgmap v318: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 13 op/s 2023-07-18T03:31:46.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:45 smithi134 ceph-mon[98216]: pgmap v319: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2023-07-18T03:31:46.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:45 smithi040 ceph-mon[96439]: pgmap v319: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2023-07-18T03:31:46.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:45 smithi064 ceph-mon[98059]: pgmap v319: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 10 op/s 2023-07-18T03:31:48.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:47 smithi134 ceph-mon[98216]: pgmap v320: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2023-07-18T03:31:48.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:47 smithi040 ceph-mon[96439]: pgmap v320: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2023-07-18T03:31:48.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:47 smithi064 ceph-mon[98059]: pgmap v320: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2023-07-18T03:31:50.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:49 smithi134 ceph-mon[98216]: pgmap v321: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2023-07-18T03:31:50.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:49 smithi064 ceph-mon[98059]: pgmap v321: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2023-07-18T03:31:50.103 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:49 smithi040 ceph-mon[96439]: pgmap v321: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2023-07-18T03:31:52.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:51 smithi134 ceph-mon[98216]: pgmap v322: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-18T03:31:52.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:51 smithi040 ceph-mon[96439]: pgmap v322: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-18T03:31:52.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:51 smithi064 ceph-mon[98059]: pgmap v322: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-18T03:31:53.017 INFO:tasks.workunit.client.0.smithi040.stderr:Updating files: 98% (11517/11710) Updating files: 99% (11593/11710) Updating files: 100% (11710/11710) Updating files: 100% (11710/11710), done. 2023-07-18T03:31:53.325 INFO:tasks.workunit.client.0.smithi040.stderr:Note: switching to '81cf785aa7781c4e08381c540f890cb9bc987ef9'. 2023-07-18T03:31:53.325 INFO:tasks.workunit.client.0.smithi040.stderr: 2023-07-18T03:31:53.325 INFO:tasks.workunit.client.0.smithi040.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-07-18T03:31:53.325 INFO:tasks.workunit.client.0.smithi040.stderr:changes and commit them, and you can discard any commits you make in this 2023-07-18T03:31:53.325 INFO:tasks.workunit.client.0.smithi040.stderr:state without impacting any branches by switching back to a branch. 2023-07-18T03:31:53.325 INFO:tasks.workunit.client.0.smithi040.stderr: 2023-07-18T03:31:53.326 INFO:tasks.workunit.client.0.smithi040.stderr:If you want to create a new branch to retain commits you create, you may 2023-07-18T03:31:53.326 INFO:tasks.workunit.client.0.smithi040.stderr:do so (now or later) by using -c with the switch command. Example: 2023-07-18T03:31:53.326 INFO:tasks.workunit.client.0.smithi040.stderr: 2023-07-18T03:31:53.326 INFO:tasks.workunit.client.0.smithi040.stderr: git switch -c 2023-07-18T03:31:53.326 INFO:tasks.workunit.client.0.smithi040.stderr: 2023-07-18T03:31:53.326 INFO:tasks.workunit.client.0.smithi040.stderr:Or undo this operation with: 2023-07-18T03:31:53.326 INFO:tasks.workunit.client.0.smithi040.stderr: 2023-07-18T03:31:53.327 INFO:tasks.workunit.client.0.smithi040.stderr: git switch - 2023-07-18T03:31:53.327 INFO:tasks.workunit.client.0.smithi040.stderr: 2023-07-18T03:31:53.327 INFO:tasks.workunit.client.0.smithi040.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-07-18T03:31:53.327 INFO:tasks.workunit.client.0.smithi040.stderr: 2023-07-18T03:31:53.327 INFO:tasks.workunit.client.0.smithi040.stderr:HEAD is now at 81cf785aa77 Merge remote-tracking branch 'temp/wip-61599' into wip-rishabh-2023Jul13-m-quota 2023-07-18T03:31:53.330 DEBUG:teuthology.orchestra.run.smithi040:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-07-18T03:31:53.371 INFO:tasks.workunit.client.0.smithi040.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-07-18T03:31:53.373 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-18T03:31:53.374 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-07-18T03:31:53.899 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-07-18T03:31:53.962 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-07-18T03:31:54.000 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-18T03:31:54.002 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-18T03:31:54.002 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-07-18T03:31:54.039 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-18T03:31:54.043 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:31:54.043 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-07-18T03:31:54.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:53 smithi134 ceph-mon[98216]: pgmap v323: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-07-18T03:31:54.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:53 smithi040 ceph-mon[96439]: pgmap v323: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-07-18T03:31:54.097 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2023-07-18T03:31:54.099 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2023-07-18T03:31:54.100 DEBUG:teuthology.orchestra.run.smithi040:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=81cf785aa7781c4e08381c540f890cb9bc987ef9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2023-07-18T03:31:54.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:53 smithi064 ceph-mon[98059]: pgmap v323: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-07-18T03:31:54.113 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:31:54.163 INFO:tasks.workunit.client.0.smithi040.stdout:getting blogbench 2023-07-18T03:31:54.163 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'getting blogbench' 2023-07-18T03:31:54.164 INFO:tasks.workunit.client.0.smithi040.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2023-07-18T03:31:54.207 INFO:tasks.workunit.client.0.smithi040.stderr:--2023-07-18 03:31:54-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2023-07-18T03:31:54.207 INFO:tasks.workunit.client.0.smithi040.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2023-07-18T03:31:54.247 INFO:tasks.workunit.client.0.smithi040.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2023-07-18T03:31:54.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.272+0000 7f3ba825c700 1 -- 172.21.15.40:0/79062554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ba00ffb10 msgr2=0x7f3ba00fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.272+0000 7f3ba825c700 1 --2- 172.21.15.40:0/79062554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ba00ffb10 0x7f3ba00fff20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009990 tx=0x7f3b9c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.272+0000 7f3ba825c700 1 -- 172.21.15.40:0/79062554 shutdown_connections 2023-07-18T03:31:54.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.272+0000 7f3ba825c700 1 --2- 172.21.15.40:0/79062554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba0101690 0x7f3ba0107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.272+0000 7f3ba825c700 1 --2- 172.21.15.40:0/79062554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ba00ffb10 0x7f3ba00fff20 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.272+0000 7f3ba825c700 1 --2- 172.21.15.40:0/79062554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ba0100d00 0x7f3ba0101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.272+0000 7f3ba825c700 1 -- 172.21.15.40:0/79062554 >> 172.21.15.40:0/79062554 conn(0x7f3ba00fb110 msgr2=0x7f3ba00fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:54.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.272+0000 7f3ba825c700 1 -- 172.21.15.40:0/79062554 shutdown_connections 2023-07-18T03:31:54.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.272+0000 7f3ba825c700 1 -- 172.21.15.40:0/79062554 wait complete. 2023-07-18T03:31:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.273+0000 7f3ba825c700 1 Processor -- start 2023-07-18T03:31:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.273+0000 7f3ba825c700 1 -- start start 2023-07-18T03:31:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.273+0000 7f3ba825c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba00ffb10 0x7f3ba00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.273+0000 7f3ba825c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ba0100d00 0x7f3ba0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.273+0000 7f3ba825c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ba0101690 0x7f3ba0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.273+0000 7f3ba825c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba00735e0 con 0x7f3ba0100d00 2023-07-18T03:31:54.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.273+0000 7f3ba825c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba0073750 con 0x7f3ba00ffb10 2023-07-18T03:31:54.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.273+0000 7f3ba825c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba019f8f0 con 0x7f3ba0101690 2023-07-18T03:31:54.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.273+0000 7f3ba57f7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ba0100d00 0x7f3ba0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.273+0000 7f3ba57f7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ba0100d00 0x7f3ba0072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58476/0 (socket says 172.21.15.40:58476) 2023-07-18T03:31:54.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.273+0000 7f3ba57f7700 1 -- 172.21.15.40:0/1644678332 learned_addr learned my addr 172.21.15.40:0/1644678332 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.273+0000 7f3ba5ff8700 1 --2- 172.21.15.40:0/1644678332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba00ffb10 0x7f3ba00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.273+0000 7f3ba67f9700 1 --2- 172.21.15.40:0/1644678332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ba0101690 0x7f3ba0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.273+0000 7f3ba57f7700 1 -- 172.21.15.40:0/1644678332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ba0101690 msgr2=0x7f3ba0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.273+0000 7f3ba57f7700 1 --2- 172.21.15.40:0/1644678332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ba0101690 0x7f3ba0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.273+0000 7f3ba57f7700 1 -- 172.21.15.40:0/1644678332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba00ffb10 msgr2=0x7f3ba00742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.273+0000 7f3ba57f7700 1 --2- 172.21.15.40:0/1644678332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba00ffb10 0x7f3ba00742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.274+0000 7f3ba57f7700 1 -- 172.21.15.40:0/1644678332 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c0095f0 con 0x7f3ba0100d00 2023-07-18T03:31:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.274+0000 7f3ba67f9700 1 --2- 172.21.15.40:0/1644678332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ba0101690 0x7f3ba0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:54.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.274+0000 7f3ba57f7700 1 --2- 172.21.15.40:0/1644678332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ba0100d00 0x7f3ba0072900 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400c920 tx=0x7f3b9400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:54.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.274+0000 7f3b92ffd700 1 -- 172.21.15.40:0/1644678332 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94007b00 con 0x7f3ba0100d00 2023-07-18T03:31:54.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.274+0000 7f3ba825c700 1 -- 172.21.15.40:0/1644678332 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba019fbd0 con 0x7f3ba0100d00 2023-07-18T03:31:54.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.274+0000 7f3b92ffd700 1 -- 172.21.15.40:0/1644678332 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b9400ce70 con 0x7f3ba0100d00 2023-07-18T03:31:54.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.274+0000 7f3b92ffd700 1 -- 172.21.15.40:0/1644678332 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94005460 con 0x7f3ba0100d00 2023-07-18T03:31:54.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.274+0000 7f3ba825c700 1 -- 172.21.15.40:0/1644678332 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba01a0130 con 0x7f3ba0100d00 2023-07-18T03:31:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.276+0000 7f3b92ffd700 1 -- 172.21.15.40:0/1644678332 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3b94004750 con 0x7f3ba0100d00 2023-07-18T03:31:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.277+0000 7f3b92ffd700 1 --2- 172.21.15.40:0/1644678332 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3b8c0a1150 0x7f3b8c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.277+0000 7f3ba5ff8700 1 --2- 172.21.15.40:0/1644678332 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3b8c0a1150 0x7f3b8c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.277+0000 7f3b92ffd700 1 -- 172.21.15.40:0/1644678332 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3b940cc520 con 0x7f3ba0100d00 2023-07-18T03:31:54.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.278+0000 7f3ba5ff8700 1 --2- 172.21.15.40:0/1644678332 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3b8c0a1150 0x7f3b8c0a3600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00b5c0 tx=0x7f3b9c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:54.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.278+0000 7f3bb43e2b80 1 Processor -- start 2023-07-18T03:31:54.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.278+0000 7f3bb43e2b80 1 -- start start 2023-07-18T03:31:54.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.278+0000 7f3bb43e2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995a4d0 0x561f4995a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.278+0000 7f3bb43e2b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995ae20 0x561f4995b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.278+0000 7f3bb43e2b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f4995c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.278+0000 7f3bb43e2b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f4985fcb0 con 0x561f4995bc70 2023-07-18T03:31:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.278+0000 7f3bb43e2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f498b2a40 con 0x561f4995a4d0 2023-07-18T03:31:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.278+0000 7f3bb43e2b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f498abed0 con 0x561f4995ae20 2023-07-18T03:31:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.279+0000 7f3ba67f9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f4995c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.279+0000 7f3ba67f9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f4995c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58486/0 (socket says 172.21.15.40:58486) 2023-07-18T03:31:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.279+0000 7f3ba67f9700 1 -- 172.21.15.40:0/3119585722 learned_addr learned my addr 172.21.15.40:0/3119585722 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.279+0000 7f3ba57f7700 1 --2- 172.21.15.40:0/3119585722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995ae20 0x561f4995b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.279+0000 7f3ba5ff8700 1 --2- 172.21.15.40:0/3119585722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995a4d0 0x561f4995a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.279+0000 7f3ba67f9700 1 -- 172.21.15.40:0/3119585722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995ae20 msgr2=0x561f4995b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.279+0000 7f3ba67f9700 1 --2- 172.21.15.40:0/3119585722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995ae20 0x561f4995b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.279+0000 7f3ba67f9700 1 -- 172.21.15.40:0/3119585722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995a4d0 msgr2=0x561f4995a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.279+0000 7f3ba67f9700 1 --2- 172.21.15.40:0/3119585722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995a4d0 0x561f4995a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.279+0000 7f3ba67f9700 1 -- 172.21.15.40:0/3119585722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f496d9de0 con 0x561f4995bc70 2023-07-18T03:31:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.279+0000 7f3ba5ff8700 1 --2- 172.21.15.40:0/3119585722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995a4d0 0x561f4995a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:54.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.279+0000 7f3ba67f9700 1 --2- 172.21.15.40:0/3119585722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f4995c0e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3b98008630 tx=0x7f3b98008940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e983487faec501 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:54.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.279+0000 7f3b8bfff700 1 -- 172.21.15.40:0/3119585722 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b98003c00 con 0x561f4995bc70 2023-07-18T03:31:54.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.279+0000 7f3b8bfff700 1 -- 172.21.15.40:0/3119585722 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b98010040 con 0x561f4995bc70 2023-07-18T03:31:54.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.279+0000 7f3b8bfff700 1 -- 172.21.15.40:0/3119585722 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b980127f0 con 0x561f4995bc70 2023-07-18T03:31:54.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.280+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/3119585722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 msgr2=0x561f4995c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.280+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/3119585722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f4995c0e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3b98008630 tx=0x7f3b98008940 comp rx=0 tx=0).stop 2023-07-18T03:31:54.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.280+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/3119585722 shutdown_connections 2023-07-18T03:31:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.280+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/3119585722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f4995c0e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.280+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/3119585722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995a4d0 0x561f4995a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.280+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/3119585722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995ae20 0x561f4995b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.280+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/3119585722 >> 172.21.15.40:0/3119585722 conn(0x561f4994fda0 msgr2=0x561f499551d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.280+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/3119585722 shutdown_connections 2023-07-18T03:31:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.280+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/3119585722 wait complete. 2023-07-18T03:31:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.280+0000 7f3bb43e2b80 1 -- start start 2023-07-18T03:31:54.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.280+0000 7f3bb43e2b80 1 Processor -- start 2023-07-18T03:31:54.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3bb43e2b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995a4d0 0x561f49964130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3bb43e2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995ae20 0x561f49964670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3bb43e2b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f4996b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3bb43e2b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f4984bac0 con 0x561f4995bc70 2023-07-18T03:31:54.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3bb43e2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f4984b7c0 con 0x561f4995ae20 2023-07-18T03:31:54.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3bb43e2b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f4972f920 con 0x561f4995a4d0 2023-07-18T03:31:54.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3bb43e2b80 10 client.0 _ll_register_callbacks cb 0x561f49849760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:54.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3ba67f9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f4996b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3ba67f9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f4996b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58492/0 (socket says 172.21.15.40:58492) 2023-07-18T03:31:54.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3ba67f9700 1 -- 172.21.15.40:0/3976070586 learned_addr learned my addr 172.21.15.40:0/3976070586 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:54.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3ba57f7700 1 --2- 172.21.15.40:0/3976070586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995ae20 0x561f49964670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3ba5ff8700 1 --2- 172.21.15.40:0/3976070586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995a4d0 0x561f49964130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3ba67f9700 1 -- 172.21.15.40:0/3976070586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995a4d0 msgr2=0x561f49964130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:54.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3ba67f9700 1 --2- 172.21.15.40:0/3976070586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995a4d0 0x561f49964130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3ba67f9700 1 -- 172.21.15.40:0/3976070586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995ae20 msgr2=0x561f49964670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:54.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3ba67f9700 1 --2- 172.21.15.40:0/3976070586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995ae20 0x561f49964670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3ba67f9700 1 -- 172.21.15.40:0/3976070586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b98007e30 con 0x561f4995bc70 2023-07-18T03:31:54.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.281+0000 7f3ba57f7700 1 --2- 172.21.15.40:0/3976070586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995ae20 0x561f49964670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:54.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.282+0000 7f3ba67f9700 1 --2- 172.21.15.40:0/3976070586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f4996b6b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3b98000f80 tx=0x7f3b98012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115204d07a2c6930 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:54.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.282+0000 7f3b897fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:54.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.282+0000 7f3b897fa700 1 -- 172.21.15.40:0/3976070586 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b98007770 con 0x561f4995bc70 2023-07-18T03:31:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.282+0000 7f3b897fa700 1 -- 172.21.15.40:0/3976070586 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3b98004ce0 con 0x561f4995bc70 2023-07-18T03:31:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.282+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/3976070586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f498054f0 con 0x561f4995bc70 2023-07-18T03:31:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.282+0000 7f3b897fa700 1 -- 172.21.15.40:0/3976070586 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b98004ea0 con 0x561f4995bc70 2023-07-18T03:31:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.282+0000 7f3bb43e2b80 10 client.15678 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.282+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/3976070586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561f496c2c60 con 0x561f4995bc70 2023-07-18T03:31:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.282+0000 7f3b897fa700 1 -- 172.21.15.40:0/3976070586 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b9802c020 con 0x561f4995bc70 2023-07-18T03:31:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.282+0000 7f3b897fa700 1 -- 172.21.15.40:0/3976070586 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3b9801c070 con 0x561f4995bc70 2023-07-18T03:31:54.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.282+0000 7f3bb43e2b80 10 client.15678 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:54.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.282+0000 7f3bb43e2b80 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:54.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.282+0000 7f3bb43e2b80 20 client.15678 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:54.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.282+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/3976070586 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561f4996fe90 0x561f499722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.282+0000 7f3bb43e2b80 4 client.15678 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:54.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.282+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/3976070586 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f498a4eb0 con 0x561f4996fe90 2023-07-18T03:31:54.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.283+0000 7f3ba5ff8700 1 --2- 172.21.15.40:0/3976070586 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561f4996fe90 0x561f499722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.283+0000 7f3ba5ff8700 1 --2- 172.21.15.40:0/3976070586 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561f4996fe90 0x561f499722a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ff3fd74573bee6 server_cookie=7662a3b7324bfb39 in_seq=0 out_seq=0 2023-07-18T03:31:54.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.283+0000 7f3b897fa700 10 client.15678 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.284+0000 7f3b897fa700 1 -- 172.21.15.40:0/3976070586 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561f498a4eb0 con 0x561f4996fe90 2023-07-18T03:31:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.284+0000 7f3b897fa700 10 client.15678 handle_command_reply: tid=0 2023-07-18T03:31:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.284+0000 7f3bb43e2b80 1 client.15678 shutdown 2023-07-18T03:31:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.284+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/3976070586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 msgr2=0x561f4996b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.284+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/3976070586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f4996b6b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3b98000f80 tx=0x7f3b98012d00 comp rx=0 tx=0).stop 2023-07-18T03:31:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.284+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/3976070586 shutdown_connections 2023-07-18T03:31:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.284+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/3976070586 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561f4996fe90 0x561f499722a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.284+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/3976070586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f4996b6b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.284+0000 7f3b897fa700 0 client.15678 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.284+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/3976070586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995a4d0 0x561f49964130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.284+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/3976070586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995ae20 0x561f49964670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.284+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/3976070586 >> 172.21.15.40:0/3976070586 conn(0x561f4994fda0 msgr2=0x561f499548b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.284+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/3976070586 shutdown_connections 2023-07-18T03:31:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.284+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/3976070586 wait complete. 2023-07-18T03:31:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.284+0000 7f3bb43e2b80 20 client.15678 trim_cache size 0 max 16384 2023-07-18T03:31:54.294 INFO:tasks.workunit.client.0.smithi040.stderr:HTTP request sent, awaiting response... 200 OK 2023-07-18T03:31:54.295 INFO:tasks.workunit.client.0.smithi040.stderr:Length: 103355 (101K) [application/octet-stream] 2023-07-18T03:31:54.295 INFO:tasks.workunit.client.0.smithi040.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2023-07-18T03:31:54.295 INFO:tasks.workunit.client.0.smithi040.stderr: 2023-07-18T03:31:54.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.302+0000 7f3bb43e2b80 1 Processor -- start 2023-07-18T03:31:54.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.302+0000 7f3bb43e2b80 1 -- start start 2023-07-18T03:31:54.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3bb43e2b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995a4d0 0x561f4995a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3bb43e2b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995ae20 0x561f4996b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3bb43e2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995bc70 0x561f49962e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3bb43e2b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f49956d30 con 0x561f4995ae20 2023-07-18T03:31:54.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3bb43e2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f499633c0 con 0x561f4995bc70 2023-07-18T03:31:54.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3bb43e2b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f49956330 con 0x561f4995a4d0 2023-07-18T03:31:54.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3ba57f7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995ae20 0x561f4996b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3ba57f7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995ae20 0x561f4996b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58500/0 (socket says 172.21.15.40:58500) 2023-07-18T03:31:54.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3ba57f7700 1 -- 172.21.15.40:0/1917844688 learned_addr learned my addr 172.21.15.40:0/1917844688 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:54.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3ba67f9700 1 --2- 172.21.15.40:0/1917844688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995bc70 0x561f49962e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3ba5ff8700 1 --2- 172.21.15.40:0/1917844688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995a4d0 0x561f4995a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3ba57f7700 1 -- 172.21.15.40:0/1917844688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995a4d0 msgr2=0x561f4995a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:54.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3ba57f7700 1 --2- 172.21.15.40:0/1917844688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995a4d0 0x561f4995a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3ba57f7700 1 -- 172.21.15.40:0/1917844688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995bc70 msgr2=0x561f49962e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:54.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3ba57f7700 1 --2- 172.21.15.40:0/1917844688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995bc70 0x561f49962e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3ba57f7700 1 -- 172.21.15.40:0/1917844688 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f49700250 con 0x561f4995ae20 2023-07-18T03:31:54.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3ba67f9700 1 --2- 172.21.15.40:0/1917844688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995bc70 0x561f49962e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:31:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3ba57f7700 1 --2- 172.21.15.40:0/1917844688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995ae20 0x561f4996b670 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3b940074a0 tx=0x7f3b94007ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb374285e507a08e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3b90ff9700 1 -- 172.21.15.40:0/1917844688 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9409d940 con 0x561f4995ae20 2023-07-18T03:31:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3b90ff9700 1 -- 172.21.15.40:0/1917844688 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b940a1d60 con 0x561f4995ae20 2023-07-18T03:31:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.303+0000 7f3b90ff9700 1 -- 172.21.15.40:0/1917844688 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94099040 con 0x561f4995ae20 2023-07-18T03:31:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.304+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/1917844688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995ae20 msgr2=0x561f4996b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.304+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/1917844688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995ae20 0x561f4996b670 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3b940074a0 tx=0x7f3b94007ce0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.304+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/1917844688 shutdown_connections 2023-07-18T03:31:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.304+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/1917844688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995bc70 0x561f49962e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.304+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/1917844688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995a4d0 0x561f4995a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.304+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/1917844688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995ae20 0x561f4996b670 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.304+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/1917844688 >> 172.21.15.40:0/1917844688 conn(0x561f4994fda0 msgr2=0x561f49976c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.304+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/1917844688 shutdown_connections 2023-07-18T03:31:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.304+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/1917844688 wait complete. 2023-07-18T03:31:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.304+0000 7f3bb43e2b80 1 -- start start 2023-07-18T03:31:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3bb43e2b80 1 Processor -- start 2023-07-18T03:31:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3bb43e2b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995a4d0 0x561f4996afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3bb43e2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995ae20 0x561f4996b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3bb43e2b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f49967c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3bb43e2b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f49968330 con 0x561f4995bc70 2023-07-18T03:31:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3bb43e2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f499684a0 con 0x561f4995ae20 2023-07-18T03:31:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3bb43e2b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f49968660 con 0x561f4995a4d0 2023-07-18T03:31:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3bb43e2b80 10 client.0 _ll_register_callbacks cb 0x7f3b98008480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3ba67f9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f49967c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3ba67f9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f49967c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58508/0 (socket says 172.21.15.40:58508) 2023-07-18T03:31:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3ba67f9700 1 -- 172.21.15.40:0/1900656883 learned_addr learned my addr 172.21.15.40:0/1900656883 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3ba5ff8700 1 --2- 172.21.15.40:0/1900656883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995a4d0 0x561f4996afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3ba57f7700 1 --2- 172.21.15.40:0/1900656883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995ae20 0x561f4996b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3ba67f9700 1 -- 172.21.15.40:0/1900656883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995a4d0 msgr2=0x561f4996afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3ba67f9700 1 --2- 172.21.15.40:0/1900656883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995a4d0 0x561f4996afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3ba67f9700 1 -- 172.21.15.40:0/1900656883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995ae20 msgr2=0x561f4996b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3ba67f9700 1 --2- 172.21.15.40:0/1900656883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995ae20 0x561f4996b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3ba67f9700 1 -- 172.21.15.40:0/1900656883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b94084040 con 0x561f4995bc70 2023-07-18T03:31:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.305+0000 7f3ba57f7700 1 --2- 172.21.15.40:0/1900656883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995ae20 0x561f4996b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.306+0000 7f3ba67f9700 1 --2- 172.21.15.40:0/1900656883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f49967c50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3b98013040 tx=0x7f3b98026bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad328fc2f062d61 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:54.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.306+0000 7f3b897fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.306+0000 7f3b897fa700 1 -- 172.21.15.40:0/1900656883 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b98035070 con 0x561f4995bc70 2023-07-18T03:31:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.306+0000 7f3b897fa700 1 -- 172.21.15.40:0/1900656883 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3b9802d080 con 0x561f4995bc70 2023-07-18T03:31:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.306+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/1900656883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b98010040 con 0x561f4995bc70 2023-07-18T03:31:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.306+0000 7f3b897fa700 1 -- 172.21.15.40:0/1900656883 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b98026e80 con 0x561f4995bc70 2023-07-18T03:31:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.306+0000 7f3bb43e2b80 10 client.15684 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.306+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/1900656883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561f496d82b0 con 0x561f4995bc70 2023-07-18T03:31:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.306+0000 7f3b897fa700 1 -- 172.21.15.40:0/1900656883 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b98040020 con 0x561f4995bc70 2023-07-18T03:31:54.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.306+0000 7f3b897fa700 1 -- 172.21.15.40:0/1900656883 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3b98038020 con 0x561f4995bc70 2023-07-18T03:31:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.306+0000 7f3bb43e2b80 10 client.15684 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.306+0000 7f3bb43e2b80 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.306+0000 7f3bb43e2b80 20 client.15684 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.306+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/1900656883 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561f499c4ea0 0x561f499c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.306+0000 7f3bb43e2b80 4 client.15684 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:31:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.306+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/1900656883 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561f49867050 con 0x561f499c4ea0 2023-07-18T03:31:54.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.307+0000 7f3ba5ff8700 1 --2- 172.21.15.40:0/1900656883 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561f499c4ea0 0x561f499c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.307+0000 7f3ba5ff8700 1 --2- 172.21.15.40:0/1900656883 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561f499c4ea0 0x561f499c72b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33fc112c83a774fc server_cookie=43079c5745350180 in_seq=0 out_seq=0 2023-07-18T03:31:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.307+0000 7f3b897fa700 10 client.15684 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.307+0000 7f3b897fa700 1 -- 172.21.15.40:0/1900656883 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561f49867050 con 0x561f499c4ea0 2023-07-18T03:31:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.307+0000 7f3b897fa700 10 client.15684 handle_command_reply: tid=0 2023-07-18T03:31:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.307+0000 7f3bb43e2b80 1 client.15684 shutdown 2023-07-18T03:31:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.307+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/1900656883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 msgr2=0x561f49967c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.307+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/1900656883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f49967c50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3b98013040 tx=0x7f3b98026bb0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.307+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/1900656883 shutdown_connections 2023-07-18T03:31:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.307+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/1900656883 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561f499c4ea0 0x561f499c72b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.307+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/1900656883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561f4995bc70 0x561f49967c50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.307+0000 7f3b897fa700 0 client.15684 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.307+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/1900656883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f4995a4d0 0x561f4996afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.307+0000 7f3bb43e2b80 1 --2- 172.21.15.40:0/1900656883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f4995ae20 0x561f4996b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.307+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/1900656883 >> 172.21.15.40:0/1900656883 conn(0x561f4994fda0 msgr2=0x561f49962200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:54.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.308+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/1900656883 shutdown_connections 2023-07-18T03:31:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.308+0000 7f3bb43e2b80 1 -- 172.21.15.40:0/1900656883 wait complete. 2023-07-18T03:31:54.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.308+0000 7f3bb43e2b80 20 client.15684 trim_cache size 0 max 16384 2023-07-18T03:31:54.315 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:31:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "no active scrubs running", 2023-07-18T03:31:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": {} 2023-07-18T03:31:54.316 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:31:54.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.308+0000 7f3ba825c700 1 -- 172.21.15.40:0/1644678332 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3b8c0a1150 msgr2=0x7f3b8c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.308+0000 7f3ba825c700 1 --2- 172.21.15.40:0/1644678332 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3b8c0a1150 0x7f3b8c0a3600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00b5c0 tx=0x7f3b9c005fb0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.308+0000 7f3ba825c700 1 -- 172.21.15.40:0/1644678332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ba0100d00 msgr2=0x7f3ba0072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.308+0000 7f3ba825c700 1 --2- 172.21.15.40:0/1644678332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ba0100d00 0x7f3ba0072900 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400c920 tx=0x7f3b9400cc30 comp rx=0 tx=0).stop 2023-07-18T03:31:54.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.308+0000 7f3ba825c700 1 -- 172.21.15.40:0/1644678332 shutdown_connections 2023-07-18T03:31:54.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.308+0000 7f3ba825c700 1 --2- 172.21.15.40:0/1644678332 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3b8c0a1150 0x7f3b8c0a3600 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.308+0000 7f3ba825c700 1 --2- 172.21.15.40:0/1644678332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ba0101690 0x7f3ba0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.308+0000 7f3ba825c700 1 --2- 172.21.15.40:0/1644678332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ba00ffb10 0x7f3ba00742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.308+0000 7f3ba825c700 1 --2- 172.21.15.40:0/1644678332 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ba0100d00 0x7f3ba0072900 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.308+0000 7f3ba825c700 1 -- 172.21.15.40:0/1644678332 >> 172.21.15.40:0/1644678332 conn(0x7f3ba00fb110 msgr2=0x7f3ba00fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:54.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.308+0000 7f3ba825c700 1 -- 172.21.15.40:0/1644678332 shutdown_connections 2023-07-18T03:31:54.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.308+0000 7f3ba825c700 1 -- 172.21.15.40:0/1644678332 wait complete. 2023-07-18T03:31:54.319 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:31:54.319 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:31:54.362 INFO:tasks.workunit.client.0.smithi040.stderr: 0K .......... .......... .......... .......... .......... 49% 674K 0s 2023-07-18T03:31:54.398 INFO:tasks.workunit.client.0.smithi040.stderr: 50K .......... .......... .......... .......... .......... 99% 1.30M 0s 2023-07-18T03:31:54.399 INFO:tasks.workunit.client.0.smithi040.stderr: 100K 100% 1779G=0.1s 2023-07-18T03:31:54.399 INFO:tasks.workunit.client.0.smithi040.stderr: 2023-07-18T03:31:54.399 INFO:tasks.workunit.client.0.smithi040.stderr:2023-07-18 03:31:54 (903 KB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2023-07-18T03:31:54.399 INFO:tasks.workunit.client.0.smithi040.stderr: 2023-07-18T03:31:54.400 INFO:tasks.workunit.client.0.smithi040.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2023-07-18T03:31:54.426 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/ 2023-07-18T03:31:54.427 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/m4/ 2023-07-18T03:31:54.429 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/m4/acx_pthread.m4 2023-07-18T03:31:54.431 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/man/ 2023-07-18T03:31:54.433 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/man/blogbench.8 2023-07-18T03:31:54.434 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/man/Makefile.am 2023-07-18T03:31:54.435 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/man/Makefile.in 2023-07-18T03:31:54.435 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/ 2023-07-18T03:31:54.438 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/mysnprintf.c 2023-07-18T03:31:54.441 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/mysnprintf.h 2023-07-18T03:31:54.441 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/blogbench.c 2023-07-18T03:31:54.442 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/blogbench.h 2023-07-18T03:31:54.442 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/fakesnprintf.c 2023-07-18T03:31:54.442 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/fakesnprintf.h 2023-07-18T03:31:54.442 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/blogbench_p.h 2023-07-18T03:31:54.442 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/bsd-getopt_long.c 2023-07-18T03:31:54.442 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/bsd-getopt_long.h 2023-07-18T03:31:54.443 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/Makefile.am 2023-07-18T03:31:54.443 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/Makefile.in 2023-07-18T03:31:54.443 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/process.c 2023-07-18T03:31:54.443 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/globals.h 2023-07-18T03:31:54.443 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/commenter.c 2023-07-18T03:31:54.443 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/rewriter.c 2023-07-18T03:31:54.443 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/reader.c 2023-07-18T03:31:54.443 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/writer.c 2023-07-18T03:31:54.444 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/src/helpers.c 2023-07-18T03:31:54.444 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/NEWS 2023-07-18T03:31:54.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.449+0000 7fd86858d700 1 -- 172.21.15.40:0/407556175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8600ffb00 msgr2=0x7fd8600fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.449+0000 7fd86858d700 1 --2- 172.21.15.40:0/407556175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8600ffb00 0x7fd8600fff10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009990 tx=0x7fd85c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.449+0000 7fd86858d700 1 -- 172.21.15.40:0/407556175 shutdown_connections 2023-07-18T03:31:54.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.449+0000 7fd86858d700 1 --2- 172.21.15.40:0/407556175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd860101680 0x7fd860107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.449+0000 7fd86858d700 1 --2- 172.21.15.40:0/407556175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8600ffb00 0x7fd8600fff10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.449+0000 7fd86858d700 1 --2- 172.21.15.40:0/407556175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860100cf0 0x7fd860101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.449+0000 7fd86858d700 1 -- 172.21.15.40:0/407556175 >> 172.21.15.40:0/407556175 conn(0x7fd8600fb0e0 msgr2=0x7fd8600fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.450+0000 7fd86858d700 1 -- 172.21.15.40:0/407556175 shutdown_connections 2023-07-18T03:31:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.450+0000 7fd86858d700 1 -- 172.21.15.40:0/407556175 wait complete. 2023-07-18T03:31:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.450+0000 7fd86858d700 1 Processor -- start 2023-07-18T03:31:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.450+0000 7fd86858d700 1 -- start start 2023-07-18T03:31:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.450+0000 7fd86858d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8600ffb00 0x7fd8601957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.450+0000 7fd86858d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd860100cf0 0x7fd860195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.450+0000 7fd86858d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860101680 0x7fd86019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.450+0000 7fd86858d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86019d340 con 0x7fd860100cf0 2023-07-18T03:31:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.450+0000 7fd86858d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86019d480 con 0x7fd8600ffb00 2023-07-18T03:31:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.450+0000 7fd86858d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86019d610 con 0x7fd860101680 2023-07-18T03:31:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.450+0000 7fd865b28700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd860100cf0 0x7fd860195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.450+0000 7fd865b28700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd860100cf0 0x7fd860195d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58520/0 (socket says 172.21.15.40:58520) 2023-07-18T03:31:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.451+0000 7fd865b28700 1 -- 172.21.15.40:0/885278023 learned_addr learned my addr 172.21.15.40:0/885278023 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.451+0000 7fd866329700 1 --2- 172.21.15.40:0/885278023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8600ffb00 0x7fd8601957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.451+0000 7fd866b2a700 1 --2- 172.21.15.40:0/885278023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860101680 0x7fd86019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.451+0000 7fd865b28700 1 -- 172.21.15.40:0/885278023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860101680 msgr2=0x7fd86019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.451+0000 7fd865b28700 1 --2- 172.21.15.40:0/885278023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860101680 0x7fd86019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.451+0000 7fd865b28700 1 -- 172.21.15.40:0/885278023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8600ffb00 msgr2=0x7fd8601957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.451+0000 7fd865b28700 1 --2- 172.21.15.40:0/885278023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8600ffb00 0x7fd8601957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.451+0000 7fd865b28700 1 -- 172.21.15.40:0/885278023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c0095f0 con 0x7fd860100cf0 2023-07-18T03:31:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.451+0000 7fd865b28700 1 --2- 172.21.15.40:0/885278023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd860100cf0 0x7fd860195d00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd85000d950 tx=0x7fd85000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.451+0000 7fd8577fe700 1 -- 172.21.15.40:0/885278023 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000f890 con 0x7fd860100cf0 2023-07-18T03:31:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.451+0000 7fd8577fe700 1 -- 172.21.15.40:0/885278023 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd850010460 con 0x7fd860100cf0 2023-07-18T03:31:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.451+0000 7fd86858d700 1 -- 172.21.15.40:0/885278023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86019d8f0 con 0x7fd860100cf0 2023-07-18T03:31:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.451+0000 7fd8577fe700 1 -- 172.21.15.40:0/885278023 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000e610 con 0x7fd860100cf0 2023-07-18T03:31:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.451+0000 7fd86858d700 1 -- 172.21.15.40:0/885278023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd86019de50 con 0x7fd860100cf0 2023-07-18T03:31:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.452+0000 7fd86858d700 1 -- 172.21.15.40:0/885278023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd860107060 con 0x7fd860100cf0 2023-07-18T03:31:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.453+0000 7fd8577fe700 1 -- 172.21.15.40:0/885278023 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd8500108d0 con 0x7fd860100cf0 2023-07-18T03:31:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.454+0000 7fd8577fe700 1 --2- 172.21.15.40:0/885278023 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd84c0a1220 0x7fd84c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.454+0000 7fd8577fe700 1 -- 172.21.15.40:0/885278023 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd8500cc6a0 con 0x7fd860100cf0 2023-07-18T03:31:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.454+0000 7fd866329700 1 --2- 172.21.15.40:0/885278023 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd84c0a1220 0x7fd84c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.454+0000 7fd866329700 1 --2- 172.21.15.40:0/885278023 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd84c0a1220 0x7fd84c0a36d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd85c006010 tx=0x7fd85c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:54.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.458+0000 7fd8577fe700 1 -- 172.21.15.40:0/885278023 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd850095dd0 con 0x7fd860100cf0 2023-07-18T03:31:54.458 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/compile 2023-07-18T03:31:54.458 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/depcomp 2023-07-18T03:31:54.458 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/aclocal.m4 2023-07-18T03:31:54.458 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/README 2023-07-18T03:31:54.458 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/configure 2023-07-18T03:31:54.462 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/configure.ac 2023-07-18T03:31:54.463 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/config.guess 2023-07-18T03:31:54.463 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/install-sh 2023-07-18T03:31:54.463 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/config.sub 2023-07-18T03:31:54.464 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/missing 2023-07-18T03:31:54.464 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/mkinstalldirs 2023-07-18T03:31:54.465 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/Makefile.am 2023-07-18T03:31:54.465 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/Makefile.in 2023-07-18T03:31:54.465 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/config.h.in 2023-07-18T03:31:54.465 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/AUTHORS 2023-07-18T03:31:54.465 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/INSTALL 2023-07-18T03:31:54.465 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/ChangeLog 2023-07-18T03:31:54.465 INFO:tasks.workunit.client.0.smithi040.stdout:blogbench-1.0/COPYING 2023-07-18T03:31:54.466 INFO:tasks.workunit.client.0.smithi040.stderr:+ cd blogbench-1.0/ 2023-07-18T03:31:54.466 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'making blogbench' 2023-07-18T03:31:54.466 INFO:tasks.workunit.client.0.smithi040.stdout:making blogbench 2023-07-18T03:31:54.467 INFO:tasks.workunit.client.0.smithi040.stderr:+ ./configure 2023-07-18T03:31:54.572 INFO:tasks.workunit.client.0.smithi040.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-07-18T03:31:54.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.704+0000 7fd86858d700 1 -- 172.21.15.40:0/885278023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd86002ca60 con 0x7fd860100cf0 2023-07-18T03:31:54.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.705+0000 7fd8577fe700 1 -- 172.21.15.40:0/885278023 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10888 (secure 0 0 0) 0x7fd850020090 con 0x7fd860100cf0 2023-07-18T03:31:54.702 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:31:54.702 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:29:26.718789+0000","modified":"2023-07-18T03:29:34.650353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24523,"mds_2":24482,"mds_3":24452,"mds_4":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14601":{"gid":14601,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1818167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1818167554},{"type":"v1","addr":"172.21.15.40:6839","nonce":1818167554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14613":{"gid":14613,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/3953074960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":3953074960},{"type":"v1","addr":"172.21.15.40:6841","nonce":3953074960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3509655945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3509655945},{"type":"v1","addr":"172.21.15.134:6837","nonce":3509655945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/3007039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3007039043},{"type":"v1","addr":"172.21.15.64:6839","nonce":3007039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:31:54.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.707+0000 7fd86858d700 1 -- 172.21.15.40:0/885278023 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd84c0a1220 msgr2=0x7fd84c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.707+0000 7fd86858d700 1 --2- 172.21.15.40:0/885278023 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd84c0a1220 0x7fd84c0a36d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd85c006010 tx=0x7fd85c005960 comp rx=0 tx=0).stop 2023-07-18T03:31:54.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.707+0000 7fd86858d700 1 -- 172.21.15.40:0/885278023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd860100cf0 msgr2=0x7fd860195d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.707+0000 7fd86858d700 1 --2- 172.21.15.40:0/885278023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd860100cf0 0x7fd860195d00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd85000d950 tx=0x7fd85000dc60 comp rx=0 tx=0).stop 2023-07-18T03:31:54.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.707+0000 7fd86858d700 1 -- 172.21.15.40:0/885278023 shutdown_connections 2023-07-18T03:31:54.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.707+0000 7fd86858d700 1 --2- 172.21.15.40:0/885278023 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd84c0a1220 0x7fd84c0a36d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.707+0000 7fd86858d700 1 --2- 172.21.15.40:0/885278023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd860101680 0x7fd86019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.707+0000 7fd86858d700 1 --2- 172.21.15.40:0/885278023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd8600ffb00 0x7fd8601957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.707+0000 7fd86858d700 1 --2- 172.21.15.40:0/885278023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd860100cf0 0x7fd860195d00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.707+0000 7fd86858d700 1 -- 172.21.15.40:0/885278023 >> 172.21.15.40:0/885278023 conn(0x7fd8600fb0e0 msgr2=0x7fd8601041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:54.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.707+0000 7fd86858d700 1 -- 172.21.15.40:0/885278023 shutdown_connections 2023-07-18T03:31:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.707+0000 7fd86858d700 1 -- 172.21.15.40:0/885278023 wait complete. 2023-07-18T03:31:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 27 2023-07-18T03:31:54.717 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-18T03:31:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.888+0000 7fb3a98c4700 1 -- 172.21.15.40:0/2519783011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a4105800 msgr2=0x7fb3a4107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.888+0000 7fb3a98c4700 1 --2- 172.21.15.40:0/2519783011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a4105800 0x7fb3a4107c60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb39c00b0a0 tx=0x7fb39c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.888+0000 7fb3a98c4700 1 -- 172.21.15.40:0/2519783011 shutdown_connections 2023-07-18T03:31:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.888+0000 7fb3a98c4700 1 --2- 172.21.15.40:0/2519783011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a4105800 0x7fb3a4107c60 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.888+0000 7fb3a98c4700 1 --2- 172.21.15.40:0/2519783011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3a41005c0 0x7fb3a41029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.888+0000 7fb3a98c4700 1 --2- 172.21.15.40:0/2519783011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3a4102ee0 0x7fb3a41052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.888+0000 7fb3a98c4700 1 -- 172.21.15.40:0/2519783011 >> 172.21.15.40:0/2519783011 conn(0x7fb3a40fa0a0 msgr2=0x7fb3a40fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:54.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.888+0000 7fb3a98c4700 1 -- 172.21.15.40:0/2519783011 shutdown_connections 2023-07-18T03:31:54.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.889+0000 7fb3a98c4700 1 -- 172.21.15.40:0/2519783011 wait complete. 2023-07-18T03:31:54.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.889+0000 7fb3a98c4700 1 Processor -- start 2023-07-18T03:31:54.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.889+0000 7fb3a98c4700 1 -- start start 2023-07-18T03:31:54.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.889+0000 7fb3a98c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3a41005c0 0x7fb3a40fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.889+0000 7fb3a98c4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a4102ee0 0x7fb3a40fe570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.889+0000 7fb3a98c4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3a4105800 0x7fb3a40feab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.889+0000 7fb3a98c4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a40ff250 con 0x7fb3a4102ee0 2023-07-18T03:31:54.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.889+0000 7fb3a98c4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a40ff3c0 con 0x7fb3a41005c0 2023-07-18T03:31:54.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.889+0000 7fb3a98c4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a419f900 con 0x7fb3a4105800 2023-07-18T03:31:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.890+0000 7fb3a27fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a4102ee0 0x7fb3a40fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.890+0000 7fb3a27fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a4102ee0 0x7fb3a40fe570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58534/0 (socket says 172.21.15.40:58534) 2023-07-18T03:31:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.890+0000 7fb3a27fc700 1 -- 172.21.15.40:0/2812669821 learned_addr learned my addr 172.21.15.40:0/2812669821 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.890+0000 7fb3a2ffd700 1 --2- 172.21.15.40:0/2812669821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3a41005c0 0x7fb3a40fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.890+0000 7fb3a37fe700 1 --2- 172.21.15.40:0/2812669821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3a4105800 0x7fb3a40feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.890+0000 7fb3a27fc700 1 -- 172.21.15.40:0/2812669821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3a4105800 msgr2=0x7fb3a40feab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.890+0000 7fb3a27fc700 1 --2- 172.21.15.40:0/2812669821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3a4105800 0x7fb3a40feab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.890+0000 7fb3a27fc700 1 -- 172.21.15.40:0/2812669821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3a41005c0 msgr2=0x7fb3a40fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.890+0000 7fb3a27fc700 1 --2- 172.21.15.40:0/2812669821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3a41005c0 0x7fb3a40fff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.890+0000 7fb3a27fc700 1 -- 172.21.15.40:0/2812669821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c009d00 con 0x7fb3a4102ee0 2023-07-18T03:31:54.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.890+0000 7fb3a2ffd700 1 --2- 172.21.15.40:0/2812669821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3a41005c0 0x7fb3a40fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:54.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.890+0000 7fb3a27fc700 1 --2- 172.21.15.40:0/2812669821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a4102ee0 0x7fb3a40fe570 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb39800e9a0 tx=0x7fb39800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:54.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.890+0000 7fb3a88c2700 1 -- 172.21.15.40:0/2812669821 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398009790 con 0x7fb3a4102ee0 2023-07-18T03:31:54.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.890+0000 7fb3a88c2700 1 -- 172.21.15.40:0/2812669821 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb39800cbd0 con 0x7fb3a4102ee0 2023-07-18T03:31:54.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.890+0000 7fb3a88c2700 1 -- 172.21.15.40:0/2812669821 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398018680 con 0x7fb3a4102ee0 2023-07-18T03:31:54.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.890+0000 7fb3a98c4700 1 -- 172.21.15.40:0/2812669821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a419fbe0 con 0x7fb3a4102ee0 2023-07-18T03:31:54.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.890+0000 7fb3a98c4700 1 -- 172.21.15.40:0/2812669821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a41a00a0 con 0x7fb3a4102ee0 2023-07-18T03:31:54.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.892+0000 7fb3a88c2700 1 -- 172.21.15.40:0/2812669821 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb39800cd40 con 0x7fb3a4102ee0 2023-07-18T03:31:54.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.893+0000 7fb3a88c2700 1 --2- 172.21.15.40:0/2812669821 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb38c0a11d0 0x7fb38c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.893+0000 7fb3a2ffd700 1 --2- 172.21.15.40:0/2812669821 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb38c0a11d0 0x7fb38c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.893+0000 7fb3a88c2700 1 -- 172.21.15.40:0/2812669821 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb398014070 con 0x7fb3a4102ee0 2023-07-18T03:31:54.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3a2ffd700 1 --2- 172.21.15.40:0/2812669821 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb38c0a11d0 0x7fb38c0a3680 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb394009cb0 tx=0x7fb394009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:54.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3b5a4ab80 1 Processor -- start 2023-07-18T03:31:54.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3b5a4ab80 1 -- start start 2023-07-18T03:31:54.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3b5a4ab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3b5a4ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f07e20 0x560a54f086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3b5a4ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f08c70 0x560a54f090e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3b5a4ab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a54e0ccb0 con 0x560a54f074d0 2023-07-18T03:31:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3b5a4ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a54e5fa40 con 0x560a54f08c70 2023-07-18T03:31:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3b5a4ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a54e58ed0 con 0x560a54f07e20 2023-07-18T03:31:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58540/0 (socket says 172.21.15.40:58540) 2023-07-18T03:31:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3a2ffd700 1 -- 172.21.15.40:0/1549272341 learned_addr learned my addr 172.21.15.40:0/1549272341 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3a27fc700 1 --2- 172.21.15.40:0/1549272341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f07e20 0x560a54f086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3a37fe700 1 --2- 172.21.15.40:0/1549272341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f08c70 0x560a54f090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3a2ffd700 1 -- 172.21.15.40:0/1549272341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f07e20 msgr2=0x560a54f086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3a2ffd700 1 --2- 172.21.15.40:0/1549272341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f07e20 0x560a54f086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3a2ffd700 1 -- 172.21.15.40:0/1549272341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f08c70 msgr2=0x560a54f090e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3a2ffd700 1 --2- 172.21.15.40:0/1549272341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f08c70 0x560a54f090e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3a2ffd700 1 -- 172.21.15.40:0/1549272341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a54c86de0 con 0x560a54f074d0 2023-07-18T03:31:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.894+0000 7fb3a27fc700 1 --2- 172.21.15.40:0/1549272341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f07e20 0x560a54f086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:31:54.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.895+0000 7fb3a2ffd700 1 --2- 172.21.15.40:0/1549272341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f078e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb394011fb0 tx=0x7fb394014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2b8a86ac7bee3b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:54.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.895+0000 7fb389ffb700 1 -- 172.21.15.40:0/1549272341 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394003880 con 0x560a54f074d0 2023-07-18T03:31:54.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.895+0000 7fb389ffb700 1 -- 172.21.15.40:0/1549272341 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb394004570 con 0x560a54f074d0 2023-07-18T03:31:54.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.895+0000 7fb389ffb700 1 -- 172.21.15.40:0/1549272341 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394012d30 con 0x560a54f074d0 2023-07-18T03:31:54.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.895+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/1549272341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 msgr2=0x560a54f078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.895+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/1549272341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f078e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb394011fb0 tx=0x7fb394014000 comp rx=0 tx=0).stop 2023-07-18T03:31:54.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.895+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/1549272341 shutdown_connections 2023-07-18T03:31:54.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.895+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/1549272341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f08c70 0x560a54f090e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.895+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/1549272341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f078e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.896+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/1549272341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f07e20 0x560a54f086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.896+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/1549272341 >> 172.21.15.40:0/1549272341 conn(0x560a54efcda0 msgr2=0x560a54f021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:54.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.896+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/1549272341 shutdown_connections 2023-07-18T03:31:54.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.896+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/1549272341 wait complete. 2023-07-18T03:31:54.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.896+0000 7fb3b5a4ab80 1 -- start start 2023-07-18T03:31:54.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.896+0000 7fb3b5a4ab80 1 Processor -- start 2023-07-18T03:31:54.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.896+0000 7fb3b5a4ab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f11220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.896+0000 7fb3b5a4ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f07e20 0x560a54f11760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.896+0000 7fb3b5a4ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f08c70 0x560a54f187a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.896+0000 7fb3b5a4ab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a54df8ac0 con 0x560a54f074d0 2023-07-18T03:31:54.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.896+0000 7fb3b5a4ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a54df87c0 con 0x560a54f07e20 2023-07-18T03:31:54.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.896+0000 7fb3b5a4ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a54cdc920 con 0x560a54f08c70 2023-07-18T03:31:54.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.896+0000 7fb3b5a4ab80 10 client.0 _ll_register_callbacks cb 0x560a54df6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:54.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f11220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f11220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58546/0 (socket says 172.21.15.40:58546) 2023-07-18T03:31:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb3a2ffd700 1 -- 172.21.15.40:0/3394928886 learned_addr learned my addr 172.21.15.40:0/3394928886 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb3a37fe700 1 --2- 172.21.15.40:0/3394928886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f08c70 0x560a54f187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb3a27fc700 1 --2- 172.21.15.40:0/3394928886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f07e20 0x560a54f11760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb3a2ffd700 1 -- 172.21.15.40:0/3394928886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f08c70 msgr2=0x560a54f187a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb3a2ffd700 1 --2- 172.21.15.40:0/3394928886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f08c70 0x560a54f187a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb3a2ffd700 1 -- 172.21.15.40:0/3394928886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f07e20 msgr2=0x560a54f11760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb3a2ffd700 1 --2- 172.21.15.40:0/3394928886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f07e20 0x560a54f11760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb3a2ffd700 1 -- 172.21.15.40:0/3394928886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb394011c60 con 0x560a54f074d0 2023-07-18T03:31:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb3a37fe700 1 --2- 172.21.15.40:0/3394928886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f08c70 0x560a54f187a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb3a27fc700 1 --2- 172.21.15.40:0/3394928886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f07e20 0x560a54f11760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:31:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb3a2ffd700 1 --2- 172.21.15.40:0/3394928886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f11220 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb394009b70 tx=0x7fb394012d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327434900c7d98ad server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb382ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb382ffd700 1 -- 172.21.15.40:0/3394928886 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394003bd0 con 0x560a54f074d0 2023-07-18T03:31:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb382ffd700 1 -- 172.21.15.40:0/3394928886 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb394018070 con 0x560a54f074d0 2023-07-18T03:31:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/3394928886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a54db24f0 con 0x560a54f074d0 2023-07-18T03:31:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb382ffd700 1 -- 172.21.15.40:0/3394928886 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb394003d30 con 0x560a54f074d0 2023-07-18T03:31:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb3b5a4ab80 10 client.15702 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.897+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/3394928886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560a54c6fc60 con 0x560a54f074d0 2023-07-18T03:31:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.898+0000 7fb382ffd700 1 -- 172.21.15.40:0/3394928886 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a54db24f0 con 0x560a54f074d0 2023-07-18T03:31:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.898+0000 7fb382ffd700 1 -- 172.21.15.40:0/3394928886 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb39401e040 con 0x560a54f074d0 2023-07-18T03:31:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.898+0000 7fb3b5a4ab80 10 client.15702 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:54.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.898+0000 7fb3b5a4ab80 10 client.15702 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2023-07-18T03:31:54.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.898+0000 7fb3b5a4ab80 20 client.15702 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:54.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.898+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/3394928886 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x560a54f1cdd0 0x560a54f1f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.898+0000 7fb3b5a4ab80 4 client.15702 mds_command: new command op to 24442 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:54.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.898+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/3394928886 --> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a54e51eb0 con 0x560a54f1cdd0 2023-07-18T03:31:54.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.898+0000 7fb3a27fc700 1 --2- 172.21.15.40:0/3394928886 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x560a54f1cdd0 0x560a54f1f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.899+0000 7fb3a27fc700 1 --2- 172.21.15.40:0/3394928886 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x560a54f1cdd0 0x560a54f1f1e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c082ad0f84584af0 server_cookie=81ad6e863bb2d54b in_seq=0 out_seq=0 2023-07-18T03:31:54.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.899+0000 7fb382ffd700 10 client.15702 ms_handle_connect on v2:172.21.15.134:6832/2345264306 2023-07-18T03:31:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.899+0000 7fb382ffd700 1 -- 172.21.15.40:0/3394928886 <== mds.4 v2:172.21.15.134:6832/2345264306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560a54e51eb0 con 0x560a54f1cdd0 2023-07-18T03:31:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.899+0000 7fb382ffd700 10 client.15702 handle_command_reply: tid=0 2023-07-18T03:31:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.899+0000 7fb3b5a4ab80 1 client.15702 shutdown 2023-07-18T03:31:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.899+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/3394928886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 msgr2=0x560a54f11220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.899+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/3394928886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f11220 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb394009b70 tx=0x7fb394012d50 comp rx=0 tx=0).stop 2023-07-18T03:31:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.899+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/3394928886 shutdown_connections 2023-07-18T03:31:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.899+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/3394928886 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x560a54f1cdd0 0x560a54f1f1e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.899+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/3394928886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f08c70 0x560a54f187a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.899+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/3394928886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f11220 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.899+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/3394928886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f07e20 0x560a54f11760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.899+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/3394928886 >> 172.21.15.40:0/3394928886 conn(0x560a54efcda0 msgr2=0x560a54f018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:54.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.899+0000 7fb382ffd700 0 client.15702 ms_handle_reset on v2:172.21.15.134:6832/2345264306 2023-07-18T03:31:54.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.900+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/3394928886 shutdown_connections 2023-07-18T03:31:54.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.900+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/3394928886 wait complete. 2023-07-18T03:31:54.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.900+0000 7fb3b5a4ab80 20 client.15702 trim_cache size 0 max 16384 2023-07-18T03:31:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.912+0000 7fb3b5a4ab80 1 Processor -- start 2023-07-18T03:31:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.912+0000 7fb3b5a4ab80 1 -- start start 2023-07-18T03:31:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb3b5a4ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f074d0 0x560a54f108d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb3b5a4ab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f07e20 0x560a54f18760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb3b5a4ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f08c70 0x560a54f0fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb3b5a4ab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a54f106d0 con 0x560a54f07e20 2023-07-18T03:31:54.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb3b5a4ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a54f1cbc0 con 0x560a54f074d0 2023-07-18T03:31:54.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb3b5a4ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a54f032e0 con 0x560a54f08c70 2023-07-18T03:31:54.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb3a27fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f07e20 0x560a54f18760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb3a27fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f07e20 0x560a54f18760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58556/0 (socket says 172.21.15.40:58556) 2023-07-18T03:31:54.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb3a27fc700 1 -- 172.21.15.40:0/3105299965 learned_addr learned my addr 172.21.15.40:0/3105299965 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:54.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb3a2ffd700 1 --2- 172.21.15.40:0/3105299965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f074d0 0x560a54f108d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb3a37fe700 1 --2- 172.21.15.40:0/3105299965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f08c70 0x560a54f0fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb3a27fc700 1 -- 172.21.15.40:0/3105299965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f08c70 msgr2=0x560a54f0fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:54.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb3a27fc700 1 --2- 172.21.15.40:0/3105299965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f08c70 0x560a54f0fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb3a27fc700 1 -- 172.21.15.40:0/3105299965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f074d0 msgr2=0x560a54f108d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:54.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb3a27fc700 1 --2- 172.21.15.40:0/3105299965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f074d0 0x560a54f108d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb3a27fc700 1 -- 172.21.15.40:0/3105299965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a54cad250 con 0x560a54f07e20 2023-07-18T03:31:54.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb3a27fc700 1 --2- 172.21.15.40:0/3105299965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f07e20 0x560a54f18760 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb398096530 tx=0x7fb398096440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41c92070efe808d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:54.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.913+0000 7fb38a7fc700 1 -- 172.21.15.40:0/3105299965 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398098170 con 0x560a54f07e20 2023-07-18T03:31:54.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.914+0000 7fb38a7fc700 1 -- 172.21.15.40:0/3105299965 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3980ca830 con 0x560a54f07e20 2023-07-18T03:31:54.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.914+0000 7fb38a7fc700 1 -- 172.21.15.40:0/3105299965 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398099780 con 0x560a54f07e20 2023-07-18T03:31:54.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.914+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/3105299965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f07e20 msgr2=0x560a54f18760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.914+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/3105299965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f07e20 0x560a54f18760 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb398096530 tx=0x7fb398096440 comp rx=0 tx=0).stop 2023-07-18T03:31:54.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.914+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/3105299965 shutdown_connections 2023-07-18T03:31:54.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.914+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/3105299965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f08c70 0x560a54f0fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.914+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/3105299965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f074d0 0x560a54f108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.914+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/3105299965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f07e20 0x560a54f18760 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.914+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/3105299965 >> 172.21.15.40:0/3105299965 conn(0x560a54efcda0 msgr2=0x560a54f23b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:54.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.914+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/3105299965 shutdown_connections 2023-07-18T03:31:54.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.914+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/3105299965 wait complete. 2023-07-18T03:31:54.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.914+0000 7fb3b5a4ab80 1 -- start start 2023-07-18T03:31:54.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3b5a4ab80 1 Processor -- start 2023-07-18T03:31:54.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3b5a4ab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f142c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3b5a4ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f07e20 0x560a54f14800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3b5a4ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f08c70 0x560a54f17820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3b5a4ab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a54f17d60 con 0x560a54f074d0 2023-07-18T03:31:54.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3b5a4ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a54f17ed0 con 0x560a54f07e20 2023-07-18T03:31:54.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3b5a4ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a54f18090 con 0x560a54f08c70 2023-07-18T03:31:54.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3b5a4ab80 10 client.0 _ll_register_callbacks cb 0x7fb394023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:54.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f142c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3a2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f142c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58560/0 (socket says 172.21.15.40:58560) 2023-07-18T03:31:54.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3a2ffd700 1 -- 172.21.15.40:0/2120803082 learned_addr learned my addr 172.21.15.40:0/2120803082 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:54.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3a27fc700 1 --2- 172.21.15.40:0/2120803082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f07e20 0x560a54f14800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3a37fe700 1 --2- 172.21.15.40:0/2120803082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f08c70 0x560a54f17820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3a2ffd700 1 -- 172.21.15.40:0/2120803082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f08c70 msgr2=0x560a54f17820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:54.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3a2ffd700 1 --2- 172.21.15.40:0/2120803082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f08c70 0x560a54f17820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3a2ffd700 1 -- 172.21.15.40:0/2120803082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f07e20 msgr2=0x560a54f14800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:54.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3a2ffd700 1 --2- 172.21.15.40:0/2120803082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f07e20 0x560a54f14800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3a2ffd700 1 -- 172.21.15.40:0/2120803082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a41a00a0 con 0x560a54f074d0 2023-07-18T03:31:54.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb3a2ffd700 1 --2- 172.21.15.40:0/2120803082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f142c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb394011a50 tx=0x7fb394005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f94f80e1c0481f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:54.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.915+0000 7fb382ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:54.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.916+0000 7fb382ffd700 1 -- 172.21.15.40:0/2120803082 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394020c50 con 0x560a54f074d0 2023-07-18T03:31:54.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.916+0000 7fb382ffd700 1 -- 172.21.15.40:0/2120803082 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb394018070 con 0x560a54f074d0 2023-07-18T03:31:54.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.916+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/2120803082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb394004570 con 0x560a54f074d0 2023-07-18T03:31:54.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.916+0000 7fb382ffd700 1 -- 172.21.15.40:0/2120803082 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb394011e80 con 0x560a54f074d0 2023-07-18T03:31:54.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.916+0000 7fb3b5a4ab80 10 client.15708 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:54.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.916+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/2120803082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560a54c852b0 con 0x560a54f074d0 2023-07-18T03:31:54.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.916+0000 7fb382ffd700 1 -- 172.21.15.40:0/2120803082 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb394004570 con 0x560a54f074d0 2023-07-18T03:31:54.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.916+0000 7fb382ffd700 1 -- 172.21.15.40:0/2120803082 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb394005340 con 0x560a54f074d0 2023-07-18T03:31:54.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.916+0000 7fb3b5a4ab80 10 client.15708 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:54.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.916+0000 7fb3b5a4ab80 10 client.15708 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2023-07-18T03:31:54.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.916+0000 7fb3b5a4ab80 20 client.15708 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:54.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.916+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/2120803082 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x560a54f71700 0x560a54f73b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:54.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.916+0000 7fb3b5a4ab80 4 client.15708 mds_command: new command op to 24442 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:31:54.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.916+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/2120803082 --> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560a54e14050 con 0x560a54f71700 2023-07-18T03:31:54.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.916+0000 7fb3a27fc700 1 --2- 172.21.15.40:0/2120803082 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x560a54f71700 0x560a54f73b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:54.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.917+0000 7fb3a27fc700 1 --2- 172.21.15.40:0/2120803082 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x560a54f71700 0x560a54f73b10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=584915fa6b60eb11 server_cookie=e88dbc7f40713a63 in_seq=0 out_seq=0 2023-07-18T03:31:54.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.917+0000 7fb382ffd700 10 client.15708 ms_handle_connect on v2:172.21.15.134:6832/2345264306 2023-07-18T03:31:54.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.917+0000 7fb382ffd700 1 -- 172.21.15.40:0/2120803082 <== mds.4 v2:172.21.15.134:6832/2345264306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560a54e14050 con 0x560a54f71700 2023-07-18T03:31:54.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.917+0000 7fb382ffd700 10 client.15708 handle_command_reply: tid=0 2023-07-18T03:31:54.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.917+0000 7fb3b5a4ab80 1 client.15708 shutdown 2023-07-18T03:31:54.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.917+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/2120803082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 msgr2=0x560a54f142c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.917+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/2120803082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f142c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb394011a50 tx=0x7fb394005a00 comp rx=0 tx=0).stop 2023-07-18T03:31:54.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.917+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/2120803082 shutdown_connections 2023-07-18T03:31:54.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.917+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/2120803082 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x560a54f71700 0x560a54f73b10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.917+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/2120803082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560a54f08c70 0x560a54f17820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.917+0000 7fb382ffd700 0 client.15708 ms_handle_reset on v2:172.21.15.134:6832/2345264306 2023-07-18T03:31:54.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.917+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/2120803082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560a54f074d0 0x560a54f142c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.917+0000 7fb3b5a4ab80 1 --2- 172.21.15.40:0/2120803082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a54f07e20 0x560a54f14800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.917+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/2120803082 >> 172.21.15.40:0/2120803082 conn(0x560a54efcda0 msgr2=0x560a54f0fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:54.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.917+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/2120803082 shutdown_connections 2023-07-18T03:31:54.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.917+0000 7fb3b5a4ab80 1 -- 172.21.15.40:0/2120803082 wait complete. 2023-07-18T03:31:54.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.918+0000 7fb3b5a4ab80 20 client.15708 trim_cache size 0 max 16384 2023-07-18T03:31:54.924 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:31:54.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.918+0000 7fb3a98c4700 1 -- 172.21.15.40:0/2812669821 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb38c0a11d0 msgr2=0x7fb38c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.918+0000 7fb3a98c4700 1 --2- 172.21.15.40:0/2812669821 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb38c0a11d0 0x7fb38c0a3680 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb394009cb0 tx=0x7fb394009260 comp rx=0 tx=0).stop 2023-07-18T03:31:54.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.918+0000 7fb3a98c4700 1 -- 172.21.15.40:0/2812669821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a4102ee0 msgr2=0x7fb3a40fe570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:54.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.918+0000 7fb3a98c4700 1 --2- 172.21.15.40:0/2812669821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a4102ee0 0x7fb3a40fe570 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb39800e9a0 tx=0x7fb39800ed60 comp rx=0 tx=0).stop 2023-07-18T03:31:54.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.918+0000 7fb3a98c4700 1 -- 172.21.15.40:0/2812669821 shutdown_connections 2023-07-18T03:31:54.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.918+0000 7fb3a98c4700 1 --2- 172.21.15.40:0/2812669821 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb38c0a11d0 0x7fb38c0a3680 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.918+0000 7fb3a98c4700 1 --2- 172.21.15.40:0/2812669821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3a4105800 0x7fb3a40feab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.918+0000 7fb3a98c4700 1 --2- 172.21.15.40:0/2812669821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3a41005c0 0x7fb3a40fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.918+0000 7fb3a98c4700 1 --2- 172.21.15.40:0/2812669821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3a4102ee0 0x7fb3a40fe570 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:54.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.918+0000 7fb3a98c4700 1 -- 172.21.15.40:0/2812669821 >> 172.21.15.40:0/2812669821 conn(0x7fb3a40fa0a0 msgr2=0x7fb3a4101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:54.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.918+0000 7fb3a98c4700 1 -- 172.21.15.40:0/2812669821 shutdown_connections 2023-07-18T03:31:54.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:54.918+0000 7fb3a98c4700 1 -- 172.21.15.40:0/2812669821 wait complete. 2023-07-18T03:31:54.928 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-18T03:31:55.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.058+0000 7f3852dc8700 1 -- 172.21.15.40:0/3309280017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0ffbd0 msgr2=0x7f384c100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.058+0000 7f3852dc8700 1 --2- 172.21.15.40:0/3309280017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0ffbd0 0x7f384c100020 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3838009990 tx=0x7f3838009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.058+0000 7f3852dc8700 1 -- 172.21.15.40:0/3309280017 shutdown_connections 2023-07-18T03:31:55.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.058+0000 7f3852dc8700 1 --2- 172.21.15.40:0/3309280017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f384c100560 0x7f384c106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.058+0000 7f3852dc8700 1 --2- 172.21.15.40:0/3309280017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f384c0fe9e0 0x7f384c0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.058+0000 7f3852dc8700 1 --2- 172.21.15.40:0/3309280017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0ffbd0 0x7f384c100020 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.058+0000 7f3852dc8700 1 -- 172.21.15.40:0/3309280017 >> 172.21.15.40:0/3309280017 conn(0x7f384c0fa000 msgr2=0x7f384c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.058+0000 7f3852dc8700 1 -- 172.21.15.40:0/3309280017 shutdown_connections 2023-07-18T03:31:55.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.058+0000 7f3852dc8700 1 -- 172.21.15.40:0/3309280017 wait complete. 2023-07-18T03:31:55.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.059+0000 7f3852dc8700 1 Processor -- start 2023-07-18T03:31:55.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.059+0000 7f3852dc8700 1 -- start start 2023-07-18T03:31:55.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.059+0000 7f3852dc8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0fe9e0 0x7f384c102970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.059+0000 7f3852dc8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f384c0ffbd0 0x7f384c100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.059+0000 7f3852dc8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f384c100560 0x7f384c1014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.059+0000 7f3852dc8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c101c90 con 0x7f384c0fe9e0 2023-07-18T03:31:55.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.059+0000 7f3852dc8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c101e00 con 0x7f384c0ffbd0 2023-07-18T03:31:55.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.059+0000 7f3852dc8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c19f8d0 con 0x7f384c100560 2023-07-18T03:31:55.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.059+0000 7f3850b64700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0fe9e0 0x7f384c102970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.059+0000 7f3850b64700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0fe9e0 0x7f384c102970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58572/0 (socket says 172.21.15.40:58572) 2023-07-18T03:31:55.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.059+0000 7f3850b64700 1 -- 172.21.15.40:0/2334700302 learned_addr learned my addr 172.21.15.40:0/2334700302 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.059+0000 7f384bfff700 1 --2- 172.21.15.40:0/2334700302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f384c0ffbd0 0x7f384c100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.059+0000 7f3851365700 1 --2- 172.21.15.40:0/2334700302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f384c100560 0x7f384c1014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.059+0000 7f3850b64700 1 -- 172.21.15.40:0/2334700302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f384c100560 msgr2=0x7f384c1014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.059+0000 7f3850b64700 1 --2- 172.21.15.40:0/2334700302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f384c100560 0x7f384c1014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.059+0000 7f3850b64700 1 -- 172.21.15.40:0/2334700302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f384c0ffbd0 msgr2=0x7f384c100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.059+0000 7f3850b64700 1 --2- 172.21.15.40:0/2334700302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f384c0ffbd0 0x7f384c100fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.060+0000 7f3850b64700 1 -- 172.21.15.40:0/2334700302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38380095f0 con 0x7f384c0fe9e0 2023-07-18T03:31:55.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.060+0000 7f3851365700 1 --2- 172.21.15.40:0/2334700302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f384c100560 0x7f384c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:55.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.060+0000 7f3850b64700 1 --2- 172.21.15.40:0/2334700302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0fe9e0 0x7f384c102970 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f384000d980 tx=0x7f384000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.060+0000 7f3849ffb700 1 -- 172.21.15.40:0/2334700302 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840004020 con 0x7f384c0fe9e0 2023-07-18T03:31:55.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.060+0000 7f3849ffb700 1 -- 172.21.15.40:0/2334700302 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3840009730 con 0x7f384c0fe9e0 2023-07-18T03:31:55.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.060+0000 7f3852dc8700 1 -- 172.21.15.40:0/2334700302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384c19fbb0 con 0x7f384c0fe9e0 2023-07-18T03:31:55.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.060+0000 7f3849ffb700 1 -- 172.21.15.40:0/2334700302 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840003a80 con 0x7f384c0fe9e0 2023-07-18T03:31:55.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.060+0000 7f3852dc8700 1 -- 172.21.15.40:0/2334700302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384c1a0110 con 0x7f384c0fe9e0 2023-07-18T03:31:55.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.062+0000 7f3849ffb700 1 -- 172.21.15.40:0/2334700302 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f38400098a0 con 0x7f384c0fe9e0 2023-07-18T03:31:55.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.063+0000 7f3849ffb700 1 --2- 172.21.15.40:0/2334700302 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f383c0a11d0 0x7f383c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.063+0000 7f3849ffb700 1 -- 172.21.15.40:0/2334700302 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f38400cc590 con 0x7f384c0fe9e0 2023-07-18T03:31:55.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.063+0000 7f384bfff700 1 --2- 172.21.15.40:0/2334700302 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f383c0a11d0 0x7f383c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.063+0000 7f384bfff700 1 --2- 172.21.15.40:0/2334700302 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f383c0a11d0 0x7f383c0a3680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f383800b2e0 tx=0x7f3838005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.063+0000 7f385ef4eb80 1 Processor -- start 2023-07-18T03:31:55.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f385ef4eb80 1 -- start start 2023-07-18T03:31:55.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f385ef4eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1d4d0 0x5593ccf1d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f385ef4eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1de20 0x5593ccf1e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f385ef4eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1ec70 0x5593ccf1f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f385ef4eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593cce22cb0 con 0x5593ccf1de20 2023-07-18T03:31:55.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f385ef4eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593cce75a40 con 0x5593ccf1d4d0 2023-07-18T03:31:55.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f385ef4eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593cce6eed0 con 0x5593ccf1ec70 2023-07-18T03:31:55.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f384bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1de20 0x5593ccf1e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f384bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1de20 0x5593ccf1e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58582/0 (socket says 172.21.15.40:58582) 2023-07-18T03:31:55.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f384bfff700 1 -- 172.21.15.40:0/87664210 learned_addr learned my addr 172.21.15.40:0/87664210 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f3850b64700 1 --2- 172.21.15.40:0/87664210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1d4d0 0x5593ccf1d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f3851365700 1 --2- 172.21.15.40:0/87664210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1ec70 0x5593ccf1f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f384bfff700 1 -- 172.21.15.40:0/87664210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1ec70 msgr2=0x5593ccf1f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f384bfff700 1 --2- 172.21.15.40:0/87664210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1ec70 0x5593ccf1f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f384bfff700 1 -- 172.21.15.40:0/87664210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1d4d0 msgr2=0x5593ccf1d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f384bfff700 1 --2- 172.21.15.40:0/87664210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1d4d0 0x5593ccf1d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f384bfff700 1 -- 172.21.15.40:0/87664210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593ccc9cde0 con 0x5593ccf1de20 2023-07-18T03:31:55.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f3850b64700 1 --2- 172.21.15.40:0/87664210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1d4d0 0x5593ccf1d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:55.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.064+0000 7f384bfff700 1 --2- 172.21.15.40:0/87664210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1de20 0x5593ccf1e6e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3838015000 tx=0x7f3838020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a30215752ff396 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.065+0000 7f3836ffd700 1 -- 172.21.15.40:0/87664210 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383800f6e0 con 0x5593ccf1de20 2023-07-18T03:31:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.065+0000 7f3836ffd700 1 -- 172.21.15.40:0/87664210 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f383801e040 con 0x5593ccf1de20 2023-07-18T03:31:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.065+0000 7f3836ffd700 1 -- 172.21.15.40:0/87664210 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383802fc50 con 0x5593ccf1de20 2023-07-18T03:31:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.065+0000 7f385ef4eb80 1 -- 172.21.15.40:0/87664210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1de20 msgr2=0x5593ccf1e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.065+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/87664210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1de20 0x5593ccf1e6e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3838015000 tx=0x7f3838020000 comp rx=0 tx=0).stop 2023-07-18T03:31:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.065+0000 7f385ef4eb80 1 -- 172.21.15.40:0/87664210 shutdown_connections 2023-07-18T03:31:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.065+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/87664210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1ec70 0x5593ccf1f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.065+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/87664210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1d4d0 0x5593ccf1d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.065+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/87664210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1de20 0x5593ccf1e6e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.065+0000 7f385ef4eb80 1 -- 172.21.15.40:0/87664210 >> 172.21.15.40:0/87664210 conn(0x5593ccf12da0 msgr2=0x5593ccf181d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.065+0000 7f385ef4eb80 1 -- 172.21.15.40:0/87664210 shutdown_connections 2023-07-18T03:31:55.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.065+0000 7f385ef4eb80 1 -- 172.21.15.40:0/87664210 wait complete. 2023-07-18T03:31:55.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.066+0000 7f385ef4eb80 1 -- start start 2023-07-18T03:31:55.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.066+0000 7f385ef4eb80 1 Processor -- start 2023-07-18T03:31:55.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.066+0000 7f385ef4eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1d4d0 0x5593ccf27160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.066+0000 7f385ef4eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1de20 0x5593ccf276a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.066+0000 7f385ef4eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1ec70 0x5593ccf2e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f385ef4eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593cce0eac0 con 0x5593ccf1ec70 2023-07-18T03:31:55.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f385ef4eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593cce0e7c0 con 0x5593ccf1d4d0 2023-07-18T03:31:55.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f385ef4eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593cccf2920 con 0x5593ccf1de20 2023-07-18T03:31:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f385ef4eb80 10 client.0 _ll_register_callbacks cb 0x5593cce0c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f3851365700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1ec70 0x5593ccf2e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f3851365700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1ec70 0x5593ccf2e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58590/0 (socket says 172.21.15.40:58590) 2023-07-18T03:31:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f3851365700 1 -- 172.21.15.40:0/1967279256 learned_addr learned my addr 172.21.15.40:0/1967279256 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f3850b64700 1 --2- 172.21.15.40:0/1967279256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1d4d0 0x5593ccf27160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f3851365700 1 -- 172.21.15.40:0/1967279256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1de20 msgr2=0x5593ccf276a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f384bfff700 1 --2- 172.21.15.40:0/1967279256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1de20 0x5593ccf276a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f3851365700 1 --2- 172.21.15.40:0/1967279256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1de20 0x5593ccf276a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f3851365700 1 -- 172.21.15.40:0/1967279256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1d4d0 msgr2=0x5593ccf27160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f3851365700 1 --2- 172.21.15.40:0/1967279256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1d4d0 0x5593ccf27160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f3851365700 1 -- 172.21.15.40:0/1967279256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384c107750 con 0x5593ccf1ec70 2023-07-18T03:31:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f3850b64700 1 --2- 172.21.15.40:0/1967279256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1d4d0 0x5593ccf27160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:31:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f384bfff700 1 --2- 172.21.15.40:0/1967279256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1de20 0x5593ccf276a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:31:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f3851365700 1 --2- 172.21.15.40:0/1967279256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1ec70 0x5593ccf2e6e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3844007a80 tx=0x7f3844007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fbbfd3edf1b724 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f382bfff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f382bfff700 1 -- 172.21.15.40:0/1967279256 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384400e070 con 0x5593ccf1ec70 2023-07-18T03:31:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f382bfff700 1 -- 172.21.15.40:0/1967279256 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3844004db0 con 0x5593ccf1ec70 2023-07-18T03:31:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1967279256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593ccdc84f0 con 0x5593ccf1ec70 2023-07-18T03:31:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f382bfff700 1 -- 172.21.15.40:0/1967279256 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3844012980 con 0x5593ccf1ec70 2023-07-18T03:31:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.067+0000 7f385ef4eb80 10 client.15720 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.068+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1967279256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5593ccc85c60 con 0x5593ccf1ec70 2023-07-18T03:31:55.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.068+0000 7f382bfff700 1 -- 172.21.15.40:0/1967279256 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5593ccdc84f0 con 0x5593ccf1ec70 2023-07-18T03:31:55.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.068+0000 7f382bfff700 1 -- 172.21.15.40:0/1967279256 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3844014040 con 0x5593ccf1ec70 2023-07-18T03:31:55.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.068+0000 7f385ef4eb80 10 client.15720 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:55.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.068+0000 7f385ef4eb80 10 client.15720 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-18T03:31:55.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.068+0000 7f385ef4eb80 20 client.15720 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:55.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.068+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/1967279256 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x5593ccf32fe0 0x5593ccf353f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.068+0000 7f385ef4eb80 4 client.15720 mds_command: new command op to 24452 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.068+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1967279256 --> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593cce67eb0 con 0x5593ccf32fe0 2023-07-18T03:31:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.068+0000 7f3850b64700 1 --2- 172.21.15.40:0/1967279256 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x5593ccf32fe0 0x5593ccf353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.069+0000 7f382bfff700 10 client.15720 ms_handle_connect on v2:172.21.15.64:6834/1864495009 2023-07-18T03:31:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.069+0000 7f3850b64700 1 --2- 172.21.15.40:0/1967279256 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x5593ccf32fe0 0x5593ccf353f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fa1bcbe205616803 server_cookie=6e897c36ce4fc687 in_seq=0 out_seq=0 2023-07-18T03:31:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.070+0000 7f382bfff700 1 -- 172.21.15.40:0/1967279256 <== mds.3 v2:172.21.15.64:6834/1864495009 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5593cce67eb0 con 0x5593ccf32fe0 2023-07-18T03:31:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.070+0000 7f382bfff700 10 client.15720 handle_command_reply: tid=0 2023-07-18T03:31:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.070+0000 7f385ef4eb80 1 client.15720 shutdown 2023-07-18T03:31:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.070+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1967279256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1ec70 msgr2=0x5593ccf2e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.070+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/1967279256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1ec70 0x5593ccf2e6e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3844007a80 tx=0x7f3844007e40 comp rx=0 tx=0).stop 2023-07-18T03:31:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.070+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1967279256 shutdown_connections 2023-07-18T03:31:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.070+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/1967279256 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x5593ccf32fe0 0x5593ccf353f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.070+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/1967279256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1ec70 0x5593ccf2e6e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.070+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/1967279256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1d4d0 0x5593ccf27160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.070+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/1967279256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1de20 0x5593ccf276a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.070+0000 7f382bfff700 0 client.15720 ms_handle_reset on v2:172.21.15.64:6834/1864495009 2023-07-18T03:31:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.070+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1967279256 >> 172.21.15.40:0/1967279256 conn(0x5593ccf12da0 msgr2=0x5593ccf178b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.070+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1967279256 shutdown_connections 2023-07-18T03:31:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.070+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1967279256 wait complete. 2023-07-18T03:31:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.070+0000 7f385ef4eb80 20 client.15720 trim_cache size 0 max 16384 2023-07-18T03:31:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.083+0000 7f385ef4eb80 1 Processor -- start 2023-07-18T03:31:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.083+0000 7f385ef4eb80 1 -- start start 2023-07-18T03:31:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f385ef4eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1d4d0 0x5593ccf26810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f385ef4eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1de20 0x5593ccf2e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f385ef4eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1ec70 0x5593ccf25e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f385ef4eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ccf129b0 con 0x5593ccf1d4d0 2023-07-18T03:31:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f385ef4eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ccf24e30 con 0x5593ccf1ec70 2023-07-18T03:31:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f385ef4eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ccf26360 con 0x5593ccf1de20 2023-07-18T03:31:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f3850b64700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1d4d0 0x5593ccf26810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f3850b64700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1d4d0 0x5593ccf26810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58594/0 (socket says 172.21.15.40:58594) 2023-07-18T03:31:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f3850b64700 1 -- 172.21.15.40:0/1596450999 learned_addr learned my addr 172.21.15.40:0/1596450999 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f3851365700 1 --2- 172.21.15.40:0/1596450999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1ec70 0x5593ccf25e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f384bfff700 1 --2- 172.21.15.40:0/1596450999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1de20 0x5593ccf2e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f3850b64700 1 -- 172.21.15.40:0/1596450999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1de20 msgr2=0x5593ccf2e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f3850b64700 1 --2- 172.21.15.40:0/1596450999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1de20 0x5593ccf2e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f3850b64700 1 -- 172.21.15.40:0/1596450999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1ec70 msgr2=0x5593ccf25e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f3850b64700 1 --2- 172.21.15.40:0/1596450999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1ec70 0x5593ccf25e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f3850b64700 1 -- 172.21.15.40:0/1596450999 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593cccc3250 con 0x5593ccf1d4d0 2023-07-18T03:31:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f384bfff700 1 --2- 172.21.15.40:0/1596450999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1de20 0x5593ccf2e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:31:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f3850b64700 1 --2- 172.21.15.40:0/1596450999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1d4d0 0x5593ccf26810 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3840095e80 tx=0x7f38400b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18ae93bc784972f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f38377fe700 1 -- 172.21.15.40:0/1596450999 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38400a1dd0 con 0x5593ccf1d4d0 2023-07-18T03:31:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f38377fe700 1 -- 172.21.15.40:0/1596450999 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38400ca180 con 0x5593ccf1d4d0 2023-07-18T03:31:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.084+0000 7f38377fe700 1 -- 172.21.15.40:0/1596450999 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840099be0 con 0x5593ccf1d4d0 2023-07-18T03:31:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.085+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1596450999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1d4d0 msgr2=0x5593ccf26810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.085+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/1596450999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1d4d0 0x5593ccf26810 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3840095e80 tx=0x7f38400b9040 comp rx=0 tx=0).stop 2023-07-18T03:31:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.085+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1596450999 shutdown_connections 2023-07-18T03:31:55.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.085+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/1596450999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1ec70 0x5593ccf25e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.085+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/1596450999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1d4d0 0x5593ccf26810 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.085+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/1596450999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1de20 0x5593ccf2e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.085+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1596450999 >> 172.21.15.40:0/1596450999 conn(0x5593ccf12da0 msgr2=0x5593ccf39d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.085+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1596450999 shutdown_connections 2023-07-18T03:31:55.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.085+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1596450999 wait complete. 2023-07-18T03:31:55.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.085+0000 7f385ef4eb80 1 -- start start 2023-07-18T03:31:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.085+0000 7f385ef4eb80 1 Processor -- start 2023-07-18T03:31:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f385ef4eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1d4d0 0x5593ccf2e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f385ef4eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1de20 0x5593ccf2aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f385ef4eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1ec70 0x5593ccf2b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f385ef4eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ccf2b8e0 con 0x5593ccf1de20 2023-07-18T03:31:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f385ef4eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ccf2e7c0 con 0x5593ccf1d4d0 2023-07-18T03:31:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f385ef4eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593ccf2e980 con 0x5593ccf1ec70 2023-07-18T03:31:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f385ef4eb80 10 client.0 _ll_register_callbacks cb 0x7f3844007470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f384bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1de20 0x5593ccf2aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f384bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1de20 0x5593ccf2aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58602/0 (socket says 172.21.15.40:58602) 2023-07-18T03:31:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f384bfff700 1 -- 172.21.15.40:0/1910193500 learned_addr learned my addr 172.21.15.40:0/1910193500 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f3851365700 1 --2- 172.21.15.40:0/1910193500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1ec70 0x5593ccf2b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f3850b64700 1 --2- 172.21.15.40:0/1910193500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1d4d0 0x5593ccf2e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f384bfff700 1 -- 172.21.15.40:0/1910193500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1ec70 msgr2=0x5593ccf2b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f384bfff700 1 --2- 172.21.15.40:0/1910193500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1ec70 0x5593ccf2b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f384bfff700 1 -- 172.21.15.40:0/1910193500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1d4d0 msgr2=0x5593ccf2e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f384bfff700 1 --2- 172.21.15.40:0/1910193500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1d4d0 0x5593ccf2e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f384bfff700 1 -- 172.21.15.40:0/1910193500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384c1a0110 con 0x5593ccf1de20 2023-07-18T03:31:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f384bfff700 1 --2- 172.21.15.40:0/1910193500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1de20 0x5593ccf2aca0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f383800bf60 tx=0x7f383800fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78993bf645fc3cb4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.086+0000 7f382bfff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.087+0000 7f382bfff700 1 -- 172.21.15.40:0/1910193500 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383802f850 con 0x5593ccf1de20 2023-07-18T03:31:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.087+0000 7f382bfff700 1 -- 172.21.15.40:0/1910193500 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f383802f9b0 con 0x5593ccf1de20 2023-07-18T03:31:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.087+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1910193500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f383801e040 con 0x5593ccf1de20 2023-07-18T03:31:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.087+0000 7f382bfff700 1 -- 172.21.15.40:0/1910193500 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f383802fb70 con 0x5593ccf1de20 2023-07-18T03:31:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.087+0000 7f385ef4eb80 10 client.15726 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.087+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1910193500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5593ccc9b2b0 con 0x5593ccf1de20 2023-07-18T03:31:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.087+0000 7f382bfff700 1 -- 172.21.15.40:0/1910193500 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f383801e040 con 0x5593ccf1de20 2023-07-18T03:31:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.087+0000 7f382bfff700 1 -- 172.21.15.40:0/1910193500 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3838039cf0 con 0x5593ccf1de20 2023-07-18T03:31:55.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.087+0000 7f385ef4eb80 10 client.15726 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:55.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.087+0000 7f385ef4eb80 10 client.15726 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-18T03:31:55.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.087+0000 7f385ef4eb80 20 client.15726 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:55.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.087+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/1910193500 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x5593ccf882a0 0x5593ccf8a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.087+0000 7f385ef4eb80 4 client.15726 mds_command: new command op to 24452 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:31:55.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.087+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1910193500 --> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5593cce2a050 con 0x5593ccf882a0 2023-07-18T03:31:55.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.087+0000 7f3850b64700 1 --2- 172.21.15.40:0/1910193500 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x5593ccf882a0 0x5593ccf8a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.088+0000 7f382bfff700 10 client.15726 ms_handle_connect on v2:172.21.15.64:6834/1864495009 2023-07-18T03:31:55.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.088+0000 7f3850b64700 1 --2- 172.21.15.40:0/1910193500 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x5593ccf882a0 0x5593ccf8a6b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=76e25070ac96d5db server_cookie=b183a8c72f6f61cc in_seq=0 out_seq=0 2023-07-18T03:31:55.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.088+0000 7f382bfff700 1 -- 172.21.15.40:0/1910193500 <== mds.3 v2:172.21.15.64:6834/1864495009 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5593cce2a050 con 0x5593ccf882a0 2023-07-18T03:31:55.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.088+0000 7f382bfff700 10 client.15726 handle_command_reply: tid=0 2023-07-18T03:31:55.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.088+0000 7f385ef4eb80 1 client.15726 shutdown 2023-07-18T03:31:55.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.088+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1910193500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1de20 msgr2=0x5593ccf2aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.088+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/1910193500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1de20 0x5593ccf2aca0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f383800bf60 tx=0x7f383800fc10 comp rx=0 tx=0).stop 2023-07-18T03:31:55.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.088+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1910193500 shutdown_connections 2023-07-18T03:31:55.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.088+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/1910193500 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x5593ccf882a0 0x5593ccf8a6b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.088+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/1910193500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5593ccf1ec70 0x5593ccf2b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.088+0000 7f382bfff700 0 client.15726 ms_handle_reset on v2:172.21.15.64:6834/1864495009 2023-07-18T03:31:55.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.088+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/1910193500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593ccf1d4d0 0x5593ccf2e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.088+0000 7f385ef4eb80 1 --2- 172.21.15.40:0/1910193500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5593ccf1de20 0x5593ccf2aca0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.088+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1910193500 >> 172.21.15.40:0/1910193500 conn(0x5593ccf12da0 msgr2=0x5593ccf25110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.088+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1910193500 shutdown_connections 2023-07-18T03:31:55.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.088+0000 7f385ef4eb80 1 -- 172.21.15.40:0/1910193500 wait complete. 2023-07-18T03:31:55.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.088+0000 7f385ef4eb80 20 client.15726 trim_cache size 0 max 16384 2023-07-18T03:31:55.098 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:31:55.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.089+0000 7f3852dc8700 1 -- 172.21.15.40:0/2334700302 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f383c0a11d0 msgr2=0x7f383c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.089+0000 7f3852dc8700 1 --2- 172.21.15.40:0/2334700302 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f383c0a11d0 0x7f383c0a3680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f383800b2e0 tx=0x7f3838005fb0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.089+0000 7f3852dc8700 1 -- 172.21.15.40:0/2334700302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0fe9e0 msgr2=0x7f384c102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.089+0000 7f3852dc8700 1 --2- 172.21.15.40:0/2334700302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0fe9e0 0x7f384c102970 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f384000d980 tx=0x7f384000dc90 comp rx=0 tx=0).stop 2023-07-18T03:31:55.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.089+0000 7f3852dc8700 1 -- 172.21.15.40:0/2334700302 shutdown_connections 2023-07-18T03:31:55.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.089+0000 7f3852dc8700 1 --2- 172.21.15.40:0/2334700302 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f383c0a11d0 0x7f383c0a3680 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.089+0000 7f3852dc8700 1 --2- 172.21.15.40:0/2334700302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f384c100560 0x7f384c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.089+0000 7f3852dc8700 1 --2- 172.21.15.40:0/2334700302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f384c0fe9e0 0x7f384c102970 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.089+0000 7f3852dc8700 1 --2- 172.21.15.40:0/2334700302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f384c0ffbd0 0x7f384c100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.089+0000 7f3852dc8700 1 -- 172.21.15.40:0/2334700302 >> 172.21.15.40:0/2334700302 conn(0x7f384c0fa000 msgr2=0x7f384c068680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.089+0000 7f3852dc8700 1 -- 172.21.15.40:0/2334700302 shutdown_connections 2023-07-18T03:31:55.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.089+0000 7f3852dc8700 1 -- 172.21.15.40:0/2334700302 wait complete. 2023-07-18T03:31:55.101 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:31:55.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.231+0000 7efead10b700 1 -- 172.21.15.40:0/2728758396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efea80ffae0 msgr2=0x7efea80ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.231+0000 7efead10b700 1 --2- 172.21.15.40:0/2728758396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efea80ffae0 0x7efea80ffef0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efe98009990 tx=0x7efe98009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.231+0000 7efead10b700 1 -- 172.21.15.40:0/2728758396 shutdown_connections 2023-07-18T03:31:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.231+0000 7efead10b700 1 --2- 172.21.15.40:0/2728758396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efea8101660 0x7efea8107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.231+0000 7efead10b700 1 --2- 172.21.15.40:0/2728758396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efea80ffae0 0x7efea80ffef0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.231+0000 7efead10b700 1 --2- 172.21.15.40:0/2728758396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efea8100cd0 0x7efea8101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.231+0000 7efead10b700 1 -- 172.21.15.40:0/2728758396 >> 172.21.15.40:0/2728758396 conn(0x7efea80fb120 msgr2=0x7efea80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.231+0000 7efead10b700 1 -- 172.21.15.40:0/2728758396 shutdown_connections 2023-07-18T03:31:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.231+0000 7efead10b700 1 -- 172.21.15.40:0/2728758396 wait complete. 2023-07-18T03:31:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.232+0000 7efead10b700 1 Processor -- start 2023-07-18T03:31:55.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.232+0000 7efead10b700 1 -- start start 2023-07-18T03:31:55.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efead10b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efea80ffae0 0x7efea80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efead10b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efea8100cd0 0x7efea8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efead10b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efea8101660 0x7efea8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efead10b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea80735e0 con 0x7efea8100cd0 2023-07-18T03:31:55.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efead10b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea8073750 con 0x7efea80ffae0 2023-07-18T03:31:55.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efead10b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea819f880 con 0x7efea8101660 2023-07-18T03:31:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efea659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efea8100cd0 0x7efea8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efea659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efea8100cd0 0x7efea8072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58616/0 (socket says 172.21.15.40:58616) 2023-07-18T03:31:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efea659c700 1 -- 172.21.15.40:0/1005774270 learned_addr learned my addr 172.21.15.40:0/1005774270 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efea759e700 1 --2- 172.21.15.40:0/1005774270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efea8101660 0x7efea8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efea6d9d700 1 --2- 172.21.15.40:0/1005774270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efea80ffae0 0x7efea80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efea659c700 1 -- 172.21.15.40:0/1005774270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efea8101660 msgr2=0x7efea8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efea659c700 1 --2- 172.21.15.40:0/1005774270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efea8101660 0x7efea8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efea659c700 1 -- 172.21.15.40:0/1005774270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efea80ffae0 msgr2=0x7efea80742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efea659c700 1 --2- 172.21.15.40:0/1005774270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efea80ffae0 0x7efea80742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efea659c700 1 -- 172.21.15.40:0/1005774270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe980095f0 con 0x7efea8100cd0 2023-07-18T03:31:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efea659c700 1 --2- 172.21.15.40:0/1005774270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efea8100cd0 0x7efea8072900 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00d950 tx=0x7efe9c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efe8ffff700 1 -- 172.21.15.40:0/1005774270 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c00f890 con 0x7efea8100cd0 2023-07-18T03:31:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efead10b700 1 -- 172.21.15.40:0/1005774270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea819fb60 con 0x7efea8100cd0 2023-07-18T03:31:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efe8ffff700 1 -- 172.21.15.40:0/1005774270 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe9c010460 con 0x7efea8100cd0 2023-07-18T03:31:55.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.233+0000 7efe8ffff700 1 -- 172.21.15.40:0/1005774270 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c00e610 con 0x7efea8100cd0 2023-07-18T03:31:55.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.234+0000 7efead10b700 1 -- 172.21.15.40:0/1005774270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea81a00c0 con 0x7efea8100cd0 2023-07-18T03:31:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.236+0000 7efe8ffff700 1 -- 172.21.15.40:0/1005774270 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efe9c0108d0 con 0x7efea8100cd0 2023-07-18T03:31:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.236+0000 7efe8ffff700 1 --2- 172.21.15.40:0/1005774270 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efe900a1150 0x7efe900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.237+0000 7efe8ffff700 1 -- 172.21.15.40:0/1005774270 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7efe9c0cca90 con 0x7efea8100cd0 2023-07-18T03:31:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.237+0000 7efea6d9d700 1 --2- 172.21.15.40:0/1005774270 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efe900a1150 0x7efe900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.237+0000 7efea6d9d700 1 --2- 172.21.15.40:0/1005774270 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efe900a1150 0x7efe900a3600 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe9800b5c0 tx=0x7efe98005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.237+0000 7efeb9291b80 1 Processor -- start 2023-07-18T03:31:55.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.237+0000 7efeb9291b80 1 -- start start 2023-07-18T03:31:55.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.237+0000 7efeb9291b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947d4d0 0x558a8947d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.237+0000 7efeb9291b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8947e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efeb9291b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 0x558a8947f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efeb9291b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a89382cb0 con 0x558a8947de20 2023-07-18T03:31:55.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efeb9291b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a893d5a40 con 0x558a8947ec70 2023-07-18T03:31:55.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efeb9291b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a893ceed0 con 0x558a8947d4d0 2023-07-18T03:31:55.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efea659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8947e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efea659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8947e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58624/0 (socket says 172.21.15.40:58624) 2023-07-18T03:31:55.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efea659c700 1 -- 172.21.15.40:0/3612120486 learned_addr learned my addr 172.21.15.40:0/3612120486 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efea6d9d700 1 --2- 172.21.15.40:0/3612120486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947d4d0 0x558a8947d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efea759e700 1 --2- 172.21.15.40:0/3612120486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 0x558a8947f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efea659c700 1 -- 172.21.15.40:0/3612120486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947d4d0 msgr2=0x558a8947d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efea659c700 1 --2- 172.21.15.40:0/3612120486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947d4d0 0x558a8947d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efea659c700 1 -- 172.21.15.40:0/3612120486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 msgr2=0x558a8947f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efea659c700 1 --2- 172.21.15.40:0/3612120486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 0x558a8947f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efea659c700 1 -- 172.21.15.40:0/3612120486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a891fcde0 con 0x558a8947de20 2023-07-18T03:31:55.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efea759e700 1 --2- 172.21.15.40:0/3612120486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 0x558a8947f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:55.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efea6d9d700 1 --2- 172.21.15.40:0/3612120486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947d4d0 0x558a8947d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:55.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efea659c700 1 --2- 172.21.15.40:0/3612120486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8947e6e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7efe9c096890 tx=0x7efe9c099ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8568a81720e85b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efe8d7fa700 1 -- 172.21.15.40:0/3612120486 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c0981c0 con 0x558a8947de20 2023-07-18T03:31:55.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efe8d7fa700 1 -- 172.21.15.40:0/3612120486 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe9c096530 con 0x558a8947de20 2023-07-18T03:31:55.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.238+0000 7efe8d7fa700 1 -- 172.21.15.40:0/3612120486 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c0a1de0 con 0x558a8947de20 2023-07-18T03:31:55.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.239+0000 7efeb9291b80 1 -- 172.21.15.40:0/3612120486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 msgr2=0x558a8947e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.239+0000 7efeb9291b80 1 --2- 172.21.15.40:0/3612120486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8947e6e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7efe9c096890 tx=0x7efe9c099ea0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.239+0000 7efeb9291b80 1 -- 172.21.15.40:0/3612120486 shutdown_connections 2023-07-18T03:31:55.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.239+0000 7efeb9291b80 1 --2- 172.21.15.40:0/3612120486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 0x558a8947f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.239+0000 7efeb9291b80 1 --2- 172.21.15.40:0/3612120486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947d4d0 0x558a8947d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.239+0000 7efeb9291b80 1 --2- 172.21.15.40:0/3612120486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8947e6e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.239+0000 7efeb9291b80 1 -- 172.21.15.40:0/3612120486 >> 172.21.15.40:0/3612120486 conn(0x558a89472da0 msgr2=0x558a894781d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.239+0000 7efeb9291b80 1 -- 172.21.15.40:0/3612120486 shutdown_connections 2023-07-18T03:31:55.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.239+0000 7efeb9291b80 1 -- 172.21.15.40:0/3612120486 wait complete. 2023-07-18T03:31:55.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.239+0000 7efeb9291b80 1 -- start start 2023-07-18T03:31:55.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efeb9291b80 1 Processor -- start 2023-07-18T03:31:55.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efeb9291b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947d4d0 0x558a89487160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efeb9291b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947de20 0x558a894876a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efeb9291b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 0x558a8948e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efeb9291b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a8936eac0 con 0x558a8947d4d0 2023-07-18T03:31:55.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efeb9291b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a8936e7c0 con 0x558a8947ec70 2023-07-18T03:31:55.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efeb9291b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a89252920 con 0x558a8947de20 2023-07-18T03:31:55.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efeb9291b80 10 client.0 _ll_register_callbacks cb 0x558a8936c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:55.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efea6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947d4d0 0x558a89487160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efea6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947d4d0 0x558a89487160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58628/0 (socket says 172.21.15.40:58628) 2023-07-18T03:31:55.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efea6d9d700 1 -- 172.21.15.40:0/113463379 learned_addr learned my addr 172.21.15.40:0/113463379 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efea659c700 1 --2- 172.21.15.40:0/113463379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947de20 0x558a894876a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efea759e700 1 --2- 172.21.15.40:0/113463379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 0x558a8948e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efea6d9d700 1 -- 172.21.15.40:0/113463379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947de20 msgr2=0x558a894876a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efea6d9d700 1 --2- 172.21.15.40:0/113463379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947de20 0x558a894876a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efea6d9d700 1 -- 172.21.15.40:0/113463379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 msgr2=0x558a8948e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efea6d9d700 1 --2- 172.21.15.40:0/113463379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 0x558a8948e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efea6d9d700 1 -- 172.21.15.40:0/113463379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea81a00c0 con 0x558a8947d4d0 2023-07-18T03:31:55.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efea759e700 1 --2- 172.21.15.40:0/113463379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 0x558a8948e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:55.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efea6d9d700 1 --2- 172.21.15.40:0/113463379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947d4d0 0x558a89487160 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efe98015000 tx=0x7efe9801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9089a78438cd68 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.240+0000 7efe8aff5700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:55.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.241+0000 7efe8aff5700 1 -- 172.21.15.40:0/113463379 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9800bed0 con 0x558a8947d4d0 2023-07-18T03:31:55.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.241+0000 7efeb9291b80 1 -- 172.21.15.40:0/113463379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a893284f0 con 0x558a8947d4d0 2023-07-18T03:31:55.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.241+0000 7efe8aff5700 1 -- 172.21.15.40:0/113463379 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7efe98003f30 con 0x558a8947d4d0 2023-07-18T03:31:55.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.241+0000 7efe8aff5700 1 -- 172.21.15.40:0/113463379 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe980040f0 con 0x558a8947d4d0 2023-07-18T03:31:55.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.241+0000 7efeb9291b80 10 client.15738 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:55.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.241+0000 7efeb9291b80 1 -- 172.21.15.40:0/113463379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558a891e5c60 con 0x558a8947d4d0 2023-07-18T03:31:55.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.241+0000 7efe8aff5700 1 -- 172.21.15.40:0/113463379 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe9803f080 con 0x558a8947d4d0 2023-07-18T03:31:55.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.241+0000 7efe8aff5700 1 -- 172.21.15.40:0/113463379 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efe98039030 con 0x558a8947d4d0 2023-07-18T03:31:55.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.241+0000 7efeb9291b80 10 client.15738 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.241+0000 7efeb9291b80 10 client.15738 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-18T03:31:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.241+0000 7efeb9291b80 20 client.15738 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.241+0000 7efeb9291b80 1 --2- 172.21.15.40:0/113463379 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x558a89492f40 0x558a89495350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.241+0000 7efeb9291b80 4 client.15738 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.241+0000 7efeb9291b80 1 -- 172.21.15.40:0/113463379 --> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a893c7eb0 con 0x558a89492f40 2023-07-18T03:31:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.241+0000 7efea659c700 1 --2- 172.21.15.40:0/113463379 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x558a89492f40 0x558a89495350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.242+0000 7efea659c700 1 --2- 172.21.15.40:0/113463379 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x558a89492f40 0x558a89495350 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da19ea72ce7a0e73 server_cookie=1da0cdda59d2b0b4 in_seq=0 out_seq=0 2023-07-18T03:31:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.242+0000 7efe8aff5700 10 client.15738 ms_handle_connect on v2:172.21.15.64:6836/3077792296 2023-07-18T03:31:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.243+0000 7efe8aff5700 1 -- 172.21.15.40:0/113463379 <== mds.2 v2:172.21.15.64:6836/3077792296 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558a893c7eb0 con 0x558a89492f40 2023-07-18T03:31:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.243+0000 7efe8aff5700 10 client.15738 handle_command_reply: tid=0 2023-07-18T03:31:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.243+0000 7efeb9291b80 1 client.15738 shutdown 2023-07-18T03:31:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.243+0000 7efeb9291b80 1 -- 172.21.15.40:0/113463379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947d4d0 msgr2=0x558a89487160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.243+0000 7efeb9291b80 1 --2- 172.21.15.40:0/113463379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947d4d0 0x558a89487160 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efe98015000 tx=0x7efe9801a050 comp rx=0 tx=0).stop 2023-07-18T03:31:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.243+0000 7efeb9291b80 1 -- 172.21.15.40:0/113463379 shutdown_connections 2023-07-18T03:31:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.243+0000 7efeb9291b80 1 --2- 172.21.15.40:0/113463379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 0x558a8948e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.243+0000 7efeb9291b80 1 --2- 172.21.15.40:0/113463379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947d4d0 0x558a89487160 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.243+0000 7efeb9291b80 1 --2- 172.21.15.40:0/113463379 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x558a89492f40 0x558a89495350 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.243+0000 7efeb9291b80 1 --2- 172.21.15.40:0/113463379 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947de20 0x558a894876a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.243+0000 7efe8aff5700 0 client.15738 ms_handle_reset on v2:172.21.15.64:6836/3077792296 2023-07-18T03:31:55.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.243+0000 7efeb9291b80 1 -- 172.21.15.40:0/113463379 >> 172.21.15.40:0/113463379 conn(0x558a89472da0 msgr2=0x558a894778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.243+0000 7efeb9291b80 1 -- 172.21.15.40:0/113463379 shutdown_connections 2023-07-18T03:31:55.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.243+0000 7efeb9291b80 1 -- 172.21.15.40:0/113463379 wait complete. 2023-07-18T03:31:55.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.243+0000 7efeb9291b80 20 client.15738 trim_cache size 0 max 16384 2023-07-18T03:31:55.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.256+0000 7efeb9291b80 1 Processor -- start 2023-07-18T03:31:55.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.256+0000 7efeb9291b80 1 -- start start 2023-07-18T03:31:55.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.256+0000 7efeb9291b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947d4d0 0x558a8948ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.256+0000 7efeb9291b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8948e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.256+0000 7efeb9291b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 0x558a89485d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.256+0000 7efeb9291b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a8947e210 con 0x558a8947de20 2023-07-18T03:31:55.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.256+0000 7efeb9291b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a89492c40 con 0x558a8947ec70 2023-07-18T03:31:55.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.256+0000 7efeb9291b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a894746d0 con 0x558a8947d4d0 2023-07-18T03:31:55.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.256+0000 7efea659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8948e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.256+0000 7efea659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8948e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58638/0 (socket says 172.21.15.40:58638) 2023-07-18T03:31:55.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.256+0000 7efea659c700 1 -- 172.21.15.40:0/1113553196 learned_addr learned my addr 172.21.15.40:0/1113553196 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.256+0000 7efea6d9d700 1 --2- 172.21.15.40:0/1113553196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947d4d0 0x558a8948ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.257+0000 7efea759e700 1 --2- 172.21.15.40:0/1113553196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 0x558a89485d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.257+0000 7efea659c700 1 -- 172.21.15.40:0/1113553196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947d4d0 msgr2=0x558a8948ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.257+0000 7efea659c700 1 --2- 172.21.15.40:0/1113553196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947d4d0 0x558a8948ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.257+0000 7efea659c700 1 -- 172.21.15.40:0/1113553196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 msgr2=0x558a89485d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.257+0000 7efea659c700 1 --2- 172.21.15.40:0/1113553196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 0x558a89485d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.257+0000 7efea659c700 1 -- 172.21.15.40:0/1113553196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a89223250 con 0x558a8947de20 2023-07-18T03:31:55.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.257+0000 7efea759e700 1 --2- 172.21.15.40:0/1113553196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 0x558a89485d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:31:55.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.257+0000 7efea659c700 1 --2- 172.21.15.40:0/1113553196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8948e6a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0a1d40 tx=0x7efe9c010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705c87fa54563969 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.257+0000 7efe8dffb700 1 -- 172.21.15.40:0/1113553196 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c097f60 con 0x558a8947de20 2023-07-18T03:31:55.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.257+0000 7efe8dffb700 1 -- 172.21.15.40:0/1113553196 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe9c09a240 con 0x558a8947de20 2023-07-18T03:31:55.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.257+0000 7efe8dffb700 1 -- 172.21.15.40:0/1113553196 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c099570 con 0x558a8947de20 2023-07-18T03:31:55.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.257+0000 7efeb9291b80 1 -- 172.21.15.40:0/1113553196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 msgr2=0x558a8948e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.257+0000 7efeb9291b80 1 --2- 172.21.15.40:0/1113553196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8948e6a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0a1d40 tx=0x7efe9c010cd0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.257+0000 7efeb9291b80 1 -- 172.21.15.40:0/1113553196 shutdown_connections 2023-07-18T03:31:55.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.257+0000 7efeb9291b80 1 --2- 172.21.15.40:0/1113553196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947ec70 0x558a89485d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efeb9291b80 1 --2- 172.21.15.40:0/1113553196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947d4d0 0x558a8948ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efeb9291b80 1 --2- 172.21.15.40:0/1113553196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8948e6a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efeb9291b80 1 -- 172.21.15.40:0/1113553196 >> 172.21.15.40:0/1113553196 conn(0x558a89472da0 msgr2=0x558a89499cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efeb9291b80 1 -- 172.21.15.40:0/1113553196 shutdown_connections 2023-07-18T03:31:55.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efeb9291b80 1 -- 172.21.15.40:0/1113553196 wait complete. 2023-07-18T03:31:55.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efeb9291b80 1 -- start start 2023-07-18T03:31:55.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efeb9291b80 1 Processor -- start 2023-07-18T03:31:55.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efeb9291b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947d4d0 0x558a8948deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efeb9291b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8948e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efeb9291b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947ec70 0x558a8948c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efeb9291b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a8948ca40 con 0x558a8947de20 2023-07-18T03:31:55.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efeb9291b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a8948cbb0 con 0x558a8947d4d0 2023-07-18T03:31:55.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efeb9291b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a8948cd70 con 0x558a8947ec70 2023-07-18T03:31:55.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efeb9291b80 10 client.0 _ll_register_callbacks cb 0x558a8936c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:55.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efea659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8948e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efea659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8948e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58644/0 (socket says 172.21.15.40:58644) 2023-07-18T03:31:55.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efea659c700 1 -- 172.21.15.40:0/334259550 learned_addr learned my addr 172.21.15.40:0/334259550 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efea6d9d700 1 --2- 172.21.15.40:0/334259550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947d4d0 0x558a8948deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.258+0000 7efea759e700 1 --2- 172.21.15.40:0/334259550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947ec70 0x558a8948c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efea659c700 1 -- 172.21.15.40:0/334259550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947ec70 msgr2=0x558a8948c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efea659c700 1 --2- 172.21.15.40:0/334259550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947ec70 0x558a8948c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efea659c700 1 -- 172.21.15.40:0/334259550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947d4d0 msgr2=0x558a8948deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efea659c700 1 --2- 172.21.15.40:0/334259550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947d4d0 0x558a8948deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efea659c700 1 -- 172.21.15.40:0/334259550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a891fcde0 con 0x558a8947de20 2023-07-18T03:31:55.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efea759e700 1 --2- 172.21.15.40:0/334259550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947ec70 0x558a8948c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:55.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efea659c700 1 --2- 172.21.15.40:0/334259550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8948e3f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efe9c099ee0 tx=0x7efe9c0982a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89ff1c4d6f6ce94 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efe8aff5700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:55.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efe8aff5700 1 -- 172.21.15.40:0/334259550 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c0ca2c0 con 0x558a8947de20 2023-07-18T03:31:55.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efe8aff5700 1 -- 172.21.15.40:0/334259550 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7efe9c0b2050 con 0x558a8947de20 2023-07-18T03:31:55.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efeb9291b80 1 -- 172.21.15.40:0/334259550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe9c096530 con 0x558a8947de20 2023-07-18T03:31:55.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efe8aff5700 1 -- 172.21.15.40:0/334259550 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe9c099310 con 0x558a8947de20 2023-07-18T03:31:55.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efeb9291b80 10 client.15744 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:55.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efeb9291b80 1 -- 172.21.15.40:0/334259550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558a891fb2b0 con 0x558a8947de20 2023-07-18T03:31:55.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efe8aff5700 1 -- 172.21.15.40:0/334259550 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe9c064030 con 0x558a8947de20 2023-07-18T03:31:55.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efe8aff5700 1 -- 172.21.15.40:0/334259550 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efe9c095d30 con 0x558a8947de20 2023-07-18T03:31:55.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efeb9291b80 10 client.15744 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:55.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efeb9291b80 10 client.15744 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-18T03:31:55.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.259+0000 7efeb9291b80 20 client.15744 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:55.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.260+0000 7efeb9291b80 1 --2- 172.21.15.40:0/334259550 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x558a894e7ae0 0x558a894e9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.260+0000 7efeb9291b80 4 client.15744 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:31:55.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.260+0000 7efeb9291b80 1 -- 172.21.15.40:0/334259550 --> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558a8938a050 con 0x558a894e7ae0 2023-07-18T03:31:55.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.260+0000 7efea6d9d700 1 --2- 172.21.15.40:0/334259550 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x558a894e7ae0 0x558a894e9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.260+0000 7efe8aff5700 10 client.15744 ms_handle_connect on v2:172.21.15.64:6836/3077792296 2023-07-18T03:31:55.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.260+0000 7efea6d9d700 1 --2- 172.21.15.40:0/334259550 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x558a894e7ae0 0x558a894e9ec0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eebf0db94743ab27 server_cookie=858dc0eba4b49b1c in_seq=0 out_seq=0 2023-07-18T03:31:55.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.260+0000 7efe8aff5700 1 -- 172.21.15.40:0/334259550 <== mds.2 v2:172.21.15.64:6836/3077792296 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558a8938a050 con 0x558a894e7ae0 2023-07-18T03:31:55.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.260+0000 7efe8aff5700 10 client.15744 handle_command_reply: tid=0 2023-07-18T03:31:55.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.260+0000 7efeb9291b80 1 client.15744 shutdown 2023-07-18T03:31:55.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.260+0000 7efeb9291b80 1 -- 172.21.15.40:0/334259550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 msgr2=0x558a8948e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.260+0000 7efeb9291b80 1 --2- 172.21.15.40:0/334259550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8948e3f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efe9c099ee0 tx=0x7efe9c0982a0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efeb9291b80 1 -- 172.21.15.40:0/334259550 shutdown_connections 2023-07-18T03:31:55.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efeb9291b80 1 --2- 172.21.15.40:0/334259550 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x558a894e7ae0 0x558a894e9ec0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efeb9291b80 1 --2- 172.21.15.40:0/334259550 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a8947ec70 0x558a8948c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efe8aff5700 0 client.15744 ms_handle_reset on v2:172.21.15.64:6836/3077792296 2023-07-18T03:31:55.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efeb9291b80 1 --2- 172.21.15.40:0/334259550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558a8947d4d0 0x558a8948deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efeb9291b80 1 --2- 172.21.15.40:0/334259550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558a8947de20 0x558a8948e3f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efeb9291b80 1 -- 172.21.15.40:0/334259550 >> 172.21.15.40:0/334259550 conn(0x558a89472da0 msgr2=0x558a8948ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efeb9291b80 1 -- 172.21.15.40:0/334259550 shutdown_connections 2023-07-18T03:31:55.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efeb9291b80 1 -- 172.21.15.40:0/334259550 wait complete. 2023-07-18T03:31:55.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efeb9291b80 20 client.15744 trim_cache size 0 max 16384 2023-07-18T03:31:55.270 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:31:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efead10b700 1 -- 172.21.15.40:0/1005774270 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efe900a1150 msgr2=0x7efe900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efead10b700 1 --2- 172.21.15.40:0/1005774270 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efe900a1150 0x7efe900a3600 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe9800b5c0 tx=0x7efe98005960 comp rx=0 tx=0).stop 2023-07-18T03:31:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efead10b700 1 -- 172.21.15.40:0/1005774270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efea8100cd0 msgr2=0x7efea8072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efead10b700 1 --2- 172.21.15.40:0/1005774270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efea8100cd0 0x7efea8072900 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00d950 tx=0x7efe9c00dc60 comp rx=0 tx=0).stop 2023-07-18T03:31:55.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efead10b700 1 -- 172.21.15.40:0/1005774270 shutdown_connections 2023-07-18T03:31:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efead10b700 1 --2- 172.21.15.40:0/1005774270 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efe900a1150 0x7efe900a3600 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efead10b700 1 --2- 172.21.15.40:0/1005774270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efea8101660 0x7efea8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efead10b700 1 --2- 172.21.15.40:0/1005774270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efea80ffae0 0x7efea80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efead10b700 1 --2- 172.21.15.40:0/1005774270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efea8100cd0 0x7efea8072900 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efead10b700 1 -- 172.21.15.40:0/1005774270 >> 172.21.15.40:0/1005774270 conn(0x7efea80fb120 msgr2=0x7efea80fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efead10b700 1 -- 172.21.15.40:0/1005774270 shutdown_connections 2023-07-18T03:31:55.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.261+0000 7efead10b700 1 -- 172.21.15.40:0/1005774270 wait complete. 2023-07-18T03:31:55.273 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:31:55.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.403+0000 7f01c31fe700 1 -- 172.21.15.40:0/4281303796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01bc101690 msgr2=0x7f01bc107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.403+0000 7f01c31fe700 1 --2- 172.21.15.40:0/4281303796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01bc101690 0x7f01bc107f50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f01b400b0a0 tx=0x7f01b400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.403+0000 7f01c31fe700 1 -- 172.21.15.40:0/4281303796 shutdown_connections 2023-07-18T03:31:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.403+0000 7f01c31fe700 1 --2- 172.21.15.40:0/4281303796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01bc101690 0x7f01bc107f50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.403+0000 7f01c31fe700 1 --2- 172.21.15.40:0/4281303796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01bc0ffb10 0x7f01bc0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.403+0000 7f01c31fe700 1 --2- 172.21.15.40:0/4281303796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01bc100d00 0x7f01bc101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.403+0000 7f01c31fe700 1 -- 172.21.15.40:0/4281303796 >> 172.21.15.40:0/4281303796 conn(0x7f01bc0fb130 msgr2=0x7f01bc0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.403+0000 7f01c31fe700 1 -- 172.21.15.40:0/4281303796 shutdown_connections 2023-07-18T03:31:55.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.403+0000 7f01c31fe700 1 -- 172.21.15.40:0/4281303796 wait complete. 2023-07-18T03:31:55.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.404+0000 7f01c31fe700 1 Processor -- start 2023-07-18T03:31:55.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.404+0000 7f01c31fe700 1 -- start start 2023-07-18T03:31:55.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.404+0000 7f01c31fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01bc0ffb10 0x7f01bc195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.404+0000 7f01c31fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01bc100d00 0x7f01bc195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.404+0000 7f01c31fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01bc101690 0x7f01bc19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.404+0000 7f01c31fe700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc19d4f0 con 0x7f01bc101690 2023-07-18T03:31:55.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.404+0000 7f01c31fe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc19d630 con 0x7f01bc0ffb10 2023-07-18T03:31:55.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.404+0000 7f01c31fe700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc19d7c0 con 0x7f01bc100d00 2023-07-18T03:31:55.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.404+0000 7f01c179b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01bc101690 0x7f01bc19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.404+0000 7f01c179b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01bc101690 0x7f01bc19cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58660/0 (socket says 172.21.15.40:58660) 2023-07-18T03:31:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.404+0000 7f01c179b700 1 -- 172.21.15.40:0/2365912059 learned_addr learned my addr 172.21.15.40:0/2365912059 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.404+0000 7f01bbfff700 1 --2- 172.21.15.40:0/2365912059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01bc100d00 0x7f01bc195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.404+0000 7f01c0f9a700 1 --2- 172.21.15.40:0/2365912059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01bc0ffb10 0x7f01bc195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.405+0000 7f01c179b700 1 -- 172.21.15.40:0/2365912059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01bc100d00 msgr2=0x7f01bc195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.405+0000 7f01c179b700 1 --2- 172.21.15.40:0/2365912059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01bc100d00 0x7f01bc195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.405+0000 7f01c179b700 1 -- 172.21.15.40:0/2365912059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01bc0ffb10 msgr2=0x7f01bc195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.405+0000 7f01c179b700 1 --2- 172.21.15.40:0/2365912059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01bc0ffb10 0x7f01bc195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.405+0000 7f01c179b700 1 -- 172.21.15.40:0/2365912059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b4009d00 con 0x7f01bc101690 2023-07-18T03:31:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.405+0000 7f01c0f9a700 1 --2- 172.21.15.40:0/2365912059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01bc0ffb10 0x7f01bc195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.405+0000 7f01c179b700 1 --2- 172.21.15.40:0/2365912059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01bc101690 0x7f01bc19cf20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f01b400d010 tx=0x7f01b4004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.405+0000 7f01b9ffb700 1 -- 172.21.15.40:0/2365912059 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b400e040 con 0x7f01bc101690 2023-07-18T03:31:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.405+0000 7f01b9ffb700 1 -- 172.21.15.40:0/2365912059 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01b4009440 con 0x7f01bc101690 2023-07-18T03:31:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.405+0000 7f01c31fe700 1 -- 172.21.15.40:0/2365912059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01bc19da40 con 0x7f01bc101690 2023-07-18T03:31:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.405+0000 7f01b9ffb700 1 -- 172.21.15.40:0/2365912059 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4004020 con 0x7f01bc101690 2023-07-18T03:31:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.405+0000 7f01c31fe700 1 -- 172.21.15.40:0/2365912059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01bc19dec0 con 0x7f01bc101690 2023-07-18T03:31:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.407+0000 7f01b9ffb700 1 -- 172.21.15.40:0/2365912059 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f01b401d690 con 0x7f01bc101690 2023-07-18T03:31:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.408+0000 7f01b9ffb700 1 --2- 172.21.15.40:0/2365912059 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f01ac0a1150 0x7f01ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.408+0000 7f01b9ffb700 1 -- 172.21.15.40:0/2365912059 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f01b40ce140 con 0x7f01bc101690 2023-07-18T03:31:55.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.408+0000 7f01c0f9a700 1 --2- 172.21.15.40:0/2365912059 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f01ac0a1150 0x7f01ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.408+0000 7f01c0f9a700 1 --2- 172.21.15.40:0/2365912059 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f01ac0a1150 0x7f01ac0a3600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f01b0005950 tx=0x7f01b00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01cf384b80 1 Processor -- start 2023-07-18T03:31:55.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01cf384b80 1 -- start start 2023-07-18T03:31:55.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01cf384b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc84d0 0x55ed28cc88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01cf384b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc8e20 0x55ed28cc96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01cf384b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc9c70 0x55ed28cca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01cf384b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed28bcdcb0 con 0x55ed28cc84d0 2023-07-18T03:31:55.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01cf384b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed28c20a40 con 0x55ed28cc9c70 2023-07-18T03:31:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01cf384b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed28c19ed0 con 0x55ed28cc8e20 2023-07-18T03:31:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01c0f9a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc84d0 0x55ed28cc88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01c0f9a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc84d0 0x55ed28cc88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58664/0 (socket says 172.21.15.40:58664) 2023-07-18T03:31:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01c0f9a700 1 -- 172.21.15.40:0/2760281143 learned_addr learned my addr 172.21.15.40:0/2760281143 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01bbfff700 1 --2- 172.21.15.40:0/2760281143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc8e20 0x55ed28cc96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01c179b700 1 --2- 172.21.15.40:0/2760281143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc9c70 0x55ed28cca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01c0f9a700 1 -- 172.21.15.40:0/2760281143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc8e20 msgr2=0x55ed28cc96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01c0f9a700 1 --2- 172.21.15.40:0/2760281143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc8e20 0x55ed28cc96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01c0f9a700 1 -- 172.21.15.40:0/2760281143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc9c70 msgr2=0x55ed28cca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01c0f9a700 1 --2- 172.21.15.40:0/2760281143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc9c70 0x55ed28cca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.409+0000 7f01c0f9a700 1 -- 172.21.15.40:0/2760281143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed28a47de0 con 0x55ed28cc84d0 2023-07-18T03:31:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.410+0000 7f01c0f9a700 1 --2- 172.21.15.40:0/2760281143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc84d0 0x55ed28cc88e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f01b000ff00 tx=0x7f01b0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5b3923b69e6bb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.410+0000 7f01a6ffd700 1 -- 172.21.15.40:0/2760281143 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b0011bb0 con 0x55ed28cc84d0 2023-07-18T03:31:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.410+0000 7f01a6ffd700 1 -- 172.21.15.40:0/2760281143 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01b0011d10 con 0x55ed28cc84d0 2023-07-18T03:31:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.410+0000 7f01a6ffd700 1 -- 172.21.15.40:0/2760281143 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b001d5b0 con 0x55ed28cc84d0 2023-07-18T03:31:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.410+0000 7f01cf384b80 1 -- 172.21.15.40:0/2760281143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc84d0 msgr2=0x55ed28cc88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.410+0000 7f01cf384b80 1 --2- 172.21.15.40:0/2760281143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc84d0 0x55ed28cc88e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f01b000ff00 tx=0x7f01b0012000 comp rx=0 tx=0).stop 2023-07-18T03:31:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.410+0000 7f01cf384b80 1 -- 172.21.15.40:0/2760281143 shutdown_connections 2023-07-18T03:31:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.410+0000 7f01cf384b80 1 --2- 172.21.15.40:0/2760281143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc9c70 0x55ed28cca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.410+0000 7f01cf384b80 1 --2- 172.21.15.40:0/2760281143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc84d0 0x55ed28cc88e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.410+0000 7f01cf384b80 1 --2- 172.21.15.40:0/2760281143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc8e20 0x55ed28cc96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.410+0000 7f01cf384b80 1 -- 172.21.15.40:0/2760281143 >> 172.21.15.40:0/2760281143 conn(0x55ed28cbdda0 msgr2=0x55ed28cc31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.410+0000 7f01cf384b80 1 -- 172.21.15.40:0/2760281143 shutdown_connections 2023-07-18T03:31:55.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.411+0000 7f01cf384b80 1 -- 172.21.15.40:0/2760281143 wait complete. 2023-07-18T03:31:55.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.411+0000 7f01cf384b80 1 -- start start 2023-07-18T03:31:55.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.411+0000 7f01cf384b80 1 Processor -- start 2023-07-18T03:31:55.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.411+0000 7f01cf384b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc84d0 0x55ed28cd2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.411+0000 7f01cf384b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc8e20 0x55ed28cd26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.411+0000 7f01cf384b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc9c70 0x55ed28cd96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.411+0000 7f01cf384b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed28bb9ac0 con 0x55ed28cc84d0 2023-07-18T03:31:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.411+0000 7f01cf384b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed28bb97c0 con 0x55ed28cc9c70 2023-07-18T03:31:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.411+0000 7f01cf384b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed28a9d920 con 0x55ed28cc8e20 2023-07-18T03:31:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.411+0000 7f01cf384b80 10 client.0 _ll_register_callbacks cb 0x55ed28bb7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.411+0000 7f01c0f9a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc84d0 0x55ed28cd2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.411+0000 7f01c0f9a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc84d0 0x55ed28cd2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58670/0 (socket says 172.21.15.40:58670) 2023-07-18T03:31:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.411+0000 7f01c0f9a700 1 -- 172.21.15.40:0/201285879 learned_addr learned my addr 172.21.15.40:0/201285879 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f01bbfff700 1 --2- 172.21.15.40:0/201285879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc8e20 0x55ed28cd26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f01c179b700 1 --2- 172.21.15.40:0/201285879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc9c70 0x55ed28cd96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f01c0f9a700 1 -- 172.21.15.40:0/201285879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc8e20 msgr2=0x55ed28cd26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f01c0f9a700 1 --2- 172.21.15.40:0/201285879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc8e20 0x55ed28cd26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f01c0f9a700 1 -- 172.21.15.40:0/201285879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc9c70 msgr2=0x55ed28cd96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f01c0f9a700 1 --2- 172.21.15.40:0/201285879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc9c70 0x55ed28cd96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f01c0f9a700 1 -- 172.21.15.40:0/201285879 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b000fbb0 con 0x55ed28cc84d0 2023-07-18T03:31:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f01c179b700 1 --2- 172.21.15.40:0/201285879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc9c70 0x55ed28cd96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f01bbfff700 1 --2- 172.21.15.40:0/201285879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc8e20 0x55ed28cd26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f01c0f9a700 1 --2- 172.21.15.40:0/201285879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc84d0 0x55ed28cd2170 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f01b0004780 tx=0x7f01b0011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c626b8591790581 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f019bfff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f019bfff700 1 -- 172.21.15.40:0/201285879 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b002d070 con 0x55ed28cc84d0 2023-07-18T03:31:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f019bfff700 1 -- 172.21.15.40:0/201285879 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f01b001e7a0 con 0x55ed28cc84d0 2023-07-18T03:31:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f01cf384b80 1 -- 172.21.15.40:0/201285879 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed28b734f0 con 0x55ed28cc84d0 2023-07-18T03:31:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f019bfff700 1 -- 172.21.15.40:0/201285879 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01b001e9a0 con 0x55ed28cc84d0 2023-07-18T03:31:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f01cf384b80 10 client.15756 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f01cf384b80 1 -- 172.21.15.40:0/201285879 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ed28a30c60 con 0x55ed28cc84d0 2023-07-18T03:31:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.412+0000 7f019bfff700 1 -- 172.21.15.40:0/201285879 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ed28b734f0 con 0x55ed28cc84d0 2023-07-18T03:31:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.413+0000 7f019bfff700 1 -- 172.21.15.40:0/201285879 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f01b00274c0 con 0x55ed28cc84d0 2023-07-18T03:31:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.413+0000 7f01cf384b80 10 client.15756 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.413+0000 7f01cf384b80 10 client.15756 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-18T03:31:55.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.413+0000 7f01cf384b80 20 client.15756 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:55.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.413+0000 7f01cf384b80 1 --2- 172.21.15.40:0/201285879 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x55ed28cdde30 0x55ed28ce0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.413+0000 7f01cf384b80 4 client.15756 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:55.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.413+0000 7f01cf384b80 1 -- 172.21.15.40:0/201285879 --> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed28c12eb0 con 0x55ed28cdde30 2023-07-18T03:31:55.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.413+0000 7f01bbfff700 1 --2- 172.21.15.40:0/201285879 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x55ed28cdde30 0x55ed28ce0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.413+0000 7f01bbfff700 1 --2- 172.21.15.40:0/201285879 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x55ed28cdde30 0x55ed28ce0240 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c7655b30810200f server_cookie=86b57bd74ed726de in_seq=0 out_seq=0 2023-07-18T03:31:55.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.413+0000 7f019bfff700 10 client.15756 ms_handle_connect on v2:172.21.15.134:6838/3192254101 2023-07-18T03:31:55.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.414+0000 7f019bfff700 1 -- 172.21.15.40:0/201285879 <== mds.1 v2:172.21.15.134:6838/3192254101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ed28c12eb0 con 0x55ed28cdde30 2023-07-18T03:31:55.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.414+0000 7f019bfff700 10 client.15756 handle_command_reply: tid=0 2023-07-18T03:31:55.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.414+0000 7f01cf384b80 1 client.15756 shutdown 2023-07-18T03:31:55.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.414+0000 7f01cf384b80 1 -- 172.21.15.40:0/201285879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc84d0 msgr2=0x55ed28cd2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.414+0000 7f01cf384b80 1 --2- 172.21.15.40:0/201285879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc84d0 0x55ed28cd2170 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f01b0004780 tx=0x7f01b0011c60 comp rx=0 tx=0).stop 2023-07-18T03:31:55.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.414+0000 7f01cf384b80 1 -- 172.21.15.40:0/201285879 shutdown_connections 2023-07-18T03:31:55.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.414+0000 7f01cf384b80 1 --2- 172.21.15.40:0/201285879 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x55ed28cdde30 0x55ed28ce0240 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.414+0000 7f01cf384b80 1 --2- 172.21.15.40:0/201285879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc9c70 0x55ed28cd96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.414+0000 7f01cf384b80 1 --2- 172.21.15.40:0/201285879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc84d0 0x55ed28cd2170 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.414+0000 7f01cf384b80 1 --2- 172.21.15.40:0/201285879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc8e20 0x55ed28cd26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.414+0000 7f019bfff700 0 client.15756 ms_handle_reset on v2:172.21.15.134:6838/3192254101 2023-07-18T03:31:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.414+0000 7f01cf384b80 1 -- 172.21.15.40:0/201285879 >> 172.21.15.40:0/201285879 conn(0x55ed28cbdda0 msgr2=0x55ed28cc28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.414+0000 7f01cf384b80 1 -- 172.21.15.40:0/201285879 shutdown_connections 2023-07-18T03:31:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.414+0000 7f01cf384b80 1 -- 172.21.15.40:0/201285879 wait complete. 2023-07-18T03:31:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.414+0000 7f01cf384b80 20 client.15756 trim_cache size 0 max 16384 2023-07-18T03:31:55.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.427+0000 7f01cf384b80 1 Processor -- start 2023-07-18T03:31:55.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.427+0000 7f01cf384b80 1 -- start start 2023-07-18T03:31:55.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.427+0000 7f01cf384b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc84d0 0x55ed28cca040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.427+0000 7f01cf384b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc8e20 0x55ed28cd96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.427+0000 7f01cf384b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc9c70 0x55ed28cd0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.427+0000 7f01cf384b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed28cc4c60 con 0x55ed28cc8e20 2023-07-18T03:31:55.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.428+0000 7f01cf384b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed28cd1390 con 0x55ed28cc84d0 2023-07-18T03:31:55.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.428+0000 7f01cf384b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed28cc8af0 con 0x55ed28cc9c70 2023-07-18T03:31:55.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.428+0000 7f01bbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc8e20 0x55ed28cd96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.428+0000 7f01bbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc8e20 0x55ed28cd96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58680/0 (socket says 172.21.15.40:58680) 2023-07-18T03:31:55.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.428+0000 7f01bbfff700 1 -- 172.21.15.40:0/1502165148 learned_addr learned my addr 172.21.15.40:0/1502165148 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.428+0000 7f01c179b700 1 --2- 172.21.15.40:0/1502165148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc9c70 0x55ed28cd0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.428+0000 7f01c0f9a700 1 --2- 172.21.15.40:0/1502165148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc84d0 0x55ed28cca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.428+0000 7f01bbfff700 1 -- 172.21.15.40:0/1502165148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc9c70 msgr2=0x55ed28cd0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.428+0000 7f01bbfff700 1 --2- 172.21.15.40:0/1502165148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc9c70 0x55ed28cd0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.428+0000 7f01bbfff700 1 -- 172.21.15.40:0/1502165148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc84d0 msgr2=0x55ed28cca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.428+0000 7f01bbfff700 1 --2- 172.21.15.40:0/1502165148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc84d0 0x55ed28cca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.428+0000 7f01bbfff700 1 -- 172.21.15.40:0/1502165148 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed28a6e250 con 0x55ed28cc8e20 2023-07-18T03:31:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.428+0000 7f01c0f9a700 1 --2- 172.21.15.40:0/1502165148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc84d0 0x55ed28cca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:31:55.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.428+0000 7f01bbfff700 1 --2- 172.21.15.40:0/1502165148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc8e20 0x55ed28cd96b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x55ed28ce0af0 tx=0x7f01a801ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e35c3f19ef36f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.428+0000 7f01a77fe700 1 -- 172.21.15.40:0/1502165148 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a80043d0 con 0x55ed28cc8e20 2023-07-18T03:31:55.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.428+0000 7f01a77fe700 1 -- 172.21.15.40:0/1502165148 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01a8017040 con 0x55ed28cc8e20 2023-07-18T03:31:55.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.428+0000 7f01a77fe700 1 -- 172.21.15.40:0/1502165148 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a8005020 con 0x55ed28cc8e20 2023-07-18T03:31:55.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.429+0000 7f01cf384b80 1 -- 172.21.15.40:0/1502165148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc8e20 msgr2=0x55ed28cd96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.429+0000 7f01cf384b80 1 --2- 172.21.15.40:0/1502165148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc8e20 0x55ed28cd96b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x55ed28ce0af0 tx=0x7f01a801ee60 comp rx=0 tx=0).stop 2023-07-18T03:31:55.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.429+0000 7f01cf384b80 1 -- 172.21.15.40:0/1502165148 shutdown_connections 2023-07-18T03:31:55.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.429+0000 7f01cf384b80 1 --2- 172.21.15.40:0/1502165148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc9c70 0x55ed28cd0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.429+0000 7f01cf384b80 1 --2- 172.21.15.40:0/1502165148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc84d0 0x55ed28cca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.429+0000 7f01cf384b80 1 --2- 172.21.15.40:0/1502165148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc8e20 0x55ed28cd96b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.429+0000 7f01cf384b80 1 -- 172.21.15.40:0/1502165148 >> 172.21.15.40:0/1502165148 conn(0x55ed28cbdda0 msgr2=0x55ed28ce4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.429+0000 7f01cf384b80 1 -- 172.21.15.40:0/1502165148 shutdown_connections 2023-07-18T03:31:55.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.429+0000 7f01cf384b80 1 -- 172.21.15.40:0/1502165148 wait complete. 2023-07-18T03:31:55.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.429+0000 7f01cf384b80 1 -- start start 2023-07-18T03:31:55.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.429+0000 7f01cf384b80 1 Processor -- start 2023-07-18T03:31:55.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01cf384b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc84d0 0x55ed28cd58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01cf384b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc8e20 0x55ed28cd5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01cf384b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc9c70 0x55ed28cd8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01cf384b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed28cd92d0 con 0x55ed28cc9c70 2023-07-18T03:31:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01cf384b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed28cd9440 con 0x55ed28cc84d0 2023-07-18T03:31:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01cf384b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed28cd9600 con 0x55ed28cc8e20 2023-07-18T03:31:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01cf384b80 10 client.0 _ll_register_callbacks cb 0x55ed28bb7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:55.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01c179b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc9c70 0x55ed28cd8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01c179b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc9c70 0x55ed28cd8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58688/0 (socket says 172.21.15.40:58688) 2023-07-18T03:31:55.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01c179b700 1 -- 172.21.15.40:0/142196206 learned_addr learned my addr 172.21.15.40:0/142196206 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01bbfff700 1 --2- 172.21.15.40:0/142196206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc8e20 0x55ed28cd5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01c0f9a700 1 --2- 172.21.15.40:0/142196206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc84d0 0x55ed28cd58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01c179b700 1 -- 172.21.15.40:0/142196206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc8e20 msgr2=0x55ed28cd5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01c179b700 1 --2- 172.21.15.40:0/142196206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc8e20 0x55ed28cd5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01c179b700 1 -- 172.21.15.40:0/142196206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc84d0 msgr2=0x55ed28cd58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01c179b700 1 --2- 172.21.15.40:0/142196206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc84d0 0x55ed28cd58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01c179b700 1 -- 172.21.15.40:0/142196206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a8007ae0 con 0x55ed28cc9c70 2023-07-18T03:31:55.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01c179b700 1 --2- 172.21.15.40:0/142196206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc9c70 0x55ed28cd8d20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f01b40cbc70 tx=0x7f01b401da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d662292e33761f56 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f019bfff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:55.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f019bfff700 1 -- 172.21.15.40:0/142196206 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b409af20 con 0x55ed28cc9c70 2023-07-18T03:31:55.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f019bfff700 1 -- 172.21.15.40:0/142196206 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f01b409b5c0 con 0x55ed28cc9c70 2023-07-18T03:31:55.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f01cf384b80 1 -- 172.21.15.40:0/142196206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01b0011d10 con 0x55ed28cc9c70 2023-07-18T03:31:55.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.430+0000 7f019bfff700 1 -- 172.21.15.40:0/142196206 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01b409f5f0 con 0x55ed28cc9c70 2023-07-18T03:31:55.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.431+0000 7f01cf384b80 10 client.15762 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:55.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.431+0000 7f01cf384b80 1 -- 172.21.15.40:0/142196206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ed28a462b0 con 0x55ed28cc9c70 2023-07-18T03:31:55.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.431+0000 7f019bfff700 1 -- 172.21.15.40:0/142196206 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01b0011d10 con 0x55ed28cc9c70 2023-07-18T03:31:55.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.431+0000 7f019bfff700 1 -- 172.21.15.40:0/142196206 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f01b409abb0 con 0x55ed28cc9c70 2023-07-18T03:31:55.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.431+0000 7f01cf384b80 10 client.15762 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:55.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.431+0000 7f01cf384b80 10 client.15762 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-18T03:31:55.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.431+0000 7f01cf384b80 20 client.15762 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:55.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.431+0000 7f01cf384b80 1 --2- 172.21.15.40:0/142196206 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x55ed28d32df0 0x55ed28d35200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.431+0000 7f01cf384b80 4 client.15762 mds_command: new command op to 24523 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:31:55.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.431+0000 7f01cf384b80 1 -- 172.21.15.40:0/142196206 --> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ed28bd5050 con 0x55ed28d32df0 2023-07-18T03:31:55.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.431+0000 7f01c0f9a700 1 --2- 172.21.15.40:0/142196206 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x55ed28d32df0 0x55ed28d35200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.432+0000 7f019bfff700 10 client.15762 ms_handle_connect on v2:172.21.15.134:6838/3192254101 2023-07-18T03:31:55.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.432+0000 7f01c0f9a700 1 --2- 172.21.15.40:0/142196206 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x55ed28d32df0 0x55ed28d35200 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52438eaed98c2691 server_cookie=fe07b472df94d8f3 in_seq=0 out_seq=0 2023-07-18T03:31:55.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.432+0000 7f019bfff700 1 -- 172.21.15.40:0/142196206 <== mds.1 v2:172.21.15.134:6838/3192254101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ed28bd5050 con 0x55ed28d32df0 2023-07-18T03:31:55.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.432+0000 7f019bfff700 10 client.15762 handle_command_reply: tid=0 2023-07-18T03:31:55.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.432+0000 7f01cf384b80 1 client.15762 shutdown 2023-07-18T03:31:55.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.432+0000 7f01cf384b80 1 -- 172.21.15.40:0/142196206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc9c70 msgr2=0x55ed28cd8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.432+0000 7f01cf384b80 1 --2- 172.21.15.40:0/142196206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc9c70 0x55ed28cd8d20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f01b40cbc70 tx=0x7f01b401da30 comp rx=0 tx=0).stop 2023-07-18T03:31:55.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.432+0000 7f01cf384b80 1 -- 172.21.15.40:0/142196206 shutdown_connections 2023-07-18T03:31:55.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.432+0000 7f01cf384b80 1 --2- 172.21.15.40:0/142196206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ed28cc9c70 0x55ed28cd8d20 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.432+0000 7f01cf384b80 1 --2- 172.21.15.40:0/142196206 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x55ed28d32df0 0x55ed28d35200 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.432+0000 7f01cf384b80 1 --2- 172.21.15.40:0/142196206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed28cc84d0 0x55ed28cd58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.432+0000 7f01cf384b80 1 --2- 172.21.15.40:0/142196206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ed28cc8e20 0x55ed28cd5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.432+0000 7f019bfff700 0 client.15762 ms_handle_reset on v2:172.21.15.134:6838/3192254101 2023-07-18T03:31:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.432+0000 7f01cf384b80 1 -- 172.21.15.40:0/142196206 >> 172.21.15.40:0/142196206 conn(0x55ed28cbdda0 msgr2=0x55ed28cccbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.432+0000 7f01cf384b80 1 -- 172.21.15.40:0/142196206 shutdown_connections 2023-07-18T03:31:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.432+0000 7f01cf384b80 1 -- 172.21.15.40:0/142196206 wait complete. 2023-07-18T03:31:55.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.432+0000 7f01cf384b80 20 client.15762 trim_cache size 0 max 16384 2023-07-18T03:31:55.443 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:31:55.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.433+0000 7f01c31fe700 1 -- 172.21.15.40:0/2365912059 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f01ac0a1150 msgr2=0x7f01ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.433+0000 7f01c31fe700 1 --2- 172.21.15.40:0/2365912059 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f01ac0a1150 0x7f01ac0a3600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f01b0005950 tx=0x7f01b00058e0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.433+0000 7f01c31fe700 1 -- 172.21.15.40:0/2365912059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01bc101690 msgr2=0x7f01bc19cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.433+0000 7f01c31fe700 1 --2- 172.21.15.40:0/2365912059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01bc101690 0x7f01bc19cf20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f01b400d010 tx=0x7f01b4004c80 comp rx=0 tx=0).stop 2023-07-18T03:31:55.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.433+0000 7f01c31fe700 1 -- 172.21.15.40:0/2365912059 shutdown_connections 2023-07-18T03:31:55.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.433+0000 7f01c31fe700 1 --2- 172.21.15.40:0/2365912059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01bc101690 0x7f01bc19cf20 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.433+0000 7f01c31fe700 1 --2- 172.21.15.40:0/2365912059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01bc0ffb10 0x7f01bc195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.433+0000 7f01c31fe700 1 --2- 172.21.15.40:0/2365912059 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f01ac0a1150 0x7f01ac0a3600 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.433+0000 7f01c31fe700 1 --2- 172.21.15.40:0/2365912059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01bc100d00 0x7f01bc195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.433+0000 7f01c31fe700 1 -- 172.21.15.40:0/2365912059 >> 172.21.15.40:0/2365912059 conn(0x7f01bc0fb130 msgr2=0x7f01bc0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.433+0000 7f01c31fe700 1 -- 172.21.15.40:0/2365912059 shutdown_connections 2023-07-18T03:31:55.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.433+0000 7f01c31fe700 1 -- 172.21.15.40:0/2365912059 wait complete. 2023-07-18T03:31:55.446 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:31:55.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.578+0000 7fcf2cddc700 1 -- 172.21.15.40:0/2488421072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf28100560 msgr2=0x7fcf28106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.578+0000 7fcf2cddc700 1 --2- 172.21.15.40:0/2488421072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf28100560 0x7fcf28106e20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000b0a0 tx=0x7fcf2000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.578+0000 7fcf2cddc700 1 -- 172.21.15.40:0/2488421072 shutdown_connections 2023-07-18T03:31:55.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.578+0000 7fcf2cddc700 1 --2- 172.21.15.40:0/2488421072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf28100560 0x7fcf28106e20 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.578+0000 7fcf2cddc700 1 --2- 172.21.15.40:0/2488421072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf280fe9e0 0x7fcf280fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.578+0000 7fcf2cddc700 1 --2- 172.21.15.40:0/2488421072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf280ffbd0 0x7fcf28100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.578+0000 7fcf2cddc700 1 -- 172.21.15.40:0/2488421072 >> 172.21.15.40:0/2488421072 conn(0x7fcf280fa000 msgr2=0x7fcf280fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.578+0000 7fcf2cddc700 1 -- 172.21.15.40:0/2488421072 shutdown_connections 2023-07-18T03:31:55.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.579+0000 7fcf2cddc700 1 -- 172.21.15.40:0/2488421072 wait complete. 2023-07-18T03:31:55.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.579+0000 7fcf2cddc700 1 Processor -- start 2023-07-18T03:31:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.579+0000 7fcf2cddc700 1 -- start start 2023-07-18T03:31:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.579+0000 7fcf2cddc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf280fe9e0 0x7fcf28102970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.579+0000 7fcf2cddc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf280ffbd0 0x7fcf28100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.579+0000 7fcf2cddc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf28100560 0x7fcf281014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.579+0000 7fcf2cddc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28101c90 con 0x7fcf28100560 2023-07-18T03:31:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.579+0000 7fcf2cddc700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28101e00 con 0x7fcf280ffbd0 2023-07-18T03:31:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.579+0000 7fcf2cddc700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2819f9b0 con 0x7fcf280fe9e0 2023-07-18T03:31:55.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.580+0000 7fcf26d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf28100560 0x7fcf281014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.580+0000 7fcf26d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf28100560 0x7fcf281014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58702/0 (socket says 172.21.15.40:58702) 2023-07-18T03:31:55.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.580+0000 7fcf26d9d700 1 -- 172.21.15.40:0/760766129 learned_addr learned my addr 172.21.15.40:0/760766129 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.580+0000 7fcf2659c700 1 --2- 172.21.15.40:0/760766129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf280fe9e0 0x7fcf28102970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.580+0000 7fcf25d9b700 1 --2- 172.21.15.40:0/760766129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf280ffbd0 0x7fcf28100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.580+0000 7fcf26d9d700 1 -- 172.21.15.40:0/760766129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf280fe9e0 msgr2=0x7fcf28102970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.580+0000 7fcf26d9d700 1 --2- 172.21.15.40:0/760766129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf280fe9e0 0x7fcf28102970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.580+0000 7fcf26d9d700 1 -- 172.21.15.40:0/760766129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf280ffbd0 msgr2=0x7fcf28100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.580+0000 7fcf26d9d700 1 --2- 172.21.15.40:0/760766129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf280ffbd0 0x7fcf28100fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.580+0000 7fcf26d9d700 1 -- 172.21.15.40:0/760766129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf18009520 con 0x7fcf28100560 2023-07-18T03:31:55.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.580+0000 7fcf26d9d700 1 --2- 172.21.15.40:0/760766129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf28100560 0x7fcf281014f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000d010 tx=0x7fcf2000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.580+0000 7fcf177fe700 1 -- 172.21.15.40:0/760766129 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000e040 con 0x7fcf28100560 2023-07-18T03:31:55.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.580+0000 7fcf177fe700 1 -- 172.21.15.40:0/760766129 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf2000bdb0 con 0x7fcf28100560 2023-07-18T03:31:55.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.580+0000 7fcf2cddc700 1 -- 172.21.15.40:0/760766129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf20009d00 con 0x7fcf28100560 2023-07-18T03:31:55.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.580+0000 7fcf177fe700 1 -- 172.21.15.40:0/760766129 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf20005260 con 0x7fcf28100560 2023-07-18T03:31:55.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.580+0000 7fcf2cddc700 1 -- 172.21.15.40:0/760766129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2819ff20 con 0x7fcf28100560 2023-07-18T03:31:55.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.583+0000 7fcf177fe700 1 -- 172.21.15.40:0/760766129 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcf2001d480 con 0x7fcf28100560 2023-07-18T03:31:55.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.583+0000 7fcf177fe700 1 --2- 172.21.15.40:0/760766129 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcf100a11d0 0x7fcf100a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.583+0000 7fcf2659c700 1 --2- 172.21.15.40:0/760766129 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcf100a11d0 0x7fcf100a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.583+0000 7fcf177fe700 1 -- 172.21.15.40:0/760766129 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcf200cd970 con 0x7fcf28100560 2023-07-18T03:31:55.585 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether build environment is sane... yes 2023-07-18T03:31:55.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.584+0000 7fcf2659c700 1 --2- 172.21.15.40:0/760766129 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcf100a11d0 0x7fcf100a3680 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcf280ffa30 tx=0x7fcf18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.584+0000 7fcf38f62b80 1 Processor -- start 2023-07-18T03:31:55.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.584+0000 7fcf38f62b80 1 -- start start 2023-07-18T03:31:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf38f62b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98e88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf38f62b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e8e20 0x5645f98e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf38f62b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e9c70 0x5645f98ea0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf38f62b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f97edcb0 con 0x5645f98e84d0 2023-07-18T03:31:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf38f62b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f9840a40 con 0x5645f98e9c70 2023-07-18T03:31:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf38f62b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f9839ed0 con 0x5645f98e8e20 2023-07-18T03:31:55.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf2659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98e88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf2659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98e88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58706/0 (socket says 172.21.15.40:58706) 2023-07-18T03:31:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf2659c700 1 -- 172.21.15.40:0/3134678930 learned_addr learned my addr 172.21.15.40:0/3134678930 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf26d9d700 1 --2- 172.21.15.40:0/3134678930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e9c70 0x5645f98ea0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf2659c700 1 -- 172.21.15.40:0/3134678930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e8e20 msgr2=0x5645f98e96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf2659c700 1 --2- 172.21.15.40:0/3134678930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e8e20 0x5645f98e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf2659c700 1 -- 172.21.15.40:0/3134678930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e9c70 msgr2=0x5645f98ea0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf2659c700 1 --2- 172.21.15.40:0/3134678930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e9c70 0x5645f98ea0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf2659c700 1 -- 172.21.15.40:0/3134678930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645f9667de0 con 0x5645f98e84d0 2023-07-18T03:31:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf26d9d700 1 --2- 172.21.15.40:0/3134678930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e9c70 0x5645f98ea0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:55.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf2659c700 1 --2- 172.21.15.40:0/3134678930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98e88e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcf180042c0 tx=0x7fcf18014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfdbd17af1cb1c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf14ff9700 1 -- 172.21.15.40:0/3134678930 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18003d10 con 0x5645f98e84d0 2023-07-18T03:31:55.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf14ff9700 1 -- 172.21.15.40:0/3134678930 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf18003e70 con 0x5645f98e84d0 2023-07-18T03:31:55.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.585+0000 7fcf14ff9700 1 -- 172.21.15.40:0/3134678930 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18026780 con 0x5645f98e84d0 2023-07-18T03:31:55.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.586+0000 7fcf38f62b80 1 -- 172.21.15.40:0/3134678930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 msgr2=0x5645f98e88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.586+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/3134678930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98e88e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcf180042c0 tx=0x7fcf18014000 comp rx=0 tx=0).stop 2023-07-18T03:31:55.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.586+0000 7fcf38f62b80 1 -- 172.21.15.40:0/3134678930 shutdown_connections 2023-07-18T03:31:55.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.586+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/3134678930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e9c70 0x5645f98ea0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.586+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/3134678930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98e88e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.586+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/3134678930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e8e20 0x5645f98e96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.586+0000 7fcf38f62b80 1 -- 172.21.15.40:0/3134678930 >> 172.21.15.40:0/3134678930 conn(0x5645f98ddda0 msgr2=0x5645f98e31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.586+0000 7fcf38f62b80 1 -- 172.21.15.40:0/3134678930 shutdown_connections 2023-07-18T03:31:55.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.586+0000 7fcf38f62b80 1 -- 172.21.15.40:0/3134678930 wait complete. 2023-07-18T03:31:55.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.587+0000 7fcf38f62b80 1 -- start start 2023-07-18T03:31:55.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.587+0000 7fcf38f62b80 1 Processor -- start 2023-07-18T03:31:55.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.587+0000 7fcf38f62b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98f2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.587+0000 7fcf38f62b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e8e20 0x5645f98f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.587+0000 7fcf38f62b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e9c70 0x5645f98f9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.587+0000 7fcf38f62b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f97d9ac0 con 0x5645f98e84d0 2023-07-18T03:31:55.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.587+0000 7fcf38f62b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f97d97c0 con 0x5645f98e8e20 2023-07-18T03:31:55.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.587+0000 7fcf38f62b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f96bd920 con 0x5645f98e9c70 2023-07-18T03:31:55.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.587+0000 7fcf38f62b80 10 client.0 _ll_register_callbacks cb 0x5645f97d7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:55.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.587+0000 7fcf2659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98f2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.587+0000 7fcf2659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98f2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58712/0 (socket says 172.21.15.40:58712) 2023-07-18T03:31:55.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.587+0000 7fcf2659c700 1 -- 172.21.15.40:0/1221861224 learned_addr learned my addr 172.21.15.40:0/1221861224 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.588+0000 7fcf25d9b700 1 --2- 172.21.15.40:0/1221861224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e8e20 0x5645f98f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.588+0000 7fcf26d9d700 1 --2- 172.21.15.40:0/1221861224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e9c70 0x5645f98f9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.588+0000 7fcf2659c700 1 -- 172.21.15.40:0/1221861224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e9c70 msgr2=0x5645f98f9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.588+0000 7fcf2659c700 1 --2- 172.21.15.40:0/1221861224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e9c70 0x5645f98f9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.588+0000 7fcf2659c700 1 -- 172.21.15.40:0/1221861224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e8e20 msgr2=0x5645f98f26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.588+0000 7fcf2659c700 1 --2- 172.21.15.40:0/1221861224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e8e20 0x5645f98f26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.588+0000 7fcf2659c700 1 -- 172.21.15.40:0/1221861224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf18017070 con 0x5645f98e84d0 2023-07-18T03:31:55.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.588+0000 7fcf2659c700 1 --2- 172.21.15.40:0/1221861224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98f2190 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009fd0 tx=0x7fcf18005460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68fbbd512aa6e058 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.588+0000 7fcf0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:55.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.588+0000 7fcf0dffb700 1 -- 172.21.15.40:0/1221861224 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1802c070 con 0x5645f98e84d0 2023-07-18T03:31:55.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.588+0000 7fcf0dffb700 1 -- 172.21.15.40:0/1221861224 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcf1801e630 con 0x5645f98e84d0 2023-07-18T03:31:55.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.588+0000 7fcf38f62b80 1 -- 172.21.15.40:0/1221861224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645f97934f0 con 0x5645f98e84d0 2023-07-18T03:31:55.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.588+0000 7fcf0dffb700 1 -- 172.21.15.40:0/1221861224 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf180056b0 con 0x5645f98e84d0 2023-07-18T03:31:55.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.588+0000 7fcf38f62b80 10 client.15774 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:55.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.588+0000 7fcf38f62b80 1 -- 172.21.15.40:0/1221861224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5645f9650c60 con 0x5645f98e84d0 2023-07-18T03:31:55.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.588+0000 7fcf0dffb700 1 -- 172.21.15.40:0/1221861224 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5645f97934f0 con 0x5645f98e84d0 2023-07-18T03:31:55.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.589+0000 7fcf0dffb700 1 -- 172.21.15.40:0/1221861224 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcf18024020 con 0x5645f98e84d0 2023-07-18T03:31:55.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.589+0000 7fcf38f62b80 10 client.15774 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:55.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.589+0000 7fcf38f62b80 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:55.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.589+0000 7fcf38f62b80 20 client.15774 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:55.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.589+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/1221861224 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5645f98fddc0 0x5645f99001d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.589+0000 7fcf38f62b80 4 client.15774 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:55.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.589+0000 7fcf38f62b80 1 -- 172.21.15.40:0/1221861224 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645f9832eb0 con 0x5645f98fddc0 2023-07-18T03:31:55.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.589+0000 7fcf25d9b700 1 --2- 172.21.15.40:0/1221861224 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5645f98fddc0 0x5645f99001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.589+0000 7fcf25d9b700 1 --2- 172.21.15.40:0/1221861224 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5645f98fddc0 0x5645f99001d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5b5bedb07370a7a server_cookie=84c43815496d48cc in_seq=0 out_seq=0 2023-07-18T03:31:55.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.589+0000 7fcf0dffb700 10 client.15774 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:55.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.590+0000 7fcf0dffb700 1 -- 172.21.15.40:0/1221861224 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5645f9832eb0 con 0x5645f98fddc0 2023-07-18T03:31:55.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.590+0000 7fcf0dffb700 10 client.15774 handle_command_reply: tid=0 2023-07-18T03:31:55.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.590+0000 7fcf38f62b80 1 client.15774 shutdown 2023-07-18T03:31:55.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.590+0000 7fcf38f62b80 1 -- 172.21.15.40:0/1221861224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 msgr2=0x5645f98f2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.590+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/1221861224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98f2190 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009fd0 tx=0x7fcf18005460 comp rx=0 tx=0).stop 2023-07-18T03:31:55.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.590+0000 7fcf38f62b80 1 -- 172.21.15.40:0/1221861224 shutdown_connections 2023-07-18T03:31:55.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.590+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/1221861224 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5645f98fddc0 0x5645f99001d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.590+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/1221861224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e9c70 0x5645f98f9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.590+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/1221861224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98f2190 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.590+0000 7fcf0dffb700 0 client.15774 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:55.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.590+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/1221861224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e8e20 0x5645f98f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.590+0000 7fcf38f62b80 1 -- 172.21.15.40:0/1221861224 >> 172.21.15.40:0/1221861224 conn(0x5645f98ddda0 msgr2=0x5645f98e28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.590+0000 7fcf38f62b80 1 -- 172.21.15.40:0/1221861224 shutdown_connections 2023-07-18T03:31:55.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.590+0000 7fcf38f62b80 1 -- 172.21.15.40:0/1221861224 wait complete. 2023-07-18T03:31:55.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.591+0000 7fcf38f62b80 20 client.15774 trim_cache size 0 max 16384 2023-07-18T03:31:55.605 INFO:tasks.workunit.client.0.smithi040.stdout:checking for gawk... gawk 2023-07-18T03:31:55.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.604+0000 7fcf38f62b80 1 Processor -- start 2023-07-18T03:31:55.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.604+0000 7fcf38f62b80 1 -- start start 2023-07-18T03:31:55.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.604+0000 7fcf38f62b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98ea040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.604+0000 7fcf38f62b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e8e20 0x5645f98f96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.604+0000 7fcf38f62b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e9c70 0x5645f98f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.604+0000 7fcf38f62b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f98efe60 con 0x5645f98e84d0 2023-07-18T03:31:55.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.604+0000 7fcf38f62b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f98fda90 con 0x5645f98e9c70 2023-07-18T03:31:55.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.604+0000 7fcf38f62b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f98f13f0 con 0x5645f98e8e20 2023-07-18T03:31:55.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.604+0000 7fcf2659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98ea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.604+0000 7fcf2659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98ea040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58720/0 (socket says 172.21.15.40:58720) 2023-07-18T03:31:55.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.604+0000 7fcf2659c700 1 -- 172.21.15.40:0/1082455956 learned_addr learned my addr 172.21.15.40:0/1082455956 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.604+0000 7fcf26d9d700 1 --2- 172.21.15.40:0/1082455956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e9c70 0x5645f98f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.604+0000 7fcf25d9b700 1 --2- 172.21.15.40:0/1082455956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e8e20 0x5645f98f96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.604+0000 7fcf2659c700 1 -- 172.21.15.40:0/1082455956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e8e20 msgr2=0x5645f98f96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.604+0000 7fcf2659c700 1 --2- 172.21.15.40:0/1082455956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e8e20 0x5645f98f96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.605+0000 7fcf2659c700 1 -- 172.21.15.40:0/1082455956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e9c70 msgr2=0x5645f98f0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.605+0000 7fcf2659c700 1 --2- 172.21.15.40:0/1082455956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e9c70 0x5645f98f0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.605+0000 7fcf2659c700 1 -- 172.21.15.40:0/1082455956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645f968e250 con 0x5645f98e84d0 2023-07-18T03:31:55.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.605+0000 7fcf2659c700 1 --2- 172.21.15.40:0/1082455956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98ea040 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcf18003e40 tx=0x7fcf18009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd6b4dd94a10321 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.605+0000 7fcf157fa700 1 -- 172.21.15.40:0/1082455956 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1802c020 con 0x5645f98e84d0 2023-07-18T03:31:55.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.605+0000 7fcf157fa700 1 -- 172.21.15.40:0/1082455956 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf1801eaf0 con 0x5645f98e84d0 2023-07-18T03:31:55.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.605+0000 7fcf157fa700 1 -- 172.21.15.40:0/1082455956 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18004680 con 0x5645f98e84d0 2023-07-18T03:31:55.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 1 -- 172.21.15.40:0/1082455956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 msgr2=0x5645f98ea040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/1082455956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98ea040 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcf18003e40 tx=0x7fcf18009b70 comp rx=0 tx=0).stop 2023-07-18T03:31:55.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 1 -- 172.21.15.40:0/1082455956 shutdown_connections 2023-07-18T03:31:55.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/1082455956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e9c70 0x5645f98f0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/1082455956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98ea040 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/1082455956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e8e20 0x5645f98f96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 1 -- 172.21.15.40:0/1082455956 >> 172.21.15.40:0/1082455956 conn(0x5645f98ddda0 msgr2=0x5645f9904af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 1 -- 172.21.15.40:0/1082455956 shutdown_connections 2023-07-18T03:31:55.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 1 -- 172.21.15.40:0/1082455956 wait complete. 2023-07-18T03:31:55.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 1 -- start start 2023-07-18T03:31:55.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 1 Processor -- start 2023-07-18T03:31:55.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98f5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e8e20 0x5645f98f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e9c70 0x5645f98f89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f98f8f10 con 0x5645f98e84d0 2023-07-18T03:31:55.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f98f9080 con 0x5645f98e8e20 2023-07-18T03:31:55.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645f98f9240 con 0x5645f98e9c70 2023-07-18T03:31:55.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf38f62b80 10 client.0 _ll_register_callbacks cb 0x7fcf18004190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:55.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf2659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf2659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98f5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58726/0 (socket says 172.21.15.40:58726) 2023-07-18T03:31:55.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.606+0000 7fcf2659c700 1 -- 172.21.15.40:0/2423428314 learned_addr learned my addr 172.21.15.40:0/2423428314 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:55.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf26d9d700 1 --2- 172.21.15.40:0/2423428314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e9c70 0x5645f98f89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf25d9b700 1 --2- 172.21.15.40:0/2423428314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e8e20 0x5645f98f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf2659c700 1 -- 172.21.15.40:0/2423428314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e9c70 msgr2=0x5645f98f89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf2659c700 1 --2- 172.21.15.40:0/2423428314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e9c70 0x5645f98f89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf2659c700 1 -- 172.21.15.40:0/2423428314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e8e20 msgr2=0x5645f98f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:55.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf2659c700 1 --2- 172.21.15.40:0/2423428314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e8e20 0x5645f98f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf2659c700 1 -- 172.21.15.40:0/2423428314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645f9650c60 con 0x5645f98e84d0 2023-07-18T03:31:55.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf26d9d700 1 --2- 172.21.15.40:0/2423428314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e9c70 0x5645f98f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:55.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf2659c700 1 --2- 172.21.15.40:0/2423428314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98f5470 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fcf18026c80 tx=0x7fcf18004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a178084118ee589b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:55.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:55.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf0dffb700 1 -- 172.21.15.40:0/2423428314 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1802c030 con 0x5645f98e84d0 2023-07-18T03:31:55.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf0dffb700 1 -- 172.21.15.40:0/2423428314 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcf1801ee40 con 0x5645f98e84d0 2023-07-18T03:31:55.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf38f62b80 1 -- 172.21.15.40:0/2423428314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf18003e70 con 0x5645f98e84d0 2023-07-18T03:31:55.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf0dffb700 1 -- 172.21.15.40:0/2423428314 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf1801e830 con 0x5645f98e84d0 2023-07-18T03:31:55.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf38f62b80 10 client.15780 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:55.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf38f62b80 1 -- 172.21.15.40:0/2423428314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5645f96662b0 con 0x5645f98e84d0 2023-07-18T03:31:55.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf0dffb700 1 -- 172.21.15.40:0/2423428314 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf18003e70 con 0x5645f98e84d0 2023-07-18T03:31:55.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf0dffb700 1 -- 172.21.15.40:0/2423428314 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcf18048050 con 0x5645f98e84d0 2023-07-18T03:31:55.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.607+0000 7fcf38f62b80 10 client.15780 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:55.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.608+0000 7fcf38f62b80 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:55.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.608+0000 7fcf38f62b80 20 client.15780 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:55.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.608+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/2423428314 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5645f9952690 0x5645f9954aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:55.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.608+0000 7fcf38f62b80 4 client.15780 mds_command: new command op to 24527 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:31:55.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.608+0000 7fcf38f62b80 1 -- 172.21.15.40:0/2423428314 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5645f97f5050 con 0x5645f9952690 2023-07-18T03:31:55.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.608+0000 7fcf25d9b700 1 --2- 172.21.15.40:0/2423428314 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5645f9952690 0x5645f9954aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:55.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.608+0000 7fcf0dffb700 10 client.15780 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:55.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.608+0000 7fcf25d9b700 1 --2- 172.21.15.40:0/2423428314 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5645f9952690 0x5645f9954aa0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fea98dcd6096fd42 server_cookie=cdd930a224ae5877 in_seq=0 out_seq=0 2023-07-18T03:31:55.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf0dffb700 1 -- 172.21.15.40:0/2423428314 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5645f97f5050 con 0x5645f9952690 2023-07-18T03:31:55.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf0dffb700 10 client.15780 handle_command_reply: tid=0 2023-07-18T03:31:55.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf38f62b80 1 client.15780 shutdown 2023-07-18T03:31:55.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf38f62b80 1 -- 172.21.15.40:0/2423428314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 msgr2=0x5645f98f5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/2423428314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98f5470 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fcf18026c80 tx=0x7fcf18004f40 comp rx=0 tx=0).stop 2023-07-18T03:31:55.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf38f62b80 1 -- 172.21.15.40:0/2423428314 shutdown_connections 2023-07-18T03:31:55.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/2423428314 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5645f9952690 0x5645f9954aa0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/2423428314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5645f98e9c70 0x5645f98f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/2423428314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5645f98e84d0 0x5645f98f5470 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf38f62b80 1 --2- 172.21.15.40:0/2423428314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5645f98e8e20 0x5645f98f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf0dffb700 0 client.15780 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:55.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf38f62b80 1 -- 172.21.15.40:0/2423428314 >> 172.21.15.40:0/2423428314 conn(0x5645f98ddda0 msgr2=0x5645f98f0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf38f62b80 1 -- 172.21.15.40:0/2423428314 shutdown_connections 2023-07-18T03:31:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf38f62b80 1 -- 172.21.15.40:0/2423428314 wait complete. 2023-07-18T03:31:55.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf38f62b80 20 client.15780 trim_cache size 0 max 16384 2023-07-18T03:31:55.631 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:31:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf2cddc700 1 -- 172.21.15.40:0/760766129 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcf100a11d0 msgr2=0x7fcf100a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf2cddc700 1 --2- 172.21.15.40:0/760766129 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcf100a11d0 0x7fcf100a3680 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcf280ffa30 tx=0x7fcf18009260 comp rx=0 tx=0).stop 2023-07-18T03:31:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf2cddc700 1 -- 172.21.15.40:0/760766129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf28100560 msgr2=0x7fcf281014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf2cddc700 1 --2- 172.21.15.40:0/760766129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf28100560 0x7fcf281014f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000d010 tx=0x7fcf2000bc80 comp rx=0 tx=0).stop 2023-07-18T03:31:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf2cddc700 1 -- 172.21.15.40:0/760766129 shutdown_connections 2023-07-18T03:31:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf2cddc700 1 --2- 172.21.15.40:0/760766129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf28100560 0x7fcf281014f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf2cddc700 1 --2- 172.21.15.40:0/760766129 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcf100a11d0 0x7fcf100a3680 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf2cddc700 1 --2- 172.21.15.40:0/760766129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf280fe9e0 0x7fcf28102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf2cddc700 1 --2- 172.21.15.40:0/760766129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf280ffbd0 0x7fcf28100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf2cddc700 1 -- 172.21.15.40:0/760766129 >> 172.21.15.40:0/760766129 conn(0x7fcf280fa000 msgr2=0x7fcf28067670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf2cddc700 1 -- 172.21.15.40:0/760766129 shutdown_connections 2023-07-18T03:31:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:55.609+0000 7fcf2cddc700 1 -- 172.21.15.40:0/760766129 wait complete. 2023-07-18T03:31:55.637 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether make sets $(MAKE)... yes 2023-07-18T03:31:55.648 INFO:tasks.workunit.client.0.smithi040.stdout:checking for gcc... gcc 2023-07-18T03:31:55.699 INFO:tasks.workunit.client.0.smithi040.stdout:checking for C compiler default output file name... a.out 2023-07-18T03:31:55.700 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether the C compiler works... yes 2023-07-18T03:31:55.703 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether we are cross compiling... no 2023-07-18T03:31:55.740 INFO:tasks.workunit.client.0.smithi040.stdout:checking for suffix of executables... 2023-07-18T03:31:55.776 INFO:tasks.workunit.client.0.smithi040.stdout:checking for suffix of object files... o 2023-07-18T03:31:55.807 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether we are using the GNU C compiler... yes 2023-07-18T03:31:55.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:55 smithi040 ceph-mon[96439]: pgmap v324: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-18T03:31:55.830 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:55 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/885278023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:31:55.837 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether gcc accepts -g... yes 2023-07-18T03:31:55.876 INFO:tasks.workunit.client.0.smithi040.stdout:checking for gcc option to accept ANSI C... none needed 2023-07-18T03:31:55.920 INFO:tasks.workunit.client.0.smithi040.stdout:checking for style of include used by make... GNU 2023-07-18T03:31:56.024 INFO:tasks.workunit.client.0.smithi040.stdout:checking dependency style of gcc... gcc3 2023-07-18T03:31:56.025 INFO:tasks.workunit.client.0.smithi040.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-07-18T03:31:56.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:55 smithi134 ceph-mon[98216]: pgmap v324: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-18T03:31:56.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:55 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/885278023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:31:56.076 INFO:tasks.workunit.client.0.smithi040.stdout:checking how to run the C preprocessor... gcc -E 2023-07-18T03:31:56.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:55 smithi064 ceph-mon[98059]: pgmap v324: 97 pgs: 97 active+clean; 664 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2023-07-18T03:31:56.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:55 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/885278023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:31:56.108 INFO:tasks.workunit.client.0.smithi040.stdout:checking for egrep... grep -E 2023-07-18T03:31:56.114 INFO:tasks.workunit.client.0.smithi040.stdout:checking for AIX... no 2023-07-18T03:31:56.179 INFO:tasks.workunit.client.0.smithi040.stdout:checking for library containing strerror... none required 2023-07-18T03:31:56.309 INFO:tasks.workunit.client.0.smithi040.stdout:checking for ANSI C header files... yes 2023-07-18T03:31:56.345 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether time.h and sys/time.h may both be included... yes 2023-07-18T03:31:56.392 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/types.h... yes 2023-07-18T03:31:56.440 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/stat.h... yes 2023-07-18T03:31:56.487 INFO:tasks.workunit.client.0.smithi040.stdout:checking for stdlib.h... yes 2023-07-18T03:31:56.535 INFO:tasks.workunit.client.0.smithi040.stdout:checking for string.h... yes 2023-07-18T03:31:56.585 INFO:tasks.workunit.client.0.smithi040.stdout:checking for memory.h... yes 2023-07-18T03:31:56.633 INFO:tasks.workunit.client.0.smithi040.stdout:checking for strings.h... yes 2023-07-18T03:31:56.638 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:31:56.638 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:31:56.691 INFO:tasks.workunit.client.0.smithi040.stdout:checking for inttypes.h... yes 2023-07-18T03:31:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.720+0000 7f3e54080700 1 -- 172.21.15.40:0/4165107697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e4c0ffc50 msgr2=0x7f3e4c1000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.720+0000 7f3e54080700 1 --2- 172.21.15.40:0/4165107697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e4c0ffc50 0x7f3e4c1000a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009990 tx=0x7f3e3c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.720+0000 7f3e54080700 1 -- 172.21.15.40:0/4165107697 shutdown_connections 2023-07-18T03:31:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.720+0000 7f3e54080700 1 --2- 172.21.15.40:0/4165107697 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e4c1005e0 0x7f3e4c106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.720+0000 7f3e54080700 1 --2- 172.21.15.40:0/4165107697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e4c0fea60 0x7f3e4c0fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.720+0000 7f3e54080700 1 --2- 172.21.15.40:0/4165107697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e4c0ffc50 0x7f3e4c1000a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.720+0000 7f3e54080700 1 -- 172.21.15.40:0/4165107697 >> 172.21.15.40:0/4165107697 conn(0x7f3e4c0fa060 msgr2=0x7f3e4c0fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.720+0000 7f3e54080700 1 -- 172.21.15.40:0/4165107697 shutdown_connections 2023-07-18T03:31:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.720+0000 7f3e54080700 1 -- 172.21.15.40:0/4165107697 wait complete. 2023-07-18T03:31:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.721+0000 7f3e54080700 1 Processor -- start 2023-07-18T03:31:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.721+0000 7f3e54080700 1 -- start start 2023-07-18T03:31:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.721+0000 7f3e54080700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e4c0fea60 0x7f3e4c195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.721+0000 7f3e54080700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e4c0ffc50 0x7f3e4c195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.721+0000 7f3e54080700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e4c1005e0 0x7f3e4c196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.721+0000 7f3e54080700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c197350 con 0x7f3e4c0fea60 2023-07-18T03:31:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.721+0000 7f3e54080700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c1974c0 con 0x7f3e4c0ffc50 2023-07-18T03:31:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.721+0000 7f3e54080700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4c197680 con 0x7f3e4c1005e0 2023-07-18T03:31:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.721+0000 7f3e51e1c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e4c0fea60 0x7f3e4c195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.721+0000 7f3e51e1c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e4c0fea60 0x7f3e4c195930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58740/0 (socket says 172.21.15.40:58740) 2023-07-18T03:31:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.721+0000 7f3e5161b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e4c0ffc50 0x7f3e4c195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.721+0000 7f3e51e1c700 1 -- 172.21.15.40:0/2891106864 learned_addr learned my addr 172.21.15.40:0/2891106864 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.722+0000 7f3e5261d700 1 --2- 172.21.15.40:0/2891106864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e4c1005e0 0x7f3e4c196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.722+0000 7f3e5161b700 1 -- 172.21.15.40:0/2891106864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e4c1005e0 msgr2=0x7f3e4c196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.722+0000 7f3e5161b700 1 --2- 172.21.15.40:0/2891106864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e4c1005e0 0x7f3e4c196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.722+0000 7f3e5161b700 1 -- 172.21.15.40:0/2891106864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e4c0fea60 msgr2=0x7f3e4c195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.722+0000 7f3e5161b700 1 --2- 172.21.15.40:0/2891106864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e4c0fea60 0x7f3e4c195930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.722+0000 7f3e5161b700 1 -- 172.21.15.40:0/2891106864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e48009520 con 0x7f3e4c0ffc50 2023-07-18T03:31:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.722+0000 7f3e5161b700 1 --2- 172.21.15.40:0/2891106864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e4c0ffc50 0x7f3e4c195e70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0059d0 tx=0x7f3e3c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.722+0000 7f3e51e1c700 1 --2- 172.21.15.40:0/2891106864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e4c0fea60 0x7f3e4c195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.722+0000 7f3e42ffd700 1 -- 172.21.15.40:0/2891106864 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c01d070 con 0x7f3e4c0ffc50 2023-07-18T03:31:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.722+0000 7f3e42ffd700 1 -- 172.21.15.40:0/2891106864 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e3c022470 con 0x7f3e4c0ffc50 2023-07-18T03:31:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.722+0000 7f3e42ffd700 1 -- 172.21.15.40:0/2891106864 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c017650 con 0x7f3e4c0ffc50 2023-07-18T03:31:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.722+0000 7f3e54080700 1 -- 172.21.15.40:0/2891106864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3c0095f0 con 0x7f3e4c0ffc50 2023-07-18T03:31:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.723+0000 7f3e54080700 1 -- 172.21.15.40:0/2891106864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4c19df40 con 0x7f3e4c0ffc50 2023-07-18T03:31:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.725+0000 7f3e42ffd700 1 -- 172.21.15.40:0/2891106864 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3e3c0225e0 con 0x7f3e4c0ffc50 2023-07-18T03:31:56.725 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.725+0000 7f3e42ffd700 1 --2- 172.21.15.40:0/2891106864 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3e380a12a0 0x7f3e380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.726+0000 7f3e51e1c700 1 --2- 172.21.15.40:0/2891106864 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3e380a12a0 0x7f3e380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.726+0000 7f3e42ffd700 1 -- 172.21.15.40:0/2891106864 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3e3c0cdce0 con 0x7f3e4c0ffc50 2023-07-18T03:31:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.726+0000 7f3e51e1c700 1 --2- 172.21.15.40:0/2891106864 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3e380a12a0 0x7f3e380a3750 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009fd0 tx=0x7f3e48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.727+0000 7f3e60206b80 1 Processor -- start 2023-07-18T03:31:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.727+0000 7f3e60206b80 1 -- start start 2023-07-18T03:31:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e60206b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0ff886b0 0x555c101306d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e60206b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c10130c10 0x555c101315d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e60206b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10131b10 0x555c10131fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e60206b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c10031c50 con 0x555c10131b10 2023-07-18T03:31:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e60206b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c100849e0 con 0x555c0ff886b0 2023-07-18T03:31:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e60206b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c1007de70 con 0x555c10130c10 2023-07-18T03:31:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e5261d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10131b10 0x555c10131fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e51e1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0ff886b0 0x555c101306d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e5261d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10131b10 0x555c10131fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58752/0 (socket says 172.21.15.40:58752) 2023-07-18T03:31:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e51e1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0ff886b0 0x555c101306d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42074/0 (socket says 172.21.15.40:42074) 2023-07-18T03:31:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e5261d700 1 -- 172.21.15.40:0/899650497 learned_addr learned my addr 172.21.15.40:0/899650497 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e5161b700 1 --2- 172.21.15.40:0/899650497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c10130c10 0x555c101315d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e5261d700 1 -- 172.21.15.40:0/899650497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c10130c10 msgr2=0x555c101315d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e5261d700 1 --2- 172.21.15.40:0/899650497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c10130c10 0x555c101315d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e5261d700 1 -- 172.21.15.40:0/899650497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0ff886b0 msgr2=0x555c101306d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e5261d700 1 --2- 172.21.15.40:0/899650497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0ff886b0 0x555c101306d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e5261d700 1 -- 172.21.15.40:0/899650497 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c0feabd80 con 0x555c10131b10 2023-07-18T03:31:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e51e1c700 1 --2- 172.21.15.40:0/899650497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0ff886b0 0x555c101306d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e5161b700 1 --2- 172.21.15.40:0/899650497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c10130c10 0x555c101315d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.728+0000 7f3e5261d700 1 --2- 172.21.15.40:0/899650497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10131b10 0x555c10131fb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009970 tx=0x7f3e44009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509ddc06153a0506 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.729+0000 7f3e37fff700 1 -- 172.21.15.40:0/899650497 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e44019070 con 0x555c10131b10 2023-07-18T03:31:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.729+0000 7f3e37fff700 1 -- 172.21.15.40:0/899650497 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e4400f040 con 0x555c10131b10 2023-07-18T03:31:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.729+0000 7f3e37fff700 1 -- 172.21.15.40:0/899650497 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e44015480 con 0x555c10131b10 2023-07-18T03:31:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.729+0000 7f3e60206b80 1 -- 172.21.15.40:0/899650497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10131b10 msgr2=0x555c10131fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.729+0000 7f3e60206b80 1 --2- 172.21.15.40:0/899650497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10131b10 0x555c10131fb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009970 tx=0x7f3e44009c80 comp rx=0 tx=0).stop 2023-07-18T03:31:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.729+0000 7f3e60206b80 1 -- 172.21.15.40:0/899650497 shutdown_connections 2023-07-18T03:31:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.729+0000 7f3e60206b80 1 --2- 172.21.15.40:0/899650497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10131b10 0x555c10131fb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.729+0000 7f3e60206b80 1 --2- 172.21.15.40:0/899650497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0ff886b0 0x555c101306d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.729+0000 7f3e60206b80 1 --2- 172.21.15.40:0/899650497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c10130c10 0x555c101315d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.729+0000 7f3e60206b80 1 -- 172.21.15.40:0/899650497 >> 172.21.15.40:0/899650497 conn(0x555c10121be0 msgr2=0x555c10126cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.730+0000 7f3e60206b80 1 -- 172.21.15.40:0/899650497 shutdown_connections 2023-07-18T03:31:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.730+0000 7f3e60206b80 1 -- 172.21.15.40:0/899650497 wait complete. 2023-07-18T03:31:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.730+0000 7f3e60206b80 1 -- start start 2023-07-18T03:31:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.730+0000 7f3e60206b80 1 Processor -- start 2023-07-18T03:31:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.730+0000 7f3e60206b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0ff886b0 0x555c10139c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.730+0000 7f3e60206b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10130c10 0x555c1013a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.730+0000 7f3e60206b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c10131b10 0x555c101411d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.730+0000 7f3e60206b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c1001da60 con 0x555c10130c10 2023-07-18T03:31:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.730+0000 7f3e60206b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c1001d760 con 0x555c0ff886b0 2023-07-18T03:31:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.730+0000 7f3e60206b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0ff018c0 con 0x555c10131b10 2023-07-18T03:31:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.731+0000 7f3e60206b80 10 client.0 _ll_register_callbacks cb 0x555c1001b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.731+0000 7f3e5161b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10130c10 0x555c1013a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.731+0000 7f3e5161b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10130c10 0x555c1013a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58756/0 (socket says 172.21.15.40:58756) 2023-07-18T03:31:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.731+0000 7f3e5161b700 1 -- 172.21.15.40:0/2822925120 learned_addr learned my addr 172.21.15.40:0/2822925120 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.731+0000 7f3e51e1c700 1 --2- 172.21.15.40:0/2822925120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0ff886b0 0x555c10139c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.731+0000 7f3e5261d700 1 --2- 172.21.15.40:0/2822925120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c10131b10 0x555c101411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.731+0000 7f3e5161b700 1 -- 172.21.15.40:0/2822925120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c10131b10 msgr2=0x555c101411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.731+0000 7f3e5161b700 1 --2- 172.21.15.40:0/2822925120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c10131b10 0x555c101411d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.731+0000 7f3e5161b700 1 -- 172.21.15.40:0/2822925120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0ff886b0 msgr2=0x555c10139c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.731+0000 7f3e5161b700 1 --2- 172.21.15.40:0/2822925120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0ff886b0 0x555c10139c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.731+0000 7f3e5161b700 1 -- 172.21.15.40:0/2822925120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e48017050 con 0x555c10130c10 2023-07-18T03:31:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.731+0000 7f3e5161b700 1 --2- 172.21.15.40:0/2822925120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10130c10 0x555c1013a190 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c097690 tx=0x7f3e3c022bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53aea7148ffd25dc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.731+0000 7f3e357fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.731+0000 7f3e357fa700 1 -- 172.21.15.40:0/2822925120 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0b2070 con 0x555c10130c10 2023-07-18T03:31:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.731+0000 7f3e357fa700 1 -- 172.21.15.40:0/2822925120 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3e3c0a3550 con 0x555c10130c10 2023-07-18T03:31:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.731+0000 7f3e60206b80 1 -- 172.21.15.40:0/2822925120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c0ffd7490 con 0x555c10130c10 2023-07-18T03:31:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.732+0000 7f3e357fa700 1 -- 172.21.15.40:0/2822925120 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e3c09a680 con 0x555c10130c10 2023-07-18T03:31:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.732+0000 7f3e60206b80 10 client.15792 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.732+0000 7f3e60206b80 1 -- 172.21.15.40:0/2822925120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f3e44003ca0 con 0x555c10130c10 2023-07-18T03:31:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.732+0000 7f3e357fa700 1 -- 172.21.15.40:0/2822925120 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c0ffd7490 con 0x555c10130c10 2023-07-18T03:31:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.732+0000 7f3e357fa700 1 -- 172.21.15.40:0/2822925120 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3e3c0b3ae0 con 0x555c10130c10 2023-07-18T03:31:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.732+0000 7f3e60206b80 10 client.15792 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.732+0000 7f3e60206b80 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.732+0000 7f3e60206b80 20 client.15792 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.732+0000 7f3e60206b80 1 --2- 172.21.15.40:0/2822925120 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555c10145660 0x555c10147a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.732+0000 7f3e60206b80 4 client.15792 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.732+0000 7f3e60206b80 1 -- 172.21.15.40:0/2822925120 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c10076e50 con 0x555c10145660 2023-07-18T03:31:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.732+0000 7f3e51e1c700 1 --2- 172.21.15.40:0/2822925120 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555c10145660 0x555c10147a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.733+0000 7f3e51e1c700 1 --2- 172.21.15.40:0/2822925120 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555c10145660 0x555c10147a70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=252d4f006e86c114 server_cookie=63b46be6a55fd2a5 in_seq=0 out_seq=0 2023-07-18T03:31:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.733+0000 7f3e357fa700 10 client.15792 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.733+0000 7f3e357fa700 1 -- 172.21.15.40:0/2822925120 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555c10076e50 con 0x555c10145660 2023-07-18T03:31:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.733+0000 7f3e357fa700 10 client.15792 handle_command_reply: tid=0 2023-07-18T03:31:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.733+0000 7f3e60206b80 1 client.15792 shutdown 2023-07-18T03:31:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.733+0000 7f3e60206b80 1 -- 172.21.15.40:0/2822925120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10130c10 msgr2=0x555c1013a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.733+0000 7f3e60206b80 1 --2- 172.21.15.40:0/2822925120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10130c10 0x555c1013a190 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c097690 tx=0x7f3e3c022bc0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.733+0000 7f3e60206b80 1 -- 172.21.15.40:0/2822925120 shutdown_connections 2023-07-18T03:31:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.733+0000 7f3e60206b80 1 --2- 172.21.15.40:0/2822925120 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555c10145660 0x555c10147a70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.734+0000 7f3e60206b80 1 --2- 172.21.15.40:0/2822925120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c10131b10 0x555c101411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.734+0000 7f3e60206b80 1 --2- 172.21.15.40:0/2822925120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c0ff886b0 0x555c10139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.734+0000 7f3e60206b80 1 --2- 172.21.15.40:0/2822925120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10130c10 0x555c1013a190 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.734+0000 7f3e60206b80 1 -- 172.21.15.40:0/2822925120 >> 172.21.15.40:0/2822925120 conn(0x555c10121be0 msgr2=0x555c10126c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.734+0000 7f3e357fa700 0 client.15792 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.734+0000 7f3e60206b80 1 -- 172.21.15.40:0/2822925120 shutdown_connections 2023-07-18T03:31:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.734+0000 7f3e60206b80 1 -- 172.21.15.40:0/2822925120 wait complete. 2023-07-18T03:31:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.734+0000 7f3e60206b80 20 client.15792 trim_cache size 0 max 16384 2023-07-18T03:31:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e60206b80 1 Processor -- start 2023-07-18T03:31:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e60206b80 1 -- start start 2023-07-18T03:31:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e60206b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0ff886b0 0x555c10139c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e60206b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c10130c10 0x555c1013a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e60206b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10131b10 0x555c101410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e60206b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c10141630 con 0x555c10131b10 2023-07-18T03:31:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e60206b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c1013ab50 con 0x555c10130c10 2023-07-18T03:31:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e60206b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c10139130 con 0x555c0ff886b0 2023-07-18T03:31:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e5161b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c10130c10 0x555c1013a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e5261d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10131b10 0x555c101410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e5161b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c10130c10 0x555c1013a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42090/0 (socket says 172.21.15.40:42090) 2023-07-18T03:31:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e5261d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10131b10 0x555c101410f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58766/0 (socket says 172.21.15.40:58766) 2023-07-18T03:31:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e5161b700 1 -- 172.21.15.40:0/1596389266 learned_addr learned my addr 172.21.15.40:0/1596389266 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e51e1c700 1 --2- 172.21.15.40:0/1596389266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0ff886b0 0x555c10139c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e5161b700 1 -- 172.21.15.40:0/1596389266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0ff886b0 msgr2=0x555c10139c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e5161b700 1 --2- 172.21.15.40:0/1596389266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0ff886b0 0x555c10139c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e5161b700 1 -- 172.21.15.40:0/1596389266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10131b10 msgr2=0x555c101410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e5161b700 1 --2- 172.21.15.40:0/1596389266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10131b10 0x555c101410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.749+0000 7f3e5161b700 1 -- 172.21.15.40:0/1596389266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c0fe94c00 con 0x555c10130c10 2023-07-18T03:31:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.750+0000 7f3e5161b700 1 --2- 172.21.15.40:0/1596389266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c10130c10 0x555c1013a150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c022890 tx=0x7f3e3c099740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a1928148ac8c1e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.750+0000 7f3e40ff9700 1 -- 172.21.15.40:0/1596389266 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0b2030 con 0x555c10130c10 2023-07-18T03:31:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.750+0000 7f3e40ff9700 1 -- 172.21.15.40:0/1596389266 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e3c0b3e60 con 0x555c10130c10 2023-07-18T03:31:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.750+0000 7f3e40ff9700 1 -- 172.21.15.40:0/1596389266 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c09a680 con 0x555c10130c10 2023-07-18T03:31:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.750+0000 7f3e60206b80 1 -- 172.21.15.40:0/1596389266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c10130c10 msgr2=0x555c1013a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.750+0000 7f3e60206b80 1 --2- 172.21.15.40:0/1596389266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c10130c10 0x555c1013a150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c022890 tx=0x7f3e3c099740 comp rx=0 tx=0).stop 2023-07-18T03:31:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.751+0000 7f3e60206b80 1 -- 172.21.15.40:0/1596389266 shutdown_connections 2023-07-18T03:31:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.751+0000 7f3e60206b80 1 --2- 172.21.15.40:0/1596389266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c10131b10 0x555c101410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.751+0000 7f3e60206b80 1 --2- 172.21.15.40:0/1596389266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c0ff886b0 0x555c10139c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.751+0000 7f3e60206b80 1 --2- 172.21.15.40:0/1596389266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c10130c10 0x555c1013a150 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.751+0000 7f3e60206b80 1 -- 172.21.15.40:0/1596389266 >> 172.21.15.40:0/1596389266 conn(0x555c10121be0 msgr2=0x555c101348e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.751+0000 7f3e60206b80 1 -- 172.21.15.40:0/1596389266 shutdown_connections 2023-07-18T03:31:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.751+0000 7f3e60206b80 1 -- 172.21.15.40:0/1596389266 wait complete. 2023-07-18T03:31:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.751+0000 7f3e60206b80 1 -- start start 2023-07-18T03:31:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.751+0000 7f3e60206b80 1 Processor -- start 2023-07-18T03:31:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.751+0000 7f3e60206b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c0ff886b0 0x555c1013cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.751+0000 7f3e60206b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c10130c10 0x555c1013d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.751+0000 7f3e60206b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c10131b10 0x555c10140050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.751+0000 7f3e60206b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c10140590 con 0x555c0ff886b0 2023-07-18T03:31:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.751+0000 7f3e60206b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c10140700 con 0x555c10130c10 2023-07-18T03:31:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.751+0000 7f3e60206b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c101408c0 con 0x555c10131b10 2023-07-18T03:31:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.751+0000 7f3e60206b80 10 client.0 _ll_register_callbacks cb 0x7f3e3c09acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.752+0000 7f3e51e1c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c0ff886b0 0x555c1013cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.752+0000 7f3e51e1c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c0ff886b0 0x555c1013cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58768/0 (socket says 172.21.15.40:58768) 2023-07-18T03:31:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.752+0000 7f3e51e1c700 1 -- 172.21.15.40:0/3998085022 learned_addr learned my addr 172.21.15.40:0/3998085022 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.752+0000 7f3e5161b700 1 --2- 172.21.15.40:0/3998085022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c10130c10 0x555c1013d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.754 INFO:tasks.workunit.client.0.smithi040.stdout:checking for stdint.h... yes 2023-07-18T03:31:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.752+0000 7f3e5261d700 1 --2- 172.21.15.40:0/3998085022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c10131b10 0x555c10140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.752+0000 7f3e5161b700 1 -- 172.21.15.40:0/3998085022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c10131b10 msgr2=0x555c10140050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.752+0000 7f3e5161b700 1 --2- 172.21.15.40:0/3998085022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c10131b10 0x555c10140050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.756 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.752+0000 7f3e5161b700 1 -- 172.21.15.40:0/3998085022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c0ff886b0 msgr2=0x555c1013cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.752+0000 7f3e5161b700 1 --2- 172.21.15.40:0/3998085022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c0ff886b0 0x555c1013cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.752+0000 7f3e5161b700 1 -- 172.21.15.40:0/3998085022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e44003ca0 con 0x555c10130c10 2023-07-18T03:31:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.752+0000 7f3e51e1c700 1 --2- 172.21.15.40:0/3998085022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c0ff886b0 0x555c1013cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.752+0000 7f3e5161b700 1 --2- 172.21.15.40:0/3998085022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c10130c10 0x555c1013d060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c09b120 tx=0x7f3e3c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=182e420da92c9c81 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.752+0000 7f3e357fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:31:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.752+0000 7f3e357fa700 1 -- 172.21.15.40:0/3998085022 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0b2070 con 0x555c10130c10 2023-07-18T03:31:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.752+0000 7f3e357fa700 1 -- 172.21.15.40:0/3998085022 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3e3c0b3070 con 0x555c10130c10 2023-07-18T03:31:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.752+0000 7f3e60206b80 1 -- 172.21.15.40:0/3998085022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e4400f040 con 0x555c10130c10 2023-07-18T03:31:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.752+0000 7f3e357fa700 1 -- 172.21.15.40:0/3998085022 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e3c0a3550 con 0x555c10130c10 2023-07-18T03:31:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.753+0000 7f3e60206b80 10 client.25460 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:56.758 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.753+0000 7f3e60206b80 1 -- 172.21.15.40:0/3998085022 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555c0fed21f0 con 0x555c10130c10 2023-07-18T03:31:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.753+0000 7f3e357fa700 1 -- 172.21.15.40:0/3998085022 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e4400f040 con 0x555c10130c10 2023-07-18T03:31:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.753+0000 7f3e357fa700 1 -- 172.21.15.40:0/3998085022 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3e3c09a0f0 con 0x555c10130c10 2023-07-18T03:31:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.753+0000 7f3e60206b80 10 client.25460 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.753+0000 7f3e60206b80 10 client.25460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.753+0000 7f3e60206b80 20 client.25460 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.753+0000 7f3e60206b80 1 --2- 172.21.15.40:0/3998085022 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555c10199af0 0x555c1019bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.753+0000 7f3e60206b80 4 client.25460 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:31:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.753+0000 7f3e60206b80 1 -- 172.21.15.40:0/3998085022 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555c10038ff0 con 0x555c10199af0 2023-07-18T03:31:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.753+0000 7f3e51e1c700 1 --2- 172.21.15.40:0/3998085022 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555c10199af0 0x555c1019bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.753+0000 7f3e357fa700 10 client.25460 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.754+0000 7f3e51e1c700 1 --2- 172.21.15.40:0/3998085022 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555c10199af0 0x555c1019bed0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=250a86b697fafd8c server_cookie=4b5568037aeb2f62 in_seq=0 out_seq=0 2023-07-18T03:31:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.799+0000 7fbe4f52b700 1 -- 172.21.15.40:0/4038922142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe480fea40 msgr2=0x7fbe480feeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.799+0000 7fbe4f52b700 1 --2- 172.21.15.40:0/4038922142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe480fea40 0x7fbe480feeb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009990 tx=0x7fbe44009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.800+0000 7fbe4f52b700 1 -- 172.21.15.40:0/4038922142 shutdown_connections 2023-07-18T03:31:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.800+0000 7fbe4f52b700 1 --2- 172.21.15.40:0/4038922142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe480ff3f0 0x7fbe48103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.800+0000 7fbe4f52b700 1 --2- 172.21.15.40:0/4038922142 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe480fd910 0x7fbe480fdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.800+0000 7fbe4f52b700 1 --2- 172.21.15.40:0/4038922142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe480fea40 0x7fbe480feeb0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.800+0000 7fbe4f52b700 1 -- 172.21.15.40:0/4038922142 >> 172.21.15.40:0/4038922142 conn(0x7fbe480f92c0 msgr2=0x7fbe480fb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:56.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.800+0000 7fbe4f52b700 1 -- 172.21.15.40:0/4038922142 shutdown_connections 2023-07-18T03:31:56.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.800+0000 7fbe4f52b700 1 -- 172.21.15.40:0/4038922142 wait complete. 2023-07-18T03:31:56.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.801+0000 7fbe4f52b700 1 Processor -- start 2023-07-18T03:31:56.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.801+0000 7fbe4f52b700 1 -- start start 2023-07-18T03:31:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.801+0000 7fbe4f52b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe480fd910 0x7fbe4819be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.801+0000 7fbe4f52b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe480fea40 0x7fbe4819c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.801+0000 7fbe4f52b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe480ff3f0 0x7fbe481a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.801+0000 7fbe4f52b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe481a39c0 con 0x7fbe480ff3f0 2023-07-18T03:31:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.801+0000 7fbe4f52b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe481a3b00 con 0x7fbe480fea40 2023-07-18T03:31:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.801+0000 7fbe4f52b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe481a3c90 con 0x7fbe480fd910 2023-07-18T03:31:56.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.801+0000 7fbe4dac8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe480ff3f0 0x7fbe481a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.801+0000 7fbe4dac8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe480ff3f0 0x7fbe481a33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58786/0 (socket says 172.21.15.40:58786) 2023-07-18T03:31:56.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.801+0000 7fbe4dac8700 1 -- 172.21.15.40:0/1869407113 learned_addr learned my addr 172.21.15.40:0/1869407113 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:56.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.801+0000 7fbe4cac6700 1 --2- 172.21.15.40:0/1869407113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe480fea40 0x7fbe4819c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.801+0000 7fbe4d2c7700 1 --2- 172.21.15.40:0/1869407113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe480fd910 0x7fbe4819be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.802+0000 7fbe4dac8700 1 -- 172.21.15.40:0/1869407113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe480fd910 msgr2=0x7fbe4819be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.802+0000 7fbe4dac8700 1 --2- 172.21.15.40:0/1869407113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe480fd910 0x7fbe4819be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.802+0000 7fbe4dac8700 1 -- 172.21.15.40:0/1869407113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe480fea40 msgr2=0x7fbe4819c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.802+0000 7fbe4dac8700 1 --2- 172.21.15.40:0/1869407113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe480fea40 0x7fbe4819c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.802+0000 7fbe4dac8700 1 -- 172.21.15.40:0/1869407113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe440095f0 con 0x7fbe480ff3f0 2023-07-18T03:31:56.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.802+0000 7fbe4dac8700 1 --2- 172.21.15.40:0/1869407113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe480ff3f0 0x7fbe481a33f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000b970 tx=0x7fbe4000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:56.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.802+0000 7fbe3e7fc700 1 -- 172.21.15.40:0/1869407113 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4000f840 con 0x7fbe480ff3f0 2023-07-18T03:31:56.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.802+0000 7fbe4f52b700 1 -- 172.21.15.40:0/1869407113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe481a3f70 con 0x7fbe480ff3f0 2023-07-18T03:31:56.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.802+0000 7fbe3e7fc700 1 -- 172.21.15.40:0/1869407113 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe4000fe60 con 0x7fbe480ff3f0 2023-07-18T03:31:56.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.802+0000 7fbe3e7fc700 1 -- 172.21.15.40:0/1869407113 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4000d3d0 con 0x7fbe480ff3f0 2023-07-18T03:31:56.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.802+0000 7fbe4f52b700 1 -- 172.21.15.40:0/1869407113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe481a43f0 con 0x7fbe480ff3f0 2023-07-18T03:31:56.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.804+0000 7fbe3e7fc700 1 -- 172.21.15.40:0/1869407113 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbe4000f9a0 con 0x7fbe480ff3f0 2023-07-18T03:31:56.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.805+0000 7fbe3e7fc700 1 --2- 172.21.15.40:0/1869407113 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe340a11a0 0x7fbe340a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.805+0000 7fbe3e7fc700 1 -- 172.21.15.40:0/1869407113 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbe40006e40 con 0x7fbe480ff3f0 2023-07-18T03:31:56.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.805+0000 7fbe4d2c7700 1 --2- 172.21.15.40:0/1869407113 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe340a11a0 0x7fbe340a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.806+0000 7fbe4d2c7700 1 --2- 172.21.15.40:0/1869407113 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe340a11a0 0x7fbe340a3650 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbe38005950 tx=0x7fbe3800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:56.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.806+0000 7fbe5b6b1b80 1 Processor -- start 2023-07-18T03:31:56.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.806+0000 7fbe5b6b1b80 1 -- start start 2023-07-18T03:31:56.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.807+0000 7fbe5b6b1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368dd550 0x55ec368dd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.807+0000 7fbe5b6b1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368ddea0 0x55ec368de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.807+0000 7fbe5b6b1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368decf0 0x55ec368df160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.807+0000 7fbe5b6b1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec367e2d30 con 0x55ec368decf0 2023-07-18T03:31:56.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.807+0000 7fbe5b6b1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec36835ac0 con 0x55ec368dd550 2023-07-18T03:31:56.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.807+0000 7fbe5b6b1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3682ef50 con 0x55ec368ddea0 2023-07-18T03:31:56.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.807+0000 7fbe4d2c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368dd550 0x55ec368dd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.807+0000 7fbe4d2c7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368dd550 0x55ec368dd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42116/0 (socket says 172.21.15.40:42116) 2023-07-18T03:31:56.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.807+0000 7fbe4d2c7700 1 -- 172.21.15.40:0/1981811643 learned_addr learned my addr 172.21.15.40:0/1981811643 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:56.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.807+0000 7fbe4cac6700 1 --2- 172.21.15.40:0/1981811643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368ddea0 0x55ec368de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.807+0000 7fbe4dac8700 1 --2- 172.21.15.40:0/1981811643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368decf0 0x55ec368df160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.807+0000 7fbe4d2c7700 1 -- 172.21.15.40:0/1981811643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368ddea0 msgr2=0x55ec368de760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.807+0000 7fbe4d2c7700 1 --2- 172.21.15.40:0/1981811643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368ddea0 0x55ec368de760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.807+0000 7fbe4d2c7700 1 -- 172.21.15.40:0/1981811643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368decf0 msgr2=0x55ec368df160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.807+0000 7fbe4d2c7700 1 --2- 172.21.15.40:0/1981811643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368decf0 0x55ec368df160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.807+0000 7fbe4d2c7700 1 -- 172.21.15.40:0/1981811643 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3665ce60 con 0x55ec368dd550 2023-07-18T03:31:56.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.807+0000 7fbe4d2c7700 1 --2- 172.21.15.40:0/1981811643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368dd550 0x55ec368dd960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbe38010fa0 tx=0x7fbe38013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f5a8986448df2f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:56.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.808+0000 7fbe337fe700 1 -- 172.21.15.40:0/1981811643 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38011db0 con 0x55ec368dd550 2023-07-18T03:31:56.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.808+0000 7fbe337fe700 1 -- 172.21.15.40:0/1981811643 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe38015070 con 0x55ec368dd550 2023-07-18T03:31:56.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.808+0000 7fbe337fe700 1 -- 172.21.15.40:0/1981811643 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3801c610 con 0x55ec368dd550 2023-07-18T03:31:56.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.809+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/1981811643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368dd550 msgr2=0x55ec368dd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.809+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/1981811643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368dd550 0x55ec368dd960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbe38010fa0 tx=0x7fbe38013000 comp rx=0 tx=0).stop 2023-07-18T03:31:56.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.809+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/1981811643 shutdown_connections 2023-07-18T03:31:56.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.809+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/1981811643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368decf0 0x55ec368df160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.809+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/1981811643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368dd550 0x55ec368dd960 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.809+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/1981811643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368ddea0 0x55ec368de760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.809+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/1981811643 >> 172.21.15.40:0/1981811643 conn(0x55ec368d2e20 msgr2=0x55ec368d8250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:56.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.809+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/1981811643 shutdown_connections 2023-07-18T03:31:56.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.809+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/1981811643 wait complete. 2023-07-18T03:31:56.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.809+0000 7fbe5b6b1b80 1 -- start start 2023-07-18T03:31:56.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.809+0000 7fbe5b6b1b80 1 Processor -- start 2023-07-18T03:31:56.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.810+0000 7fbe5b6b1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368dd550 0x55ec368e7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.810+0000 7fbe5b6b1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 0x55ec368e76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.810+0000 7fbe5b6b1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368decf0 0x55ec368ee710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.810+0000 7fbe5b6b1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec367ceb40 con 0x55ec368decf0 2023-07-18T03:31:56.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.810+0000 7fbe5b6b1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec367ce840 con 0x55ec368ddea0 2023-07-18T03:31:56.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.810+0000 7fbe5b6b1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec366b29a0 con 0x55ec368dd550 2023-07-18T03:31:56.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.810+0000 7fbe5b6b1b80 10 client.0 _ll_register_callbacks cb 0x55ec367cc7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:56.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.810+0000 7fbe4d2c7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368dd550 0x55ec368e7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.810+0000 7fbe4cac6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 0x55ec368e76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.810+0000 7fbe4dac8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368decf0 0x55ec368ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.810+0000 7fbe4cac6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 0x55ec368e76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42124/0 (socket says 172.21.15.40:42124) 2023-07-18T03:31:56.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.810+0000 7fbe4cac6700 1 -- 172.21.15.40:0/3136203716 learned_addr learned my addr 172.21.15.40:0/3136203716 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.810+0000 7f3e357fa700 1 -- 172.21.15.40:0/3998085022 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555c10038ff0 con 0x555c10199af0 2023-07-18T03:31:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.810+0000 7f3e357fa700 10 client.25460 handle_command_reply: tid=0 2023-07-18T03:31:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.810+0000 7f3e60206b80 1 client.25460 shutdown 2023-07-18T03:31:56.815 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.810+0000 7f3e60206b80 1 -- 172.21.15.40:0/3998085022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c10130c10 msgr2=0x555c1013d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.815 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.810+0000 7f3e60206b80 1 --2- 172.21.15.40:0/3998085022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c10130c10 0x555c1013d060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c09b120 tx=0x7f3e3c009780 comp rx=0 tx=0).stop 2023-07-18T03:31:56.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.810+0000 7fbe4cac6700 1 -- 172.21.15.40:0/3136203716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368dd550 msgr2=0x55ec368e7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.810+0000 7fbe4cac6700 1 --2- 172.21.15.40:0/3136203716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368dd550 0x55ec368e7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.810+0000 7fbe4cac6700 1 -- 172.21.15.40:0/3136203716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368decf0 msgr2=0x55ec368ee710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.810+0000 7fbe4cac6700 1 --2- 172.21.15.40:0/3136203716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368decf0 0x55ec368ee710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.810+0000 7fbe4cac6700 1 -- 172.21.15.40:0/3136203716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe38010c50 con 0x55ec368ddea0 2023-07-18T03:31:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.811+0000 7f3e60206b80 1 -- 172.21.15.40:0/3998085022 shutdown_connections 2023-07-18T03:31:56.816 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.811+0000 7f3e60206b80 1 --2- 172.21.15.40:0/3998085022 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555c10199af0 0x555c1019bed0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.811+0000 7f3e60206b80 1 --2- 172.21.15.40:0/3998085022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555c10131b10 0x555c10140050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.811+0000 7f3e60206b80 1 --2- 172.21.15.40:0/3998085022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555c0ff886b0 0x555c1013cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.811+0000 7f3e357fa700 0 client.25460 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.811+0000 7f3e60206b80 1 --2- 172.21.15.40:0/3998085022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555c10130c10 0x555c1013d060 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.811+0000 7f3e60206b80 1 -- 172.21.15.40:0/3998085022 >> 172.21.15.40:0/3998085022 conn(0x555c10121be0 msgr2=0x555c101348e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:56.817 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.811+0000 7f3e60206b80 1 -- 172.21.15.40:0/3998085022 shutdown_connections 2023-07-18T03:31:56.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.811+0000 7fbe4cac6700 1 --2- 172.21.15.40:0/3136203716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 0x55ec368e76d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009990 tx=0x7fbe44005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dedb83b63fddeb57 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:56.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.811+0000 7fbe30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:31:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.811+0000 7f3e60206b80 1 -- 172.21.15.40:0/3998085022 wait complete. 2023-07-18T03:31:56.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.811+0000 7fbe30ff9700 1 -- 172.21.15.40:0/3136203716 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4401d070 con 0x55ec368ddea0 2023-07-18T03:31:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.811+0000 7f3e60206b80 20 client.25460 trim_cache size 0 max 16384 2023-07-18T03:31:56.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.811+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/3136203716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec36788570 con 0x55ec368ddea0 2023-07-18T03:31:56.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.811+0000 7fbe30ff9700 1 -- 172.21.15.40:0/3136203716 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbe44004710 con 0x55ec368ddea0 2023-07-18T03:31:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:31:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:31:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:31:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:31:56.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.811+0000 7fbe30ff9700 1 -- 172.21.15.40:0/3136203716 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe4400bed0 con 0x55ec368ddea0 2023-07-18T03:31:56.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.811+0000 7fbe5b6b1b80 10 client.25469 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:56.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.811+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/3136203716 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ec36645ce0 con 0x55ec368ddea0 2023-07-18T03:31:56.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.811+0000 7fbe30ff9700 1 -- 172.21.15.40:0/3136203716 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec36788570 con 0x55ec368ddea0 2023-07-18T03:31:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.811+0000 7f3e54080700 1 -- 172.21.15.40:0/2891106864 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3e380a12a0 msgr2=0x7f3e380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.811+0000 7f3e54080700 1 --2- 172.21.15.40:0/2891106864 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3e380a12a0 0x7f3e380a3750 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009fd0 tx=0x7f3e48009260 comp rx=0 tx=0).stop 2023-07-18T03:31:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.811+0000 7f3e54080700 1 -- 172.21.15.40:0/2891106864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e4c0ffc50 msgr2=0x7f3e4c195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.811+0000 7f3e54080700 1 --2- 172.21.15.40:0/2891106864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e4c0ffc50 0x7f3e4c195e70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0059d0 tx=0x7f3e3c004730 comp rx=0 tx=0).stop 2023-07-18T03:31:56.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.812+0000 7fbe30ff9700 1 -- 172.21.15.40:0/3136203716 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbe4400fa60 con 0x55ec368ddea0 2023-07-18T03:31:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.812+0000 7f3e54080700 1 -- 172.21.15.40:0/2891106864 shutdown_connections 2023-07-18T03:31:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.812+0000 7f3e54080700 1 --2- 172.21.15.40:0/2891106864 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3e380a12a0 0x7f3e380a3750 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.812+0000 7f3e54080700 1 --2- 172.21.15.40:0/2891106864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e4c1005e0 0x7f3e4c196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.812+0000 7f3e54080700 1 --2- 172.21.15.40:0/2891106864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e4c0fea60 0x7f3e4c195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.812+0000 7f3e54080700 1 --2- 172.21.15.40:0/2891106864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e4c0ffc50 0x7f3e4c195e70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.812+0000 7f3e54080700 1 -- 172.21.15.40:0/2891106864 >> 172.21.15.40:0/2891106864 conn(0x7f3e4c0fa060 msgr2=0x7f3e4c0fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.812+0000 7fbe5b6b1b80 10 client.25469 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.812+0000 7fbe5b6b1b80 10 client.25469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.812+0000 7fbe5b6b1b80 20 client.25469 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.812+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/3136203716 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ec368f3060 0x55ec368f5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.812+0000 7fbe5b6b1b80 4 client.25469 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:56.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.812+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/3136203716 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec36827f30 con 0x55ec368f3060 2023-07-18T03:31:56.825 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.812+0000 7f3e54080700 1 -- 172.21.15.40:0/2891106864 shutdown_connections 2023-07-18T03:31:56.826 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:31:56.812+0000 7f3e54080700 1 -- 172.21.15.40:0/2891106864 wait complete. 2023-07-18T03:31:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.812+0000 7fbe4d2c7700 1 --2- 172.21.15.40:0/3136203716 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ec368f3060 0x55ec368f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.812+0000 7fbe30ff9700 10 client.25469 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.812+0000 7fbe4d2c7700 1 --2- 172.21.15.40:0/3136203716 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ec368f3060 0x55ec368f5470 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87a732a9c545422 server_cookie=a223867e0c1a0ded in_seq=0 out_seq=0 2023-07-18T03:31:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.813+0000 7fbe30ff9700 1 -- 172.21.15.40:0/3136203716 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ec36827f30 con 0x55ec368f3060 2023-07-18T03:31:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.813+0000 7fbe30ff9700 10 client.25469 handle_command_reply: tid=0 2023-07-18T03:31:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.813+0000 7fbe5b6b1b80 1 client.25469 shutdown 2023-07-18T03:31:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.813+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/3136203716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 msgr2=0x55ec368e76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.813+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/3136203716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 0x55ec368e76d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009990 tx=0x7fbe44005270 comp rx=0 tx=0).stop 2023-07-18T03:31:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.813+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/3136203716 shutdown_connections 2023-07-18T03:31:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.813+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/3136203716 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ec368f3060 0x55ec368f5470 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.813+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/3136203716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368decf0 0x55ec368ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.813+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/3136203716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368dd550 0x55ec368e7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.813+0000 7fbe30ff9700 0 client.25469 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:56.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.813+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/3136203716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 0x55ec368e76d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.813+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/3136203716 >> 172.21.15.40:0/3136203716 conn(0x55ec368d2e20 msgr2=0x55ec368d7930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:56.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.813+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/3136203716 shutdown_connections 2023-07-18T03:31:56.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.813+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/3136203716 wait complete. 2023-07-18T03:31:56.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.814+0000 7fbe5b6b1b80 20 client.25469 trim_cache size 0 max 16384 2023-07-18T03:31:56.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.832+0000 7fbe5b6b1b80 1 Processor -- start 2023-07-18T03:31:56.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.832+0000 7fbe5b6b1b80 1 -- start start 2023-07-18T03:31:56.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe5b6b1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368dd550 0x55ec368e7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe5b6b1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 0x55ec368e76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe5b6b1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368decf0 0x55ec368ee630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe5b6b1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec368eeb70 con 0x55ec368dd550 2023-07-18T03:31:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe5b6b1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec368e7e10 con 0x55ec368ddea0 2023-07-18T03:31:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe5b6b1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec368e4e30 con 0x55ec368decf0 2023-07-18T03:31:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe4d2c7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368dd550 0x55ec368e7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe4cac6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 0x55ec368e76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe4d2c7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368dd550 0x55ec368e7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58804/0 (socket says 172.21.15.40:58804) 2023-07-18T03:31:56.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe4d2c7700 1 -- 172.21.15.40:0/3739600672 learned_addr learned my addr 172.21.15.40:0/3739600672 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:56.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe4dac8700 1 --2- 172.21.15.40:0/3739600672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368decf0 0x55ec368ee630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe4d2c7700 1 -- 172.21.15.40:0/3739600672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368decf0 msgr2=0x55ec368ee630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe4d2c7700 1 --2- 172.21.15.40:0/3739600672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368decf0 0x55ec368ee630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe4d2c7700 1 -- 172.21.15.40:0/3739600672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 msgr2=0x55ec368e76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe4d2c7700 1 --2- 172.21.15.40:0/3739600672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 0x55ec368e76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe4d2c7700 1 -- 172.21.15.40:0/3739600672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec368e61d0 con 0x55ec368dd550 2023-07-18T03:31:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe4cac6700 1 --2- 172.21.15.40:0/3739600672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 0x55ec368e76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.833+0000 7fbe4d2c7700 1 --2- 172.21.15.40:0/3739600672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368dd550 0x55ec368e7190 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbe38011010 tx=0x7fbe38016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e99d501f078efcd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.834+0000 7fbe33fff700 1 -- 172.21.15.40:0/3739600672 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3802c030 con 0x55ec368dd550 2023-07-18T03:31:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.834+0000 7fbe33fff700 1 -- 172.21.15.40:0/3739600672 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe3802d040 con 0x55ec368dd550 2023-07-18T03:31:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.834+0000 7fbe33fff700 1 -- 172.21.15.40:0/3739600672 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38033290 con 0x55ec368dd550 2023-07-18T03:31:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.834+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/3739600672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368dd550 msgr2=0x55ec368e7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.834+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/3739600672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368dd550 0x55ec368e7190 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbe38011010 tx=0x7fbe38016000 comp rx=0 tx=0).stop 2023-07-18T03:31:56.835 INFO:tasks.workunit.client.0.smithi040.stdout:checking for unistd.h... yes 2023-07-18T03:31:56.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.834+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/3739600672 shutdown_connections 2023-07-18T03:31:56.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.834+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/3739600672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368decf0 0x55ec368ee630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.834+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/3739600672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368dd550 0x55ec368e7190 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.834+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/3739600672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 0x55ec368e76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.834+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/3739600672 >> 172.21.15.40:0/3739600672 conn(0x55ec368d2e20 msgr2=0x55ec368e5080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:56.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.835+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/3739600672 shutdown_connections 2023-07-18T03:31:56.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.835+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/3739600672 wait complete. 2023-07-18T03:31:56.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.835+0000 7fbe5b6b1b80 1 -- start start 2023-07-18T03:31:56.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.835+0000 7fbe5b6b1b80 1 Processor -- start 2023-07-18T03:31:56.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.835+0000 7fbe5b6b1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368dd550 0x55ec368eb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.835+0000 7fbe5b6b1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 0x55ec368ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.835+0000 7fbe5b6b1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368decf0 0x55ec368ece90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.835+0000 7fbe5b6b1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec368ed3d0 con 0x55ec368decf0 2023-07-18T03:31:56.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.835+0000 7fbe5b6b1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec368ed540 con 0x55ec368ddea0 2023-07-18T03:31:56.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.835+0000 7fbe5b6b1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec368ed700 con 0x55ec368dd550 2023-07-18T03:31:56.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.835+0000 7fbe5b6b1b80 10 client.0 _ll_register_callbacks cb 0x55ec367cc7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:56.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.835+0000 7fbe4dac8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368decf0 0x55ec368ece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.835+0000 7fbe4dac8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368decf0 0x55ec368ece90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58814/0 (socket says 172.21.15.40:58814) 2023-07-18T03:31:56.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.836+0000 7fbe4dac8700 1 -- 172.21.15.40:0/1341965730 learned_addr learned my addr 172.21.15.40:0/1341965730 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:56.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.836+0000 7fbe4d2c7700 1 --2- 172.21.15.40:0/1341965730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368dd550 0x55ec368eb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.836+0000 7fbe4cac6700 1 --2- 172.21.15.40:0/1341965730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 0x55ec368ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.836+0000 7fbe4dac8700 1 -- 172.21.15.40:0/1341965730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368dd550 msgr2=0x55ec368eb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.836+0000 7fbe4dac8700 1 --2- 172.21.15.40:0/1341965730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368dd550 0x55ec368eb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.836+0000 7fbe4dac8700 1 -- 172.21.15.40:0/1341965730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 msgr2=0x55ec368ebe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.836+0000 7fbe4dac8700 1 --2- 172.21.15.40:0/1341965730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 0x55ec368ebe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.836+0000 7fbe4dac8700 1 -- 172.21.15.40:0/1341965730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec36645ce0 con 0x55ec368decf0 2023-07-18T03:31:56.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.836+0000 7fbe4dac8700 1 --2- 172.21.15.40:0/1341965730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368decf0 0x55ec368ece90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbe40099b00 tx=0x7fbe40004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d885045aa7c1ab7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:56.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.836+0000 7fbe30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:56.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.836+0000 7fbe30ff9700 1 -- 172.21.15.40:0/1341965730 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe400a2010 con 0x55ec368decf0 2023-07-18T03:31:56.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.836+0000 7fbe30ff9700 1 -- 172.21.15.40:0/1341965730 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbe40017600 con 0x55ec368decf0 2023-07-18T03:31:56.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.836+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/1341965730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe38015070 con 0x55ec368decf0 2023-07-18T03:31:56.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.836+0000 7fbe30ff9700 1 -- 172.21.15.40:0/1341965730 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe400b2240 con 0x55ec368decf0 2023-07-18T03:31:56.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.836+0000 7fbe5b6b1b80 10 client.15816 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:56.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.836+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/1341965730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ec3665ce60 con 0x55ec368decf0 2023-07-18T03:31:56.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.837+0000 7fbe30ff9700 1 -- 172.21.15.40:0/1341965730 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe40063080 con 0x55ec368decf0 2023-07-18T03:31:56.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.837+0000 7fbe30ff9700 1 -- 172.21.15.40:0/1341965730 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbe400a1b80 con 0x55ec368decf0 2023-07-18T03:31:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.837+0000 7fbe5b6b1b80 10 client.15816 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.837+0000 7fbe5b6b1b80 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.837+0000 7fbe5b6b1b80 20 client.15816 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.837+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/1341965730 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ec36949590 0x55ec3694b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.837+0000 7fbe5b6b1b80 4 client.15816 mds_command: new command op to 24527 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:31:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.837+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/1341965730 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ec367ea0d0 con 0x55ec36949590 2023-07-18T03:31:56.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.837+0000 7fbe4d2c7700 1 --2- 172.21.15.40:0/1341965730 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ec36949590 0x55ec3694b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.837+0000 7fbe4d2c7700 1 --2- 172.21.15.40:0/1341965730 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ec36949590 0x55ec3694b9a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff0e7e6924e45ec2 server_cookie=6fedaacdf91c849a in_seq=0 out_seq=0 2023-07-18T03:31:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.837+0000 7fbe30ff9700 10 client.15816 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.839+0000 7fbe30ff9700 1 -- 172.21.15.40:0/1341965730 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ec367ea0d0 con 0x55ec36949590 2023-07-18T03:31:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.839+0000 7fbe30ff9700 10 client.15816 handle_command_reply: tid=0 2023-07-18T03:31:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.839+0000 7fbe5b6b1b80 1 client.15816 shutdown 2023-07-18T03:31:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.839+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/1341965730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368decf0 msgr2=0x55ec368ece90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.839+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/1341965730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368decf0 0x55ec368ece90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbe40099b00 tx=0x7fbe40004ad0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.839+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/1341965730 shutdown_connections 2023-07-18T03:31:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.839+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/1341965730 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ec36949590 0x55ec3694b9a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.839+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/1341965730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ec368decf0 0x55ec368ece90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.839+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/1341965730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec368dd550 0x55ec368eb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.839+0000 7fbe30ff9700 0 client.15816 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.839+0000 7fbe5b6b1b80 1 --2- 172.21.15.40:0/1341965730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec368ddea0 0x55ec368ebe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.839+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/1341965730 >> 172.21.15.40:0/1341965730 conn(0x55ec368d2e20 msgr2=0x55ec368e7130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:56.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.839+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/1341965730 shutdown_connections 2023-07-18T03:31:56.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.839+0000 7fbe5b6b1b80 1 -- 172.21.15.40:0/1341965730 wait complete. 2023-07-18T03:31:56.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.839+0000 7fbe5b6b1b80 20 client.15816 trim_cache size 0 max 16384 2023-07-18T03:31:56.847 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:31:56.847 INFO:teuthology.orchestra.run.smithi040.stdout: "return_code": 0, 2023-07-18T03:31:56.847 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_tag": "5e1c5c11-079c-45b0-a60e-91da5f7ba4f1", 2023-07-18T03:31:56.848 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": "asynchronous" 2023-07-18T03:31:56.848 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:31:56.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.840+0000 7fbe4f52b700 1 -- 172.21.15.40:0/1869407113 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe340a11a0 msgr2=0x7fbe340a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.840+0000 7fbe4f52b700 1 --2- 172.21.15.40:0/1869407113 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe340a11a0 0x7fbe340a3650 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbe38005950 tx=0x7fbe3800b410 comp rx=0 tx=0).stop 2023-07-18T03:31:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.840+0000 7fbe4f52b700 1 -- 172.21.15.40:0/1869407113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe480ff3f0 msgr2=0x7fbe481a33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.840+0000 7fbe4f52b700 1 --2- 172.21.15.40:0/1869407113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe480ff3f0 0x7fbe481a33f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000b970 tx=0x7fbe4000bc80 comp rx=0 tx=0).stop 2023-07-18T03:31:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.840+0000 7fbe4f52b700 1 -- 172.21.15.40:0/1869407113 shutdown_connections 2023-07-18T03:31:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.840+0000 7fbe4f52b700 1 --2- 172.21.15.40:0/1869407113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe480ff3f0 0x7fbe481a33f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.840+0000 7fbe4f52b700 1 --2- 172.21.15.40:0/1869407113 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbe340a11a0 0x7fbe340a3650 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.840+0000 7fbe4f52b700 1 --2- 172.21.15.40:0/1869407113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe480fd910 0x7fbe4819be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.840+0000 7fbe4f52b700 1 --2- 172.21.15.40:0/1869407113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe480fea40 0x7fbe4819c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.840+0000 7fbe4f52b700 1 -- 172.21.15.40:0/1869407113 >> 172.21.15.40:0/1869407113 conn(0x7fbe480f92c0 msgr2=0x7fbe480f9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.840+0000 7fbe4f52b700 1 -- 172.21.15.40:0/1869407113 shutdown_connections 2023-07-18T03:31:56.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.840+0000 7fbe4f52b700 1 -- 172.21.15.40:0/1869407113 wait complete. 2023-07-18T03:31:56.850 INFO:tasks.workunit.client.0.smithi040.stdout:checking for memory.h... (cached) yes 2023-07-18T03:31:56.851 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:31:56.852 INFO:tasks.workunit.client.0.smithi040.stdout:checking for strings.h... (cached) yes 2023-07-18T03:31:56.857 INFO:tasks.workunit.client.0.smithi040.stdout:checking for unistd.h... (cached) yes 2023-07-18T03:31:56.901 INFO:tasks.workunit.client.0.smithi040.stdout:checking locale.h usability... yes 2023-07-18T03:31:56.916 INFO:tasks.workunit.client.0.smithi040.stdout:checking locale.h presence... yes 2023-07-18T03:31:56.917 INFO:tasks.workunit.client.0.smithi040.stdout:checking for locale.h... yes 2023-07-18T03:31:56.935 INFO:tasks.workunit.client.0.smithi040.stdout:checking for stdlib.h... (cached) yes 2023-07-18T03:31:56.941 INFO:tasks.workunit.client.0.smithi040.stdout:checking for string.h... (cached) yes 2023-07-18T03:31:56.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.987+0000 7fa093849700 1 -- 172.21.15.40:0/929056647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c100cd0 msgr2=0x7fa08c101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.987+0000 7fa093849700 1 --2- 172.21.15.40:0/929056647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c100cd0 0x7fa08c101120 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009990 tx=0x7fa07c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.988+0000 7fa093849700 1 -- 172.21.15.40:0/929056647 shutdown_connections 2023-07-18T03:31:56.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.988+0000 7fa093849700 1 --2- 172.21.15.40:0/929056647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa08c101660 0x7fa08c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.988+0000 7fa093849700 1 --2- 172.21.15.40:0/929056647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa08c0ffae0 0x7fa08c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.988+0000 7fa093849700 1 --2- 172.21.15.40:0/929056647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c100cd0 0x7fa08c101120 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.988+0000 7fa093849700 1 -- 172.21.15.40:0/929056647 >> 172.21.15.40:0/929056647 conn(0x7fa08c0fb120 msgr2=0x7fa08c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:56.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.988+0000 7fa093849700 1 -- 172.21.15.40:0/929056647 shutdown_connections 2023-07-18T03:31:56.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.988+0000 7fa093849700 1 -- 172.21.15.40:0/929056647 wait complete. 2023-07-18T03:31:56.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.988+0000 7fa093849700 1 Processor -- start 2023-07-18T03:31:56.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.988+0000 7fa093849700 1 -- start start 2023-07-18T03:31:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa093849700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa08c0ffae0 0x7fa08c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa093849700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c100cd0 0x7fa08c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa093849700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa08c101660 0x7fa08c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa093849700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c0735e0 con 0x7fa08c100cd0 2023-07-18T03:31:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa093849700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c073750 con 0x7fa08c0ffae0 2023-07-18T03:31:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa093849700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c19f880 con 0x7fa08c101660 2023-07-18T03:31:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa090de4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c100cd0 0x7fa08c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa090de4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c100cd0 0x7fa08c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58826/0 (socket says 172.21.15.40:58826) 2023-07-18T03:31:56.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa0915e5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa08c0ffae0 0x7fa08c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa090de4700 1 -- 172.21.15.40:0/3875424152 learned_addr learned my addr 172.21.15.40:0/3875424152 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:56.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa091de6700 1 --2- 172.21.15.40:0/3875424152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa08c101660 0x7fa08c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa0915e5700 1 -- 172.21.15.40:0/3875424152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa08c101660 msgr2=0x7fa08c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa0915e5700 1 --2- 172.21.15.40:0/3875424152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa08c101660 0x7fa08c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa0915e5700 1 -- 172.21.15.40:0/3875424152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c100cd0 msgr2=0x7fa08c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa0915e5700 1 --2- 172.21.15.40:0/3875424152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c100cd0 0x7fa08c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa0915e5700 1 -- 172.21.15.40:0/3875424152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c0095f0 con 0x7fa08c0ffae0 2023-07-18T03:31:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa0915e5700 1 --2- 172.21.15.40:0/3875424152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa08c0ffae0 0x7fa08c0742c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa08800c940 tx=0x7fa08800cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.989+0000 7fa0827fc700 1 -- 172.21.15.40:0/3875424152 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa088004220 con 0x7fa08c0ffae0 2023-07-18T03:31:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.990+0000 7fa093849700 1 -- 172.21.15.40:0/3875424152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08c19fb60 con 0x7fa08c0ffae0 2023-07-18T03:31:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.990+0000 7fa0827fc700 1 -- 172.21.15.40:0/3875424152 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa088004cf0 con 0x7fa08c0ffae0 2023-07-18T03:31:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.990+0000 7fa0827fc700 1 -- 172.21.15.40:0/3875424152 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0880056c0 con 0x7fa08c0ffae0 2023-07-18T03:31:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.990+0000 7fa093849700 1 -- 172.21.15.40:0/3875424152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08c1a0070 con 0x7fa08c0ffae0 2023-07-18T03:31:56.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.992+0000 7fa0827fc700 1 -- 172.21.15.40:0/3875424152 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa088004380 con 0x7fa08c0ffae0 2023-07-18T03:31:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.993+0000 7fa0827fc700 1 --2- 172.21.15.40:0/3875424152 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0780a1150 0x7fa0780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.993+0000 7fa09f9cfb80 1 Processor -- start 2023-07-18T03:31:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.993+0000 7fa09f9cfb80 1 -- start start 2023-07-18T03:31:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.993+0000 7fa0827fc700 1 -- 172.21.15.40:0/3875424152 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa0880cc3f0 con 0x7fa08c0ffae0 2023-07-18T03:31:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.993+0000 7fa090de4700 1 --2- 172.21.15.40:0/3875424152 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0780a1150 0x7fa0780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.993+0000 7fa09f9cfb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259e4d0 0x55a51259e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.993+0000 7fa09f9cfb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259ee20 0x55a51259f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.993+0000 7fa09f9cfb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259fc70 0x55a5125a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.993+0000 7fa09f9cfb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5124a3cb0 con 0x55a51259ee20 2023-07-18T03:31:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.993+0000 7fa09f9cfb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5124f6a40 con 0x55a51259fc70 2023-07-18T03:31:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.993+0000 7fa09f9cfb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5124efed0 con 0x55a51259e4d0 2023-07-18T03:31:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.994+0000 7fa090de4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259ee20 0x55a51259f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.994+0000 7fa090de4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259ee20 0x55a51259f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58834/0 (socket says 172.21.15.40:58834) 2023-07-18T03:31:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.994+0000 7fa090de4700 1 -- 172.21.15.40:0/498622220 learned_addr learned my addr 172.21.15.40:0/498622220 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.994+0000 7fa0915e5700 1 --2- 172.21.15.40:0/498622220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259e4d0 0x55a51259e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.994+0000 7fa091de6700 1 --2- 172.21.15.40:0/498622220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259fc70 0x55a5125a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.994+0000 7fa090de4700 1 -- 172.21.15.40:0/498622220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259e4d0 msgr2=0x55a51259e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.994+0000 7fa090de4700 1 --2- 172.21.15.40:0/498622220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259e4d0 0x55a51259e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.994+0000 7fa090de4700 1 -- 172.21.15.40:0/498622220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259fc70 msgr2=0x55a5125a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.994+0000 7fa090de4700 1 --2- 172.21.15.40:0/498622220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259fc70 0x55a5125a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.994+0000 7fa090de4700 1 -- 172.21.15.40:0/498622220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a51231dde0 con 0x55a51259ee20 2023-07-18T03:31:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.994+0000 7fa091de6700 1 --2- 172.21.15.40:0/498622220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259fc70 0x55a5125a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.994+0000 7fa090de4700 1 --2- 172.21.15.40:0/3875424152 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0780a1150 0x7fa0780a3600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa07c005670 tx=0x7fa07c0056a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.994+0000 7fa090de4700 1 --2- 172.21.15.40:0/498622220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259ee20 0x55a51259f6e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00bad0 tx=0x7fa07c029030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f46cbf164d74ff server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.994+0000 7fa0777fe700 1 -- 172.21.15.40:0/498622220 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c01d030 con 0x55a51259ee20 2023-07-18T03:31:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.994+0000 7fa0777fe700 1 -- 172.21.15.40:0/498622220 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa07c004cd0 con 0x55a51259ee20 2023-07-18T03:31:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.994+0000 7fa0777fe700 1 -- 172.21.15.40:0/498622220 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c00fb90 con 0x55a51259ee20 2023-07-18T03:31:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.995+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/498622220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259ee20 msgr2=0x55a51259f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.995+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/498622220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259ee20 0x55a51259f6e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00bad0 tx=0x7fa07c029030 comp rx=0 tx=0).stop 2023-07-18T03:31:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.995+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/498622220 shutdown_connections 2023-07-18T03:31:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.995+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/498622220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259fc70 0x55a5125a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.995+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/498622220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259e4d0 0x55a51259e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.995+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/498622220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259ee20 0x55a51259f6e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.995+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/498622220 >> 172.21.15.40:0/498622220 conn(0x55a512593da0 msgr2=0x55a5125991d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.995+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/498622220 shutdown_connections 2023-07-18T03:31:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.995+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/498622220 wait complete. 2023-07-18T03:31:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.995+0000 7fa09f9cfb80 1 -- start start 2023-07-18T03:31:57.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa09f9cfb80 1 Processor -- start 2023-07-18T03:31:57.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa09f9cfb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259e4d0 0x55a5125a8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:57.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa09f9cfb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259ee20 0x55a5125a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:57.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa09f9cfb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259fc70 0x55a5125af580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:57.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa09f9cfb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51248fac0 con 0x55a51259e4d0 2023-07-18T03:31:57.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa09f9cfb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51248f7c0 con 0x55a51259ee20 2023-07-18T03:31:57.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa09f9cfb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a512373920 con 0x55a51259fc70 2023-07-18T03:31:57.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa09f9cfb80 10 client.0 _ll_register_callbacks cb 0x55a51248d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:57.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa0915e5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259e4d0 0x55a5125a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:57.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa0915e5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259e4d0 0x55a5125a8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58838/0 (socket says 172.21.15.40:58838) 2023-07-18T03:31:57.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa0915e5700 1 -- 172.21.15.40:0/2950285097 learned_addr learned my addr 172.21.15.40:0/2950285097 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:57.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa090de4700 1 --2- 172.21.15.40:0/2950285097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259ee20 0x55a5125a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:57.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa091de6700 1 --2- 172.21.15.40:0/2950285097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259fc70 0x55a5125af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:57.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa0915e5700 1 -- 172.21.15.40:0/2950285097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259fc70 msgr2=0x55a5125af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:57.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa0915e5700 1 --2- 172.21.15.40:0/2950285097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259fc70 0x55a5125af580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa0915e5700 1 -- 172.21.15.40:0/2950285097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259ee20 msgr2=0x55a5125a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:57.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa0915e5700 1 --2- 172.21.15.40:0/2950285097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259ee20 0x55a5125a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.996+0000 7fa0915e5700 1 -- 172.21.15.40:0/2950285097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07c020040 con 0x55a51259e4d0 2023-07-18T03:31:57.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.997+0000 7fa0915e5700 1 --2- 172.21.15.40:0/2950285097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259e4d0 0x55a5125a8000 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa08800cfd0 tx=0x7fa08809d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11960c89ddca9195 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:57.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.997+0000 7fa074ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:31:57.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.997+0000 7fa074ff9700 1 -- 172.21.15.40:0/2950285097 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0880992c0 con 0x55a51259e4d0 2023-07-18T03:31:57.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.997+0000 7fa074ff9700 1 -- 172.21.15.40:0/2950285097 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa0880b90d0 con 0x55a51259e4d0 2023-07-18T03:31:57.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.997+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/2950285097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5124494f0 con 0x55a51259e4d0 2023-07-18T03:31:57.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.997+0000 7fa074ff9700 1 -- 172.21.15.40:0/2950285097 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0880b9290 con 0x55a51259e4d0 2023-07-18T03:31:57.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.997+0000 7fa09f9cfb80 10 client.15828 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:57.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.997+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/2950285097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a512306c60 con 0x55a51259e4d0 2023-07-18T03:31:57.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.997+0000 7fa074ff9700 1 -- 172.21.15.40:0/2950285097 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0880c7070 con 0x55a51259e4d0 2023-07-18T03:31:57.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.997+0000 7fa074ff9700 1 -- 172.21.15.40:0/2950285097 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa0880a1740 con 0x55a51259e4d0 2023-07-18T03:31:57.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.997+0000 7fa09f9cfb80 10 client.15828 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:57.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.997+0000 7fa09f9cfb80 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:57.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.997+0000 7fa09f9cfb80 20 client.15828 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:57.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.997+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/2950285097 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55a5125b3de0 0x55a5125b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:57.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.997+0000 7fa09f9cfb80 4 client.15828 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:31:57.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.997+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/2950285097 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5124e8eb0 con 0x55a5125b3de0 2023-07-18T03:31:57.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.998+0000 7fa090de4700 1 --2- 172.21.15.40:0/2950285097 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55a5125b3de0 0x55a5125b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:57.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.998+0000 7fa074ff9700 10 client.15828 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:57.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.998+0000 7fa090de4700 1 --2- 172.21.15.40:0/2950285097 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55a5125b3de0 0x55a5125b61f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f5ccc640d6beaf8 server_cookie=6ed83dc5b21fa588 in_seq=0 out_seq=0 2023-07-18T03:31:57.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.999+0000 7fa074ff9700 1 -- 172.21.15.40:0/2950285097 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a5124e8eb0 con 0x55a5125b3de0 2023-07-18T03:31:57.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.999+0000 7fa074ff9700 10 client.15828 handle_command_reply: tid=0 2023-07-18T03:31:57.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.999+0000 7fa09f9cfb80 1 client.15828 shutdown 2023-07-18T03:31:57.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.999+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/2950285097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259e4d0 msgr2=0x55a5125a8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:57.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.999+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/2950285097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259e4d0 0x55a5125a8000 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa08800cfd0 tx=0x7fa08809d7b0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.999+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/2950285097 shutdown_connections 2023-07-18T03:31:57.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.999+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/2950285097 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55a5125b3de0 0x55a5125b61f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.999+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/2950285097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259fc70 0x55a5125af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.999+0000 7fa074ff9700 0 client.15828 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:57.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.999+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/2950285097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259e4d0 0x55a5125a8000 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.999+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/2950285097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259ee20 0x55a5125a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.999+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/2950285097 >> 172.21.15.40:0/2950285097 conn(0x55a512593da0 msgr2=0x55a5125988b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:57.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.999+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/2950285097 shutdown_connections 2023-07-18T03:31:57.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.999+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/2950285097 wait complete. 2023-07-18T03:31:57.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:56.999+0000 7fa09f9cfb80 20 client.15828 trim_cache size 0 max 16384 2023-07-18T03:31:57.009 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/socket.h usability... yes 2023-07-18T03:31:57.013 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/socket.h presence... yes 2023-07-18T03:31:57.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.016+0000 7fa09f9cfb80 1 Processor -- start 2023-07-18T03:31:57.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.016+0000 7fa09f9cfb80 1 -- start start 2023-07-18T03:31:57.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa09f9cfb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259e4d0 0x55a5125988b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:57.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa09f9cfb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259ee20 0x55a5125af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:57.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa09f9cfb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259fc70 0x55a5125a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:57.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa09f9cfb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5125b3900 con 0x55a51259fc70 2023-07-18T03:31:57.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa09f9cfb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5125a5e70 con 0x55a51259ee20 2023-07-18T03:31:57.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa09f9cfb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5125b3ae0 con 0x55a51259e4d0 2023-07-18T03:31:57.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa091de6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259fc70 0x55a5125a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:57.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa091de6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259fc70 0x55a5125a60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58850/0 (socket says 172.21.15.40:58850) 2023-07-18T03:31:57.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa090de4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259ee20 0x55a5125af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:57.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa091de6700 1 -- 172.21.15.40:0/814564880 learned_addr learned my addr 172.21.15.40:0/814564880 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:57.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa0915e5700 1 --2- 172.21.15.40:0/814564880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259e4d0 0x55a5125988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:57.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa090de4700 1 -- 172.21.15.40:0/814564880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259e4d0 msgr2=0x55a5125988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:57.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa090de4700 1 --2- 172.21.15.40:0/814564880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259e4d0 0x55a5125988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa090de4700 1 -- 172.21.15.40:0/814564880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259fc70 msgr2=0x55a5125a60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:57.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa090de4700 1 --2- 172.21.15.40:0/814564880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259fc70 0x55a5125a60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa090de4700 1 -- 172.21.15.40:0/814564880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a512344250 con 0x55a51259ee20 2023-07-18T03:31:57.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa091de6700 1 --2- 172.21.15.40:0/814564880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259fc70 0x55a5125a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:57.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.017+0000 7fa0915e5700 1 --2- 172.21.15.40:0/814564880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259e4d0 0x55a5125988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:31:57.018 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/socket.h... yes 2023-07-18T03:31:57.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.018+0000 7fa090de4700 1 --2- 172.21.15.40:0/814564880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259ee20 0x55a5125af540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009ce0 tx=0x7fa07c00b2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bd7612042f719b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:57.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.018+0000 7fa077fff700 1 -- 172.21.15.40:0/814564880 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c01d070 con 0x55a51259ee20 2023-07-18T03:31:57.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.018+0000 7fa077fff700 1 -- 172.21.15.40:0/814564880 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa07c00f460 con 0x55a51259ee20 2023-07-18T03:31:57.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.018+0000 7fa077fff700 1 -- 172.21.15.40:0/814564880 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c03e500 con 0x55a51259ee20 2023-07-18T03:31:57.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.018+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/814564880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259ee20 msgr2=0x55a5125af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:57.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.018+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/814564880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259ee20 0x55a5125af540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009ce0 tx=0x7fa07c00b2e0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.018+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/814564880 shutdown_connections 2023-07-18T03:31:57.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.018+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/814564880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259fc70 0x55a5125a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.018+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/814564880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259e4d0 0x55a5125988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.018+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/814564880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259ee20 0x55a5125af540 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.019+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/814564880 >> 172.21.15.40:0/814564880 conn(0x55a512593da0 msgr2=0x55a5125a6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:57.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.019+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/814564880 shutdown_connections 2023-07-18T03:31:57.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.019+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/814564880 wait complete. 2023-07-18T03:31:57.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.019+0000 7fa09f9cfb80 1 -- start start 2023-07-18T03:31:57.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.019+0000 7fa09f9cfb80 1 Processor -- start 2023-07-18T03:31:57.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.019+0000 7fa09f9cfb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259e4d0 0x55a5125af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:57.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.019+0000 7fa09f9cfb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259ee20 0x55a5125af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:57.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.019+0000 7fa09f9cfb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259fc70 0x55a5125abc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:57.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.019+0000 7fa09f9cfb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5125ac310 con 0x55a51259fc70 2023-07-18T03:31:57.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.019+0000 7fa09f9cfb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5125ac480 con 0x55a51259e4d0 2023-07-18T03:31:57.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.019+0000 7fa09f9cfb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5125ac640 con 0x55a51259ee20 2023-07-18T03:31:57.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.019+0000 7fa09f9cfb80 10 client.0 _ll_register_callbacks cb 0x55a51248d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:31:57.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa0915e5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259e4d0 0x55a5125af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:57.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa091de6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259fc70 0x55a5125abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:57.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa0915e5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259e4d0 0x55a5125af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42178/0 (socket says 172.21.15.40:42178) 2023-07-18T03:31:57.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa0915e5700 1 -- 172.21.15.40:0/573547249 learned_addr learned my addr 172.21.15.40:0/573547249 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:31:57.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa091de6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259fc70 0x55a5125abc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58856/0 (socket says 172.21.15.40:58856) 2023-07-18T03:31:57.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa090de4700 1 --2- 172.21.15.40:0/573547249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259ee20 0x55a5125af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:57.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa0915e5700 1 -- 172.21.15.40:0/573547249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259ee20 msgr2=0x55a5125af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:57.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa0915e5700 1 --2- 172.21.15.40:0/573547249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259ee20 0x55a5125af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa0915e5700 1 -- 172.21.15.40:0/573547249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259fc70 msgr2=0x55a5125abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:31:57.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa0915e5700 1 --2- 172.21.15.40:0/573547249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259fc70 0x55a5125abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa0915e5700 1 -- 172.21.15.40:0/573547249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa084009170 con 0x55a51259e4d0 2023-07-18T03:31:57.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa091de6700 1 --2- 172.21.15.40:0/573547249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259fc70 0x55a5125abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:31:57.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa0915e5700 1 --2- 172.21.15.40:0/573547249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259e4d0 0x55a5125af050 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa088098700 tx=0x7fa088099cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f90081023c2d63c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:31:57.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa074ff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:31:57.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa074ff9700 1 -- 172.21.15.40:0/573547249 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa088098970 con 0x55a51259e4d0 2023-07-18T03:31:57.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/573547249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa07c004cd0 con 0x55a51259e4d0 2023-07-18T03:31:57.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa074ff9700 1 -- 172.21.15.40:0/573547249 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa0880a1740 con 0x55a51259e4d0 2023-07-18T03:31:57.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa074ff9700 1 -- 172.21.15.40:0/573547249 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa088098ad0 con 0x55a51259e4d0 2023-07-18T03:31:57.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa09f9cfb80 10 client.25493 fetch_fsmap learned FSMap version 27 2023-07-18T03:31:57.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.020+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/573547249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fa07c035040 con 0x55a51259e4d0 2023-07-18T03:31:57.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.021+0000 7fa074ff9700 1 -- 172.21.15.40:0/573547249 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa07c004cd0 con 0x55a51259e4d0 2023-07-18T03:31:57.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.021+0000 7fa074ff9700 1 -- 172.21.15.40:0/573547249 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa088098d00 con 0x55a51259e4d0 2023-07-18T03:31:57.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.021+0000 7fa09f9cfb80 10 client.25493 fetch_fsmap finished waiting for FSMap version 27 2023-07-18T03:31:57.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.021+0000 7fa09f9cfb80 10 client.25493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:31:57.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.021+0000 7fa09f9cfb80 20 client.25493 populate_metadata read hostname 'smithi040' 2023-07-18T03:31:57.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.021+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/573547249 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55a512609000 0x55a51260b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:31:57.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.021+0000 7fa09f9cfb80 4 client.25493 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:31:57.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.021+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/573547249 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a5124ab050 con 0x55a512609000 2023-07-18T03:31:57.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.021+0000 7fa090de4700 1 --2- 172.21.15.40:0/573547249 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55a512609000 0x55a51260b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:31:57.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.021+0000 7fa090de4700 1 --2- 172.21.15.40:0/573547249 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55a512609000 0x55a51260b410 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e64c46359a2a0a server_cookie=3c87fbb3951c9f6c in_seq=0 out_seq=0 2023-07-18T03:31:57.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.021+0000 7fa074ff9700 10 client.25493 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:57.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa074ff9700 1 -- 172.21.15.40:0/573547249 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a5124ab050 con 0x55a512609000 2023-07-18T03:31:57.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa074ff9700 10 client.25493 handle_command_reply: tid=0 2023-07-18T03:31:57.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa09f9cfb80 1 client.25493 shutdown 2023-07-18T03:31:57.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/573547249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259e4d0 msgr2=0x55a5125af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:57.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/573547249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259e4d0 0x55a5125af050 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa088098700 tx=0x7fa088099cc0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/573547249 shutdown_connections 2023-07-18T03:31:57.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/573547249 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55a512609000 0x55a51260b410 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/573547249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a51259fc70 0x55a5125abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa074ff9700 0 client.25493 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:31:57.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/573547249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a51259e4d0 0x55a5125af050 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa09f9cfb80 1 --2- 172.21.15.40:0/573547249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a51259ee20 0x55a5125af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/573547249 >> 172.21.15.40:0/573547249 conn(0x55a512593da0 msgr2=0x55a5125a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:57.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/573547249 shutdown_connections 2023-07-18T03:31:57.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa09f9cfb80 1 -- 172.21.15.40:0/573547249 wait complete. 2023-07-18T03:31:57.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa09f9cfb80 20 client.25493 trim_cache size 0 max 16384 2023-07-18T03:31:57.032 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:31:57.032 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:31:57.032 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:31:57.032 INFO:teuthology.orchestra.run.smithi040.stdout: "5e1c5c11-079c-45b0-a60e-91da5f7ba4f1": { 2023-07-18T03:31:57.032 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:31:57.032 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "5e1c5c11-079c-45b0-a60e-91da5f7ba4f1", 2023-07-18T03:31:57.033 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:31:57.033 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:31:57.033 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:31:57.033 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:31:57.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa093849700 1 -- 172.21.15.40:0/3875424152 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0780a1150 msgr2=0x7fa0780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa093849700 1 --2- 172.21.15.40:0/3875424152 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0780a1150 0x7fa0780a3600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa07c005670 tx=0x7fa07c0056a0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa093849700 1 -- 172.21.15.40:0/3875424152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa08c0ffae0 msgr2=0x7fa08c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:31:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa093849700 1 --2- 172.21.15.40:0/3875424152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa08c0ffae0 0x7fa08c0742c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa08800c940 tx=0x7fa08800cc50 comp rx=0 tx=0).stop 2023-07-18T03:31:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa093849700 1 -- 172.21.15.40:0/3875424152 shutdown_connections 2023-07-18T03:31:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa093849700 1 --2- 172.21.15.40:0/3875424152 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0780a1150 0x7fa0780a3600 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.023+0000 7fa093849700 1 --2- 172.21.15.40:0/3875424152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa08c101660 0x7fa08c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.024+0000 7fa093849700 1 --2- 172.21.15.40:0/3875424152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa08c0ffae0 0x7fa08c0742c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.024+0000 7fa093849700 1 --2- 172.21.15.40:0/3875424152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa08c100cd0 0x7fa08c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:31:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.024+0000 7fa093849700 1 -- 172.21.15.40:0/3875424152 >> 172.21.15.40:0/3875424152 conn(0x7fa08c0fb120 msgr2=0x7fa08c0fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:31:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.024+0000 7fa093849700 1 -- 172.21.15.40:0/3875424152 shutdown_connections 2023-07-18T03:31:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:31:57.024+0000 7fa093849700 1 -- 172.21.15.40:0/3875424152 wait complete. 2023-07-18T03:31:57.036 INFO:tasks.cephfs.filesystem:scrub status for tag:5e1c5c11-079c-45b0-a60e-91da5f7ba4f1 - {'path': '/', 'tag': '5e1c5c11-079c-45b0-a60e-91da5f7ba4f1', 'options': 'recursive,force'} 2023-07-18T03:31:57.068 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/time.h usability... yes 2023-07-18T03:31:57.084 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/time.h presence... yes 2023-07-18T03:31:57.084 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/time.h... yes 2023-07-18T03:31:57.139 INFO:tasks.workunit.client.0.smithi040.stdout:checking fcntl.h usability... yes 2023-07-18T03:31:57.154 INFO:tasks.workunit.client.0.smithi040.stdout:checking fcntl.h presence... yes 2023-07-18T03:31:57.155 INFO:tasks.workunit.client.0.smithi040.stdout:checking for fcntl.h... yes 2023-07-18T03:31:57.208 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/fcntl.h usability... yes 2023-07-18T03:31:57.224 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/fcntl.h presence... yes 2023-07-18T03:31:57.225 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/fcntl.h... yes 2023-07-18T03:31:57.287 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/param.h usability... yes 2023-07-18T03:31:57.305 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/param.h presence... yes 2023-07-18T03:31:57.306 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/param.h... yes 2023-07-18T03:31:57.352 INFO:tasks.workunit.client.0.smithi040.stdout:checking ioctl.h usability... no 2023-07-18T03:31:57.368 INFO:tasks.workunit.client.0.smithi040.stdout:checking ioctl.h presence... no 2023-07-18T03:31:57.369 INFO:tasks.workunit.client.0.smithi040.stdout:checking for ioctl.h... no 2023-07-18T03:31:57.419 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/ioctl.h usability... yes 2023-07-18T03:31:57.434 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/ioctl.h presence... yes 2023-07-18T03:31:57.435 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/ioctl.h... yes 2023-07-18T03:31:57.486 INFO:tasks.workunit.client.0.smithi040.stdout:checking return type of signal handlers... void 2023-07-18T03:31:57.526 INFO:tasks.workunit.client.0.smithi040.stdout:checking for size_t... yes 2023-07-18T03:31:57.561 INFO:tasks.workunit.client.0.smithi040.stdout:checking for dev_t... yes 2023-07-18T03:31:57.602 INFO:tasks.workunit.client.0.smithi040.stdout:checking for short... yes 2023-07-18T03:31:57.679 INFO:tasks.workunit.client.0.smithi040.stdout:checking size of short... 2 2023-07-18T03:31:57.722 INFO:tasks.workunit.client.0.smithi040.stdout:checking for int... yes 2023-07-18T03:31:57.787 INFO:tasks.workunit.client.0.smithi040.stdout:checking size of int... 4 2023-07-18T03:31:57.830 INFO:tasks.workunit.client.0.smithi040.stdout:checking for long... yes 2023-07-18T03:31:57.897 INFO:tasks.workunit.client.0.smithi040.stdout:checking size of long... 8 2023-07-18T03:31:57.939 INFO:tasks.workunit.client.0.smithi040.stdout:checking for long long... yes 2023-07-18T03:31:58.005 INFO:tasks.workunit.client.0.smithi040.stdout:checking size of long long... 8 2023-07-18T03:31:58.045 INFO:tasks.workunit.client.0.smithi040.stdout:checking for ssize_t... yes 2023-07-18T03:31:58.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:57 smithi134 ceph-mon[98216]: pgmap v325: 97 pgs: 97 active+clean; 813 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 11 op/s 2023-07-18T03:31:58.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:57 smithi134 ceph-mon[98216]: scrub queued for path: / 2023-07-18T03:31:58.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:57 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:31:58.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:57 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:31:58.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:57 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:31:58.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:57 smithi040 ceph-mon[96439]: pgmap v325: 97 pgs: 97 active+clean; 813 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 11 op/s 2023-07-18T03:31:58.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:57 smithi040 ceph-mon[96439]: scrub queued for path: / 2023-07-18T03:31:58.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:57 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:31:58.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:57 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:31:58.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:57 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:31:58.081 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sig_atomic_t... yes 2023-07-18T03:31:58.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:57 smithi064 ceph-mon[98059]: pgmap v325: 97 pgs: 97 active+clean; 813 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 11 op/s 2023-07-18T03:31:58.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:57 smithi064 ceph-mon[98059]: scrub queued for path: / 2023-07-18T03:31:58.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:57 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:31:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:57 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:31:58.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:57 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:31:58.127 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether gcc needs -traditional... no 2023-07-18T03:31:58.160 INFO:tasks.workunit.client.0.smithi040.stdout:checking for an ANSI C-conforming const... yes 2023-07-18T03:31:58.179 INFO:tasks.workunit.client.0.smithi040.stdout:checking for inline... inline 2023-07-18T03:31:58.238 INFO:tasks.workunit.client.0.smithi040.stdout:checking for working alloca.h... yes 2023-07-18T03:31:58.282 INFO:tasks.workunit.client.0.smithi040.stdout:checking for alloca... yes 2023-07-18T03:31:58.348 INFO:tasks.workunit.client.0.smithi040.stdout:checking for working memcmp... yes 2023-07-18T03:31:58.401 INFO:tasks.workunit.client.0.smithi040.stdout:checking for strftime... yes 2023-07-18T03:31:58.474 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2023-07-18T03:31:58.533 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether stat accepts an empty string... no 2023-07-18T03:31:58.586 INFO:tasks.workunit.client.0.smithi040.stdout:checking for vprintf... yes 2023-07-18T03:31:58.643 INFO:tasks.workunit.client.0.smithi040.stdout:checking for _doprnt... no 2023-07-18T03:31:58.667 INFO:tasks.workunit.client.0.smithi040.stdout:checking build system type... x86_64-unknown-linux-gnu 2023-07-18T03:31:58.680 INFO:tasks.workunit.client.0.smithi040.stdout:checking host system type... x86_64-unknown-linux-gnu 2023-07-18T03:31:58.739 INFO:tasks.workunit.client.0.smithi040.stdout:checking for the pthreads library -lpthreads... no 2023-07-18T03:31:58.788 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether pthreads work without any flags... no 2023-07-18T03:31:58.803 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether pthreads work with -Kthread... no 2023-07-18T03:31:58.817 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether pthreads work with -kthread... no 2023-07-18T03:31:58.858 INFO:tasks.workunit.client.0.smithi040.stdout:checking for the pthreads library -llthread... no 2023-07-18T03:31:58.907 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether pthreads work with -pthread... yes 2023-07-18T03:31:58.950 INFO:tasks.workunit.client.0.smithi040.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2023-07-18T03:31:58.950 INFO:tasks.workunit.client.0.smithi040.stdout:checking if more special flags are required for pthreads... no 2023-07-18T03:31:58.950 INFO:tasks.workunit.client.0.smithi040.stdout:checking for cc_r... gcc 2023-07-18T03:31:59.049 INFO:tasks.workunit.client.0.smithi040.stdout:checking for setrlimit... yes 2023-07-18T03:31:59.109 INFO:tasks.workunit.client.0.smithi040.stdout:checking for setproctitle... no 2023-07-18T03:31:59.166 INFO:tasks.workunit.client.0.smithi040.stdout:checking for getopt_long... yes 2023-07-18T03:31:59.226 INFO:tasks.workunit.client.0.smithi040.stdout:checking for strtoull... yes 2023-07-18T03:31:59.287 INFO:tasks.workunit.client.0.smithi040.stdout:checking for strtoq... yes 2023-07-18T03:31:59.348 INFO:tasks.workunit.client.0.smithi040.stdout:checking for snprintf... yes 2023-07-18T03:31:59.410 INFO:tasks.workunit.client.0.smithi040.stdout:checking for vsnprintf... yes 2023-07-18T03:31:59.471 INFO:tasks.workunit.client.0.smithi040.stdout:checking for vfprintf... yes 2023-07-18T03:31:59.664 INFO:tasks.workunit.client.0.smithi040.stdout:checking for gethostname... yes 2023-07-18T03:31:59.827 INFO:tasks.workunit.client.0.smithi040.stdout:checking for setlocale... yes 2023-07-18T03:31:59.898 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sync... yes 2023-07-18T03:31:59.963 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether snprintf is C99 conformant... done 2023-07-18T03:31:59.974 INFO:tasks.workunit.client.0.smithi040.stdout:configure: creating ./config.status 2023-07-18T03:32:00.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:31:59 smithi134 ceph-mon[98216]: pgmap v326: 97 pgs: 97 active+clean; 813 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 19 op/s 2023-07-18T03:32:00.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:31:59 smithi040 ceph-mon[96439]: pgmap v326: 97 pgs: 97 active+clean; 813 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 19 op/s 2023-07-18T03:32:00.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:31:59 smithi064 ceph-mon[98059]: pgmap v326: 97 pgs: 97 active+clean; 813 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 19 op/s 2023-07-18T03:32:00.120 INFO:tasks.workunit.client.0.smithi040.stdout:config.status: creating Makefile 2023-07-18T03:32:00.136 INFO:tasks.workunit.client.0.smithi040.stdout:config.status: creating src/Makefile 2023-07-18T03:32:00.152 INFO:tasks.workunit.client.0.smithi040.stdout:config.status: creating man/Makefile 2023-07-18T03:32:00.161 INFO:tasks.workunit.client.0.smithi040.stdout:config.status: creating config.h 2023-07-18T03:32:00.208 INFO:tasks.workunit.client.0.smithi040.stdout:config.status: executing depfiles commands 2023-07-18T03:32:00.329 INFO:tasks.workunit.client.0.smithi040.stderr:+ make 2023-07-18T03:32:00.341 INFO:tasks.workunit.client.0.smithi040.stdout:make all-recursive 2023-07-18T03:32:00.342 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-07-18T03:32:00.344 INFO:tasks.workunit.client.0.smithi040.stdout:Making all in src 2023-07-18T03:32:00.353 INFO:tasks.workunit.client.0.smithi040.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2023-07-18T03:32:00.353 INFO:tasks.workunit.client.0.smithi040.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2023-07-18T03:32:00.353 INFO:tasks.workunit.client.0.smithi040.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2023-07-18T03:32:00.423 INFO:tasks.workunit.client.0.smithi040.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2023-07-18T03:32:00.424 INFO:tasks.workunit.client.0.smithi040.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2023-07-18T03:32:00.485 INFO:tasks.workunit.client.0.smithi040.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2023-07-18T03:32:00.485 INFO:tasks.workunit.client.0.smithi040.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2023-07-18T03:32:00.534 INFO:tasks.workunit.client.0.smithi040.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2023-07-18T03:32:00.534 INFO:tasks.workunit.client.0.smithi040.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2023-07-18T03:32:00.582 INFO:tasks.workunit.client.0.smithi040.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2023-07-18T03:32:00.582 INFO:tasks.workunit.client.0.smithi040.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2023-07-18T03:32:00.629 INFO:tasks.workunit.client.0.smithi040.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2023-07-18T03:32:00.629 INFO:tasks.workunit.client.0.smithi040.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2023-07-18T03:32:00.676 INFO:tasks.workunit.client.0.smithi040.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2023-07-18T03:32:00.676 INFO:tasks.workunit.client.0.smithi040.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2023-07-18T03:32:00.761 INFO:tasks.workunit.client.0.smithi040.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2023-07-18T03:32:00.761 INFO:tasks.workunit.client.0.smithi040.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2023-07-18T03:32:00.793 INFO:tasks.workunit.client.0.smithi040.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2023-07-18T03:32:00.794 INFO:tasks.workunit.client.0.smithi040.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2023-07-18T03:32:00.824 INFO:tasks.workunit.client.0.smithi040.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2023-07-18T03:32:00.825 INFO:tasks.workunit.client.0.smithi040.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2023-07-18T03:32:00.885 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -g -O2 -pthread -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2023-07-18T03:32:00.922 INFO:tasks.workunit.client.0.smithi040.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2023-07-18T03:32:00.923 INFO:tasks.workunit.client.0.smithi040.stdout:Making all in man 2023-07-18T03:32:00.928 INFO:tasks.workunit.client.0.smithi040.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2023-07-18T03:32:00.928 INFO:tasks.workunit.client.0.smithi040.stdout:make[2]: Nothing to be done for 'all'. 2023-07-18T03:32:00.928 INFO:tasks.workunit.client.0.smithi040.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2023-07-18T03:32:00.929 INFO:tasks.workunit.client.0.smithi040.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-07-18T03:32:00.930 INFO:tasks.workunit.client.0.smithi040.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-07-18T03:32:00.930 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-07-18T03:32:00.931 INFO:tasks.workunit.client.0.smithi040.stderr:+ cd src 2023-07-18T03:32:00.931 INFO:tasks.workunit.client.0.smithi040.stderr:+ mkdir blogtest_in 2023-07-18T03:32:00.933 INFO:tasks.workunit.client.0.smithi040.stdout:running blogbench 2023-07-18T03:32:00.934 INFO:tasks.workunit.client.0.smithi040.stderr:+ echo 'running blogbench' 2023-07-18T03:32:00.934 INFO:tasks.workunit.client.0.smithi040.stderr:+ ./blogbench -d blogtest_in 2023-07-18T03:32:00.935 INFO:tasks.workunit.client.0.smithi040.stdout: 2023-07-18T03:32:00.935 INFO:tasks.workunit.client.0.smithi040.stdout:Frequency = 10 secs 2023-07-18T03:32:00.935 INFO:tasks.workunit.client.0.smithi040.stdout:Scratch dir = [blogtest_in] 2023-07-18T03:32:01.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:00 smithi134 ceph-mon[98216]: scrub summary: idle 2023-07-18T03:32:01.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:00 smithi040 ceph-mon[96439]: scrub summary: idle 2023-07-18T03:32:01.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:00 smithi064 ceph-mon[98059]: scrub summary: idle 2023-07-18T03:32:02.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:01 smithi134 ceph-mon[98216]: pgmap v327: 97 pgs: 97 active+clean; 934 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 KiB/s wr, 24 op/s 2023-07-18T03:32:02.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:01 smithi040 ceph-mon[96439]: pgmap v327: 97 pgs: 97 active+clean; 934 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 KiB/s wr, 24 op/s 2023-07-18T03:32:02.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:01 smithi064 ceph-mon[98059]: pgmap v327: 97 pgs: 97 active+clean; 934 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 KiB/s wr, 24 op/s 2023-07-18T03:32:03.141 INFO:tasks.workunit.client.0.smithi040.stdout:Spawning 3 writers... 2023-07-18T03:32:03.141 INFO:tasks.workunit.client.0.smithi040.stdout:Spawning 1 rewriters... 2023-07-18T03:32:03.142 INFO:tasks.workunit.client.0.smithi040.stdout:Spawning 5 commenters... 2023-07-18T03:32:03.142 INFO:tasks.workunit.client.0.smithi040.stdout:Spawning 100 readers... 2023-07-18T03:32:03.144 INFO:tasks.workunit.client.0.smithi040.stdout:Benchmarking for 30 iterations. 2023-07-18T03:32:03.144 INFO:tasks.workunit.client.0.smithi040.stdout:The test will run during 5 minutes. 2023-07-18T03:32:03.144 INFO:tasks.workunit.client.0.smithi040.stdout: 2023-07-18T03:32:03.144 INFO:tasks.workunit.client.0.smithi040.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2023-07-18T03:32:04.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:03 smithi134 ceph-mon[98216]: pgmap v328: 97 pgs: 97 active+clean; 4.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 369 KiB/s wr, 80 op/s 2023-07-18T03:32:04.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:03 smithi040 ceph-mon[96439]: pgmap v328: 97 pgs: 97 active+clean; 4.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 369 KiB/s wr, 80 op/s 2023-07-18T03:32:04.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:03 smithi064 ceph-mon[98059]: pgmap v328: 97 pgs: 97 active+clean; 4.4 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 369 KiB/s wr, 80 op/s 2023-07-18T03:32:06.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:05 smithi134 ceph-mon[98216]: pgmap v329: 97 pgs: 97 active+clean; 4.6 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 410 KiB/s wr, 89 op/s 2023-07-18T03:32:06.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:05 smithi040 ceph-mon[96439]: pgmap v329: 97 pgs: 97 active+clean; 4.6 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 410 KiB/s wr, 89 op/s 2023-07-18T03:32:06.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:05 smithi064 ceph-mon[98059]: pgmap v329: 97 pgs: 97 active+clean; 4.6 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 410 KiB/s wr, 89 op/s 2023-07-18T03:32:07.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:06 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:32:07.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:06 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:32:07.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:06 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:32:08.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:07 smithi134 ceph-mon[98216]: pgmap v330: 97 pgs: 97 active+clean; 5.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 538 KiB/s wr, 99 op/s 2023-07-18T03:32:08.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:07 smithi040 ceph-mon[96439]: pgmap v330: 97 pgs: 97 active+clean; 5.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 538 KiB/s wr, 99 op/s 2023-07-18T03:32:08.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:07 smithi064 ceph-mon[98059]: pgmap v330: 97 pgs: 97 active+clean; 5.8 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 538 KiB/s wr, 99 op/s 2023-07-18T03:32:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.925+0000 7fde6788b700 1 -- 172.21.15.40:0/2453639211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde600feab0 msgr2=0x7fde600feec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.925+0000 7fde6788b700 1 --2- 172.21.15.40:0/2453639211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde600feab0 0x7fde600feec0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009990 tx=0x7fde5c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.925+0000 7fde6788b700 1 -- 172.21.15.40:0/2453639211 shutdown_connections 2023-07-18T03:32:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.925+0000 7fde6788b700 1 --2- 172.21.15.40:0/2453639211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde60100630 0x7fde60106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.925+0000 7fde6788b700 1 --2- 172.21.15.40:0/2453639211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde600feab0 0x7fde600feec0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.925+0000 7fde6788b700 1 --2- 172.21.15.40:0/2453639211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde600ffca0 0x7fde601000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.925+0000 7fde6788b700 1 -- 172.21.15.40:0/2453639211 >> 172.21.15.40:0/2453639211 conn(0x7fde600fa0d0 msgr2=0x7fde600fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.926+0000 7fde6788b700 1 -- 172.21.15.40:0/2453639211 shutdown_connections 2023-07-18T03:32:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.926+0000 7fde6788b700 1 -- 172.21.15.40:0/2453639211 wait complete. 2023-07-18T03:32:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.926+0000 7fde6788b700 1 Processor -- start 2023-07-18T03:32:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.926+0000 7fde6788b700 1 -- start start 2023-07-18T03:32:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.926+0000 7fde6788b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde600feab0 0x7fde601948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.926+0000 7fde6788b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde600ffca0 0x7fde60194de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.926+0000 7fde6788b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde60100630 0x7fde60195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.926+0000 7fde6788b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde601962c0 con 0x7fde600ffca0 2023-07-18T03:32:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.926+0000 7fde6788b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde60196430 con 0x7fde600feab0 2023-07-18T03:32:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.926+0000 7fde6788b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde601965f0 con 0x7fde60100630 2023-07-18T03:32:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.926+0000 7fde64e26700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde600ffca0 0x7fde60194de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.927+0000 7fde64e26700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde600ffca0 0x7fde60194de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58890/0 (socket says 172.21.15.40:58890) 2023-07-18T03:32:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.927+0000 7fde65e28700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde60100630 0x7fde60195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.927+0000 7fde64e26700 1 -- 172.21.15.40:0/668310121 learned_addr learned my addr 172.21.15.40:0/668310121 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.927+0000 7fde65627700 1 --2- 172.21.15.40:0/668310121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde600feab0 0x7fde601948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.927+0000 7fde64e26700 1 -- 172.21.15.40:0/668310121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde60100630 msgr2=0x7fde60195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.927+0000 7fde64e26700 1 --2- 172.21.15.40:0/668310121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde60100630 0x7fde60195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.927+0000 7fde64e26700 1 -- 172.21.15.40:0/668310121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde600feab0 msgr2=0x7fde601948a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.927+0000 7fde64e26700 1 --2- 172.21.15.40:0/668310121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde600feab0 0x7fde601948a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.927+0000 7fde64e26700 1 -- 172.21.15.40:0/668310121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c0095f0 con 0x7fde600ffca0 2023-07-18T03:32:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.927+0000 7fde65e28700 1 --2- 172.21.15.40:0/668310121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde60100630 0x7fde60195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.927+0000 7fde65627700 1 --2- 172.21.15.40:0/668310121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde600feab0 0x7fde601948a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:32:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.927+0000 7fde64e26700 1 --2- 172.21.15.40:0/668310121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde600ffca0 0x7fde60194de0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fde5000e9a0 tx=0x7fde5000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.927+0000 7fde567fc700 1 -- 172.21.15.40:0/668310121 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50009790 con 0x7fde600ffca0 2023-07-18T03:32:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.927+0000 7fde567fc700 1 -- 172.21.15.40:0/668310121 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde5000cbd0 con 0x7fde600ffca0 2023-07-18T03:32:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.927+0000 7fde6788b700 1 -- 172.21.15.40:0/668310121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde601968d0 con 0x7fde600ffca0 2023-07-18T03:32:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.927+0000 7fde567fc700 1 -- 172.21.15.40:0/668310121 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50018680 con 0x7fde600ffca0 2023-07-18T03:32:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.927+0000 7fde6788b700 1 -- 172.21.15.40:0/668310121 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6019d090 con 0x7fde600ffca0 2023-07-18T03:32:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.929+0000 7fde567fc700 1 -- 172.21.15.40:0/668310121 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fde5000cd40 con 0x7fde600ffca0 2023-07-18T03:32:09.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.930+0000 7fde567fc700 1 --2- 172.21.15.40:0/668310121 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fde4c0a1150 0x7fde4c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.930+0000 7fde65627700 1 --2- 172.21.15.40:0/668310121 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fde4c0a1150 0x7fde4c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.930+0000 7fde567fc700 1 -- 172.21.15.40:0/668310121 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fde50014070 con 0x7fde600ffca0 2023-07-18T03:32:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.930+0000 7fde65627700 1 --2- 172.21.15.40:0/668310121 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fde4c0a1150 0x7fde4c0a3600 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fde5c000c00 tx=0x7fde5c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.931+0000 7fde73a11b80 1 Processor -- start 2023-07-18T03:32:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.931+0000 7fde73a11b80 1 -- start start 2023-07-18T03:32:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.931+0000 7fde73a11b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb565b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.931+0000 7fde73a11b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb56af0 0x55d33bb57370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.931+0000 7fde73a11b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb578b0 0x55d33bb57d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.931+0000 7fde73a11b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33ba59c50 con 0x55d33b9b06b0 2023-07-18T03:32:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.931+0000 7fde73a11b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33baac9e0 con 0x55d33bb578b0 2023-07-18T03:32:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.931+0000 7fde73a11b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33baa5e70 con 0x55d33bb56af0 2023-07-18T03:32:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.931+0000 7fde65627700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.931+0000 7fde64e26700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb56af0 0x55d33bb57370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.931+0000 7fde65627700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58896/0 (socket says 172.21.15.40:58896) 2023-07-18T03:32:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.931+0000 7fde65627700 1 -- 172.21.15.40:0/2175354819 learned_addr learned my addr 172.21.15.40:0/2175354819 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.932+0000 7fde65e28700 1 --2- 172.21.15.40:0/2175354819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb578b0 0x55d33bb57d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.932+0000 7fde65627700 1 -- 172.21.15.40:0/2175354819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb56af0 msgr2=0x55d33bb57370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.932+0000 7fde65627700 1 --2- 172.21.15.40:0/2175354819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb56af0 0x55d33bb57370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.932+0000 7fde65627700 1 -- 172.21.15.40:0/2175354819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb578b0 msgr2=0x55d33bb57d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.932+0000 7fde65627700 1 --2- 172.21.15.40:0/2175354819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb578b0 0x55d33bb57d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.932+0000 7fde65627700 1 -- 172.21.15.40:0/2175354819 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d33b8d3d80 con 0x55d33b9b06b0 2023-07-18T03:32:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.932+0000 7fde64e26700 1 --2- 172.21.15.40:0/2175354819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb56af0 0x55d33bb57370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.932+0000 7fde65627700 1 --2- 172.21.15.40:0/2175354819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb565b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fde5c015000 tx=0x7fde5c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ab4fda8d65a261 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.932+0000 7fde4b7fe700 1 -- 172.21.15.40:0/2175354819 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c02b070 con 0x55d33b9b06b0 2023-07-18T03:32:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.932+0000 7fde4b7fe700 1 -- 172.21.15.40:0/2175354819 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde5c01e040 con 0x55d33b9b06b0 2023-07-18T03:32:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.932+0000 7fde4b7fe700 1 -- 172.21.15.40:0/2175354819 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c024810 con 0x55d33b9b06b0 2023-07-18T03:32:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.933+0000 7fde73a11b80 1 -- 172.21.15.40:0/2175354819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 msgr2=0x55d33bb565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.933+0000 7fde73a11b80 1 --2- 172.21.15.40:0/2175354819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb565b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fde5c015000 tx=0x7fde5c01f000 comp rx=0 tx=0).stop 2023-07-18T03:32:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.933+0000 7fde73a11b80 1 -- 172.21.15.40:0/2175354819 shutdown_connections 2023-07-18T03:32:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.933+0000 7fde73a11b80 1 --2- 172.21.15.40:0/2175354819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb578b0 0x55d33bb57d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.933+0000 7fde73a11b80 1 --2- 172.21.15.40:0/2175354819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb565b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.933+0000 7fde73a11b80 1 --2- 172.21.15.40:0/2175354819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb56af0 0x55d33bb57370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.933+0000 7fde73a11b80 1 -- 172.21.15.40:0/2175354819 >> 172.21.15.40:0/2175354819 conn(0x55d33bb49be0 msgr2=0x55d33bb4ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.933+0000 7fde73a11b80 1 -- 172.21.15.40:0/2175354819 shutdown_connections 2023-07-18T03:32:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.933+0000 7fde73a11b80 1 -- 172.21.15.40:0/2175354819 wait complete. 2023-07-18T03:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.933+0000 7fde73a11b80 1 -- start start 2023-07-18T03:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.933+0000 7fde73a11b80 1 Processor -- start 2023-07-18T03:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde73a11b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb5f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde73a11b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb56af0 0x55d33bb5ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde73a11b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb578b0 0x55d33bb66f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde73a11b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33ba45a60 con 0x55d33b9b06b0 2023-07-18T03:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde73a11b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33ba45760 con 0x55d33bb56af0 2023-07-18T03:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde73a11b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33b9298c0 con 0x55d33bb578b0 2023-07-18T03:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde73a11b80 10 client.0 _ll_register_callbacks cb 0x55d33ba43700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde65627700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb5f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde65627700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb5f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58902/0 (socket says 172.21.15.40:58902) 2023-07-18T03:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde65627700 1 -- 172.21.15.40:0/965164616 learned_addr learned my addr 172.21.15.40:0/965164616 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde65e28700 1 --2- 172.21.15.40:0/965164616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb578b0 0x55d33bb66f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde64e26700 1 --2- 172.21.15.40:0/965164616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb56af0 0x55d33bb5ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde65627700 1 -- 172.21.15.40:0/965164616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb578b0 msgr2=0x55d33bb66f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde65627700 1 --2- 172.21.15.40:0/965164616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb578b0 0x55d33bb66f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde65627700 1 -- 172.21.15.40:0/965164616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb56af0 msgr2=0x55d33bb5ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde65627700 1 --2- 172.21.15.40:0/965164616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb56af0 0x55d33bb5ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde65627700 1 -- 172.21.15.40:0/965164616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6019d090 con 0x55d33b9b06b0 2023-07-18T03:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde65e28700 1 --2- 172.21.15.40:0/965164616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb578b0 0x55d33bb66f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde65627700 1 --2- 172.21.15.40:0/965164616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb5f9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009ce0 tx=0x7fde5c00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a164ed0acc7b9746 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde48ff9700 1 -- 172.21.15.40:0/965164616 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c02b070 con 0x55d33b9b06b0 2023-07-18T03:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.934+0000 7fde48ff9700 1 -- 172.21.15.40:0/965164616 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fde5c00f6f0 con 0x55d33b9b06b0 2023-07-18T03:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.935+0000 7fde73a11b80 1 -- 172.21.15.40:0/965164616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d33b9ff490 con 0x55d33b9b06b0 2023-07-18T03:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.935+0000 7fde48ff9700 1 -- 172.21.15.40:0/965164616 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde5c00f8b0 con 0x55d33b9b06b0 2023-07-18T03:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.935+0000 7fde73a11b80 10 client.15846 fetch_fsmap learned FSMap version 28 2023-07-18T03:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.935+0000 7fde73a11b80 1 -- 172.21.15.40:0/965164616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fde60107820 con 0x55d33b9b06b0 2023-07-18T03:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.935+0000 7fde48ff9700 1 -- 172.21.15.40:0/965164616 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d33b9ff490 con 0x55d33b9b06b0 2023-07-18T03:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.935+0000 7fde48ff9700 1 -- 172.21.15.40:0/965164616 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fde5c040030 con 0x55d33b9b06b0 2023-07-18T03:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.935+0000 7fde73a11b80 10 client.15846 fetch_fsmap finished waiting for FSMap version 28 2023-07-18T03:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.935+0000 7fde73a11b80 10 client.15846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.935+0000 7fde73a11b80 20 client.15846 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.935+0000 7fde73a11b80 1 --2- 172.21.15.40:0/965164616 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d33bb6b2a0 0x55d33bb6d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.935+0000 7fde73a11b80 4 client.15846 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.935+0000 7fde73a11b80 1 -- 172.21.15.40:0/965164616 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d33ba9ee50 con 0x55d33bb6b2a0 2023-07-18T03:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.935+0000 7fde64e26700 1 --2- 172.21.15.40:0/965164616 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d33bb6b2a0 0x55d33bb6d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.936+0000 7fde48ff9700 10 client.15846 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.936+0000 7fde64e26700 1 --2- 172.21.15.40:0/965164616 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d33bb6b2a0 0x55d33bb6d6b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=735e3ae71162fa5b server_cookie=2881d15b23a8c227 in_seq=0 out_seq=0 2023-07-18T03:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.936+0000 7fde48ff9700 1 -- 172.21.15.40:0/965164616 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d33ba9ee50 con 0x55d33bb6b2a0 2023-07-18T03:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.936+0000 7fde48ff9700 10 client.15846 handle_command_reply: tid=0 2023-07-18T03:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.936+0000 7fde73a11b80 1 client.15846 shutdown 2023-07-18T03:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.937+0000 7fde73a11b80 1 -- 172.21.15.40:0/965164616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 msgr2=0x55d33bb5f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.937+0000 7fde73a11b80 1 --2- 172.21.15.40:0/965164616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb5f9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009ce0 tx=0x7fde5c00fce0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.937+0000 7fde73a11b80 1 -- 172.21.15.40:0/965164616 shutdown_connections 2023-07-18T03:32:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.937+0000 7fde73a11b80 1 --2- 172.21.15.40:0/965164616 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d33bb6b2a0 0x55d33bb6d6b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.937+0000 7fde73a11b80 1 --2- 172.21.15.40:0/965164616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb578b0 0x55d33bb66f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.937+0000 7fde73a11b80 1 --2- 172.21.15.40:0/965164616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb5f9c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.937+0000 7fde48ff9700 0 client.15846 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.937+0000 7fde73a11b80 1 --2- 172.21.15.40:0/965164616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb56af0 0x55d33bb5ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.937+0000 7fde73a11b80 1 -- 172.21.15.40:0/965164616 >> 172.21.15.40:0/965164616 conn(0x55d33bb49be0 msgr2=0x55d33bb4e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.937+0000 7fde73a11b80 1 -- 172.21.15.40:0/965164616 shutdown_connections 2023-07-18T03:32:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.937+0000 7fde73a11b80 1 -- 172.21.15.40:0/965164616 wait complete. 2023-07-18T03:32:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.937+0000 7fde73a11b80 20 client.15846 trim_cache size 0 max 16384 2023-07-18T03:32:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.951+0000 7fde73a11b80 1 Processor -- start 2023-07-18T03:32:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde73a11b80 1 -- start start 2023-07-18T03:32:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde73a11b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb5f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde73a11b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb56af0 0x55d33bb72010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde73a11b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb578b0 0x55d33bb5a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde73a11b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33bb5ee70 con 0x55d33b9b06b0 2023-07-18T03:32:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde73a11b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33bb57110 con 0x55d33bb56af0 2023-07-18T03:32:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde73a11b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33bb4bd00 con 0x55d33bb578b0 2023-07-18T03:32:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde65627700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb5f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde65627700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb5f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58910/0 (socket says 172.21.15.40:58910) 2023-07-18T03:32:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde65627700 1 -- 172.21.15.40:0/1763205058 learned_addr learned my addr 172.21.15.40:0/1763205058 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde65e28700 1 --2- 172.21.15.40:0/1763205058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb578b0 0x55d33bb5a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde65627700 1 -- 172.21.15.40:0/1763205058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb578b0 msgr2=0x55d33bb5a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde65627700 1 --2- 172.21.15.40:0/1763205058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb578b0 0x55d33bb5a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde65627700 1 -- 172.21.15.40:0/1763205058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb56af0 msgr2=0x55d33bb72010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde65627700 1 --2- 172.21.15.40:0/1763205058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb56af0 0x55d33bb72010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde65627700 1 -- 172.21.15.40:0/1763205058 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d33b8bcc00 con 0x55d33b9b06b0 2023-07-18T03:32:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde65e28700 1 --2- 172.21.15.40:0/1763205058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb578b0 0x55d33bb5a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:32:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde65627700 1 --2- 172.21.15.40:0/1763205058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb5f070 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009990 tx=0x7fde5c005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c901d14c88a40449 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde4bfff700 1 -- 172.21.15.40:0/1763205058 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c02b070 con 0x55d33b9b06b0 2023-07-18T03:32:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde4bfff700 1 -- 172.21.15.40:0/1763205058 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde5c024410 con 0x55d33b9b06b0 2023-07-18T03:32:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.952+0000 7fde4bfff700 1 -- 172.21.15.40:0/1763205058 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c042640 con 0x55d33b9b06b0 2023-07-18T03:32:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.953+0000 7fde73a11b80 1 -- 172.21.15.40:0/1763205058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 msgr2=0x55d33bb5f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.953+0000 7fde73a11b80 1 --2- 172.21.15.40:0/1763205058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb5f070 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009990 tx=0x7fde5c005f00 comp rx=0 tx=0).stop 2023-07-18T03:32:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.953+0000 7fde73a11b80 1 -- 172.21.15.40:0/1763205058 shutdown_connections 2023-07-18T03:32:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.953+0000 7fde73a11b80 1 --2- 172.21.15.40:0/1763205058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb578b0 0x55d33bb5a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.953+0000 7fde73a11b80 1 --2- 172.21.15.40:0/1763205058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb5f070 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.953+0000 7fde73a11b80 1 --2- 172.21.15.40:0/1763205058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb56af0 0x55d33bb72010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.953+0000 7fde73a11b80 1 -- 172.21.15.40:0/1763205058 >> 172.21.15.40:0/1763205058 conn(0x55d33bb49be0 msgr2=0x55d33bb5e730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.953+0000 7fde73a11b80 1 -- 172.21.15.40:0/1763205058 shutdown_connections 2023-07-18T03:32:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.953+0000 7fde73a11b80 1 -- 172.21.15.40:0/1763205058 wait complete. 2023-07-18T03:32:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.953+0000 7fde73a11b80 1 -- start start 2023-07-18T03:32:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.953+0000 7fde73a11b80 1 Processor -- start 2023-07-18T03:32:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde73a11b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde73a11b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb56af0 0x55d33bb53c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde73a11b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb578b0 0x55d33bb54bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde73a11b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33bb55130 con 0x55d33b9b06b0 2023-07-18T03:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde73a11b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33bb552a0 con 0x55d33bb56af0 2023-07-18T03:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde73a11b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d33bb55460 con 0x55d33bb578b0 2023-07-18T03:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde73a11b80 10 client.0 _ll_register_callbacks cb 0x55d33ba43700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde65627700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde65627700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58916/0 (socket says 172.21.15.40:58916) 2023-07-18T03:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde65627700 1 -- 172.21.15.40:0/2460059234 learned_addr learned my addr 172.21.15.40:0/2460059234 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde65e28700 1 --2- 172.21.15.40:0/2460059234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb578b0 0x55d33bb54bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde64e26700 1 --2- 172.21.15.40:0/2460059234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb56af0 0x55d33bb53c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde65627700 1 -- 172.21.15.40:0/2460059234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb578b0 msgr2=0x55d33bb54bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde65627700 1 --2- 172.21.15.40:0/2460059234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb578b0 0x55d33bb54bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde65627700 1 -- 172.21.15.40:0/2460059234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb56af0 msgr2=0x55d33bb53c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde65627700 1 --2- 172.21.15.40:0/2460059234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb56af0 0x55d33bb53c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde65627700 1 -- 172.21.15.40:0/2460059234 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde60107820 con 0x55d33b9b06b0 2023-07-18T03:32:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde64e26700 1 --2- 172.21.15.40:0/2460059234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb56af0 0x55d33bb53c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:32:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde65627700 1 --2- 172.21.15.40:0/2460059234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb536f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fde5c018d00 tx=0x7fde5c018bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1397181c7b03e37c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.954+0000 7fde48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:32:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.955+0000 7fde48ff9700 1 -- 172.21.15.40:0/2460059234 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c02b040 con 0x55d33b9b06b0 2023-07-18T03:32:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.955+0000 7fde48ff9700 1 -- 172.21.15.40:0/2460059234 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fde5c00f900 con 0x55d33b9b06b0 2023-07-18T03:32:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.955+0000 7fde73a11b80 1 -- 172.21.15.40:0/2460059234 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d33ba60ff0 con 0x55d33b9b06b0 2023-07-18T03:32:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.955+0000 7fde48ff9700 1 -- 172.21.15.40:0/2460059234 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde5c00fae0 con 0x55d33b9b06b0 2023-07-18T03:32:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.955+0000 7fde73a11b80 10 client.15852 fetch_fsmap learned FSMap version 28 2023-07-18T03:32:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.955+0000 7fde73a11b80 1 -- 172.21.15.40:0/2460059234 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d33b8fa1f0 con 0x55d33b9b06b0 2023-07-18T03:32:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.955+0000 7fde48ff9700 1 -- 172.21.15.40:0/2460059234 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d33ba60ff0 con 0x55d33b9b06b0 2023-07-18T03:32:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.955+0000 7fde48ff9700 1 -- 172.21.15.40:0/2460059234 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fde5c00bca0 con 0x55d33b9b06b0 2023-07-18T03:32:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.955+0000 7fde73a11b80 10 client.15852 fetch_fsmap finished waiting for FSMap version 28 2023-07-18T03:32:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.955+0000 7fde73a11b80 10 client.15852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:32:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.955+0000 7fde73a11b80 20 client.15852 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.955+0000 7fde73a11b80 1 --2- 172.21.15.40:0/2460059234 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d33bbc17a0 0x55d33bbc3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.955+0000 7fde73a11b80 4 client.15852 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:32:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.955+0000 7fde73a11b80 1 -- 172.21.15.40:0/2460059234 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d33b8f4f10 con 0x55d33bbc17a0 2023-07-18T03:32:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.956+0000 7fde64e26700 1 --2- 172.21.15.40:0/2460059234 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d33bbc17a0 0x55d33bbc3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.956+0000 7fde48ff9700 10 client.15852 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:09.956+0000 7fde64e26700 1 --2- 172.21.15.40:0/2460059234 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d33bbc17a0 0x55d33bbc3bb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e8b8a9abc45cffa server_cookie=5613c44aff2a7d92 in_seq=0 out_seq=0 2023-07-18T03:32:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.045+0000 7fde48ff9700 1 -- 172.21.15.40:0/2460059234 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d33b8f4f10 con 0x55d33bbc17a0 2023-07-18T03:32:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.045+0000 7fde48ff9700 10 client.15852 handle_command_reply: tid=0 2023-07-18T03:32:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.046+0000 7fde73a11b80 1 client.15852 shutdown 2023-07-18T03:32:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.046+0000 7fde73a11b80 1 -- 172.21.15.40:0/2460059234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 msgr2=0x55d33bb536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.046+0000 7fde73a11b80 1 --2- 172.21.15.40:0/2460059234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb536f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fde5c018d00 tx=0x7fde5c018bf0 comp rx=0 tx=0).stop 2023-07-18T03:32:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.046+0000 7fde73a11b80 1 -- 172.21.15.40:0/2460059234 shutdown_connections 2023-07-18T03:32:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.046+0000 7fde73a11b80 1 --2- 172.21.15.40:0/2460059234 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d33bbc17a0 0x55d33bbc3bb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.046+0000 7fde73a11b80 1 --2- 172.21.15.40:0/2460059234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d33bb578b0 0x55d33bb54bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.046+0000 7fde48ff9700 0 client.15852 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.046+0000 7fde73a11b80 1 --2- 172.21.15.40:0/2460059234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d33b9b06b0 0x55d33bb536f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.046+0000 7fde73a11b80 1 --2- 172.21.15.40:0/2460059234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d33bb56af0 0x55d33bb53c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.046+0000 7fde73a11b80 1 -- 172.21.15.40:0/2460059234 >> 172.21.15.40:0/2460059234 conn(0x55d33bb49be0 msgr2=0x55d33bb5a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.046+0000 7fde73a11b80 1 -- 172.21.15.40:0/2460059234 shutdown_connections 2023-07-18T03:32:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.046+0000 7fde73a11b80 1 -- 172.21.15.40:0/2460059234 wait complete. 2023-07-18T03:32:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.046+0000 7fde73a11b80 20 client.15852 trim_cache size 0 max 16384 2023-07-18T03:32:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:32:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:32:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:32:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:32:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.047+0000 7fde6788b700 1 -- 172.21.15.40:0/668310121 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fde4c0a1150 msgr2=0x7fde4c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.047+0000 7fde6788b700 1 --2- 172.21.15.40:0/668310121 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fde4c0a1150 0x7fde4c0a3600 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fde5c000c00 tx=0x7fde5c005fb0 comp rx=0 tx=0).stop 2023-07-18T03:32:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.047+0000 7fde6788b700 1 -- 172.21.15.40:0/668310121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde600ffca0 msgr2=0x7fde60194de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.047+0000 7fde6788b700 1 --2- 172.21.15.40:0/668310121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde600ffca0 0x7fde60194de0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fde5000e9a0 tx=0x7fde5000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:32:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.047+0000 7fde6788b700 1 -- 172.21.15.40:0/668310121 shutdown_connections 2023-07-18T03:32:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.047+0000 7fde6788b700 1 --2- 172.21.15.40:0/668310121 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fde4c0a1150 0x7fde4c0a3600 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.047+0000 7fde6788b700 1 --2- 172.21.15.40:0/668310121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde60100630 0x7fde60195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.047+0000 7fde6788b700 1 --2- 172.21.15.40:0/668310121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde600feab0 0x7fde601948a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.047+0000 7fde6788b700 1 --2- 172.21.15.40:0/668310121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde600ffca0 0x7fde60194de0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.047+0000 7fde6788b700 1 -- 172.21.15.40:0/668310121 >> 172.21.15.40:0/668310121 conn(0x7fde600fa0d0 msgr2=0x7fde600fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.047+0000 7fde6788b700 1 -- 172.21.15.40:0/668310121 shutdown_connections 2023-07-18T03:32:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:10.047+0000 7fde6788b700 1 -- 172.21.15.40:0/668310121 wait complete. 2023-07-18T03:32:10.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:09 smithi134 ceph-mon[98216]: pgmap v331: 97 pgs: 97 active+clean; 9.2 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 830 KiB/s wr, 106 op/s 2023-07-18T03:32:10.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:09 smithi040 ceph-mon[96439]: pgmap v331: 97 pgs: 97 active+clean; 9.2 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 830 KiB/s wr, 106 op/s 2023-07-18T03:32:10.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:09 smithi064 ceph-mon[98059]: pgmap v331: 97 pgs: 97 active+clean; 9.2 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 830 KiB/s wr, 106 op/s 2023-07-18T03:32:12.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:11 smithi134 ceph-mon[98216]: pgmap v332: 97 pgs: 97 active+clean; 9.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 824 KiB/s wr, 97 op/s 2023-07-18T03:32:12.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:11 smithi040 ceph-mon[96439]: pgmap v332: 97 pgs: 97 active+clean; 9.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 824 KiB/s wr, 97 op/s 2023-07-18T03:32:12.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:11 smithi064 ceph-mon[98059]: pgmap v332: 97 pgs: 97 active+clean; 9.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 824 KiB/s wr, 97 op/s 2023-07-18T03:32:13.144 INFO:tasks.workunit.client.0.smithi040.stdout: 8 67273 431 53491 473 9644 513 2023-07-18T03:32:14.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:13 smithi134 ceph-mon[98216]: pgmap v333: 97 pgs: 97 active+clean; 7.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 1009 KiB/s wr, 105 op/s 2023-07-18T03:32:14.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:13 smithi040 ceph-mon[96439]: pgmap v333: 97 pgs: 97 active+clean; 7.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 1009 KiB/s wr, 105 op/s 2023-07-18T03:32:14.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:13 smithi064 ceph-mon[98059]: pgmap v333: 97 pgs: 97 active+clean; 7.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 1009 KiB/s wr, 105 op/s 2023-07-18T03:32:16.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:15 smithi134 ceph-mon[98216]: pgmap v334: 97 pgs: 97 active+clean; 7.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 685 KiB/s wr, 53 op/s 2023-07-18T03:32:16.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:15 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:32:16.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:15 smithi040 ceph-mon[96439]: pgmap v334: 97 pgs: 97 active+clean; 7.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 685 KiB/s wr, 53 op/s 2023-07-18T03:32:16.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:15 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:32:16.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:15 smithi064 ceph-mon[98059]: pgmap v334: 97 pgs: 97 active+clean; 7.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 685 KiB/s wr, 53 op/s 2023-07-18T03:32:16.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:15 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:32:18.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:17 smithi134 ceph-mon[98216]: pgmap v335: 97 pgs: 97 active+clean; 16 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.5 MiB/s wr, 92 op/s 2023-07-18T03:32:18.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:17 smithi040 ceph-mon[96439]: pgmap v335: 97 pgs: 97 active+clean; 16 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.5 MiB/s wr, 92 op/s 2023-07-18T03:32:18.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:17 smithi064 ceph-mon[98059]: pgmap v335: 97 pgs: 97 active+clean; 16 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.5 MiB/s wr, 92 op/s 2023-07-18T03:32:20.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:19 smithi134 ceph-mon[98216]: pgmap v336: 97 pgs: 97 active+clean; 41 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 181 op/s 2023-07-18T03:32:20.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:19 smithi040 ceph-mon[96439]: pgmap v336: 97 pgs: 97 active+clean; 41 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 181 op/s 2023-07-18T03:32:20.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:19 smithi064 ceph-mon[98059]: pgmap v336: 97 pgs: 97 active+clean; 41 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 181 op/s 2023-07-18T03:32:21.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:20 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:32:21.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:20 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:32:21.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:20 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:32:22.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:21 smithi134 ceph-mon[98216]: pgmap v337: 97 pgs: 97 active+clean; 43 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 189 op/s 2023-07-18T03:32:22.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:21 smithi040 ceph-mon[96439]: pgmap v337: 97 pgs: 97 active+clean; 43 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 189 op/s 2023-07-18T03:32:22.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:21 smithi064 ceph-mon[98059]: pgmap v337: 97 pgs: 97 active+clean; 43 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 189 op/s 2023-07-18T03:32:23.144 INFO:tasks.workunit.client.0.smithi040.stdout: 17 287068 510 215494 475 89618 916 2023-07-18T03:32:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.164+0000 7fbc14d9b700 1 -- 172.21.15.40:0/2216374744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc10100640 msgr2=0x7fbc10106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.164+0000 7fbc14d9b700 1 --2- 172.21.15.40:0/2216374744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc10100640 0x7fbc10106ee0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009ec0 tx=0x7fbc0800b330 comp rx=0 tx=0).stop 2023-07-18T03:32:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.164+0000 7fbc14d9b700 1 -- 172.21.15.40:0/2216374744 shutdown_connections 2023-07-18T03:32:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.164+0000 7fbc14d9b700 1 --2- 172.21.15.40:0/2216374744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc10100640 0x7fbc10106ee0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.164+0000 7fbc14d9b700 1 --2- 172.21.15.40:0/2216374744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc100fea00 0x7fbc100fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.164+0000 7fbc14d9b700 1 --2- 172.21.15.40:0/2216374744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc100ffc90 0x7fbc10100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.164+0000 7fbc14d9b700 1 -- 172.21.15.40:0/2216374744 >> 172.21.15.40:0/2216374744 conn(0x7fbc100fa000 msgr2=0x7fbc100fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.164+0000 7fbc14d9b700 1 -- 172.21.15.40:0/2216374744 shutdown_connections 2023-07-18T03:32:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.164+0000 7fbc14d9b700 1 -- 172.21.15.40:0/2216374744 wait complete. 2023-07-18T03:32:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.165+0000 7fbc14d9b700 1 Processor -- start 2023-07-18T03:32:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.165+0000 7fbc14d9b700 1 -- start start 2023-07-18T03:32:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc14d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc100fea00 0x7fbc101984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc14d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc100ffc90 0x7fbc1019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc14d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc10100640 0x7fbc1019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc14d9b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1019d540 con 0x7fbc100fea00 2023-07-18T03:32:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc14d9b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1019d680 con 0x7fbc10100640 2023-07-18T03:32:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc14d9b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1019d840 con 0x7fbc100ffc90 2023-07-18T03:32:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc0e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc100fea00 0x7fbc101984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc0e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc100fea00 0x7fbc101984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58932/0 (socket says 172.21.15.40:58932) 2023-07-18T03:32:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc0e59c700 1 -- 172.21.15.40:0/597512252 learned_addr learned my addr 172.21.15.40:0/597512252 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc07fff700 1 --2- 172.21.15.40:0/597512252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc100ffc90 0x7fbc1019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc0ed9d700 1 --2- 172.21.15.40:0/597512252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc10100640 0x7fbc1019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc0e59c700 1 -- 172.21.15.40:0/597512252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc100ffc90 msgr2=0x7fbc1019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc0e59c700 1 --2- 172.21.15.40:0/597512252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc100ffc90 0x7fbc1019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc0e59c700 1 -- 172.21.15.40:0/597512252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc10100640 msgr2=0x7fbc1019cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc0e59c700 1 --2- 172.21.15.40:0/597512252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc10100640 0x7fbc1019cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc0e59c700 1 -- 172.21.15.40:0/597512252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc08005e70 con 0x7fbc100fea00 2023-07-18T03:32:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc0e59c700 1 --2- 172.21.15.40:0/597512252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc100fea00 0x7fbc101984d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000e9a0 tx=0x7fbc0000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc077fe700 1 -- 172.21.15.40:0/597512252 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc00009790 con 0x7fbc100fea00 2023-07-18T03:32:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc077fe700 1 -- 172.21.15.40:0/597512252 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc00010430 con 0x7fbc100fea00 2023-07-18T03:32:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.166+0000 7fbc14d9b700 1 -- 172.21.15.40:0/597512252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1019db20 con 0x7fbc100fea00 2023-07-18T03:32:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.167+0000 7fbc077fe700 1 -- 172.21.15.40:0/597512252 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc00018680 con 0x7fbc100fea00 2023-07-18T03:32:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.167+0000 7fbc14d9b700 1 -- 172.21.15.40:0/597512252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1019e030 con 0x7fbc100fea00 2023-07-18T03:32:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.169+0000 7fbc077fe700 1 -- 172.21.15.40:0/597512252 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbc000105a0 con 0x7fbc100fea00 2023-07-18T03:32:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.169+0000 7fbc077fe700 1 --2- 172.21.15.40:0/597512252 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbbfc0a1150 0x7fbbfc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.170+0000 7fbc07fff700 1 --2- 172.21.15.40:0/597512252 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbbfc0a1150 0x7fbbfc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.170+0000 7fbc077fe700 1 -- 172.21.15.40:0/597512252 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbc00014070 con 0x7fbc100fea00 2023-07-18T03:32:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.170+0000 7fbc07fff700 1 --2- 172.21.15.40:0/597512252 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbbfc0a1150 0x7fbbfc0a3600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbc10067db0 tx=0x7fbbf8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.170+0000 7fbc20f21b80 1 Processor -- start 2023-07-18T03:32:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc20f21b80 1 -- start start 2023-07-18T03:32:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc20f21b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e732d6b0 0x5627e74d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc20f21b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e74d17b0 0x5627e74decf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc20f21b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e74d1c20 0x5627e74e1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc20f21b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e73d6c50 con 0x5627e732d6b0 2023-07-18T03:32:23.177 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc20f21b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e74299e0 con 0x5627e74d17b0 2023-07-18T03:32:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc20f21b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e7422e70 con 0x5627e74d1c20 2023-07-18T03:32:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc0e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e732d6b0 0x5627e74d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc0e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e732d6b0 0x5627e74d1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58940/0 (socket says 172.21.15.40:58940) 2023-07-18T03:32:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc0e59c700 1 -- 172.21.15.40:0/3860542078 learned_addr learned my addr 172.21.15.40:0/3860542078 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc0ed9d700 1 --2- 172.21.15.40:0/3860542078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e74d1c20 0x5627e74e1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc07fff700 1 --2- 172.21.15.40:0/3860542078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e74d17b0 0x5627e74decf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc0e59c700 1 -- 172.21.15.40:0/3860542078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e74d1c20 msgr2=0x5627e74e1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc0e59c700 1 --2- 172.21.15.40:0/3860542078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e74d1c20 0x5627e74e1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc0e59c700 1 -- 172.21.15.40:0/3860542078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e74d17b0 msgr2=0x5627e74decf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc0e59c700 1 --2- 172.21.15.40:0/3860542078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e74d17b0 0x5627e74decf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc0e59c700 1 -- 172.21.15.40:0/3860542078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627e7250d80 con 0x5627e732d6b0 2023-07-18T03:32:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc07fff700 1 --2- 172.21.15.40:0/3860542078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e74d17b0 0x5627e74decf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:32:23.180 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.171+0000 7fbc0e59c700 1 --2- 172.21.15.40:0/3860542078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e732d6b0 0x5627e74d1270 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbc000963e0 tx=0x7fbc00099910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a340ad374bdda3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.172+0000 7fbc04ff9700 1 -- 172.21.15.40:0/3860542078 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc000a2330 con 0x5627e732d6b0 2023-07-18T03:32:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.172+0000 7fbc04ff9700 1 -- 172.21.15.40:0/3860542078 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc00098280 con 0x5627e732d6b0 2023-07-18T03:32:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.172+0000 7fbc04ff9700 1 -- 172.21.15.40:0/3860542078 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc000a2100 con 0x5627e732d6b0 2023-07-18T03:32:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.172+0000 7fbc20f21b80 1 -- 172.21.15.40:0/3860542078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e732d6b0 msgr2=0x5627e74d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.172+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/3860542078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e732d6b0 0x5627e74d1270 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbc000963e0 tx=0x7fbc00099910 comp rx=0 tx=0).stop 2023-07-18T03:32:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.172+0000 7fbc20f21b80 1 -- 172.21.15.40:0/3860542078 shutdown_connections 2023-07-18T03:32:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.172+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/3860542078 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e74d1c20 0x5627e74e1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.172+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/3860542078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e732d6b0 0x5627e74d1270 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.172+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/3860542078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e74d17b0 0x5627e74decf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.172+0000 7fbc20f21b80 1 -- 172.21.15.40:0/3860542078 >> 172.21.15.40:0/3860542078 conn(0x5627e74c6be0 msgr2=0x5627e74cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.172+0000 7fbc20f21b80 1 -- 172.21.15.40:0/3860542078 shutdown_connections 2023-07-18T03:32:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.173+0000 7fbc20f21b80 1 -- 172.21.15.40:0/3860542078 wait complete. 2023-07-18T03:32:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.173+0000 7fbc20f21b80 1 -- start start 2023-07-18T03:32:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.173+0000 7fbc20f21b80 1 Processor -- start 2023-07-18T03:32:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.173+0000 7fbc20f21b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e732d6b0 0x5627e74e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.173+0000 7fbc20f21b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d17b0 0x5627e74e35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.173+0000 7fbc20f21b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e74d1c20 0x5627e74ea610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.173+0000 7fbc20f21b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e73c2a60 con 0x5627e74d17b0 2023-07-18T03:32:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.173+0000 7fbc20f21b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e73c2760 con 0x5627e74d1c20 2023-07-18T03:32:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.173+0000 7fbc20f21b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e72a68c0 con 0x5627e732d6b0 2023-07-18T03:32:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.173+0000 7fbc20f21b80 10 client.0 _ll_register_callbacks cb 0x5627e73c0700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.173+0000 7fbc07fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d17b0 0x5627e74e35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.173+0000 7fbc0e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e732d6b0 0x5627e74e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.173+0000 7fbc07fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d17b0 0x5627e74e35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58948/0 (socket says 172.21.15.40:58948) 2023-07-18T03:32:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.173+0000 7fbc07fff700 1 -- 172.21.15.40:0/2921851793 learned_addr learned my addr 172.21.15.40:0/2921851793 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.174+0000 7fbc0ed9d700 1 --2- 172.21.15.40:0/2921851793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e74d1c20 0x5627e74ea610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.174+0000 7fbc07fff700 1 -- 172.21.15.40:0/2921851793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e732d6b0 msgr2=0x5627e74e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.174+0000 7fbc07fff700 1 --2- 172.21.15.40:0/2921851793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e732d6b0 0x5627e74e3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.174+0000 7fbc07fff700 1 -- 172.21.15.40:0/2921851793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e74d1c20 msgr2=0x5627e74ea610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.174+0000 7fbc07fff700 1 --2- 172.21.15.40:0/2921851793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e74d1c20 0x5627e74ea610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.174+0000 7fbc07fff700 1 -- 172.21.15.40:0/2921851793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1019e030 con 0x5627e74d17b0 2023-07-18T03:32:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.174+0000 7fbc07fff700 1 --2- 172.21.15.40:0/2921851793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d17b0 0x5627e74e35d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbbf801d850 tx=0x7fbbf80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3fb4af77e0bbf8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.174+0000 7fbbf5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:32:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.174+0000 7fbbf5ffb700 1 -- 172.21.15.40:0/2921851793 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf800e840 con 0x5627e74d17b0 2023-07-18T03:32:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.174+0000 7fbc20f21b80 1 -- 172.21.15.40:0/2921851793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627e737c490 con 0x5627e74d17b0 2023-07-18T03:32:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.174+0000 7fbbf5ffb700 1 -- 172.21.15.40:0/2921851793 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbbf800e9a0 con 0x5627e74d17b0 2023-07-18T03:32:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.174+0000 7fbbf5ffb700 1 -- 172.21.15.40:0/2921851793 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbf800eb60 con 0x5627e74d17b0 2023-07-18T03:32:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.174+0000 7fbc20f21b80 10 client.15864 fetch_fsmap learned FSMap version 30 2023-07-18T03:32:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.174+0000 7fbc20f21b80 1 -- 172.21.15.40:0/2921851793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5627e7239c00 con 0x5627e74d17b0 2023-07-18T03:32:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.174+0000 7fbbf5ffb700 1 -- 172.21.15.40:0/2921851793 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5627e737c490 con 0x5627e74d17b0 2023-07-18T03:32:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.175+0000 7fbbf5ffb700 1 -- 172.21.15.40:0/2921851793 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbbf801b400 con 0x5627e74d17b0 2023-07-18T03:32:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.175+0000 7fbc20f21b80 10 client.15864 fetch_fsmap finished waiting for FSMap version 30 2023-07-18T03:32:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.175+0000 7fbc20f21b80 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:32:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.175+0000 7fbc20f21b80 20 client.15864 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.175+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/2921851793 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5627e74eeee0 0x5627e74f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.175+0000 7fbc20f21b80 4 client.15864 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:32:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.175+0000 7fbc20f21b80 1 -- 172.21.15.40:0/2921851793 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627e741be50 con 0x5627e74eeee0 2023-07-18T03:32:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.175+0000 7fbc0e59c700 1 --2- 172.21.15.40:0/2921851793 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5627e74eeee0 0x5627e74f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.175+0000 7fbbf5ffb700 10 client.15864 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.175+0000 7fbc0e59c700 1 --2- 172.21.15.40:0/2921851793 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5627e74eeee0 0x5627e74f12f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d1ae269aba41d56 server_cookie=cbf5a955aaad14ea in_seq=0 out_seq=0 2023-07-18T03:32:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.176+0000 7fbbf5ffb700 1 -- 172.21.15.40:0/2921851793 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5627e741be50 con 0x5627e74eeee0 2023-07-18T03:32:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.176+0000 7fbbf5ffb700 10 client.15864 handle_command_reply: tid=0 2023-07-18T03:32:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.176+0000 7fbc20f21b80 1 client.15864 shutdown 2023-07-18T03:32:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.176+0000 7fbc20f21b80 1 -- 172.21.15.40:0/2921851793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d17b0 msgr2=0x5627e74e35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.176+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/2921851793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d17b0 0x5627e74e35d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbbf801d850 tx=0x7fbbf80047c0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.176+0000 7fbc20f21b80 1 -- 172.21.15.40:0/2921851793 shutdown_connections 2023-07-18T03:32:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.176+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/2921851793 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5627e74eeee0 0x5627e74f12f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.176+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/2921851793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e74d1c20 0x5627e74ea610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.176+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/2921851793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e732d6b0 0x5627e74e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.176+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/2921851793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d17b0 0x5627e74e35d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.176+0000 7fbc20f21b80 1 -- 172.21.15.40:0/2921851793 >> 172.21.15.40:0/2921851793 conn(0x5627e74c6be0 msgr2=0x5627e74cb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.176+0000 7fbbf5ffb700 0 client.15864 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.176+0000 7fbc20f21b80 1 -- 172.21.15.40:0/2921851793 shutdown_connections 2023-07-18T03:32:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.176+0000 7fbc20f21b80 1 -- 172.21.15.40:0/2921851793 wait complete. 2023-07-18T03:32:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.176+0000 7fbc20f21b80 20 client.15864 trim_cache size 0 max 16384 2023-07-18T03:32:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.191+0000 7fbc20f21b80 1 Processor -- start 2023-07-18T03:32:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.191+0000 7fbc20f21b80 1 -- start start 2023-07-18T03:32:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.191+0000 7fbc20f21b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e732d6b0 0x5627e74f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.191+0000 7fbc20f21b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e74d17b0 0x5627e74ea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.191+0000 7fbc20f21b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d1c20 0x5627e74e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.191+0000 7fbc20f21b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e74e2c40 con 0x5627e74d1c20 2023-07-18T03:32:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.191+0000 7fbc20f21b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e74e1710 con 0x5627e74d17b0 2023-07-18T03:32:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.191+0000 7fbc20f21b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e74d2020 con 0x5627e732d6b0 2023-07-18T03:32:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.191+0000 7fbc0ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d1c20 0x5627e74e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.191+0000 7fbc0e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e732d6b0 0x5627e74f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.191+0000 7fbc0ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d1c20 0x5627e74e3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58958/0 (socket says 172.21.15.40:58958) 2023-07-18T03:32:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.191+0000 7fbc0ed9d700 1 -- 172.21.15.40:0/4079891042 learned_addr learned my addr 172.21.15.40:0/4079891042 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.192+0000 7fbc07fff700 1 --2- 172.21.15.40:0/4079891042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e74d17b0 0x5627e74ea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.192+0000 7fbc0ed9d700 1 -- 172.21.15.40:0/4079891042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e732d6b0 msgr2=0x5627e74f0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.192+0000 7fbc0ed9d700 1 --2- 172.21.15.40:0/4079891042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e732d6b0 0x5627e74f0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.192+0000 7fbc0ed9d700 1 -- 172.21.15.40:0/4079891042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e74d17b0 msgr2=0x5627e74ea5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.192+0000 7fbc0ed9d700 1 --2- 172.21.15.40:0/4079891042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e74d17b0 0x5627e74ea5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.192+0000 7fbc0ed9d700 1 -- 172.21.15.40:0/4079891042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627e72771f0 con 0x5627e74d1c20 2023-07-18T03:32:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.192+0000 7fbc07fff700 1 --2- 172.21.15.40:0/4079891042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e74d17b0 0x5627e74ea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:32:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.192+0000 7fbc0e59c700 1 --2- 172.21.15.40:0/4079891042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e732d6b0 0x5627e74f0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:23.197 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.192+0000 7fbc0ed9d700 1 --2- 172.21.15.40:0/4079891042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d1c20 0x5627e74e3090 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009ec0 tx=0x7fbc0800b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edcdf9b52468cf1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.192+0000 7fbc057fa700 1 -- 172.21.15.40:0/4079891042 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0802a070 con 0x5627e74d1c20 2023-07-18T03:32:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.192+0000 7fbc057fa700 1 -- 172.21.15.40:0/4079891042 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc08007610 con 0x5627e74d1c20 2023-07-18T03:32:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.192+0000 7fbc057fa700 1 -- 172.21.15.40:0/4079891042 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0800e040 con 0x5627e74d1c20 2023-07-18T03:32:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.193+0000 7fbc20f21b80 1 -- 172.21.15.40:0/4079891042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d1c20 msgr2=0x5627e74e3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.193+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/4079891042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d1c20 0x5627e74e3090 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009ec0 tx=0x7fbc0800b680 comp rx=0 tx=0).stop 2023-07-18T03:32:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.193+0000 7fbc20f21b80 1 -- 172.21.15.40:0/4079891042 shutdown_connections 2023-07-18T03:32:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.193+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/4079891042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d1c20 0x5627e74e3090 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.193+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/4079891042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e732d6b0 0x5627e74f0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.193+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/4079891042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e74d17b0 0x5627e74ea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.193+0000 7fbc20f21b80 1 -- 172.21.15.40:0/4079891042 >> 172.21.15.40:0/4079891042 conn(0x5627e74c6be0 msgr2=0x5627e74e0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.193+0000 7fbc20f21b80 1 -- 172.21.15.40:0/4079891042 shutdown_connections 2023-07-18T03:32:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.193+0000 7fbc20f21b80 1 -- 172.21.15.40:0/4079891042 wait complete. 2023-07-18T03:32:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.193+0000 7fbc20f21b80 1 -- start start 2023-07-18T03:32:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.193+0000 7fbc20f21b80 1 Processor -- start 2023-07-18T03:32:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc20f21b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e732d6b0 0x5627e74e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc20f21b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e74d17b0 0x5627e74e8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc20f21b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d1c20 0x5627e74e9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc20f21b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e74e9c60 con 0x5627e74d1c20 2023-07-18T03:32:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc20f21b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e74e9dd0 con 0x5627e732d6b0 2023-07-18T03:32:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc20f21b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e74e9f90 con 0x5627e74d17b0 2023-07-18T03:32:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc20f21b80 10 client.0 _ll_register_callbacks cb 0x7fbbf801d410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc0ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d1c20 0x5627e74e9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc0ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d1c20 0x5627e74e9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58964/0 (socket says 172.21.15.40:58964) 2023-07-18T03:32:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc0ed9d700 1 -- 172.21.15.40:0/4245489294 learned_addr learned my addr 172.21.15.40:0/4245489294 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc07fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e74d17b0 0x5627e74e8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc0e59c700 1 --2- 172.21.15.40:0/4245489294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e732d6b0 0x5627e74e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc0ed9d700 1 -- 172.21.15.40:0/4245489294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e74d17b0 msgr2=0x5627e74e8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc0ed9d700 1 --2- 172.21.15.40:0/4245489294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e74d17b0 0x5627e74e8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc0ed9d700 1 -- 172.21.15.40:0/4245489294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e732d6b0 msgr2=0x5627e74e81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc0ed9d700 1 --2- 172.21.15.40:0/4245489294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e732d6b0 0x5627e74e81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc0ed9d700 1 -- 172.21.15.40:0/4245489294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627e7250d80 con 0x5627e74d1c20 2023-07-18T03:32:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc0ed9d700 1 --2- 172.21.15.40:0/4245489294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d1c20 0x5627e74e9720 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009690 tx=0x7fbc0800bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96f6b4dfe20a989 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbbf5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:32:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbbf5ffb700 1 -- 172.21.15.40:0/4245489294 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0802a040 con 0x5627e74d1c20 2023-07-18T03:32:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbbf5ffb700 1 -- 172.21.15.40:0/4245489294 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fbc08020040 con 0x5627e74d1c20 2023-07-18T03:32:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.194+0000 7fbc20f21b80 1 -- 172.21.15.40:0/4245489294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc00098280 con 0x5627e74d1c20 2023-07-18T03:32:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.195+0000 7fbbf5ffb700 1 -- 172.21.15.40:0/4245489294 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc0800e040 con 0x5627e74d1c20 2023-07-18T03:32:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.195+0000 7fbc20f21b80 10 client.15870 fetch_fsmap learned FSMap version 30 2023-07-18T03:32:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.195+0000 7fbc20f21b80 1 -- 172.21.15.40:0/4245489294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fbc10107750 con 0x5627e74d1c20 2023-07-18T03:32:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.195+0000 7fbbf5ffb700 1 -- 172.21.15.40:0/4245489294 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc00098280 con 0x5627e74d1c20 2023-07-18T03:32:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.195+0000 7fbbf5ffb700 1 -- 172.21.15.40:0/4245489294 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbc08024050 con 0x5627e74d1c20 2023-07-18T03:32:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.195+0000 7fbc20f21b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 30 2023-07-18T03:32:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.195+0000 7fbc20f21b80 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:32:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.195+0000 7fbc20f21b80 20 client.15870 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.195+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/4245489294 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5627e7543830 0x5627e7545c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.195+0000 7fbc20f21b80 4 client.15870 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:32:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.195+0000 7fbc20f21b80 1 -- 172.21.15.40:0/4245489294 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5627e73ddff0 con 0x5627e7543830 2023-07-18T03:32:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.195+0000 7fbc0e59c700 1 --2- 172.21.15.40:0/4245489294 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5627e7543830 0x5627e7545c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.196+0000 7fbc0e59c700 1 --2- 172.21.15.40:0/4245489294 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5627e7543830 0x5627e7545c40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc96fddc678fd4b server_cookie=4fcbebde6aa66244 in_seq=0 out_seq=0 2023-07-18T03:32:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.196+0000 7fbbf5ffb700 10 client.15870 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.293+0000 7fbbf5ffb700 1 -- 172.21.15.40:0/4245489294 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5627e73ddff0 con 0x5627e7543830 2023-07-18T03:32:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.293+0000 7fbbf5ffb700 10 client.15870 handle_command_reply: tid=0 2023-07-18T03:32:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.293+0000 7fbc20f21b80 1 client.15870 shutdown 2023-07-18T03:32:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.293+0000 7fbc20f21b80 1 -- 172.21.15.40:0/4245489294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d1c20 msgr2=0x5627e74e9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.293+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/4245489294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d1c20 0x5627e74e9720 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009690 tx=0x7fbc0800bc00 comp rx=0 tx=0).stop 2023-07-18T03:32:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.293+0000 7fbc20f21b80 1 -- 172.21.15.40:0/4245489294 shutdown_connections 2023-07-18T03:32:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.293+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/4245489294 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5627e7543830 0x5627e7545c40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.293+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/4245489294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5627e74d1c20 0x5627e74e9720 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.293+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/4245489294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5627e732d6b0 0x5627e74e81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.293+0000 7fbc20f21b80 1 --2- 172.21.15.40:0/4245489294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5627e74d17b0 0x5627e74e8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.293+0000 7fbbf5ffb700 0 client.15870 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.293+0000 7fbc20f21b80 1 -- 172.21.15.40:0/4245489294 >> 172.21.15.40:0/4245489294 conn(0x5627e74c6be0 msgr2=0x5627e74f0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.293+0000 7fbc20f21b80 1 -- 172.21.15.40:0/4245489294 shutdown_connections 2023-07-18T03:32:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.293+0000 7fbc20f21b80 1 -- 172.21.15.40:0/4245489294 wait complete. 2023-07-18T03:32:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.293+0000 7fbc20f21b80 20 client.15870 trim_cache size 0 max 16384 2023-07-18T03:32:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:32:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:32:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:32:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:32:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.294+0000 7fbc14d9b700 1 -- 172.21.15.40:0/597512252 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbbfc0a1150 msgr2=0x7fbbfc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.294+0000 7fbc14d9b700 1 --2- 172.21.15.40:0/597512252 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbbfc0a1150 0x7fbbfc0a3600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbc10067db0 tx=0x7fbbf8009260 comp rx=0 tx=0).stop 2023-07-18T03:32:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.294+0000 7fbc14d9b700 1 -- 172.21.15.40:0/597512252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc100fea00 msgr2=0x7fbc101984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.294+0000 7fbc14d9b700 1 --2- 172.21.15.40:0/597512252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc100fea00 0x7fbc101984d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000e9a0 tx=0x7fbc0000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.294+0000 7fbc14d9b700 1 -- 172.21.15.40:0/597512252 shutdown_connections 2023-07-18T03:32:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.294+0000 7fbc14d9b700 1 --2- 172.21.15.40:0/597512252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc10100640 0x7fbc1019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.294+0000 7fbc14d9b700 1 --2- 172.21.15.40:0/597512252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc100fea00 0x7fbc101984d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.294+0000 7fbc14d9b700 1 --2- 172.21.15.40:0/597512252 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fbbfc0a1150 0x7fbbfc0a3600 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.294+0000 7fbc14d9b700 1 --2- 172.21.15.40:0/597512252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc100ffc90 0x7fbc1019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.294+0000 7fbc14d9b700 1 -- 172.21.15.40:0/597512252 >> 172.21.15.40:0/597512252 conn(0x7fbc100fa000 msgr2=0x7fbc100fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.294+0000 7fbc14d9b700 1 -- 172.21.15.40:0/597512252 shutdown_connections 2023-07-18T03:32:23.296 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:23.294+0000 7fbc14d9b700 1 -- 172.21.15.40:0/597512252 wait complete. 2023-07-18T03:32:24.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:23 smithi134 ceph-mon[98216]: pgmap v338: 97 pgs: 97 active+clean; 47 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 210 op/s 2023-07-18T03:32:24.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:23 smithi040 ceph-mon[96439]: pgmap v338: 97 pgs: 97 active+clean; 47 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 210 op/s 2023-07-18T03:32:24.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:23 smithi064 ceph-mon[98059]: pgmap v338: 97 pgs: 97 active+clean; 47 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 210 op/s 2023-07-18T03:32:25.748 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:25 smithi040 ceph-mon[96439]: pgmap v339: 97 pgs: 97 active+clean; 47 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 201 op/s 2023-07-18T03:32:25.749 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:25 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:32:26.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:25 smithi134 ceph-mon[98216]: pgmap v339: 97 pgs: 97 active+clean; 47 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 201 op/s 2023-07-18T03:32:26.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:25 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:32:26.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:25 smithi064 ceph-mon[98059]: pgmap v339: 97 pgs: 97 active+clean; 47 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 201 op/s 2023-07-18T03:32:26.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:25 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:32:27.038 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:32:27.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.194+0000 7fc3adc98700 1 -- 172.21.15.40:0/1043757623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a80fea10 msgr2=0x7fc3a80fee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.194+0000 7fc3adc98700 1 --2- 172.21.15.40:0/1043757623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a80fea10 0x7fc3a80fee20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc390009990 tx=0x7fc390009ca0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.194+0000 7fc3adc98700 1 -- 172.21.15.40:0/1043757623 shutdown_connections 2023-07-18T03:32:27.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.194+0000 7fc3adc98700 1 --2- 172.21.15.40:0/1043757623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3a8100590 0x7fc3a8106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.194+0000 7fc3adc98700 1 --2- 172.21.15.40:0/1043757623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a80fea10 0x7fc3a80fee20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.194+0000 7fc3adc98700 1 --2- 172.21.15.40:0/1043757623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3a80ffc00 0x7fc3a8100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.194+0000 7fc3adc98700 1 -- 172.21.15.40:0/1043757623 >> 172.21.15.40:0/1043757623 conn(0x7fc3a80fa030 msgr2=0x7fc3a80fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:27.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.194+0000 7fc3adc98700 1 -- 172.21.15.40:0/1043757623 shutdown_connections 2023-07-18T03:32:27.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.194+0000 7fc3adc98700 1 -- 172.21.15.40:0/1043757623 wait complete. 2023-07-18T03:32:27.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.195+0000 7fc3adc98700 1 Processor -- start 2023-07-18T03:32:27.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.195+0000 7fc3adc98700 1 -- start start 2023-07-18T03:32:27.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.195+0000 7fc3adc98700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3a80fea10 0x7fc3a80691b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.195+0000 7fc3adc98700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3a80ffc00 0x7fc3a819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.195+0000 7fc3adc98700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a8100590 0x7fc3a819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.195+0000 7fc3adc98700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a819d370 con 0x7fc3a8100590 2023-07-18T03:32:27.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.195+0000 7fc3adc98700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a819d4e0 con 0x7fc3a80fea10 2023-07-18T03:32:27.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.195+0000 7fc3adc98700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a819d6a0 con 0x7fc3a80ffc00 2023-07-18T03:32:27.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.195+0000 7fc3a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a8100590 0x7fc3a819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.195+0000 7fc3a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a8100590 0x7fc3a819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58980/0 (socket says 172.21.15.40:58980) 2023-07-18T03:32:27.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.195+0000 7fc3a7fff700 1 -- 172.21.15.40:0/3144740274 learned_addr learned my addr 172.21.15.40:0/3144740274 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:27.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.195+0000 7fc3a6ffd700 1 --2- 172.21.15.40:0/3144740274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3a80ffc00 0x7fc3a819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.195+0000 7fc3a77fe700 1 --2- 172.21.15.40:0/3144740274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3a80fea10 0x7fc3a80691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.195+0000 7fc3a7fff700 1 -- 172.21.15.40:0/3144740274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3a80ffc00 msgr2=0x7fc3a819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.195+0000 7fc3a7fff700 1 --2- 172.21.15.40:0/3144740274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3a80ffc00 0x7fc3a819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.195+0000 7fc3a7fff700 1 -- 172.21.15.40:0/3144740274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3a80fea10 msgr2=0x7fc3a80691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.196+0000 7fc3a7fff700 1 --2- 172.21.15.40:0/3144740274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3a80fea10 0x7fc3a80691b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.196+0000 7fc3a7fff700 1 -- 172.21.15.40:0/3144740274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3900095f0 con 0x7fc3a8100590 2023-07-18T03:32:27.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.196+0000 7fc3a7fff700 1 --2- 172.21.15.40:0/3144740274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a8100590 0x7fc3a819cbd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0007ee0 tx=0x7fc3a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:27.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.196+0000 7fc3a4ff9700 1 -- 172.21.15.40:0/3144740274 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0017070 con 0x7fc3a8100590 2023-07-18T03:32:27.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.196+0000 7fc3adc98700 1 -- 172.21.15.40:0/3144740274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a81943c0 con 0x7fc3a8100590 2023-07-18T03:32:27.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.196+0000 7fc3a4ff9700 1 -- 172.21.15.40:0/3144740274 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3a000f040 con 0x7fc3a8100590 2023-07-18T03:32:27.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.196+0000 7fc3a4ff9700 1 -- 172.21.15.40:0/3144740274 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0005550 con 0x7fc3a8100590 2023-07-18T03:32:27.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.196+0000 7fc3adc98700 1 -- 172.21.15.40:0/3144740274 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a81948d0 con 0x7fc3a8100590 2023-07-18T03:32:27.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.198+0000 7fc3a4ff9700 1 -- 172.21.15.40:0/3144740274 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc3a000db20 con 0x7fc3a8100590 2023-07-18T03:32:27.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.199+0000 7fc3a4ff9700 1 --2- 172.21.15.40:0/3144740274 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3940a1150 0x7fc3940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.199+0000 7fc3a77fe700 1 --2- 172.21.15.40:0/3144740274 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3940a1150 0x7fc3940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.199+0000 7fc3a4ff9700 1 -- 172.21.15.40:0/3144740274 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc3a00cc010 con 0x7fc3a8100590 2023-07-18T03:32:27.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3a77fe700 1 --2- 172.21.15.40:0/3144740274 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3940a1150 0x7fc3940a3600 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc390006010 tx=0x7fc390005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:27.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3b9e1eb80 1 Processor -- start 2023-07-18T03:32:27.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3b9e1eb80 1 -- start start 2023-07-18T03:32:27.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3b9e1eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe24d0 0x557693fe28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3b9e1eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe2e20 0x557693fe36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3b9e1eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe3c70 0x557693fe40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3b9e1eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557693ee7cb0 con 0x557693fe2e20 2023-07-18T03:32:27.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3b9e1eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557693f3aa40 con 0x557693fe3c70 2023-07-18T03:32:27.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3b9e1eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557693f33ed0 con 0x557693fe24d0 2023-07-18T03:32:27.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3a77fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe24d0 0x557693fe28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3a6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe2e20 0x557693fe36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3a77fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe24d0 0x557693fe28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48012/0 (socket says 172.21.15.40:48012) 2023-07-18T03:32:27.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3a6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe2e20 0x557693fe36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58986/0 (socket says 172.21.15.40:58986) 2023-07-18T03:32:27.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3a77fe700 1 -- 172.21.15.40:0/1345910271 learned_addr learned my addr 172.21.15.40:0/1345910271 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:27.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3a7fff700 1 --2- 172.21.15.40:0/1345910271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe3c70 0x557693fe40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3a6ffd700 1 -- 172.21.15.40:0/1345910271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe24d0 msgr2=0x557693fe28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:27.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3a6ffd700 1 --2- 172.21.15.40:0/1345910271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe24d0 0x557693fe28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3a6ffd700 1 -- 172.21.15.40:0/1345910271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe3c70 msgr2=0x557693fe40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:27.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.200+0000 7fc3a6ffd700 1 --2- 172.21.15.40:0/1345910271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe3c70 0x557693fe40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.201+0000 7fc3a6ffd700 1 -- 172.21.15.40:0/1345910271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557693d61de0 con 0x557693fe2e20 2023-07-18T03:32:27.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.201+0000 7fc3a7fff700 1 --2- 172.21.15.40:0/1345910271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe3c70 0x557693fe40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:27.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.201+0000 7fc3a6ffd700 1 --2- 172.21.15.40:0/1345910271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe2e20 0x557693fe36e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc398009cb0 tx=0x7fc398011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b255c4c97412f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:27.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.201+0000 7fc39dffb700 1 -- 172.21.15.40:0/1345910271 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398011d20 con 0x557693fe2e20 2023-07-18T03:32:27.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.201+0000 7fc39dffb700 1 -- 172.21.15.40:0/1345910271 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc398011e80 con 0x557693fe2e20 2023-07-18T03:32:27.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.201+0000 7fc39dffb700 1 -- 172.21.15.40:0/1345910271 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398005280 con 0x557693fe2e20 2023-07-18T03:32:27.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.202+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/1345910271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe2e20 msgr2=0x557693fe36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.202+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/1345910271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe2e20 0x557693fe36e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc398009cb0 tx=0x7fc398011440 comp rx=0 tx=0).stop 2023-07-18T03:32:27.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.202+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/1345910271 shutdown_connections 2023-07-18T03:32:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.202+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/1345910271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe3c70 0x557693fe40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.202+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/1345910271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe24d0 0x557693fe28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.202+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/1345910271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe2e20 0x557693fe36e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.202+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/1345910271 >> 172.21.15.40:0/1345910271 conn(0x557693fd7da0 msgr2=0x557693fdd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.202+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/1345910271 shutdown_connections 2023-07-18T03:32:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.202+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/1345910271 wait complete. 2023-07-18T03:32:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.202+0000 7fc3b9e1eb80 1 -- start start 2023-07-18T03:32:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.202+0000 7fc3b9e1eb80 1 Processor -- start 2023-07-18T03:32:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3b9e1eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe24d0 0x557693fec0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3b9e1eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe2e20 0x557693fec620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3b9e1eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe3c70 0x557693ff3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3b9e1eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557693ed3ac0 con 0x557693fe2e20 2023-07-18T03:32:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3b9e1eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557693ed37c0 con 0x557693fe24d0 2023-07-18T03:32:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3b9e1eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557693db7920 con 0x557693fe3c70 2023-07-18T03:32:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3b9e1eb80 10 client.0 _ll_register_callbacks cb 0x557693ed1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3a6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe2e20 0x557693fec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3a6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe2e20 0x557693fec620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58992/0 (socket says 172.21.15.40:58992) 2023-07-18T03:32:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3a6ffd700 1 -- 172.21.15.40:0/3645876459 learned_addr learned my addr 172.21.15.40:0/3645876459 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3a7fff700 1 --2- 172.21.15.40:0/3645876459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe3c70 0x557693ff3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3a77fe700 1 --2- 172.21.15.40:0/3645876459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe24d0 0x557693fec0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3a6ffd700 1 -- 172.21.15.40:0/3645876459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe3c70 msgr2=0x557693ff3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3a6ffd700 1 --2- 172.21.15.40:0/3645876459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe3c70 0x557693ff3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3a6ffd700 1 -- 172.21.15.40:0/3645876459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe24d0 msgr2=0x557693fec0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3a6ffd700 1 --2- 172.21.15.40:0/3645876459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe24d0 0x557693fec0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3a6ffd700 1 -- 172.21.15.40:0/3645876459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a81948d0 con 0x557693fe2e20 2023-07-18T03:32:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3a6ffd700 1 --2- 172.21.15.40:0/3645876459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe2e20 0x557693fec620 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc398005c50 tx=0x7fc398011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c0306638548332 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc386ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:32:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc386ffd700 1 -- 172.21.15.40:0/3645876459 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398004260 con 0x557693fe2e20 2023-07-18T03:32:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc386ffd700 1 -- 172.21.15.40:0/3645876459 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc3980043c0 con 0x557693fe2e20 2023-07-18T03:32:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.203+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/3645876459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557693f2ceb0 con 0x557693fe2e20 2023-07-18T03:32:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.204+0000 7fc386ffd700 1 -- 172.21.15.40:0/3645876459 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc398004580 con 0x557693fe2e20 2023-07-18T03:32:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.204+0000 7fc3b9e1eb80 10 client.15882 fetch_fsmap learned FSMap version 31 2023-07-18T03:32:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.204+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/3645876459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fc3980098e0 con 0x557693fe2e20 2023-07-18T03:32:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.204+0000 7fc386ffd700 1 -- 172.21.15.40:0/3645876459 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc398031020 con 0x557693fe2e20 2023-07-18T03:32:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.204+0000 7fc386ffd700 1 -- 172.21.15.40:0/3645876459 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc398003b70 con 0x557693fe2e20 2023-07-18T03:32:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.204+0000 7fc3b9e1eb80 10 client.15882 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:32:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.204+0000 7fc3b9e1eb80 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:32:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.204+0000 7fc3b9e1eb80 20 client.15882 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.204+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/3645876459 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557693ff7d70 0x557693ffa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.204+0000 7fc3b9e1eb80 4 client.15882 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:32:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.204+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/3645876459 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557693eef050 con 0x557693ff7d70 2023-07-18T03:32:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.204+0000 7fc3a77fe700 1 --2- 172.21.15.40:0/3645876459 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557693ff7d70 0x557693ffa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.205+0000 7fc386ffd700 10 client.15882 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.205+0000 7fc3a77fe700 1 --2- 172.21.15.40:0/3645876459 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557693ff7d70 0x557693ffa180 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86c7e541401add95 server_cookie=701cb97e9b3b154f in_seq=0 out_seq=0 2023-07-18T03:32:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.205+0000 7fc386ffd700 1 -- 172.21.15.40:0/3645876459 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557693eef050 con 0x557693ff7d70 2023-07-18T03:32:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.205+0000 7fc386ffd700 10 client.15882 handle_command_reply: tid=0 2023-07-18T03:32:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.205+0000 7fc3b9e1eb80 1 client.15882 shutdown 2023-07-18T03:32:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.205+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/3645876459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe2e20 msgr2=0x557693fec620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.205+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/3645876459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe2e20 0x557693fec620 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc398005c50 tx=0x7fc398011ba0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.205+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/3645876459 shutdown_connections 2023-07-18T03:32:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.205+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/3645876459 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557693ff7d70 0x557693ffa180 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.206+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/3645876459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe3c70 0x557693ff3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.206+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/3645876459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe24d0 0x557693fec0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.206+0000 7fc386ffd700 0 client.15882 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.206+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/3645876459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe2e20 0x557693fec620 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.206+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/3645876459 >> 172.21.15.40:0/3645876459 conn(0x557693fd7da0 msgr2=0x557693fdc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.206+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/3645876459 shutdown_connections 2023-07-18T03:32:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.206+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/3645876459 wait complete. 2023-07-18T03:32:27.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.206+0000 7fc3b9e1eb80 20 client.15882 trim_cache size 0 max 16384 2023-07-18T03:32:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3b9e1eb80 1 Processor -- start 2023-07-18T03:32:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3b9e1eb80 1 -- start start 2023-07-18T03:32:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3b9e1eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe24d0 0x557693fe6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3b9e1eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe2e20 0x557693ff3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3b9e1eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe3c70 0x557693fe70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3b9e1eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557693fd4f60 con 0x557693fe3c70 2023-07-18T03:32:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3b9e1eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557693fecd60 con 0x557693fe24d0 2023-07-18T03:32:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3b9e1eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557693fe9e30 con 0x557693fe2e20 2023-07-18T03:32:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe3c70 0x557693fe70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe3c70 0x557693fe70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59002/0 (socket says 172.21.15.40:59002) 2023-07-18T03:32:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3a7fff700 1 -- 172.21.15.40:0/2473690747 learned_addr learned my addr 172.21.15.40:0/2473690747 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3a6ffd700 1 --2- 172.21.15.40:0/2473690747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe2e20 0x557693ff3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3a77fe700 1 --2- 172.21.15.40:0/2473690747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe24d0 0x557693fe6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3a7fff700 1 -- 172.21.15.40:0/2473690747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe2e20 msgr2=0x557693ff3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:27.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3a7fff700 1 --2- 172.21.15.40:0/2473690747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe2e20 0x557693ff3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3a7fff700 1 -- 172.21.15.40:0/2473690747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe24d0 msgr2=0x557693fe6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:27.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3a7fff700 1 --2- 172.21.15.40:0/2473690747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe24d0 0x557693fe6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3a7fff700 1 -- 172.21.15.40:0/2473690747 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557693d4ac60 con 0x557693fe3c70 2023-07-18T03:32:27.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.223+0000 7fc3a7fff700 1 --2- 172.21.15.40:0/2473690747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe3c70 0x557693fe70f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0095e70 tx=0x7fc3a0084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee6e07e53a07113 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:27.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.224+0000 7fc39e7fc700 1 -- 172.21.15.40:0/2473690747 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a00a1850 con 0x557693fe3c70 2023-07-18T03:32:27.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.224+0000 7fc39e7fc700 1 -- 172.21.15.40:0/2473690747 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3a000ddd0 con 0x557693fe3c70 2023-07-18T03:32:27.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.224+0000 7fc39e7fc700 1 -- 172.21.15.40:0/2473690747 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0004d60 con 0x557693fe3c70 2023-07-18T03:32:27.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.224+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/2473690747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe3c70 msgr2=0x557693fe70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.224+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/2473690747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe3c70 0x557693fe70f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0095e70 tx=0x7fc3a0084040 comp rx=0 tx=0).stop 2023-07-18T03:32:27.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.224+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/2473690747 shutdown_connections 2023-07-18T03:32:27.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.224+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/2473690747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe3c70 0x557693fe70f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.224+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/2473690747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe24d0 0x557693fe6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.224+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/2473690747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe2e20 0x557693ff3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.224+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/2473690747 >> 172.21.15.40:0/2473690747 conn(0x557693fd7da0 msgr2=0x557693feae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:27.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.224+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/2473690747 shutdown_connections 2023-07-18T03:32:27.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.224+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/2473690747 wait complete. 2023-07-18T03:32:27.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.225+0000 7fc3b9e1eb80 1 -- start start 2023-07-18T03:32:27.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.225+0000 7fc3b9e1eb80 1 Processor -- start 2023-07-18T03:32:27.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.225+0000 7fc3b9e1eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe24d0 0x557693ff2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.225+0000 7fc3b9e1eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe2e20 0x557693ff3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.225+0000 7fc3b9e1eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe3c70 0x557693fef9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.225+0000 7fc3b9e1eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557693ff00b0 con 0x557693fe3c70 2023-07-18T03:32:27.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.225+0000 7fc3b9e1eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557693ff0220 con 0x557693fe24d0 2023-07-18T03:32:27.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.225+0000 7fc3b9e1eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557693ff03e0 con 0x557693fe2e20 2023-07-18T03:32:27.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.225+0000 7fc3b9e1eb80 10 client.0 _ll_register_callbacks cb 0x7fc3a0004b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:27.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.225+0000 7fc3a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe3c70 0x557693fef9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.225+0000 7fc3a6ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe2e20 0x557693ff3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.225+0000 7fc3a77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe24d0 0x557693ff2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.225+0000 7fc3a6ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe2e20 0x557693ff3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48032/0 (socket says 172.21.15.40:48032) 2023-07-18T03:32:27.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.225+0000 7fc3a77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe24d0 0x557693ff2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42330/0 (socket says 172.21.15.40:42330) 2023-07-18T03:32:27.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.225+0000 7fc3a6ffd700 1 -- 172.21.15.40:0/1355043849 learned_addr learned my addr 172.21.15.40:0/1355043849 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:27.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.225+0000 7fc3a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe3c70 0x557693fef9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59008/0 (socket says 172.21.15.40:59008) 2023-07-18T03:32:27.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.226+0000 7fc3a6ffd700 1 -- 172.21.15.40:0/1355043849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe24d0 msgr2=0x557693ff2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:27.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.226+0000 7fc3a6ffd700 1 --2- 172.21.15.40:0/1355043849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe24d0 0x557693ff2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.226+0000 7fc3a6ffd700 1 -- 172.21.15.40:0/1355043849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe3c70 msgr2=0x557693fef9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:27.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.226+0000 7fc3a6ffd700 1 --2- 172.21.15.40:0/1355043849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe3c70 0x557693fef9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.226+0000 7fc3a6ffd700 1 -- 172.21.15.40:0/1355043849 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a0021070 con 0x557693fe2e20 2023-07-18T03:32:27.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.226+0000 7fc3a77fe700 1 --2- 172.21.15.40:0/1355043849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe24d0 0x557693ff2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:27.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.226+0000 7fc3a6ffd700 1 --2- 172.21.15.40:0/1355043849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe2e20 0x557693ff3470 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc39802b850 tx=0x7fc398004490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4681e081d76e78da server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:27.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.226+0000 7fc386ffd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:32:27.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.226+0000 7fc386ffd700 1 -- 172.21.15.40:0/1355043849 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3980092d0 con 0x557693fe2e20 2023-07-18T03:32:27.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.226+0000 7fc386ffd700 1 -- 172.21.15.40:0/1355043849 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc398032080 con 0x557693fe2e20 2023-07-18T03:32:27.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.226+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/1355043849 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557693e8d4f0 con 0x557693fe2e20 2023-07-18T03:32:27.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.226+0000 7fc386ffd700 1 -- 172.21.15.40:0/1355043849 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc398011cc0 con 0x557693fe2e20 2023-07-18T03:32:27.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.226+0000 7fc3b9e1eb80 10 client.25540 fetch_fsmap learned FSMap version 31 2023-07-18T03:32:27.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.226+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/1355043849 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557693d88250 con 0x557693fe2e20 2023-07-18T03:32:27.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.226+0000 7fc386ffd700 1 -- 172.21.15.40:0/1355043849 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557693e8d4f0 con 0x557693fe2e20 2023-07-18T03:32:27.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.227+0000 7fc386ffd700 1 -- 172.21.15.40:0/1355043849 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc398037470 con 0x557693fe2e20 2023-07-18T03:32:27.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.227+0000 7fc3b9e1eb80 10 client.25540 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:32:27.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.227+0000 7fc3b9e1eb80 10 client.25540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:32:27.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.227+0000 7fc3b9e1eb80 20 client.25540 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:27.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.227+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/1355043849 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55769404ced0 0x55769404f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.227+0000 7fc3b9e1eb80 4 client.25540 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:32:27.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.227+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/1355043849 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557693d82f70 con 0x55769404ced0 2023-07-18T03:32:27.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.227+0000 7fc3a77fe700 1 --2- 172.21.15.40:0/1355043849 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55769404ced0 0x55769404f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.227+0000 7fc3a77fe700 1 --2- 172.21.15.40:0/1355043849 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55769404ced0 0x55769404f2e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bae41d73c105ac6d server_cookie=789d38c551813d8e in_seq=0 out_seq=0 2023-07-18T03:32:27.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.227+0000 7fc386ffd700 10 client.25540 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:27.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.227+0000 7fc386ffd700 1 -- 172.21.15.40:0/1355043849 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557693d82f70 con 0x55769404ced0 2023-07-18T03:32:27.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.227+0000 7fc386ffd700 10 client.25540 handle_command_reply: tid=0 2023-07-18T03:32:27.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.227+0000 7fc3b9e1eb80 1 client.25540 shutdown 2023-07-18T03:32:27.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/1355043849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe2e20 msgr2=0x557693ff3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/1355043849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe2e20 0x557693ff3470 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc39802b850 tx=0x7fc398004490 comp rx=0 tx=0).stop 2023-07-18T03:32:27.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/1355043849 shutdown_connections 2023-07-18T03:32:27.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/1355043849 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55769404ced0 0x55769404f2e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/1355043849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557693fe3c70 0x557693fef9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc386ffd700 0 client.25540 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:27.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/1355043849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557693fe24d0 0x557693ff2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3b9e1eb80 1 --2- 172.21.15.40:0/1355043849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557693fe2e20 0x557693ff3470 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/1355043849 >> 172.21.15.40:0/1355043849 conn(0x557693fd7da0 msgr2=0x557693fe6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:27.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/1355043849 shutdown_connections 2023-07-18T03:32:27.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3b9e1eb80 1 -- 172.21.15.40:0/1355043849 wait complete. 2023-07-18T03:32:27.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3b9e1eb80 20 client.25540 trim_cache size 0 max 16384 2023-07-18T03:32:27.235 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:32:27.235 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "no active scrubs running", 2023-07-18T03:32:27.235 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": {} 2023-07-18T03:32:27.235 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:32:27.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3adc98700 1 -- 172.21.15.40:0/3144740274 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3940a1150 msgr2=0x7fc3940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3adc98700 1 --2- 172.21.15.40:0/3144740274 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3940a1150 0x7fc3940a3600 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc390006010 tx=0x7fc390005960 comp rx=0 tx=0).stop 2023-07-18T03:32:27.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3adc98700 1 -- 172.21.15.40:0/3144740274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a8100590 msgr2=0x7fc3a819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3adc98700 1 --2- 172.21.15.40:0/3144740274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a8100590 0x7fc3a819cbd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0007ee0 tx=0x7fc3a000d3b0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3adc98700 1 -- 172.21.15.40:0/3144740274 shutdown_connections 2023-07-18T03:32:27.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3adc98700 1 --2- 172.21.15.40:0/3144740274 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3a8100590 0x7fc3a819cbd0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3adc98700 1 --2- 172.21.15.40:0/3144740274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3a80fea10 0x7fc3a80691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3adc98700 1 --2- 172.21.15.40:0/3144740274 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc3940a1150 0x7fc3940a3600 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3adc98700 1 --2- 172.21.15.40:0/3144740274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3a80ffc00 0x7fc3a819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3adc98700 1 -- 172.21.15.40:0/3144740274 >> 172.21.15.40:0/3144740274 conn(0x7fc3a80fa030 msgr2=0x7fc3a80fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:27.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3adc98700 1 -- 172.21.15.40:0/3144740274 shutdown_connections 2023-07-18T03:32:27.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.228+0000 7fc3adc98700 1 -- 172.21.15.40:0/3144740274 wait complete. 2023-07-18T03:32:27.239 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:32:27.239 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:32:27.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.372+0000 7fe293783700 1 -- 172.21.15.40:0/3905803959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe28c0ffae0 msgr2=0x7fe28c0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.372+0000 7fe293783700 1 --2- 172.21.15.40:0/3905803959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe28c0ffae0 0x7fe28c0ffef0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe288009990 tx=0x7fe288009ca0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.372+0000 7fe293783700 1 -- 172.21.15.40:0/3905803959 shutdown_connections 2023-07-18T03:32:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.372+0000 7fe293783700 1 --2- 172.21.15.40:0/3905803959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe28c101660 0x7fe28c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.372+0000 7fe293783700 1 --2- 172.21.15.40:0/3905803959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe28c0ffae0 0x7fe28c0ffef0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.372+0000 7fe293783700 1 --2- 172.21.15.40:0/3905803959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe28c100cd0 0x7fe28c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.372+0000 7fe293783700 1 -- 172.21.15.40:0/3905803959 >> 172.21.15.40:0/3905803959 conn(0x7fe28c0fb100 msgr2=0x7fe28c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.372+0000 7fe293783700 1 -- 172.21.15.40:0/3905803959 shutdown_connections 2023-07-18T03:32:27.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.372+0000 7fe293783700 1 -- 172.21.15.40:0/3905803959 wait complete. 2023-07-18T03:32:27.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.372+0000 7fe293783700 1 Processor -- start 2023-07-18T03:32:27.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.372+0000 7fe293783700 1 -- start start 2023-07-18T03:32:27.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe293783700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe28c0ffae0 0x7fe28c1957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe293783700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe28c100cd0 0x7fe28c195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe293783700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe28c101660 0x7fe28c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe293783700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c19d350 con 0x7fe28c100cd0 2023-07-18T03:32:27.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe293783700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c19d490 con 0x7fe28c0ffae0 2023-07-18T03:32:27.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe293783700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c19d620 con 0x7fe28c101660 2023-07-18T03:32:27.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe290d1e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe28c100cd0 0x7fe28c195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe290d1e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe28c100cd0 0x7fe28c195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59020/0 (socket says 172.21.15.40:59020) 2023-07-18T03:32:27.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe290d1e700 1 -- 172.21.15.40:0/1725648042 learned_addr learned my addr 172.21.15.40:0/1725648042 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:27.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe29151f700 1 --2- 172.21.15.40:0/1725648042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe28c0ffae0 0x7fe28c1957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe291d20700 1 --2- 172.21.15.40:0/1725648042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe28c101660 0x7fe28c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe290d1e700 1 -- 172.21.15.40:0/1725648042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe28c101660 msgr2=0x7fe28c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe290d1e700 1 --2- 172.21.15.40:0/1725648042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe28c101660 0x7fe28c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe290d1e700 1 -- 172.21.15.40:0/1725648042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe28c0ffae0 msgr2=0x7fe28c1957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe290d1e700 1 --2- 172.21.15.40:0/1725648042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe28c0ffae0 0x7fe28c1957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe290d1e700 1 -- 172.21.15.40:0/1725648042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2880095f0 con 0x7fe28c100cd0 2023-07-18T03:32:27.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe290d1e700 1 --2- 172.21.15.40:0/1725648042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe28c100cd0 0x7fe28c195d10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00e9a0 tx=0x7fe27c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:27.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe2827fc700 1 -- 172.21.15.40:0/1725648042 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c009790 con 0x7fe28c100cd0 2023-07-18T03:32:27.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe2827fc700 1 -- 172.21.15.40:0/1725648042 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe27c010430 con 0x7fe28c100cd0 2023-07-18T03:32:27.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe293783700 1 -- 172.21.15.40:0/1725648042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe28c19d900 con 0x7fe28c100cd0 2023-07-18T03:32:27.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.373+0000 7fe2827fc700 1 -- 172.21.15.40:0/1725648042 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c018680 con 0x7fe28c100cd0 2023-07-18T03:32:27.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.374+0000 7fe293783700 1 -- 172.21.15.40:0/1725648042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28c19de60 con 0x7fe28c100cd0 2023-07-18T03:32:27.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.374+0000 7fe293783700 1 -- 172.21.15.40:0/1725648042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe28c107020 con 0x7fe28c100cd0 2023-07-18T03:32:27.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.377+0000 7fe2827fc700 1 -- 172.21.15.40:0/1725648042 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe27c0105a0 con 0x7fe28c100cd0 2023-07-18T03:32:27.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.377+0000 7fe2827fc700 1 --2- 172.21.15.40:0/1725648042 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe2780a1220 0x7fe2780a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.377+0000 7fe2827fc700 1 -- 172.21.15.40:0/1725648042 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe27c014070 con 0x7fe28c100cd0 2023-07-18T03:32:27.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.377+0000 7fe29151f700 1 --2- 172.21.15.40:0/1725648042 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe2780a1220 0x7fe2780a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.378+0000 7fe29151f700 1 --2- 172.21.15.40:0/1725648042 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe2780a1220 0x7fe2780a36d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe288006010 tx=0x7fe288005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:27.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.380+0000 7fe2827fc700 1 -- 172.21.15.40:0/1725648042 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe27c0963d0 con 0x7fe28c100cd0 2023-07-18T03:32:27.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.657+0000 7fe293783700 1 -- 172.21.15.40:0/1725648042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe28c02ca60 con 0x7fe28c100cd0 2023-07-18T03:32:27.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.657+0000 7fe2827fc700 1 -- 172.21.15.40:0/1725648042 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10896 (secure 0 0 0) 0x7fe27c026020 con 0x7fe28c100cd0 2023-07-18T03:32:27.654 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:32:27.654 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:29:26.718789+0000","modified":"2023-07-18T03:32:24.747159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24523,"mds_2":24482,"mds_3":24452,"mds_4":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14601":{"gid":14601,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1818167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1818167554},{"type":"v1","addr":"172.21.15.40:6839","nonce":1818167554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14613":{"gid":14613,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/3953074960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":3953074960},{"type":"v1","addr":"172.21.15.40:6841","nonce":3953074960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3509655945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3509655945},{"type":"v1","addr":"172.21.15.134:6837","nonce":3509655945}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/3007039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3007039043},{"type":"v1","addr":"172.21.15.64:6839","nonce":3007039043}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:32:27.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.660+0000 7fe293783700 1 -- 172.21.15.40:0/1725648042 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe2780a1220 msgr2=0x7fe2780a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.660+0000 7fe293783700 1 --2- 172.21.15.40:0/1725648042 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe2780a1220 0x7fe2780a36d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe288006010 tx=0x7fe288005960 comp rx=0 tx=0).stop 2023-07-18T03:32:27.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.660+0000 7fe293783700 1 -- 172.21.15.40:0/1725648042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe28c100cd0 msgr2=0x7fe28c195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.660+0000 7fe293783700 1 --2- 172.21.15.40:0/1725648042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe28c100cd0 0x7fe28c195d10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00e9a0 tx=0x7fe27c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.660+0000 7fe293783700 1 -- 172.21.15.40:0/1725648042 shutdown_connections 2023-07-18T03:32:27.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.660+0000 7fe293783700 1 --2- 172.21.15.40:0/1725648042 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe2780a1220 0x7fe2780a36d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.660+0000 7fe293783700 1 --2- 172.21.15.40:0/1725648042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe28c101660 0x7fe28c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.660+0000 7fe293783700 1 --2- 172.21.15.40:0/1725648042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe28c0ffae0 0x7fe28c1957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.660+0000 7fe293783700 1 --2- 172.21.15.40:0/1725648042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe28c100cd0 0x7fe28c195d10 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.660+0000 7fe293783700 1 -- 172.21.15.40:0/1725648042 >> 172.21.15.40:0/1725648042 conn(0x7fe28c0fb100 msgr2=0x7fe28c104160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:27.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.660+0000 7fe293783700 1 -- 172.21.15.40:0/1725648042 shutdown_connections 2023-07-18T03:32:27.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.660+0000 7fe293783700 1 -- 172.21.15.40:0/1725648042 wait complete. 2023-07-18T03:32:27.658 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 31 2023-07-18T03:32:27.669 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-18T03:32:27.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.889+0000 7f443759e700 1 -- 172.21.15.40:0/1305126656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44380ffc90 msgr2=0x7f4438106530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.889+0000 7f443759e700 1 --2- 172.21.15.40:0/1305126656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44380ffc90 0x7f4438106530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4430009f90 tx=0x7f443000b330 comp rx=0 tx=0).stop 2023-07-18T03:32:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.889+0000 7f443759e700 1 -- 172.21.15.40:0/1305126656 shutdown_connections 2023-07-18T03:32:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.889+0000 7f443759e700 1 --2- 172.21.15.40:0/1305126656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44380ffc90 0x7f4438106530 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.889+0000 7f443759e700 1 --2- 172.21.15.40:0/1305126656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44380fe050 0x7f44380fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.889+0000 7f443759e700 1 --2- 172.21.15.40:0/1305126656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44380ff2e0 0x7f44380ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.889+0000 7f443759e700 1 -- 172.21.15.40:0/1305126656 >> 172.21.15.40:0/1305126656 conn(0x7f44380f9670 msgr2=0x7f44380fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.889+0000 7f443759e700 1 -- 172.21.15.40:0/1305126656 shutdown_connections 2023-07-18T03:32:27.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.889+0000 7f443759e700 1 -- 172.21.15.40:0/1305126656 wait complete. 2023-07-18T03:32:27.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.891+0000 7f443759e700 1 Processor -- start 2023-07-18T03:32:27.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.891+0000 7f443759e700 1 -- start start 2023-07-18T03:32:27.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.891+0000 7f443759e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44380fe050 0x7f4438194c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.891+0000 7f443759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44380ff2e0 0x7f4438195170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.891+0000 7f443759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44380ffc90 0x7f443819bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.891+0000 7f443759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443819c100 con 0x7f44380ffc90 2023-07-18T03:32:27.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.891+0000 7f443759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443819c240 con 0x7f44380ff2e0 2023-07-18T03:32:27.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.891+0000 7f443759e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4438193950 con 0x7f44380fe050 2023-07-18T03:32:27.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.891+0000 7f4436d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44380ffc90 0x7f443819bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.891+0000 7f4436d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44380ffc90 0x7f443819bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59032/0 (socket says 172.21.15.40:59032) 2023-07-18T03:32:27.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.891+0000 7f4436d9d700 1 -- 172.21.15.40:0/1838574740 learned_addr learned my addr 172.21.15.40:0/1838574740 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.891+0000 7f443659c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44380fe050 0x7f4438194c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.891+0000 7f4435d9b700 1 --2- 172.21.15.40:0/1838574740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44380ff2e0 0x7f4438195170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.892+0000 7f443659c700 1 -- 172.21.15.40:0/1838574740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44380ff2e0 msgr2=0x7f4438195170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.892+0000 7f443659c700 1 --2- 172.21.15.40:0/1838574740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44380ff2e0 0x7f4438195170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.892+0000 7f443659c700 1 -- 172.21.15.40:0/1838574740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44380ffc90 msgr2=0x7f443819bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.892+0000 7f443659c700 1 --2- 172.21.15.40:0/1838574740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44380ffc90 0x7f443819bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.892+0000 7f443659c700 1 -- 172.21.15.40:0/1838574740 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44200095f0 con 0x7f44380fe050 2023-07-18T03:32:27.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.892+0000 7f4436d9d700 1 --2- 172.21.15.40:0/1838574740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44380ffc90 0x7f443819bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:27.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.892+0000 7f443659c700 1 --2- 172.21.15.40:0/1838574740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44380fe050 0x7f4438194c30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f442800d950 tx=0x7f442800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:27.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.892+0000 7f442f7fe700 1 -- 172.21.15.40:0/1838574740 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442800f890 con 0x7f44380fe050 2023-07-18T03:32:27.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.892+0000 7f442f7fe700 1 -- 172.21.15.40:0/1838574740 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4428010460 con 0x7f44380fe050 2023-07-18T03:32:27.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.892+0000 7f443759e700 1 -- 172.21.15.40:0/1838574740 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4430009c40 con 0x7f44380fe050 2023-07-18T03:32:27.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.892+0000 7f442f7fe700 1 -- 172.21.15.40:0/1838574740 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442800e610 con 0x7f44380fe050 2023-07-18T03:32:27.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.892+0000 7f443759e700 1 -- 172.21.15.40:0/1838574740 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4438193fb0 con 0x7f44380fe050 2023-07-18T03:32:27.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.898+0000 7f4448b85b80 1 Processor -- start 2023-07-18T03:32:27.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.898+0000 7f4448b85b80 1 -- start start 2023-07-18T03:32:27.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.898+0000 7f4448b85b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20eca720 0x55ea20ecab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.898+0000 7f4448b85b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 0x55ea20ecb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.898+0000 7f4448b85b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20ecbe30 0x55ea20ecc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.898+0000 7f4448b85b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea20dcfcb0 con 0x55ea20eca720 2023-07-18T03:32:27.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.898+0000 7f4448b85b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea20e22a40 con 0x55ea20ecbe30 2023-07-18T03:32:27.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.898+0000 7f4448b85b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea20e1bed0 con 0x55ea20ecb070 2023-07-18T03:32:27.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.898+0000 7f4436d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20eca720 0x55ea20ecab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.898+0000 7f4436d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20eca720 0x55ea20ecab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59036/0 (socket says 172.21.15.40:59036) 2023-07-18T03:32:27.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.898+0000 7f4436d9d700 1 -- 172.21.15.40:0/1027885076 learned_addr learned my addr 172.21.15.40:0/1027885076 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:27.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.899+0000 7f4435d9b700 1 --2- 172.21.15.40:0/1027885076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20ecbe30 0x55ea20ecc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.899+0000 7f4436d9d700 1 -- 172.21.15.40:0/1027885076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 msgr2=0x55ea20ecb8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:32:27.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.899+0000 7f4436d9d700 1 --2- 172.21.15.40:0/1027885076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 0x55ea20ecb8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.899+0000 7f4436d9d700 1 -- 172.21.15.40:0/1027885076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20ecbe30 msgr2=0x55ea20ecc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:27.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.899+0000 7f4436d9d700 1 --2- 172.21.15.40:0/1027885076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20ecbe30 0x55ea20ecc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.899+0000 7f4436d9d700 1 -- 172.21.15.40:0/1027885076 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea20c49de0 con 0x55ea20eca720 2023-07-18T03:32:27.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.899+0000 7f4436d9d700 1 --2- 172.21.15.40:0/1027885076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20eca720 0x55ea20ecab30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4430009b70 tx=0x7f4430009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663b0a0dc337819 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:27.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.899+0000 7f442cff9700 1 -- 172.21.15.40:0/1027885076 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443000e040 con 0x55ea20eca720 2023-07-18T03:32:27.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.900+0000 7f442cff9700 1 -- 172.21.15.40:0/1027885076 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4430007620 con 0x55ea20eca720 2023-07-18T03:32:27.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.900+0000 7f442cff9700 1 -- 172.21.15.40:0/1027885076 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430005330 con 0x55ea20eca720 2023-07-18T03:32:27.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.900+0000 7f442f7fe700 1 -- 172.21.15.40:0/1838574740 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f44280108d0 con 0x7f44380fe050 2023-07-18T03:32:27.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.900+0000 7f4448b85b80 1 -- 172.21.15.40:0/1027885076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20eca720 msgr2=0x55ea20ecab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.900+0000 7f4448b85b80 1 --2- 172.21.15.40:0/1027885076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20eca720 0x55ea20ecab30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4430009b70 tx=0x7f4430009ba0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.900+0000 7f442f7fe700 1 --2- 172.21.15.40:0/1838574740 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f44240a11f0 0x7f44240a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.900+0000 7f4448b85b80 1 -- 172.21.15.40:0/1027885076 shutdown_connections 2023-07-18T03:32:27.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.900+0000 7f4448b85b80 1 --2- 172.21.15.40:0/1027885076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20ecbe30 0x55ea20ecc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.900+0000 7f4448b85b80 1 --2- 172.21.15.40:0/1027885076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20eca720 0x55ea20ecab30 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.900+0000 7f4448b85b80 1 --2- 172.21.15.40:0/1027885076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 0x55ea20ecb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.900+0000 7f4448b85b80 1 -- 172.21.15.40:0/1027885076 >> 172.21.15.40:0/1027885076 conn(0x55ea20ebfff0 msgr2=0x55ea20ec5460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:27.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.901+0000 7f442f7fe700 1 -- 172.21.15.40:0/1838574740 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f44280cc5a0 con 0x7f44380fe050 2023-07-18T03:32:27.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.901+0000 7f4436d9d700 1 --2- 172.21.15.40:0/1838574740 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f44240a11f0 0x7f44240a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.901+0000 7f4448b85b80 1 -- 172.21.15.40:0/1027885076 shutdown_connections 2023-07-18T03:32:27.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.901+0000 7f4448b85b80 1 -- 172.21.15.40:0/1027885076 wait complete. 2023-07-18T03:32:27.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.901+0000 7f4436d9d700 1 --2- 172.21.15.40:0/1838574740 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f44240a11f0 0x7f44240a36a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4430007900 tx=0x7f443000b330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:27.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.901+0000 7f4448b85b80 1 -- start start 2023-07-18T03:32:27.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.901+0000 7f4448b85b80 1 Processor -- start 2023-07-18T03:32:27.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.902+0000 7f4448b85b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20eca720 0x55ea20ed42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.902+0000 7f4448b85b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 0x55ea20ed4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.902+0000 7f4448b85b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 0x55ea20edb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.902+0000 7f4448b85b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea20dbbac0 con 0x55ea20ecbe30 2023-07-18T03:32:27.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.902+0000 7f4448b85b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea20dbb7c0 con 0x55ea20eca720 2023-07-18T03:32:27.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.902+0000 7f4448b85b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea20c9f920 con 0x55ea20ecb070 2023-07-18T03:32:27.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.902+0000 7f4448b85b80 10 client.0 _ll_register_callbacks cb 0x55ea20db9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:27.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.902+0000 7f4436d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 0x55ea20edb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.902+0000 7f443659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20eca720 0x55ea20ed42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.902+0000 7f443659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20eca720 0x55ea20ed42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42368/0 (socket says 172.21.15.40:42368) 2023-07-18T03:32:27.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.902+0000 7f4436d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 0x55ea20edb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59046/0 (socket says 172.21.15.40:59046) 2023-07-18T03:32:27.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.902+0000 7f443659c700 1 -- 172.21.15.40:0/3118085498 learned_addr learned my addr 172.21.15.40:0/3118085498 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:27.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.902+0000 7f4436d9d700 1 -- 172.21.15.40:0/3118085498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 msgr2=0x55ea20ed4820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:32:27.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.902+0000 7f4436d9d700 1 --2- 172.21.15.40:0/3118085498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 0x55ea20ed4820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.902+0000 7f4436d9d700 1 -- 172.21.15.40:0/3118085498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20eca720 msgr2=0x55ea20ed42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:27.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.902+0000 7f4436d9d700 1 --2- 172.21.15.40:0/3118085498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20eca720 0x55ea20ed42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.902+0000 7f4436d9d700 1 -- 172.21.15.40:0/3118085498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4438106da0 con 0x55ea20ecbe30 2023-07-18T03:32:27.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.903+0000 7f4436d9d700 1 --2- 172.21.15.40:0/3118085498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 0x55ea20edb860 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4430009790 tx=0x7f4430009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63773e0045339a45 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:27.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.903+0000 7f441dffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:32:27.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.903+0000 7f441dffb700 1 -- 172.21.15.40:0/3118085498 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443000e040 con 0x55ea20ecbe30 2023-07-18T03:32:27.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.903+0000 7f441dffb700 1 -- 172.21.15.40:0/3118085498 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f44300058e0 con 0x55ea20ecbe30 2023-07-18T03:32:27.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.903+0000 7f4448b85b80 1 -- 172.21.15.40:0/3118085498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea20d754f0 con 0x55ea20ecbe30 2023-07-18T03:32:27.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.903+0000 7f441dffb700 1 -- 172.21.15.40:0/3118085498 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4430025460 con 0x55ea20ecbe30 2023-07-18T03:32:27.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.903+0000 7f4448b85b80 10 client.15906 fetch_fsmap learned FSMap version 31 2023-07-18T03:32:27.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.903+0000 7f4448b85b80 1 -- 172.21.15.40:0/3118085498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55ea20c32c60 con 0x55ea20ecbe30 2023-07-18T03:32:27.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.903+0000 7f441dffb700 1 -- 172.21.15.40:0/3118085498 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4430048080 con 0x55ea20ecbe30 2023-07-18T03:32:27.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.904+0000 7f441dffb700 1 -- 172.21.15.40:0/3118085498 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f4430042030 con 0x55ea20ecbe30 2023-07-18T03:32:27.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.904+0000 7f4448b85b80 10 client.15906 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:32:27.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.904+0000 7f4448b85b80 10 client.15906 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2023-07-18T03:32:27.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.904+0000 7f4448b85b80 20 client.15906 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.904+0000 7f4448b85b80 1 --2- 172.21.15.40:0/3118085498 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55ea20edfcf0 0x55ea20ee20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.904+0000 7f4448b85b80 4 client.15906 mds_command: new command op to 24442 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:32:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.904+0000 7f4448b85b80 1 -- 172.21.15.40:0/3118085498 --> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea20e14eb0 con 0x55ea20edfcf0 2023-07-18T03:32:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.906+0000 7f443659c700 1 --2- 172.21.15.40:0/3118085498 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55ea20edfcf0 0x55ea20ee20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.907+0000 7f441dffb700 10 client.15906 ms_handle_connect on v2:172.21.15.134:6832/2345264306 2023-07-18T03:32:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.907+0000 7f443659c700 1 --2- 172.21.15.40:0/3118085498 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55ea20edfcf0 0x55ea20ee20d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=92a7dbc6c9febe80 server_cookie=977b143ec4f27e23 in_seq=0 out_seq=0 2023-07-18T03:32:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.907+0000 7f441dffb700 1 -- 172.21.15.40:0/3118085498 <== mds.4 v2:172.21.15.134:6832/2345264306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ea20e14eb0 con 0x55ea20edfcf0 2023-07-18T03:32:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.907+0000 7f441dffb700 10 client.15906 handle_command_reply: tid=0 2023-07-18T03:32:27.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.907+0000 7f4448b85b80 1 client.15906 shutdown 2023-07-18T03:32:27.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.907+0000 7f4448b85b80 1 -- 172.21.15.40:0/3118085498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 msgr2=0x55ea20edb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.907+0000 7f4448b85b80 1 --2- 172.21.15.40:0/3118085498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 0x55ea20edb860 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4430009790 tx=0x7f4430009ba0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.907+0000 7f4448b85b80 1 -- 172.21.15.40:0/3118085498 shutdown_connections 2023-07-18T03:32:27.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.907+0000 7f4448b85b80 1 --2- 172.21.15.40:0/3118085498 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55ea20edfcf0 0x55ea20ee20d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.907+0000 7f4448b85b80 1 --2- 172.21.15.40:0/3118085498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 0x55ea20edb860 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.907+0000 7f4448b85b80 1 --2- 172.21.15.40:0/3118085498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20eca720 0x55ea20ed42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.908+0000 7f4448b85b80 1 --2- 172.21.15.40:0/3118085498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 0x55ea20ed4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.907+0000 7f441dffb700 0 client.15906 ms_handle_reset on v2:172.21.15.134:6832/2345264306 2023-07-18T03:32:27.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.908+0000 7f4448b85b80 1 -- 172.21.15.40:0/3118085498 >> 172.21.15.40:0/3118085498 conn(0x55ea20ebfff0 msgr2=0x55ea20ec4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:27.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.908+0000 7f4448b85b80 1 -- 172.21.15.40:0/3118085498 shutdown_connections 2023-07-18T03:32:27.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.908+0000 7f4448b85b80 1 -- 172.21.15.40:0/3118085498 wait complete. 2023-07-18T03:32:27.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.908+0000 7f4448b85b80 20 client.15906 trim_cache size 0 max 16384 2023-07-18T03:32:27.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.931+0000 7f4448b85b80 1 Processor -- start 2023-07-18T03:32:27.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.935+0000 7f4448b85b80 1 -- start start 2023-07-18T03:32:27.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.935+0000 7f4448b85b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20eca720 0x55ea20ed21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.935+0000 7f4448b85b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 0x55ea20edb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.935+0000 7f4448b85b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 0x55ea20ececf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.935+0000 7f4448b85b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea20ec6400 con 0x55ea20ecbe30 2023-07-18T03:32:27.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.935+0000 7f4448b85b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea20ed4f60 con 0x55ea20eca720 2023-07-18T03:32:27.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.935+0000 7f4448b85b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea20ed1f70 con 0x55ea20ecb070 2023-07-18T03:32:27.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.938+0000 7f4436d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 0x55ea20ececf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.938+0000 7f4436d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 0x55ea20ececf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59050/0 (socket says 172.21.15.40:59050) 2023-07-18T03:32:27.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.938+0000 7f4436d9d700 1 -- 172.21.15.40:0/1748736093 learned_addr learned my addr 172.21.15.40:0/1748736093 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:27.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.943+0000 7f443659c700 1 --2- 172.21.15.40:0/1748736093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20eca720 0x55ea20ed21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.943+0000 7f4435d9b700 1 --2- 172.21.15.40:0/1748736093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 0x55ea20edb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.944+0000 7f4436d9d700 1 -- 172.21.15.40:0/1748736093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 msgr2=0x55ea20edb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:27.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.944+0000 7f4436d9d700 1 --2- 172.21.15.40:0/1748736093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 0x55ea20edb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.944+0000 7f4436d9d700 1 -- 172.21.15.40:0/1748736093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20eca720 msgr2=0x55ea20ed21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:27.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.944+0000 7f4436d9d700 1 --2- 172.21.15.40:0/1748736093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20eca720 0x55ea20ed21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.944+0000 7f4436d9d700 1 -- 172.21.15.40:0/1748736093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea20c70250 con 0x55ea20ecbe30 2023-07-18T03:32:27.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.949+0000 7f4436d9d700 1 --2- 172.21.15.40:0/1748736093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 0x55ea20ececf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4430000f80 tx=0x7f4430006180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44035b8760fb6ea server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:27.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.953+0000 7f442d7fa700 1 -- 172.21.15.40:0/1748736093 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443000e070 con 0x55ea20ecbe30 2023-07-18T03:32:27.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.953+0000 7f442d7fa700 1 -- 172.21.15.40:0/1748736093 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ea20d754f0 con 0x55ea20ecbe30 2023-07-18T03:32:27.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.953+0000 7f442d7fa700 1 -- 172.21.15.40:0/1748736093 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430025850 con 0x55ea20ecbe30 2023-07-18T03:32:27.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.954+0000 7f4448b85b80 1 -- 172.21.15.40:0/1748736093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 msgr2=0x55ea20ececf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.954+0000 7f4448b85b80 1 --2- 172.21.15.40:0/1748736093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 0x55ea20ececf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4430000f80 tx=0x7f4430006180 comp rx=0 tx=0).stop 2023-07-18T03:32:27.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.954+0000 7f4448b85b80 1 -- 172.21.15.40:0/1748736093 shutdown_connections 2023-07-18T03:32:27.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.954+0000 7f4448b85b80 1 --2- 172.21.15.40:0/1748736093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 0x55ea20ececf0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.954+0000 7f4448b85b80 1 --2- 172.21.15.40:0/1748736093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20eca720 0x55ea20ed21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.954+0000 7f4448b85b80 1 --2- 172.21.15.40:0/1748736093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 0x55ea20edb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.954+0000 7f4448b85b80 1 -- 172.21.15.40:0/1748736093 >> 172.21.15.40:0/1748736093 conn(0x55ea20ebfff0 msgr2=0x55ea20ed2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:27.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.960+0000 7f4448b85b80 1 -- 172.21.15.40:0/1748736093 shutdown_connections 2023-07-18T03:32:27.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.962+0000 7f4448b85b80 1 -- 172.21.15.40:0/1748736093 wait complete. 2023-07-18T03:32:27.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.962+0000 7f4448b85b80 1 -- start start 2023-07-18T03:32:27.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.962+0000 7f4448b85b80 1 Processor -- start 2023-07-18T03:32:27.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.963+0000 7f4448b85b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20eca720 0x55ea20ed5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.963+0000 7f4448b85b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 0x55ea20ed5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.963+0000 7f4448b85b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 0x55ea20edaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.963+0000 7f4448b85b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea20edb480 con 0x55ea20ecbe30 2023-07-18T03:32:27.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.963+0000 7f4448b85b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea20edb5f0 con 0x55ea20eca720 2023-07-18T03:32:27.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.963+0000 7f4448b85b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea20edb7b0 con 0x55ea20ecb070 2023-07-18T03:32:27.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.963+0000 7f4448b85b80 10 client.0 _ll_register_callbacks cb 0x7f4430003d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:27.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.963+0000 7f4435d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 0x55ea20ed5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.963+0000 7f4435d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 0x55ea20ed5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48086/0 (socket says 172.21.15.40:48086) 2023-07-18T03:32:27.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.963+0000 7f4435d9b700 1 -- 172.21.15.40:0/2640853560 learned_addr learned my addr 172.21.15.40:0/2640853560 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:27.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.963+0000 7f443659c700 1 --2- 172.21.15.40:0/2640853560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20eca720 0x55ea20ed5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.963+0000 7f4436d9d700 1 --2- 172.21.15.40:0/2640853560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 0x55ea20edaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.964+0000 7f4435d9b700 1 -- 172.21.15.40:0/2640853560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20eca720 msgr2=0x55ea20ed5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:27.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.964+0000 7f4435d9b700 1 --2- 172.21.15.40:0/2640853560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20eca720 0x55ea20ed5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.964+0000 7f4435d9b700 1 -- 172.21.15.40:0/2640853560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 msgr2=0x55ea20edaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:27.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.964+0000 7f4435d9b700 1 --2- 172.21.15.40:0/2640853560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 0x55ea20edaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.964+0000 7f4435d9b700 1 -- 172.21.15.40:0/2640853560 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea20c32c60 con 0x55ea20ecb070 2023-07-18T03:32:27.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.964+0000 7f443659c700 1 --2- 172.21.15.40:0/2640853560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20eca720 0x55ea20ed5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:27.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.964+0000 7f4435d9b700 1 --2- 172.21.15.40:0/2640853560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 0x55ea20ed5f60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4420009fd0 tx=0x7f4420010ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6398862c56c0fb1f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:27.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.964+0000 7f441dffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:32:27.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.974+0000 7f441dffb700 1 -- 172.21.15.40:0/2640853560 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442000f8f0 con 0x55ea20ecb070 2023-07-18T03:32:27.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.974+0000 7f441dffb700 1 -- 172.21.15.40:0/2640853560 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f442000fa50 con 0x55ea20ecb070 2023-07-18T03:32:27.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.974+0000 7f4448b85b80 1 -- 172.21.15.40:0/2640853560 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4430007620 con 0x55ea20ecb070 2023-07-18T03:32:27.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.974+0000 7f441dffb700 1 -- 172.21.15.40:0/2640853560 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f442000fc10 con 0x55ea20ecb070 2023-07-18T03:32:27.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.975+0000 7f4448b85b80 10 client.25555 fetch_fsmap learned FSMap version 31 2023-07-18T03:32:27.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.975+0000 7f4448b85b80 1 -- 172.21.15.40:0/2640853560 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55ea20c482b0 con 0x55ea20ecb070 2023-07-18T03:32:27.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.977+0000 7f441dffb700 1 -- 172.21.15.40:0/2640853560 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4430007620 con 0x55ea20ecb070 2023-07-18T03:32:27.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.977+0000 7f441dffb700 1 -- 172.21.15.40:0/2640853560 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f4420023070 con 0x55ea20ecb070 2023-07-18T03:32:27.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.977+0000 7f4448b85b80 10 client.25555 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:32:27.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.977+0000 7f4448b85b80 10 client.25555 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2023-07-18T03:32:27.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.977+0000 7f4448b85b80 20 client.25555 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:27.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.977+0000 7f4448b85b80 1 --2- 172.21.15.40:0/2640853560 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55ea20f4e780 0x55ea20f50b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:27.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.977+0000 7f4448b85b80 4 client.25555 mds_command: new command op to 24442 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:32:27.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.977+0000 7f4448b85b80 1 -- 172.21.15.40:0/2640853560 --> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ea20dd7050 con 0x55ea20f4e780 2023-07-18T03:32:27.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.977+0000 7f443659c700 1 --2- 172.21.15.40:0/2640853560 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55ea20f4e780 0x55ea20f50b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:27.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.978+0000 7f441dffb700 10 client.25555 ms_handle_connect on v2:172.21.15.134:6832/2345264306 2023-07-18T03:32:27.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.978+0000 7f443659c700 1 --2- 172.21.15.40:0/2640853560 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55ea20f4e780 0x55ea20f50b90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b129b9cc90261d87 server_cookie=f6f0080eb585d0c5 in_seq=0 out_seq=0 2023-07-18T03:32:27.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.978+0000 7f441dffb700 1 -- 172.21.15.40:0/2640853560 <== mds.4 v2:172.21.15.134:6832/2345264306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ea20dd7050 con 0x55ea20f4e780 2023-07-18T03:32:27.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.978+0000 7f441dffb700 10 client.25555 handle_command_reply: tid=0 2023-07-18T03:32:27.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.978+0000 7f4448b85b80 1 client.25555 shutdown 2023-07-18T03:32:27.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.978+0000 7f4448b85b80 1 -- 172.21.15.40:0/2640853560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 msgr2=0x55ea20ed5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.978+0000 7f4448b85b80 1 --2- 172.21.15.40:0/2640853560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 0x55ea20ed5f60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4420009fd0 tx=0x7f4420010ad0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.979+0000 7f4448b85b80 1 -- 172.21.15.40:0/2640853560 shutdown_connections 2023-07-18T03:32:27.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.979+0000 7f4448b85b80 1 --2- 172.21.15.40:0/2640853560 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x55ea20f4e780 0x55ea20f50b90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.979+0000 7f4448b85b80 1 --2- 172.21.15.40:0/2640853560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ea20ecbe30 0x55ea20edaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.979+0000 7f4448b85b80 1 --2- 172.21.15.40:0/2640853560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea20eca720 0x55ea20ed5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.979+0000 7f4448b85b80 1 --2- 172.21.15.40:0/2640853560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea20ecb070 0x55ea20ed5f60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.979+0000 7f441dffb700 0 client.25555 ms_handle_reset on v2:172.21.15.134:6832/2345264306 2023-07-18T03:32:27.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.979+0000 7f4448b85b80 1 -- 172.21.15.40:0/2640853560 >> 172.21.15.40:0/2640853560 conn(0x55ea20ebfff0 msgr2=0x55ea20ed21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:27.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.979+0000 7f4448b85b80 1 -- 172.21.15.40:0/2640853560 shutdown_connections 2023-07-18T03:32:27.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.979+0000 7f4448b85b80 1 -- 172.21.15.40:0/2640853560 wait complete. 2023-07-18T03:32:27.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.979+0000 7f4448b85b80 20 client.25555 trim_cache size 0 max 16384 2023-07-18T03:32:27.979 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:32:27.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.980+0000 7f443759e700 1 -- 172.21.15.40:0/1838574740 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f44240a11f0 msgr2=0x7f44240a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.980+0000 7f443759e700 1 --2- 172.21.15.40:0/1838574740 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f44240a11f0 0x7f44240a36a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4430007900 tx=0x7f443000b330 comp rx=0 tx=0).stop 2023-07-18T03:32:27.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.980+0000 7f443759e700 1 -- 172.21.15.40:0/1838574740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44380fe050 msgr2=0x7f4438194c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:27.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.980+0000 7f443759e700 1 --2- 172.21.15.40:0/1838574740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44380fe050 0x7f4438194c30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f442800d950 tx=0x7f442800dc60 comp rx=0 tx=0).stop 2023-07-18T03:32:27.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.980+0000 7f443759e700 1 -- 172.21.15.40:0/1838574740 shutdown_connections 2023-07-18T03:32:27.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.980+0000 7f443759e700 1 --2- 172.21.15.40:0/1838574740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44380ffc90 0x7f443819bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.980+0000 7f443759e700 1 --2- 172.21.15.40:0/1838574740 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f44240a11f0 0x7f44240a36a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.980+0000 7f443759e700 1 --2- 172.21.15.40:0/1838574740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44380fe050 0x7f4438194c30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.980+0000 7f443759e700 1 --2- 172.21.15.40:0/1838574740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44380ff2e0 0x7f4438195170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:27.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.980+0000 7f443759e700 1 -- 172.21.15.40:0/1838574740 >> 172.21.15.40:0/1838574740 conn(0x7f44380f9670 msgr2=0x7f44380fb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:27.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.981+0000 7f443759e700 1 -- 172.21.15.40:0/1838574740 shutdown_connections 2023-07-18T03:32:27.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:27.981+0000 7f443759e700 1 -- 172.21.15.40:0/1838574740 wait complete. 2023-07-18T03:32:27.993 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:27 smithi040 ceph-mon[96439]: pgmap v340: 97 pgs: 97 active+clean; 47 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 204 op/s 2023-07-18T03:32:27.994 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:27 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1725648042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:32:27.995 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-18T03:32:28.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:27 smithi134 ceph-mon[98216]: pgmap v340: 97 pgs: 97 active+clean; 47 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 204 op/s 2023-07-18T03:32:28.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:27 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1725648042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:32:28.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:27 smithi064 ceph-mon[98059]: pgmap v340: 97 pgs: 97 active+clean; 47 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 204 op/s 2023-07-18T03:32:28.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:27 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1725648042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:32:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.138+0000 7ff10ad53700 1 -- 172.21.15.40:0/3638381297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff104101670 msgr2=0x7ff104107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.138+0000 7ff10ad53700 1 --2- 172.21.15.40:0/3638381297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff104101670 0x7ff104107f30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00b0a0 tx=0x7ff0fc00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.138+0000 7ff10ad53700 1 -- 172.21.15.40:0/3638381297 shutdown_connections 2023-07-18T03:32:28.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.138+0000 7ff10ad53700 1 --2- 172.21.15.40:0/3638381297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff104101670 0x7ff104107f30 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.138+0000 7ff10ad53700 1 --2- 172.21.15.40:0/3638381297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1040ffaf0 0x7ff1040fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.138+0000 7ff10ad53700 1 --2- 172.21.15.40:0/3638381297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff104100ce0 0x7ff104101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.138+0000 7ff10ad53700 1 -- 172.21.15.40:0/3638381297 >> 172.21.15.40:0/3638381297 conn(0x7ff1040fb130 msgr2=0x7ff1040fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.138+0000 7ff10ad53700 1 -- 172.21.15.40:0/3638381297 shutdown_connections 2023-07-18T03:32:28.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.138+0000 7ff10ad53700 1 -- 172.21.15.40:0/3638381297 wait complete. 2023-07-18T03:32:28.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff10ad53700 1 Processor -- start 2023-07-18T03:32:28.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff10ad53700 1 -- start start 2023-07-18T03:32:28.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff10ad53700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1040ffaf0 0x7ff1040742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff10ad53700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff104100ce0 0x7ff104072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff10ad53700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff104101670 0x7ff104072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff10ad53700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040735e0 con 0x7ff104101670 2023-07-18T03:32:28.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff10ad53700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff104073750 con 0x7ff104100ce0 2023-07-18T03:32:28.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff10ad53700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10419f950 con 0x7ff1040ffaf0 2023-07-18T03:32:28.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff1092f0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff104101670 0x7ff104072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff1092f0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff104101670 0x7ff104072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59074/0 (socket says 172.21.15.40:59074) 2023-07-18T03:32:28.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff1092f0700 1 -- 172.21.15.40:0/1930224075 learned_addr learned my addr 172.21.15.40:0/1930224075 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff103fff700 1 --2- 172.21.15.40:0/1930224075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff104100ce0 0x7ff104072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff108aef700 1 --2- 172.21.15.40:0/1930224075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1040ffaf0 0x7ff1040742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff1092f0700 1 -- 172.21.15.40:0/1930224075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1040ffaf0 msgr2=0x7ff1040742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff1092f0700 1 --2- 172.21.15.40:0/1930224075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1040ffaf0 0x7ff1040742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff1092f0700 1 -- 172.21.15.40:0/1930224075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff104100ce0 msgr2=0x7ff104072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff1092f0700 1 --2- 172.21.15.40:0/1930224075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff104100ce0 0x7ff104072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.139+0000 7ff1092f0700 1 -- 172.21.15.40:0/1930224075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc009d00 con 0x7ff104101670 2023-07-18T03:32:28.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.140+0000 7ff1092f0700 1 --2- 172.21.15.40:0/1930224075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff104101670 0x7ff104072e40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00b780 tx=0x7ff0fc00bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.140+0000 7ff101ffb700 1 -- 172.21.15.40:0/1930224075 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc00e040 con 0x7ff104101670 2023-07-18T03:32:28.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.140+0000 7ff10ad53700 1 -- 172.21.15.40:0/1930224075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10419fbd0 con 0x7ff104101670 2023-07-18T03:32:28.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.140+0000 7ff101ffb700 1 -- 172.21.15.40:0/1930224075 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0fc004c60 con 0x7ff104101670 2023-07-18T03:32:28.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.140+0000 7ff101ffb700 1 -- 172.21.15.40:0/1930224075 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc01b6b0 con 0x7ff104101670 2023-07-18T03:32:28.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.140+0000 7ff10ad53700 1 -- 172.21.15.40:0/1930224075 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1041a0050 con 0x7ff104101670 2023-07-18T03:32:28.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.143+0000 7ff101ffb700 1 -- 172.21.15.40:0/1930224075 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff0fc01d480 con 0x7ff104101670 2023-07-18T03:32:28.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.143+0000 7ff101ffb700 1 --2- 172.21.15.40:0/1930224075 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff0f40a1150 0x7ff0f40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.143+0000 7ff101ffb700 1 -- 172.21.15.40:0/1930224075 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff0fc0ce230 con 0x7ff104101670 2023-07-18T03:32:28.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.144+0000 7ff108aef700 1 --2- 172.21.15.40:0/1930224075 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff0f40a1150 0x7ff0f40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.144+0000 7ff108aef700 1 --2- 172.21.15.40:0/1930224075 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff0f40a1150 0x7ff0f40a3600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff104100b40 tx=0x7ff0f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.144+0000 7ff116ed9b80 1 Processor -- start 2023-07-18T03:32:28.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.144+0000 7ff116ed9b80 1 -- start start 2023-07-18T03:32:28.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.144+0000 7ff116ed9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5c4d0 0x56527ca5c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.144+0000 7ff116ed9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5ce20 0x56527ca5d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.144+0000 7ff116ed9b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5dc70 0x56527ca5e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.144+0000 7ff116ed9b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527c961cb0 con 0x56527ca5dc70 2023-07-18T03:32:28.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.144+0000 7ff116ed9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527c9b4a40 con 0x56527ca5ce20 2023-07-18T03:32:28.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.144+0000 7ff116ed9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527c9aded0 con 0x56527ca5c4d0 2023-07-18T03:32:28.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.145+0000 7ff108aef700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5c4d0 0x56527ca5c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.145+0000 7ff108aef700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5c4d0 0x56527ca5c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48106/0 (socket says 172.21.15.40:48106) 2023-07-18T03:32:28.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.145+0000 7ff108aef700 1 -- 172.21.15.40:0/1475271442 learned_addr learned my addr 172.21.15.40:0/1475271442 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.145+0000 7ff1092f0700 1 --2- 172.21.15.40:0/1475271442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5dc70 0x56527ca5e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.145+0000 7ff103fff700 1 --2- 172.21.15.40:0/1475271442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5ce20 0x56527ca5d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.145+0000 7ff108aef700 1 -- 172.21.15.40:0/1475271442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5ce20 msgr2=0x56527ca5d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.145+0000 7ff108aef700 1 --2- 172.21.15.40:0/1475271442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5ce20 0x56527ca5d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.145+0000 7ff108aef700 1 -- 172.21.15.40:0/1475271442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5dc70 msgr2=0x56527ca5e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.145+0000 7ff108aef700 1 --2- 172.21.15.40:0/1475271442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5dc70 0x56527ca5e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.145+0000 7ff108aef700 1 -- 172.21.15.40:0/1475271442 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56527c7dbde0 con 0x56527ca5c4d0 2023-07-18T03:32:28.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.145+0000 7ff108aef700 1 --2- 172.21.15.40:0/1475271442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5c4d0 0x56527ca5c8e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8011d50 tx=0x7ff0f8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=946b91e5f2c34245 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.145+0000 7ff0eeffd700 1 -- 172.21.15.40:0/1475271442 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800ed20 con 0x56527ca5c4d0 2023-07-18T03:32:28.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.145+0000 7ff0eeffd700 1 -- 172.21.15.40:0/1475271442 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0f8021070 con 0x56527ca5c4d0 2023-07-18T03:32:28.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.145+0000 7ff0eeffd700 1 -- 172.21.15.40:0/1475271442 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f801d670 con 0x56527ca5c4d0 2023-07-18T03:32:28.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.146+0000 7ff116ed9b80 1 -- 172.21.15.40:0/1475271442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5c4d0 msgr2=0x56527ca5c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.146+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/1475271442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5c4d0 0x56527ca5c8e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8011d50 tx=0x7ff0f8014000 comp rx=0 tx=0).stop 2023-07-18T03:32:28.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.146+0000 7ff116ed9b80 1 -- 172.21.15.40:0/1475271442 shutdown_connections 2023-07-18T03:32:28.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.146+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/1475271442 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5dc70 0x56527ca5e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.146+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/1475271442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5c4d0 0x56527ca5c8e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.146+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/1475271442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5ce20 0x56527ca5d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.146+0000 7ff116ed9b80 1 -- 172.21.15.40:0/1475271442 >> 172.21.15.40:0/1475271442 conn(0x56527ca51da0 msgr2=0x56527ca571d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.146+0000 7ff116ed9b80 1 -- 172.21.15.40:0/1475271442 shutdown_connections 2023-07-18T03:32:28.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.146+0000 7ff116ed9b80 1 -- 172.21.15.40:0/1475271442 wait complete. 2023-07-18T03:32:28.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.146+0000 7ff116ed9b80 1 -- start start 2023-07-18T03:32:28.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.147+0000 7ff116ed9b80 1 Processor -- start 2023-07-18T03:32:28.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.147+0000 7ff116ed9b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 0x56527ca66120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.147+0000 7ff116ed9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5ce20 0x56527ca66660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.147+0000 7ff116ed9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5dc70 0x56527ca6d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.147+0000 7ff116ed9b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527c94dac0 con 0x56527ca5c4d0 2023-07-18T03:32:28.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.147+0000 7ff116ed9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527c94d7c0 con 0x56527ca5ce20 2023-07-18T03:32:28.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.147+0000 7ff116ed9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527c831920 con 0x56527ca5dc70 2023-07-18T03:32:28.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.147+0000 7ff116ed9b80 10 client.0 _ll_register_callbacks cb 0x56527c94b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:28.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.147+0000 7ff108aef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 0x56527ca66120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.147+0000 7ff108aef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 0x56527ca66120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59084/0 (socket says 172.21.15.40:59084) 2023-07-18T03:32:28.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.147+0000 7ff1092f0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5dc70 0x56527ca6d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.147+0000 7ff108aef700 1 -- 172.21.15.40:0/2127566404 learned_addr learned my addr 172.21.15.40:0/2127566404 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.147+0000 7ff103fff700 1 --2- 172.21.15.40:0/2127566404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5ce20 0x56527ca66660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.148+0000 7ff1092f0700 1 -- 172.21.15.40:0/2127566404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5ce20 msgr2=0x56527ca66660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.148+0000 7ff1092f0700 1 --2- 172.21.15.40:0/2127566404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5ce20 0x56527ca66660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.148+0000 7ff1092f0700 1 -- 172.21.15.40:0/2127566404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 msgr2=0x56527ca66120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.148+0000 7ff1092f0700 1 --2- 172.21.15.40:0/2127566404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 0x56527ca66120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.148+0000 7ff1092f0700 1 -- 172.21.15.40:0/2127566404 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f8017070 con 0x56527ca5dc70 2023-07-18T03:32:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.148+0000 7ff1092f0700 1 --2- 172.21.15.40:0/2127566404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5dc70 0x56527ca6d6a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0cc080 tx=0x7ff0fc00b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99005a546855c269 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.148+0000 7ff0e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:32:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.148+0000 7ff0e3fff700 1 -- 172.21.15.40:0/2127566404 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc09b6b0 con 0x56527ca5dc70 2023-07-18T03:32:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.148+0000 7ff116ed9b80 1 -- 172.21.15.40:0/2127566404 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56527c9074f0 con 0x56527ca5dc70 2023-07-18T03:32:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.148+0000 7ff0e3fff700 1 -- 172.21.15.40:0/2127566404 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff0fc09f6e0 con 0x56527ca5dc70 2023-07-18T03:32:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.148+0000 7ff0e3fff700 1 -- 172.21.15.40:0/2127566404 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0fc0a3a70 con 0x56527ca5dc70 2023-07-18T03:32:28.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.148+0000 7ff116ed9b80 10 client.25567 fetch_fsmap learned FSMap version 31 2023-07-18T03:32:28.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.148+0000 7ff116ed9b80 1 -- 172.21.15.40:0/2127566404 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56527c7c4c60 con 0x56527ca5dc70 2023-07-18T03:32:28.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.148+0000 7ff0e3fff700 1 -- 172.21.15.40:0/2127566404 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56527c9074f0 con 0x56527ca5dc70 2023-07-18T03:32:28.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.148+0000 7ff0e3fff700 1 -- 172.21.15.40:0/2127566404 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff0fc0a35e0 con 0x56527ca5dc70 2023-07-18T03:32:28.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.149+0000 7ff116ed9b80 10 client.25567 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:32:28.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.149+0000 7ff116ed9b80 10 client.25567 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-18T03:32:28.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.149+0000 7ff116ed9b80 20 client.25567 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.149+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/2127566404 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x56527ca71bd0 0x56527ca73fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.149+0000 7ff116ed9b80 4 client.25567 mds_command: new command op to 24452 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:32:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.149+0000 7ff116ed9b80 1 -- 172.21.15.40:0/2127566404 --> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56527c9a6eb0 con 0x56527ca71bd0 2023-07-18T03:32:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.149+0000 7ff108aef700 1 --2- 172.21.15.40:0/2127566404 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x56527ca71bd0 0x56527ca73fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.149+0000 7ff108aef700 1 --2- 172.21.15.40:0/2127566404 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x56527ca71bd0 0x56527ca73fb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e172b9ac6030691b server_cookie=7d40f099efe6233d in_seq=0 out_seq=0 2023-07-18T03:32:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.149+0000 7ff0e3fff700 10 client.25567 ms_handle_connect on v2:172.21.15.64:6834/1864495009 2023-07-18T03:32:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.150+0000 7ff0e3fff700 1 -- 172.21.15.40:0/2127566404 <== mds.3 v2:172.21.15.64:6834/1864495009 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56527c9a6eb0 con 0x56527ca71bd0 2023-07-18T03:32:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.150+0000 7ff0e3fff700 10 client.25567 handle_command_reply: tid=0 2023-07-18T03:32:28.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.150+0000 7ff116ed9b80 1 client.25567 shutdown 2023-07-18T03:32:28.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.150+0000 7ff116ed9b80 1 -- 172.21.15.40:0/2127566404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5dc70 msgr2=0x56527ca6d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.150+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/2127566404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5dc70 0x56527ca6d6a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0cc080 tx=0x7ff0fc00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.150+0000 7ff116ed9b80 1 -- 172.21.15.40:0/2127566404 shutdown_connections 2023-07-18T03:32:28.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.150+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/2127566404 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x56527ca71bd0 0x56527ca73fb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.150+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/2127566404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5dc70 0x56527ca6d6a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.150+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/2127566404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 0x56527ca66120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.150+0000 7ff0e3fff700 0 client.25567 ms_handle_reset on v2:172.21.15.64:6834/1864495009 2023-07-18T03:32:28.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.150+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/2127566404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5ce20 0x56527ca66660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.150+0000 7ff116ed9b80 1 -- 172.21.15.40:0/2127566404 >> 172.21.15.40:0/2127566404 conn(0x56527ca51da0 msgr2=0x56527ca568b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.151+0000 7ff116ed9b80 1 -- 172.21.15.40:0/2127566404 shutdown_connections 2023-07-18T03:32:28.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.151+0000 7ff116ed9b80 1 -- 172.21.15.40:0/2127566404 wait complete. 2023-07-18T03:32:28.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.151+0000 7ff116ed9b80 20 client.25567 trim_cache size 0 max 16384 2023-07-18T03:32:28.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.164+0000 7ff116ed9b80 1 Processor -- start 2023-07-18T03:32:28.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.164+0000 7ff116ed9b80 1 -- start start 2023-07-18T03:32:28.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.164+0000 7ff116ed9b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 0x56527ca5c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff116ed9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5ce20 0x56527ca6d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff116ed9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5dc70 0x56527ca571a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff116ed9b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527ca58a00 con 0x56527ca5c4d0 2023-07-18T03:32:28.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff116ed9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527ca576e0 con 0x56527ca5dc70 2023-07-18T03:32:28.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff116ed9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527ca58c20 con 0x56527ca5ce20 2023-07-18T03:32:28.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff108aef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 0x56527ca5c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff108aef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 0x56527ca5c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59092/0 (socket says 172.21.15.40:59092) 2023-07-18T03:32:28.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff108aef700 1 -- 172.21.15.40:0/4072726311 learned_addr learned my addr 172.21.15.40:0/4072726311 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff103fff700 1 --2- 172.21.15.40:0/4072726311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5ce20 0x56527ca6d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff1092f0700 1 --2- 172.21.15.40:0/4072726311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5dc70 0x56527ca571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff108aef700 1 -- 172.21.15.40:0/4072726311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5ce20 msgr2=0x56527ca6d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff108aef700 1 --2- 172.21.15.40:0/4072726311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5ce20 0x56527ca6d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff108aef700 1 -- 172.21.15.40:0/4072726311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5dc70 msgr2=0x56527ca571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff108aef700 1 --2- 172.21.15.40:0/4072726311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5dc70 0x56527ca571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff108aef700 1 -- 172.21.15.40:0/4072726311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56527c802250 con 0x56527ca5c4d0 2023-07-18T03:32:28.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff108aef700 1 --2- 172.21.15.40:0/4072726311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 0x56527ca5c8a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x56527ca74890 tx=0x7ff0f8011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94864fd3a9e0716 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff0ef7fe700 1 -- 172.21.15.40:0/4072726311 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f801cc10 con 0x56527ca5c4d0 2023-07-18T03:32:28.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff0ef7fe700 1 -- 172.21.15.40:0/4072726311 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0f801ce50 con 0x56527ca5c4d0 2023-07-18T03:32:28.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.165+0000 7ff0ef7fe700 1 -- 172.21.15.40:0/4072726311 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8004550 con 0x56527ca5c4d0 2023-07-18T03:32:28.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.166+0000 7ff116ed9b80 1 -- 172.21.15.40:0/4072726311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 msgr2=0x56527ca5c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.166+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/4072726311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 0x56527ca5c8a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x56527ca74890 tx=0x7ff0f8011cd0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.166+0000 7ff116ed9b80 1 -- 172.21.15.40:0/4072726311 shutdown_connections 2023-07-18T03:32:28.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.166+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/4072726311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5dc70 0x56527ca571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.166+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/4072726311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 0x56527ca5c8a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.166+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/4072726311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5ce20 0x56527ca6d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.166+0000 7ff116ed9b80 1 -- 172.21.15.40:0/4072726311 >> 172.21.15.40:0/4072726311 conn(0x56527ca51da0 msgr2=0x56527ca64e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.166+0000 7ff116ed9b80 1 -- 172.21.15.40:0/4072726311 shutdown_connections 2023-07-18T03:32:28.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.166+0000 7ff116ed9b80 1 -- 172.21.15.40:0/4072726311 wait complete. 2023-07-18T03:32:28.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.166+0000 7ff116ed9b80 1 -- start start 2023-07-18T03:32:28.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff116ed9b80 1 Processor -- start 2023-07-18T03:32:28.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff116ed9b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 0x56527ca69890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff116ed9b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5ce20 0x56527ca69dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff116ed9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5dc70 0x56527ca6c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff116ed9b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527ca6cf10 con 0x56527ca5c4d0 2023-07-18T03:32:28.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff116ed9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527ca6d050 con 0x56527ca5dc70 2023-07-18T03:32:28.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff116ed9b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527ca6d210 con 0x56527ca5ce20 2023-07-18T03:32:28.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff116ed9b80 10 client.0 _ll_register_callbacks cb 0x7ff0fc09b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:28.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff108aef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 0x56527ca69890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff108aef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 0x56527ca69890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59098/0 (socket says 172.21.15.40:59098) 2023-07-18T03:32:28.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff103fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5ce20 0x56527ca69dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff108aef700 1 -- 172.21.15.40:0/313762068 learned_addr learned my addr 172.21.15.40:0/313762068 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff1092f0700 1 --2- 172.21.15.40:0/313762068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5dc70 0x56527ca6c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff108aef700 1 -- 172.21.15.40:0/313762068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5ce20 msgr2=0x56527ca69dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff108aef700 1 --2- 172.21.15.40:0/313762068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5ce20 0x56527ca69dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff108aef700 1 -- 172.21.15.40:0/313762068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5dc70 msgr2=0x56527ca6c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff108aef700 1 --2- 172.21.15.40:0/313762068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5dc70 0x56527ca6c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff108aef700 1 -- 172.21.15.40:0/313762068 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f802a040 con 0x56527ca5c4d0 2023-07-18T03:32:28.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.167+0000 7ff103fff700 1 --2- 172.21.15.40:0/313762068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5ce20 0x56527ca69dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:28.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.168+0000 7ff108aef700 1 --2- 172.21.15.40:0/313762068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 0x56527ca69890 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800eb20 tx=0x7ff0f800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d01914d8fd51f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.168+0000 7ff0e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:32:28.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.168+0000 7ff0e3fff700 1 -- 172.21.15.40:0/313762068 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f80044d0 con 0x56527ca5c4d0 2023-07-18T03:32:28.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.168+0000 7ff0e3fff700 1 -- 172.21.15.40:0/313762068 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff0f800eca0 con 0x56527ca5c4d0 2023-07-18T03:32:28.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.168+0000 7ff116ed9b80 1 -- 172.21.15.40:0/313762068 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0f8021070 con 0x56527ca5c4d0 2023-07-18T03:32:28.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.168+0000 7ff0e3fff700 1 -- 172.21.15.40:0/313762068 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0f801cc10 con 0x56527ca5c4d0 2023-07-18T03:32:28.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.168+0000 7ff116ed9b80 10 client.15930 fetch_fsmap learned FSMap version 31 2023-07-18T03:32:28.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.168+0000 7ff116ed9b80 1 -- 172.21.15.40:0/313762068 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56527c7da2b0 con 0x56527ca5c4d0 2023-07-18T03:32:28.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.168+0000 7ff0e3fff700 1 -- 172.21.15.40:0/313762068 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0f8021070 con 0x56527ca5c4d0 2023-07-18T03:32:28.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.168+0000 7ff0e3fff700 1 -- 172.21.15.40:0/313762068 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff0f8038020 con 0x56527ca5c4d0 2023-07-18T03:32:28.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.168+0000 7ff116ed9b80 10 client.15930 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:32:28.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.168+0000 7ff116ed9b80 10 client.15930 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-18T03:32:28.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.168+0000 7ff116ed9b80 20 client.15930 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:28.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.168+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/313762068 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x56527cac6630 0x56527cac8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.168+0000 7ff116ed9b80 4 client.15930 mds_command: new command op to 24452 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:32:28.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.168+0000 7ff116ed9b80 1 -- 172.21.15.40:0/313762068 --> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56527c969050 con 0x56527cac6630 2023-07-18T03:32:28.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.169+0000 7ff103fff700 1 --2- 172.21.15.40:0/313762068 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x56527cac6630 0x56527cac8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.169+0000 7ff103fff700 1 --2- 172.21.15.40:0/313762068 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x56527cac6630 0x56527cac8a10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=982fcdfc743c2823 server_cookie=9c390b15592ba081 in_seq=0 out_seq=0 2023-07-18T03:32:28.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.169+0000 7ff0e3fff700 10 client.15930 ms_handle_connect on v2:172.21.15.64:6834/1864495009 2023-07-18T03:32:28.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.169+0000 7ff0e3fff700 1 -- 172.21.15.40:0/313762068 <== mds.3 v2:172.21.15.64:6834/1864495009 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56527c969050 con 0x56527cac6630 2023-07-18T03:32:28.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.169+0000 7ff0e3fff700 10 client.15930 handle_command_reply: tid=0 2023-07-18T03:32:28.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.169+0000 7ff116ed9b80 1 client.15930 shutdown 2023-07-18T03:32:28.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.169+0000 7ff116ed9b80 1 -- 172.21.15.40:0/313762068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 msgr2=0x56527ca69890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.169+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/313762068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 0x56527ca69890 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800eb20 tx=0x7ff0f800ee90 comp rx=0 tx=0).stop 2023-07-18T03:32:28.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.169+0000 7ff116ed9b80 1 -- 172.21.15.40:0/313762068 shutdown_connections 2023-07-18T03:32:28.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.169+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/313762068 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x56527cac6630 0x56527cac8a10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.169+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/313762068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56527ca5dc70 0x56527ca6c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.169+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/313762068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56527ca5c4d0 0x56527ca69890 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.169+0000 7ff116ed9b80 1 --2- 172.21.15.40:0/313762068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56527ca5ce20 0x56527ca69dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.169+0000 7ff0e3fff700 0 client.15930 ms_handle_reset on v2:172.21.15.64:6834/1864495009 2023-07-18T03:32:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.169+0000 7ff116ed9b80 1 -- 172.21.15.40:0/313762068 >> 172.21.15.40:0/313762068 conn(0x56527ca51da0 msgr2=0x56527ca64e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.170+0000 7ff116ed9b80 1 -- 172.21.15.40:0/313762068 shutdown_connections 2023-07-18T03:32:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.170+0000 7ff116ed9b80 1 -- 172.21.15.40:0/313762068 wait complete. 2023-07-18T03:32:28.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.170+0000 7ff116ed9b80 20 client.15930 trim_cache size 0 max 16384 2023-07-18T03:32:28.187 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:32:28.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.170+0000 7ff10ad53700 1 -- 172.21.15.40:0/1930224075 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff0f40a1150 msgr2=0x7ff0f40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.170+0000 7ff10ad53700 1 --2- 172.21.15.40:0/1930224075 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff0f40a1150 0x7ff0f40a3600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff104100b40 tx=0x7ff0f8009260 comp rx=0 tx=0).stop 2023-07-18T03:32:28.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.170+0000 7ff10ad53700 1 -- 172.21.15.40:0/1930224075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff104101670 msgr2=0x7ff104072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.170+0000 7ff10ad53700 1 --2- 172.21.15.40:0/1930224075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff104101670 0x7ff104072e40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00b780 tx=0x7ff0fc00bd00 comp rx=0 tx=0).stop 2023-07-18T03:32:28.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.170+0000 7ff10ad53700 1 -- 172.21.15.40:0/1930224075 shutdown_connections 2023-07-18T03:32:28.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.170+0000 7ff10ad53700 1 --2- 172.21.15.40:0/1930224075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff104101670 0x7ff104072e40 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.170+0000 7ff10ad53700 1 --2- 172.21.15.40:0/1930224075 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff0f40a1150 0x7ff0f40a3600 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.170+0000 7ff10ad53700 1 --2- 172.21.15.40:0/1930224075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1040ffaf0 0x7ff1040742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.170+0000 7ff10ad53700 1 --2- 172.21.15.40:0/1930224075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff104100ce0 0x7ff104072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.170+0000 7ff10ad53700 1 -- 172.21.15.40:0/1930224075 >> 172.21.15.40:0/1930224075 conn(0x7ff1040fb130 msgr2=0x7ff1040fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.170+0000 7ff10ad53700 1 -- 172.21.15.40:0/1930224075 shutdown_connections 2023-07-18T03:32:28.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.170+0000 7ff10ad53700 1 -- 172.21.15.40:0/1930224075 wait complete. 2023-07-18T03:32:28.190 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:32:28.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.325+0000 7f6fbd740700 1 -- 172.21.15.40:0/1859031224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fb8102e80 msgr2=0x7f6fb8105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.325+0000 7f6fbd740700 1 --2- 172.21.15.40:0/1859031224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fb8102e80 0x7f6fb8105260 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009990 tx=0x7f6fa8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.325+0000 7f6fbd740700 1 -- 172.21.15.40:0/1859031224 shutdown_connections 2023-07-18T03:32:28.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.325+0000 7f6fbd740700 1 --2- 172.21.15.40:0/1859031224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fb81057a0 0x7f6fb8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.325+0000 7f6fbd740700 1 --2- 172.21.15.40:0/1859031224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb8100560 0x7f6fb8102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.325+0000 7f6fbd740700 1 --2- 172.21.15.40:0/1859031224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fb8102e80 0x7f6fb8105260 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.325+0000 7f6fbd740700 1 -- 172.21.15.40:0/1859031224 >> 172.21.15.40:0/1859031224 conn(0x7f6fb80fa000 msgr2=0x7f6fb80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.325+0000 7f6fbd740700 1 -- 172.21.15.40:0/1859031224 shutdown_connections 2023-07-18T03:32:28.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.325+0000 7f6fbd740700 1 -- 172.21.15.40:0/1859031224 wait complete. 2023-07-18T03:32:28.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.326+0000 7f6fbd740700 1 Processor -- start 2023-07-18T03:32:28.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.326+0000 7f6fbd740700 1 -- start start 2023-07-18T03:32:28.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.326+0000 7f6fbd740700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fb8100560 0x7f6fb80ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.326+0000 7f6fbd740700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb8102e80 0x7f6fb80fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.326+0000 7f6fbd740700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fb81057a0 0x7f6fb80fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.326+0000 7f6fbd740700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb80ff1f0 con 0x7f6fb8100560 2023-07-18T03:32:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.326+0000 7f6fbd740700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb80ff360 con 0x7f6fb8102e80 2023-07-18T03:32:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.326+0000 7f6fbd740700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb819f880 con 0x7f6fb81057a0 2023-07-18T03:32:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.326+0000 7f6fb6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fb8100560 0x7f6fb80ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.326+0000 7f6fb6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fb8100560 0x7f6fb80ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59112/0 (socket says 172.21.15.40:59112) 2023-07-18T03:32:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.326+0000 7f6fb6ffd700 1 -- 172.21.15.40:0/1009180612 learned_addr learned my addr 172.21.15.40:0/1009180612 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.327+0000 7f6fb77fe700 1 --2- 172.21.15.40:0/1009180612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fb81057a0 0x7f6fb80fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.327+0000 7f6fb67fc700 1 --2- 172.21.15.40:0/1009180612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb8102e80 0x7f6fb80fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.327+0000 7f6fb6ffd700 1 -- 172.21.15.40:0/1009180612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fb81057a0 msgr2=0x7f6fb80fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.327+0000 7f6fb6ffd700 1 --2- 172.21.15.40:0/1009180612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fb81057a0 0x7f6fb80fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.327+0000 7f6fb6ffd700 1 -- 172.21.15.40:0/1009180612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb8102e80 msgr2=0x7f6fb80fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.327+0000 7f6fb6ffd700 1 --2- 172.21.15.40:0/1009180612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb8102e80 0x7f6fb80fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.327+0000 7f6fb6ffd700 1 -- 172.21.15.40:0/1009180612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa80095f0 con 0x7f6fb8100560 2023-07-18T03:32:28.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.327+0000 7f6fb6ffd700 1 --2- 172.21.15.40:0/1009180612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fb8100560 0x7f6fb80ffed0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000d950 tx=0x7f6fa000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.327+0000 7f6faffff700 1 -- 172.21.15.40:0/1009180612 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa000f890 con 0x7f6fb8100560 2023-07-18T03:32:28.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.327+0000 7f6faffff700 1 -- 172.21.15.40:0/1009180612 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fa0010460 con 0x7f6fb8100560 2023-07-18T03:32:28.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.327+0000 7f6fbd740700 1 -- 172.21.15.40:0/1009180612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb819fb60 con 0x7f6fb8100560 2023-07-18T03:32:28.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.327+0000 7f6faffff700 1 -- 172.21.15.40:0/1009180612 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa000e610 con 0x7f6fb8100560 2023-07-18T03:32:28.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.327+0000 7f6fbd740700 1 -- 172.21.15.40:0/1009180612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb81a0070 con 0x7f6fb8100560 2023-07-18T03:32:28.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.329+0000 7f6faffff700 1 -- 172.21.15.40:0/1009180612 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6fa00108d0 con 0x7f6fb8100560 2023-07-18T03:32:28.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.330+0000 7f6faffff700 1 --2- 172.21.15.40:0/1009180612 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6fa40a1150 0x7f6fa40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.330+0000 7f6faffff700 1 -- 172.21.15.40:0/1009180612 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6fa00cc740 con 0x7f6fb8100560 2023-07-18T03:32:28.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.330+0000 7f6fb67fc700 1 --2- 172.21.15.40:0/1009180612 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6fa40a1150 0x7f6fa40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.331+0000 7f6fb67fc700 1 --2- 172.21.15.40:0/1009180612 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6fa40a1150 0x7f6fa40a3600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800b2e0 tx=0x7f6fa8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.331+0000 7f6fc98c6b80 1 Processor -- start 2023-07-18T03:32:28.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.331+0000 7f6fc98c6b80 1 -- start start 2023-07-18T03:32:28.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.331+0000 7f6fc98c6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb872024d0 0x55cb872028e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.331+0000 7f6fc98c6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb87202e20 0x55cb872036e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.331+0000 7f6fc98c6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb87203c70 0x55cb872040e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.331+0000 7f6fc98c6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb87107cb0 con 0x55cb87202e20 2023-07-18T03:32:28.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.331+0000 7f6fc98c6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb8715aa40 con 0x55cb872024d0 2023-07-18T03:32:28.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.331+0000 7f6fc98c6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb87153ed0 con 0x55cb87203c70 2023-07-18T03:32:28.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.332+0000 7f6fb77fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb87203c70 0x55cb872040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.332+0000 7f6fb67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb87202e20 0x55cb872036e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.332+0000 7f6fb67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb87202e20 0x55cb872036e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59122/0 (socket says 172.21.15.40:59122) 2023-07-18T03:32:28.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.332+0000 7f6fb67fc700 1 -- 172.21.15.40:0/3817774489 learned_addr learned my addr 172.21.15.40:0/3817774489 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.332+0000 7f6fb77fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb87203c70 0x55cb872040e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48152/0 (socket says 172.21.15.40:48152) 2023-07-18T03:32:28.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.332+0000 7f6fb6ffd700 1 --2- 172.21.15.40:0/3817774489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb872024d0 0x55cb872028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.332+0000 7f6fb67fc700 1 -- 172.21.15.40:0/3817774489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb87203c70 msgr2=0x55cb872040e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.332+0000 7f6fb67fc700 1 --2- 172.21.15.40:0/3817774489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb87203c70 0x55cb872040e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.332+0000 7f6fb67fc700 1 -- 172.21.15.40:0/3817774489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb872024d0 msgr2=0x55cb872028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.332+0000 7f6fb67fc700 1 --2- 172.21.15.40:0/3817774489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb872024d0 0x55cb872028e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.332+0000 7f6fb67fc700 1 -- 172.21.15.40:0/3817774489 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb86f81de0 con 0x55cb87202e20 2023-07-18T03:32:28.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.332+0000 7f6fb67fc700 1 --2- 172.21.15.40:0/3817774489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb87202e20 0x55cb872036e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800bfc0 tx=0x7f6fa8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb3f3d5ffced648 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.332+0000 7f6fad7fa700 1 -- 172.21.15.40:0/3817774489 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa802b070 con 0x55cb87202e20 2023-07-18T03:32:28.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.332+0000 7f6fad7fa700 1 -- 172.21.15.40:0/3817774489 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fa801e040 con 0x55cb87202e20 2023-07-18T03:32:28.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.332+0000 7f6fad7fa700 1 -- 172.21.15.40:0/3817774489 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800f8e0 con 0x55cb87202e20 2023-07-18T03:32:28.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.333+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/3817774489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb87202e20 msgr2=0x55cb872036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.333+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/3817774489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb87202e20 0x55cb872036e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800bfc0 tx=0x7f6fa8020000 comp rx=0 tx=0).stop 2023-07-18T03:32:28.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.333+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/3817774489 shutdown_connections 2023-07-18T03:32:28.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.333+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/3817774489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb87203c70 0x55cb872040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.333+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/3817774489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb872024d0 0x55cb872028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.333+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/3817774489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb87202e20 0x55cb872036e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.333+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/3817774489 >> 172.21.15.40:0/3817774489 conn(0x55cb871f7da0 msgr2=0x55cb871fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.333+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/3817774489 shutdown_connections 2023-07-18T03:32:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.333+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/3817774489 wait complete. 2023-07-18T03:32:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.333+0000 7f6fc98c6b80 1 -- start start 2023-07-18T03:32:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fc98c6b80 1 Processor -- start 2023-07-18T03:32:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fc98c6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb872024d0 0x55cb8720c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fc98c6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb87202e20 0x55cb8720c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fc98c6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb87203c70 0x55cb872135c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fc98c6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb870f3ac0 con 0x55cb872024d0 2023-07-18T03:32:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fc98c6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb870f37c0 con 0x55cb87203c70 2023-07-18T03:32:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fc98c6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb86fd7920 con 0x55cb87202e20 2023-07-18T03:32:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fc98c6b80 10 client.0 _ll_register_callbacks cb 0x55cb870f1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fb6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb872024d0 0x55cb8720c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fb6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb872024d0 0x55cb8720c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59126/0 (socket says 172.21.15.40:59126) 2023-07-18T03:32:28.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fb6ffd700 1 -- 172.21.15.40:0/537620012 learned_addr learned my addr 172.21.15.40:0/537620012 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fb67fc700 1 --2- 172.21.15.40:0/537620012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb87202e20 0x55cb8720c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fb77fe700 1 --2- 172.21.15.40:0/537620012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb87203c70 0x55cb872135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fb6ffd700 1 -- 172.21.15.40:0/537620012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb87202e20 msgr2=0x55cb8720c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fb6ffd700 1 --2- 172.21.15.40:0/537620012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb87202e20 0x55cb8720c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fb6ffd700 1 -- 172.21.15.40:0/537620012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb87203c70 msgr2=0x55cb872135c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fb6ffd700 1 --2- 172.21.15.40:0/537620012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb87203c70 0x55cb872135c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fb6ffd700 1 -- 172.21.15.40:0/537620012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb81087f0 con 0x55cb872024d0 2023-07-18T03:32:28.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6fb6ffd700 1 --2- 172.21.15.40:0/537620012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb872024d0 0x55cb8720c040 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000b2f0 tx=0x7f6fa009db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff4a0358ab26474 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6f9e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:32:28.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.334+0000 7f6f9e7fc700 1 -- 172.21.15.40:0/537620012 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0099f30 con 0x55cb872024d0 2023-07-18T03:32:28.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.335+0000 7f6f9e7fc700 1 -- 172.21.15.40:0/537620012 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6fa0098fc0 con 0x55cb872024d0 2023-07-18T03:32:28.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.335+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/537620012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb870ad4f0 con 0x55cb872024d0 2023-07-18T03:32:28.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.335+0000 7f6f9e7fc700 1 -- 172.21.15.40:0/537620012 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fa00b2200 con 0x55cb872024d0 2023-07-18T03:32:28.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.335+0000 7f6fc98c6b80 10 client.15942 fetch_fsmap learned FSMap version 31 2023-07-18T03:32:28.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.335+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/537620012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55cb86f6ac60 con 0x55cb872024d0 2023-07-18T03:32:28.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.335+0000 7f6f9e7fc700 1 -- 172.21.15.40:0/537620012 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cb870ad4f0 con 0x55cb872024d0 2023-07-18T03:32:28.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.335+0000 7f6f9e7fc700 1 -- 172.21.15.40:0/537620012 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f6fa0098a30 con 0x55cb872024d0 2023-07-18T03:32:28.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.335+0000 7f6fc98c6b80 10 client.15942 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:32:28.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.335+0000 7f6fc98c6b80 10 client.15942 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-18T03:32:28.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.335+0000 7f6fc98c6b80 20 client.15942 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:28.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.335+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/537620012 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55cb87217c90 0x55cb8721a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.335+0000 7f6fc98c6b80 4 client.15942 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:32:28.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.335+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/537620012 --> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb8714ceb0 con 0x55cb87217c90 2023-07-18T03:32:28.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.335+0000 7f6fb67fc700 1 --2- 172.21.15.40:0/537620012 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55cb87217c90 0x55cb8721a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.336+0000 7f6f9e7fc700 10 client.15942 ms_handle_connect on v2:172.21.15.64:6836/3077792296 2023-07-18T03:32:28.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.336+0000 7f6fb67fc700 1 --2- 172.21.15.40:0/537620012 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55cb87217c90 0x55cb8721a0a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=160e9f095b9c2d86 server_cookie=a28c1448aae1aa61 in_seq=0 out_seq=0 2023-07-18T03:32:28.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.336+0000 7f6f9e7fc700 1 -- 172.21.15.40:0/537620012 <== mds.2 v2:172.21.15.64:6836/3077792296 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cb8714ceb0 con 0x55cb87217c90 2023-07-18T03:32:28.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.336+0000 7f6f9e7fc700 10 client.15942 handle_command_reply: tid=0 2023-07-18T03:32:28.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.337+0000 7f6fc98c6b80 1 client.15942 shutdown 2023-07-18T03:32:28.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.337+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/537620012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb872024d0 msgr2=0x55cb8720c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.337+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/537620012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb872024d0 0x55cb8720c040 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000b2f0 tx=0x7f6fa009db00 comp rx=0 tx=0).stop 2023-07-18T03:32:28.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.337+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/537620012 shutdown_connections 2023-07-18T03:32:28.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.337+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/537620012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb87203c70 0x55cb872135c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.337+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/537620012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb872024d0 0x55cb8720c040 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.337+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/537620012 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55cb87217c90 0x55cb8721a0a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.337+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/537620012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb87202e20 0x55cb8720c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.337+0000 7f6f9e7fc700 0 client.15942 ms_handle_reset on v2:172.21.15.64:6836/3077792296 2023-07-18T03:32:28.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.337+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/537620012 >> 172.21.15.40:0/537620012 conn(0x55cb871f7da0 msgr2=0x55cb871fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.337+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/537620012 shutdown_connections 2023-07-18T03:32:28.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.337+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/537620012 wait complete. 2023-07-18T03:32:28.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.337+0000 7f6fc98c6b80 20 client.15942 trim_cache size 0 max 16384 2023-07-18T03:32:28.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.350+0000 7f6fc98c6b80 1 Processor -- start 2023-07-18T03:32:28.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.350+0000 7f6fc98c6b80 1 -- start start 2023-07-18T03:32:28.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.350+0000 7f6fc98c6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb872024d0 0x55cb8720b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.350+0000 7f6fc98c6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb87202e20 0x55cb87213580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.350+0000 7f6fc98c6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb87203c70 0x55cb8720adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.350+0000 7f6fc98c6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb871fea00 con 0x55cb87203c70 2023-07-18T03:32:28.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.350+0000 7f6fc98c6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb87209d30 con 0x55cb87202e20 2023-07-18T03:32:28.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.350+0000 7f6fc98c6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb87217960 con 0x55cb872024d0 2023-07-18T03:32:28.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.350+0000 7f6fb77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb87203c70 0x55cb8720adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.350+0000 7f6fb6ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb872024d0 0x55cb8720b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.350+0000 7f6fb6ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb872024d0 0x55cb8720b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48162/0 (socket says 172.21.15.40:48162) 2023-07-18T03:32:28.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.350+0000 7f6fb77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb87203c70 0x55cb8720adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59138/0 (socket says 172.21.15.40:59138) 2023-07-18T03:32:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.350+0000 7f6fb6ffd700 1 -- 172.21.15.40:0/2229023675 learned_addr learned my addr 172.21.15.40:0/2229023675 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.350+0000 7f6fb67fc700 1 --2- 172.21.15.40:0/2229023675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb87202e20 0x55cb87213580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.351+0000 7f6fb6ffd700 1 -- 172.21.15.40:0/2229023675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb87202e20 msgr2=0x55cb87213580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.351+0000 7f6fb6ffd700 1 --2- 172.21.15.40:0/2229023675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb87202e20 0x55cb87213580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.351+0000 7f6fb6ffd700 1 -- 172.21.15.40:0/2229023675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb87203c70 msgr2=0x55cb8720adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.351+0000 7f6fb6ffd700 1 --2- 172.21.15.40:0/2229023675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb87203c70 0x55cb8720adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.351+0000 7f6fb6ffd700 1 -- 172.21.15.40:0/2229023675 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb86fa8250 con 0x55cb872024d0 2023-07-18T03:32:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.351+0000 7f6fb6ffd700 1 --2- 172.21.15.40:0/2229023675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb872024d0 0x55cb8720b6f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0010fd0 tx=0x7f6fa0098080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ea78b99b8dfa09e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.351+0000 7f6fadffb700 1 -- 172.21.15.40:0/2229023675 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0098a30 con 0x55cb872024d0 2023-07-18T03:32:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.351+0000 7f6fadffb700 1 -- 172.21.15.40:0/2229023675 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fa0098b90 con 0x55cb872024d0 2023-07-18T03:32:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.351+0000 7f6fadffb700 1 -- 172.21.15.40:0/2229023675 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0099f90 con 0x55cb872024d0 2023-07-18T03:32:28.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.352+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/2229023675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb872024d0 msgr2=0x55cb8720b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.352+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/2229023675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb872024d0 0x55cb8720b6f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0010fd0 tx=0x7f6fa0098080 comp rx=0 tx=0).stop 2023-07-18T03:32:28.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.352+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/2229023675 shutdown_connections 2023-07-18T03:32:28.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.352+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/2229023675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb87203c70 0x55cb8720adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.352+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/2229023675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb872024d0 0x55cb8720b6f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.352+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/2229023675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb87202e20 0x55cb87213580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.352+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/2229023675 >> 172.21.15.40:0/2229023675 conn(0x55cb871f7da0 msgr2=0x55cb8721e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.352+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/2229023675 shutdown_connections 2023-07-18T03:32:28.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.352+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/2229023675 wait complete. 2023-07-18T03:32:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.352+0000 7f6fc98c6b80 1 -- start start 2023-07-18T03:32:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.352+0000 7f6fc98c6b80 1 Processor -- start 2023-07-18T03:32:28.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.352+0000 7f6fc98c6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb872024d0 0x55cb8720f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.352+0000 7f6fc98c6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb87202e20 0x55cb8720f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.352+0000 7f6fc98c6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb87203c70 0x55cb87212a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.352+0000 7f6fc98c6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb87212f40 con 0x55cb872024d0 2023-07-18T03:32:28.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6fc98c6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb872130b0 con 0x55cb87202e20 2023-07-18T03:32:28.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6fc98c6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb87213270 con 0x55cb87203c70 2023-07-18T03:32:28.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6fc98c6b80 10 client.0 _ll_register_callbacks cb 0x7f6fa0099450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:28.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6fb6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb872024d0 0x55cb8720f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6fb6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb872024d0 0x55cb8720f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59140/0 (socket says 172.21.15.40:59140) 2023-07-18T03:32:28.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6fb77fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb87203c70 0x55cb87212a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6fb6ffd700 1 -- 172.21.15.40:0/3329152683 learned_addr learned my addr 172.21.15.40:0/3329152683 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6fb67fc700 1 --2- 172.21.15.40:0/3329152683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb87202e20 0x55cb8720f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6fb77fe700 1 -- 172.21.15.40:0/3329152683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb87202e20 msgr2=0x55cb8720f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6fb77fe700 1 --2- 172.21.15.40:0/3329152683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb87202e20 0x55cb8720f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6fb77fe700 1 -- 172.21.15.40:0/3329152683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb872024d0 msgr2=0x55cb8720f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6fb77fe700 1 --2- 172.21.15.40:0/3329152683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb872024d0 0x55cb8720f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6fb77fe700 1 -- 172.21.15.40:0/3329152683 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb86f6ac60 con 0x55cb87203c70 2023-07-18T03:32:28.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6fb77fe700 1 --2- 172.21.15.40:0/3329152683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb87203c70 0x55cb87212a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0009510 tx=0x7f6fb0018700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=233841f0f8020a17 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6f9e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:32:28.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6f9e7fc700 1 -- 172.21.15.40:0/3329152683 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb001d070 con 0x55cb87203c70 2023-07-18T03:32:28.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6f9e7fc700 1 -- 172.21.15.40:0/3329152683 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6fb0004bb0 con 0x55cb87203c70 2023-07-18T03:32:28.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/3329152683 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fa801e040 con 0x55cb87203c70 2023-07-18T03:32:28.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.353+0000 7f6f9e7fc700 1 -- 172.21.15.40:0/3329152683 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fb000e040 con 0x55cb87203c70 2023-07-18T03:32:28.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.354+0000 7f6fc98c6b80 10 client.25591 fetch_fsmap learned FSMap version 31 2023-07-18T03:32:28.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.354+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/3329152683 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55cb86f802b0 con 0x55cb87203c70 2023-07-18T03:32:28.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.354+0000 7f6f9e7fc700 1 -- 172.21.15.40:0/3329152683 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fa801e040 con 0x55cb87203c70 2023-07-18T03:32:28.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.354+0000 7f6f9e7fc700 1 -- 172.21.15.40:0/3329152683 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f6fb002b050 con 0x55cb87203c70 2023-07-18T03:32:28.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.354+0000 7f6fc98c6b80 10 client.25591 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:32:28.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.354+0000 7f6fc98c6b80 10 client.25591 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-18T03:32:28.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.354+0000 7f6fc98c6b80 20 client.25591 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:28.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.354+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/3329152683 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55cb8726cd50 0x55cb8726f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.354+0000 7f6fc98c6b80 4 client.25591 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:32:28.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.354+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/3329152683 --> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cb8710f050 con 0x55cb8726cd50 2023-07-18T03:32:28.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.354+0000 7f6fb6ffd700 1 --2- 172.21.15.40:0/3329152683 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55cb8726cd50 0x55cb8726f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.355+0000 7f6fb6ffd700 1 --2- 172.21.15.40:0/3329152683 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55cb8726cd50 0x55cb8726f160 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8daa7d62b17402d4 server_cookie=53e269d51b9bf336 in_seq=0 out_seq=0 2023-07-18T03:32:28.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.355+0000 7f6f9e7fc700 10 client.25591 ms_handle_connect on v2:172.21.15.64:6836/3077792296 2023-07-18T03:32:28.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.355+0000 7f6f9e7fc700 1 -- 172.21.15.40:0/3329152683 <== mds.2 v2:172.21.15.64:6836/3077792296 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cb8710f050 con 0x55cb8726cd50 2023-07-18T03:32:28.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.355+0000 7f6f9e7fc700 10 client.25591 handle_command_reply: tid=0 2023-07-18T03:32:28.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.355+0000 7f6fc98c6b80 1 client.25591 shutdown 2023-07-18T03:32:28.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.355+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/3329152683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb87203c70 msgr2=0x55cb87212a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.355+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/3329152683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb87203c70 0x55cb87212a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0009510 tx=0x7f6fb0018700 comp rx=0 tx=0).stop 2023-07-18T03:32:28.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.355+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/3329152683 shutdown_connections 2023-07-18T03:32:28.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.355+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/3329152683 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55cb8726cd50 0x55cb8726f160 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.355+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/3329152683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb87203c70 0x55cb87212a00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.355+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/3329152683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb872024d0 0x55cb8720f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.355+0000 7f6fc98c6b80 1 --2- 172.21.15.40:0/3329152683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb87202e20 0x55cb8720f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.355+0000 7f6f9e7fc700 0 client.25591 ms_handle_reset on v2:172.21.15.64:6836/3077792296 2023-07-18T03:32:28.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.355+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/3329152683 >> 172.21.15.40:0/3329152683 conn(0x55cb871f7da0 msgr2=0x55cb87209f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.355+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/3329152683 shutdown_connections 2023-07-18T03:32:28.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.355+0000 7f6fc98c6b80 1 -- 172.21.15.40:0/3329152683 wait complete. 2023-07-18T03:32:28.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.355+0000 7f6fc98c6b80 20 client.25591 trim_cache size 0 max 16384 2023-07-18T03:32:28.369 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:32:28.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.356+0000 7f6fbd740700 1 -- 172.21.15.40:0/1009180612 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6fa40a1150 msgr2=0x7f6fa40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.356+0000 7f6fbd740700 1 --2- 172.21.15.40:0/1009180612 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6fa40a1150 0x7f6fa40a3600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800b2e0 tx=0x7f6fa8005fb0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.356+0000 7f6fbd740700 1 -- 172.21.15.40:0/1009180612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fb8100560 msgr2=0x7f6fb80ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.356+0000 7f6fbd740700 1 --2- 172.21.15.40:0/1009180612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fb8100560 0x7f6fb80ffed0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000d950 tx=0x7f6fa000dc60 comp rx=0 tx=0).stop 2023-07-18T03:32:28.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.356+0000 7f6fbd740700 1 -- 172.21.15.40:0/1009180612 shutdown_connections 2023-07-18T03:32:28.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.356+0000 7f6fbd740700 1 --2- 172.21.15.40:0/1009180612 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6fa40a1150 0x7f6fa40a3600 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.356+0000 7f6fbd740700 1 --2- 172.21.15.40:0/1009180612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fb81057a0 0x7f6fb80fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.356+0000 7f6fbd740700 1 --2- 172.21.15.40:0/1009180612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6fb8100560 0x7f6fb80ffed0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.356+0000 7f6fbd740700 1 --2- 172.21.15.40:0/1009180612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb8102e80 0x7f6fb80fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.356+0000 7f6fbd740700 1 -- 172.21.15.40:0/1009180612 >> 172.21.15.40:0/1009180612 conn(0x7f6fb80fa000 msgr2=0x7f6fb81011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.356+0000 7f6fbd740700 1 -- 172.21.15.40:0/1009180612 shutdown_connections 2023-07-18T03:32:28.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.356+0000 7f6fbd740700 1 -- 172.21.15.40:0/1009180612 wait complete. 2023-07-18T03:32:28.372 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:32:28.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.517+0000 7f7ad2978700 1 -- 172.21.15.40:0/774386636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc102e80 msgr2=0x7f7acc105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.517+0000 7f7ad2978700 1 --2- 172.21.15.40:0/774386636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc102e80 0x7f7acc105260 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009990 tx=0x7f7ab4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.517+0000 7f7ad2978700 1 -- 172.21.15.40:0/774386636 shutdown_connections 2023-07-18T03:32:28.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.517+0000 7f7ad2978700 1 --2- 172.21.15.40:0/774386636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7acc1057a0 0x7f7acc107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.517+0000 7f7ad2978700 1 --2- 172.21.15.40:0/774386636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7acc100560 0x7f7acc102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.517+0000 7f7ad2978700 1 --2- 172.21.15.40:0/774386636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc102e80 0x7f7acc105260 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.517+0000 7f7ad2978700 1 -- 172.21.15.40:0/774386636 >> 172.21.15.40:0/774386636 conn(0x7f7acc0fa000 msgr2=0x7f7acc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.517+0000 7f7ad2978700 1 -- 172.21.15.40:0/774386636 shutdown_connections 2023-07-18T03:32:28.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.517+0000 7f7ad2978700 1 -- 172.21.15.40:0/774386636 wait complete. 2023-07-18T03:32:28.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.517+0000 7f7ad2978700 1 Processor -- start 2023-07-18T03:32:28.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7ad2978700 1 -- start start 2023-07-18T03:32:28.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7ad2978700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7acc100560 0x7f7acc195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7ad2978700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7acc102e80 0x7f7acc195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7ad2978700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc1057a0 0x7f7acc19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7ad2978700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc19d3b0 con 0x7f7acc1057a0 2023-07-18T03:32:28.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7ad2978700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc19d4f0 con 0x7f7acc100560 2023-07-18T03:32:28.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7ad2978700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7acc19d680 con 0x7f7acc102e80 2023-07-18T03:32:28.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7ad0f15700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc1057a0 0x7f7acc19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7ad0f15700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc1057a0 0x7f7acc19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59158/0 (socket says 172.21.15.40:59158) 2023-07-18T03:32:28.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7ad0f15700 1 -- 172.21.15.40:0/2286374279 learned_addr learned my addr 172.21.15.40:0/2286374279 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7acb7fe700 1 --2- 172.21.15.40:0/2286374279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7acc102e80 0x7f7acc195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7acbfff700 1 --2- 172.21.15.40:0/2286374279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7acc100560 0x7f7acc195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7ad0f15700 1 -- 172.21.15.40:0/2286374279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7acc102e80 msgr2=0x7f7acc195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7ad0f15700 1 --2- 172.21.15.40:0/2286374279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7acc102e80 0x7f7acc195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7ad0f15700 1 -- 172.21.15.40:0/2286374279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7acc100560 msgr2=0x7f7acc195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7ad0f15700 1 --2- 172.21.15.40:0/2286374279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7acc100560 0x7f7acc195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7ad0f15700 1 -- 172.21.15.40:0/2286374279 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab40095f0 con 0x7f7acc1057a0 2023-07-18T03:32:28.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.518+0000 7f7acb7fe700 1 --2- 172.21.15.40:0/2286374279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7acc102e80 0x7f7acc195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:28.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.519+0000 7f7ad0f15700 1 --2- 172.21.15.40:0/2286374279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc1057a0 0x7f7acc19cde0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400ef30 tx=0x7f7ac400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.519+0000 7f7ac97fa700 1 -- 172.21.15.40:0/2286374279 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4018070 con 0x7f7acc1057a0 2023-07-18T03:32:28.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.519+0000 7f7ac97fa700 1 -- 172.21.15.40:0/2286374279 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ac400f040 con 0x7f7acc1057a0 2023-07-18T03:32:28.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.519+0000 7f7ad2978700 1 -- 172.21.15.40:0/2286374279 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7acc19d900 con 0x7f7acc1057a0 2023-07-18T03:32:28.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.519+0000 7f7ac97fa700 1 -- 172.21.15.40:0/2286374279 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac400cdb0 con 0x7f7acc1057a0 2023-07-18T03:32:28.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.519+0000 7f7ad2978700 1 -- 172.21.15.40:0/2286374279 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7acc19de10 con 0x7f7acc1057a0 2023-07-18T03:32:28.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.521+0000 7f7ac97fa700 1 -- 172.21.15.40:0/2286374279 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7ac4003a00 con 0x7f7acc1057a0 2023-07-18T03:32:28.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.521+0000 7f7ac97fa700 1 --2- 172.21.15.40:0/2286374279 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7ab80a1150 0x7f7ab80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.522+0000 7f7ac97fa700 1 -- 172.21.15.40:0/2286374279 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7ac40cc4e0 con 0x7f7acc1057a0 2023-07-18T03:32:28.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.522+0000 7f7acbfff700 1 --2- 172.21.15.40:0/2286374279 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7ab80a1150 0x7f7ab80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.522+0000 7f7acbfff700 1 --2- 172.21.15.40:0/2286374279 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7ab80a1150 0x7f7ab80a3600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009bf0 tx=0x7f7abc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.522+0000 7f7adeafeb80 1 Processor -- start 2023-07-18T03:32:28.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.522+0000 7f7adeafeb80 1 -- start start 2023-07-18T03:32:28.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7adeafeb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e65974d0 0x5615e65978e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7adeafeb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6597e20 0x5615e65986e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7adeafeb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6598c70 0x5615e65990e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7adeafeb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e649ccb0 con 0x5615e6598c70 2023-07-18T03:32:28.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7adeafeb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e64efa40 con 0x5615e6597e20 2023-07-18T03:32:28.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7adeafeb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e64e8ed0 con 0x5615e65974d0 2023-07-18T03:32:28.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7ad0f15700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6598c70 0x5615e65990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7acbfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e65974d0 0x5615e65978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7ad0f15700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6598c70 0x5615e65990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59166/0 (socket says 172.21.15.40:59166) 2023-07-18T03:32:28.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7ad0f15700 1 -- 172.21.15.40:0/344166918 learned_addr learned my addr 172.21.15.40:0/344166918 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7acb7fe700 1 --2- 172.21.15.40:0/344166918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6597e20 0x5615e65986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7ad0f15700 1 -- 172.21.15.40:0/344166918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e65974d0 msgr2=0x5615e65978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7ad0f15700 1 --2- 172.21.15.40:0/344166918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e65974d0 0x5615e65978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7ad0f15700 1 -- 172.21.15.40:0/344166918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6597e20 msgr2=0x5615e65986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7ad0f15700 1 --2- 172.21.15.40:0/344166918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6597e20 0x5615e65986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7ad0f15700 1 -- 172.21.15.40:0/344166918 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615e6316de0 con 0x5615e6598c70 2023-07-18T03:32:28.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7acb7fe700 1 --2- 172.21.15.40:0/344166918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6597e20 0x5615e65986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:32:28.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.523+0000 7f7ad0f15700 1 --2- 172.21.15.40:0/344166918 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6598c70 0x5615e65990e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40a17f0 tx=0x7f7ac40092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd4fb3d7221c990 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.524+0000 7f7ac27fc700 1 -- 172.21.15.40:0/344166918 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4004ad0 con 0x5615e6598c70 2023-07-18T03:32:28.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.524+0000 7f7ac27fc700 1 -- 172.21.15.40:0/344166918 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ac4099960 con 0x5615e6598c70 2023-07-18T03:32:28.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.524+0000 7f7ac27fc700 1 -- 172.21.15.40:0/344166918 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac40990b0 con 0x5615e6598c70 2023-07-18T03:32:28.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.524+0000 7f7adeafeb80 1 -- 172.21.15.40:0/344166918 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6598c70 msgr2=0x5615e65990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.524+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/344166918 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6598c70 0x5615e65990e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40a17f0 tx=0x7f7ac40092e0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.524+0000 7f7adeafeb80 1 -- 172.21.15.40:0/344166918 shutdown_connections 2023-07-18T03:32:28.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.524+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/344166918 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6598c70 0x5615e65990e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.524+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/344166918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e65974d0 0x5615e65978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.524+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/344166918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6597e20 0x5615e65986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.524+0000 7f7adeafeb80 1 -- 172.21.15.40:0/344166918 >> 172.21.15.40:0/344166918 conn(0x5615e658cda0 msgr2=0x5615e65921d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.524+0000 7f7adeafeb80 1 -- 172.21.15.40:0/344166918 shutdown_connections 2023-07-18T03:32:28.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.524+0000 7f7adeafeb80 1 -- 172.21.15.40:0/344166918 wait complete. 2023-07-18T03:32:28.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.525+0000 7f7adeafeb80 1 -- start start 2023-07-18T03:32:28.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.525+0000 7f7adeafeb80 1 Processor -- start 2023-07-18T03:32:28.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.525+0000 7f7adeafeb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e65974d0 0x5615e65a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.525+0000 7f7adeafeb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6597e20 0x5615e65a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.525+0000 7f7adeafeb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6598c70 0x5615e65a8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.525+0000 7f7adeafeb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e6488ac0 con 0x5615e6597e20 2023-07-18T03:32:28.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.525+0000 7f7adeafeb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e64887c0 con 0x5615e6598c70 2023-07-18T03:32:28.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.525+0000 7f7adeafeb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e636c920 con 0x5615e65974d0 2023-07-18T03:32:28.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.525+0000 7f7adeafeb80 10 client.0 _ll_register_callbacks cb 0x5615e6486760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:28.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.525+0000 7f7acb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6597e20 0x5615e65a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.525+0000 7f7acbfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e65974d0 0x5615e65a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.525+0000 7f7acbfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e65974d0 0x5615e65a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48196/0 (socket says 172.21.15.40:48196) 2023-07-18T03:32:28.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.525+0000 7f7acb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6597e20 0x5615e65a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59170/0 (socket says 172.21.15.40:59170) 2023-07-18T03:32:28.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.525+0000 7f7acbfff700 1 -- 172.21.15.40:0/303637852 learned_addr learned my addr 172.21.15.40:0/303637852 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.526+0000 7f7ad0f15700 1 --2- 172.21.15.40:0/303637852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6598c70 0x5615e65a8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.526+0000 7f7acbfff700 1 -- 172.21.15.40:0/303637852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6598c70 msgr2=0x5615e65a8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.526+0000 7f7acbfff700 1 --2- 172.21.15.40:0/303637852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6598c70 0x5615e65a8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.526+0000 7f7acbfff700 1 -- 172.21.15.40:0/303637852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6597e20 msgr2=0x5615e65a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.526+0000 7f7acbfff700 1 --2- 172.21.15.40:0/303637852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6597e20 0x5615e65a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.526+0000 7f7acbfff700 1 -- 172.21.15.40:0/303637852 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7acc19de10 con 0x5615e65974d0 2023-07-18T03:32:28.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.526+0000 7f7acbfff700 1 --2- 172.21.15.40:0/303637852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e65974d0 0x5615e65a11a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7abc012910 tx=0x7f7abc0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64ffa6d404466520 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.526+0000 7f7aab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:32:28.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.526+0000 7f7aab7fe700 1 -- 172.21.15.40:0/303637852 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc020840 con 0x5615e65974d0 2023-07-18T03:32:28.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.526+0000 7f7adeafeb80 1 -- 172.21.15.40:0/303637852 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615e64424f0 con 0x5615e65974d0 2023-07-18T03:32:28.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.526+0000 7f7aab7fe700 1 -- 172.21.15.40:0/303637852 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7abc018070 con 0x5615e65974d0 2023-07-18T03:32:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.526+0000 7f7aab7fe700 1 -- 172.21.15.40:0/303637852 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7abc012e80 con 0x5615e65974d0 2023-07-18T03:32:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.526+0000 7f7adeafeb80 10 client.25603 fetch_fsmap learned FSMap version 31 2023-07-18T03:32:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.526+0000 7f7adeafeb80 1 -- 172.21.15.40:0/303637852 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5615e62ffc60 con 0x5615e65974d0 2023-07-18T03:32:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.527+0000 7f7aab7fe700 1 -- 172.21.15.40:0/303637852 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615e64424f0 con 0x5615e65974d0 2023-07-18T03:32:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.527+0000 7f7aab7fe700 1 -- 172.21.15.40:0/303637852 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7abc029bf0 con 0x5615e65974d0 2023-07-18T03:32:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.527+0000 7f7adeafeb80 10 client.25603 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:32:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.527+0000 7f7adeafeb80 10 client.25603 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-18T03:32:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.527+0000 7f7adeafeb80 20 client.25603 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:28.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.527+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/303637852 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x5615e65acf40 0x5615e65af350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.527+0000 7f7adeafeb80 4 client.25603 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:32:28.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.527+0000 7f7adeafeb80 1 -- 172.21.15.40:0/303637852 --> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615e64e1eb0 con 0x5615e65acf40 2023-07-18T03:32:28.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.527+0000 7f7acb7fe700 1 --2- 172.21.15.40:0/303637852 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x5615e65acf40 0x5615e65af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.527+0000 7f7aab7fe700 10 client.25603 ms_handle_connect on v2:172.21.15.134:6838/3192254101 2023-07-18T03:32:28.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.527+0000 7f7acb7fe700 1 --2- 172.21.15.40:0/303637852 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x5615e65acf40 0x5615e65af350 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75d790d5df6f20fb server_cookie=bdf24ae0e45d9811 in_seq=0 out_seq=0 2023-07-18T03:32:28.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.530+0000 7f7aab7fe700 1 -- 172.21.15.40:0/303637852 <== mds.1 v2:172.21.15.134:6838/3192254101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5615e64e1eb0 con 0x5615e65acf40 2023-07-18T03:32:28.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.530+0000 7f7aab7fe700 10 client.25603 handle_command_reply: tid=0 2023-07-18T03:32:28.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.530+0000 7f7adeafeb80 1 client.25603 shutdown 2023-07-18T03:32:28.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.531+0000 7f7adeafeb80 1 -- 172.21.15.40:0/303637852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e65974d0 msgr2=0x5615e65a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.531+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/303637852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e65974d0 0x5615e65a11a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7abc012910 tx=0x7f7abc0047c0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.531+0000 7f7adeafeb80 1 -- 172.21.15.40:0/303637852 shutdown_connections 2023-07-18T03:32:28.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.531+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/303637852 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x5615e65acf40 0x5615e65af350 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.531+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/303637852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6598c70 0x5615e65a8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.531+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/303637852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e65974d0 0x5615e65a11a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.531+0000 7f7aab7fe700 0 client.25603 ms_handle_reset on v2:172.21.15.134:6838/3192254101 2023-07-18T03:32:28.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.531+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/303637852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6597e20 0x5615e65a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.531+0000 7f7adeafeb80 1 -- 172.21.15.40:0/303637852 >> 172.21.15.40:0/303637852 conn(0x5615e658cda0 msgr2=0x5615e65918b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.531+0000 7f7adeafeb80 1 -- 172.21.15.40:0/303637852 shutdown_connections 2023-07-18T03:32:28.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.531+0000 7f7adeafeb80 1 -- 172.21.15.40:0/303637852 wait complete. 2023-07-18T03:32:28.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.531+0000 7f7adeafeb80 20 client.25603 trim_cache size 0 max 16384 2023-07-18T03:32:28.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.544+0000 7f7adeafeb80 1 Processor -- start 2023-07-18T03:32:28.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.544+0000 7f7adeafeb80 1 -- start start 2023-07-18T03:32:28.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.544+0000 7f7adeafeb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e65974d0 0x5615e659bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.544+0000 7f7adeafeb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6597e20 0x5615e659c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.544+0000 7f7adeafeb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6598c70 0x5615e65a86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.544+0000 7f7adeafeb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e65aa300 con 0x5615e6597e20 2023-07-18T03:32:28.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.544+0000 7f7adeafeb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e65aca50 con 0x5615e6598c70 2023-07-18T03:32:28.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.544+0000 7f7adeafeb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e65acc60 con 0x5615e65974d0 2023-07-18T03:32:28.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.544+0000 7f7acb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6597e20 0x5615e659c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.544+0000 7f7acb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6597e20 0x5615e659c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59178/0 (socket says 172.21.15.40:59178) 2023-07-18T03:32:28.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.544+0000 7f7acb7fe700 1 -- 172.21.15.40:0/1030485439 learned_addr learned my addr 172.21.15.40:0/1030485439 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.544+0000 7f7acbfff700 1 --2- 172.21.15.40:0/1030485439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e65974d0 0x5615e659bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.544+0000 7f7ad0f15700 1 --2- 172.21.15.40:0/1030485439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6598c70 0x5615e65a86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.545+0000 7f7acb7fe700 1 -- 172.21.15.40:0/1030485439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e65974d0 msgr2=0x5615e659bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.545+0000 7f7acb7fe700 1 --2- 172.21.15.40:0/1030485439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e65974d0 0x5615e659bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.545+0000 7f7acb7fe700 1 -- 172.21.15.40:0/1030485439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6598c70 msgr2=0x5615e65a86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.545+0000 7f7acb7fe700 1 --2- 172.21.15.40:0/1030485439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6598c70 0x5615e65a86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.545+0000 7f7acb7fe700 1 -- 172.21.15.40:0/1030485439 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615e633d250 con 0x5615e6597e20 2023-07-18T03:32:28.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.545+0000 7f7ad0f15700 1 --2- 172.21.15.40:0/1030485439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6598c70 0x5615e65a86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:32:28.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.545+0000 7f7acb7fe700 1 --2- 172.21.15.40:0/1030485439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6597e20 0x5615e659c0b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009990 tx=0x7f7ab4005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca385767493aa90 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.545+0000 7f7ac2ffd700 1 -- 172.21.15.40:0/1030485439 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab400bc00 con 0x5615e6597e20 2023-07-18T03:32:28.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.545+0000 7f7ac2ffd700 1 -- 172.21.15.40:0/1030485439 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab401e040 con 0x5615e6597e20 2023-07-18T03:32:28.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.545+0000 7f7ac2ffd700 1 -- 172.21.15.40:0/1030485439 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab40052a0 con 0x5615e6597e20 2023-07-18T03:32:28.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.546+0000 7f7adeafeb80 1 -- 172.21.15.40:0/1030485439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6597e20 msgr2=0x5615e659c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.546+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/1030485439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6597e20 0x5615e659c0b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4009990 tx=0x7f7ab4005fb0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.546+0000 7f7adeafeb80 1 -- 172.21.15.40:0/1030485439 shutdown_connections 2023-07-18T03:32:28.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.546+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/1030485439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6598c70 0x5615e65a86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.546+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/1030485439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e65974d0 0x5615e659bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.546+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/1030485439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e6597e20 0x5615e659c0b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.546+0000 7f7adeafeb80 1 -- 172.21.15.40:0/1030485439 >> 172.21.15.40:0/1030485439 conn(0x5615e658cda0 msgr2=0x5615e65a9010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.546+0000 7f7adeafeb80 1 -- 172.21.15.40:0/1030485439 shutdown_connections 2023-07-18T03:32:28.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.546+0000 7f7adeafeb80 1 -- 172.21.15.40:0/1030485439 wait complete. 2023-07-18T03:32:28.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.546+0000 7f7adeafeb80 1 -- start start 2023-07-18T03:32:28.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.546+0000 7f7adeafeb80 1 Processor -- start 2023-07-18T03:32:28.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7adeafeb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e65974d0 0x5615e65a5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7adeafeb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6597e20 0x5615e65a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7adeafeb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e6598c70 0x5615e6600db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7adeafeb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e65a6190 con 0x5615e65974d0 2023-07-18T03:32:28.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7adeafeb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e66012f0 con 0x5615e6597e20 2023-07-18T03:32:28.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7adeafeb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e6601480 con 0x5615e6598c70 2023-07-18T03:32:28.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7adeafeb80 10 client.0 _ll_register_callbacks cb 0x7f7abc0124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:28.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7acbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e65974d0 0x5615e65a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7acbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e65974d0 0x5615e65a5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59182/0 (socket says 172.21.15.40:59182) 2023-07-18T03:32:28.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7acbfff700 1 -- 172.21.15.40:0/2864199085 learned_addr learned my addr 172.21.15.40:0/2864199085 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7ad0f15700 1 --2- 172.21.15.40:0/2864199085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e6598c70 0x5615e6600db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7acb7fe700 1 --2- 172.21.15.40:0/2864199085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6597e20 0x5615e65a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7acbfff700 1 -- 172.21.15.40:0/2864199085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e6598c70 msgr2=0x5615e6600db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7acbfff700 1 --2- 172.21.15.40:0/2864199085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e6598c70 0x5615e6600db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7acbfff700 1 -- 172.21.15.40:0/2864199085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6597e20 msgr2=0x5615e65a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7acbfff700 1 --2- 172.21.15.40:0/2864199085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6597e20 0x5615e65a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7acbfff700 1 -- 172.21.15.40:0/2864199085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7acc1087f0 con 0x5615e65974d0 2023-07-18T03:32:28.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7acb7fe700 1 --2- 172.21.15.40:0/2864199085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6597e20 0x5615e65a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:32:28.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7acbfff700 1 --2- 172.21.15.40:0/2864199085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e65974d0 0x5615e65a5570 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7abc029f20 tx=0x7f7abc01f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1325cea9e77b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7aab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:32:28.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7aab7fe700 1 -- 172.21.15.40:0/2864199085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc01fe70 con 0x5615e65974d0 2023-07-18T03:32:28.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7aab7fe700 1 -- 172.21.15.40:0/2864199085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7abc018040 con 0x5615e65974d0 2023-07-18T03:32:28.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.547+0000 7f7adeafeb80 1 -- 172.21.15.40:0/2864199085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ac4099960 con 0x5615e65974d0 2023-07-18T03:32:28.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.548+0000 7f7aab7fe700 1 -- 172.21.15.40:0/2864199085 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7abc004020 con 0x5615e65974d0 2023-07-18T03:32:28.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.548+0000 7f7adeafeb80 10 client.15966 fetch_fsmap learned FSMap version 31 2023-07-18T03:32:28.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.548+0000 7f7adeafeb80 1 -- 172.21.15.40:0/2864199085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5615e63152b0 con 0x5615e65974d0 2023-07-18T03:32:28.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.548+0000 7f7aab7fe700 1 -- 172.21.15.40:0/2864199085 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ac4099960 con 0x5615e65974d0 2023-07-18T03:32:28.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.548+0000 7f7aab7fe700 1 -- 172.21.15.40:0/2864199085 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7abc044020 con 0x5615e65974d0 2023-07-18T03:32:28.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.548+0000 7f7adeafeb80 10 client.15966 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:32:28.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.548+0000 7f7adeafeb80 10 client.15966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-18T03:32:28.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.548+0000 7f7adeafeb80 20 client.15966 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:28.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.548+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/2864199085 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x5615e6602f30 0x5615e6605340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.548+0000 7f7adeafeb80 4 client.15966 mds_command: new command op to 24523 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:32:28.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.548+0000 7f7adeafeb80 1 -- 172.21.15.40:0/2864199085 --> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5615e64a4050 con 0x5615e6602f30 2023-07-18T03:32:28.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.548+0000 7f7acb7fe700 1 --2- 172.21.15.40:0/2864199085 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x5615e6602f30 0x5615e6605340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.549+0000 7f7aab7fe700 10 client.15966 ms_handle_connect on v2:172.21.15.134:6838/3192254101 2023-07-18T03:32:28.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.549+0000 7f7acb7fe700 1 --2- 172.21.15.40:0/2864199085 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x5615e6602f30 0x5615e6605340 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c64adf81a2073bd2 server_cookie=f79b3a1f01a07a14 in_seq=0 out_seq=0 2023-07-18T03:32:28.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.550+0000 7f7aab7fe700 1 -- 172.21.15.40:0/2864199085 <== mds.1 v2:172.21.15.134:6838/3192254101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5615e64a4050 con 0x5615e6602f30 2023-07-18T03:32:28.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.550+0000 7f7aab7fe700 10 client.15966 handle_command_reply: tid=0 2023-07-18T03:32:28.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.550+0000 7f7adeafeb80 1 client.15966 shutdown 2023-07-18T03:32:28.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7adeafeb80 1 -- 172.21.15.40:0/2864199085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e65974d0 msgr2=0x5615e65a5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/2864199085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e65974d0 0x5615e65a5570 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7abc029f20 tx=0x7f7abc01f820 comp rx=0 tx=0).stop 2023-07-18T03:32:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7adeafeb80 1 -- 172.21.15.40:0/2864199085 shutdown_connections 2023-07-18T03:32:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/2864199085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615e6598c70 0x5615e6600db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/2864199085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615e65974d0 0x5615e65a5570 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/2864199085 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x5615e6602f30 0x5615e6605340 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7adeafeb80 1 --2- 172.21.15.40:0/2864199085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e6597e20 0x5615e65a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7adeafeb80 1 -- 172.21.15.40:0/2864199085 >> 172.21.15.40:0/2864199085 conn(0x5615e658cda0 msgr2=0x5615e659bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7aab7fe700 0 client.15966 ms_handle_reset on v2:172.21.15.134:6838/3192254101 2023-07-18T03:32:28.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7adeafeb80 1 -- 172.21.15.40:0/2864199085 shutdown_connections 2023-07-18T03:32:28.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7adeafeb80 1 -- 172.21.15.40:0/2864199085 wait complete. 2023-07-18T03:32:28.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7adeafeb80 20 client.15966 trim_cache size 0 max 16384 2023-07-18T03:32:28.561 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:32:28.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7ad2978700 1 -- 172.21.15.40:0/2286374279 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7ab80a1150 msgr2=0x7f7ab80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7ad2978700 1 --2- 172.21.15.40:0/2286374279 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7ab80a1150 0x7f7ab80a3600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009bf0 tx=0x7f7abc009260 comp rx=0 tx=0).stop 2023-07-18T03:32:28.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7ad2978700 1 -- 172.21.15.40:0/2286374279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc1057a0 msgr2=0x7f7acc19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7ad2978700 1 --2- 172.21.15.40:0/2286374279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc1057a0 0x7f7acc19cde0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400ef30 tx=0x7f7ac400c3a0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7ad2978700 1 -- 172.21.15.40:0/2286374279 shutdown_connections 2023-07-18T03:32:28.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7ad2978700 1 --2- 172.21.15.40:0/2286374279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7acc1057a0 0x7f7acc19cde0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7ad2978700 1 --2- 172.21.15.40:0/2286374279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7acc100560 0x7f7acc195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7ad2978700 1 --2- 172.21.15.40:0/2286374279 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7ab80a1150 0x7f7ab80a3600 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7ad2978700 1 --2- 172.21.15.40:0/2286374279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7acc102e80 0x7f7acc195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.551+0000 7f7ad2978700 1 -- 172.21.15.40:0/2286374279 >> 172.21.15.40:0/2286374279 conn(0x7f7acc0fa000 msgr2=0x7f7acc1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.552+0000 7f7ad2978700 1 -- 172.21.15.40:0/2286374279 shutdown_connections 2023-07-18T03:32:28.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.552+0000 7f7ad2978700 1 -- 172.21.15.40:0/2286374279 wait complete. 2023-07-18T03:32:28.567 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:32:28.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.704+0000 7f0e908d8700 1 -- 172.21.15.40:0/3145131676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e8c100cd0 msgr2=0x7f0e8c101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.704+0000 7f0e908d8700 1 --2- 172.21.15.40:0/3145131676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e8c100cd0 0x7f0e8c101120 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009990 tx=0x7f0e74009ca0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.704+0000 7f0e908d8700 1 -- 172.21.15.40:0/3145131676 shutdown_connections 2023-07-18T03:32:28.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.704+0000 7f0e908d8700 1 --2- 172.21.15.40:0/3145131676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e8c101660 0x7f0e8c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.704+0000 7f0e908d8700 1 --2- 172.21.15.40:0/3145131676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e8c0ffae0 0x7f0e8c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.704+0000 7f0e908d8700 1 --2- 172.21.15.40:0/3145131676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e8c100cd0 0x7f0e8c101120 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.704+0000 7f0e908d8700 1 -- 172.21.15.40:0/3145131676 >> 172.21.15.40:0/3145131676 conn(0x7f0e8c0fb120 msgr2=0x7f0e8c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.704+0000 7f0e908d8700 1 -- 172.21.15.40:0/3145131676 shutdown_connections 2023-07-18T03:32:28.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.704+0000 7f0e908d8700 1 -- 172.21.15.40:0/3145131676 wait complete. 2023-07-18T03:32:28.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e908d8700 1 Processor -- start 2023-07-18T03:32:28.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e908d8700 1 -- start start 2023-07-18T03:32:28.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e908d8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e8c0ffae0 0x7f0e8c195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e908d8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e8c100cd0 0x7f0e8c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e908d8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e8c101660 0x7f0e8c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e908d8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c19d3a0 con 0x7f0e8c100cd0 2023-07-18T03:32:28.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e908d8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c19d4e0 con 0x7f0e8c101660 2023-07-18T03:32:28.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e908d8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c19d670 con 0x7f0e8c0ffae0 2023-07-18T03:32:28.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e89d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e8c100cd0 0x7f0e8c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e89d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e8c100cd0 0x7f0e8c195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59198/0 (socket says 172.21.15.40:59198) 2023-07-18T03:32:28.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e89d9b700 1 -- 172.21.15.40:0/2267109273 learned_addr learned my addr 172.21.15.40:0/2267109273 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e8a59c700 1 --2- 172.21.15.40:0/2267109273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e8c0ffae0 0x7f0e8c195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e8ad9d700 1 --2- 172.21.15.40:0/2267109273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e8c101660 0x7f0e8c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e89d9b700 1 -- 172.21.15.40:0/2267109273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e8c0ffae0 msgr2=0x7f0e8c195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e89d9b700 1 --2- 172.21.15.40:0/2267109273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e8c0ffae0 0x7f0e8c195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e89d9b700 1 -- 172.21.15.40:0/2267109273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e8c101660 msgr2=0x7f0e8c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e89d9b700 1 --2- 172.21.15.40:0/2267109273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e8c101660 0x7f0e8c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e89d9b700 1 -- 172.21.15.40:0/2267109273 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e740095f0 con 0x7f0e8c100cd0 2023-07-18T03:32:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.705+0000 7f0e8a59c700 1 --2- 172.21.15.40:0/2267109273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e8c0ffae0 0x7f0e8c195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.706+0000 7f0e89d9b700 1 --2- 172.21.15.40:0/2267109273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e8c100cd0 0x7f0e8c195d60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400b620 tx=0x7f0e74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.706+0000 7f0e837fe700 1 -- 172.21.15.40:0/2267109273 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7401d070 con 0x7f0e8c100cd0 2023-07-18T03:32:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.706+0000 7f0e837fe700 1 -- 172.21.15.40:0/2267109273 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e74005030 con 0x7f0e8c100cd0 2023-07-18T03:32:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.706+0000 7f0e908d8700 1 -- 172.21.15.40:0/2267109273 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8c19d8f0 con 0x7f0e8c100cd0 2023-07-18T03:32:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.706+0000 7f0e837fe700 1 -- 172.21.15.40:0/2267109273 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74004b80 con 0x7f0e8c100cd0 2023-07-18T03:32:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.706+0000 7f0e908d8700 1 -- 172.21.15.40:0/2267109273 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8c19dd10 con 0x7f0e8c100cd0 2023-07-18T03:32:28.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.708+0000 7f0e837fe700 1 -- 172.21.15.40:0/2267109273 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0e740051a0 con 0x7f0e8c100cd0 2023-07-18T03:32:28.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.709+0000 7f0e837fe700 1 --2- 172.21.15.40:0/2267109273 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0e780a1150 0x7f0e780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.709+0000 7f0e837fe700 1 -- 172.21.15.40:0/2267109273 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0e740cd930 con 0x7f0e8c100cd0 2023-07-18T03:32:28.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.709+0000 7f0e8a59c700 1 --2- 172.21.15.40:0/2267109273 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0e780a1150 0x7f0e780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.709+0000 7f0e8a59c700 1 --2- 172.21.15.40:0/2267109273 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0e780a1150 0x7f0e780a3600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c100b30 tx=0x7f0e7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.710+0000 7f0e9ca5eb80 1 Processor -- start 2023-07-18T03:32:28.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.710+0000 7f0e9ca5eb80 1 -- start start 2023-07-18T03:32:28.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.710+0000 7f0e9ca5eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd14d0 0x56018edd18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.710+0000 7f0e9ca5eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd1e20 0x56018edd26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.710+0000 7f0e9ca5eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd2c70 0x56018edd30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.710+0000 7f0e9ca5eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018ecd6cb0 con 0x56018edd14d0 2023-07-18T03:32:28.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.710+0000 7f0e9ca5eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018ed29a40 con 0x56018edd1e20 2023-07-18T03:32:28.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.710+0000 7f0e9ca5eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018ed22ed0 con 0x56018edd2c70 2023-07-18T03:32:28.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.710+0000 7f0e8a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd14d0 0x56018edd18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.710+0000 7f0e8a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd14d0 0x56018edd18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59204/0 (socket says 172.21.15.40:59204) 2023-07-18T03:32:28.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.710+0000 7f0e8a59c700 1 -- 172.21.15.40:0/3493480743 learned_addr learned my addr 172.21.15.40:0/3493480743 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.710+0000 7f0e8ad9d700 1 --2- 172.21.15.40:0/3493480743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd2c70 0x56018edd30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.710+0000 7f0e89d9b700 1 --2- 172.21.15.40:0/3493480743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd1e20 0x56018edd26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.711+0000 7f0e8ad9d700 1 -- 172.21.15.40:0/3493480743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd1e20 msgr2=0x56018edd26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.711+0000 7f0e8ad9d700 1 --2- 172.21.15.40:0/3493480743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd1e20 0x56018edd26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.711+0000 7f0e8ad9d700 1 -- 172.21.15.40:0/3493480743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd14d0 msgr2=0x56018edd18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.711+0000 7f0e8ad9d700 1 --2- 172.21.15.40:0/3493480743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd14d0 0x56018edd18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.711+0000 7f0e8ad9d700 1 -- 172.21.15.40:0/3493480743 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56018eb50de0 con 0x56018edd2c70 2023-07-18T03:32:28.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.711+0000 7f0e8a59c700 1 --2- 172.21.15.40:0/3493480743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd14d0 0x56018edd18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:28.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.711+0000 7f0e8ad9d700 1 --2- 172.21.15.40:0/3493480743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd2c70 0x56018edd30e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0e84011580 tx=0x7f0e84009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5a69da8c4db5f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.711+0000 7f0e80ff9700 1 -- 172.21.15.40:0/3493480743 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8400e040 con 0x56018edd2c70 2023-07-18T03:32:28.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.711+0000 7f0e80ff9700 1 -- 172.21.15.40:0/3493480743 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e8400f040 con 0x56018edd2c70 2023-07-18T03:32:28.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.711+0000 7f0e80ff9700 1 -- 172.21.15.40:0/3493480743 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e84004390 con 0x56018edd2c70 2023-07-18T03:32:28.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.712+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3493480743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd2c70 msgr2=0x56018edd30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.712+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/3493480743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd2c70 0x56018edd30e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0e84011580 tx=0x7f0e84009920 comp rx=0 tx=0).stop 2023-07-18T03:32:28.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.712+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3493480743 shutdown_connections 2023-07-18T03:32:28.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.712+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/3493480743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd2c70 0x56018edd30e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.712+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/3493480743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd14d0 0x56018edd18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.712+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/3493480743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd1e20 0x56018edd26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.712+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3493480743 >> 172.21.15.40:0/3493480743 conn(0x56018edc6da0 msgr2=0x56018edcc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.712+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3493480743 shutdown_connections 2023-07-18T03:32:28.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.712+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3493480743 wait complete. 2023-07-18T03:32:28.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.712+0000 7f0e9ca5eb80 1 -- start start 2023-07-18T03:32:28.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.712+0000 7f0e9ca5eb80 1 Processor -- start 2023-07-18T03:32:28.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e9ca5eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd14d0 0x56018eddb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e9ca5eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd1e20 0x56018eddb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e9ca5eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd2c70 0x56018ede26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e9ca5eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018ecc2ac0 con 0x56018edd14d0 2023-07-18T03:32:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e9ca5eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018ecc27c0 con 0x56018edd1e20 2023-07-18T03:32:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e9ca5eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018eba6920 con 0x56018edd2c70 2023-07-18T03:32:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e9ca5eb80 10 client.0 _ll_register_callbacks cb 0x56018ecc0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e8a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd14d0 0x56018eddb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e8a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd14d0 0x56018eddb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59210/0 (socket says 172.21.15.40:59210) 2023-07-18T03:32:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e8a59c700 1 -- 172.21.15.40:0/230686594 learned_addr learned my addr 172.21.15.40:0/230686594 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e8ad9d700 1 --2- 172.21.15.40:0/230686594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd2c70 0x56018ede26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e89d9b700 1 --2- 172.21.15.40:0/230686594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd1e20 0x56018eddb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e8a59c700 1 -- 172.21.15.40:0/230686594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd2c70 msgr2=0x56018ede26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e8a59c700 1 --2- 172.21.15.40:0/230686594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd2c70 0x56018ede26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e8a59c700 1 -- 172.21.15.40:0/230686594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd1e20 msgr2=0x56018eddb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e8a59c700 1 --2- 172.21.15.40:0/230686594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd1e20 0x56018eddb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e8a59c700 1 -- 172.21.15.40:0/230686594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7c017070 con 0x56018edd14d0 2023-07-18T03:32:28.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e8a59c700 1 --2- 172.21.15.40:0/230686594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd14d0 0x56018eddb130 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c003aa0 tx=0x7f0e7c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfa6eba870fddf9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:32:28.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.713+0000 7f0e71ffb700 1 -- 172.21.15.40:0/230686594 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c01f8c0 con 0x56018edd14d0 2023-07-18T03:32:28.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.714+0000 7f0e71ffb700 1 -- 172.21.15.40:0/230686594 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0e7c01fa20 con 0x56018edd14d0 2023-07-18T03:32:28.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.714+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/230686594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56018ec7c4f0 con 0x56018edd14d0 2023-07-18T03:32:28.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.714+0000 7f0e71ffb700 1 -- 172.21.15.40:0/230686594 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e7c01fbe0 con 0x56018edd14d0 2023-07-18T03:32:28.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.714+0000 7f0e9ca5eb80 10 client.15978 fetch_fsmap learned FSMap version 31 2023-07-18T03:32:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.714+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/230686594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f0e84011230 con 0x56018edd14d0 2023-07-18T03:32:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.714+0000 7f0e71ffb700 1 -- 172.21.15.40:0/230686594 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56018ec7c4f0 con 0x56018edd14d0 2023-07-18T03:32:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.714+0000 7f0e71ffb700 1 -- 172.21.15.40:0/230686594 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f0e7c031050 con 0x56018edd14d0 2023-07-18T03:32:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.714+0000 7f0e9ca5eb80 10 client.15978 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:32:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.714+0000 7f0e9ca5eb80 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:32:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.714+0000 7f0e9ca5eb80 20 client.15978 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.714+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/230686594 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56018ede6f00 0x56018ede9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.714+0000 7f0e9ca5eb80 4 client.15978 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:32:28.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.714+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/230686594 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56018ed1beb0 con 0x56018ede6f00 2023-07-18T03:32:28.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.714+0000 7f0e89d9b700 1 --2- 172.21.15.40:0/230686594 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56018ede6f00 0x56018ede9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.715+0000 7f0e71ffb700 10 client.15978 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:28.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.715+0000 7f0e89d9b700 1 --2- 172.21.15.40:0/230686594 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56018ede6f00 0x56018ede9310 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba1f253cbd11c6bd server_cookie=d1867fbee443119f in_seq=0 out_seq=0 2023-07-18T03:32:28.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.715+0000 7f0e71ffb700 1 -- 172.21.15.40:0/230686594 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56018ed1beb0 con 0x56018ede6f00 2023-07-18T03:32:28.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.715+0000 7f0e71ffb700 10 client.15978 handle_command_reply: tid=0 2023-07-18T03:32:28.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.715+0000 7f0e9ca5eb80 1 client.15978 shutdown 2023-07-18T03:32:28.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.716+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/230686594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd14d0 msgr2=0x56018eddb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.716+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/230686594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd14d0 0x56018eddb130 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c003aa0 tx=0x7f0e7c0047c0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.716+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/230686594 shutdown_connections 2023-07-18T03:32:28.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.716+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/230686594 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56018ede6f00 0x56018ede9310 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.716+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/230686594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd2c70 0x56018ede26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.716+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/230686594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd14d0 0x56018eddb130 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.716+0000 7f0e71ffb700 0 client.15978 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:28.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.716+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/230686594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd1e20 0x56018eddb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.716+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/230686594 >> 172.21.15.40:0/230686594 conn(0x56018edc6da0 msgr2=0x56018edcb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.716+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/230686594 shutdown_connections 2023-07-18T03:32:28.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.716+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/230686594 wait complete. 2023-07-18T03:32:28.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.716+0000 7f0e9ca5eb80 20 client.15978 trim_cache size 0 max 16384 2023-07-18T03:32:28.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e9ca5eb80 1 Processor -- start 2023-07-18T03:32:28.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e9ca5eb80 1 -- start start 2023-07-18T03:32:28.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e9ca5eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd14d0 0x56018edda7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e9ca5eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd1e20 0x56018ede2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e9ca5eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd2c70 0x56018ededc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e9ca5eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018edd9bb0 con 0x56018edd2c70 2023-07-18T03:32:28.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e9ca5eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018edd8df0 con 0x56018edd14d0 2023-07-18T03:32:28.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e9ca5eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018edee1b0 con 0x56018edd1e20 2023-07-18T03:32:28.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e8ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd2c70 0x56018ededc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e8ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd2c70 0x56018ededc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59222/0 (socket says 172.21.15.40:59222) 2023-07-18T03:32:28.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e8ad9d700 1 -- 172.21.15.40:0/3022407597 learned_addr learned my addr 172.21.15.40:0/3022407597 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e8a59c700 1 --2- 172.21.15.40:0/3022407597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd14d0 0x56018edda7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e89d9b700 1 --2- 172.21.15.40:0/3022407597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd1e20 0x56018ede2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e8ad9d700 1 -- 172.21.15.40:0/3022407597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd1e20 msgr2=0x56018ede2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e8ad9d700 1 --2- 172.21.15.40:0/3022407597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd1e20 0x56018ede2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e8ad9d700 1 -- 172.21.15.40:0/3022407597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd14d0 msgr2=0x56018edda7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e8ad9d700 1 --2- 172.21.15.40:0/3022407597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd14d0 0x56018edda7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.729+0000 7f0e8ad9d700 1 -- 172.21.15.40:0/3022407597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56018eb39c60 con 0x56018edd2c70 2023-07-18T03:32:28.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.730+0000 7f0e8ad9d700 1 --2- 172.21.15.40:0/3022407597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd2c70 0x56018ededc70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009b30 tx=0x7f0e84011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d969c1cbf6c0d878 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.730+0000 7f0e817fa700 1 -- 172.21.15.40:0/3022407597 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8400e040 con 0x56018edd2c70 2023-07-18T03:32:28.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.730+0000 7f0e817fa700 1 -- 172.21.15.40:0/3022407597 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e840093e0 con 0x56018edd2c70 2023-07-18T03:32:28.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.730+0000 7f0e817fa700 1 -- 172.21.15.40:0/3022407597 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8401d440 con 0x56018edd2c70 2023-07-18T03:32:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3022407597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd2c70 msgr2=0x56018ededc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/3022407597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd2c70 0x56018ededc70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009b30 tx=0x7f0e84011f30 comp rx=0 tx=0).stop 2023-07-18T03:32:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3022407597 shutdown_connections 2023-07-18T03:32:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/3022407597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd2c70 0x56018ededc70 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/3022407597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd14d0 0x56018edda7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/3022407597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd1e20 0x56018ede2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3022407597 >> 172.21.15.40:0/3022407597 conn(0x56018edc6da0 msgr2=0x56018edcc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3022407597 shutdown_connections 2023-07-18T03:32:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3022407597 wait complete. 2023-07-18T03:32:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 1 -- start start 2023-07-18T03:32:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 1 Processor -- start 2023-07-18T03:32:28.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd14d0 0x56018edde850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd1e20 0x56018edded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd2c70 0x56018ede1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018ede22f0 con 0x56018edd2c70 2023-07-18T03:32:28.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018ede2460 con 0x56018edd1e20 2023-07-18T03:32:28.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56018ede2620 con 0x56018edd14d0 2023-07-18T03:32:28.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e9ca5eb80 10 client.0 _ll_register_callbacks cb 0x7f0e7c003680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e8ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd2c70 0x56018ede1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e8ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd2c70 0x56018ede1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59228/0 (socket says 172.21.15.40:59228) 2023-07-18T03:32:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e8a59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd14d0 0x56018edde850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.731+0000 7f0e8ad9d700 1 -- 172.21.15.40:0/3917531149 learned_addr learned my addr 172.21.15.40:0/3917531149 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.732+0000 7f0e89d9b700 1 --2- 172.21.15.40:0/3917531149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd1e20 0x56018edded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.732+0000 7f0e8ad9d700 1 -- 172.21.15.40:0/3917531149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd14d0 msgr2=0x56018edde850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.732+0000 7f0e8ad9d700 1 --2- 172.21.15.40:0/3917531149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd14d0 0x56018edde850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.732+0000 7f0e8ad9d700 1 -- 172.21.15.40:0/3917531149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd1e20 msgr2=0x56018edded90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.732+0000 7f0e8ad9d700 1 --2- 172.21.15.40:0/3917531149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd1e20 0x56018edded90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.732+0000 7f0e8ad9d700 1 -- 172.21.15.40:0/3917531149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56018eb50de0 con 0x56018edd2c70 2023-07-18T03:32:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.732+0000 7f0e8ad9d700 1 --2- 172.21.15.40:0/3917531149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd2c70 0x56018ede1db0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009550 tx=0x7f0e84003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402c2c3ca7690769 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.732+0000 7f0e71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:32:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.732+0000 7f0e71ffb700 1 -- 172.21.15.40:0/3917531149 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8402d070 con 0x56018edd2c70 2023-07-18T03:32:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.732+0000 7f0e71ffb700 1 -- 172.21.15.40:0/3917531149 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0e840077d0 con 0x56018edd2c70 2023-07-18T03:32:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.732+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3917531149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e8400f040 con 0x56018edd2c70 2023-07-18T03:32:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.732+0000 7f0e71ffb700 1 -- 172.21.15.40:0/3917531149 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e84007990 con 0x56018edd2c70 2023-07-18T03:32:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.732+0000 7f0e9ca5eb80 10 client.15984 fetch_fsmap learned FSMap version 31 2023-07-18T03:32:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.732+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3917531149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56018eb77250 con 0x56018edd2c70 2023-07-18T03:32:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.732+0000 7f0e71ffb700 1 -- 172.21.15.40:0/3917531149 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e84037080 con 0x56018edd2c70 2023-07-18T03:32:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.732+0000 7f0e71ffb700 1 -- 172.21.15.40:0/3917531149 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f0e84030030 con 0x56018edd2c70 2023-07-18T03:32:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.733+0000 7f0e9ca5eb80 10 client.15984 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:32:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.733+0000 7f0e9ca5eb80 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:32:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.733+0000 7f0e9ca5eb80 20 client.15984 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.733+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/3917531149 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56018ee3bcb0 0x56018ee3e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.733+0000 7f0e9ca5eb80 4 client.15984 mds_command: new command op to 24527 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:32:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.733+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3917531149 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56018ecde050 con 0x56018ee3bcb0 2023-07-18T03:32:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.733+0000 7f0e8a59c700 1 --2- 172.21.15.40:0/3917531149 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56018ee3bcb0 0x56018ee3e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.733+0000 7f0e71ffb700 10 client.15984 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.733+0000 7f0e8a59c700 1 --2- 172.21.15.40:0/3917531149 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56018ee3bcb0 0x56018ee3e0c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e35745095f9730e server_cookie=ac47862d6658cf8f in_seq=0 out_seq=0 2023-07-18T03:32:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.733+0000 7f0e71ffb700 1 -- 172.21.15.40:0/3917531149 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56018ecde050 con 0x56018ee3bcb0 2023-07-18T03:32:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.733+0000 7f0e71ffb700 10 client.15984 handle_command_reply: tid=0 2023-07-18T03:32:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.733+0000 7f0e9ca5eb80 1 client.15984 shutdown 2023-07-18T03:32:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.733+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3917531149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd2c70 msgr2=0x56018ede1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.733+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/3917531149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd2c70 0x56018ede1db0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009550 tx=0x7f0e84003d70 comp rx=0 tx=0).stop 2023-07-18T03:32:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3917531149 shutdown_connections 2023-07-18T03:32:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/3917531149 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56018ee3bcb0 0x56018ee3e0c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/3917531149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56018edd2c70 0x56018ede1db0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e71ffb700 0 client.15984 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/3917531149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56018edd14d0 0x56018edde850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e9ca5eb80 1 --2- 172.21.15.40:0/3917531149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56018edd1e20 0x56018edded90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3917531149 >> 172.21.15.40:0/3917531149 conn(0x56018edc6da0 msgr2=0x56018edd9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3917531149 shutdown_connections 2023-07-18T03:32:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e9ca5eb80 1 -- 172.21.15.40:0/3917531149 wait complete. 2023-07-18T03:32:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e9ca5eb80 20 client.15984 trim_cache size 0 max 16384 2023-07-18T03:32:28.749 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:32:28.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e908d8700 1 -- 172.21.15.40:0/2267109273 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0e780a1150 msgr2=0x7f0e780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e908d8700 1 --2- 172.21.15.40:0/2267109273 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0e780a1150 0x7f0e780a3600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c100b30 tx=0x7f0e7c009260 comp rx=0 tx=0).stop 2023-07-18T03:32:28.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e908d8700 1 -- 172.21.15.40:0/2267109273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e8c100cd0 msgr2=0x7f0e8c195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:28.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e908d8700 1 --2- 172.21.15.40:0/2267109273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e8c100cd0 0x7f0e8c195d60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400b620 tx=0x7f0e74004730 comp rx=0 tx=0).stop 2023-07-18T03:32:28.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e908d8700 1 -- 172.21.15.40:0/2267109273 shutdown_connections 2023-07-18T03:32:28.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e908d8700 1 --2- 172.21.15.40:0/2267109273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e8c101660 0x7f0e8c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e908d8700 1 --2- 172.21.15.40:0/2267109273 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0e780a1150 0x7f0e780a3600 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e908d8700 1 --2- 172.21.15.40:0/2267109273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e8c0ffae0 0x7f0e8c195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e908d8700 1 --2- 172.21.15.40:0/2267109273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e8c100cd0 0x7f0e8c195d60 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:28.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e908d8700 1 -- 172.21.15.40:0/2267109273 >> 172.21.15.40:0/2267109273 conn(0x7f0e8c0fb120 msgr2=0x7f0e8c0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:28.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e908d8700 1 -- 172.21.15.40:0/2267109273 shutdown_connections 2023-07-18T03:32:28.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:28.734+0000 7f0e908d8700 1 -- 172.21.15.40:0/2267109273 wait complete. 2023-07-18T03:32:29.753 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-18T03:32:29.753 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-18T03:32:29.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.924+0000 7f5f62820700 1 -- 172.21.15.40:0/407249479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c1016f0 msgr2=0x7f5f5c107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:29.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.924+0000 7f5f62820700 1 --2- 172.21.15.40:0/407249479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c1016f0 0x7f5f5c107fb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400b0a0 tx=0x7f5f5400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.924+0000 7f5f62820700 1 -- 172.21.15.40:0/407249479 shutdown_connections 2023-07-18T03:32:29.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.924+0000 7f5f62820700 1 --2- 172.21.15.40:0/407249479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c1016f0 0x7f5f5c107fb0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.924+0000 7f5f62820700 1 --2- 172.21.15.40:0/407249479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f5c0fdd90 0x7f5f5c0fe1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.924+0000 7f5f62820700 1 --2- 172.21.15.40:0/407249479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f5c100d60 0x7f5f5c1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.924+0000 7f5f62820700 1 -- 172.21.15.40:0/407249479 >> 172.21.15.40:0/407249479 conn(0x7f5f5c0f9770 msgr2=0x7f5f5c0fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:29.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.924+0000 7f5f62820700 1 -- 172.21.15.40:0/407249479 shutdown_connections 2023-07-18T03:32:29.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.924+0000 7f5f62820700 1 -- 172.21.15.40:0/407249479 wait complete. 2023-07-18T03:32:29.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.925+0000 7f5f62820700 1 Processor -- start 2023-07-18T03:32:29.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.925+0000 7f5f62820700 1 -- start start 2023-07-18T03:32:29.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.925+0000 7f5f62820700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f5c0fdd90 0x7f5f5c1006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.925+0000 7f5f62820700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c100d60 0x7f5f5c0fed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.925+0000 7f5f62820700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f5c1016f0 0x7f5f5c0ff250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.925+0000 7f5f62820700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0ff9f0 con 0x7f5f5c100d60 2023-07-18T03:32:29.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.925+0000 7f5f62820700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0ffb60 con 0x7f5f5c0fdd90 2023-07-18T03:32:29.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.925+0000 7f5f62820700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c19f9e0 con 0x7f5f5c1016f0 2023-07-18T03:32:29.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.925+0000 7f5f5b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c100d60 0x7f5f5c0fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:29.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.925+0000 7f5f5b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c100d60 0x7f5f5c0fed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59240/0 (socket says 172.21.15.40:59240) 2023-07-18T03:32:29.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.925+0000 7f5f5b7fe700 1 -- 172.21.15.40:0/2657513589 learned_addr learned my addr 172.21.15.40:0/2657513589 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:29.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.925+0000 7f5f60dbd700 1 --2- 172.21.15.40:0/2657513589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f5c1016f0 0x7f5f5c0ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:29.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.925+0000 7f5f5bfff700 1 --2- 172.21.15.40:0/2657513589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f5c0fdd90 0x7f5f5c1006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:29.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.926+0000 7f5f60dbd700 1 -- 172.21.15.40:0/2657513589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f5c0fdd90 msgr2=0x7f5f5c1006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:29.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.926+0000 7f5f60dbd700 1 --2- 172.21.15.40:0/2657513589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f5c0fdd90 0x7f5f5c1006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.926+0000 7f5f60dbd700 1 -- 172.21.15.40:0/2657513589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c100d60 msgr2=0x7f5f5c0fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:29.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.926+0000 7f5f60dbd700 1 --2- 172.21.15.40:0/2657513589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c100d60 0x7f5f5c0fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.926+0000 7f5f60dbd700 1 -- 172.21.15.40:0/2657513589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c009520 con 0x7f5f5c1016f0 2023-07-18T03:32:29.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.926+0000 7f5f5b7fe700 1 --2- 172.21.15.40:0/2657513589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c100d60 0x7f5f5c0fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:29.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.926+0000 7f5f60dbd700 1 --2- 172.21.15.40:0/2657513589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f5c1016f0 0x7f5f5c0ff250 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009fd0 tx=0x7f5f5400bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:29.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.926+0000 7f5f597fa700 1 -- 172.21.15.40:0/2657513589 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5400e040 con 0x7f5f5c1016f0 2023-07-18T03:32:29.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.926+0000 7f5f62820700 1 -- 172.21.15.40:0/2657513589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f54009d00 con 0x7f5f5c1016f0 2023-07-18T03:32:29.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.926+0000 7f5f597fa700 1 -- 172.21.15.40:0/2657513589 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f540094c0 con 0x7f5f5c1016f0 2023-07-18T03:32:29.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.926+0000 7f5f597fa700 1 -- 172.21.15.40:0/2657513589 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f540052a0 con 0x7f5f5c1016f0 2023-07-18T03:32:29.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.926+0000 7f5f62820700 1 -- 172.21.15.40:0/2657513589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5c19ffa0 con 0x7f5f5c1016f0 2023-07-18T03:32:29.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.928+0000 7f5f597fa700 1 -- 172.21.15.40:0/2657513589 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5f5401d470 con 0x7f5f5c1016f0 2023-07-18T03:32:29.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.929+0000 7f5f597fa700 1 --2- 172.21.15.40:0/2657513589 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5f440a1150 0x7f5f440a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.929+0000 7f5f597fa700 1 -- 172.21.15.40:0/2657513589 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5f540cde60 con 0x7f5f5c1016f0 2023-07-18T03:32:29.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.929+0000 7f5f5bfff700 1 --2- 172.21.15.40:0/2657513589 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5f440a1150 0x7f5f440a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:29.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.930+0000 7f5f5bfff700 1 --2- 172.21.15.40:0/2657513589 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5f440a1150 0x7f5f440a3600 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c067470 tx=0x7f5f4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:29.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.932+0000 7f5f6e9a6b80 1 Processor -- start 2023-07-18T03:32:29.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.932+0000 7f5f6e9a6b80 1 -- start start 2023-07-18T03:32:29.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.932+0000 7f5f6e9a6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 0x561a99ffa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.932+0000 7f5f6e9a6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a99ffa6b0 0x561a99ffaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.932+0000 7f5f6e9a6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 0x561a99ffb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.932+0000 7f5f6e9a6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a99ef6d30 con 0x561a99ff9d60 2023-07-18T03:32:29.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.932+0000 7f5f6e9a6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a99f49ac0 con 0x561a99ffa6b0 2023-07-18T03:32:29.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.932+0000 7f5f6e9a6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a99f42f50 con 0x561a99ffb500 2023-07-18T03:32:29.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.932+0000 7f5f5bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 0x561a99ffa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:29.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.933+0000 7f5f60dbd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 0x561a99ffb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:29.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.933+0000 7f5f5bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 0x561a99ffa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59246/0 (socket says 172.21.15.40:59246) 2023-07-18T03:32:29.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.933+0000 7f5f5bfff700 1 -- 172.21.15.40:0/2930985371 learned_addr learned my addr 172.21.15.40:0/2930985371 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:29.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.933+0000 7f5f60dbd700 1 -- 172.21.15.40:0/2930985371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a99ffa6b0 msgr2=0x561a99ffaf70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:32:29.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.933+0000 7f5f60dbd700 1 --2- 172.21.15.40:0/2930985371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a99ffa6b0 0x561a99ffaf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.933+0000 7f5f60dbd700 1 -- 172.21.15.40:0/2930985371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 msgr2=0x561a99ffa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:29.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.933+0000 7f5f60dbd700 1 --2- 172.21.15.40:0/2930985371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 0x561a99ffa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.933+0000 7f5f60dbd700 1 -- 172.21.15.40:0/2930985371 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a99d70e60 con 0x561a99ffb500 2023-07-18T03:32:29.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.933+0000 7f5f60dbd700 1 --2- 172.21.15.40:0/2930985371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 0x561a99ffb970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5f540cb990 tx=0x7f5f5401dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7667dc535fc62c56 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:29.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.933+0000 7f5f427fc700 1 -- 172.21.15.40:0/2930985371 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5402d2f0 con 0x561a99ffb500 2023-07-18T03:32:29.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.933+0000 7f5f427fc700 1 -- 172.21.15.40:0/2930985371 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f5409b2e0 con 0x561a99ffb500 2023-07-18T03:32:29.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.933+0000 7f5f427fc700 1 -- 172.21.15.40:0/2930985371 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f54099680 con 0x561a99ffb500 2023-07-18T03:32:29.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.934+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/2930985371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 msgr2=0x561a99ffb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:29.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.934+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/2930985371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 0x561a99ffb970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5f540cb990 tx=0x7f5f5401dc60 comp rx=0 tx=0).stop 2023-07-18T03:32:29.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.934+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/2930985371 shutdown_connections 2023-07-18T03:32:29.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.934+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/2930985371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 0x561a99ffb970 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.934+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/2930985371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 0x561a99ffa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.934+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/2930985371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a99ffa6b0 0x561a99ffaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.934+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/2930985371 >> 172.21.15.40:0/2930985371 conn(0x561a99fe6e20 msgr2=0x561a99fec250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:29.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.935+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/2930985371 shutdown_connections 2023-07-18T03:32:29.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.935+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/2930985371 wait complete. 2023-07-18T03:32:29.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.936+0000 7f5f6e9a6b80 1 -- start start 2023-07-18T03:32:29.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.936+0000 7f5f6e9a6b80 1 Processor -- start 2023-07-18T03:32:29.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f6e9a6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 0x561a9a003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f6e9a6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 0x561a9a003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f6e9a6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a9a008f70 0x561a9a00b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f6e9a6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a99ee2b40 con 0x561a99ff9d60 2023-07-18T03:32:29.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f6e9a6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a99ee2840 con 0x561a9a008f70 2023-07-18T03:32:29.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f6e9a6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a99dc69a0 con 0x561a99ffb500 2023-07-18T03:32:29.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f6e9a6b80 10 client.0 _ll_register_callbacks cb 0x561a99ee07e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:29.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f5bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 0x561a9a003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:29.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f5bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 0x561a9a003a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59250/0 (socket says 172.21.15.40:59250) 2023-07-18T03:32:29.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f5bfff700 1 -- 172.21.15.40:0/3316596434 learned_addr learned my addr 172.21.15.40:0/3316596434 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:29.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f60dbd700 1 --2- 172.21.15.40:0/3316596434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a9a008f70 0x561a9a00b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:29.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f5bfff700 1 -- 172.21.15.40:0/3316596434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 msgr2=0x561a9a003f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:32:29.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f5bfff700 1 --2- 172.21.15.40:0/3316596434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 0x561a9a003f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f5bfff700 1 -- 172.21.15.40:0/3316596434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a9a008f70 msgr2=0x561a9a00b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:29.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f5bfff700 1 --2- 172.21.15.40:0/3316596434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a9a008f70 0x561a9a00b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f5bfff700 1 -- 172.21.15.40:0/3316596434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5c19ffa0 con 0x561a99ff9d60 2023-07-18T03:32:29.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f5bfff700 1 --2- 172.21.15.40:0/3316596434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 0x561a9a003a00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0039f0 tx=0x7f5f4c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571025da6e0b2090 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:29.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.937+0000 7f5f3b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:32:29.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.938+0000 7f5f3b7fe700 1 -- 172.21.15.40:0/3316596434 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c01fcd0 con 0x561a99ff9d60 2023-07-18T03:32:29.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.938+0000 7f5f3b7fe700 1 -- 172.21.15.40:0/3316596434 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5f4c01fe30 con 0x561a99ff9d60 2023-07-18T03:32:29.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.938+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/3316596434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a99e9c570 con 0x561a99ff9d60 2023-07-18T03:32:29.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.938+0000 7f5f3b7fe700 1 -- 172.21.15.40:0/3316596434 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f4c003e60 con 0x561a99ff9d60 2023-07-18T03:32:29.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.938+0000 7f5f6e9a6b80 10 client.15996 fetch_fsmap learned FSMap version 31 2023-07-18T03:32:29.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.938+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/3316596434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x561a99d59ce0 con 0x561a99ff9d60 2023-07-18T03:32:29.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.938+0000 7f5f3b7fe700 1 -- 172.21.15.40:0/3316596434 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561a99e9c570 con 0x561a99ff9d60 2023-07-18T03:32:29.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.938+0000 7f5f3b7fe700 1 -- 172.21.15.40:0/3316596434 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5f4c030020 con 0x561a99ff9d60 2023-07-18T03:32:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.938+0000 7f5f6e9a6b80 10 client.15996 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:32:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.938+0000 7f5f6e9a6b80 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:32:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.939+0000 7f5f6e9a6b80 20 client.15996 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.939+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/3316596434 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561a9a00fc10 0x561a9a012020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.939+0000 7f5f6e9a6b80 4 client.15996 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:32:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.939+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/3316596434 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a99f3bf30 con 0x561a9a00fc10 2023-07-18T03:32:29.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.942+0000 7f5f5b7fe700 1 --2- 172.21.15.40:0/3316596434 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561a9a00fc10 0x561a9a012020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.943+0000 7f5f3b7fe700 10 client.15996 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.943+0000 7f5f5b7fe700 1 --2- 172.21.15.40:0/3316596434 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561a9a00fc10 0x561a9a012020 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a21084014c6d79b server_cookie=e81f14c3009f480 in_seq=0 out_seq=0 2023-07-18T03:32:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.943+0000 7f5f3b7fe700 1 -- 172.21.15.40:0/3316596434 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561a99f3bf30 con 0x561a9a00fc10 2023-07-18T03:32:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.943+0000 7f5f3b7fe700 10 client.15996 handle_command_reply: tid=0 2023-07-18T03:32:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.943+0000 7f5f6e9a6b80 1 client.15996 shutdown 2023-07-18T03:32:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.944+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/3316596434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 msgr2=0x561a9a003a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.944+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/3316596434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 0x561a9a003a00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0039f0 tx=0x7f5f4c0047c0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.944+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/3316596434 shutdown_connections 2023-07-18T03:32:29.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.944+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/3316596434 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561a9a00fc10 0x561a9a012020 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.944+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/3316596434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a9a008f70 0x561a9a00b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.944+0000 7f5f3b7fe700 0 client.15996 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:29.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.944+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/3316596434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 0x561a9a003a00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.944+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/3316596434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 0x561a9a003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.944+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/3316596434 >> 172.21.15.40:0/3316596434 conn(0x561a99fe6e20 msgr2=0x561a99fec1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:29.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.944+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/3316596434 shutdown_connections 2023-07-18T03:32:29.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.944+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/3316596434 wait complete. 2023-07-18T03:32:29.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.945+0000 7f5f6e9a6b80 20 client.15996 trim_cache size 0 max 16384 2023-07-18T03:32:29.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.964+0000 7f5f6e9a6b80 1 Processor -- start 2023-07-18T03:32:29.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f6e9a6b80 1 -- start start 2023-07-18T03:32:29.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f6e9a6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99fe6e20 0x561a9a01f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f6e9a6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a99ff9d60 0x561a9a0037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f6e9a6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 0x561a9a003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f6e9a6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a99fed540 con 0x561a99fe6e20 2023-07-18T03:32:29.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f6e9a6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9a004900 con 0x561a99ff9d60 2023-07-18T03:32:29.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f6e9a6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9a00d020 con 0x561a99ffb500 2023-07-18T03:32:29.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f5bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99fe6e20 0x561a9a01f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:29.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f5bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99fe6e20 0x561a9a01f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59258/0 (socket says 172.21.15.40:59258) 2023-07-18T03:32:29.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f5bfff700 1 -- 172.21.15.40:0/2779897189 learned_addr learned my addr 172.21.15.40:0/2779897189 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:29.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f5b7fe700 1 --2- 172.21.15.40:0/2779897189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a99ff9d60 0x561a9a0037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:29.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f60dbd700 1 --2- 172.21.15.40:0/2779897189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 0x561a9a003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:29.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f5bfff700 1 -- 172.21.15.40:0/2779897189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 msgr2=0x561a9a003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:29.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f5bfff700 1 --2- 172.21.15.40:0/2779897189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 0x561a9a003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f5bfff700 1 -- 172.21.15.40:0/2779897189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a99ff9d60 msgr2=0x561a9a0037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:29.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f5bfff700 1 --2- 172.21.15.40:0/2779897189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a99ff9d60 0x561a9a0037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f5bfff700 1 -- 172.21.15.40:0/2779897189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a99feb160 con 0x561a99fe6e20 2023-07-18T03:32:29.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f60dbd700 1 --2- 172.21.15.40:0/2779897189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 0x561a9a003cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:32:29.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.965+0000 7f5f5bfff700 1 --2- 172.21.15.40:0/2779897189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99fe6e20 0x561a9a01f8e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c016040 tx=0x7f5f4c0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19168b1cc9b468c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:29.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.966+0000 7f5f42ffd700 1 -- 172.21.15.40:0/2779897189 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c032420 con 0x561a99fe6e20 2023-07-18T03:32:29.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.966+0000 7f5f42ffd700 1 -- 172.21.15.40:0/2779897189 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f4c032c40 con 0x561a99fe6e20 2023-07-18T03:32:29.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.966+0000 7f5f42ffd700 1 -- 172.21.15.40:0/2779897189 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c0289c0 con 0x561a99fe6e20 2023-07-18T03:32:29.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.966+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/2779897189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99fe6e20 msgr2=0x561a9a01f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:29.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.966+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/2779897189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99fe6e20 0x561a9a01f8e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c016040 tx=0x7f5f4c0092d0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.967+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/2779897189 shutdown_connections 2023-07-18T03:32:29.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.967+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/2779897189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 0x561a9a003cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.967+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/2779897189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99fe6e20 0x561a9a01f8e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.967+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/2779897189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a99ff9d60 0x561a9a0037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.967+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/2779897189 >> 172.21.15.40:0/2779897189 conn(0x561a99fe8680 msgr2=0x561a99ffe440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:29.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.967+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/2779897189 shutdown_connections 2023-07-18T03:32:29.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.967+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/2779897189 wait complete. 2023-07-18T03:32:29.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.967+0000 7f5f6e9a6b80 1 -- start start 2023-07-18T03:32:29.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.967+0000 7f5f6e9a6b80 1 Processor -- start 2023-07-18T03:32:29.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.967+0000 7f5f6e9a6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a99fe6e20 0x561a9a01ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.967+0000 7f5f6e9a6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 0x561a9a01f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.967+0000 7f5f6e9a6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 0x561a9a064330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.967+0000 7f5f6e9a6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9a0648a0 con 0x561a99ff9d60 2023-07-18T03:32:29.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f6e9a6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9a064a10 con 0x561a99fe6e20 2023-07-18T03:32:29.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f6e9a6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9a064bd0 con 0x561a99ffb500 2023-07-18T03:32:29.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f6e9a6b80 10 client.0 _ll_register_callbacks cb 0x561a99ee07e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:29.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f5b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 0x561a9a01f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:29.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f5b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 0x561a9a01f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59266/0 (socket says 172.21.15.40:59266) 2023-07-18T03:32:29.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f5b7fe700 1 -- 172.21.15.40:0/1352442783 learned_addr learned my addr 172.21.15.40:0/1352442783 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:29.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f60dbd700 1 --2- 172.21.15.40:0/1352442783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 0x561a9a064330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:29.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f5bfff700 1 --2- 172.21.15.40:0/1352442783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a99fe6e20 0x561a9a01ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:29.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f5b7fe700 1 -- 172.21.15.40:0/1352442783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 msgr2=0x561a9a064330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:29.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f5b7fe700 1 --2- 172.21.15.40:0/1352442783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 0x561a9a064330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f5b7fe700 1 -- 172.21.15.40:0/1352442783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a99fe6e20 msgr2=0x561a9a01ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:29.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f5b7fe700 1 --2- 172.21.15.40:0/1352442783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a99fe6e20 0x561a9a01ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f5b7fe700 1 -- 172.21.15.40:0/1352442783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a99d59ce0 con 0x561a99ff9d60 2023-07-18T03:32:29.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f5b7fe700 1 --2- 172.21.15.40:0/1352442783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 0x561a9a01f150 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5f50005d90 tx=0x7f5f50019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396ec4d6c8bfa95c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:29.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f3b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:32:29.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f3b7fe700 1 -- 172.21.15.40:0/1352442783 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f50004230 con 0x561a99ff9d60 2023-07-18T03:32:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f3b7fe700 1 -- 172.21.15.40:0/1352442783 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5f50016070 con 0x561a99ff9d60 2023-07-18T03:32:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/1352442783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f5409b2e0 con 0x561a99ff9d60 2023-07-18T03:32:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f3b7fe700 1 -- 172.21.15.40:0/1352442783 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f50004390 con 0x561a99ff9d60 2023-07-18T03:32:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f6e9a6b80 10 client.16002 fetch_fsmap learned FSMap version 31 2023-07-18T03:32:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.968+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/1352442783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x561a99d972d0 con 0x561a99ff9d60 2023-07-18T03:32:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.969+0000 7f5f3b7fe700 1 -- 172.21.15.40:0/1352442783 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f5409b2e0 con 0x561a99ff9d60 2023-07-18T03:32:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.969+0000 7f5f3b7fe700 1 -- 172.21.15.40:0/1352442783 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5f5002a050 con 0x561a99ff9d60 2023-07-18T03:32:29.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.969+0000 7f5f6e9a6b80 10 client.16002 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:32:29.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.969+0000 7f5f6e9a6b80 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:32:29.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.969+0000 7f5f6e9a6b80 20 client.16002 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:29.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.969+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/1352442783 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561a9a008f70 0x561a9a068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:29.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.969+0000 7f5f6e9a6b80 4 client.16002 mds_command: new command op to 24527 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-18T03:32:29.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.969+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/1352442783 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561a99efe0d0 con 0x561a9a008f70 2023-07-18T03:32:29.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.969+0000 7f5f5bfff700 1 --2- 172.21.15.40:0/1352442783 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561a9a008f70 0x561a9a068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:29.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.969+0000 7f5f3b7fe700 10 client.16002 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:29.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.969+0000 7f5f5bfff700 1 --2- 172.21.15.40:0/1352442783 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561a9a008f70 0x561a9a068670 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e431465f2d674f4 server_cookie=52eab62d93134d4d in_seq=0 out_seq=0 2023-07-18T03:32:29.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.970+0000 7f5f3b7fe700 1 -- 172.21.15.40:0/1352442783 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561a99efe0d0 con 0x561a9a008f70 2023-07-18T03:32:29.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.970+0000 7f5f3b7fe700 10 client.16002 handle_command_reply: tid=0 2023-07-18T03:32:29.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.971+0000 7f5f6e9a6b80 1 client.16002 shutdown 2023-07-18T03:32:29.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.971+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/1352442783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 msgr2=0x561a9a01f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:29.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.971+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/1352442783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 0x561a9a01f150 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5f50005d90 tx=0x7f5f50019660 comp rx=0 tx=0).stop 2023-07-18T03:32:29.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.971+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/1352442783 shutdown_connections 2023-07-18T03:32:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.971+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/1352442783 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561a9a008f70 0x561a9a068670 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.971+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/1352442783 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561a99ffb500 0x561a9a064330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.971+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/1352442783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a99fe6e20 0x561a9a01ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.971+0000 7f5f6e9a6b80 1 --2- 172.21.15.40:0/1352442783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561a99ff9d60 0x561a9a01f150 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.971+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/1352442783 >> 172.21.15.40:0/1352442783 conn(0x561a99fe8680 msgr2=0x561a9a0037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.971+0000 7f5f3b7fe700 0 client.16002 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.971+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/1352442783 shutdown_connections 2023-07-18T03:32:29.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.971+0000 7f5f6e9a6b80 1 -- 172.21.15.40:0/1352442783 wait complete. 2023-07-18T03:32:29.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.971+0000 7f5f6e9a6b80 20 client.16002 trim_cache size 0 max 16384 2023-07-18T03:32:29.977 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:32:29.977 INFO:teuthology.orchestra.run.smithi040.stdout: "return_code": 0, 2023-07-18T03:32:29.977 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:32:29.977 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": "asynchronous" 2023-07-18T03:32:29.978 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:32:29.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.971+0000 7f5f62820700 1 -- 172.21.15.40:0/2657513589 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5f440a1150 msgr2=0x7f5f440a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:29.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.971+0000 7f5f62820700 1 --2- 172.21.15.40:0/2657513589 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5f440a1150 0x7f5f440a3600 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c067470 tx=0x7f5f4c009260 comp rx=0 tx=0).stop 2023-07-18T03:32:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.972+0000 7f5f62820700 1 -- 172.21.15.40:0/2657513589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f5c1016f0 msgr2=0x7f5f5c0ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.972+0000 7f5f62820700 1 --2- 172.21.15.40:0/2657513589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f5c1016f0 0x7f5f5c0ff250 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009fd0 tx=0x7f5f5400bce0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.972+0000 7f5f62820700 1 -- 172.21.15.40:0/2657513589 shutdown_connections 2023-07-18T03:32:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.972+0000 7f5f62820700 1 --2- 172.21.15.40:0/2657513589 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5f440a1150 0x7f5f440a3600 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.972+0000 7f5f62820700 1 --2- 172.21.15.40:0/2657513589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f5c1016f0 0x7f5f5c0ff250 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.972+0000 7f5f62820700 1 --2- 172.21.15.40:0/2657513589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f5c0fdd90 0x7f5f5c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.972+0000 7f5f62820700 1 --2- 172.21.15.40:0/2657513589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f5c100d60 0x7f5f5c0fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.972+0000 7f5f62820700 1 -- 172.21.15.40:0/2657513589 >> 172.21.15.40:0/2657513589 conn(0x7f5f5c0f9770 msgr2=0x7f5f5c1041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.972+0000 7f5f62820700 1 -- 172.21.15.40:0/2657513589 shutdown_connections 2023-07-18T03:32:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:29.972+0000 7f5f62820700 1 -- 172.21.15.40:0/2657513589 wait complete. 2023-07-18T03:32:29.981 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:32:30.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:29 smithi134 ceph-mon[98216]: pgmap v341: 97 pgs: 97 active+clean; 55 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 190 op/s 2023-07-18T03:32:30.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:29 smithi040 ceph-mon[96439]: pgmap v341: 97 pgs: 97 active+clean; 55 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 190 op/s 2023-07-18T03:32:30.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:29 smithi064 ceph-mon[98059]: pgmap v341: 97 pgs: 97 active+clean; 55 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 190 op/s 2023-07-18T03:32:30.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.118+0000 7fb4e902e700 1 -- 172.21.15.40:0/2026526433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4e4100d00 msgr2=0x7fb4e4101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:30.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.118+0000 7fb4e902e700 1 --2- 172.21.15.40:0/2026526433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4e4100d00 0x7fb4e4101150 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009990 tx=0x7fb4d8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.118+0000 7fb4e902e700 1 -- 172.21.15.40:0/2026526433 shutdown_connections 2023-07-18T03:32:30.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.118+0000 7fb4e902e700 1 --2- 172.21.15.40:0/2026526433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4e4101690 0x7fb4e4107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.118+0000 7fb4e902e700 1 --2- 172.21.15.40:0/2026526433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4e40ffb10 0x7fb4e40fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.118+0000 7fb4e902e700 1 --2- 172.21.15.40:0/2026526433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4e4100d00 0x7fb4e4101150 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.118+0000 7fb4e902e700 1 -- 172.21.15.40:0/2026526433 >> 172.21.15.40:0/2026526433 conn(0x7fb4e40fb130 msgr2=0x7fb4e40fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:30.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.118+0000 7fb4e902e700 1 -- 172.21.15.40:0/2026526433 shutdown_connections 2023-07-18T03:32:30.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.118+0000 7fb4e902e700 1 -- 172.21.15.40:0/2026526433 wait complete. 2023-07-18T03:32:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.119+0000 7fb4e902e700 1 Processor -- start 2023-07-18T03:32:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.119+0000 7fb4e902e700 1 -- start start 2023-07-18T03:32:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.119+0000 7fb4e902e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4e40ffb10 0x7fb4e4195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.119+0000 7fb4e902e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4e4100d00 0x7fb4e4195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.119+0000 7fb4e902e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4e4101690 0x7fb4e419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.119+0000 7fb4e902e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e419d390 con 0x7fb4e40ffb10 2023-07-18T03:32:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.119+0000 7fb4e902e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e419d4d0 con 0x7fb4e4100d00 2023-07-18T03:32:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.119+0000 7fb4e902e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e419d660 con 0x7fb4e4101690 2023-07-18T03:32:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.119+0000 7fb4e2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4e40ffb10 0x7fb4e4195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.119+0000 7fb4e2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4e40ffb10 0x7fb4e4195810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59278/0 (socket says 172.21.15.40:59278) 2023-07-18T03:32:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.119+0000 7fb4e2d9d700 1 -- 172.21.15.40:0/116993120 learned_addr learned my addr 172.21.15.40:0/116993120 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.120+0000 7fb4e259c700 1 --2- 172.21.15.40:0/116993120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4e4100d00 0x7fb4e4195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.120+0000 7fb4e359e700 1 --2- 172.21.15.40:0/116993120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4e4101690 0x7fb4e419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.120+0000 7fb4e2d9d700 1 -- 172.21.15.40:0/116993120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4e4101690 msgr2=0x7fb4e419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.120+0000 7fb4e2d9d700 1 --2- 172.21.15.40:0/116993120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4e4101690 0x7fb4e419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.120+0000 7fb4e2d9d700 1 -- 172.21.15.40:0/116993120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4e4100d00 msgr2=0x7fb4e4195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.120+0000 7fb4e2d9d700 1 --2- 172.21.15.40:0/116993120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4e4100d00 0x7fb4e4195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.120+0000 7fb4e2d9d700 1 -- 172.21.15.40:0/116993120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d80095f0 con 0x7fb4e40ffb10 2023-07-18T03:32:30.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.120+0000 7fb4e2d9d700 1 --2- 172.21.15.40:0/116993120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4e40ffb10 0x7fb4e4195810 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400e970 tx=0x7fb4d400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:30.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.120+0000 7fb4cbfff700 1 -- 172.21.15.40:0/116993120 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4009790 con 0x7fb4e40ffb10 2023-07-18T03:32:30.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.120+0000 7fb4e902e700 1 -- 172.21.15.40:0/116993120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e419d940 con 0x7fb4e40ffb10 2023-07-18T03:32:30.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.120+0000 7fb4cbfff700 1 -- 172.21.15.40:0/116993120 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d400c7e0 con 0x7fb4e40ffb10 2023-07-18T03:32:30.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.120+0000 7fb4cbfff700 1 -- 172.21.15.40:0/116993120 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4003c70 con 0x7fb4e40ffb10 2023-07-18T03:32:30.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.120+0000 7fb4e902e700 1 -- 172.21.15.40:0/116993120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e419dea0 con 0x7fb4e40ffb10 2023-07-18T03:32:30.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.122+0000 7fb4cbfff700 1 -- 172.21.15.40:0/116993120 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb4d400cc50 con 0x7fb4e40ffb10 2023-07-18T03:32:30.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.123+0000 7fb4cbfff700 1 --2- 172.21.15.40:0/116993120 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb4cc0a1150 0x7fb4cc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.123+0000 7fb4cbfff700 1 -- 172.21.15.40:0/116993120 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb4d4014070 con 0x7fb4e40ffb10 2023-07-18T03:32:30.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.123+0000 7fb4e259c700 1 --2- 172.21.15.40:0/116993120 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb4cc0a1150 0x7fb4cc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.123+0000 7fb4e259c700 1 --2- 172.21.15.40:0/116993120 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb4cc0a1150 0x7fb4cc0a3600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800b2e0 tx=0x7fb4d8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:30.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4f51b4b80 1 Processor -- start 2023-07-18T03:32:30.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4f51b4b80 1 -- start start 2023-07-18T03:32:30.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4f51b4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e4634d0 0x563a0e4638e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4f51b4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e463e20 0x563a0e4646e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4f51b4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e464c70 0x563a0e4650e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4f51b4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0e368cb0 con 0x563a0e4634d0 2023-07-18T03:32:30.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4f51b4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0e3bba40 con 0x563a0e463e20 2023-07-18T03:32:30.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4f51b4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0e3b4ed0 con 0x563a0e464c70 2023-07-18T03:32:30.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4e2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e4634d0 0x563a0e4638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4e2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e4634d0 0x563a0e4638e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59286/0 (socket says 172.21.15.40:59286) 2023-07-18T03:32:30.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4e2d9d700 1 -- 172.21.15.40:0/3715950106 learned_addr learned my addr 172.21.15.40:0/3715950106 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:30.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4e359e700 1 --2- 172.21.15.40:0/3715950106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e464c70 0x563a0e4650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4e259c700 1 --2- 172.21.15.40:0/3715950106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e463e20 0x563a0e4646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4e2d9d700 1 -- 172.21.15.40:0/3715950106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e464c70 msgr2=0x563a0e4650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:30.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4e2d9d700 1 --2- 172.21.15.40:0/3715950106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e464c70 0x563a0e4650e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4e2d9d700 1 -- 172.21.15.40:0/3715950106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e463e20 msgr2=0x563a0e4646e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:30.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4e2d9d700 1 --2- 172.21.15.40:0/3715950106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e463e20 0x563a0e4646e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4e2d9d700 1 -- 172.21.15.40:0/3715950106 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a0e1e2de0 con 0x563a0e4634d0 2023-07-18T03:32:30.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.124+0000 7fb4e359e700 1 --2- 172.21.15.40:0/3715950106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e464c70 0x563a0e4650e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:30.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.125+0000 7fb4e2d9d700 1 --2- 172.21.15.40:0/3715950106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e4634d0 0x563a0e4638e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40963b0 tx=0x7fb4d4099890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c09ccecbeb2f255 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:30.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.125+0000 7fb4c97fa700 1 -- 172.21.15.40:0/3715950106 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d409dee0 con 0x563a0e4634d0 2023-07-18T03:32:30.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.125+0000 7fb4c97fa700 1 -- 172.21.15.40:0/3715950106 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d40a2300 con 0x563a0e4634d0 2023-07-18T03:32:30.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.125+0000 7fb4c97fa700 1 -- 172.21.15.40:0/3715950106 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d40993a0 con 0x563a0e4634d0 2023-07-18T03:32:30.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.125+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/3715950106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e4634d0 msgr2=0x563a0e4638e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:30.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.125+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/3715950106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e4634d0 0x563a0e4638e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40963b0 tx=0x7fb4d4099890 comp rx=0 tx=0).stop 2023-07-18T03:32:30.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/3715950106 shutdown_connections 2023-07-18T03:32:30.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/3715950106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e464c70 0x563a0e4650e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/3715950106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e4634d0 0x563a0e4638e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/3715950106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e463e20 0x563a0e4646e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/3715950106 >> 172.21.15.40:0/3715950106 conn(0x563a0e458da0 msgr2=0x563a0e45e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:30.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/3715950106 shutdown_connections 2023-07-18T03:32:30.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/3715950106 wait complete. 2023-07-18T03:32:30.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4f51b4b80 1 -- start start 2023-07-18T03:32:30.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4f51b4b80 1 Processor -- start 2023-07-18T03:32:30.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4f51b4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e4634d0 0x563a0e46d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4f51b4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e463e20 0x563a0e46d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4f51b4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e464c70 0x563a0e474750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4f51b4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0e354ac0 con 0x563a0e4634d0 2023-07-18T03:32:30.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4f51b4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0e3547c0 con 0x563a0e464c70 2023-07-18T03:32:30.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4f51b4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0e238920 con 0x563a0e463e20 2023-07-18T03:32:30.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4f51b4b80 10 client.0 _ll_register_callbacks cb 0x563a0e352760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:30.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4e2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e4634d0 0x563a0e46d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4e259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e463e20 0x563a0e46d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.127+0000 7fb4e259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e463e20 0x563a0e46d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48322/0 (socket says 172.21.15.40:48322) 2023-07-18T03:32:30.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.127+0000 7fb4e259c700 1 -- 172.21.15.40:0/3947036099 learned_addr learned my addr 172.21.15.40:0/3947036099 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:30.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.126+0000 7fb4e2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e4634d0 0x563a0e46d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59292/0 (socket says 172.21.15.40:59292) 2023-07-18T03:32:30.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.127+0000 7fb4e359e700 1 --2- 172.21.15.40:0/3947036099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e464c70 0x563a0e474750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.127+0000 7fb4e259c700 1 -- 172.21.15.40:0/3947036099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e464c70 msgr2=0x563a0e474750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:30.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.127+0000 7fb4e259c700 1 --2- 172.21.15.40:0/3947036099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e464c70 0x563a0e474750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.127+0000 7fb4e259c700 1 -- 172.21.15.40:0/3947036099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e4634d0 msgr2=0x563a0e46d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:30.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.127+0000 7fb4e259c700 1 --2- 172.21.15.40:0/3947036099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e4634d0 0x563a0e46d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.127+0000 7fb4e259c700 1 -- 172.21.15.40:0/3947036099 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4dc003ac0 con 0x563a0e463e20 2023-07-18T03:32:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.127+0000 7fb4e259c700 1 --2- 172.21.15.40:0/3947036099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e463e20 0x563a0e46d710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009990 tx=0x7fb4d8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcc7a8aa7d0973bd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.127+0000 7fb4c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:32:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.127+0000 7fb4c6ff5700 1 -- 172.21.15.40:0/3947036099 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8004e80 con 0x563a0e463e20 2023-07-18T03:32:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.127+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/3947036099 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a0e30e4f0 con 0x563a0e463e20 2023-07-18T03:32:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.127+0000 7fb4c6ff5700 1 -- 172.21.15.40:0/3947036099 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb4d800f460 con 0x563a0e463e20 2023-07-18T03:32:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.127+0000 7fb4c6ff5700 1 -- 172.21.15.40:0/3947036099 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4d800f620 con 0x563a0e463e20 2023-07-18T03:32:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.127+0000 7fb4f51b4b80 10 client.25654 fetch_fsmap learned FSMap version 31 2023-07-18T03:32:30.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.127+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/3947036099 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fb4e419dea0 con 0x563a0e463e20 2023-07-18T03:32:30.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.128+0000 7fb4c6ff5700 1 -- 172.21.15.40:0/3947036099 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d803f020 con 0x563a0e463e20 2023-07-18T03:32:30.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.128+0000 7fb4c6ff5700 1 -- 172.21.15.40:0/3947036099 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fb4d802f070 con 0x563a0e463e20 2023-07-18T03:32:30.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.128+0000 7fb4f51b4b80 10 client.25654 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:32:30.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.128+0000 7fb4f51b4b80 10 client.25654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:32:30.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.128+0000 7fb4f51b4b80 20 client.25654 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:30.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.128+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/3947036099 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563a0e479000 0x563a0e47b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.128+0000 7fb4f51b4b80 4 client.25654 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:32:30.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.128+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/3947036099 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a0e3adeb0 con 0x563a0e479000 2023-07-18T03:32:30.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.128+0000 7fb4e2d9d700 1 --2- 172.21.15.40:0/3947036099 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563a0e479000 0x563a0e47b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.129+0000 7fb4e2d9d700 1 --2- 172.21.15.40:0/3947036099 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563a0e479000 0x563a0e47b410 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=115fd91a91cda4a2 server_cookie=5f7dc6f8f730e81 in_seq=0 out_seq=0 2023-07-18T03:32:30.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.129+0000 7fb4c6ff5700 10 client.25654 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:30.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.129+0000 7fb4c6ff5700 1 -- 172.21.15.40:0/3947036099 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563a0e3adeb0 con 0x563a0e479000 2023-07-18T03:32:30.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.129+0000 7fb4c6ff5700 10 client.25654 handle_command_reply: tid=0 2023-07-18T03:32:30.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.129+0000 7fb4f51b4b80 1 client.25654 shutdown 2023-07-18T03:32:30.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.129+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/3947036099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e463e20 msgr2=0x563a0e46d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:30.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.130+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/3947036099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e463e20 0x563a0e46d710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009990 tx=0x7fb4d8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.130+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/3947036099 shutdown_connections 2023-07-18T03:32:30.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.130+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/3947036099 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563a0e479000 0x563a0e47b410 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.130+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/3947036099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e464c70 0x563a0e474750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.130+0000 7fb4c6ff5700 0 client.25654 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:30.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.130+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/3947036099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e4634d0 0x563a0e46d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.130+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/3947036099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e463e20 0x563a0e46d710 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.130+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/3947036099 >> 172.21.15.40:0/3947036099 conn(0x563a0e458da0 msgr2=0x563a0e45d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:30.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.130+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/3947036099 shutdown_connections 2023-07-18T03:32:30.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.130+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/3947036099 wait complete. 2023-07-18T03:32:30.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.130+0000 7fb4f51b4b80 20 client.25654 trim_cache size 0 max 16384 2023-07-18T03:32:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.147+0000 7fb4f51b4b80 1 Processor -- start 2023-07-18T03:32:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.147+0000 7fb4f51b4b80 1 -- start start 2023-07-18T03:32:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.147+0000 7fb4f51b4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e4634d0 0x563a0e45d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.147+0000 7fb4f51b4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e463e20 0x563a0e474710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.147+0000 7fb4f51b4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e464c70 0x563a0e46be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.147+0000 7fb4f51b4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0e4641f0 con 0x563a0e464c70 2023-07-18T03:32:30.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.147+0000 7fb4f51b4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0e46ae30 con 0x563a0e4634d0 2023-07-18T03:32:30.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.147+0000 7fb4f51b4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0e478d00 con 0x563a0e463e20 2023-07-18T03:32:30.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.147+0000 7fb4e259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e463e20 0x563a0e474710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.147+0000 7fb4e359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e464c70 0x563a0e46be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.147+0000 7fb4e259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e463e20 0x563a0e474710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48330/0 (socket says 172.21.15.40:48330) 2023-07-18T03:32:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.147+0000 7fb4e259c700 1 -- 172.21.15.40:0/4270622397 learned_addr learned my addr 172.21.15.40:0/4270622397 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.147+0000 7fb4e2d9d700 1 --2- 172.21.15.40:0/4270622397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e4634d0 0x563a0e45d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.148+0000 7fb4e259c700 1 -- 172.21.15.40:0/4270622397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e4634d0 msgr2=0x563a0e45d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.148+0000 7fb4e259c700 1 --2- 172.21.15.40:0/4270622397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e4634d0 0x563a0e45d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.148+0000 7fb4e259c700 1 -- 172.21.15.40:0/4270622397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e464c70 msgr2=0x563a0e46be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.148+0000 7fb4e259c700 1 --2- 172.21.15.40:0/4270622397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e464c70 0x563a0e46be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.148+0000 7fb4e259c700 1 -- 172.21.15.40:0/4270622397 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a0e1cbc60 con 0x563a0e463e20 2023-07-18T03:32:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.148+0000 7fb4e2d9d700 1 --2- 172.21.15.40:0/4270622397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e4634d0 0x563a0e45d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:32:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.148+0000 7fb4e359e700 1 --2- 172.21.15.40:0/4270622397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e464c70 0x563a0e46be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.148+0000 7fb4e259c700 1 --2- 172.21.15.40:0/4270622397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e463e20 0x563a0e474710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8000c00 tx=0x7fb4d8004b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9fb00d3b3fd853e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.148+0000 7fb4c9ffb700 1 -- 172.21.15.40:0/4270622397 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8024610 con 0x563a0e463e20 2023-07-18T03:32:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.148+0000 7fb4c9ffb700 1 -- 172.21.15.40:0/4270622397 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d803f070 con 0x563a0e463e20 2023-07-18T03:32:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.148+0000 7fb4c9ffb700 1 -- 172.21.15.40:0/4270622397 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d800fca0 con 0x563a0e463e20 2023-07-18T03:32:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/4270622397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e463e20 msgr2=0x563a0e474710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/4270622397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e463e20 0x563a0e474710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8000c00 tx=0x7fb4d8004b70 comp rx=0 tx=0).stop 2023-07-18T03:32:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/4270622397 shutdown_connections 2023-07-18T03:32:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/4270622397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e464c70 0x563a0e46be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/4270622397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e4634d0 0x563a0e45d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/4270622397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e463e20 0x563a0e474710 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/4270622397 >> 172.21.15.40:0/4270622397 conn(0x563a0e458da0 msgr2=0x563a0e47fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/4270622397 shutdown_connections 2023-07-18T03:32:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/4270622397 wait complete. 2023-07-18T03:32:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 1 -- start start 2023-07-18T03:32:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 1 Processor -- start 2023-07-18T03:32:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e4634d0 0x563a0e473fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e463e20 0x563a0e4744f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e464c70 0x563a0e4725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0e472be0 con 0x563a0e463e20 2023-07-18T03:32:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0e472d50 con 0x563a0e4634d0 2023-07-18T03:32:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0e472f10 con 0x563a0e464c70 2023-07-18T03:32:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4f51b4b80 10 client.0 _ll_register_callbacks cb 0x7fb4d800bc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.149+0000 7fb4e259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e463e20 0x563a0e4744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4e259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e463e20 0x563a0e4744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59308/0 (socket says 172.21.15.40:59308) 2023-07-18T03:32:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4e259c700 1 -- 172.21.15.40:0/506996871 learned_addr learned my addr 172.21.15.40:0/506996871 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4e359e700 1 --2- 172.21.15.40:0/506996871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e464c70 0x563a0e4725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4e2d9d700 1 --2- 172.21.15.40:0/506996871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e4634d0 0x563a0e473fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4e259c700 1 -- 172.21.15.40:0/506996871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e464c70 msgr2=0x563a0e4725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4e259c700 1 --2- 172.21.15.40:0/506996871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e464c70 0x563a0e4725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4e259c700 1 -- 172.21.15.40:0/506996871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e4634d0 msgr2=0x563a0e473fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4e259c700 1 --2- 172.21.15.40:0/506996871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e4634d0 0x563a0e473fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4e259c700 1 -- 172.21.15.40:0/506996871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4dc009b30 con 0x563a0e463e20 2023-07-18T03:32:30.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4e359e700 1 --2- 172.21.15.40:0/506996871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e464c70 0x563a0e4725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:30.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4e259c700 1 --2- 172.21.15.40:0/506996871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e463e20 0x563a0e4744f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8024e70 tx=0x7fb4d800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18997694a40ae0f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:30.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:32:30.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4c6ff5700 1 -- 172.21.15.40:0/506996871 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d800fe10 con 0x563a0e463e20 2023-07-18T03:32:30.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4c6ff5700 1 -- 172.21.15.40:0/506996871 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb4d8040080 con 0x563a0e463e20 2023-07-18T03:32:30.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/506996871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4d40a2300 con 0x563a0e463e20 2023-07-18T03:32:30.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4c6ff5700 1 -- 172.21.15.40:0/506996871 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4d8004e80 con 0x563a0e463e20 2023-07-18T03:32:30.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4f51b4b80 10 client.16020 fetch_fsmap learned FSMap version 31 2023-07-18T03:32:30.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/506996871 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fb4e419dea0 con 0x563a0e463e20 2023-07-18T03:32:30.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.150+0000 7fb4c6ff5700 1 -- 172.21.15.40:0/506996871 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d40a2300 con 0x563a0e463e20 2023-07-18T03:32:30.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.151+0000 7fb4c6ff5700 1 -- 172.21.15.40:0/506996871 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fb4d802f070 con 0x563a0e463e20 2023-07-18T03:32:30.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.151+0000 7fb4f51b4b80 10 client.16020 fetch_fsmap finished waiting for FSMap version 31 2023-07-18T03:32:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.151+0000 7fb4f51b4b80 10 client.16020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:32:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.151+0000 7fb4f51b4b80 20 client.16020 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.151+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/506996871 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563a0e4cdda0 0x563a0e4d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.151+0000 7fb4f51b4b80 4 client.16020 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:32:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.151+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/506996871 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563a0e370050 con 0x563a0e4cdda0 2023-07-18T03:32:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.151+0000 7fb4e2d9d700 1 --2- 172.21.15.40:0/506996871 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563a0e4cdda0 0x563a0e4d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.151+0000 7fb4c6ff5700 10 client.16020 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.151+0000 7fb4e2d9d700 1 --2- 172.21.15.40:0/506996871 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563a0e4cdda0 0x563a0e4d01b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=803280c2667ecd1e server_cookie=2fc790aeeeba13e5 in_seq=0 out_seq=0 2023-07-18T03:32:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4c6ff5700 1 -- 172.21.15.40:0/506996871 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563a0e370050 con 0x563a0e4cdda0 2023-07-18T03:32:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4c6ff5700 10 client.16020 handle_command_reply: tid=0 2023-07-18T03:32:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4f51b4b80 1 client.16020 shutdown 2023-07-18T03:32:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/506996871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e463e20 msgr2=0x563a0e4744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/506996871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e463e20 0x563a0e4744f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8024e70 tx=0x7fb4d800f690 comp rx=0 tx=0).stop 2023-07-18T03:32:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/506996871 shutdown_connections 2023-07-18T03:32:30.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/506996871 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563a0e4cdda0 0x563a0e4d01b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/506996871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a0e464c70 0x563a0e4725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/506996871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e4634d0 0x563a0e473fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4c6ff5700 0 client.16020 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:30.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4f51b4b80 1 --2- 172.21.15.40:0/506996871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563a0e463e20 0x563a0e4744f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/506996871 >> 172.21.15.40:0/506996871 conn(0x563a0e458da0 msgr2=0x563a0e45d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:30.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/506996871 shutdown_connections 2023-07-18T03:32:30.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4f51b4b80 1 -- 172.21.15.40:0/506996871 wait complete. 2023-07-18T03:32:30.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4f51b4b80 20 client.16020 trim_cache size 0 max 16384 2023-07-18T03:32:30.164 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:32:30.165 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "scrub active (0 inodes in the stack)", 2023-07-18T03:32:30.165 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:32:30.165 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:32:30.165 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:32:30.165 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:32:30.165 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:32:30.165 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:32:30.166 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:32:30.166 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:32:30.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4e902e700 1 -- 172.21.15.40:0/116993120 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb4cc0a1150 msgr2=0x7fb4cc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:30.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4e902e700 1 --2- 172.21.15.40:0/116993120 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb4cc0a1150 0x7fb4cc0a3600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800b2e0 tx=0x7fb4d8005960 comp rx=0 tx=0).stop 2023-07-18T03:32:30.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4e902e700 1 -- 172.21.15.40:0/116993120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4e40ffb10 msgr2=0x7fb4e4195810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:30.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4e902e700 1 --2- 172.21.15.40:0/116993120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4e40ffb10 0x7fb4e4195810 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400e970 tx=0x7fb4d400ec80 comp rx=0 tx=0).stop 2023-07-18T03:32:30.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4e902e700 1 -- 172.21.15.40:0/116993120 shutdown_connections 2023-07-18T03:32:30.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4e902e700 1 --2- 172.21.15.40:0/116993120 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb4cc0a1150 0x7fb4cc0a3600 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4e902e700 1 --2- 172.21.15.40:0/116993120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4e4101690 0x7fb4e419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4e902e700 1 --2- 172.21.15.40:0/116993120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4e40ffb10 0x7fb4e4195810 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4e902e700 1 --2- 172.21.15.40:0/116993120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb4e4100d00 0x7fb4e4195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:30.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.152+0000 7fb4e902e700 1 -- 172.21.15.40:0/116993120 >> 172.21.15.40:0/116993120 conn(0x7fb4e40fb130 msgr2=0x7fb4e40fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:30.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.153+0000 7fb4e902e700 1 -- 172.21.15.40:0/116993120 shutdown_connections 2023-07-18T03:32:30.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:32:30.153+0000 7fb4e902e700 1 -- 172.21.15.40:0/116993120 wait complete. 2023-07-18T03:32:30.169 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:32:30.780 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:30.434+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005e8 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench.c auth v62 ap=1 s=2117 n(v0 rc2005-02-11T14:49:01.000000+0000 b2117 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55bde700a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005e8:[<0x100000007de/blogbench.c v6>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000005e8:[<0x100000007de/blogbench.c v62>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:30.780 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:30.441+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005f2 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/globals.h auth v52 ap=1 s=1419 n(v0 rc2005-02-11T13:50:02.000000+0000 b1419 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55bde700bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005f2:[<0x100000007de/globals.h v26>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000005f2:[<0x100000007de/globals.h v52>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:30.781 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:30.442+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005f6 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/writer.c auth v42 ap=1 s=1910 n(v0 rc2005-02-11T14:29:28.000000+0000 b1910 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55bde700c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005f6:[<0x100000007de/writer.c v34>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000005f6:[<0x100000007de/writer.c v42>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:30.781 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:30.442+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007df [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/mysnprintf.c auth v56 ap=1 s=729 n(v0 rc2023-07-18T03:31:54.444517+0000 b729 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55bde700c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007df:[<0x100000007de/mysnprintf.c v2>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000007df:[<0x100000007de/mysnprintf.c v56>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:31.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:30 smithi134 ceph-mon[98216]: scrub queued for path: / 2023-07-18T03:32:31.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:30 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:31.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:30 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:32:31.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:30 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:32:31.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:30 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:31.068 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:30.882+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ee [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h auth v60 ap=1 s=4944 n(v0 rc2005-02-11T11:45:33.000000+0000 b4944 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55bde700cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005ee:[<0x100000007de/bsd-getopt_long.h v18>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000005ee:[<0x100000007de/bsd-getopt_long.h v60>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:31.068 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:30.891+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005f3 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/commenter.c auth v40 ap=1 s=939 n(v0 rc2005-02-11T12:58:54.000000+0000 b939 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55bde700d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005f3:[<0x100000007de/commenter.c v28>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000005f3:[<0x100000007de/commenter.c v40>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:31.068 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:30.901+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005f7 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/helpers.c auth v70 ap=1 s=4200 n(v0 rc2005-02-11T14:46:26.000000+0000 b4200 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55bde700d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005f7:[<0x100000007de/helpers.c v36>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000005f7:[<0x100000007de/helpers.c v70>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:31.068 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:30.902+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005e7 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/mysnprintf.h auth{0=1} v38 ap=1 s=696 n(v0 rc2005-02-11T11:45:33.000000+0000 b696 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x55bde7012000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005e7:[<0x100000007de/mysnprintf.h v4>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000005e7:[<0x100000007de/mysnprintf.h v38>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:31.069 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:30.904+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ec [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench_p.h auth v58 ap=1 s=460 n(v0 rc2005-02-11T13:43:53.000000+0000 b460 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55bde7012b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005ec:[<0x100000007de/blogbench_p.h v14>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000005ec:[<0x100000007de/blogbench_p.h v58>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:31.069 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:30.905+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005e9 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench.h auth v54 ap=1 s=3764 n(v0 rc2005-02-11T19:50:06.000000+0000 b3764 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55bde7013600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005e9:[<0x100000007de/blogbench.h v8>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000005e9:[<0x100000007de/blogbench.h v54>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:31.069 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:30.905+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005f4 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/rewriter.c auth v48 ap=1 s=1581 n(v0 rc2005-02-11T12:50:44.000000+0000 b1581 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55bde7013b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005f4:[<0x100000007de/rewriter.c v30>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000005f4:[<0x100000007de/rewriter.c v48>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:31.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:30 smithi040 ceph-mon[96439]: scrub queued for path: / 2023-07-18T03:32:31.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:30 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:31.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:30 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:32:31.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:30 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:32:31.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:30 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:31.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:30 smithi064 ceph-mon[98059]: scrub queued for path: / 2023-07-18T03:32:31.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:30 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:31.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:30 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:32:31.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:30 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:32:31.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:30 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:31.567 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:31.234+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005f0 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/Makefile.in auth v50 ap=1 s=13775 n(v0 rc2005-02-11T20:29:05.000000+0000 b13775 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/pFscr@1},l=15309 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55bde7014c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005f0:[<0x100000007de/Makefile.in v22>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000005f0:[<0x100000007de/Makefile.in v50>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:31.567 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:31.236+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ef [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/Makefile.am auth v72 ap=1 s=299 n(v0 rc2005-02-11T14:07:02.000000+0000 b299 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55bde7015700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005ef:[<0x100000007de/Makefile.am v20>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000005ef:[<0x100000007de/Makefile.am v72>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:31.567 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:31.238+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ea [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/fakesnprintf.c auth v64 ap=1 s=10682 n(v0 rc2005-02-11T11:54:44.000000+0000 b10682 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55bde7015180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005ea:[<0x100000007de/fakesnprintf.c v10>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000005ea:[<0x100000007de/fakesnprintf.c v64>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:31.567 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:31.244+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ed [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c auth v68 ap=1 s=18847 n(v0 rc2005-02-11T11:48:28.000000+0000 b18847 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55bde6e6c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005ed:[<0x100000007de/bsd-getopt_long.c v16>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000005ed:[<0x100000007de/bsd-getopt_long.c v68>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:31.568 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:31.248+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005f5 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/reader.c auth v44 ap=1 s=1605 n(v0 rc2005-02-11T10:45:16.000000+0000 b1605 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55bde6e6cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005f5:[<0x100000007de/reader.c v32>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000005f5:[<0x100000007de/reader.c v44>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:31.568 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:31.250+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005f1 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/process.c auth v46 ap=1 s=4649 n(v0 rc2005-02-11T21:32:46.000000+0000 b4649 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55bde6e6d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005f1:[<0x100000007de/process.c v24>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000005f1:[<0x100000007de/process.c v46>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:31.568 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:32:31.257+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005eb [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/fakesnprintf.h auth v66 ap=1 s=606 n(v0 rc2005-02-11T11:45:33.000000+0000 b606 1=1+0) (iversion lock) caps={15309=pAsLsXsFscr/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55bde6e6db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005eb:[<0x100000007de/fakesnprintf.h v12>,<0x100000007d9/src v50>,<0x100000005e3/blogbench-1.0 v416>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000005eb:[<0x100000007de/fakesnprintf.h v66>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v41>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v51>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:32:32.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: pgmap v342: 97 pgs: 97 active+clean; 55 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 93 op/s 2023-07-18T03:32:32.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000005e8 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000005f2 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/globals.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000005f6 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/writer.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000007df (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/mysnprintf.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000005ee (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000005f3 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/commenter.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000005f7 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/helpers.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000005e7 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/mysnprintf.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000005ec (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench_p.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000005e9 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000005f4 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/rewriter.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:32:32.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:32:32.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:32.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:32.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:32:32.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:31 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:32.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: pgmap v342: 97 pgs: 97 active+clean; 55 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 93 op/s 2023-07-18T03:32:32.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000005e8 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000005f2 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/globals.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000005f6 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/writer.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000007df (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/mysnprintf.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000005ee (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000005f3 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/commenter.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000005f7 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/helpers.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000005e7 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/mysnprintf.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000005ec (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench_p.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000005e9 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000005f4 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/rewriter.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:32:32.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:32:32.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:32.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:32.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:32:32.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:31 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:32.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: pgmap v342: 97 pgs: 97 active+clean; 55 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 93 op/s 2023-07-18T03:32:32.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000005e8 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000005f2 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/globals.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000005f6 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/writer.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000007df (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/mysnprintf.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000005ee (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000005f3 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/commenter.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000005f7 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/helpers.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000005e7 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/mysnprintf.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000005ec (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench_p.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000005e9 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000005f4 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/rewriter.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:32.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:32:32.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:32:32.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:32.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:32.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:32:32.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:31 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:33.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:32 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000005f0 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/Makefile.in) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:32 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000005ef (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/Makefile.am) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:32 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000005ea (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/fakesnprintf.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:32 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000005ed (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:32 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000005f5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/reader.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:32 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000005f1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/process.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:32 smithi134 ceph-mon[98216]: Scrub error on inode 0x100000005eb (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/fakesnprintf.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:32 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:32:33.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:32 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:33.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:32 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000005f0 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/Makefile.in) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:32 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000005ef (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/Makefile.am) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:32 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000005ea (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/fakesnprintf.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:32 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000005ed (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:32 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000005f5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/reader.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:32 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000005f1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/process.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:32 smithi040 ceph-mon[96439]: Scrub error on inode 0x100000005eb (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/fakesnprintf.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:32 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:32:33.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:32 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:33.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:32 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000005f0 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/Makefile.in) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:32 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000005ef (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/Makefile.am) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:32 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000005ea (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/fakesnprintf.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:32 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000005ed (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:32 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000005f5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/reader.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:32 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000005f1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/process.c) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:32 smithi064 ceph-mon[98059]: Scrub error on inode 0x100000005eb (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/fakesnprintf.h) see mds.i log and `damage ls` output for details 2023-07-18T03:32:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:32 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:32:33.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:32 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:32:33.144 INFO:tasks.workunit.client.0.smithi040.stdout: 36 587736 971 361473 928 201371 570 2023-07-18T03:32:34.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:33 smithi064 ceph-mon[98059]: pgmap v343: 97 pgs: 97 active+clean; 61 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 107 op/s 2023-07-18T03:32:34.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:33 smithi134 ceph-mon[98216]: pgmap v343: 97 pgs: 97 active+clean; 61 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 107 op/s 2023-07-18T03:32:34.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:33 smithi040 ceph-mon[96439]: pgmap v343: 97 pgs: 97 active+clean; 61 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 107 op/s 2023-07-18T03:32:35.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:34 smithi134 ceph-mon[98216]: pgmap v344: 97 pgs: 97 active+clean; 62 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 94 op/s 2023-07-18T03:32:35.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:34 smithi134 ceph-mon[98216]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-18T03:32:35.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:34 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] up:active 2023-07-18T03:32:35.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:34 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:32:35.338 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:34 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mon-a[96415]: 2023-07-18T03:32:34.839+0000 7f858afe7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-18T03:32:35.338 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:34 smithi040 ceph-mon[96439]: pgmap v344: 97 pgs: 97 active+clean; 62 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 94 op/s 2023-07-18T03:32:35.338 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:34 smithi040 ceph-mon[96439]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-18T03:32:35.338 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:34 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] up:active 2023-07-18T03:32:35.339 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:34 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:32:35.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:34 smithi064 ceph-mon[98059]: pgmap v344: 97 pgs: 97 active+clean; 62 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 94 op/s 2023-07-18T03:32:35.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:34 smithi064 ceph-mon[98059]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-18T03:32:35.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:34 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] up:active 2023-07-18T03:32:35.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:34 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:32:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.425+0000 7f4724291700 1 -- 172.21.15.40:0/1960511555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f471c0ffb30 msgr2=0x7f471c0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.425+0000 7f4724291700 1 --2- 172.21.15.40:0/1960511555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f471c0ffb30 0x7f471c0fff40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4718009990 tx=0x7f4718009ca0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.425+0000 7f4724291700 1 -- 172.21.15.40:0/1960511555 shutdown_connections 2023-07-18T03:32:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.425+0000 7f4724291700 1 --2- 172.21.15.40:0/1960511555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f471c1016b0 0x7f471c107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.425+0000 7f4724291700 1 --2- 172.21.15.40:0/1960511555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f471c0ffb30 0x7f471c0fff40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.425+0000 7f4724291700 1 --2- 172.21.15.40:0/1960511555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f471c100d20 0x7f471c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.425+0000 7f4724291700 1 -- 172.21.15.40:0/1960511555 >> 172.21.15.40:0/1960511555 conn(0x7f471c0fb130 msgr2=0x7f471c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.425+0000 7f4724291700 1 -- 172.21.15.40:0/1960511555 shutdown_connections 2023-07-18T03:32:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.425+0000 7f4724291700 1 -- 172.21.15.40:0/1960511555 wait complete. 2023-07-18T03:32:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.426+0000 7f4724291700 1 Processor -- start 2023-07-18T03:32:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.426+0000 7f4724291700 1 -- start start 2023-07-18T03:32:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.426+0000 7f4724291700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f471c0ffb30 0x7f471c1959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.426+0000 7f4724291700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f471c100d20 0x7f471c195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.426+0000 7f4724291700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f471c1016b0 0x7f471c196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.426+0000 7f4724291700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c1973f0 con 0x7f471c1016b0 2023-07-18T03:32:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.426+0000 7f4724291700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c197560 con 0x7f471c100d20 2023-07-18T03:32:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.426+0000 7f4724291700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c197720 con 0x7f471c0ffb30 2023-07-18T03:32:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.426+0000 7f472282e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f471c1016b0 0x7f471c196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.426+0000 7f472202d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f471c0ffb30 0x7f471c1959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.426+0000 7f472202d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f471c0ffb30 0x7f471c1959d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48348/0 (socket says 172.21.15.40:48348) 2023-07-18T03:32:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.426+0000 7f472182c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f471c100d20 0x7f471c195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.426+0000 7f472282e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f471c1016b0 0x7f471c196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59324/0 (socket says 172.21.15.40:59324) 2023-07-18T03:32:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.426+0000 7f472202d700 1 -- 172.21.15.40:0/3541691616 learned_addr learned my addr 172.21.15.40:0/3541691616 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.427+0000 7f472202d700 1 -- 172.21.15.40:0/3541691616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f471c100d20 msgr2=0x7f471c195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.427+0000 7f472202d700 1 --2- 172.21.15.40:0/3541691616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f471c100d20 0x7f471c195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.427+0000 7f472202d700 1 -- 172.21.15.40:0/3541691616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f471c1016b0 msgr2=0x7f471c196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.427+0000 7f472202d700 1 --2- 172.21.15.40:0/3541691616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f471c1016b0 0x7f471c196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.427+0000 7f472202d700 1 -- 172.21.15.40:0/3541691616 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47180095f0 con 0x7f471c0ffb30 2023-07-18T03:32:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.427+0000 7f472282e700 1 --2- 172.21.15.40:0/3541691616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f471c1016b0 0x7f471c196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.427+0000 7f472202d700 1 --2- 172.21.15.40:0/3541691616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f471c0ffb30 0x7f471c1959d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f471800b620 tx=0x7f4718004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.427+0000 7f47137fe700 1 -- 172.21.15.40:0/3541691616 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471801d070 con 0x7f471c0ffb30 2023-07-18T03:32:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.427+0000 7f47137fe700 1 -- 172.21.15.40:0/3541691616 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4718005030 con 0x7f471c0ffb30 2023-07-18T03:32:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.427+0000 7f4724291700 1 -- 172.21.15.40:0/3541691616 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471c1979a0 con 0x7f471c0ffb30 2023-07-18T03:32:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.427+0000 7f47137fe700 1 -- 172.21.15.40:0/3541691616 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4718004b80 con 0x7f471c0ffb30 2023-07-18T03:32:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.427+0000 7f4724291700 1 -- 172.21.15.40:0/3541691616 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471c19e1c0 con 0x7f471c0ffb30 2023-07-18T03:32:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.429+0000 7f47137fe700 1 -- 172.21.15.40:0/3541691616 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f47180051a0 con 0x7f471c0ffb30 2023-07-18T03:32:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.430+0000 7f47137fe700 1 --2- 172.21.15.40:0/3541691616 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f47080a1150 0x7f47080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.430+0000 7f47137fe700 1 -- 172.21.15.40:0/3541691616 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f47180cd960 con 0x7f471c0ffb30 2023-07-18T03:32:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.430+0000 7f472182c700 1 --2- 172.21.15.40:0/3541691616 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f47080a1150 0x7f47080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.431+0000 7f472182c700 1 --2- 172.21.15.40:0/3541691616 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f47080a1150 0x7f47080a3600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f470c009c60 tx=0x7f470c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.433+0000 7f4730417b80 1 Processor -- start 2023-07-18T03:32:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.433+0000 7f4730417b80 1 -- start start 2023-07-18T03:32:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.433+0000 7f4730417b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da25a16b0 0x560da2749710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.433+0000 7f4730417b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 0x560da274a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.433+0000 7f4730417b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da274aaf0 0x560da274af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.433+0000 7f4730417b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560da264ac50 con 0x560da274aaf0 2023-07-18T03:32:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.433+0000 7f4730417b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560da269d9e0 con 0x560da25a16b0 2023-07-18T03:32:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.433+0000 7f4730417b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560da2696e70 con 0x560da2749c50 2023-07-18T03:32:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.434+0000 7f472182c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 0x560da274a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.434+0000 7f472202d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da25a16b0 0x560da2749710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.434+0000 7f472282e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da274aaf0 0x560da274af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.434+0000 7f472182c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 0x560da274a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48358/0 (socket says 172.21.15.40:48358) 2023-07-18T03:32:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.434+0000 7f472202d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da25a16b0 0x560da2749710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42654/0 (socket says 172.21.15.40:42654) 2023-07-18T03:32:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.434+0000 7f472182c700 1 -- 172.21.15.40:0/1097673029 learned_addr learned my addr 172.21.15.40:0/1097673029 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.434+0000 7f472182c700 1 -- 172.21.15.40:0/1097673029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da25a16b0 msgr2=0x560da2749710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.434+0000 7f472182c700 1 --2- 172.21.15.40:0/1097673029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da25a16b0 0x560da2749710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.434+0000 7f472182c700 1 -- 172.21.15.40:0/1097673029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da274aaf0 msgr2=0x560da274af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.434+0000 7f472182c700 1 --2- 172.21.15.40:0/1097673029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da274aaf0 0x560da274af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.434+0000 7f472182c700 1 -- 172.21.15.40:0/1097673029 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560da24c4d80 con 0x560da2749c50 2023-07-18T03:32:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.434+0000 7f472182c700 1 --2- 172.21.15.40:0/1097673029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 0x560da274a5b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f470c00e4e0 tx=0x7f470c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dd2ce817e9e635d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.434+0000 7f4710dac700 1 -- 172.21.15.40:0/1097673029 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c00ee00 con 0x560da2749c50 2023-07-18T03:32:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.434+0000 7f4710dac700 1 -- 172.21.15.40:0/1097673029 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f470c021070 con 0x560da2749c50 2023-07-18T03:32:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.434+0000 7f4710dac700 1 -- 172.21.15.40:0/1097673029 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c01d810 con 0x560da2749c50 2023-07-18T03:32:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.435+0000 7f4730417b80 1 -- 172.21.15.40:0/1097673029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 msgr2=0x560da274a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.435+0000 7f4730417b80 1 --2- 172.21.15.40:0/1097673029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 0x560da274a5b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f470c00e4e0 tx=0x7f470c014000 comp rx=0 tx=0).stop 2023-07-18T03:32:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.435+0000 7f4730417b80 1 -- 172.21.15.40:0/1097673029 shutdown_connections 2023-07-18T03:32:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.435+0000 7f4730417b80 1 --2- 172.21.15.40:0/1097673029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da274aaf0 0x560da274af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.435+0000 7f4730417b80 1 --2- 172.21.15.40:0/1097673029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da25a16b0 0x560da2749710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.436+0000 7f4730417b80 1 --2- 172.21.15.40:0/1097673029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 0x560da274a5b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.436+0000 7f4730417b80 1 -- 172.21.15.40:0/1097673029 >> 172.21.15.40:0/1097673029 conn(0x560da273abe0 msgr2=0x560da273fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.436+0000 7f4730417b80 1 -- 172.21.15.40:0/1097673029 shutdown_connections 2023-07-18T03:32:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.436+0000 7f4730417b80 1 -- 172.21.15.40:0/1097673029 wait complete. 2023-07-18T03:32:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.436+0000 7f4730417b80 1 -- start start 2023-07-18T03:32:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.436+0000 7f4730417b80 1 Processor -- start 2023-07-18T03:32:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f4730417b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da25a16b0 0x560da2752c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f4730417b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 0x560da2753190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f4730417b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da274aaf0 0x560da275a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f4730417b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560da2636a60 con 0x560da274aaf0 2023-07-18T03:32:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f4730417b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560da2636760 con 0x560da25a16b0 2023-07-18T03:32:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f4730417b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560da251a8c0 con 0x560da2749c50 2023-07-18T03:32:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f4730417b80 10 client.0 _ll_register_callbacks cb 0x560da2634700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f472202d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da25a16b0 0x560da2752c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f472182c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 0x560da2753190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f472182c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 0x560da2753190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48362/0 (socket says 172.21.15.40:48362) 2023-07-18T03:32:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f472202d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da25a16b0 0x560da2752c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42662/0 (socket says 172.21.15.40:42662) 2023-07-18T03:32:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f472182c700 1 -- 172.21.15.40:0/448203718 learned_addr learned my addr 172.21.15.40:0/448203718 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f472282e700 1 --2- 172.21.15.40:0/448203718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da274aaf0 0x560da275a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f472182c700 1 -- 172.21.15.40:0/448203718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da25a16b0 msgr2=0x560da2752c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f472182c700 1 --2- 172.21.15.40:0/448203718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da25a16b0 0x560da2752c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f472182c700 1 -- 172.21.15.40:0/448203718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da274aaf0 msgr2=0x560da275a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f472182c700 1 --2- 172.21.15.40:0/448203718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da274aaf0 0x560da275a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f472182c700 1 -- 172.21.15.40:0/448203718 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470c017070 con 0x560da2749c50 2023-07-18T03:32:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f472282e700 1 --2- 172.21.15.40:0/448203718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da274aaf0 0x560da275a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:32:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f472182c700 1 --2- 172.21.15.40:0/448203718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 0x560da2753190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f470c00ed90 tx=0x7f470c00edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=163b2ab7b3f6d9a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.437+0000 7f4705ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:32:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.438+0000 7f4705ffb700 1 -- 172.21.15.40:0/448203718 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c005350 con 0x560da2749c50 2023-07-18T03:32:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.438+0000 7f4705ffb700 1 -- 172.21.15.40:0/448203718 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f470c0054b0 con 0x560da2749c50 2023-07-18T03:32:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.438+0000 7f4730417b80 1 -- 172.21.15.40:0/448203718 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560da25f0490 con 0x560da2749c50 2023-07-18T03:32:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.438+0000 7f4705ffb700 1 -- 172.21.15.40:0/448203718 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f470c005670 con 0x560da2749c50 2023-07-18T03:32:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.438+0000 7f4730417b80 10 client.25672 fetch_fsmap learned FSMap version 32 2023-07-18T03:32:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.438+0000 7f4730417b80 1 -- 172.21.15.40:0/448203718 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560da24adc00 con 0x560da2749c50 2023-07-18T03:32:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.438+0000 7f4705ffb700 1 -- 172.21.15.40:0/448203718 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f470c03d080 con 0x560da2749c50 2023-07-18T03:32:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.438+0000 7f4705ffb700 1 -- 172.21.15.40:0/448203718 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f470c037030 con 0x560da2749c50 2023-07-18T03:32:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.438+0000 7f4730417b80 10 client.25672 fetch_fsmap finished waiting for FSMap version 32 2023-07-18T03:32:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.438+0000 7f4730417b80 10 client.25672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:32:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.438+0000 7f4730417b80 20 client.25672 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.438+0000 7f4730417b80 1 --2- 172.21.15.40:0/448203718 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560da275e940 0x560da2760d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.438+0000 7f4730417b80 4 client.25672 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:32:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.438+0000 7f4730417b80 1 -- 172.21.15.40:0/448203718 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560da268fe50 con 0x560da275e940 2023-07-18T03:32:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.439+0000 7f472202d700 1 --2- 172.21.15.40:0/448203718 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560da275e940 0x560da2760d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.439+0000 7f4705ffb700 10 client.25672 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.439+0000 7f472202d700 1 --2- 172.21.15.40:0/448203718 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560da275e940 0x560da2760d50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7018033cb0cc601 server_cookie=e74a21834a97783f in_seq=0 out_seq=0 2023-07-18T03:32:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.441+0000 7f4705ffb700 1 -- 172.21.15.40:0/448203718 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560da268fe50 con 0x560da275e940 2023-07-18T03:32:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.441+0000 7f4705ffb700 10 client.25672 handle_command_reply: tid=0 2023-07-18T03:32:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.441+0000 7f4730417b80 1 client.25672 shutdown 2023-07-18T03:32:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.441+0000 7f4730417b80 1 -- 172.21.15.40:0/448203718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 msgr2=0x560da2753190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.441+0000 7f4730417b80 1 --2- 172.21.15.40:0/448203718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 0x560da2753190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f470c00ed90 tx=0x7f470c00edc0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.441+0000 7f4730417b80 1 -- 172.21.15.40:0/448203718 shutdown_connections 2023-07-18T03:32:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.441+0000 7f4730417b80 1 --2- 172.21.15.40:0/448203718 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560da275e940 0x560da2760d50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.441+0000 7f4730417b80 1 --2- 172.21.15.40:0/448203718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da274aaf0 0x560da275a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.441+0000 7f4705ffb700 0 client.25672 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.441+0000 7f4730417b80 1 --2- 172.21.15.40:0/448203718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da25a16b0 0x560da2752c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.441+0000 7f4730417b80 1 --2- 172.21.15.40:0/448203718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 0x560da2753190 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.441+0000 7f4730417b80 1 -- 172.21.15.40:0/448203718 >> 172.21.15.40:0/448203718 conn(0x560da273abe0 msgr2=0x560da273f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.441+0000 7f4730417b80 1 -- 172.21.15.40:0/448203718 shutdown_connections 2023-07-18T03:32:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.441+0000 7f4730417b80 1 -- 172.21.15.40:0/448203718 wait complete. 2023-07-18T03:32:36.451 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.442+0000 7f4730417b80 20 client.25672 trim_cache size 0 max 16384 2023-07-18T03:32:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.458+0000 7f4730417b80 1 Processor -- start 2023-07-18T03:32:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.458+0000 7f4730417b80 1 -- start start 2023-07-18T03:32:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.458+0000 7f4730417b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da25a16b0 0x560da2765670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.458+0000 7f4730417b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da2749c50 0x560da275a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.458+0000 7f4730417b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da274aaf0 0x560da2752c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.458+0000 7f4730417b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560da27538d0 con 0x560da25a16b0 2023-07-18T03:32:36.457 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.458+0000 7f4730417b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560da2751720 con 0x560da2749c50 2023-07-18T03:32:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.458+0000 7f4730417b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560da2750c40 con 0x560da274aaf0 2023-07-18T03:32:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.458+0000 7f472202d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da25a16b0 0x560da2765670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.458+0000 7f472182c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da2749c50 0x560da275a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.458+0000 7f472282e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da274aaf0 0x560da2752c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.458 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.458+0000 7f472182c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da2749c50 0x560da275a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42670/0 (socket says 172.21.15.40:42670) 2023-07-18T03:32:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.458+0000 7f472202d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da25a16b0 0x560da2765670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59342/0 (socket says 172.21.15.40:59342) 2023-07-18T03:32:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.458+0000 7f472182c700 1 -- 172.21.15.40:0/4231348541 learned_addr learned my addr 172.21.15.40:0/4231348541 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.459+0000 7f472182c700 1 -- 172.21.15.40:0/4231348541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da274aaf0 msgr2=0x560da2752c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.459+0000 7f472182c700 1 --2- 172.21.15.40:0/4231348541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da274aaf0 0x560da2752c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.459+0000 7f472182c700 1 -- 172.21.15.40:0/4231348541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da25a16b0 msgr2=0x560da2765670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.459+0000 7f472182c700 1 --2- 172.21.15.40:0/4231348541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da25a16b0 0x560da2765670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.459 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.459+0000 7f472182c700 1 -- 172.21.15.40:0/4231348541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560da24eb1f0 con 0x560da2749c50 2023-07-18T03:32:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.459+0000 7f472202d700 1 --2- 172.21.15.40:0/4231348541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da25a16b0 0x560da2765670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.459+0000 7f472282e700 1 --2- 172.21.15.40:0/4231348541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da274aaf0 0x560da2752c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.459+0000 7f472182c700 1 --2- 172.21.15.40:0/4231348541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da2749c50 0x560da275a190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f470c016040 tx=0x7f470c01ca80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73cb9f3eab5ac296 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.459+0000 7f47115ad700 1 -- 172.21.15.40:0/4231348541 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c01cb50 con 0x560da2749c50 2023-07-18T03:32:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.459+0000 7f47115ad700 1 -- 172.21.15.40:0/4231348541 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560da25f0490 con 0x560da2749c50 2023-07-18T03:32:36.460 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.459+0000 7f47115ad700 1 -- 172.21.15.40:0/4231348541 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c003a00 con 0x560da2749c50 2023-07-18T03:32:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.460+0000 7f4730417b80 1 -- 172.21.15.40:0/4231348541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da2749c50 msgr2=0x560da275a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.460+0000 7f4730417b80 1 --2- 172.21.15.40:0/4231348541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da2749c50 0x560da275a190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f470c016040 tx=0x7f470c01ca80 comp rx=0 tx=0).stop 2023-07-18T03:32:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.460+0000 7f4730417b80 1 -- 172.21.15.40:0/4231348541 shutdown_connections 2023-07-18T03:32:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.460+0000 7f4730417b80 1 --2- 172.21.15.40:0/4231348541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da274aaf0 0x560da2752c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.460+0000 7f4730417b80 1 --2- 172.21.15.40:0/4231348541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da25a16b0 0x560da2765670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.460+0000 7f4730417b80 1 --2- 172.21.15.40:0/4231348541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da2749c50 0x560da275a190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.460+0000 7f4730417b80 1 -- 172.21.15.40:0/4231348541 >> 172.21.15.40:0/4231348541 conn(0x560da273abe0 msgr2=0x560da27519b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:36.464 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.460+0000 7f4730417b80 1 -- 172.21.15.40:0/4231348541 shutdown_connections 2023-07-18T03:32:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.460+0000 7f4730417b80 1 -- 172.21.15.40:0/4231348541 wait complete. 2023-07-18T03:32:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.460+0000 7f4730417b80 1 -- start start 2023-07-18T03:32:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.460+0000 7f4730417b80 1 Processor -- start 2023-07-18T03:32:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.460+0000 7f4730417b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da25a16b0 0x560da27488d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.460+0000 7f4730417b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 0x560da2748e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.460+0000 7f4730417b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da274aaf0 0x560da2759db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.465 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f4730417b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560da275a4c0 con 0x560da25a16b0 2023-07-18T03:32:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f4730417b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560da2745390 con 0x560da274aaf0 2023-07-18T03:32:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f4730417b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560da2745550 con 0x560da2749c50 2023-07-18T03:32:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f4730417b80 10 client.0 _ll_register_callbacks cb 0x7f4714009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f472202d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da25a16b0 0x560da27488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f472202d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da25a16b0 0x560da27488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59348/0 (socket says 172.21.15.40:59348) 2023-07-18T03:32:36.466 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f472182c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 0x560da2748e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f472202d700 1 -- 172.21.15.40:0/2808283545 learned_addr learned my addr 172.21.15.40:0/2808283545 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f472282e700 1 --2- 172.21.15.40:0/2808283545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da274aaf0 0x560da2759db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f472202d700 1 -- 172.21.15.40:0/2808283545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 msgr2=0x560da2748e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f472202d700 1 --2- 172.21.15.40:0/2808283545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 0x560da2748e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f472202d700 1 -- 172.21.15.40:0/2808283545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da274aaf0 msgr2=0x560da2759db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f472202d700 1 --2- 172.21.15.40:0/2808283545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da274aaf0 0x560da2759db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:36.467 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f472202d700 1 -- 172.21.15.40:0/2808283545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471c19e1c0 con 0x560da25a16b0 2023-07-18T03:32:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f472202d700 1 --2- 172.21.15.40:0/2808283545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da25a16b0 0x560da27488d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4718097280 tx=0x7f4718005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb26220c7c7f3bd server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f4705ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:32:36.468 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f4705ffb700 1 -- 172.21.15.40:0/2808283545 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47180a3140 con 0x560da25a16b0 2023-07-18T03:32:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f4705ffb700 1 -- 172.21.15.40:0/2808283545 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4718099120 con 0x560da25a16b0 2023-07-18T03:32:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f4730417b80 1 -- 172.21.15.40:0/2808283545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f470c021070 con 0x560da25a16b0 2023-07-18T03:32:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f4705ffb700 1 -- 172.21.15.40:0/2808283545 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f471809afe0 con 0x560da25a16b0 2023-07-18T03:32:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f4730417b80 10 client.16038 fetch_fsmap learned FSMap version 32 2023-07-18T03:32:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.461+0000 7f4730417b80 1 -- 172.21.15.40:0/2808283545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f47140091c0 con 0x560da25a16b0 2023-07-18T03:32:36.469 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.462+0000 7f4705ffb700 1 -- 172.21.15.40:0/2808283545 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47180cb020 con 0x560da25a16b0 2023-07-18T03:32:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.462+0000 7f4705ffb700 1 -- 172.21.15.40:0/2808283545 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f47180a2cb0 con 0x560da25a16b0 2023-07-18T03:32:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.462+0000 7f4730417b80 10 client.16038 fetch_fsmap finished waiting for FSMap version 32 2023-07-18T03:32:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.462+0000 7f4730417b80 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:32:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.462+0000 7f4730417b80 20 client.16038 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.462+0000 7f4730417b80 1 --2- 172.21.15.40:0/2808283545 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560da27b79b0 0x560da27b9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.462+0000 7f4730417b80 4 client.16038 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:32:36.471 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.462+0000 7f4730417b80 1 -- 172.21.15.40:0/2808283545 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560da2651ff0 con 0x560da27b79b0 2023-07-18T03:32:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.462+0000 7f472182c700 1 --2- 172.21.15.40:0/2808283545 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560da27b79b0 0x560da27b9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.462+0000 7f4705ffb700 10 client.16038 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:36.462+0000 7f472182c700 1 --2- 172.21.15.40:0/2808283545 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560da27b79b0 0x560da27b9dc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d928312e56df2a9 server_cookie=713d90e0bcf0c071 in_seq=0 out_seq=0 2023-07-18T03:32:37.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:36 smithi134 ceph-mon[98216]: pgmap v345: 97 pgs: 97 active+clean; 96 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 232 op/s 2023-07-18T03:32:37.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:36 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:32:37.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:36 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:32:37.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:36 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:32:37.318 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:36 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:32:37.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:36 smithi040 ceph-mon[96439]: pgmap v345: 97 pgs: 97 active+clean; 96 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 232 op/s 2023-07-18T03:32:37.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:36 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:32:37.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:36 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:32:37.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:36 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:32:37.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:36 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:32:37.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:36 smithi064 ceph-mon[98059]: pgmap v345: 97 pgs: 97 active+clean; 96 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 232 op/s 2023-07-18T03:32:37.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:36 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:32:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:36 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:32:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:36 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:32:37.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:36 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:32:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.568+0000 7f4705ffb700 1 -- 172.21.15.40:0/2808283545 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560da2651ff0 con 0x560da27b79b0 2023-07-18T03:32:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.568+0000 7f4705ffb700 10 client.16038 handle_command_reply: tid=0 2023-07-18T03:32:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.568+0000 7f4730417b80 1 client.16038 shutdown 2023-07-18T03:32:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.568+0000 7f4730417b80 1 -- 172.21.15.40:0/2808283545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da25a16b0 msgr2=0x560da27488d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.568+0000 7f4730417b80 1 --2- 172.21.15.40:0/2808283545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da25a16b0 0x560da27488d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4718097280 tx=0x7f4718005ef0 comp rx=0 tx=0).stop 2023-07-18T03:32:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4730417b80 1 -- 172.21.15.40:0/2808283545 shutdown_connections 2023-07-18T03:32:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4730417b80 1 --2- 172.21.15.40:0/2808283545 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560da27b79b0 0x560da27b9dc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4730417b80 1 --2- 172.21.15.40:0/2808283545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560da274aaf0 0x560da2759db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4730417b80 1 --2- 172.21.15.40:0/2808283545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560da25a16b0 0x560da27488d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4705ffb700 0 client.16038 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4730417b80 1 --2- 172.21.15.40:0/2808283545 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560da2749c50 0x560da2748e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4730417b80 1 -- 172.21.15.40:0/2808283545 >> 172.21.15.40:0/2808283545 conn(0x560da273abe0 msgr2=0x560da2765670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4730417b80 1 -- 172.21.15.40:0/2808283545 shutdown_connections 2023-07-18T03:32:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4730417b80 1 -- 172.21.15.40:0/2808283545 wait complete. 2023-07-18T03:32:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4730417b80 20 client.16038 trim_cache size 0 max 16384 2023-07-18T03:32:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:32:37.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:32:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:32:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:32:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4724291700 1 -- 172.21.15.40:0/3541691616 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f47080a1150 msgr2=0x7f47080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4724291700 1 --2- 172.21.15.40:0/3541691616 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f47080a1150 0x7f47080a3600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f470c009c60 tx=0x7f470c009260 comp rx=0 tx=0).stop 2023-07-18T03:32:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4724291700 1 -- 172.21.15.40:0/3541691616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f471c0ffb30 msgr2=0x7f471c1959d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4724291700 1 --2- 172.21.15.40:0/3541691616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f471c0ffb30 0x7f471c1959d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f471800b620 tx=0x7f4718004730 comp rx=0 tx=0).stop 2023-07-18T03:32:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4724291700 1 -- 172.21.15.40:0/3541691616 shutdown_connections 2023-07-18T03:32:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4724291700 1 --2- 172.21.15.40:0/3541691616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f471c1016b0 0x7f471c196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4724291700 1 --2- 172.21.15.40:0/3541691616 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f47080a1150 0x7f47080a3600 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:37.571 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4724291700 1 --2- 172.21.15.40:0/3541691616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f471c0ffb30 0x7f471c1959d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4724291700 1 --2- 172.21.15.40:0/3541691616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f471c100d20 0x7f471c195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.569+0000 7f4724291700 1 -- 172.21.15.40:0/3541691616 >> 172.21.15.40:0/3541691616 conn(0x7f471c0fb130 msgr2=0x7f471c104250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.570+0000 7f4724291700 1 -- 172.21.15.40:0/3541691616 shutdown_connections 2023-07-18T03:32:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:37.570+0000 7f4724291700 1 -- 172.21.15.40:0/3541691616 wait complete. 2023-07-18T03:32:39.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:38 smithi134 ceph-mon[98216]: pgmap v346: 97 pgs: 97 active+clean; 149 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.1 MiB/s wr, 486 op/s 2023-07-18T03:32:39.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:38 smithi040 ceph-mon[96439]: pgmap v346: 97 pgs: 97 active+clean; 149 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.1 MiB/s wr, 486 op/s 2023-07-18T03:32:39.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:38 smithi064 ceph-mon[98059]: pgmap v346: 97 pgs: 97 active+clean; 149 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.1 MiB/s wr, 486 op/s 2023-07-18T03:32:41.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:40 smithi134 ceph-mon[98216]: pgmap v347: 97 pgs: 97 active+clean; 155 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.9 MiB/s wr, 497 op/s 2023-07-18T03:32:41.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:40 smithi040 ceph-mon[96439]: pgmap v347: 97 pgs: 97 active+clean; 155 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.9 MiB/s wr, 497 op/s 2023-07-18T03:32:41.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:40 smithi064 ceph-mon[98059]: pgmap v347: 97 pgs: 97 active+clean; 155 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.9 MiB/s wr, 497 op/s 2023-07-18T03:32:43.144 INFO:tasks.workunit.client.0.smithi040.stdout: 49 346423 653 218421 913 103634 428 2023-07-18T03:32:44.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:43 smithi134 ceph-mon[98216]: pgmap v348: 97 pgs: 97 active+clean; 157 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.3 MiB/s wr, 592 op/s 2023-07-18T03:32:44.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:43 smithi040 ceph-mon[96439]: pgmap v348: 97 pgs: 97 active+clean; 157 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.3 MiB/s wr, 592 op/s 2023-07-18T03:32:44.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:43 smithi064 ceph-mon[98059]: pgmap v348: 97 pgs: 97 active+clean; 157 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.3 MiB/s wr, 592 op/s 2023-07-18T03:32:46.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:45 smithi134 ceph-mon[98216]: pgmap v349: 97 pgs: 97 active+clean; 162 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.1 MiB/s wr, 588 op/s 2023-07-18T03:32:46.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:45 smithi040 ceph-mon[96439]: pgmap v349: 97 pgs: 97 active+clean; 162 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.1 MiB/s wr, 588 op/s 2023-07-18T03:32:46.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:45 smithi064 ceph-mon[98059]: pgmap v349: 97 pgs: 97 active+clean; 162 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.1 MiB/s wr, 588 op/s 2023-07-18T03:32:48.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:47 smithi134 ceph-mon[98216]: pgmap v350: 97 pgs: 97 active+clean; 163 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.2 MiB/s wr, 584 op/s 2023-07-18T03:32:48.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:47 smithi040 ceph-mon[96439]: pgmap v350: 97 pgs: 97 active+clean; 163 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.2 MiB/s wr, 584 op/s 2023-07-18T03:32:48.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:47 smithi064 ceph-mon[98059]: pgmap v350: 97 pgs: 97 active+clean; 163 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.2 MiB/s wr, 584 op/s 2023-07-18T03:32:50.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:49 smithi134 ceph-mon[98216]: pgmap v351: 97 pgs: 97 active+clean; 168 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 459 op/s 2023-07-18T03:32:50.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:49 smithi040 ceph-mon[96439]: pgmap v351: 97 pgs: 97 active+clean; 168 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 459 op/s 2023-07-18T03:32:50.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:49 smithi064 ceph-mon[98059]: pgmap v351: 97 pgs: 97 active+clean; 168 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 459 op/s 2023-07-18T03:32:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.687+0000 7fd0460fb700 1 -- 172.21.15.40:0/2767255387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0400ffb10 msgr2=0x7fd0400fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.687+0000 7fd0460fb700 1 --2- 172.21.15.40:0/2767255387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0400ffb10 0x7fd0400fff20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd0300098f0 tx=0x7fd030009c00 comp rx=0 tx=0).stop 2023-07-18T03:32:50.685 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.687+0000 7fd0460fb700 1 -- 172.21.15.40:0/2767255387 shutdown_connections 2023-07-18T03:32:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.687+0000 7fd0460fb700 1 --2- 172.21.15.40:0/2767255387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd040101690 0x7fd040107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.687+0000 7fd0460fb700 1 --2- 172.21.15.40:0/2767255387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd0400ffb10 0x7fd0400fff20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.687+0000 7fd0460fb700 1 --2- 172.21.15.40:0/2767255387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd040100d00 0x7fd040101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.687+0000 7fd0460fb700 1 -- 172.21.15.40:0/2767255387 >> 172.21.15.40:0/2767255387 conn(0x7fd0400fb130 msgr2=0x7fd0400fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.687+0000 7fd0460fb700 1 -- 172.21.15.40:0/2767255387 shutdown_connections 2023-07-18T03:32:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.687+0000 7fd0460fb700 1 -- 172.21.15.40:0/2767255387 wait complete. 2023-07-18T03:32:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd0460fb700 1 Processor -- start 2023-07-18T03:32:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd0460fb700 1 -- start start 2023-07-18T03:32:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd0460fb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0400ffb10 0x7fd040072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd0460fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd040100d00 0x7fd040072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd0460fb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd040101690 0x7fd040073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd0460fb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04019f900 con 0x7fd040101690 2023-07-18T03:32:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd0460fb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04019fa40 con 0x7fd040100d00 2023-07-18T03:32:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd0460fb700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04019fb80 con 0x7fd0400ffb10 2023-07-18T03:32:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd03f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0400ffb10 0x7fd040072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd03ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd040101690 0x7fd040073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd03effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd040100d00 0x7fd040072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd03f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0400ffb10 0x7fd040072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48390/0 (socket says 172.21.15.40:48390) 2023-07-18T03:32:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd03f7fe700 1 -- 172.21.15.40:0/3723786342 learned_addr learned my addr 172.21.15.40:0/3723786342 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd03effd700 1 -- 172.21.15.40:0/3723786342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0400ffb10 msgr2=0x7fd040072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd03effd700 1 --2- 172.21.15.40:0/3723786342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0400ffb10 0x7fd040072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd03effd700 1 -- 172.21.15.40:0/3723786342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd040101690 msgr2=0x7fd040073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd03effd700 1 --2- 172.21.15.40:0/3723786342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd040101690 0x7fd040073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd03effd700 1 -- 172.21.15.40:0/3723786342 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0340095f0 con 0x7fd040100d00 2023-07-18T03:32:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.688+0000 7fd03ffff700 1 --2- 172.21.15.40:0/3723786342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd040101690 0x7fd040073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.689+0000 7fd03effd700 1 --2- 172.21.15.40:0/3723786342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd040100d00 0x7fd040072e40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd03400eb90 tx=0x7fd03400eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.689+0000 7fd03cff9700 1 -- 172.21.15.40:0/3723786342 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034009970 con 0x7fd040100d00 2023-07-18T03:32:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.689+0000 7fd0460fb700 1 -- 172.21.15.40:0/3723786342 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd030009520 con 0x7fd040100d00 2023-07-18T03:32:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.689+0000 7fd03cff9700 1 -- 172.21.15.40:0/3723786342 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0340044e0 con 0x7fd040100d00 2023-07-18T03:32:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.689+0000 7fd03cff9700 1 -- 172.21.15.40:0/3723786342 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034003e50 con 0x7fd040100d00 2023-07-18T03:32:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.689+0000 7fd0460fb700 1 -- 172.21.15.40:0/3723786342 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0401a0190 con 0x7fd040100d00 2023-07-18T03:32:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.691+0000 7fd03cff9700 1 -- 172.21.15.40:0/3723786342 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd034004000 con 0x7fd040100d00 2023-07-18T03:32:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.692+0000 7fd03cff9700 1 --2- 172.21.15.40:0/3723786342 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd0280a1150 0x7fd0280a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.692+0000 7fd03f7fe700 1 --2- 172.21.15.40:0/3723786342 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd0280a1150 0x7fd0280a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.692+0000 7fd03cff9700 1 -- 172.21.15.40:0/3723786342 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd034014070 con 0x7fd040100d00 2023-07-18T03:32:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.692+0000 7fd03f7fe700 1 --2- 172.21.15.40:0/3723786342 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd0280a1150 0x7fd0280a3600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd03000c010 tx=0x7fd030019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.692+0000 7fd052281b80 1 Processor -- start 2023-07-18T03:32:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd052281b80 1 -- start start 2023-07-18T03:32:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd052281b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653a5e6b0 0x564653c02300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd052281b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653c02840 0x564653c031a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd052281b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c036e0 0x564653c03b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd052281b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564653b07c50 con 0x564653c036e0 2023-07-18T03:32:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd052281b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564653b5a9e0 con 0x564653a5e6b0 2023-07-18T03:32:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd052281b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564653b53e70 con 0x564653c02840 2023-07-18T03:32:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd03ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c036e0 0x564653c03b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd03f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653a5e6b0 0x564653c02300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd03ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c036e0 0x564653c03b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59374/0 (socket says 172.21.15.40:59374) 2023-07-18T03:32:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd03ffff700 1 -- 172.21.15.40:0/2962193466 learned_addr learned my addr 172.21.15.40:0/2962193466 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd03f7fe700 1 --2- 172.21.15.40:0/2962193466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653a5e6b0 0x564653c02300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42696/0 (socket says 172.21.15.40:42696) 2023-07-18T03:32:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd03effd700 1 --2- 172.21.15.40:0/2962193466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653c02840 0x564653c031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd03f7fe700 1 -- 172.21.15.40:0/2962193466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653c02840 msgr2=0x564653c031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd03f7fe700 1 --2- 172.21.15.40:0/2962193466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653c02840 0x564653c031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd03f7fe700 1 -- 172.21.15.40:0/2962193466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c036e0 msgr2=0x564653c03b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd03f7fe700 1 --2- 172.21.15.40:0/2962193466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c036e0 0x564653c03b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd03f7fe700 1 -- 172.21.15.40:0/2962193466 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564653981d80 con 0x564653a5e6b0 2023-07-18T03:32:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd03ffff700 1 --2- 172.21.15.40:0/2962193466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c036e0 0x564653c03b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.693+0000 7fd03f7fe700 1 --2- 172.21.15.40:0/2962193466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653a5e6b0 0x564653c02300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd03000f9e0 tx=0x7fd03001e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78186f71b8195063 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.694+0000 7fd025ffb700 1 -- 172.21.15.40:0/2962193466 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03000f6e0 con 0x564653a5e6b0 2023-07-18T03:32:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.694+0000 7fd025ffb700 1 -- 172.21.15.40:0/2962193466 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd03001d040 con 0x564653a5e6b0 2023-07-18T03:32:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.694+0000 7fd025ffb700 1 -- 172.21.15.40:0/2962193466 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0300056a0 con 0x564653a5e6b0 2023-07-18T03:32:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.694+0000 7fd052281b80 1 -- 172.21.15.40:0/2962193466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653a5e6b0 msgr2=0x564653c02300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.694+0000 7fd052281b80 1 --2- 172.21.15.40:0/2962193466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653a5e6b0 0x564653c02300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd03000f9e0 tx=0x7fd03001e000 comp rx=0 tx=0).stop 2023-07-18T03:32:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.694+0000 7fd052281b80 1 -- 172.21.15.40:0/2962193466 shutdown_connections 2023-07-18T03:32:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.694+0000 7fd052281b80 1 --2- 172.21.15.40:0/2962193466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c036e0 0x564653c03b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.694+0000 7fd052281b80 1 --2- 172.21.15.40:0/2962193466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653a5e6b0 0x564653c02300 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.694+0000 7fd052281b80 1 --2- 172.21.15.40:0/2962193466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653c02840 0x564653c031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.694+0000 7fd052281b80 1 -- 172.21.15.40:0/2962193466 >> 172.21.15.40:0/2962193466 conn(0x564653bf7be0 msgr2=0x564653bfccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.694+0000 7fd052281b80 1 -- 172.21.15.40:0/2962193466 shutdown_connections 2023-07-18T03:32:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.694+0000 7fd052281b80 1 -- 172.21.15.40:0/2962193466 wait complete. 2023-07-18T03:32:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.695+0000 7fd052281b80 1 -- start start 2023-07-18T03:32:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.695+0000 7fd052281b80 1 Processor -- start 2023-07-18T03:32:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.695+0000 7fd052281b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653a5e6b0 0x564653c0b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.695+0000 7fd052281b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 0x564653c0bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.695+0000 7fd052281b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653c036e0 0x564653c12df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.695+0000 7fd052281b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564653af3a60 con 0x564653c02840 2023-07-18T03:32:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.695+0000 7fd052281b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564653af3760 con 0x564653c036e0 2023-07-18T03:32:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.695+0000 7fd052281b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646539d78c0 con 0x564653a5e6b0 2023-07-18T03:32:50.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.695+0000 7fd052281b80 10 client.0 _ll_register_callbacks cb 0x564653af1700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.695+0000 7fd03f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653a5e6b0 0x564653c0b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.695+0000 7fd03effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 0x564653c0bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.695+0000 7fd03f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653a5e6b0 0x564653c0b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48404/0 (socket says 172.21.15.40:48404) 2023-07-18T03:32:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.695+0000 7fd03ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653c036e0 0x564653c12df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.695+0000 7fd03f7fe700 1 -- 172.21.15.40:0/3656043011 learned_addr learned my addr 172.21.15.40:0/3656043011 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:50.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.696+0000 7fd03f7fe700 1 -- 172.21.15.40:0/3656043011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653c036e0 msgr2=0x564653c12df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.696+0000 7fd03f7fe700 1 --2- 172.21.15.40:0/3656043011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653c036e0 0x564653c12df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.696+0000 7fd03f7fe700 1 -- 172.21.15.40:0/3656043011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 msgr2=0x564653c0bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.696+0000 7fd03f7fe700 1 --2- 172.21.15.40:0/3656043011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 0x564653c0bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.696+0000 7fd03f7fe700 1 -- 172.21.15.40:0/3656043011 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd038003b90 con 0x564653a5e6b0 2023-07-18T03:32:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.696+0000 7fd03ffff700 1 --2- 172.21.15.40:0/3656043011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653c036e0 0x564653c12df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:32:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.696+0000 7fd03f7fe700 1 --2- 172.21.15.40:0/3656043011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653a5e6b0 0x564653c0b870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd030005850 tx=0x7fd03000f710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6be77e0e266d50f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.696+0000 7fd01effd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:32:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.696+0000 7fd01effd700 1 -- 172.21.15.40:0/3656043011 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03000b910 con 0x564653a5e6b0 2023-07-18T03:32:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.696+0000 7fd01effd700 1 -- 172.21.15.40:0/3656043011 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd03000ba70 con 0x564653a5e6b0 2023-07-18T03:32:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.696+0000 7fd052281b80 1 -- 172.21.15.40:0/3656043011 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564653aad490 con 0x564653a5e6b0 2023-07-18T03:32:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.696+0000 7fd01effd700 1 -- 172.21.15.40:0/3656043011 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd030003720 con 0x564653a5e6b0 2023-07-18T03:32:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.696+0000 7fd052281b80 10 client.25690 fetch_fsmap learned FSMap version 32 2023-07-18T03:32:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.696+0000 7fd052281b80 1 -- 172.21.15.40:0/3656043011 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fd040108880 con 0x564653a5e6b0 2023-07-18T03:32:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.696+0000 7fd01effd700 1 -- 172.21.15.40:0/3656043011 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564653aad490 con 0x564653a5e6b0 2023-07-18T03:32:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.696+0000 7fd01effd700 1 -- 172.21.15.40:0/3656043011 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd03002f960 con 0x564653a5e6b0 2023-07-18T03:32:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.697+0000 7fd052281b80 10 client.25690 fetch_fsmap finished waiting for FSMap version 32 2023-07-18T03:32:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.697+0000 7fd052281b80 10 client.25690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:32:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.697+0000 7fd052281b80 20 client.25690 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.697+0000 7fd052281b80 1 --2- 172.21.15.40:0/3656043011 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564653c17190 0x564653c195a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.697+0000 7fd052281b80 4 client.25690 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:32:50.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.697+0000 7fd052281b80 1 -- 172.21.15.40:0/3656043011 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564653b4ce50 con 0x564653c17190 2023-07-18T03:32:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.697+0000 7fd03effd700 1 --2- 172.21.15.40:0/3656043011 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564653c17190 0x564653c195a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.697+0000 7fd01effd700 10 client.25690 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.697+0000 7fd03effd700 1 --2- 172.21.15.40:0/3656043011 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564653c17190 0x564653c195a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f698da1affceabe3 server_cookie=d2b9d66c0274a7dc in_seq=0 out_seq=0 2023-07-18T03:32:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.698+0000 7fd01effd700 1 -- 172.21.15.40:0/3656043011 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564653b4ce50 con 0x564653c17190 2023-07-18T03:32:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.698+0000 7fd01effd700 10 client.25690 handle_command_reply: tid=0 2023-07-18T03:32:50.709 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.698+0000 7fd052281b80 1 client.25690 shutdown 2023-07-18T03:32:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.698+0000 7fd052281b80 1 -- 172.21.15.40:0/3656043011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653a5e6b0 msgr2=0x564653c0b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.698+0000 7fd052281b80 1 --2- 172.21.15.40:0/3656043011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653a5e6b0 0x564653c0b870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd030005850 tx=0x7fd03000f710 comp rx=0 tx=0).stop 2023-07-18T03:32:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.698+0000 7fd052281b80 1 -- 172.21.15.40:0/3656043011 shutdown_connections 2023-07-18T03:32:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.698+0000 7fd052281b80 1 --2- 172.21.15.40:0/3656043011 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564653c17190 0x564653c195a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.698+0000 7fd052281b80 1 --2- 172.21.15.40:0/3656043011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653c036e0 0x564653c12df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.698+0000 7fd01effd700 0 client.25690 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:50.710 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.698+0000 7fd052281b80 1 --2- 172.21.15.40:0/3656043011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653a5e6b0 0x564653c0b870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.698+0000 7fd052281b80 1 --2- 172.21.15.40:0/3656043011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 0x564653c0bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.698+0000 7fd052281b80 1 -- 172.21.15.40:0/3656043011 >> 172.21.15.40:0/3656043011 conn(0x564653bf7be0 msgr2=0x564653bfc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.698+0000 7fd052281b80 1 -- 172.21.15.40:0/3656043011 shutdown_connections 2023-07-18T03:32:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.698+0000 7fd052281b80 1 -- 172.21.15.40:0/3656043011 wait complete. 2023-07-18T03:32:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.698+0000 7fd052281b80 20 client.25690 trim_cache size 0 max 16384 2023-07-18T03:32:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd052281b80 1 Processor -- start 2023-07-18T03:32:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd052281b80 1 -- start start 2023-07-18T03:32:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd052281b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653a5e6b0 0x564653c03ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd052281b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 0x564653c12db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd052281b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653c036e0 0x564653c0a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd052281b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564653aad490 con 0x564653c02840 2023-07-18T03:32:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd052281b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564653bc8060 con 0x564653c036e0 2023-07-18T03:32:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd052281b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564653c0ad20 con 0x564653a5e6b0 2023-07-18T03:32:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd03effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 0x564653c12db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd03f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653a5e6b0 0x564653c03ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd03ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653c036e0 0x564653c0a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd03f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653a5e6b0 0x564653c03ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48412/0 (socket says 172.21.15.40:48412) 2023-07-18T03:32:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd03ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653c036e0 0x564653c0a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42714/0 (socket says 172.21.15.40:42714) 2023-07-18T03:32:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd03effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 0x564653c12db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59386/0 (socket says 172.21.15.40:59386) 2023-07-18T03:32:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd03f7fe700 1 -- 172.21.15.40:0/4287058450 learned_addr learned my addr 172.21.15.40:0/4287058450 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd03ffff700 1 -- 172.21.15.40:0/4287058450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653a5e6b0 msgr2=0x564653c03ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd03ffff700 1 --2- 172.21.15.40:0/4287058450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653a5e6b0 0x564653c03ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd03ffff700 1 -- 172.21.15.40:0/4287058450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 msgr2=0x564653c12db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd03ffff700 1 --2- 172.21.15.40:0/4287058450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 0x564653c12db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.714+0000 7fd03ffff700 1 -- 172.21.15.40:0/4287058450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56465396ac00 con 0x564653c036e0 2023-07-18T03:32:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.715+0000 7fd03ffff700 1 --2- 172.21.15.40:0/4287058450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653c036e0 0x564653c0a5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd038009fd0 tx=0x7fd038009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e29fe39cd3b49b55 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.715+0000 7fd0267fc700 1 -- 172.21.15.40:0/4287058450 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038004db0 con 0x564653c036e0 2023-07-18T03:32:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.715+0000 7fd0267fc700 1 -- 172.21.15.40:0/4287058450 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd03800f040 con 0x564653c036e0 2023-07-18T03:32:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.715+0000 7fd0267fc700 1 -- 172.21.15.40:0/4287058450 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0380059c0 con 0x564653c036e0 2023-07-18T03:32:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.715+0000 7fd052281b80 1 -- 172.21.15.40:0/4287058450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653c036e0 msgr2=0x564653c0a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.715+0000 7fd052281b80 1 --2- 172.21.15.40:0/4287058450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653c036e0 0x564653c0a5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd038009fd0 tx=0x7fd038009920 comp rx=0 tx=0).stop 2023-07-18T03:32:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.715+0000 7fd052281b80 1 -- 172.21.15.40:0/4287058450 shutdown_connections 2023-07-18T03:32:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.715+0000 7fd052281b80 1 --2- 172.21.15.40:0/4287058450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653c036e0 0x564653c0a5a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.715+0000 7fd052281b80 1 --2- 172.21.15.40:0/4287058450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653a5e6b0 0x564653c03ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.715+0000 7fd052281b80 1 --2- 172.21.15.40:0/4287058450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 0x564653c12db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.715+0000 7fd052281b80 1 -- 172.21.15.40:0/4287058450 >> 172.21.15.40:0/4287058450 conn(0x564653bf7be0 msgr2=0x564653bf9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.716+0000 7fd052281b80 1 -- 172.21.15.40:0/4287058450 shutdown_connections 2023-07-18T03:32:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.716+0000 7fd052281b80 1 -- 172.21.15.40:0/4287058450 wait complete. 2023-07-18T03:32:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.716+0000 7fd052281b80 1 -- start start 2023-07-18T03:32:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.716+0000 7fd052281b80 1 Processor -- start 2023-07-18T03:32:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.716+0000 7fd052281b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653a5e6b0 0x564653c0e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.716+0000 7fd052281b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 0x564653c0ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.716+0000 7fd052281b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653c036e0 0x564653c11ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.716+0000 7fd052281b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564653c121e0 con 0x564653c02840 2023-07-18T03:32:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.716+0000 7fd052281b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564653c12350 con 0x564653a5e6b0 2023-07-18T03:32:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.716+0000 7fd052281b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564653c12510 con 0x564653c036e0 2023-07-18T03:32:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.716+0000 7fd052281b80 10 client.0 _ll_register_callbacks cb 0x7fd038003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:32:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.716+0000 7fd03effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 0x564653c0ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.716+0000 7fd03effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 0x564653c0ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59392/0 (socket says 172.21.15.40:59392) 2023-07-18T03:32:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.716+0000 7fd03effd700 1 -- 172.21.15.40:0/2869783691 learned_addr learned my addr 172.21.15.40:0/2869783691 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:32:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.716+0000 7fd03ffff700 1 --2- 172.21.15.40:0/2869783691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653c036e0 0x564653c11ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.717+0000 7fd03f7fe700 1 --2- 172.21.15.40:0/2869783691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653a5e6b0 0x564653c0e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.725 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.717+0000 7fd03effd700 1 -- 172.21.15.40:0/2869783691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653c036e0 msgr2=0x564653c11ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.717+0000 7fd03effd700 1 --2- 172.21.15.40:0/2869783691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653c036e0 0x564653c11ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.717+0000 7fd03effd700 1 -- 172.21.15.40:0/2869783691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653a5e6b0 msgr2=0x564653c0e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:32:50.726 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.717+0000 7fd03effd700 1 --2- 172.21.15.40:0/2869783691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653a5e6b0 0x564653c0e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.727 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.717+0000 7fd03effd700 1 -- 172.21.15.40:0/2869783691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd038003eb0 con 0x564653c02840 2023-07-18T03:32:50.727 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.717+0000 7fd03effd700 1 --2- 172.21.15.40:0/2869783691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 0x564653c0ec80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd034005fd0 tx=0x7fd03409a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4221db0af0e0faf7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:32:50.727 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.717+0000 7fd01effd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:32:50.727 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.717+0000 7fd01effd700 1 -- 172.21.15.40:0/2869783691 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0340a2600 con 0x564653c02840 2023-07-18T03:32:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.717+0000 7fd052281b80 1 -- 172.21.15.40:0/2869783691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd03001d040 con 0x564653c02840 2023-07-18T03:32:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.717+0000 7fd01effd700 1 -- 172.21.15.40:0/2869783691 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd0340982f0 con 0x564653c02840 2023-07-18T03:32:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.717+0000 7fd01effd700 1 -- 172.21.15.40:0/2869783691 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd034098550 con 0x564653c02840 2023-07-18T03:32:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.717+0000 7fd052281b80 10 client.16056 fetch_fsmap learned FSMap version 32 2023-07-18T03:32:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.717+0000 7fd052281b80 1 -- 172.21.15.40:0/2869783691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5646539a81f0 con 0x564653c02840 2023-07-18T03:32:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.717+0000 7fd01effd700 1 -- 172.21.15.40:0/2869783691 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd034063070 con 0x564653c02840 2023-07-18T03:32:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.718+0000 7fd01effd700 1 -- 172.21.15.40:0/2869783691 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd0340997a0 con 0x564653c02840 2023-07-18T03:32:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.718+0000 7fd052281b80 10 client.16056 fetch_fsmap finished waiting for FSMap version 32 2023-07-18T03:32:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.718+0000 7fd052281b80 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:32:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.718+0000 7fd052281b80 20 client.16056 populate_metadata read hostname 'smithi040' 2023-07-18T03:32:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.718+0000 7fd052281b80 1 --2- 172.21.15.40:0/2869783691 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564653c6bc00 0x564653c6e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:32:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.718+0000 7fd052281b80 4 client.16056 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:32:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.718+0000 7fd052281b80 1 -- 172.21.15.40:0/2869783691 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564653b0eff0 con 0x564653c6bc00 2023-07-18T03:32:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.718+0000 7fd03f7fe700 1 --2- 172.21.15.40:0/2869783691 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564653c6bc00 0x564653c6e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:32:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.718+0000 7fd01effd700 10 client.16056 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:50.732 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.718+0000 7fd03f7fe700 1 --2- 172.21.15.40:0/2869783691 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564653c6bc00 0x564653c6e010 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab9950760e806151 server_cookie=c59b74d82bbda99f in_seq=0 out_seq=0 2023-07-18T03:32:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.910+0000 7fd01effd700 1 -- 172.21.15.40:0/2869783691 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564653b0eff0 con 0x564653c6bc00 2023-07-18T03:32:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.910+0000 7fd01effd700 10 client.16056 handle_command_reply: tid=0 2023-07-18T03:32:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.910+0000 7fd052281b80 1 client.16056 shutdown 2023-07-18T03:32:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.910+0000 7fd052281b80 1 -- 172.21.15.40:0/2869783691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 msgr2=0x564653c0ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.910+0000 7fd052281b80 1 --2- 172.21.15.40:0/2869783691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 0x564653c0ec80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd034005fd0 tx=0x7fd03409a2a0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.910+0000 7fd052281b80 1 -- 172.21.15.40:0/2869783691 shutdown_connections 2023-07-18T03:32:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.910+0000 7fd052281b80 1 --2- 172.21.15.40:0/2869783691 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564653c6bc00 0x564653c6e010 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.910+0000 7fd052281b80 1 --2- 172.21.15.40:0/2869783691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564653c036e0 0x564653c11ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.910+0000 7fd052281b80 1 --2- 172.21.15.40:0/2869783691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564653a5e6b0 0x564653c0e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.910+0000 7fd052281b80 1 --2- 172.21.15.40:0/2869783691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564653c02840 0x564653c0ec80 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.910+0000 7fd01effd700 0 client.16056 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:32:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.910+0000 7fd052281b80 1 -- 172.21.15.40:0/2869783691 >> 172.21.15.40:0/2869783691 conn(0x564653bf7be0 msgr2=0x564653c1df00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.910+0000 7fd052281b80 1 -- 172.21.15.40:0/2869783691 shutdown_connections 2023-07-18T03:32:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.910+0000 7fd052281b80 1 -- 172.21.15.40:0/2869783691 wait complete. 2023-07-18T03:32:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.911+0000 7fd052281b80 20 client.16056 trim_cache size 0 max 16384 2023-07-18T03:32:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:32:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:32:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:32:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:32:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.911+0000 7fd0460fb700 1 -- 172.21.15.40:0/3723786342 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd0280a1150 msgr2=0x7fd0280a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.911+0000 7fd0460fb700 1 --2- 172.21.15.40:0/3723786342 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd0280a1150 0x7fd0280a3600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd03000c010 tx=0x7fd030019040 comp rx=0 tx=0).stop 2023-07-18T03:32:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.911+0000 7fd0460fb700 1 -- 172.21.15.40:0/3723786342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd040100d00 msgr2=0x7fd040072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:32:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.911+0000 7fd0460fb700 1 --2- 172.21.15.40:0/3723786342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd040100d00 0x7fd040072e40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd03400eb90 tx=0x7fd03400eea0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.911+0000 7fd0460fb700 1 -- 172.21.15.40:0/3723786342 shutdown_connections 2023-07-18T03:32:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.911+0000 7fd0460fb700 1 --2- 172.21.15.40:0/3723786342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd040101690 0x7fd040073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.911+0000 7fd0460fb700 1 --2- 172.21.15.40:0/3723786342 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd0280a1150 0x7fd0280a3600 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.911+0000 7fd0460fb700 1 --2- 172.21.15.40:0/3723786342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0400ffb10 0x7fd040072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.911+0000 7fd0460fb700 1 --2- 172.21.15.40:0/3723786342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd040100d00 0x7fd040072e40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:32:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.911+0000 7fd0460fb700 1 -- 172.21.15.40:0/3723786342 >> 172.21.15.40:0/3723786342 conn(0x7fd0400fb130 msgr2=0x7fd0400fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:32:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.911+0000 7fd0460fb700 1 -- 172.21.15.40:0/3723786342 shutdown_connections 2023-07-18T03:32:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:32:50.911+0000 7fd0460fb700 1 -- 172.21.15.40:0/3723786342 wait complete. 2023-07-18T03:32:52.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:51 smithi134 ceph-mon[98216]: pgmap v352: 97 pgs: 97 active+clean; 169 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 203 op/s 2023-07-18T03:32:52.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:51 smithi040 ceph-mon[96439]: pgmap v352: 97 pgs: 97 active+clean; 169 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 203 op/s 2023-07-18T03:32:52.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:51 smithi064 ceph-mon[98059]: pgmap v352: 97 pgs: 97 active+clean; 169 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 203 op/s 2023-07-18T03:32:53.144 INFO:tasks.workunit.client.0.smithi040.stdout: 70 436761 1021 299503 1129 102465 35 2023-07-18T03:32:54.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:53 smithi134 ceph-mon[98216]: pgmap v353: 97 pgs: 97 active+clean; 174 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 208 op/s 2023-07-18T03:32:54.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:53 smithi040 ceph-mon[96439]: pgmap v353: 97 pgs: 97 active+clean; 174 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 208 op/s 2023-07-18T03:32:54.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:53 smithi064 ceph-mon[98059]: pgmap v353: 97 pgs: 97 active+clean; 174 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 208 op/s 2023-07-18T03:32:56.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:55 smithi134 ceph-mon[98216]: pgmap v354: 97 pgs: 97 active+clean; 175 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 141 op/s 2023-07-18T03:32:56.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:55 smithi040 ceph-mon[96439]: pgmap v354: 97 pgs: 97 active+clean; 175 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 141 op/s 2023-07-18T03:32:56.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:55 smithi064 ceph-mon[98059]: pgmap v354: 97 pgs: 97 active+clean; 175 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 141 op/s 2023-07-18T03:32:58.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:57 smithi134 ceph-mon[98216]: pgmap v355: 97 pgs: 97 active+clean; 178 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 147 op/s 2023-07-18T03:32:58.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:57 smithi040 ceph-mon[96439]: pgmap v355: 97 pgs: 97 active+clean; 178 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 147 op/s 2023-07-18T03:32:58.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:57 smithi064 ceph-mon[98059]: pgmap v355: 97 pgs: 97 active+clean; 178 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 147 op/s 2023-07-18T03:33:00.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:32:59 smithi134 ceph-mon[98216]: pgmap v356: 97 pgs: 97 active+clean; 183 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2023-07-18T03:33:00.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:32:59 smithi040 ceph-mon[96439]: pgmap v356: 97 pgs: 97 active+clean; 183 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2023-07-18T03:33:00.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:32:59 smithi064 ceph-mon[98059]: pgmap v356: 97 pgs: 97 active+clean; 183 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2023-07-18T03:33:00.169 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:33:00.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.307+0000 7f922d9d1700 1 -- 172.21.15.40:0/768749355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9228102e80 msgr2=0x7f9228105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:00.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.307+0000 7f922d9d1700 1 --2- 172.21.15.40:0/768749355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9228102e80 0x7f9228105260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f921c009990 tx=0x7f921c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.307+0000 7f922d9d1700 1 -- 172.21.15.40:0/768749355 shutdown_connections 2023-07-18T03:33:00.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.307+0000 7f922d9d1700 1 --2- 172.21.15.40:0/768749355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92281057a0 0x7f9228107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.307+0000 7f922d9d1700 1 --2- 172.21.15.40:0/768749355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9228100560 0x7f9228102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.307+0000 7f922d9d1700 1 --2- 172.21.15.40:0/768749355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9228102e80 0x7f9228105260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.307+0000 7f922d9d1700 1 -- 172.21.15.40:0/768749355 >> 172.21.15.40:0/768749355 conn(0x7f92280fa000 msgr2=0x7f92280fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:00.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.307+0000 7f922d9d1700 1 -- 172.21.15.40:0/768749355 shutdown_connections 2023-07-18T03:33:00.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.307+0000 7f922d9d1700 1 -- 172.21.15.40:0/768749355 wait complete. 2023-07-18T03:33:00.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.308+0000 7f922d9d1700 1 Processor -- start 2023-07-18T03:33:00.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.308+0000 7f922d9d1700 1 -- start start 2023-07-18T03:33:00.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.309+0000 7f922d9d1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9228100560 0x7f92281957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.309+0000 7f922d9d1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9228102e80 0x7f9228195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.309+0000 7f922d9d1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92281057a0 0x7f922819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.309+0000 7f922d9d1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922819d330 con 0x7f9228100560 2023-07-18T03:33:00.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.309+0000 7f922d9d1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922819d470 con 0x7f92281057a0 2023-07-18T03:33:00.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.309+0000 7f922d9d1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922819d600 con 0x7f9228102e80 2023-07-18T03:33:00.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.309+0000 7f92267fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9228102e80 0x7f9228195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.309+0000 7f9226ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9228100560 0x7f92281957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.309+0000 7f92267fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9228102e80 0x7f9228195cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48434/0 (socket says 172.21.15.40:48434) 2023-07-18T03:33:00.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.309+0000 7f92267fc700 1 -- 172.21.15.40:0/1584669309 learned_addr learned my addr 172.21.15.40:0/1584669309 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:00.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.309+0000 7f92277fe700 1 --2- 172.21.15.40:0/1584669309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92281057a0 0x7f922819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.309+0000 7f9226ffd700 1 -- 172.21.15.40:0/1584669309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9228102e80 msgr2=0x7f9228195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:00.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.309+0000 7f9226ffd700 1 --2- 172.21.15.40:0/1584669309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9228102e80 0x7f9228195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.309+0000 7f9226ffd700 1 -- 172.21.15.40:0/1584669309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92281057a0 msgr2=0x7f922819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:00.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.309+0000 7f9226ffd700 1 --2- 172.21.15.40:0/1584669309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92281057a0 0x7f922819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.309+0000 7f9226ffd700 1 -- 172.21.15.40:0/1584669309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c0095f0 con 0x7f9228100560 2023-07-18T03:33:00.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.309+0000 7f9226ffd700 1 --2- 172.21.15.40:0/1584669309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9228100560 0x7f92281957b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f921800e9f0 tx=0x7f921800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:00.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.310+0000 7f922c9cf700 1 -- 172.21.15.40:0/1584669309 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921800c9c0 con 0x7f9228100560 2023-07-18T03:33:00.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.310+0000 7f922c9cf700 1 -- 172.21.15.40:0/1584669309 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9218010430 con 0x7f9228100560 2023-07-18T03:33:00.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.310+0000 7f922d9d1700 1 -- 172.21.15.40:0/1584669309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922819d8e0 con 0x7f9228100560 2023-07-18T03:33:00.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.310+0000 7f922c9cf700 1 -- 172.21.15.40:0/1584669309 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9218018680 con 0x7f9228100560 2023-07-18T03:33:00.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.310+0000 7f922d9d1700 1 -- 172.21.15.40:0/1584669309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922819de40 con 0x7f9228100560 2023-07-18T03:33:00.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.312+0000 7f922c9cf700 1 -- 172.21.15.40:0/1584669309 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9218010860 con 0x7f9228100560 2023-07-18T03:33:00.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.312+0000 7f922c9cf700 1 --2- 172.21.15.40:0/1584669309 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f92100a11d0 0x7f92100a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.313+0000 7f922c9cf700 1 -- 172.21.15.40:0/1584669309 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9218014070 con 0x7f9228100560 2023-07-18T03:33:00.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.313+0000 7f92267fc700 1 --2- 172.21.15.40:0/1584669309 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f92100a11d0 0x7f92100a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.313+0000 7f92267fc700 1 --2- 172.21.15.40:0/1584669309 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f92100a11d0 0x7f92100a3680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f921c00b2e0 tx=0x7f921c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:00.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.315+0000 7f9239b57b80 1 Processor -- start 2023-07-18T03:33:00.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.315+0000 7f9239b57b80 1 -- start start 2023-07-18T03:33:00.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f9239b57b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a734d0 0x555bf2a738e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f9239b57b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a73e20 0x555bf2a746e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f9239b57b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a74c70 0x555bf2a750e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f9239b57b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf2978cb0 con 0x555bf2a73e20 2023-07-18T03:33:00.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f9239b57b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf29cba40 con 0x555bf2a734d0 2023-07-18T03:33:00.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f9239b57b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf29c4ed0 con 0x555bf2a74c70 2023-07-18T03:33:00.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f92267fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a73e20 0x555bf2a746e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f9226ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a734d0 0x555bf2a738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f92267fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a73e20 0x555bf2a746e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59414/0 (socket says 172.21.15.40:59414) 2023-07-18T03:33:00.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f92267fc700 1 -- 172.21.15.40:0/3701850774 learned_addr learned my addr 172.21.15.40:0/3701850774 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:00.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f92277fe700 1 --2- 172.21.15.40:0/3701850774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a74c70 0x555bf2a750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f9226ffd700 1 -- 172.21.15.40:0/3701850774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a74c70 msgr2=0x555bf2a750e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:00.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f9226ffd700 1 --2- 172.21.15.40:0/3701850774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a74c70 0x555bf2a750e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f9226ffd700 1 -- 172.21.15.40:0/3701850774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a73e20 msgr2=0x555bf2a746e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:00.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f9226ffd700 1 --2- 172.21.15.40:0/3701850774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a73e20 0x555bf2a746e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f9226ffd700 1 -- 172.21.15.40:0/3701850774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bf27f2de0 con 0x555bf2a734d0 2023-07-18T03:33:00.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f92267fc700 1 --2- 172.21.15.40:0/3701850774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a73e20 0x555bf2a746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:00.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.316+0000 7f9226ffd700 1 --2- 172.21.15.40:0/3701850774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a734d0 0x555bf2a738e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f921801e000 tx=0x7f9218099920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=769e9e5374e0408e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:00.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.317+0000 7f920dffb700 1 -- 172.21.15.40:0/3701850774 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9218099de0 con 0x555bf2a734d0 2023-07-18T03:33:00.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.317+0000 7f920dffb700 1 -- 172.21.15.40:0/3701850774 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f921809de10 con 0x555bf2a734d0 2023-07-18T03:33:00.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.317+0000 7f920dffb700 1 -- 172.21.15.40:0/3701850774 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92180a1da0 con 0x555bf2a734d0 2023-07-18T03:33:00.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.318+0000 7f9239b57b80 1 -- 172.21.15.40:0/3701850774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a734d0 msgr2=0x555bf2a738e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:00.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.318+0000 7f9239b57b80 1 --2- 172.21.15.40:0/3701850774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a734d0 0x555bf2a738e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f921801e000 tx=0x7f9218099920 comp rx=0 tx=0).stop 2023-07-18T03:33:00.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.318+0000 7f9239b57b80 1 -- 172.21.15.40:0/3701850774 shutdown_connections 2023-07-18T03:33:00.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.318+0000 7f9239b57b80 1 --2- 172.21.15.40:0/3701850774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a74c70 0x555bf2a750e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.318+0000 7f9239b57b80 1 --2- 172.21.15.40:0/3701850774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a734d0 0x555bf2a738e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.318+0000 7f9239b57b80 1 --2- 172.21.15.40:0/3701850774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a73e20 0x555bf2a746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.318+0000 7f9239b57b80 1 -- 172.21.15.40:0/3701850774 >> 172.21.15.40:0/3701850774 conn(0x555bf2a68da0 msgr2=0x555bf2a6e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:00.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.318+0000 7f9239b57b80 1 -- 172.21.15.40:0/3701850774 shutdown_connections 2023-07-18T03:33:00.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.318+0000 7f9239b57b80 1 -- 172.21.15.40:0/3701850774 wait complete. 2023-07-18T03:33:00.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.318+0000 7f9239b57b80 1 -- start start 2023-07-18T03:33:00.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.319+0000 7f9239b57b80 1 Processor -- start 2023-07-18T03:33:00.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.319+0000 7f9239b57b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 0x555bf2a7d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.319+0000 7f9239b57b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a73e20 0x555bf2a7d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.319+0000 7f9239b57b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a74c70 0x555bf2a846d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.319+0000 7f9239b57b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf2964ac0 con 0x555bf2a74c70 2023-07-18T03:33:00.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.319+0000 7f9239b57b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf29647c0 con 0x555bf2a73e20 2023-07-18T03:33:00.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.319+0000 7f9239b57b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf2848920 con 0x555bf2a734d0 2023-07-18T03:33:00.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.319+0000 7f9239b57b80 10 client.0 _ll_register_callbacks cb 0x555bf2962760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:00.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.319+0000 7f9226ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 0x555bf2a7d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.319+0000 7f92267fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a73e20 0x555bf2a7d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.319+0000 7f9226ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 0x555bf2a7d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48446/0 (socket says 172.21.15.40:48446) 2023-07-18T03:33:00.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.319+0000 7f9226ffd700 1 -- 172.21.15.40:0/1635009344 learned_addr learned my addr 172.21.15.40:0/1635009344 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:00.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.319+0000 7f92277fe700 1 --2- 172.21.15.40:0/1635009344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a74c70 0x555bf2a846d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.319+0000 7f9226ffd700 1 -- 172.21.15.40:0/1635009344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a73e20 msgr2=0x555bf2a7d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:00.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.319+0000 7f9226ffd700 1 --2- 172.21.15.40:0/1635009344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a73e20 0x555bf2a7d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.319+0000 7f9226ffd700 1 -- 172.21.15.40:0/1635009344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a74c70 msgr2=0x555bf2a846d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:00.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.319+0000 7f9226ffd700 1 --2- 172.21.15.40:0/1635009344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a74c70 0x555bf2a846d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.320+0000 7f9226ffd700 1 -- 172.21.15.40:0/1635009344 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92281087f0 con 0x555bf2a734d0 2023-07-18T03:33:00.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.320+0000 7f92267fc700 1 --2- 172.21.15.40:0/1635009344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a73e20 0x555bf2a7d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:00.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.320+0000 7f9226ffd700 1 --2- 172.21.15.40:0/1635009344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 0x555bf2a7d150 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9218095b60 tx=0x7f9218010610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36bdfc008adfd518 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:00.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.320+0000 7f9206ffd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:33:00.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.320+0000 7f9206ffd700 1 -- 172.21.15.40:0/1635009344 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92180ca270 con 0x555bf2a734d0 2023-07-18T03:33:00.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.320+0000 7f9239b57b80 1 -- 172.21.15.40:0/1635009344 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bf291e4f0 con 0x555bf2a734d0 2023-07-18T03:33:00.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.320+0000 7f9206ffd700 1 -- 172.21.15.40:0/1635009344 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9218099de0 con 0x555bf2a734d0 2023-07-18T03:33:00.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.320+0000 7f9206ffd700 1 -- 172.21.15.40:0/1635009344 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92180b6bd0 con 0x555bf2a734d0 2023-07-18T03:33:00.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.320+0000 7f9239b57b80 10 client.25708 fetch_fsmap learned FSMap version 32 2023-07-18T03:33:00.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.320+0000 7f9239b57b80 1 -- 172.21.15.40:0/1635009344 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f922819de40 con 0x555bf2a734d0 2023-07-18T03:33:00.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.320+0000 7f9206ffd700 1 -- 172.21.15.40:0/1635009344 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555bf291e4f0 con 0x555bf2a734d0 2023-07-18T03:33:00.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.321+0000 7f9206ffd700 1 -- 172.21.15.40:0/1635009344 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f92180adae0 con 0x555bf2a734d0 2023-07-18T03:33:00.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.321+0000 7f9239b57b80 10 client.25708 fetch_fsmap finished waiting for FSMap version 32 2023-07-18T03:33:00.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.321+0000 7f9239b57b80 10 client.25708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:33:00.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.321+0000 7f9239b57b80 20 client.25708 populate_metadata read hostname 'smithi040' 2023-07-18T03:33:00.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.321+0000 7f9239b57b80 1 --2- 172.21.15.40:0/1635009344 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555bf2a88a60 0x555bf2a8ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.321+0000 7f9239b57b80 4 client.25708 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:33:00.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.321+0000 7f9239b57b80 1 -- 172.21.15.40:0/1635009344 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bf29bdeb0 con 0x555bf2a88a60 2023-07-18T03:33:00.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.321+0000 7f92267fc700 1 --2- 172.21.15.40:0/1635009344 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555bf2a88a60 0x555bf2a8ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.321+0000 7f9206ffd700 10 client.25708 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:00.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.321+0000 7f92267fc700 1 --2- 172.21.15.40:0/1635009344 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555bf2a88a60 0x555bf2a8ae70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d183ab58f0de32c server_cookie=b295b20f39662b5b in_seq=0 out_seq=0 2023-07-18T03:33:00.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.322+0000 7f9206ffd700 1 -- 172.21.15.40:0/1635009344 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555bf29bdeb0 con 0x555bf2a88a60 2023-07-18T03:33:00.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.322+0000 7f9206ffd700 10 client.25708 handle_command_reply: tid=0 2023-07-18T03:33:00.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.322+0000 7f9239b57b80 1 client.25708 shutdown 2023-07-18T03:33:00.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.322+0000 7f9239b57b80 1 -- 172.21.15.40:0/1635009344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 msgr2=0x555bf2a7d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:00.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.322+0000 7f9239b57b80 1 --2- 172.21.15.40:0/1635009344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 0x555bf2a7d150 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9218095b60 tx=0x7f9218010610 comp rx=0 tx=0).stop 2023-07-18T03:33:00.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.322+0000 7f9239b57b80 1 -- 172.21.15.40:0/1635009344 shutdown_connections 2023-07-18T03:33:00.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.322+0000 7f9239b57b80 1 --2- 172.21.15.40:0/1635009344 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555bf2a88a60 0x555bf2a8ae70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.322+0000 7f9239b57b80 1 --2- 172.21.15.40:0/1635009344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a74c70 0x555bf2a846d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.322+0000 7f9239b57b80 1 --2- 172.21.15.40:0/1635009344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 0x555bf2a7d150 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.322+0000 7f9206ffd700 0 client.25708 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:00.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.322+0000 7f9239b57b80 1 --2- 172.21.15.40:0/1635009344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a73e20 0x555bf2a7d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.322+0000 7f9239b57b80 1 -- 172.21.15.40:0/1635009344 >> 172.21.15.40:0/1635009344 conn(0x555bf2a68da0 msgr2=0x555bf2a6d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:00.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.322+0000 7f9239b57b80 1 -- 172.21.15.40:0/1635009344 shutdown_connections 2023-07-18T03:33:00.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.322+0000 7f9239b57b80 1 -- 172.21.15.40:0/1635009344 wait complete. 2023-07-18T03:33:00.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.323+0000 7f9239b57b80 20 client.25708 trim_cache size 0 max 16384 2023-07-18T03:33:00.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.340+0000 7f9239b57b80 1 Processor -- start 2023-07-18T03:33:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.340+0000 7f9239b57b80 1 -- start start 2023-07-18T03:33:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.340+0000 7f9239b57b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 0x555bf2a7be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.340+0000 7f9239b57b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a73e20 0x555bf2a84690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.340+0000 7f9239b57b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a74c70 0x555bf2a84fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.340+0000 7f9239b57b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf2a7cd20 con 0x555bf2a74c70 2023-07-18T03:33:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.340+0000 7f9239b57b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf2a6fc60 con 0x555bf2a73e20 2023-07-18T03:33:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.340+0000 7f9239b57b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf2a68970 con 0x555bf2a734d0 2023-07-18T03:33:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.340+0000 7f9226ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 0x555bf2a7be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.340+0000 7f92277fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a74c70 0x555bf2a84fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.340+0000 7f9226ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 0x555bf2a7be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48454/0 (socket says 172.21.15.40:48454) 2023-07-18T03:33:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.340+0000 7f9226ffd700 1 -- 172.21.15.40:0/3503357544 learned_addr learned my addr 172.21.15.40:0/3503357544 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.340+0000 7f92267fc700 1 --2- 172.21.15.40:0/3503357544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a73e20 0x555bf2a84690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.341+0000 7f9226ffd700 1 -- 172.21.15.40:0/3503357544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a73e20 msgr2=0x555bf2a84690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:00.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.341+0000 7f9226ffd700 1 --2- 172.21.15.40:0/3503357544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a73e20 0x555bf2a84690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.341+0000 7f9226ffd700 1 -- 172.21.15.40:0/3503357544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a74c70 msgr2=0x555bf2a84fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:00.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.341+0000 7f9226ffd700 1 --2- 172.21.15.40:0/3503357544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a74c70 0x555bf2a84fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.341+0000 7f9226ffd700 1 -- 172.21.15.40:0/3503357544 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bf27dbc60 con 0x555bf2a734d0 2023-07-18T03:33:00.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.341+0000 7f92277fe700 1 --2- 172.21.15.40:0/3503357544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a74c70 0x555bf2a84fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:00.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.341+0000 7f9226ffd700 1 --2- 172.21.15.40:0/3503357544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 0x555bf2a7be50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f921801e000 tx=0x7f9218098390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab0ba5bfe731e624 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:00.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.341+0000 7f920e7fc700 1 -- 172.21.15.40:0/3503357544 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921807af10 con 0x555bf2a734d0 2023-07-18T03:33:00.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.341+0000 7f920e7fc700 1 -- 172.21.15.40:0/3503357544 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9218099920 con 0x555bf2a734d0 2023-07-18T03:33:00.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.341+0000 7f920e7fc700 1 -- 172.21.15.40:0/3503357544 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92180b6bd0 con 0x555bf2a734d0 2023-07-18T03:33:00.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.342+0000 7f9239b57b80 1 -- 172.21.15.40:0/3503357544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 msgr2=0x555bf2a7be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:00.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.342+0000 7f9239b57b80 1 --2- 172.21.15.40:0/3503357544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 0x555bf2a7be50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f921801e000 tx=0x7f9218098390 comp rx=0 tx=0).stop 2023-07-18T03:33:00.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.342+0000 7f9239b57b80 1 -- 172.21.15.40:0/3503357544 shutdown_connections 2023-07-18T03:33:00.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.342+0000 7f9239b57b80 1 --2- 172.21.15.40:0/3503357544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a74c70 0x555bf2a84fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.342+0000 7f9239b57b80 1 --2- 172.21.15.40:0/3503357544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 0x555bf2a7be50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.342+0000 7f9239b57b80 1 --2- 172.21.15.40:0/3503357544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a73e20 0x555bf2a84690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.342+0000 7f9239b57b80 1 -- 172.21.15.40:0/3503357544 >> 172.21.15.40:0/3503357544 conn(0x555bf2a68da0 msgr2=0x555bf2a8f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:00.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.342+0000 7f9239b57b80 1 -- 172.21.15.40:0/3503357544 shutdown_connections 2023-07-18T03:33:00.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.342+0000 7f9239b57b80 1 -- 172.21.15.40:0/3503357544 wait complete. 2023-07-18T03:33:00.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.342+0000 7f9239b57b80 1 -- start start 2023-07-18T03:33:00.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.342+0000 7f9239b57b80 1 Processor -- start 2023-07-18T03:33:00.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f9239b57b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 0x555bf2a804e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f9239b57b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a73e20 0x555bf2a80a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f9239b57b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a74c70 0x555bf2a83a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f9239b57b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf2a83f80 con 0x555bf2a73e20 2023-07-18T03:33:00.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f9239b57b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf2a840f0 con 0x555bf2a74c70 2023-07-18T03:33:00.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f9239b57b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bf2a842b0 con 0x555bf2a734d0 2023-07-18T03:33:00.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f9239b57b80 10 client.0 _ll_register_callbacks cb 0x7f921c005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:00.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f92267fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a73e20 0x555bf2a80a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f92267fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a73e20 0x555bf2a80a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59434/0 (socket says 172.21.15.40:59434) 2023-07-18T03:33:00.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f9226ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 0x555bf2a804e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f92267fc700 1 -- 172.21.15.40:0/683818072 learned_addr learned my addr 172.21.15.40:0/683818072 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:00.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f92277fe700 1 --2- 172.21.15.40:0/683818072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a74c70 0x555bf2a83a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f92267fc700 1 -- 172.21.15.40:0/683818072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 msgr2=0x555bf2a804e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:00.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f92267fc700 1 --2- 172.21.15.40:0/683818072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 0x555bf2a804e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f92267fc700 1 -- 172.21.15.40:0/683818072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a74c70 msgr2=0x555bf2a83a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:00.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f92267fc700 1 --2- 172.21.15.40:0/683818072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a74c70 0x555bf2a83a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f92267fc700 1 -- 172.21.15.40:0/683818072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92200124e0 con 0x555bf2a73e20 2023-07-18T03:33:00.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f92277fe700 1 --2- 172.21.15.40:0/683818072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a74c70 0x555bf2a83a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:00.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f9226ffd700 1 --2- 172.21.15.40:0/683818072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 0x555bf2a804e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:00.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f92267fc700 1 --2- 172.21.15.40:0/683818072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a73e20 0x555bf2a80a20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f921c005cd0 tx=0x7f921c01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3aaf6a3abbfcfc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:00.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f9206ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:33:00.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.343+0000 7f9206ffd700 1 -- 172.21.15.40:0/683818072 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c02b040 con 0x555bf2a73e20 2023-07-18T03:33:00.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.344+0000 7f9239b57b80 1 -- 172.21.15.40:0/683818072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f921809de10 con 0x555bf2a73e20 2023-07-18T03:33:00.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.344+0000 7f9206ffd700 1 -- 172.21.15.40:0/683818072 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f921c0349d0 con 0x555bf2a73e20 2023-07-18T03:33:00.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.344+0000 7f9206ffd700 1 -- 172.21.15.40:0/683818072 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f921c02d040 con 0x555bf2a73e20 2023-07-18T03:33:00.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.344+0000 7f9239b57b80 10 client.16074 fetch_fsmap learned FSMap version 32 2023-07-18T03:33:00.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.344+0000 7f9239b57b80 1 -- 172.21.15.40:0/683818072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f922819de40 con 0x555bf2a73e20 2023-07-18T03:33:00.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.344+0000 7f9206ffd700 1 -- 172.21.15.40:0/683818072 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f921809de10 con 0x555bf2a73e20 2023-07-18T03:33:00.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.344+0000 7f9206ffd700 1 -- 172.21.15.40:0/683818072 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f921c00fa40 con 0x555bf2a73e20 2023-07-18T03:33:00.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.344+0000 7f9239b57b80 10 client.16074 fetch_fsmap finished waiting for FSMap version 32 2023-07-18T03:33:00.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.344+0000 7f9239b57b80 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:33:00.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.344+0000 7f9239b57b80 20 client.16074 populate_metadata read hostname 'smithi040' 2023-07-18T03:33:00.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.344+0000 7f9239b57b80 1 --2- 172.21.15.40:0/683818072 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555bf2add900 0x555bf2adfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:00.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.344+0000 7f9239b57b80 4 client.16074 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:33:00.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.344+0000 7f9239b57b80 1 -- 172.21.15.40:0/683818072 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555bf2980050 con 0x555bf2add900 2023-07-18T03:33:00.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.344+0000 7f9226ffd700 1 --2- 172.21.15.40:0/683818072 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555bf2add900 0x555bf2adfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:00.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.345+0000 7f9206ffd700 10 client.16074 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:00.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.345+0000 7f9226ffd700 1 --2- 172.21.15.40:0/683818072 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555bf2add900 0x555bf2adfd10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=444e1e47b814a3d4 server_cookie=7b18d6094ab88f0 in_seq=0 out_seq=0 2023-07-18T03:33:00.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.345+0000 7f9206ffd700 1 -- 172.21.15.40:0/683818072 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555bf2980050 con 0x555bf2add900 2023-07-18T03:33:00.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.345+0000 7f9206ffd700 10 client.16074 handle_command_reply: tid=0 2023-07-18T03:33:00.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.345+0000 7f9239b57b80 1 client.16074 shutdown 2023-07-18T03:33:00.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.345+0000 7f9239b57b80 1 -- 172.21.15.40:0/683818072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a73e20 msgr2=0x555bf2a80a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:00.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.345+0000 7f9239b57b80 1 --2- 172.21.15.40:0/683818072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a73e20 0x555bf2a80a20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f921c005cd0 tx=0x7f921c01a050 comp rx=0 tx=0).stop 2023-07-18T03:33:00.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.345+0000 7f9239b57b80 1 -- 172.21.15.40:0/683818072 shutdown_connections 2023-07-18T03:33:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.345+0000 7f9239b57b80 1 --2- 172.21.15.40:0/683818072 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555bf2add900 0x555bf2adfd10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.345+0000 7f9239b57b80 1 --2- 172.21.15.40:0/683818072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf2a74c70 0x555bf2a83a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.345+0000 7f9239b57b80 1 --2- 172.21.15.40:0/683818072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555bf2a734d0 0x555bf2a804e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.345+0000 7f9206ffd700 0 client.16074 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.345+0000 7f9239b57b80 1 --2- 172.21.15.40:0/683818072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555bf2a73e20 0x555bf2a80a20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.345+0000 7f9239b57b80 1 -- 172.21.15.40:0/683818072 >> 172.21.15.40:0/683818072 conn(0x555bf2a68da0 msgr2=0x555bf2a7be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.345+0000 7f9239b57b80 1 -- 172.21.15.40:0/683818072 shutdown_connections 2023-07-18T03:33:00.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.345+0000 7f9239b57b80 1 -- 172.21.15.40:0/683818072 wait complete. 2023-07-18T03:33:00.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.345+0000 7f9239b57b80 20 client.16074 trim_cache size 0 max 16384 2023-07-18T03:33:00.353 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:33:00.353 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:33:00.353 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:33:00.353 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:33:00.354 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:33:00.354 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:33:00.354 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:33:00.354 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:33:00.354 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:33:00.354 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:33:00.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.346+0000 7f922d9d1700 1 -- 172.21.15.40:0/1584669309 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f92100a11d0 msgr2=0x7f92100a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:00.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.346+0000 7f922d9d1700 1 --2- 172.21.15.40:0/1584669309 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f92100a11d0 0x7f92100a3680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f921c00b2e0 tx=0x7f921c005960 comp rx=0 tx=0).stop 2023-07-18T03:33:00.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.346+0000 7f922d9d1700 1 -- 172.21.15.40:0/1584669309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9228100560 msgr2=0x7f92281957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:00.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.346+0000 7f922d9d1700 1 --2- 172.21.15.40:0/1584669309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9228100560 0x7f92281957b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f921800e9f0 tx=0x7f921800ed00 comp rx=0 tx=0).stop 2023-07-18T03:33:00.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.346+0000 7f922d9d1700 1 -- 172.21.15.40:0/1584669309 shutdown_connections 2023-07-18T03:33:00.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.346+0000 7f922d9d1700 1 --2- 172.21.15.40:0/1584669309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92281057a0 0x7f922819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.346+0000 7f922d9d1700 1 --2- 172.21.15.40:0/1584669309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9228100560 0x7f92281957b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.346+0000 7f922d9d1700 1 --2- 172.21.15.40:0/1584669309 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f92100a11d0 0x7f92100a3680 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.346+0000 7f922d9d1700 1 --2- 172.21.15.40:0/1584669309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9228102e80 0x7f9228195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:00.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.346+0000 7f922d9d1700 1 -- 172.21.15.40:0/1584669309 >> 172.21.15.40:0/1584669309 conn(0x7f92280fa000 msgr2=0x7f92281011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:00.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.346+0000 7f922d9d1700 1 -- 172.21.15.40:0/1584669309 shutdown_connections 2023-07-18T03:33:00.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:00.346+0000 7f922d9d1700 1 -- 172.21.15.40:0/1584669309 wait complete. 2023-07-18T03:33:00.359 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:33:02.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:01 smithi134 ceph-mon[98216]: pgmap v357: 97 pgs: 97 active+clean; 183 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 144 op/s 2023-07-18T03:33:02.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:01 smithi040 ceph-mon[96439]: pgmap v357: 97 pgs: 97 active+clean; 183 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 144 op/s 2023-07-18T03:33:02.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:01 smithi064 ceph-mon[98059]: pgmap v357: 97 pgs: 97 active+clean; 183 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 144 op/s 2023-07-18T03:33:03.144 INFO:tasks.workunit.client.0.smithi040.stdout: 91 771793 995 552124 1259 141935 21 2023-07-18T03:33:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.066+0000 7f0ced37c700 1 -- 172.21.15.40:0/520536336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ce80ffc10 msgr2=0x7f0ce8100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.066+0000 7f0ced37c700 1 --2- 172.21.15.40:0/520536336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ce80ffc10 0x7f0ce8100060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009990 tx=0x7f0cdc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.066+0000 7f0ced37c700 1 -- 172.21.15.40:0/520536336 shutdown_connections 2023-07-18T03:33:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.066+0000 7f0ced37c700 1 --2- 172.21.15.40:0/520536336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ce81005a0 0x7f0ce8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.066+0000 7f0ced37c700 1 --2- 172.21.15.40:0/520536336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ce80fea20 0x7f0ce80fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.066+0000 7f0ced37c700 1 --2- 172.21.15.40:0/520536336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ce80ffc10 0x7f0ce8100060 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.066+0000 7f0ced37c700 1 -- 172.21.15.40:0/520536336 >> 172.21.15.40:0/520536336 conn(0x7f0ce80fa020 msgr2=0x7f0ce80fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.067+0000 7f0ced37c700 1 -- 172.21.15.40:0/520536336 shutdown_connections 2023-07-18T03:33:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.067+0000 7f0ced37c700 1 -- 172.21.15.40:0/520536336 wait complete. 2023-07-18T03:33:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.068+0000 7f0ced37c700 1 Processor -- start 2023-07-18T03:33:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.068+0000 7f0ced37c700 1 -- start start 2023-07-18T03:33:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.068+0000 7f0ced37c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ce80fea20 0x7f0ce8195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.068+0000 7f0ced37c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ce80ffc10 0x7f0ce8195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.068+0000 7f0ced37c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ce81005a0 0x7f0ce8196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.068+0000 7f0ced37c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce8197370 con 0x7f0ce80ffc10 2023-07-18T03:33:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.068+0000 7f0ced37c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce81974e0 con 0x7f0ce80fea20 2023-07-18T03:33:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.068+0000 7f0ced37c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce81976a0 con 0x7f0ce81005a0 2023-07-18T03:33:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.069+0000 7f0ce67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ce80ffc10 0x7f0ce8195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.069+0000 7f0ce67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ce80ffc10 0x7f0ce8195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59448/0 (socket says 172.21.15.40:59448) 2023-07-18T03:33:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.069+0000 7f0ce67fc700 1 -- 172.21.15.40:0/2789640456 learned_addr learned my addr 172.21.15.40:0/2789640456 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.069+0000 7f0ce77fe700 1 --2- 172.21.15.40:0/2789640456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ce81005a0 0x7f0ce8196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.069+0000 7f0ce67fc700 1 -- 172.21.15.40:0/2789640456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ce81005a0 msgr2=0x7f0ce8196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.069+0000 7f0ce67fc700 1 --2- 172.21.15.40:0/2789640456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ce81005a0 0x7f0ce8196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.069+0000 7f0ce67fc700 1 -- 172.21.15.40:0/2789640456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ce80fea20 msgr2=0x7f0ce8195950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:33:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.069+0000 7f0ce67fc700 1 --2- 172.21.15.40:0/2789640456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ce80fea20 0x7f0ce8195950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.069+0000 7f0ce67fc700 1 -- 172.21.15.40:0/2789640456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc0095f0 con 0x7f0ce80ffc10 2023-07-18T03:33:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.069+0000 7f0ce77fe700 1 --2- 172.21.15.40:0/2789640456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ce81005a0 0x7f0ce8196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.069+0000 7f0ce67fc700 1 --2- 172.21.15.40:0/2789640456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ce80ffc10 0x7f0ce8195e90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc004760 tx=0x7f0cdc004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.069+0000 7f0ccffff700 1 -- 172.21.15.40:0/2789640456 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc01d070 con 0x7f0ce80ffc10 2023-07-18T03:33:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.070+0000 7f0ccffff700 1 -- 172.21.15.40:0/2789640456 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cdc004830 con 0x7f0ce80ffc10 2023-07-18T03:33:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.070+0000 7f0ced37c700 1 -- 172.21.15.40:0/2789640456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce8197920 con 0x7f0ce80ffc10 2023-07-18T03:33:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.070+0000 7f0ccffff700 1 -- 172.21.15.40:0/2789640456 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc005460 con 0x7f0ce80ffc10 2023-07-18T03:33:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.070+0000 7f0ced37c700 1 -- 172.21.15.40:0/2789640456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce8197b90 con 0x7f0ce80ffc10 2023-07-18T03:33:04.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:03 smithi134 ceph-mon[98216]: pgmap v358: 97 pgs: 97 active+clean; 183 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 149 op/s 2023-07-18T03:33:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.072+0000 7f0ccffff700 1 -- 172.21.15.40:0/2789640456 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0cdc004c10 con 0x7f0ce80ffc10 2023-07-18T03:33:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.073+0000 7f0ccffff700 1 --2- 172.21.15.40:0/2789640456 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0cd00a1150 0x7f0cd00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.073+0000 7f0ce6ffd700 1 --2- 172.21.15.40:0/2789640456 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0cd00a1150 0x7f0cd00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.073+0000 7f0ccffff700 1 -- 172.21.15.40:0/2789640456 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0cdc0cdaf0 con 0x7f0ce80ffc10 2023-07-18T03:33:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.074+0000 7f0ce6ffd700 1 --2- 172.21.15.40:0/2789640456 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0cd00a1150 0x7f0cd00a3600 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80ffa70 tx=0x7f0cd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0cf9502b80 1 Processor -- start 2023-07-18T03:33:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0cf9502b80 1 -- start start 2023-07-18T03:33:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0cf9502b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 0x561c62f1f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0cf9502b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f1f840 0x561c62f201a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0cf9502b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62f206e0 0x561c62f20b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0cf9502b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c62e24c50 con 0x561c62d7b6b0 2023-07-18T03:33:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0cf9502b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c62e779e0 con 0x561c62f1f840 2023-07-18T03:33:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0cf9502b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c62e70e70 con 0x561c62f206e0 2023-07-18T03:33:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0ce6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 0x561c62f1f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0ce6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 0x561c62f1f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59454/0 (socket says 172.21.15.40:59454) 2023-07-18T03:33:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0ce6ffd700 1 -- 172.21.15.40:0/3487006856 learned_addr learned my addr 172.21.15.40:0/3487006856 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0ce77fe700 1 --2- 172.21.15.40:0/3487006856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62f206e0 0x561c62f20b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0ce67fc700 1 --2- 172.21.15.40:0/3487006856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f1f840 0x561c62f201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0ce77fe700 1 -- 172.21.15.40:0/3487006856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f1f840 msgr2=0x561c62f201a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0ce77fe700 1 --2- 172.21.15.40:0/3487006856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f1f840 0x561c62f201a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0ce77fe700 1 -- 172.21.15.40:0/3487006856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 msgr2=0x561c62f1f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0ce77fe700 1 --2- 172.21.15.40:0/3487006856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 0x561c62f1f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0ce77fe700 1 -- 172.21.15.40:0/3487006856 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c62c9ed80 con 0x561c62f206e0 2023-07-18T03:33:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.076+0000 7f0ce6ffd700 1 --2- 172.21.15.40:0/3487006856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 0x561c62f1f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.077+0000 7f0ce77fe700 1 --2- 172.21.15.40:0/3487006856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62f206e0 0x561c62f20b80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0003db0 tx=0x7f0ce0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91cc45bf2cbf474e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.077+0000 7f0ccd7fa700 1 -- 172.21.15.40:0/3487006856 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0004bb0 con 0x561c62f206e0 2023-07-18T03:33:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.077+0000 7f0ccd7fa700 1 -- 172.21.15.40:0/3487006856 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ce000f040 con 0x561c62f206e0 2023-07-18T03:33:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.077+0000 7f0ccd7fa700 1 -- 172.21.15.40:0/3487006856 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0007c20 con 0x561c62f206e0 2023-07-18T03:33:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.078+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3487006856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62f206e0 msgr2=0x561c62f20b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.078+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/3487006856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62f206e0 0x561c62f20b80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0003db0 tx=0x7f0ce0009920 comp rx=0 tx=0).stop 2023-07-18T03:33:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.078+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3487006856 shutdown_connections 2023-07-18T03:33:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.078+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/3487006856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62f206e0 0x561c62f20b80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.078+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/3487006856 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 0x561c62f1f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.078+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/3487006856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f1f840 0x561c62f201a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.078+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3487006856 >> 172.21.15.40:0/3487006856 conn(0x561c62f14be0 msgr2=0x561c62f19cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.078+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3487006856 shutdown_connections 2023-07-18T03:33:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.078+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3487006856 wait complete. 2023-07-18T03:33:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.078+0000 7f0cf9502b80 1 -- start start 2023-07-18T03:33:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.078+0000 7f0cf9502b80 1 Processor -- start 2023-07-18T03:33:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.078+0000 7f0cf9502b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62d7b6b0 0x561c62f28800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.078+0000 7f0cf9502b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f1f840 0x561c62f28d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.078+0000 7f0cf9502b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62f206e0 0x561c62f2fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0cf9502b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c62e10a60 con 0x561c62f206e0 2023-07-18T03:33:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0cf9502b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c62e10760 con 0x561c62f1f840 2023-07-18T03:33:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0cf9502b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c62cf48c0 con 0x561c62d7b6b0 2023-07-18T03:33:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0cf9502b80 10 client.0 _ll_register_callbacks cb 0x561c62e0e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0ce6ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62d7b6b0 0x561c62f28800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0ce6ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62d7b6b0 0x561c62f28800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48488/0 (socket says 172.21.15.40:48488) 2023-07-18T03:33:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0ce6ffd700 1 -- 172.21.15.40:0/3939525852 learned_addr learned my addr 172.21.15.40:0/3939525852 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0ce67fc700 1 --2- 172.21.15.40:0/3939525852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f1f840 0x561c62f28d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0ce6ffd700 1 -- 172.21.15.40:0/3939525852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f1f840 msgr2=0x561c62f28d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0ce6ffd700 1 --2- 172.21.15.40:0/3939525852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f1f840 0x561c62f28d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0ce6ffd700 1 -- 172.21.15.40:0/3939525852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62f206e0 msgr2=0x561c62f2fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0ce6ffd700 1 --2- 172.21.15.40:0/3939525852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62f206e0 0x561c62f2fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0ce6ffd700 1 -- 172.21.15.40:0/3939525852 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd800e7e0 con 0x561c62d7b6b0 2023-07-18T03:33:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0ce67fc700 1 --2- 172.21.15.40:0/3939525852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f1f840 0x561c62f28d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0ce6ffd700 1 --2- 172.21.15.40:0/3939525852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62d7b6b0 0x561c62f28800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0cd801d6e0 tx=0x7f0cd80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98b72555dd0e350d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0ccaff5700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:33:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0ccaff5700 1 -- 172.21.15.40:0/3939525852 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8010640 con 0x561c62d7b6b0 2023-07-18T03:33:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3939525852 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c62dca490 con 0x561c62d7b6b0 2023-07-18T03:33:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.079+0000 7f0ccaff5700 1 -- 172.21.15.40:0/3939525852 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0cd80107a0 con 0x561c62d7b6b0 2023-07-18T03:33:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.080+0000 7f0ccaff5700 1 -- 172.21.15.40:0/3939525852 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cd8010960 con 0x561c62d7b6b0 2023-07-18T03:33:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.080+0000 7f0cf9502b80 10 client.25726 fetch_fsmap learned FSMap version 32 2023-07-18T03:33:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.080+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3939525852 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f0ce0003a60 con 0x561c62d7b6b0 2023-07-18T03:33:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.080+0000 7f0ccaff5700 1 -- 172.21.15.40:0/3939525852 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cd8034080 con 0x561c62d7b6b0 2023-07-18T03:33:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.080+0000 7f0ccaff5700 1 -- 172.21.15.40:0/3939525852 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f0cd802e030 con 0x561c62d7b6b0 2023-07-18T03:33:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.080+0000 7f0cf9502b80 10 client.25726 fetch_fsmap finished waiting for FSMap version 32 2023-07-18T03:33:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.080+0000 7f0cf9502b80 10 client.25726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:33:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.080+0000 7f0cf9502b80 20 client.25726 populate_metadata read hostname 'smithi040' 2023-07-18T03:33:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.080+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/3939525852 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561c62f38a70 0x561c62f3ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.080+0000 7f0cf9502b80 4 client.25726 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:33:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.080+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3939525852 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c62e69e50 con 0x561c62f38a70 2023-07-18T03:33:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.080+0000 7f0ce67fc700 1 --2- 172.21.15.40:0/3939525852 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561c62f38a70 0x561c62f3ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.081+0000 7f0ccaff5700 10 client.25726 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.081+0000 7f0ce67fc700 1 --2- 172.21.15.40:0/3939525852 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561c62f38a70 0x561c62f3ae80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df719fb7bb96c8e0 server_cookie=2be4725c168ed2ea in_seq=0 out_seq=0 2023-07-18T03:33:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.082+0000 7f0ccaff5700 1 -- 172.21.15.40:0/3939525852 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561c62e69e50 con 0x561c62f38a70 2023-07-18T03:33:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.082+0000 7f0ccaff5700 10 client.25726 handle_command_reply: tid=0 2023-07-18T03:33:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.082+0000 7f0cf9502b80 1 client.25726 shutdown 2023-07-18T03:33:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.082+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3939525852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62d7b6b0 msgr2=0x561c62f28800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.082+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/3939525852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62d7b6b0 0x561c62f28800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0cd801d6e0 tx=0x7f0cd80047c0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.082+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3939525852 shutdown_connections 2023-07-18T03:33:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.082+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/3939525852 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561c62f38a70 0x561c62f3ae80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.082+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/3939525852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62f206e0 0x561c62f2fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.082+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/3939525852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62d7b6b0 0x561c62f28800 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.082+0000 7f0ccaff5700 0 client.25726 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.082+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/3939525852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f1f840 0x561c62f28d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.082+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3939525852 >> 172.21.15.40:0/3939525852 conn(0x561c62f14be0 msgr2=0x561c62f194e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.082+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3939525852 shutdown_connections 2023-07-18T03:33:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.082+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3939525852 wait complete. 2023-07-18T03:33:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.082+0000 7f0cf9502b80 20 client.25726 trim_cache size 0 max 16384 2023-07-18T03:33:04.088 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:03 smithi040 ceph-mon[96439]: pgmap v358: 97 pgs: 97 active+clean; 183 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 149 op/s 2023-07-18T03:33:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.102+0000 7f0cf9502b80 1 Processor -- start 2023-07-18T03:33:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.102+0000 7f0cf9502b80 1 -- start start 2023-07-18T03:33:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.102+0000 7f0cf9502b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 0x561c62f235b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.102+0000 7f0cf9502b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f1f840 0x561c62f2fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.102+0000 7f0cf9502b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62f206e0 0x561c62f23af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.102+0000 7f0cf9502b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c62f27cb0 con 0x561c62d7b6b0 2023-07-18T03:33:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.102+0000 7f0cf9502b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c62f16d00 con 0x561c62f1f840 2023-07-18T03:33:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.102+0000 7f0cf9502b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c62f1a9a0 con 0x561c62f206e0 2023-07-18T03:33:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.102+0000 7f0ce6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 0x561c62f235b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.102+0000 7f0ce6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 0x561c62f235b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59468/0 (socket says 172.21.15.40:59468) 2023-07-18T03:33:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.102+0000 7f0ce6ffd700 1 -- 172.21.15.40:0/3021263677 learned_addr learned my addr 172.21.15.40:0/3021263677 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.102+0000 7f0ce77fe700 1 --2- 172.21.15.40:0/3021263677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62f206e0 0x561c62f23af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:04.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:03 smithi064 ceph-mon[98059]: pgmap v358: 97 pgs: 97 active+clean; 183 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 149 op/s 2023-07-18T03:33:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.103+0000 7f0ce67fc700 1 --2- 172.21.15.40:0/3021263677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f1f840 0x561c62f2fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.103+0000 7f0ce6ffd700 1 -- 172.21.15.40:0/3021263677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62f206e0 msgr2=0x561c62f23af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.103+0000 7f0ce6ffd700 1 --2- 172.21.15.40:0/3021263677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62f206e0 0x561c62f23af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.103+0000 7f0ce6ffd700 1 -- 172.21.15.40:0/3021263677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f1f840 msgr2=0x561c62f2fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.103+0000 7f0ce6ffd700 1 --2- 172.21.15.40:0/3021263677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f1f840 0x561c62f2fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.103+0000 7f0ce6ffd700 1 -- 172.21.15.40:0/3021263677 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c62c87c00 con 0x561c62d7b6b0 2023-07-18T03:33:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.103+0000 7f0ce67fc700 1 --2- 172.21.15.40:0/3021263677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f1f840 0x561c62f2fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:33:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.103+0000 7f0ce6ffd700 1 --2- 172.21.15.40:0/3021263677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 0x561c62f235b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8013040 tx=0x7f0cd8030de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c304f9a9ebb84b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.103+0000 7f0ccdffb700 1 -- 172.21.15.40:0/3021263677 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8030400 con 0x561c62d7b6b0 2023-07-18T03:33:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.103+0000 7f0ccdffb700 1 -- 172.21.15.40:0/3021263677 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c62dca490 con 0x561c62d7b6b0 2023-07-18T03:33:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.103+0000 7f0ccdffb700 1 -- 172.21.15.40:0/3021263677 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8025560 con 0x561c62d7b6b0 2023-07-18T03:33:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.104+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3021263677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 msgr2=0x561c62f235b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.104+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/3021263677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 0x561c62f235b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8013040 tx=0x7f0cd8030de0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.105+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3021263677 shutdown_connections 2023-07-18T03:33:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.105+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/3021263677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62f206e0 0x561c62f23af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.105+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/3021263677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 0x561c62f235b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.105+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/3021263677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f1f840 0x561c62f2fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.105+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3021263677 >> 172.21.15.40:0/3021263677 conn(0x561c62f14be0 msgr2=0x561c62f3f820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.105+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3021263677 shutdown_connections 2023-07-18T03:33:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.105+0000 7f0cf9502b80 1 -- 172.21.15.40:0/3021263677 wait complete. 2023-07-18T03:33:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.105+0000 7f0cf9502b80 1 -- start start 2023-07-18T03:33:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.105+0000 7f0cf9502b80 1 Processor -- start 2023-07-18T03:33:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.106+0000 7f0cf9502b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 0x561c62f2bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.106+0000 7f0cf9502b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62f1f840 0x561c62f2c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.106+0000 7f0cf9502b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f206e0 0x561c62f2ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.106+0000 7f0cf9502b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c62f2f490 con 0x561c62d7b6b0 2023-07-18T03:33:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.106+0000 7f0cf9502b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c62f2f600 con 0x561c62f206e0 2023-07-18T03:33:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.106+0000 7f0cf9502b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c62f2f7c0 con 0x561c62f1f840 2023-07-18T03:33:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.106+0000 7f0cf9502b80 10 client.0 _ll_register_callbacks cb 0x561c62e0e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.106+0000 7f0ce6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 0x561c62f2bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.106+0000 7f0ce77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f206e0 0x561c62f2ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.106+0000 7f0ce77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f206e0 0x561c62f2ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42802/0 (socket says 172.21.15.40:42802) 2023-07-18T03:33:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.106+0000 7f0ce77fe700 1 -- 172.21.15.40:0/1158959523 learned_addr learned my addr 172.21.15.40:0/1158959523 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.106+0000 7f0ce67fc700 1 --2- 172.21.15.40:0/1158959523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62f1f840 0x561c62f2c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.106+0000 7f0ce77fe700 1 -- 172.21.15.40:0/1158959523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62f1f840 msgr2=0x561c62f2c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.106+0000 7f0ce77fe700 1 --2- 172.21.15.40:0/1158959523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62f1f840 0x561c62f2c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.106+0000 7f0ce77fe700 1 -- 172.21.15.40:0/1158959523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 msgr2=0x561c62f2bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.106+0000 7f0ce77fe700 1 --2- 172.21.15.40:0/1158959523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 0x561c62f2bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.106+0000 7f0ce77fe700 1 -- 172.21.15.40:0/1158959523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce0003a60 con 0x561c62f206e0 2023-07-18T03:33:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.107+0000 7f0ce77fe700 1 --2- 172.21.15.40:0/1158959523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f206e0 0x561c62f2ef50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0004cd0 tx=0x7f0ce0004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbd5d715d8e3cfef server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.107+0000 7f0ccaff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:33:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.107+0000 7f0ccaff5700 1 -- 172.21.15.40:0/1158959523 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0007500 con 0x561c62f206e0 2023-07-18T03:33:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.107+0000 7f0ccaff5700 1 -- 172.21.15.40:0/1158959523 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0ce0007660 con 0x561c62f206e0 2023-07-18T03:33:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.107+0000 7f0cf9502b80 1 -- 172.21.15.40:0/1158959523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ce000f040 con 0x561c62f206e0 2023-07-18T03:33:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.107+0000 7f0ccaff5700 1 -- 172.21.15.40:0/1158959523 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ce0007820 con 0x561c62f206e0 2023-07-18T03:33:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.107+0000 7f0cf9502b80 10 client.25721 fetch_fsmap learned FSMap version 32 2023-07-18T03:33:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.107+0000 7f0cf9502b80 1 -- 172.21.15.40:0/1158959523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561c62cc51f0 con 0x561c62f206e0 2023-07-18T03:33:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.107+0000 7f0ccaff5700 1 -- 172.21.15.40:0/1158959523 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ce000f040 con 0x561c62f206e0 2023-07-18T03:33:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.107+0000 7f0ccaff5700 1 -- 172.21.15.40:0/1158959523 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f0ce0031050 con 0x561c62f206e0 2023-07-18T03:33:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.108+0000 7f0cf9502b80 10 client.25721 fetch_fsmap finished waiting for FSMap version 32 2023-07-18T03:33:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.108+0000 7f0cf9502b80 10 client.25721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:33:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.108+0000 7f0cf9502b80 20 client.25721 populate_metadata read hostname 'smithi040' 2023-07-18T03:33:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.108+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/1158959523 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561c62f8d790 0x561c62f8fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.108+0000 7f0cf9502b80 4 client.25721 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:33:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.108+0000 7f0cf9502b80 1 -- 172.21.15.40:0/1158959523 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561c62e2bff0 con 0x561c62f8d790 2023-07-18T03:33:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.108+0000 7f0ce6ffd700 1 --2- 172.21.15.40:0/1158959523 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561c62f8d790 0x561c62f8fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.108+0000 7f0ccaff5700 10 client.25721 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.108+0000 7f0ce6ffd700 1 --2- 172.21.15.40:0/1158959523 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561c62f8d790 0x561c62f8fba0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bac238b35f31746 server_cookie=3320be7a88f076a7 in_seq=0 out_seq=0 2023-07-18T03:33:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.251+0000 7f0ccaff5700 1 -- 172.21.15.40:0/1158959523 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561c62e2bff0 con 0x561c62f8d790 2023-07-18T03:33:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.251+0000 7f0ccaff5700 10 client.25721 handle_command_reply: tid=0 2023-07-18T03:33:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.251+0000 7f0cf9502b80 1 client.25721 shutdown 2023-07-18T03:33:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.251+0000 7f0cf9502b80 1 -- 172.21.15.40:0/1158959523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f206e0 msgr2=0x561c62f2ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.251+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/1158959523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f206e0 0x561c62f2ef50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0004cd0 tx=0x7f0ce0004d00 comp rx=0 tx=0).stop 2023-07-18T03:33:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.251+0000 7f0cf9502b80 1 -- 172.21.15.40:0/1158959523 shutdown_connections 2023-07-18T03:33:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.251+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/1158959523 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561c62f8d790 0x561c62f8fba0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.251+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/1158959523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c62f206e0 0x561c62f2ef50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.251+0000 7f0ccaff5700 0 client.25721 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.251+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/1158959523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561c62d7b6b0 0x561c62f2bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.251+0000 7f0cf9502b80 1 --2- 172.21.15.40:0/1158959523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561c62f1f840 0x561c62f2c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.251+0000 7f0cf9502b80 1 -- 172.21.15.40:0/1158959523 >> 172.21.15.40:0/1158959523 conn(0x561c62f14be0 msgr2=0x561c62f235b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.252+0000 7f0cf9502b80 1 -- 172.21.15.40:0/1158959523 shutdown_connections 2023-07-18T03:33:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.252+0000 7f0cf9502b80 1 -- 172.21.15.40:0/1158959523 wait complete. 2023-07-18T03:33:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.252+0000 7f0cf9502b80 20 client.25721 trim_cache size 0 max 16384 2023-07-18T03:33:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:33:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:33:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:33:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:33:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.252+0000 7f0ced37c700 1 -- 172.21.15.40:0/2789640456 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0cd00a1150 msgr2=0x7f0cd00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.252+0000 7f0ced37c700 1 --2- 172.21.15.40:0/2789640456 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0cd00a1150 0x7f0cd00a3600 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80ffa70 tx=0x7f0cd8009260 comp rx=0 tx=0).stop 2023-07-18T03:33:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.252+0000 7f0ced37c700 1 -- 172.21.15.40:0/2789640456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ce80ffc10 msgr2=0x7f0ce8195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.252+0000 7f0ced37c700 1 --2- 172.21.15.40:0/2789640456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ce80ffc10 0x7f0ce8195e90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc004760 tx=0x7f0cdc004790 comp rx=0 tx=0).stop 2023-07-18T03:33:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.252+0000 7f0ced37c700 1 -- 172.21.15.40:0/2789640456 shutdown_connections 2023-07-18T03:33:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.252+0000 7f0ced37c700 1 --2- 172.21.15.40:0/2789640456 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0cd00a1150 0x7f0cd00a3600 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.252+0000 7f0ced37c700 1 --2- 172.21.15.40:0/2789640456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ce81005a0 0x7f0ce8196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.252+0000 7f0ced37c700 1 --2- 172.21.15.40:0/2789640456 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ce80fea20 0x7f0ce8195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.252+0000 7f0ced37c700 1 --2- 172.21.15.40:0/2789640456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ce80ffc10 0x7f0ce8195e90 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.252+0000 7f0ced37c700 1 -- 172.21.15.40:0/2789640456 >> 172.21.15.40:0/2789640456 conn(0x7f0ce80fa020 msgr2=0x7f0ce80fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.252+0000 7f0ced37c700 1 -- 172.21.15.40:0/2789640456 shutdown_connections 2023-07-18T03:33:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:04.252+0000 7f0ced37c700 1 -- 172.21.15.40:0/2789640456 wait complete. 2023-07-18T03:33:06.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:05 smithi134 ceph-mon[98216]: pgmap v359: 97 pgs: 97 active+clean; 193 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 111 op/s 2023-07-18T03:33:06.084 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:05 smithi040 ceph-mon[96439]: pgmap v359: 97 pgs: 97 active+clean; 193 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 111 op/s 2023-07-18T03:33:06.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:05 smithi064 ceph-mon[98059]: pgmap v359: 97 pgs: 97 active+clean; 193 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 111 op/s 2023-07-18T03:33:08.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:07 smithi134 ceph-mon[98216]: pgmap v360: 97 pgs: 97 active+clean; 236 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 256 op/s 2023-07-18T03:33:08.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:07 smithi040 ceph-mon[96439]: pgmap v360: 97 pgs: 97 active+clean; 236 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 256 op/s 2023-07-18T03:33:08.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:07 smithi064 ceph-mon[98059]: pgmap v360: 97 pgs: 97 active+clean; 236 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 256 op/s 2023-07-18T03:33:10.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:09 smithi134 ceph-mon[98216]: pgmap v361: 97 pgs: 97 active+clean; 338 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 14 MiB/s wr, 627 op/s 2023-07-18T03:33:10.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:09 smithi040 ceph-mon[96439]: pgmap v361: 97 pgs: 97 active+clean; 338 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 14 MiB/s wr, 627 op/s 2023-07-18T03:33:10.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:09 smithi064 ceph-mon[98059]: pgmap v361: 97 pgs: 97 active+clean; 338 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 14 MiB/s wr, 627 op/s 2023-07-18T03:33:12.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:11 smithi134 ceph-mon[98216]: pgmap v362: 97 pgs: 97 active+clean; 349 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 14 MiB/s wr, 653 op/s 2023-07-18T03:33:12.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:11 smithi040 ceph-mon[96439]: pgmap v362: 97 pgs: 97 active+clean; 349 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 14 MiB/s wr, 653 op/s 2023-07-18T03:33:12.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:11 smithi064 ceph-mon[98059]: pgmap v362: 97 pgs: 97 active+clean; 349 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 14 MiB/s wr, 653 op/s 2023-07-18T03:33:13.144 INFO:tasks.workunit.client.0.smithi040.stdout: 106 810996 816 598457 885 120473 63 2023-07-18T03:33:14.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:13 smithi134 ceph-mon[98216]: pgmap v363: 97 pgs: 97 active+clean; 367 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 16 MiB/s wr, 719 op/s 2023-07-18T03:33:14.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:13 smithi040 ceph-mon[96439]: pgmap v363: 97 pgs: 97 active+clean; 367 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 16 MiB/s wr, 719 op/s 2023-07-18T03:33:14.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:13 smithi064 ceph-mon[98059]: pgmap v363: 97 pgs: 97 active+clean; 367 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 16 MiB/s wr, 719 op/s 2023-07-18T03:33:16.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:15 smithi134 ceph-mon[98216]: pgmap v364: 97 pgs: 97 active+clean; 375 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 16 MiB/s wr, 718 op/s 2023-07-18T03:33:16.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:15 smithi040 ceph-mon[96439]: pgmap v364: 97 pgs: 97 active+clean; 375 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 16 MiB/s wr, 718 op/s 2023-07-18T03:33:16.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:15 smithi064 ceph-mon[98059]: pgmap v364: 97 pgs: 97 active+clean; 375 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 16 MiB/s wr, 718 op/s 2023-07-18T03:33:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.529+0000 7fc7b96b5700 1 -- 172.21.15.40:0/2056149642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b40ff480 msgr2=0x7fc7b4103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.529+0000 7fc7b96b5700 1 --2- 172.21.15.40:0/2056149642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b40ff480 0x7fc7b4103d30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40098f0 tx=0x7fc7a4009c00 comp rx=0 tx=0).stop 2023-07-18T03:33:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.530+0000 7fc7b96b5700 1 -- 172.21.15.40:0/2056149642 shutdown_connections 2023-07-18T03:33:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.530+0000 7fc7b96b5700 1 --2- 172.21.15.40:0/2056149642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b40ff480 0x7fc7b4103d30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.530+0000 7fc7b96b5700 1 --2- 172.21.15.40:0/2056149642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7b40fd9a0 0x7fc7b40fddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.530+0000 7fc7b96b5700 1 --2- 172.21.15.40:0/2056149642 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b40fead0 0x7fc7b40fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.530+0000 7fc7b96b5700 1 -- 172.21.15.40:0/2056149642 >> 172.21.15.40:0/2056149642 conn(0x7fc7b40f9330 msgr2=0x7fc7b40fb780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.534+0000 7fc7b96b5700 1 -- 172.21.15.40:0/2056149642 shutdown_connections 2023-07-18T03:33:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.535+0000 7fc7b96b5700 1 -- 172.21.15.40:0/2056149642 wait complete. 2023-07-18T03:33:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.535+0000 7fc7b96b5700 1 Processor -- start 2023-07-18T03:33:17.551 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.536+0000 7fc7b96b5700 1 -- start start 2023-07-18T03:33:17.551 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.536+0000 7fc7b96b5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b40fd9a0 0x7fc7b41a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.551 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.536+0000 7fc7b96b5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b40fead0 0x7fc7b41a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.551 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.536+0000 7fc7b96b5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7b40ff480 0x7fc7b410e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.536+0000 7fc7b96b5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b410ea80 con 0x7fc7b40ff480 2023-07-18T03:33:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.536+0000 7fc7b96b5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b410ebc0 con 0x7fc7b40fead0 2023-07-18T03:33:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.536+0000 7fc7b96b5700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b410ed80 con 0x7fc7b40fd9a0 2023-07-18T03:33:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.542+0000 7fc7b2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b40fd9a0 0x7fc7b41a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.542+0000 7fc7b2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b40fd9a0 0x7fc7b41a5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48516/0 (socket says 172.21.15.40:48516) 2023-07-18T03:33:17.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.542+0000 7fc7b2ffd700 1 -- 172.21.15.40:0/983109992 learned_addr learned my addr 172.21.15.40:0/983109992 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.548+0000 7fc7b2ffd700 1 -- 172.21.15.40:0/983109992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b40fead0 msgr2=0x7fc7b41a6430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:33:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.548+0000 7fc7b2ffd700 1 --2- 172.21.15.40:0/983109992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b40fead0 0x7fc7b41a6430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.548+0000 7fc7b2ffd700 1 -- 172.21.15.40:0/983109992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7b40ff480 msgr2=0x7fc7b410e540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:33:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.548+0000 7fc7b2ffd700 1 --2- 172.21.15.40:0/983109992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7b40ff480 0x7fc7b410e540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.548+0000 7fc7b2ffd700 1 -- 172.21.15.40:0/983109992 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a4009520 con 0x7fc7b40fd9a0 2023-07-18T03:33:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.553+0000 7fc7b2ffd700 1 --2- 172.21.15.40:0/983109992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b40fd9a0 0x7fc7b41a5ef0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac007fa0 tx=0x7fc7ac00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.577+0000 7fc79bfff700 1 -- 172.21.15.40:0/983109992 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac00de60 con 0x7fc7b40fd9a0 2023-07-18T03:33:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.577+0000 7fc79bfff700 1 -- 172.21.15.40:0/983109992 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7ac00f040 con 0x7fc7b40fd9a0 2023-07-18T03:33:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.577+0000 7fc79bfff700 1 -- 172.21.15.40:0/983109992 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac014720 con 0x7fc7b40fd9a0 2023-07-18T03:33:17.580 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.578+0000 7fc7b96b5700 1 -- 172.21.15.40:0/983109992 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b410f060 con 0x7fc7b40fd9a0 2023-07-18T03:33:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.578+0000 7fc7b96b5700 1 -- 172.21.15.40:0/983109992 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b4069f30 con 0x7fc7b40fd9a0 2023-07-18T03:33:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.595+0000 7fc79bfff700 1 -- 172.21.15.40:0/983109992 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc7ac004ad0 con 0x7fc7b40fd9a0 2023-07-18T03:33:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.596+0000 7fc79bfff700 1 --2- 172.21.15.40:0/983109992 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc79c0a1350 0x7fc79c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.597+0000 7fc79bfff700 1 -- 172.21.15.40:0/983109992 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc7ac0cc160 con 0x7fc7b40fd9a0 2023-07-18T03:33:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.601+0000 7fc7c583bb80 1 Processor -- start 2023-07-18T03:33:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.604+0000 7fc7c583bb80 1 -- start start 2023-07-18T03:33:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.604+0000 7fc7b27fc700 1 --2- 172.21.15.40:0/983109992 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc79c0a1350 0x7fc79c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.604+0000 7fc7c583bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2bf416b0 0x558d2c0e5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.604+0000 7fc7c583bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0e61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.605+0000 7fc7c583bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c0e66e0 0x558d2c0e6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.605+0000 7fc7c583bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2bfeac50 con 0x558d2bf416b0 2023-07-18T03:33:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.605+0000 7fc7c583bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2c03d9e0 con 0x558d2c0e5840 2023-07-18T03:33:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.605+0000 7fc7c583bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2c036e70 con 0x558d2c0e66e0 2023-07-18T03:33:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.613+0000 7fc7b27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0e61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.613+0000 7fc7b27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0e61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42818/0 (socket says 172.21.15.40:42818) 2023-07-18T03:33:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.613+0000 7fc7b27fc700 1 -- 172.21.15.40:0/1150050709 learned_addr learned my addr 172.21.15.40:0/1150050709 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.619+0000 7fc7b37fe700 1 --2- 172.21.15.40:0/1150050709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c0e66e0 0x558d2c0e6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.619+0000 7fc7b27fc700 1 -- 172.21.15.40:0/1150050709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c0e66e0 msgr2=0x558d2c0e6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.619+0000 7fc7b27fc700 1 --2- 172.21.15.40:0/1150050709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c0e66e0 0x558d2c0e6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.619+0000 7fc7b27fc700 1 -- 172.21.15.40:0/1150050709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2bf416b0 msgr2=0x558d2c0e5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.619+0000 7fc7b27fc700 1 --2- 172.21.15.40:0/1150050709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2bf416b0 0x558d2c0e5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.619+0000 7fc7b27fc700 1 -- 172.21.15.40:0/1150050709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d2be64d80 con 0x558d2c0e5840 2023-07-18T03:33:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.620+0000 7fc7b37fe700 1 --2- 172.21.15.40:0/1150050709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c0e66e0 0x558d2c0e6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:33:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.620+0000 7fc7b27fc700 1 --2- 172.21.15.40:0/1150050709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0e61a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8014a00 tx=0x7fc7a8014d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6356acc41177eacc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.625+0000 7fc7997fa700 1 -- 172.21.15.40:0/1150050709 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8019070 con 0x558d2c0e5840 2023-07-18T03:33:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.625+0000 7fc7997fa700 1 -- 172.21.15.40:0/1150050709 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7a8012d50 con 0x558d2c0e5840 2023-07-18T03:33:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.625+0000 7fc7997fa700 1 -- 172.21.15.40:0/1150050709 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8005580 con 0x558d2c0e5840 2023-07-18T03:33:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.626+0000 7fc7c583bb80 1 -- 172.21.15.40:0/1150050709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 msgr2=0x558d2c0e61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.626+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/1150050709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0e61a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8014a00 tx=0x7fc7a8014d10 comp rx=0 tx=0).stop 2023-07-18T03:33:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.626+0000 7fc7b27fc700 1 --2- 172.21.15.40:0/983109992 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc79c0a1350 0x7fc79c0a3800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8012670 tx=0x7fc7a800d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.627+0000 7fc7c583bb80 1 -- 172.21.15.40:0/1150050709 shutdown_connections 2023-07-18T03:33:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.627+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/1150050709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c0e66e0 0x558d2c0e6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.627+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/1150050709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2bf416b0 0x558d2c0e5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.627+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/1150050709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0e61a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.627+0000 7fc7c583bb80 1 -- 172.21.15.40:0/1150050709 >> 172.21.15.40:0/1150050709 conn(0x558d2c0dabe0 msgr2=0x558d2c0dfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.627+0000 7fc7c583bb80 1 -- 172.21.15.40:0/1150050709 shutdown_connections 2023-07-18T03:33:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.627+0000 7fc7c583bb80 1 -- 172.21.15.40:0/1150050709 wait complete. 2023-07-18T03:33:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.628+0000 7fc7c583bb80 1 -- start start 2023-07-18T03:33:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.628+0000 7fc7c583bb80 1 Processor -- start 2023-07-18T03:33:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.628+0000 7fc7c583bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2bf416b0 0x558d2c0ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.628+0000 7fc7c583bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0eee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.628+0000 7fc7c583bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c0e66e0 0x558d2c0f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.628+0000 7fc7c583bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2bfd6a60 con 0x558d2bf416b0 2023-07-18T03:33:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.628+0000 7fc7c583bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2bfd6760 con 0x558d2c0e5840 2023-07-18T03:33:17.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.628+0000 7fc7c583bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2beba8c0 con 0x558d2c0e66e0 2023-07-18T03:33:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.628+0000 7fc7c583bb80 10 client.0 _ll_register_callbacks cb 0x558d2bfd4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.629+0000 7fc7b27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0eee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.629+0000 7fc7b27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0eee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42828/0 (socket says 172.21.15.40:42828) 2023-07-18T03:33:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.629+0000 7fc7b27fc700 1 -- 172.21.15.40:0/4157736419 learned_addr learned my addr 172.21.15.40:0/4157736419 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.630+0000 7fc7b27fc700 1 -- 172.21.15.40:0/4157736419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c0e66e0 msgr2=0x558d2c0f5e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:33:17.632 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.630+0000 7fc7b27fc700 1 --2- 172.21.15.40:0/4157736419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c0e66e0 0x558d2c0f5e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.630+0000 7fc7b27fc700 1 -- 172.21.15.40:0/4157736419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2bf416b0 msgr2=0x558d2c0ee8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.630+0000 7fc7b27fc700 1 --2- 172.21.15.40:0/4157736419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2bf416b0 0x558d2c0ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.630+0000 7fc7b27fc700 1 -- 172.21.15.40:0/4157736419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a80146b0 con 0x558d2c0e5840 2023-07-18T03:33:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.630+0000 7fc7b27fc700 1 --2- 172.21.15.40:0/4157736419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0eee30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8000c00 tx=0x7fc7a8004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c33f4bb4540c56cb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:17.640 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.643+0000 7fc796ff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:33:17.640 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.643+0000 7fc796ff5700 1 -- 172.21.15.40:0/4157736419 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8019030 con 0x558d2c0e5840 2023-07-18T03:33:17.640 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.643+0000 7fc796ff5700 1 -- 172.21.15.40:0/4157736419 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc7a801a070 con 0x558d2c0e5840 2023-07-18T03:33:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.643+0000 7fc7c583bb80 1 -- 172.21.15.40:0/4157736419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d2bf90490 con 0x558d2c0e5840 2023-07-18T03:33:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.644+0000 7fc796ff5700 1 -- 172.21.15.40:0/4157736419 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7a8005770 con 0x558d2c0e5840 2023-07-18T03:33:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.644+0000 7fc7c583bb80 10 client.25730 fetch_fsmap learned FSMap version 32 2023-07-18T03:33:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.644+0000 7fc7c583bb80 1 -- 172.21.15.40:0/4157736419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558d2be4dc00 con 0x558d2c0e5840 2023-07-18T03:33:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.652+0000 7fc796ff5700 1 -- 172.21.15.40:0/4157736419 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d2bf90490 con 0x558d2c0e5840 2023-07-18T03:33:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.653+0000 7fc796ff5700 1 -- 172.21.15.40:0/4157736419 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc7a8026470 con 0x558d2c0e5840 2023-07-18T03:33:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.653+0000 7fc7c583bb80 10 client.25730 fetch_fsmap finished waiting for FSMap version 32 2023-07-18T03:33:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.653+0000 7fc7c583bb80 10 client.25730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:33:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.653+0000 7fc7c583bb80 20 client.25730 populate_metadata read hostname 'smithi040' 2023-07-18T03:33:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.653+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/4157736419 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x558d2c0fa6f0 0x558d2c0fcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.653+0000 7fc7c583bb80 4 client.25730 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:33:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.653+0000 7fc7c583bb80 1 -- 172.21.15.40:0/4157736419 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d2c02fe50 con 0x558d2c0fa6f0 2023-07-18T03:33:17.651 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.653+0000 7fc7b2ffd700 1 --2- 172.21.15.40:0/4157736419 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x558d2c0fa6f0 0x558d2c0fcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.653+0000 7fc7b2ffd700 1 --2- 172.21.15.40:0/4157736419 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x558d2c0fa6f0 0x558d2c0fcb00 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf2d8ee5934dd1c7 server_cookie=37a6dd65f987edde in_seq=0 out_seq=0 2023-07-18T03:33:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.653+0000 7fc796ff5700 10 client.25730 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.654+0000 7fc796ff5700 1 -- 172.21.15.40:0/4157736419 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558d2c02fe50 con 0x558d2c0fa6f0 2023-07-18T03:33:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.654+0000 7fc796ff5700 10 client.25730 handle_command_reply: tid=0 2023-07-18T03:33:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.654+0000 7fc7c583bb80 1 client.25730 shutdown 2023-07-18T03:33:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.654+0000 7fc7c583bb80 1 -- 172.21.15.40:0/4157736419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 msgr2=0x558d2c0eee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.654+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/4157736419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0eee30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8000c00 tx=0x7fc7a8004a30 comp rx=0 tx=0).stop 2023-07-18T03:33:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.655+0000 7fc7c583bb80 1 -- 172.21.15.40:0/4157736419 shutdown_connections 2023-07-18T03:33:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.655+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/4157736419 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x558d2c0fa6f0 0x558d2c0fcb00 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.655+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/4157736419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2c0e66e0 0x558d2c0f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.655+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/4157736419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2bf416b0 0x558d2c0ee8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.655+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/4157736419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0eee30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.655+0000 7fc7c583bb80 1 -- 172.21.15.40:0/4157736419 >> 172.21.15.40:0/4157736419 conn(0x558d2c0dabe0 msgr2=0x558d2c0df4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.655+0000 7fc796ff5700 0 client.25730 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.655+0000 7fc7c583bb80 1 -- 172.21.15.40:0/4157736419 shutdown_connections 2023-07-18T03:33:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.655+0000 7fc7c583bb80 1 -- 172.21.15.40:0/4157736419 wait complete. 2023-07-18T03:33:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.655+0000 7fc7c583bb80 20 client.25730 trim_cache size 0 max 16384 2023-07-18T03:33:17.674 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.677+0000 7fc7c583bb80 1 Processor -- start 2023-07-18T03:33:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.677+0000 7fc7c583bb80 1 -- start start 2023-07-18T03:33:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.677+0000 7fc7c583bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2bf416b0 0x558d2c0ecb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.677+0000 7fc7c583bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.677+0000 7fc7c583bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2c0e66e0 0x558d2c0ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.675 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.677+0000 7fc7c583bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2c0ef7f0 con 0x558d2c0e66e0 2023-07-18T03:33:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.677+0000 7fc7c583bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2c0edda0 con 0x558d2c0e5840 2023-07-18T03:33:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.677+0000 7fc7c583bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2c0e0b80 con 0x558d2bf416b0 2023-07-18T03:33:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.677+0000 7fc7b27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.677+0000 7fc7b27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0f5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42832/0 (socket says 172.21.15.40:42832) 2023-07-18T03:33:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.677+0000 7fc7b27fc700 1 -- 172.21.15.40:0/887457938 learned_addr learned my addr 172.21.15.40:0/887457938 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.677+0000 7fc7b37fe700 1 --2- 172.21.15.40:0/887457938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2c0e66e0 0x558d2c0ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.678+0000 7fc7b27fc700 1 -- 172.21.15.40:0/887457938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2bf416b0 msgr2=0x558d2c0ecb40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:33:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.678+0000 7fc7b27fc700 1 --2- 172.21.15.40:0/887457938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2bf416b0 0x558d2c0ecb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.678+0000 7fc7b27fc700 1 -- 172.21.15.40:0/887457938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2c0e66e0 msgr2=0x558d2c0ee8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.678+0000 7fc7b27fc700 1 --2- 172.21.15.40:0/887457938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2c0e66e0 0x558d2c0ee8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.678+0000 7fc7b27fc700 1 -- 172.21.15.40:0/887457938 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d2be8b1f0 con 0x558d2c0e5840 2023-07-18T03:33:17.677 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.678+0000 7fc7b27fc700 1 --2- 172.21.15.40:0/887457938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0f5d90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8026800 tx=0x7fc7a8012740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd04ffc27255e8ec server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.678+0000 7fc799ffb700 1 -- 172.21.15.40:0/887457938 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8019070 con 0x558d2c0e5840 2023-07-18T03:33:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.678+0000 7fc799ffb700 1 -- 172.21.15.40:0/887457938 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7a8026af0 con 0x558d2c0e5840 2023-07-18T03:33:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.678+0000 7fc799ffb700 1 -- 172.21.15.40:0/887457938 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a802f440 con 0x558d2c0e5840 2023-07-18T03:33:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.679+0000 7fc7c583bb80 1 -- 172.21.15.40:0/887457938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 msgr2=0x558d2c0f5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.679+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/887457938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0f5d90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8026800 tx=0x7fc7a8012740 comp rx=0 tx=0).stop 2023-07-18T03:33:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.679+0000 7fc7c583bb80 1 -- 172.21.15.40:0/887457938 shutdown_connections 2023-07-18T03:33:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.679+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/887457938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2c0e66e0 0x558d2c0ee8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.679+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/887457938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2bf416b0 0x558d2c0ecb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.679+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/887457938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0f5d90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.679+0000 7fc7c583bb80 1 -- 172.21.15.40:0/887457938 >> 172.21.15.40:0/887457938 conn(0x558d2c0dabe0 msgr2=0x558d2c0ed560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.679+0000 7fc7c583bb80 1 -- 172.21.15.40:0/887457938 shutdown_connections 2023-07-18T03:33:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.679+0000 7fc7c583bb80 1 -- 172.21.15.40:0/887457938 wait complete. 2023-07-18T03:33:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.679+0000 7fc7c583bb80 1 -- start start 2023-07-18T03:33:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.679+0000 7fc7c583bb80 1 Processor -- start 2023-07-18T03:33:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.679+0000 7fc7c583bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2bf416b0 0x558d2c0f1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.679+0000 7fc7c583bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc7c583bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2c0e66e0 0x558d2c0f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc7c583bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2c0f5020 con 0x558d2c0e66e0 2023-07-18T03:33:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc7c583bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2c0f5190 con 0x558d2c0e5840 2023-07-18T03:33:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc7c583bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d2c0f5350 con 0x558d2bf416b0 2023-07-18T03:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc7c583bb80 10 client.0 _ll_register_callbacks cb 0x7fc7a80148d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc7b2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2bf416b0 0x558d2c0f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc7b27fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc7b2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2bf416b0 0x558d2c0f1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48540/0 (socket says 172.21.15.40:48540) 2023-07-18T03:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc7b2ffd700 1 -- 172.21.15.40:0/3529170107 learned_addr learned my addr 172.21.15.40:0/3529170107 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc7b37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2c0e66e0 0x558d2c0f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc7b27fc700 1 -- 172.21.15.40:0/3529170107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2bf416b0 msgr2=0x558d2c0f1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc7b27fc700 1 --2- 172.21.15.40:0/3529170107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2bf416b0 0x558d2c0f1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc7b27fc700 1 -- 172.21.15.40:0/3529170107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2c0e66e0 msgr2=0x558d2c0f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc7b27fc700 1 --2- 172.21.15.40:0/3529170107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2c0e66e0 0x558d2c0f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc7b27fc700 1 -- 172.21.15.40:0/3529170107 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d2be4dc00 con 0x558d2c0e5840 2023-07-18T03:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc7b27fc700 1 --2- 172.21.15.40:0/3529170107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0f1ac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8026880 tx=0x7fc7a80127b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f563e89c0eecdef0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc796ff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc796ff5700 1 -- 172.21.15.40:0/3529170107 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8019070 con 0x558d2c0e5840 2023-07-18T03:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc796ff5700 1 -- 172.21.15.40:0/3529170107 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc7a800bda0 con 0x558d2c0e5840 2023-07-18T03:33:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.680+0000 7fc7c583bb80 1 -- 172.21.15.40:0/3529170107 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7a8012d50 con 0x558d2c0e5840 2023-07-18T03:33:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.681+0000 7fc796ff5700 1 -- 172.21.15.40:0/3529170107 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7a800b2d0 con 0x558d2c0e5840 2023-07-18T03:33:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.681+0000 7fc7c583bb80 10 client.25736 fetch_fsmap learned FSMap version 32 2023-07-18T03:33:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.681+0000 7fc7c583bb80 1 -- 172.21.15.40:0/3529170107 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558d2be63250 con 0x558d2c0e5840 2023-07-18T03:33:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.681+0000 7fc796ff5700 1 -- 172.21.15.40:0/3529170107 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7a8012d50 con 0x558d2c0e5840 2023-07-18T03:33:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.681+0000 7fc796ff5700 1 -- 172.21.15.40:0/3529170107 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fc7a800cd20 con 0x558d2c0e5840 2023-07-18T03:33:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.681+0000 7fc7c583bb80 10 client.25736 fetch_fsmap finished waiting for FSMap version 32 2023-07-18T03:33:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.681+0000 7fc7c583bb80 10 client.25736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:33:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.681+0000 7fc7c583bb80 20 client.25736 populate_metadata read hostname 'smithi040' 2023-07-18T03:33:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.681+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/3529170107 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x558d2c14eab0 0x558d2c150e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.681+0000 7fc7c583bb80 4 client.25736 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:33:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.681+0000 7fc7c583bb80 1 -- 172.21.15.40:0/3529170107 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558d2bff1ff0 con 0x558d2c14eab0 2023-07-18T03:33:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.681+0000 7fc7b2ffd700 1 --2- 172.21.15.40:0/3529170107 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x558d2c14eab0 0x558d2c150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.682+0000 7fc796ff5700 10 client.25736 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:17.682+0000 7fc7b2ffd700 1 --2- 172.21.15.40:0/3529170107 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x558d2c14eab0 0x558d2c150e90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be54b89adef77e3a server_cookie=e55e528f91870964 in_seq=0 out_seq=0 2023-07-18T03:33:18.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:17 smithi134 ceph-mon[98216]: pgmap v365: 97 pgs: 97 active+clean; 379 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 16 MiB/s wr, 723 op/s 2023-07-18T03:33:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.079+0000 7fc796ff5700 1 -- 172.21.15.40:0/3529170107 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558d2bff1ff0 con 0x558d2c14eab0 2023-07-18T03:33:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.079+0000 7fc796ff5700 10 client.25736 handle_command_reply: tid=0 2023-07-18T03:33:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.079+0000 7fc7c583bb80 1 client.25736 shutdown 2023-07-18T03:33:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.079+0000 7fc7c583bb80 1 -- 172.21.15.40:0/3529170107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 msgr2=0x558d2c0f1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.079+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/3529170107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0f1ac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8026880 tx=0x7fc7a80127b0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.079+0000 7fc7c583bb80 1 -- 172.21.15.40:0/3529170107 shutdown_connections 2023-07-18T03:33:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.079+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/3529170107 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x558d2c14eab0 0x558d2c150e90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.079+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/3529170107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x558d2c0e66e0 0x558d2c0f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.079+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/3529170107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558d2bf416b0 0x558d2c0f1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.079+0000 7fc796ff5700 0 client.25736 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.079+0000 7fc7c583bb80 1 --2- 172.21.15.40:0/3529170107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558d2c0e5840 0x558d2c0f1ac0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.079+0000 7fc7c583bb80 1 -- 172.21.15.40:0/3529170107 >> 172.21.15.40:0/3529170107 conn(0x558d2c0dabe0 msgr2=0x558d2c0ecb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.079+0000 7fc7c583bb80 1 -- 172.21.15.40:0/3529170107 shutdown_connections 2023-07-18T03:33:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.079+0000 7fc7c583bb80 1 -- 172.21.15.40:0/3529170107 wait complete. 2023-07-18T03:33:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.080+0000 7fc7c583bb80 20 client.25736 trim_cache size 0 max 16384 2023-07-18T03:33:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:33:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:33:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:33:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:33:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.080+0000 7fc7b96b5700 1 -- 172.21.15.40:0/983109992 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc79c0a1350 msgr2=0x7fc79c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.080+0000 7fc7b96b5700 1 --2- 172.21.15.40:0/983109992 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc79c0a1350 0x7fc79c0a3800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8012670 tx=0x7fc7a800d000 comp rx=0 tx=0).stop 2023-07-18T03:33:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.080+0000 7fc7b96b5700 1 -- 172.21.15.40:0/983109992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b40fd9a0 msgr2=0x7fc7b41a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.080+0000 7fc7b96b5700 1 --2- 172.21.15.40:0/983109992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b40fd9a0 0x7fc7b41a5ef0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac007fa0 tx=0x7fc7ac00d3b0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.080+0000 7fc7b96b5700 1 -- 172.21.15.40:0/983109992 shutdown_connections 2023-07-18T03:33:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.080+0000 7fc7b96b5700 1 --2- 172.21.15.40:0/983109992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7b40ff480 0x7fc7b410e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.081+0000 7fc7b96b5700 1 --2- 172.21.15.40:0/983109992 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc79c0a1350 0x7fc79c0a3800 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.081+0000 7fc7b96b5700 1 --2- 172.21.15.40:0/983109992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b40fd9a0 0x7fc7b41a5ef0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.081+0000 7fc7b96b5700 1 --2- 172.21.15.40:0/983109992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b40fead0 0x7fc7b41a6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.081+0000 7fc7b96b5700 1 -- 172.21.15.40:0/983109992 >> 172.21.15.40:0/983109992 conn(0x7fc7b40f9330 msgr2=0x7fc7b40fb530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.081+0000 7fc7b96b5700 1 -- 172.21.15.40:0/983109992 shutdown_connections 2023-07-18T03:33:18.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:18.081+0000 7fc7b96b5700 1 -- 172.21.15.40:0/983109992 wait complete. 2023-07-18T03:33:18.088 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:17 smithi040 ceph-mon[96439]: pgmap v365: 97 pgs: 97 active+clean; 379 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 16 MiB/s wr, 723 op/s 2023-07-18T03:33:18.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:17 smithi064 ceph-mon[98059]: pgmap v365: 97 pgs: 97 active+clean; 379 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 16 MiB/s wr, 723 op/s 2023-07-18T03:33:20.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:19 smithi134 ceph-mon[98216]: pgmap v366: 97 pgs: 97 active+clean; 382 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 13 MiB/s wr, 567 op/s 2023-07-18T03:33:20.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:19 smithi040 ceph-mon[96439]: pgmap v366: 97 pgs: 97 active+clean; 382 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 13 MiB/s wr, 567 op/s 2023-07-18T03:33:20.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:19 smithi064 ceph-mon[98059]: pgmap v366: 97 pgs: 97 active+clean; 382 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 13 MiB/s wr, 567 op/s 2023-07-18T03:33:22.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:21 smithi134 ceph-mon[98216]: pgmap v367: 97 pgs: 97 active+clean; 382 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 170 op/s 2023-07-18T03:33:22.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:21 smithi040 ceph-mon[96439]: pgmap v367: 97 pgs: 97 active+clean; 382 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 170 op/s 2023-07-18T03:33:22.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:21 smithi064 ceph-mon[98059]: pgmap v367: 97 pgs: 97 active+clean; 382 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 170 op/s 2023-07-18T03:33:23.150 INFO:tasks.workunit.client.0.smithi040.stdout: 117 1098205 658 804654 706 171141 594 2023-07-18T03:33:24.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:23 smithi064 ceph-mon[98059]: pgmap v368: 97 pgs: 97 active+clean; 385 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2023-07-18T03:33:24.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:23 smithi134 ceph-mon[98216]: pgmap v368: 97 pgs: 97 active+clean; 385 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2023-07-18T03:33:24.339 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:23 smithi040 ceph-mon[96439]: pgmap v368: 97 pgs: 97 active+clean; 385 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2023-07-18T03:33:26.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:25 smithi134 ceph-mon[98216]: pgmap v369: 97 pgs: 97 active+clean; 391 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 148 op/s 2023-07-18T03:33:26.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:25 smithi040 ceph-mon[96439]: pgmap v369: 97 pgs: 97 active+clean; 391 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 148 op/s 2023-07-18T03:33:26.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:25 smithi064 ceph-mon[98059]: pgmap v369: 97 pgs: 97 active+clean; 391 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 148 op/s 2023-07-18T03:33:28.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:27 smithi134 ceph-mon[98216]: pgmap v370: 97 pgs: 97 active+clean; 391 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 146 op/s 2023-07-18T03:33:28.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:27 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:28.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:27 smithi040 ceph-mon[96439]: pgmap v370: 97 pgs: 97 active+clean; 391 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 146 op/s 2023-07-18T03:33:28.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:27 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:28.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:27 smithi064 ceph-mon[98059]: pgmap v370: 97 pgs: 97 active+clean; 391 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 146 op/s 2023-07-18T03:33:28.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:27 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:30.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:29 smithi134 ceph-mon[98216]: pgmap v371: 97 pgs: 97 active+clean; 398 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 148 op/s 2023-07-18T03:33:30.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:29 smithi040 ceph-mon[96439]: pgmap v371: 97 pgs: 97 active+clean; 398 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 148 op/s 2023-07-18T03:33:30.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:29 smithi064 ceph-mon[98059]: pgmap v371: 97 pgs: 97 active+clean; 398 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 148 op/s 2023-07-18T03:33:30.361 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:33:30.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.535+0000 7f7776bba700 1 -- 172.21.15.40:0/3247624737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77700fe4b0 msgr2=0x7f77700fe950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:30.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.535+0000 7f7776bba700 1 --2- 172.21.15.40:0/3247624737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77700fe4b0 0x7f77700fe950 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7768009f90 tx=0x7f776800b330 comp rx=0 tx=0).stop 2023-07-18T03:33:30.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.535+0000 7f7776bba700 1 -- 172.21.15.40:0/3247624737 shutdown_connections 2023-07-18T03:33:30.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.535+0000 7f7776bba700 1 --2- 172.21.15.40:0/3247624737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77700fe4b0 0x7f77700fe950 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.535+0000 7f7776bba700 1 --2- 172.21.15.40:0/3247624737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77700fffa0 0x7f77701003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.535+0000 7f7776bba700 1 --2- 172.21.15.40:0/3247624737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77700fdb20 0x7f77700fdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.535+0000 7f7776bba700 1 -- 172.21.15.40:0/3247624737 >> 172.21.15.40:0/3247624737 conn(0x7f77700f9770 msgr2=0x7f77700fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:30.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.535+0000 7f7776bba700 1 -- 172.21.15.40:0/3247624737 shutdown_connections 2023-07-18T03:33:30.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.535+0000 7f7776bba700 1 -- 172.21.15.40:0/3247624737 wait complete. 2023-07-18T03:33:30.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.536+0000 7f7776bba700 1 Processor -- start 2023-07-18T03:33:30.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.536+0000 7f7776bba700 1 -- start start 2023-07-18T03:33:30.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.536+0000 7f7776bba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77700fdb20 0x7f7770197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.536+0000 7f7776bba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77700fe4b0 0x7f7770198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.536+0000 7f7776bba700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77700fffa0 0x7f777019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.536+0000 7f7776bba700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777019f690 con 0x7f77700fdb20 2023-07-18T03:33:30.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.536+0000 7f7776bba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777019f7d0 con 0x7f77700fe4b0 2023-07-18T03:33:30.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.536+0000 7f7776bba700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777019f960 con 0x7f77700fffa0 2023-07-18T03:33:30.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.536+0000 7f7774956700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77700fdb20 0x7f7770197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.536+0000 7f7774956700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77700fdb20 0x7f7770197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59526/0 (socket says 172.21.15.40:59526) 2023-07-18T03:33:30.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.536+0000 7f7774956700 1 -- 172.21.15.40:0/1470190742 learned_addr learned my addr 172.21.15.40:0/1470190742 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:30.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.536+0000 7f7775157700 1 --2- 172.21.15.40:0/1470190742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77700fffa0 0x7f777019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.537+0000 7f776ffff700 1 --2- 172.21.15.40:0/1470190742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77700fe4b0 0x7f7770198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.537+0000 7f7775157700 1 -- 172.21.15.40:0/1470190742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77700fe4b0 msgr2=0x7f7770198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:30.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.537+0000 7f7775157700 1 --2- 172.21.15.40:0/1470190742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77700fe4b0 0x7f7770198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.537+0000 7f7775157700 1 -- 172.21.15.40:0/1470190742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77700fdb20 msgr2=0x7f7770197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:30.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.537+0000 7f7775157700 1 --2- 172.21.15.40:0/1470190742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77700fdb20 0x7f7770197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.537+0000 7f7775157700 1 -- 172.21.15.40:0/1470190742 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7768009c40 con 0x7f77700fffa0 2023-07-18T03:33:30.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.537+0000 7f7775157700 1 --2- 172.21.15.40:0/1470190742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77700fffa0 0x7f777019f0c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f776800b6c0 tx=0x7f7768004cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:30.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.537+0000 7f776dffb700 1 -- 172.21.15.40:0/1470190742 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776800e040 con 0x7f77700fffa0 2023-07-18T03:33:30.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.537+0000 7f7776bba700 1 -- 172.21.15.40:0/1470190742 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777019fbe0 con 0x7f77700fffa0 2023-07-18T03:33:30.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.537+0000 7f776dffb700 1 -- 172.21.15.40:0/1470190742 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77680093a0 con 0x7f77700fffa0 2023-07-18T03:33:30.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.538+0000 7f776dffb700 1 -- 172.21.15.40:0/1470190742 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7768005040 con 0x7f77700fffa0 2023-07-18T03:33:30.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.538+0000 7f7776bba700 1 -- 172.21.15.40:0/1470190742 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77701a0060 con 0x7f77700fffa0 2023-07-18T03:33:30.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.540+0000 7f776dffb700 1 -- 172.21.15.40:0/1470190742 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f776801d470 con 0x7f77700fffa0 2023-07-18T03:33:30.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.541+0000 7f776dffb700 1 --2- 172.21.15.40:0/1470190742 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f77600a1150 0x7f77600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.541+0000 7f776dffb700 1 -- 172.21.15.40:0/1470190742 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f77680cdec0 con 0x7f77700fffa0 2023-07-18T03:33:30.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.541+0000 7f7774956700 1 --2- 172.21.15.40:0/1470190742 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f77600a1150 0x7f77600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.542+0000 7f7774956700 1 --2- 172.21.15.40:0/1470190742 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f77600a1150 0x7f77600a3600 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7764009d20 tx=0x7f7764009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:30.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.543+0000 7f7782d40b80 1 Processor -- start 2023-07-18T03:33:30.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.543+0000 7f7782d40b80 1 -- start start 2023-07-18T03:33:30.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.543+0000 7f7782d40b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.543+0000 7f7782d40b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 0x564d504cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.543+0000 7f7782d40b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504ccc70 0x564d504cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.543+0000 7f7782d40b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d503d0cb0 con 0x564d504cbe20 2023-07-18T03:33:30.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.543+0000 7f7782d40b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d50423a40 con 0x564d504cb4d0 2023-07-18T03:33:30.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.543+0000 7f7782d40b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d5041ced0 con 0x564d504ccc70 2023-07-18T03:33:30.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.543+0000 7f7774956700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.543+0000 7f776ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 0x564d504cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.543+0000 7f776ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 0x564d504cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59536/0 (socket says 172.21.15.40:59536) 2023-07-18T03:33:30.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.543+0000 7f7774956700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42860/0 (socket says 172.21.15.40:42860) 2023-07-18T03:33:30.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.543+0000 7f776ffff700 1 -- 172.21.15.40:0/3784447811 learned_addr learned my addr 172.21.15.40:0/3784447811 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:30.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.543+0000 7f7775157700 1 --2- 172.21.15.40:0/3784447811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504ccc70 0x564d504cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.544+0000 7f7775157700 1 -- 172.21.15.40:0/3784447811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 msgr2=0x564d504cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:30.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.544+0000 7f7775157700 1 --2- 172.21.15.40:0/3784447811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.544+0000 7f7775157700 1 -- 172.21.15.40:0/3784447811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 msgr2=0x564d504cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:30.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.544+0000 7f7775157700 1 --2- 172.21.15.40:0/3784447811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 0x564d504cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.544+0000 7f7775157700 1 -- 172.21.15.40:0/3784447811 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d5024ade0 con 0x564d504ccc70 2023-07-18T03:33:30.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.544+0000 7f7775157700 1 --2- 172.21.15.40:0/3784447811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504ccc70 0x564d504cd0e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f77680cb9f0 tx=0x7f7768008fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f38cac3be2d0494d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:30.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.544+0000 7f775affd700 1 -- 172.21.15.40:0/3784447811 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776809aca0 con 0x564d504ccc70 2023-07-18T03:33:30.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.544+0000 7f775affd700 1 -- 172.21.15.40:0/3784447811 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f776809b340 con 0x564d504ccc70 2023-07-18T03:33:30.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.544+0000 7f775affd700 1 -- 172.21.15.40:0/3784447811 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776809b5a0 con 0x564d504ccc70 2023-07-18T03:33:30.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.545+0000 7f7782d40b80 1 -- 172.21.15.40:0/3784447811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504ccc70 msgr2=0x564d504cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:30.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.545+0000 7f7782d40b80 1 --2- 172.21.15.40:0/3784447811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504ccc70 0x564d504cd0e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f77680cb9f0 tx=0x7f7768008fd0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.545+0000 7f7782d40b80 1 -- 172.21.15.40:0/3784447811 shutdown_connections 2023-07-18T03:33:30.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.545+0000 7f7782d40b80 1 --2- 172.21.15.40:0/3784447811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504ccc70 0x564d504cd0e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.545+0000 7f7782d40b80 1 --2- 172.21.15.40:0/3784447811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.545+0000 7f7782d40b80 1 --2- 172.21.15.40:0/3784447811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 0x564d504cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.545+0000 7f7782d40b80 1 -- 172.21.15.40:0/3784447811 >> 172.21.15.40:0/3784447811 conn(0x564d504c0da0 msgr2=0x564d504c61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:30.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.545+0000 7f7782d40b80 1 -- 172.21.15.40:0/3784447811 shutdown_connections 2023-07-18T03:33:30.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.545+0000 7f7782d40b80 1 -- 172.21.15.40:0/3784447811 wait complete. 2023-07-18T03:33:30.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.546+0000 7f7782d40b80 1 -- start start 2023-07-18T03:33:30.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.546+0000 7f7782d40b80 1 Processor -- start 2023-07-18T03:33:30.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.546+0000 7f7782d40b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.546+0000 7f7782d40b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 0x564d504d5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.546+0000 7f7782d40b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504ccc70 0x564d504dc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.546+0000 7f7782d40b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d503bcac0 con 0x564d504cbe20 2023-07-18T03:33:30.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.546+0000 7f7782d40b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d503bc7c0 con 0x564d504cb4d0 2023-07-18T03:33:30.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.546+0000 7f7782d40b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d502a0920 con 0x564d504ccc70 2023-07-18T03:33:30.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.546+0000 7f7782d40b80 10 client.0 _ll_register_callbacks cb 0x564d503ba760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:30.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.547+0000 7f7774956700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.547+0000 7f776ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 0x564d504d5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.547+0000 7f7774956700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42866/0 (socket says 172.21.15.40:42866) 2023-07-18T03:33:30.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.547+0000 7f776ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 0x564d504d5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59542/0 (socket says 172.21.15.40:59542) 2023-07-18T03:33:30.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.547+0000 7f7774956700 1 -- 172.21.15.40:0/1108317681 learned_addr learned my addr 172.21.15.40:0/1108317681 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:30.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.547+0000 7f7775157700 1 --2- 172.21.15.40:0/1108317681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504ccc70 0x564d504dc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.547+0000 7f7774956700 1 -- 172.21.15.40:0/1108317681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504ccc70 msgr2=0x564d504dc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:30.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.547+0000 7f7774956700 1 --2- 172.21.15.40:0/1108317681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504ccc70 0x564d504dc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.547+0000 7f7774956700 1 -- 172.21.15.40:0/1108317681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 msgr2=0x564d504d5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:30.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.547+0000 7f7774956700 1 --2- 172.21.15.40:0/1108317681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 0x564d504d5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.547+0000 7f7774956700 1 -- 172.21.15.40:0/1108317681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77701a0060 con 0x564d504cb4d0 2023-07-18T03:33:30.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.547+0000 7f7774956700 1 --2- 172.21.15.40:0/1108317681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d4f00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f776401dcb0 tx=0x7f77640047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=297aacc243dde3ec server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:30.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.547+0000 7f774ffff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:33:30.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.547+0000 7f774ffff700 1 -- 172.21.15.40:0/1108317681 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764022070 con 0x564d504cb4d0 2023-07-18T03:33:30.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.547+0000 7f774ffff700 1 -- 172.21.15.40:0/1108317681 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f776401cad0 con 0x564d504cb4d0 2023-07-18T03:33:30.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.547+0000 7f7782d40b80 1 -- 172.21.15.40:0/1108317681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d503764f0 con 0x564d504cb4d0 2023-07-18T03:33:30.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.548+0000 7f774ffff700 1 -- 172.21.15.40:0/1108317681 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f776401cc90 con 0x564d504cb4d0 2023-07-18T03:33:30.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.548+0000 7f7782d40b80 10 client.25748 fetch_fsmap learned FSMap version 32 2023-07-18T03:33:30.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.548+0000 7f7782d40b80 1 -- 172.21.15.40:0/1108317681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564d50233c60 con 0x564d504cb4d0 2023-07-18T03:33:30.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.548+0000 7f774ffff700 1 -- 172.21.15.40:0/1108317681 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7764035080 con 0x564d504cb4d0 2023-07-18T03:33:30.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.548+0000 7f774ffff700 1 -- 172.21.15.40:0/1108317681 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f776402f030 con 0x564d504cb4d0 2023-07-18T03:33:30.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.548+0000 7f7782d40b80 10 client.25748 fetch_fsmap finished waiting for FSMap version 32 2023-07-18T03:33:30.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.548+0000 7f7782d40b80 10 client.25748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:33:30.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.548+0000 7f7782d40b80 20 client.25748 populate_metadata read hostname 'smithi040' 2023-07-18T03:33:30.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.548+0000 7f7782d40b80 1 --2- 172.21.15.40:0/1108317681 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564d504e0d30 0x564d504e3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.548+0000 7f7782d40b80 4 client.25748 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:33:30.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.548+0000 7f7782d40b80 1 -- 172.21.15.40:0/1108317681 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d50415eb0 con 0x564d504e0d30 2023-07-18T03:33:30.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.548+0000 7f776ffff700 1 --2- 172.21.15.40:0/1108317681 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564d504e0d30 0x564d504e3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.549+0000 7f776ffff700 1 --2- 172.21.15.40:0/1108317681 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564d504e0d30 0x564d504e3140 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f454ad32f51ef680 server_cookie=2242fcd2aebd72ca in_seq=0 out_seq=0 2023-07-18T03:33:30.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.549+0000 7f774ffff700 10 client.25748 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:30.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.550+0000 7f774ffff700 1 -- 172.21.15.40:0/1108317681 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564d50415eb0 con 0x564d504e0d30 2023-07-18T03:33:30.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.550+0000 7f774ffff700 10 client.25748 handle_command_reply: tid=0 2023-07-18T03:33:30.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.550+0000 7f7782d40b80 1 client.25748 shutdown 2023-07-18T03:33:30.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.550+0000 7f7782d40b80 1 -- 172.21.15.40:0/1108317681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 msgr2=0x564d504d4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:30.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.550+0000 7f7782d40b80 1 --2- 172.21.15.40:0/1108317681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d4f00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f776401dcb0 tx=0x7f77640047c0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.550+0000 7f7782d40b80 1 -- 172.21.15.40:0/1108317681 shutdown_connections 2023-07-18T03:33:30.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.550+0000 7f7782d40b80 1 --2- 172.21.15.40:0/1108317681 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564d504e0d30 0x564d504e3140 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.550+0000 7f7782d40b80 1 --2- 172.21.15.40:0/1108317681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504ccc70 0x564d504dc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.550+0000 7f7782d40b80 1 --2- 172.21.15.40:0/1108317681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d4f00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.550+0000 7f774ffff700 0 client.25748 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:30.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.550+0000 7f7782d40b80 1 --2- 172.21.15.40:0/1108317681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 0x564d504d5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.550+0000 7f7782d40b80 1 -- 172.21.15.40:0/1108317681 >> 172.21.15.40:0/1108317681 conn(0x564d504c0da0 msgr2=0x564d504c58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:30.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.550+0000 7f7782d40b80 1 -- 172.21.15.40:0/1108317681 shutdown_connections 2023-07-18T03:33:30.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.550+0000 7f7782d40b80 1 -- 172.21.15.40:0/1108317681 wait complete. 2023-07-18T03:33:30.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.550+0000 7f7782d40b80 20 client.25748 trim_cache size 0 max 16384 2023-07-18T03:33:30.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7782d40b80 1 Processor -- start 2023-07-18T03:33:30.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7782d40b80 1 -- start start 2023-07-18T03:33:30.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7782d40b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7782d40b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504cbe20 0x564d504dc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7782d40b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504ccc70 0x564d504cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7782d40b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d504de120 con 0x564d504ccc70 2023-07-18T03:33:30.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7782d40b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d504c7070 con 0x564d504cb4d0 2023-07-18T03:33:30.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7782d40b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d504c0970 con 0x564d504cbe20 2023-07-18T03:33:30.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7775157700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504ccc70 0x564d504cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7774956700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f776ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504cbe20 0x564d504dc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7775157700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504ccc70 0x564d504cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59552/0 (socket says 172.21.15.40:59552) 2023-07-18T03:33:30.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7774956700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42874/0 (socket says 172.21.15.40:42874) 2023-07-18T03:33:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7775157700 1 -- 172.21.15.40:0/3534777749 learned_addr learned my addr 172.21.15.40:0/3534777749 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7774956700 1 -- 172.21.15.40:0/3534777749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504cbe20 msgr2=0x564d504dc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7774956700 1 --2- 172.21.15.40:0/3534777749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504cbe20 0x564d504dc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7774956700 1 -- 172.21.15.40:0/3534777749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504ccc70 msgr2=0x564d504cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7774956700 1 --2- 172.21.15.40:0/3534777749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504ccc70 0x564d504cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7774956700 1 -- 172.21.15.40:0/3534777749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d50271250 con 0x564d504cb4d0 2023-07-18T03:33:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f776ffff700 1 --2- 172.21.15.40:0/3534777749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504cbe20 0x564d504dc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:30.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.569+0000 7f7775157700 1 --2- 172.21.15.40:0/3534777749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504ccc70 0x564d504cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:30.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.570+0000 7f7774956700 1 --2- 172.21.15.40:0/3534777749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d3060 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7764013010 tx=0x7f7764009e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0389431a43390e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:30.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.570+0000 7f775b7fe700 1 -- 172.21.15.40:0/3534777749 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764022070 con 0x564d504cb4d0 2023-07-18T03:33:30.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.570+0000 7f775b7fe700 1 -- 172.21.15.40:0/3534777749 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564d503764f0 con 0x564d504cb4d0 2023-07-18T03:33:30.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.570+0000 7f775b7fe700 1 -- 172.21.15.40:0/3534777749 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764031d30 con 0x564d504cb4d0 2023-07-18T03:33:30.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.571+0000 7f7782d40b80 1 -- 172.21.15.40:0/3534777749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 msgr2=0x564d504d3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:30.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.571+0000 7f7782d40b80 1 --2- 172.21.15.40:0/3534777749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d3060 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7764013010 tx=0x7f7764009e80 comp rx=0 tx=0).stop 2023-07-18T03:33:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.571+0000 7f7782d40b80 1 -- 172.21.15.40:0/3534777749 shutdown_connections 2023-07-18T03:33:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.571+0000 7f7782d40b80 1 --2- 172.21.15.40:0/3534777749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504ccc70 0x564d504cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.571+0000 7f7782d40b80 1 --2- 172.21.15.40:0/3534777749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d3060 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.571+0000 7f7782d40b80 1 --2- 172.21.15.40:0/3534777749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504cbe20 0x564d504dc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.571+0000 7f7782d40b80 1 -- 172.21.15.40:0/3534777749 >> 172.21.15.40:0/3534777749 conn(0x564d504c0da0 msgr2=0x564d504d3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.571+0000 7f7782d40b80 1 -- 172.21.15.40:0/3534777749 shutdown_connections 2023-07-18T03:33:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.571+0000 7f7782d40b80 1 -- 172.21.15.40:0/3534777749 wait complete. 2023-07-18T03:33:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.571+0000 7f7782d40b80 1 -- start start 2023-07-18T03:33:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.571+0000 7f7782d40b80 1 Processor -- start 2023-07-18T03:33:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f7782d40b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f7782d40b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 0x564d504d9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f7782d40b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504ccc70 0x564d504daa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f7782d40b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d504dafa0 con 0x564d504cbe20 2023-07-18T03:33:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f7782d40b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d504db110 con 0x564d504cb4d0 2023-07-18T03:33:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f7782d40b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d504db2d0 con 0x564d504ccc70 2023-07-18T03:33:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f7782d40b80 10 client.0 _ll_register_callbacks cb 0x564d503ba760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f7774956700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f776ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 0x564d504d9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f7774956700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42880/0 (socket says 172.21.15.40:42880) 2023-07-18T03:33:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f776ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 0x564d504d9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59556/0 (socket says 172.21.15.40:59556) 2023-07-18T03:33:30.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f7774956700 1 -- 172.21.15.40:0/1293718005 learned_addr learned my addr 172.21.15.40:0/1293718005 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:30.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f7775157700 1 --2- 172.21.15.40:0/1293718005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504ccc70 0x564d504daa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f7774956700 1 -- 172.21.15.40:0/1293718005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504ccc70 msgr2=0x564d504daa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:30.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f7774956700 1 --2- 172.21.15.40:0/1293718005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504ccc70 0x564d504daa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f7774956700 1 -- 172.21.15.40:0/1293718005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 msgr2=0x564d504d9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:30.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f7774956700 1 --2- 172.21.15.40:0/1293718005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 0x564d504d9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f7774956700 1 -- 172.21.15.40:0/1293718005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775c009bf0 con 0x564d504cb4d0 2023-07-18T03:33:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.572+0000 7f7775157700 1 --2- 172.21.15.40:0/1293718005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504ccc70 0x564d504daa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.573+0000 7f7774956700 1 --2- 172.21.15.40:0/1293718005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d9510 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7764026f60 tx=0x7f776401cd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67411a2c29388be server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.573+0000 7f774ffff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:33:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.573+0000 7f774ffff700 1 -- 172.21.15.40:0/1293718005 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764022030 con 0x564d504cb4d0 2023-07-18T03:33:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.573+0000 7f774ffff700 1 -- 172.21.15.40:0/1293718005 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7764023070 con 0x564d504cb4d0 2023-07-18T03:33:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.573+0000 7f7782d40b80 1 -- 172.21.15.40:0/1293718005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f776809b340 con 0x564d504cb4d0 2023-07-18T03:33:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.573+0000 7f774ffff700 1 -- 172.21.15.40:0/1293718005 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f776401cea0 con 0x564d504cb4d0 2023-07-18T03:33:30.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.573+0000 7f7782d40b80 10 client.25754 fetch_fsmap learned FSMap version 32 2023-07-18T03:33:30.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.573+0000 7f7782d40b80 1 -- 172.21.15.40:0/1293718005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564d5024ade0 con 0x564d504cb4d0 2023-07-18T03:33:30.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.573+0000 7f774ffff700 1 -- 172.21.15.40:0/1293718005 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f776404e020 con 0x564d504cb4d0 2023-07-18T03:33:30.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.573+0000 7f774ffff700 1 -- 172.21.15.40:0/1293718005 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f776401dce0 con 0x564d504cb4d0 2023-07-18T03:33:30.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.573+0000 7f7782d40b80 10 client.25754 fetch_fsmap finished waiting for FSMap version 32 2023-07-18T03:33:30.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.573+0000 7f7782d40b80 10 client.25754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:33:30.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.573+0000 7f7782d40b80 20 client.25754 populate_metadata read hostname 'smithi040' 2023-07-18T03:33:30.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.573+0000 7f7782d40b80 1 --2- 172.21.15.40:0/1293718005 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564d504e0d30 0x564d50538c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:30.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.573+0000 7f7782d40b80 4 client.25754 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:33:30.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.573+0000 7f7782d40b80 1 -- 172.21.15.40:0/1293718005 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564d503d8050 con 0x564d504e0d30 2023-07-18T03:33:30.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.574+0000 7f776ffff700 1 --2- 172.21.15.40:0/1293718005 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564d504e0d30 0x564d50538c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:30.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.574+0000 7f774ffff700 10 client.25754 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:30.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.574+0000 7f776ffff700 1 --2- 172.21.15.40:0/1293718005 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564d504e0d30 0x564d50538c00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea2ef1a9b73f59d8 server_cookie=31ba50eeb74626bf in_seq=0 out_seq=0 2023-07-18T03:33:30.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.574+0000 7f774ffff700 1 -- 172.21.15.40:0/1293718005 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x564d503d8050 con 0x564d504e0d30 2023-07-18T03:33:30.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.574+0000 7f774ffff700 10 client.25754 handle_command_reply: tid=0 2023-07-18T03:33:30.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.574+0000 7f7782d40b80 1 client.25754 shutdown 2023-07-18T03:33:30.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.574+0000 7f7782d40b80 1 -- 172.21.15.40:0/1293718005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 msgr2=0x564d504d9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:30.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.574+0000 7f7782d40b80 1 --2- 172.21.15.40:0/1293718005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d9510 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7764026f60 tx=0x7f776401cd00 comp rx=0 tx=0).stop 2023-07-18T03:33:30.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.574+0000 7f7782d40b80 1 -- 172.21.15.40:0/1293718005 shutdown_connections 2023-07-18T03:33:30.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.574+0000 7f7782d40b80 1 --2- 172.21.15.40:0/1293718005 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564d504e0d30 0x564d50538c00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.574+0000 7f7782d40b80 1 --2- 172.21.15.40:0/1293718005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d504ccc70 0x564d504daa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.574+0000 7f7782d40b80 1 --2- 172.21.15.40:0/1293718005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d504cb4d0 0x564d504d9510 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.574+0000 7f7782d40b80 1 --2- 172.21.15.40:0/1293718005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d504cbe20 0x564d504d9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.574+0000 7f774ffff700 0 client.25754 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:30.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.574+0000 7f7782d40b80 1 -- 172.21.15.40:0/1293718005 >> 172.21.15.40:0/1293718005 conn(0x564d504c0da0 msgr2=0x564d504cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:30.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.575+0000 7f7782d40b80 1 -- 172.21.15.40:0/1293718005 shutdown_connections 2023-07-18T03:33:30.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.575+0000 7f7782d40b80 1 -- 172.21.15.40:0/1293718005 wait complete. 2023-07-18T03:33:30.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.575+0000 7f7782d40b80 20 client.25754 trim_cache size 0 max 16384 2023-07-18T03:33:30.596 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:33:30.596 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-18T03:33:30.596 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:33:30.596 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:33:30.597 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:33:30.597 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:33:30.597 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:33:30.597 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:33:30.597 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:33:30.597 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:33:30.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.575+0000 7f7776bba700 1 -- 172.21.15.40:0/1470190742 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f77600a1150 msgr2=0x7f77600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.575+0000 7f7776bba700 1 --2- 172.21.15.40:0/1470190742 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f77600a1150 0x7f77600a3600 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7764009d20 tx=0x7f7764009260 comp rx=0 tx=0).stop 2023-07-18T03:33:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.575+0000 7f7776bba700 1 -- 172.21.15.40:0/1470190742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77700fffa0 msgr2=0x7f777019f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.575+0000 7f7776bba700 1 --2- 172.21.15.40:0/1470190742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77700fffa0 0x7f777019f0c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f776800b6c0 tx=0x7f7768004cc0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.575+0000 7f7776bba700 1 -- 172.21.15.40:0/1470190742 shutdown_connections 2023-07-18T03:33:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.575+0000 7f7776bba700 1 --2- 172.21.15.40:0/1470190742 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f77600a1150 0x7f77600a3600 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.575+0000 7f7776bba700 1 --2- 172.21.15.40:0/1470190742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77700fffa0 0x7f777019f0c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.575+0000 7f7776bba700 1 --2- 172.21.15.40:0/1470190742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77700fdb20 0x7f7770197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.575+0000 7f7776bba700 1 --2- 172.21.15.40:0/1470190742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77700fe4b0 0x7f7770198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:30.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.575+0000 7f7776bba700 1 -- 172.21.15.40:0/1470190742 >> 172.21.15.40:0/1470190742 conn(0x7f77700f9770 msgr2=0x7f77700fb380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:30.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.575+0000 7f7776bba700 1 -- 172.21.15.40:0/1470190742 shutdown_connections 2023-07-18T03:33:30.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:33:30.575+0000 7f7776bba700 1 -- 172.21.15.40:0/1470190742 wait complete. 2023-07-18T03:33:30.600 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:33:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.292+0000 7f1b382d5700 1 -- 172.21.15.40:0/827362855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b30102c10 msgr2=0x7f1b30105050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.292+0000 7f1b382d5700 1 --2- 172.21.15.40:0/827362855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b30102c10 0x7f1b30105050 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00b0a0 tx=0x7f1b2c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.292+0000 7f1b382d5700 1 -- 172.21.15.40:0/827362855 shutdown_connections 2023-07-18T03:33:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.292+0000 7f1b382d5700 1 --2- 172.21.15.40:0/827362855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b301056e0 0x7f1b30107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.292+0000 7f1b382d5700 1 --2- 172.21.15.40:0/827362855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b30068600 0x7f1b301026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.292+0000 7f1b382d5700 1 --2- 172.21.15.40:0/827362855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b30102c10 0x7f1b30105050 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00b0a0 tx=0x7f1b2c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.292+0000 7f1b382d5700 1 -- 172.21.15.40:0/827362855 >> 172.21.15.40:0/827362855 conn(0x7f1b300fa000 msgr2=0x7f1b300fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.292+0000 7f1b382d5700 1 -- 172.21.15.40:0/827362855 shutdown_connections 2023-07-18T03:33:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.292+0000 7f1b382d5700 1 -- 172.21.15.40:0/827362855 wait complete. 2023-07-18T03:33:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.293+0000 7f1b382d5700 1 Processor -- start 2023-07-18T03:33:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.293+0000 7f1b382d5700 1 -- start start 2023-07-18T03:33:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.294+0000 7f1b382d5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b30068600 0x7f1b30071a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.294+0000 7f1b382d5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b301056e0 0x7f1b30071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.294+0000 7f1b382d5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b3010c6b0 0x7f1b30070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.294+0000 7f1b382d5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b30070c60 con 0x7f1b30068600 2023-07-18T03:33:31.292 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.294+0000 7f1b382d5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b30070dd0 con 0x7f1b301056e0 2023-07-18T03:33:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.294+0000 7f1b382d5700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b30070f90 con 0x7f1b3010c6b0 2023-07-18T03:33:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.295+0000 7f1b36071700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b30068600 0x7f1b30071a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.295+0000 7f1b36071700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b30068600 0x7f1b30071a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59568/0 (socket says 172.21.15.40:59568) 2023-07-18T03:33:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.295+0000 7f1b36071700 1 -- 172.21.15.40:0/1138726109 learned_addr learned my addr 172.21.15.40:0/1138726109 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.295+0000 7f1b36071700 1 -- 172.21.15.40:0/1138726109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b3010c6b0 msgr2=0x7f1b30070580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:33:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.295+0000 7f1b36071700 1 --2- 172.21.15.40:0/1138726109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b3010c6b0 0x7f1b30070580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.295+0000 7f1b36071700 1 -- 172.21.15.40:0/1138726109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b301056e0 msgr2=0x7f1b30071fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:33:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.295+0000 7f1b36071700 1 --2- 172.21.15.40:0/1138726109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b301056e0 0x7f1b30071fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.295+0000 7f1b36071700 1 -- 172.21.15.40:0/1138726109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2c009d00 con 0x7f1b30068600 2023-07-18T03:33:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.296+0000 7f1b36071700 1 --2- 172.21.15.40:0/1138726109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b30068600 0x7f1b30071a70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800e970 tx=0x7f1b2800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.296+0000 7f1b277fe700 1 -- 172.21.15.40:0/1138726109 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28009790 con 0x7f1b30068600 2023-07-18T03:33:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.296+0000 7f1b277fe700 1 -- 172.21.15.40:0/1138726109 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b28004cf0 con 0x7f1b30068600 2023-07-18T03:33:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.296+0000 7f1b382d5700 1 -- 172.21.15.40:0/1138726109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b30071270 con 0x7f1b30068600 2023-07-18T03:33:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.296+0000 7f1b277fe700 1 -- 172.21.15.40:0/1138726109 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28005680 con 0x7f1b30068600 2023-07-18T03:33:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.296+0000 7f1b382d5700 1 -- 172.21.15.40:0/1138726109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3006f100 con 0x7f1b30068600 2023-07-18T03:33:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.301+0000 7f1b277fe700 1 -- 172.21.15.40:0/1138726109 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1b28004750 con 0x7f1b30068600 2023-07-18T03:33:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.302+0000 7f1b277fe700 1 --2- 172.21.15.40:0/1138726109 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1b1c0a11d0 0x7f1b1c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.302+0000 7f1b4445bb80 1 Processor -- start 2023-07-18T03:33:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.302+0000 7f1b277fe700 1 -- 172.21.15.40:0/1138726109 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1b28014070 con 0x7f1b30068600 2023-07-18T03:33:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.302+0000 7f1b35870700 1 --2- 172.21.15.40:0/1138726109 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1b1c0a11d0 0x7f1b1c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.303+0000 7f1b4445bb80 1 -- start start 2023-07-18T03:33:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.303+0000 7f1b4445bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46a746b0 0x564d46c18300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.303+0000 7f1b4445bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 0x564d46c191a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.303+0000 7f1b4445bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46c196e0 0x564d46c19b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.303+0000 7f1b4445bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d46b1dc50 con 0x564d46a746b0 2023-07-18T03:33:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.303+0000 7f1b4445bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d46b709e0 con 0x564d46c18840 2023-07-18T03:33:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.303+0000 7f1b4445bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d46b69e70 con 0x564d46c196e0 2023-07-18T03:33:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.303+0000 7f1b36071700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46a746b0 0x564d46c18300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.304+0000 7f1b35870700 1 --2- 172.21.15.40:0/1138726109 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1b1c0a11d0 0x7f1b1c0a3680 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00b070 tx=0x7f1b2c006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.304+0000 7f1b35870700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 0x564d46c191a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.304+0000 7f1b36071700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46a746b0 0x564d46c18300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59576/0 (socket says 172.21.15.40:59576) 2023-07-18T03:33:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.304+0000 7f1b35870700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 0x564d46c191a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42906/0 (socket says 172.21.15.40:42906) 2023-07-18T03:33:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.304+0000 7f1b36071700 1 -- 172.21.15.40:0/3520381139 learned_addr learned my addr 172.21.15.40:0/3520381139 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.304+0000 7f1b36872700 1 --2- 172.21.15.40:0/3520381139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46c196e0 0x564d46c19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.304+0000 7f1b36071700 1 -- 172.21.15.40:0/3520381139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46c196e0 msgr2=0x564d46c19b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.304+0000 7f1b36071700 1 --2- 172.21.15.40:0/3520381139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46c196e0 0x564d46c19b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.304+0000 7f1b36071700 1 -- 172.21.15.40:0/3520381139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 msgr2=0x564d46c191a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:31.303 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.304+0000 7f1b36071700 1 --2- 172.21.15.40:0/3520381139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 0x564d46c191a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.304+0000 7f1b36071700 1 -- 172.21.15.40:0/3520381139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d46997d80 con 0x564d46a746b0 2023-07-18T03:33:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.304+0000 7f1b36071700 1 --2- 172.21.15.40:0/3520381139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46a746b0 0x564d46c18300 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1b280966a0 tx=0x7f1b280996c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc72885994e54d1a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.306+0000 7f1b24dac700 1 -- 172.21.15.40:0/3520381139 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2809dc60 con 0x564d46a746b0 2023-07-18T03:33:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.306+0000 7f1b24dac700 1 -- 172.21.15.40:0/3520381139 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b280a2080 con 0x564d46a746b0 2023-07-18T03:33:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.306+0000 7f1b24dac700 1 -- 172.21.15.40:0/3520381139 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28099e90 con 0x564d46a746b0 2023-07-18T03:33:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.307+0000 7f1b4445bb80 1 -- 172.21.15.40:0/3520381139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46a746b0 msgr2=0x564d46c18300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.307+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/3520381139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46a746b0 0x564d46c18300 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1b280966a0 tx=0x7f1b280996c0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.307+0000 7f1b4445bb80 1 -- 172.21.15.40:0/3520381139 shutdown_connections 2023-07-18T03:33:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.307+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/3520381139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46c196e0 0x564d46c19b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.307+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/3520381139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46a746b0 0x564d46c18300 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.307+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/3520381139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 0x564d46c191a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.307+0000 7f1b4445bb80 1 -- 172.21.15.40:0/3520381139 >> 172.21.15.40:0/3520381139 conn(0x564d46c0dbe0 msgr2=0x564d46c12cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.307+0000 7f1b4445bb80 1 -- 172.21.15.40:0/3520381139 shutdown_connections 2023-07-18T03:33:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.307+0000 7f1b4445bb80 1 -- 172.21.15.40:0/3520381139 wait complete. 2023-07-18T03:33:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.308+0000 7f1b4445bb80 1 -- start start 2023-07-18T03:33:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.309+0000 7f1b4445bb80 1 Processor -- start 2023-07-18T03:33:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.309+0000 7f1b4445bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46a746b0 0x564d46c21980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.309+0000 7f1b4445bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 0x564d46c21ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.309+0000 7f1b4445bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46c196e0 0x564d46c28f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.309+0000 7f1b4445bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d46b09a60 con 0x564d46c196e0 2023-07-18T03:33:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.309+0000 7f1b4445bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d46b09760 con 0x564d46c18840 2023-07-18T03:33:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.309+0000 7f1b4445bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d469ed8c0 con 0x564d46a746b0 2023-07-18T03:33:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.309+0000 7f1b4445bb80 10 client.0 _ll_register_callbacks cb 0x564d46b07700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.309+0000 7f1b36872700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46c196e0 0x564d46c28f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.309+0000 7f1b36872700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46c196e0 0x564d46c28f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59582/0 (socket says 172.21.15.40:59582) 2023-07-18T03:33:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.309+0000 7f1b36872700 1 -- 172.21.15.40:0/1634217375 learned_addr learned my addr 172.21.15.40:0/1634217375 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.310+0000 7f1b36071700 1 --2- 172.21.15.40:0/1634217375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46a746b0 0x564d46c21980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.310+0000 7f1b35870700 1 --2- 172.21.15.40:0/1634217375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 0x564d46c21ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.310+0000 7f1b36071700 1 -- 172.21.15.40:0/1634217375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 msgr2=0x564d46c21ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.310+0000 7f1b36071700 1 --2- 172.21.15.40:0/1634217375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 0x564d46c21ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.310+0000 7f1b36071700 1 -- 172.21.15.40:0/1634217375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46c196e0 msgr2=0x564d46c28f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.310+0000 7f1b36071700 1 --2- 172.21.15.40:0/1634217375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46c196e0 0x564d46c28f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.310+0000 7f1b36071700 1 -- 172.21.15.40:0/1634217375 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3006f100 con 0x564d46a746b0 2023-07-18T03:33:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.313+0000 7f1b36071700 1 --2- 172.21.15.40:0/1634217375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46a746b0 0x564d46c21980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b28004ed0 tx=0x7f1b28005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=263878bcf21c4823 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:31.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.313+0000 7f1b19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:33:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.313+0000 7f1b19ffb700 1 -- 172.21.15.40:0/1634217375 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2800ccd0 con 0x564d46a746b0 2023-07-18T03:33:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.313+0000 7f1b19ffb700 1 -- 172.21.15.40:0/1634217375 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1b2809dc60 con 0x564d46a746b0 2023-07-18T03:33:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.313+0000 7f1b4445bb80 1 -- 172.21.15.40:0/1634217375 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d46ac3490 con 0x564d46a746b0 2023-07-18T03:33:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.313+0000 7f1b19ffb700 1 -- 172.21.15.40:0/1634217375 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b2809a0a0 con 0x564d46a746b0 2023-07-18T03:33:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.313+0000 7f1b4445bb80 10 client.25768 fetch_fsmap learned FSMap version 32 2023-07-18T03:33:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.313+0000 7f1b4445bb80 1 -- 172.21.15.40:0/1634217375 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564d46980c00 con 0x564d46a746b0 2023-07-18T03:33:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.314+0000 7f1b19ffb700 1 -- 172.21.15.40:0/1634217375 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564d46ac3490 con 0x564d46a746b0 2023-07-18T03:33:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.314+0000 7f1b19ffb700 1 -- 172.21.15.40:0/1634217375 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f1b28098b90 con 0x564d46a746b0 2023-07-18T03:33:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.314+0000 7f1b4445bb80 10 client.25768 fetch_fsmap finished waiting for FSMap version 32 2023-07-18T03:33:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.314+0000 7f1b4445bb80 10 client.25768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:33:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.314+0000 7f1b4445bb80 20 client.25768 populate_metadata read hostname 'smithi040' 2023-07-18T03:33:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.314+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/1634217375 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564d46c2d2a0 0x564d46c2f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.314+0000 7f1b4445bb80 4 client.25768 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:33:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.314+0000 7f1b4445bb80 1 -- 172.21.15.40:0/1634217375 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d46b62e50 con 0x564d46c2d2a0 2023-07-18T03:33:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.314+0000 7f1b35870700 1 --2- 172.21.15.40:0/1634217375 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564d46c2d2a0 0x564d46c2f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.315+0000 7f1b19ffb700 10 client.25768 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.315+0000 7f1b35870700 1 --2- 172.21.15.40:0/1634217375 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564d46c2d2a0 0x564d46c2f6b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9cc0a8a7e63ad7d server_cookie=e6171d36d6a51367 in_seq=0 out_seq=0 2023-07-18T03:33:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.315+0000 7f1b19ffb700 1 -- 172.21.15.40:0/1634217375 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564d46b62e50 con 0x564d46c2d2a0 2023-07-18T03:33:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.315+0000 7f1b19ffb700 10 client.25768 handle_command_reply: tid=0 2023-07-18T03:33:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.315+0000 7f1b4445bb80 1 client.25768 shutdown 2023-07-18T03:33:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.316+0000 7f1b4445bb80 1 -- 172.21.15.40:0/1634217375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46a746b0 msgr2=0x564d46c21980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.316+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/1634217375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46a746b0 0x564d46c21980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b28004ed0 tx=0x7f1b28005ce0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.316+0000 7f1b4445bb80 1 -- 172.21.15.40:0/1634217375 shutdown_connections 2023-07-18T03:33:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.316+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/1634217375 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564d46c2d2a0 0x564d46c2f6b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.316+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/1634217375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46c196e0 0x564d46c28f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.316+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/1634217375 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46a746b0 0x564d46c21980 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.316+0000 7f1b19ffb700 0 client.25768 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.316+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/1634217375 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 0x564d46c21ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.316+0000 7f1b4445bb80 1 -- 172.21.15.40:0/1634217375 >> 172.21.15.40:0/1634217375 conn(0x564d46c0dbe0 msgr2=0x564d46c124e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.316+0000 7f1b4445bb80 1 -- 172.21.15.40:0/1634217375 shutdown_connections 2023-07-18T03:33:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.316+0000 7f1b4445bb80 1 -- 172.21.15.40:0/1634217375 wait complete. 2023-07-18T03:33:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.316+0000 7f1b4445bb80 20 client.25768 trim_cache size 0 max 16384 2023-07-18T03:33:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.346+0000 7f1b4445bb80 1 Processor -- start 2023-07-18T03:33:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.346+0000 7f1b4445bb80 1 -- start start 2023-07-18T03:33:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.346+0000 7f1b4445bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46a746b0 0x564d46c34010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.346+0000 7f1b4445bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 0x564d46c28ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.346+0000 7f1b4445bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46c196e0 0x564d46c296d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.346+0000 7f1b4445bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d46c18d40 con 0x564d46a746b0 2023-07-18T03:33:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.346+0000 7f1b4445bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d46b62e50 con 0x564d46c18840 2023-07-18T03:33:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.346+0000 7f1b4445bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d46c22b70 con 0x564d46c196e0 2023-07-18T03:33:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.347+0000 7f1b35870700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 0x564d46c28ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.347+0000 7f1b35870700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 0x564d46c28ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42916/0 (socket says 172.21.15.40:42916) 2023-07-18T03:33:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.347+0000 7f1b35870700 1 -- 172.21.15.40:0/1899146890 learned_addr learned my addr 172.21.15.40:0/1899146890 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.347+0000 7f1b36872700 1 --2- 172.21.15.40:0/1899146890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46c196e0 0x564d46c296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.347+0000 7f1b35870700 1 -- 172.21.15.40:0/1899146890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46c196e0 msgr2=0x564d46c296d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.347+0000 7f1b35870700 1 --2- 172.21.15.40:0/1899146890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46c196e0 0x564d46c296d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.347+0000 7f1b35870700 1 -- 172.21.15.40:0/1899146890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46a746b0 msgr2=0x564d46c34010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.347+0000 7f1b35870700 1 --2- 172.21.15.40:0/1899146890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46a746b0 0x564d46c34010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.347+0000 7f1b35870700 1 -- 172.21.15.40:0/1899146890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d469be1f0 con 0x564d46c18840 2023-07-18T03:33:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.347+0000 7f1b35870700 1 --2- 172.21.15.40:0/1899146890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 0x564d46c28ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00b0a0 tx=0x7f1b2c01a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5af807fa37c8e1d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.349+0000 7f1b255ad700 1 -- 172.21.15.40:0/1899146890 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c00e040 con 0x564d46c18840 2023-07-18T03:33:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.349+0000 7f1b255ad700 1 -- 172.21.15.40:0/1899146890 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b2c003a00 con 0x564d46c18840 2023-07-18T03:33:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.349+0000 7f1b255ad700 1 -- 172.21.15.40:0/1899146890 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c005690 con 0x564d46c18840 2023-07-18T03:33:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.350+0000 7f1b4445bb80 1 -- 172.21.15.40:0/1899146890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 msgr2=0x564d46c28ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.350+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/1899146890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 0x564d46c28ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00b0a0 tx=0x7f1b2c01a000 comp rx=0 tx=0).stop 2023-07-18T03:33:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.350+0000 7f1b4445bb80 1 -- 172.21.15.40:0/1899146890 shutdown_connections 2023-07-18T03:33:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.350+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/1899146890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46c196e0 0x564d46c296d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.350+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/1899146890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46a746b0 0x564d46c34010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.350+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/1899146890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c18840 0x564d46c28ec0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.350+0000 7f1b4445bb80 1 -- 172.21.15.40:0/1899146890 >> 172.21.15.40:0/1899146890 conn(0x564d46c0dbe0 msgr2=0x564d46c12c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.350+0000 7f1b4445bb80 1 -- 172.21.15.40:0/1899146890 shutdown_connections 2023-07-18T03:33:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.350+0000 7f1b4445bb80 1 -- 172.21.15.40:0/1899146890 wait complete. 2023-07-18T03:33:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.351+0000 7f1b4445bb80 1 -- start start 2023-07-18T03:33:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.351+0000 7f1b4445bb80 1 Processor -- start 2023-07-18T03:33:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.351+0000 7f1b4445bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46a746b0 0x564d46c26e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.351+0000 7f1b4445bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46c18840 0x564d46c27370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.351+0000 7f1b4445bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c196e0 0x564d46c28380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.351+0000 7f1b4445bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d46c288c0 con 0x564d46c18840 2023-07-18T03:33:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.351+0000 7f1b4445bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d46c28a30 con 0x564d46c196e0 2023-07-18T03:33:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.351+0000 7f1b4445bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d46c28bf0 con 0x564d46a746b0 2023-07-18T03:33:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.351+0000 7f1b4445bb80 10 client.0 _ll_register_callbacks cb 0x7f1b2c0092e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.352+0000 7f1b35870700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46c18840 0x564d46c27370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.353+0000 7f1b35870700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46c18840 0x564d46c27370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59596/0 (socket says 172.21.15.40:59596) 2023-07-18T03:33:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.353+0000 7f1b35870700 1 -- 172.21.15.40:0/2535133223 learned_addr learned my addr 172.21.15.40:0/2535133223 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.353+0000 7f1b36071700 1 --2- 172.21.15.40:0/2535133223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46a746b0 0x564d46c26e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.353+0000 7f1b35870700 1 -- 172.21.15.40:0/2535133223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46a746b0 msgr2=0x564d46c26e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.353+0000 7f1b35870700 1 --2- 172.21.15.40:0/2535133223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46a746b0 0x564d46c26e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.353+0000 7f1b35870700 1 -- 172.21.15.40:0/2535133223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c196e0 msgr2=0x564d46c28380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:33:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.353+0000 7f1b35870700 1 --2- 172.21.15.40:0/2535133223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c196e0 0x564d46c28380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.353+0000 7f1b35870700 1 -- 172.21.15.40:0/2535133223 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2c022040 con 0x564d46c18840 2023-07-18T03:33:31.359 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.354+0000 7f1b35870700 1 --2- 172.21.15.40:0/2535133223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46c18840 0x564d46c27370 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c007f00 tx=0x7f1b2c003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a69631f303032c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.354+0000 7f1b19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:33:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.354+0000 7f1b19ffb700 1 -- 172.21.15.40:0/2535133223 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c00e050 con 0x564d46c18840 2023-07-18T03:33:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.354+0000 7f1b19ffb700 1 -- 172.21.15.40:0/2535133223 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1b2c0074a0 con 0x564d46c18840 2023-07-18T03:33:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.354+0000 7f1b4445bb80 1 -- 172.21.15.40:0/2535133223 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b280a2080 con 0x564d46c18840 2023-07-18T03:33:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.354+0000 7f1b19ffb700 1 -- 172.21.15.40:0/2535133223 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b2c007680 con 0x564d46c18840 2023-07-18T03:33:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.354+0000 7f1b4445bb80 10 client.16131 fetch_fsmap learned FSMap version 32 2023-07-18T03:33:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.354+0000 7f1b4445bb80 1 -- 172.21.15.40:0/2535133223 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564d46996250 con 0x564d46c18840 2023-07-18T03:33:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.355+0000 7f1b19ffb700 1 -- 172.21.15.40:0/2535133223 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b280a2080 con 0x564d46c18840 2023-07-18T03:33:31.362 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.356+0000 7f1b19ffb700 1 -- 172.21.15.40:0/2535133223 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f1b2c01d6d0 con 0x564d46c18840 2023-07-18T03:33:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.362+0000 7f1b4445bb80 10 client.16131 fetch_fsmap finished waiting for FSMap version 32 2023-07-18T03:33:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.362+0000 7f1b4445bb80 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:33:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.362+0000 7f1b4445bb80 20 client.16131 populate_metadata read hostname 'smithi040' 2023-07-18T03:33:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.362+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/2535133223 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564d46c81f60 0x564d46c84370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.362+0000 7f1b4445bb80 4 client.16131 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:33:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.362+0000 7f1b4445bb80 1 -- 172.21.15.40:0/2535133223 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564d46b24ff0 con 0x564d46c81f60 2023-07-18T03:33:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.362+0000 7f1b36071700 1 --2- 172.21.15.40:0/2535133223 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564d46c81f60 0x564d46c84370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.362+0000 7f1b19ffb700 10 client.16131 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:31.362+0000 7f1b36071700 1 --2- 172.21.15.40:0/2535133223 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564d46c81f60 0x564d46c84370 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7c286cc69390a6b server_cookie=e4fb9983f04ca92c in_seq=0 out_seq=0 2023-07-18T03:33:32.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:31 smithi134 ceph-mon[98216]: pgmap v372: 97 pgs: 97 active+clean; 398 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 131 op/s 2023-07-18T03:33:32.084 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:31 smithi040 ceph-mon[96439]: pgmap v372: 97 pgs: 97 active+clean; 398 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 131 op/s 2023-07-18T03:33:32.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:31 smithi064 ceph-mon[98059]: pgmap v372: 97 pgs: 97 active+clean; 398 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 131 op/s 2023-07-18T03:33:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.427+0000 7f1b19ffb700 1 -- 172.21.15.40:0/2535133223 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564d46b24ff0 con 0x564d46c81f60 2023-07-18T03:33:32.424 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.427+0000 7f1b19ffb700 10 client.16131 handle_command_reply: tid=0 2023-07-18T03:33:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.427+0000 7f1b4445bb80 1 client.16131 shutdown 2023-07-18T03:33:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.427+0000 7f1b4445bb80 1 -- 172.21.15.40:0/2535133223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46c18840 msgr2=0x564d46c27370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.427+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/2535133223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46c18840 0x564d46c27370 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c007f00 tx=0x7f1b2c003be0 comp rx=0 tx=0).stop 2023-07-18T03:33:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.427+0000 7f1b4445bb80 1 -- 172.21.15.40:0/2535133223 shutdown_connections 2023-07-18T03:33:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.427+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/2535133223 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564d46c81f60 0x564d46c84370 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.427+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/2535133223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d46c196e0 0x564d46c28380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.427+0000 7f1b19ffb700 0 client.16131 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:32.426 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.427+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/2535133223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d46a746b0 0x564d46c26e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:32.426 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.427+0000 7f1b4445bb80 1 --2- 172.21.15.40:0/2535133223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564d46c18840 0x564d46c27370 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:32.426 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.427+0000 7f1b4445bb80 1 -- 172.21.15.40:0/2535133223 >> 172.21.15.40:0/2535133223 conn(0x564d46c0dbe0 msgr2=0x564d46c296d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:32.426 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.427+0000 7f1b4445bb80 1 -- 172.21.15.40:0/2535133223 shutdown_connections 2023-07-18T03:33:32.426 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.427+0000 7f1b4445bb80 1 -- 172.21.15.40:0/2535133223 wait complete. 2023-07-18T03:33:32.426 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.427+0000 7f1b4445bb80 20 client.16131 trim_cache size 0 max 16384 2023-07-18T03:33:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:33:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:33:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:33:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:33:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.428+0000 7f1b382d5700 1 -- 172.21.15.40:0/1138726109 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1b1c0a11d0 msgr2=0x7f1b1c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.428+0000 7f1b382d5700 1 --2- 172.21.15.40:0/1138726109 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1b1c0a11d0 0x7f1b1c0a3680 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00b070 tx=0x7f1b2c006000 comp rx=0 tx=0).stop 2023-07-18T03:33:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.428+0000 7f1b382d5700 1 -- 172.21.15.40:0/1138726109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b30068600 msgr2=0x7f1b30071a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.428+0000 7f1b382d5700 1 --2- 172.21.15.40:0/1138726109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b30068600 0x7f1b30071a70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800e970 tx=0x7f1b2800ed30 comp rx=0 tx=0).stop 2023-07-18T03:33:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.428+0000 7f1b382d5700 1 -- 172.21.15.40:0/1138726109 shutdown_connections 2023-07-18T03:33:32.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.428+0000 7f1b382d5700 1 --2- 172.21.15.40:0/1138726109 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1b1c0a11d0 0x7f1b1c0a3680 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.428+0000 7f1b382d5700 1 --2- 172.21.15.40:0/1138726109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1b3010c6b0 0x7f1b30070580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.428+0000 7f1b382d5700 1 --2- 172.21.15.40:0/1138726109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b30068600 0x7f1b30071a70 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.428+0000 7f1b382d5700 1 --2- 172.21.15.40:0/1138726109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b301056e0 0x7f1b30071fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.428+0000 7f1b382d5700 1 -- 172.21.15.40:0/1138726109 >> 172.21.15.40:0/1138726109 conn(0x7f1b300fa000 msgr2=0x7f1b301063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.428+0000 7f1b382d5700 1 -- 172.21.15.40:0/1138726109 shutdown_connections 2023-07-18T03:33:32.429 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:32.428+0000 7f1b382d5700 1 -- 172.21.15.40:0/1138726109 wait complete. 2023-07-18T03:33:33.150 INFO:tasks.workunit.client.0.smithi040.stdout: 136 1663926 871 1225325 950 291941 442 2023-07-18T03:33:34.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:33 smithi134 ceph-mon[98216]: pgmap v373: 97 pgs: 97 active+clean; 401 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2023-07-18T03:33:34.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:33 smithi040 ceph-mon[96439]: pgmap v373: 97 pgs: 97 active+clean; 401 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2023-07-18T03:33:34.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:33 smithi064 ceph-mon[98059]: pgmap v373: 97 pgs: 97 active+clean; 401 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2023-07-18T03:33:36.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:35 smithi134 ceph-mon[98216]: pgmap v374: 97 pgs: 97 active+clean; 402 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2023-07-18T03:33:36.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:35 smithi040 ceph-mon[96439]: pgmap v374: 97 pgs: 97 active+clean; 402 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2023-07-18T03:33:36.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:35 smithi064 ceph-mon[98059]: pgmap v374: 97 pgs: 97 active+clean; 402 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2023-07-18T03:33:38.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: pgmap v375: 97 pgs: 97 active+clean; 420 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 274 op/s 2023-07-18T03:33:38.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.072 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.072 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.072 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:38.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:33:38.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:33:38.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:37 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:33:38.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: pgmap v375: 97 pgs: 97 active+clean; 420 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 274 op/s 2023-07-18T03:33:38.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.083 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.083 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.083 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.083 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.083 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.084 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.084 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.084 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.084 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.084 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.084 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.084 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.085 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.085 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.085 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.085 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.085 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.085 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.085 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.085 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.086 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.086 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.086 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.086 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.086 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.086 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.086 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.086 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.087 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.087 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.087 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.087 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.087 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.087 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.087 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.087 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.088 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.088 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.088 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.088 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.088 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.088 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.088 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.089 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.089 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.089 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.089 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.089 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.089 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.089 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.091 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.091 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:38.091 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:33:38.091 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:33:38.091 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:37 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:33:38.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: pgmap v375: 97 pgs: 97 active+clean; 420 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 274 op/s 2023-07-18T03:33:38.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.102 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.103 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.104 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.105 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.106 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.107 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.108 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.108 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.108 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.108 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.108 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.108 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.108 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.110 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.110 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.110 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.110 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.110 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.110 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.110 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.111 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.111 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.111 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.111 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.111 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.111 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.111 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.111 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.112 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.112 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.112 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.112 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.112 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.112 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.112 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.112 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.113 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:38.113 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:38.113 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:33:38.113 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:33:38.113 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:37 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:33:39.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.072 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.072 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.072 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.072 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.072 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.072 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.080 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.080 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.080 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.080 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.080 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.080 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.080 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.082 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.082 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.082 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.082 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.082 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.082 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.082 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.083 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.083 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.083 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.083 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.083 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.083 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.083 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.084 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.084 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.085 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.085 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.087 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.087 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.087 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.087 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.087 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.087 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.088 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.088 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.088 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.088 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.088 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.088 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.088 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.089 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.089 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.089 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.089 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.089 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.089 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.090 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.090 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.090 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.090 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.090 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.090 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.090 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.090 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.091 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.091 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.091 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.091 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.091 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.091 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.091 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.091 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.092 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.092 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.092 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.092 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.092 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.092 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.092 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.093 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.093 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.093 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.093 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.093 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.093 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.093 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.093 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.094 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.094 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.094 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.094 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.094 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.094 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.094 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.094 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.096 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.096 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.096 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.096 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.096 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.096 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.096 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.097 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.097 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.097 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.097 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.097 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.097 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.098 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.098 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.098 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.098 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.098 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.098 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.098 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.098 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.099 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.099 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.099 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.099 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.099 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.099 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.100 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.100 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.100 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.100 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.100 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.100 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.100 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.101 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.101 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.101 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.101 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.101 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.101 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.101 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.101 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.102 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.103 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.104 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.105 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.105 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.105 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.105 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.105 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.105 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.105 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.105 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.106 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.106 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.106 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.106 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.106 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.106 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.106 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.107 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.107 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.107 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.107 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.107 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.107 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.107 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.108 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.108 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.108 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.108 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.108 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.109 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.109 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.109 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.109 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.109 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.109 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.110 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.110 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.110 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.110 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.110 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.110 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.111 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.111 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.111 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.111 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.111 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.111 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.111 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.112 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.112 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.112 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.112 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.112 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.112 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.112 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.112 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.113 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.113 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.113 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.113 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.113 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.113 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.113 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.114 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.114 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.114 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.114 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.114 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.114 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.114 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.114 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.115 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.115 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.115 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.115 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.115 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.115 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.115 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.115 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.116 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.116 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.116 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.116 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.116 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.116 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.116 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.117 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.117 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.117 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.117 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.117 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.117 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.117 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.117 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.118 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.118 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.118 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.118 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.119 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.119 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.119 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.119 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.119 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.119 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.120 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.120 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.120 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.120 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.120 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.120 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.120 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.120 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.121 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.121 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.121 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.121 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.121 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.121 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.121 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.121 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.122 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.122 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.122 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.122 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.122 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.122 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.122 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.122 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.123 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.124 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.124 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.124 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.124 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.124 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.124 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.124 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.125 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.125 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.125 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.125 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.125 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.125 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.125 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.125 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.126 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.126 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.126 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.126 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.126 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.126 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.126 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.126 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.127 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.127 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.127 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.127 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.127 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.127 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.127 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.128 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.128 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.128 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.128 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.128 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.128 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.129 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.129 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.129 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.129 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.129 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.129 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.129 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.131 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.132 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.134 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.135 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.136 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.137 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.137 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.137 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.137 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.137 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.137 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.138 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.138 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.138 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.138 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.138 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.138 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.138 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.138 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.139 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.139 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.139 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.139 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.139 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.139 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.140 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.140 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.140 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.140 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.140 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.140 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.140 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.141 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.141 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.141 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.141 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.141 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.142 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.142 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.142 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.142 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.142 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.142 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.143 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.143 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.143 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:39.143 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:33:39.144 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:33:39.144 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:38 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:33:39.145 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.145 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.145 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.145 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.145 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.145 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.145 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.146 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.146 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.146 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.146 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.146 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.146 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.147 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.147 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.147 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.147 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.147 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.147 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.147 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.148 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.148 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.148 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.148 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.149 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.149 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.149 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.149 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.150 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.150 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.150 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.150 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.150 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.151 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.151 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.151 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.151 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.151 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.151 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.152 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.152 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.152 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.152 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.153 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.153 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.153 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.153 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.154 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.154 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.154 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.154 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.154 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.154 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.155 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.155 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.155 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.155 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.155 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.155 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.155 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.156 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.156 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.156 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.156 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.156 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.156 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.156 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.157 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.157 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.157 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.157 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.157 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.157 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.158 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.158 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.158 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.158 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.158 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.158 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.158 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.159 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.159 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.159 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.159 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.159 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.159 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.160 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.160 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.160 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.160 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.160 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.160 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.161 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.161 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.161 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.161 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.162 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.162 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.162 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.162 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.162 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.162 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.162 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.162 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.163 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.163 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.163 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.163 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.163 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.163 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.163 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.164 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.164 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.164 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.164 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.164 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.164 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.164 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.164 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.165 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.165 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.165 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.165 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.165 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.165 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.165 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.165 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.166 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.166 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.166 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.166 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.166 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.166 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.166 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.166 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.167 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.167 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.167 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.167 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.168 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.168 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.168 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.168 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.168 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.169 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.169 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.169 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.169 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.169 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.169 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.170 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.170 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.170 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.170 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.170 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.171 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.171 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.171 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.171 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.171 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.172 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.172 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.172 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.172 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.172 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.172 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.172 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.173 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.173 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.173 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.173 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.173 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.173 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.173 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.173 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.174 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.174 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.174 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.174 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.174 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.174 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.174 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.174 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.175 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.175 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.175 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.175 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.175 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.175 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.175 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.175 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.176 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.176 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.176 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.176 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.176 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.176 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.176 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.177 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.177 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.177 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.177 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.177 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.177 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.177 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.177 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.178 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.178 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.178 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.178 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.179 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.179 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.179 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.180 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.180 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.180 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.180 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.181 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.181 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.181 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.182 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.182 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.182 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.182 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.183 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.183 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.183 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.183 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.184 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.184 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.184 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.185 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.185 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.185 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.185 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.185 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.185 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.186 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.186 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.186 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.186 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.186 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.186 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.186 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.186 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.187 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.187 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.187 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.187 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.187 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.187 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.188 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.188 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.188 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.188 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.189 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.189 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.189 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.189 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.189 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.190 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.190 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.190 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.190 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.190 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.190 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.191 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.191 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.191 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.191 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.191 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.191 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.191 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.191 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.192 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.192 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.192 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.192 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.192 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.192 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.192 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.193 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.193 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.193 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.193 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.193 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.193 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.193 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.194 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.194 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.194 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.194 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.194 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.195 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.195 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.195 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.195 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.195 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.195 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.195 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.196 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.196 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.196 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.196 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.197 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.197 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.197 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.197 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.197 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.197 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.197 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.198 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.198 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.198 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.198 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.198 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.198 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.198 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.198 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.199 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.199 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.199 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.199 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.199 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.199 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.199 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.200 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.200 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.200 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.200 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.200 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.201 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.201 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.201 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.201 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.201 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.202 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.202 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.202 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.202 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.202 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.202 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.202 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.202 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.203 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.203 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.203 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.203 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.203 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.203 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.203 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.203 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.204 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.204 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.204 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.204 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.204 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.204 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.205 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.205 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.205 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.205 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.205 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.205 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.205 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.205 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.206 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.206 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.206 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.206 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.206 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.206 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.206 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.207 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.207 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.207 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.207 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.207 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.207 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.208 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.208 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.208 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.208 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.208 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.208 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.208 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.209 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.209 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.209 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.209 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.209 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.209 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.209 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.210 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.210 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.210 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.210 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.210 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.210 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.211 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.211 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.211 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.211 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.211 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.211 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.211 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.212 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.212 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.212 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.212 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.212 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.212 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.212 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.213 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.213 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.213 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.213 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.213 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.213 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.213 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.214 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.214 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.214 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.214 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.215 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.215 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.215 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.215 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.216 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.216 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.216 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.217 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.217 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.217 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.217 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.218 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.218 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.218 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.218 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.219 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.219 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.219 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.220 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.220 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.220 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.221 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.221 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.221 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.221 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.222 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.222 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.222 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.222 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.223 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.223 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.223 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.223 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.223 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.224 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.224 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.224 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.225 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.225 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.225 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.225 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.225 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.225 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.226 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.226 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.226 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.226 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.226 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.226 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.226 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.226 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.227 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.227 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.227 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.227 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.227 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.227 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.227 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.228 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.228 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.228 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.228 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.228 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.228 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.228 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.228 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.229 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.229 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.229 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.229 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.229 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.229 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.229 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.230 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.230 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.230 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.230 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.230 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.230 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.230 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.231 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.231 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.231 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.231 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.231 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:33:39.231 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:33:39.231 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:38 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:33:39.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.233 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.234 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.235 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.235 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.235 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.235 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.235 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.235 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.235 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.236 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.236 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.236 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.237 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.237 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.237 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.237 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.238 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.238 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.238 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.238 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.239 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.239 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.239 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.240 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.240 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.240 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.240 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.241 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.242 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.242 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.242 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.242 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.243 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.243 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.243 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.243 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.244 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.244 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.244 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.244 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.244 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.244 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.244 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.245 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.245 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.245 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.245 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.245 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.245 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.245 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.246 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.246 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.246 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.246 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.246 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.246 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.246 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.246 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.247 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.247 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.247 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.247 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.247 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.247 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.247 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.247 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.248 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.248 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.248 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.248 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.248 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.248 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.248 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.248 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.250 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.250 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.250 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.250 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.250 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.250 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.250 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.250 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.251 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.251 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.251 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.251 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.251 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.251 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.251 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.252 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.252 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.252 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.252 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.252 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.252 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.252 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.252 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.253 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.253 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.253 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.253 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.253 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.253 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.253 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.253 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.254 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.254 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.254 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.254 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.254 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.254 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.254 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.254 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.255 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.255 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.255 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.255 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.255 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.255 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.255 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.255 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.260 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.260 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.260 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.260 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.260 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.260 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.260 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.260 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.261 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.261 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.261 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.261 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.261 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.261 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.261 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.261 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.262 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.262 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.262 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.262 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.262 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.262 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.262 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.262 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.263 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.263 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.263 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.263 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.263 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.263 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.263 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.263 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.264 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.264 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.264 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.264 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.264 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.264 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.264 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.264 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.266 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.266 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.266 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.266 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.266 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.266 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.266 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.266 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.267 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.267 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.267 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.267 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.267 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.267 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.267 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.268 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.268 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.268 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.268 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.268 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.268 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.268 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.268 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.269 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.269 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.269 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.269 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.269 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.269 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.269 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.269 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.271 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.271 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.271 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.271 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.271 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.271 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.271 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.271 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.272 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.272 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.272 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.272 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.272 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.272 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.272 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.272 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.273 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.273 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.273 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.273 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.273 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.273 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.273 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.273 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.276 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.276 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.276 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.276 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.276 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.276 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.276 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.277 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.277 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.277 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.277 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.277 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.277 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.277 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.277 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.278 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.278 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.278 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.278 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.278 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.278 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.278 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.278 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.279 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.279 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.279 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.279 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.279 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.279 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.279 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.279 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.280 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.280 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.280 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.280 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.280 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.280 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.280 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.282 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.282 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.282 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.282 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.282 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.282 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.282 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.282 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.283 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.283 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.283 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.283 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.283 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.283 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.283 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.283 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.287 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.287 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.287 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.287 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.287 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.287 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.287 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.287 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.288 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.288 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.288 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.288 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.288 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.288 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.288 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.289 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.289 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.289 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.289 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.289 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.289 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.289 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.289 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.291 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.291 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.291 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.291 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.291 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.291 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.291 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.291 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.292 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.292 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.292 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.292 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.292 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.292 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.292 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.292 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.293 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.293 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.293 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.293 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.293 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.293 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.293 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.293 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.294 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.294 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.294 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.294 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.294 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.294 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.294 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.294 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.295 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.295 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.295 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.295 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.295 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.295 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.295 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.295 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.296 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.296 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.296 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.296 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.296 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.296 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.296 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.297 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.297 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.297 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.297 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.297 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.297 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.297 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.297 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.299 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.299 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.299 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.299 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.299 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.299 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.299 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.299 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.300 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.300 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.300 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.300 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.300 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.300 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.300 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.300 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.301 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.301 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.301 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.301 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.301 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.301 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.301 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.301 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.302 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.302 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.302 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:39.302 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:33:39.302 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:33:39.302 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:38 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:33:39.749 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.749 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.749 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.749 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.750 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.750 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.750 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.750 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.750 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.750 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.750 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.750 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.751 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.751 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.751 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.751 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.751 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.751 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.751 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.752 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.752 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.752 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.752 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.752 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.752 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.752 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.753 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.753 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.753 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.753 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.753 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.753 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.754 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.754 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.754 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.754 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.754 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.754 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.754 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.754 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.755 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.755 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.755 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.755 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.755 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.755 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.756 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.756 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.756 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.756 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: pgmap v376: 97 pgs: 97 active+clean; 496 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.6 MiB/s wr, 722 op/s 2023-07-18T03:33:39.756 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.756 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.756 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.757 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.757 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.757 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.757 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.757 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.757 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.758 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.758 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.758 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.758 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.758 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.758 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.759 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.759 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.759 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.759 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.759 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.759 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.759 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.760 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.760 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.760 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.760 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.760 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.761 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.761 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.761 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.761 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.761 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.762 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.762 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.762 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.762 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.762 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.762 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.762 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.763 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.763 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.763 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.763 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.763 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.763 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.764 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.764 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.764 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.764 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.764 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.764 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.764 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.765 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.765 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.765 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.765 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.765 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.765 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.765 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.765 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.766 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.766 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.766 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.766 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.766 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.766 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.766 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.767 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.767 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.767 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.767 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.767 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.767 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.767 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.767 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.768 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.768 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.768 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.768 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.768 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.768 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.768 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.768 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.769 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.769 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.769 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.769 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.769 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.769 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.769 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.769 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.770 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.770 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.770 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.770 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.770 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.770 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.770 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.771 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.771 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.771 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.771 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.771 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.771 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.771 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.771 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.772 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.772 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.772 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.772 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.772 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.772 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.772 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.772 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.773 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.773 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.773 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.773 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.773 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.773 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.773 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.773 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.774 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.774 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.774 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.774 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.774 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.774 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.774 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.775 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.775 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.775 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.775 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.775 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.775 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.775 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.775 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.776 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.776 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.776 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.776 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.776 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.776 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.776 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.777 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.777 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.777 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.777 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.777 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.777 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.777 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.777 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.778 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.778 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.778 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.778 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.778 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.778 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.778 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.778 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.779 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.779 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.779 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.779 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.779 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.779 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.779 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.780 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.780 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.780 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.780 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.780 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.780 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.781 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.781 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.781 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.781 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.781 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.781 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.781 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.782 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.782 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.782 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:39.782 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:39.782 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.070 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.071 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.072 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.072 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.072 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.072 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.072 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.072 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.073 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.074 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.075 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: pgmap v376: 97 pgs: 97 active+clean; 496 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.6 MiB/s wr, 722 op/s 2023-07-18T03:33:40.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.076 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.080 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.080 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.080 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.080 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.080 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.080 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.080 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.082 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.082 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.082 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.082 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.082 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.082 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.082 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.083 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.083 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.083 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.083 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.083 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.086 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.086 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.086 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.086 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.086 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.087 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.087 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.087 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.087 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.087 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.087 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.087 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.088 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.088 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.088 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.088 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.088 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.088 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.088 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.088 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.089 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.089 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.089 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.089 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.089 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.089 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.089 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.089 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.091 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.091 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.091 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.091 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.091 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.091 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.092 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.092 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.092 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.092 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.092 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.092 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.092 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.093 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.093 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.093 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.093 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.093 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.093 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.093 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.094 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.094 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.094 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.094 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.094 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.094 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.094 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.095 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.095 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.095 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.095 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.095 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.095 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.096 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.096 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.096 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.096 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.096 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.096 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.097 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.097 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.097 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.097 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.097 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.097 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.098 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.098 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.098 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.098 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.098 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.098 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.099 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.099 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.099 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.100 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.100 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.100 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.100 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.100 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.100 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.101 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.101 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.101 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.101 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.101 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.101 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.101 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.101 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.102 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.102 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.102 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.102 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.102 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.102 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.102 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.103 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.103 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.103 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.103 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.103 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.103 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.103 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.103 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.104 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.104 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.104 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.104 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.104 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.104 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.105 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.105 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.105 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.105 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.105 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.105 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.105 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.105 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.106 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.106 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.106 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.106 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.106 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.106 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.107 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.107 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.107 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.107 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.107 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.107 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.107 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.108 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.108 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.108 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.108 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.108 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.108 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.108 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.108 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.109 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.109 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.109 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.109 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.109 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.109 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.109 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.109 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.110 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.110 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.110 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.110 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.110 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.111 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.111 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.111 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.111 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.111 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.111 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.111 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.111 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.112 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.112 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.112 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.112 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.112 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.112 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.112 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.113 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.113 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.113 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.113 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.113 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.113 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.113 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.113 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.114 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.114 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.114 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.114 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.114 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.114 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.114 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.114 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.115 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.115 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.115 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.115 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.115 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.115 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.115 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.115 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.116 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.116 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.116 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.116 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.116 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.116 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.116 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.116 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.117 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.117 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.117 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.117 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.117 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.117 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.117 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.118 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.118 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.118 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.118 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.118 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.118 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.118 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.118 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.119 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.119 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.119 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.119 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.119 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.119 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.119 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.119 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.120 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.120 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.120 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.120 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.120 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.120 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.121 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.121 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.121 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.121 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.121 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.121 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.121 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.122 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.122 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.122 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.122 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.122 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.122 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.122 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.123 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.123 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.123 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.123 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.123 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.123 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.124 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.124 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.124 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.124 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.124 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.124 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.125 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.125 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.125 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.125 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.125 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.125 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.125 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.126 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.126 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.126 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.126 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.126 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.126 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.127 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.127 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.127 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.127 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.127 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.127 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.127 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.128 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.128 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.128 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.128 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.128 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.128 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.129 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.129 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.129 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.129 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.129 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.130 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.130 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.130 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.130 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.130 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.130 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.131 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.131 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.131 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.131 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.131 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.131 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.132 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.132 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.132 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.132 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.132 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.132 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.133 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.133 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.133 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.133 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.133 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.133 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.133 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.134 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.134 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.134 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.134 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.134 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.134 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.134 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.134 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.135 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.135 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.135 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.135 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.135 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.135 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.135 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.136 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.136 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.136 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.136 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.136 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.136 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.136 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.136 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.137 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.137 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.137 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.137 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.137 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.137 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.137 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.138 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.138 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.138 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.138 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.138 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.138 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.138 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.139 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.139 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.139 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.139 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.139 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.139 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.139 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.139 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.140 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.140 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.140 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.140 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.140 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.140 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.140 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.140 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.141 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.141 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.141 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.141 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.141 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.141 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.141 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.141 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.142 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.142 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.142 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.142 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.142 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.142 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.142 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.143 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.143 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.143 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.143 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.143 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.143 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.143 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.143 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.144 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.144 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.144 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.144 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.144 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.144 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.144 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.144 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.145 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.145 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.145 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.145 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.145 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.145 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.145 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.145 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.146 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.146 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.146 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.146 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.146 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.146 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.146 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.147 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.147 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.147 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.147 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.147 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.147 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.147 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.147 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.148 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.148 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.148 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.148 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.148 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.148 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.148 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.149 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.149 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.149 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.149 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.149 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.149 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.149 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.150 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.150 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.150 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.150 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.150 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.150 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.150 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.151 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.151 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.151 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.151 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.151 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.151 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.151 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.152 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.152 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.152 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.152 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.152 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.152 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.153 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.153 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.153 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.153 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.153 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.153 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.153 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.154 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.154 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.154 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.154 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.154 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.154 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.154 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.155 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.155 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.155 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.155 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.155 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.156 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.156 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.156 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.156 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.156 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.156 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.157 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.157 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.157 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.157 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.157 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.157 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.158 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.158 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.158 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.158 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.158 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.158 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.158 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.159 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.159 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.159 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.159 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.159 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.159 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.159 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.159 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.160 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.160 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.160 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.160 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.160 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.160 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.160 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.161 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.161 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.161 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.161 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.161 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.161 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.161 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.162 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.162 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.162 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.162 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.162 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.162 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.162 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.162 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.163 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.163 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.163 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.163 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.163 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.163 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.163 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.164 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.164 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.164 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.164 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.164 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.164 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.164 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.165 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.165 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.165 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.165 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.165 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.165 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.165 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.166 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.166 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.166 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.166 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.166 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.166 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.166 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.166 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.167 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.167 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.167 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.167 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.167 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.167 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.167 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.167 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.168 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.168 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.168 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.168 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.168 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:33:40.168 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.169 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.169 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.170 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.170 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.170 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.170 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.170 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.170 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.170 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.171 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.171 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.171 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.171 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.171 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.171 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.171 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.172 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.172 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.172 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.172 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.172 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.172 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.172 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.172 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.173 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.174 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.175 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.184 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.185 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.186 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.186 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.186 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.186 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.186 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.186 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.187 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.187 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.187 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.187 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.188 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.189 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.192 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.193 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.194 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.195 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.196 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.196 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.196 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.196 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.196 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.196 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.196 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.197 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.197 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.197 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.197 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.197 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.197 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.197 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.197 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.198 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.198 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.198 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.198 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.198 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.198 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.198 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.198 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.199 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.199 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.199 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.199 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.199 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.199 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.199 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.199 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.200 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.201 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.202 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.205 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.205 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.205 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.205 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.205 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.205 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.205 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.205 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.206 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.206 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.206 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.206 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.206 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.206 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.207 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.207 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.207 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.207 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.207 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.207 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.207 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.207 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.208 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.208 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.208 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.208 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.208 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.208 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.209 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.209 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.209 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.209 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.209 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.209 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.209 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.209 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.210 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.210 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.210 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.210 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.210 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.210 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.211 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.211 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.211 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.211 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.211 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.211 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.212 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.212 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.212 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.212 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.212 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.212 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.213 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.213 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.213 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.213 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.213 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.213 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.214 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.214 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.214 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.214 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.214 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.215 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.215 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.215 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.215 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.215 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.215 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.215 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.216 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.216 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.216 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.216 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.216 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.216 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.216 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.217 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.217 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.217 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.217 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.217 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.217 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.217 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.218 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.218 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.218 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.218 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.218 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.218 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.218 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.219 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.219 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.219 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.219 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.219 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.219 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.219 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.219 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.220 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.220 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.220 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.220 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.220 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.220 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.220 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.220 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.221 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.221 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.221 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.221 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.221 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.221 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.221 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.221 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.222 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.222 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.222 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.222 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.222 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.222 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.222 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.222 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.223 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.223 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.223 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.223 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.223 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.223 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.223 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.223 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.224 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.224 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.224 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.224 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.224 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.224 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.224 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.224 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.225 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.227 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.228 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.229 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.229 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.229 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.229 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.229 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.229 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.229 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.229 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.230 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.230 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.230 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.230 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.230 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.230 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.230 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.230 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.231 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.231 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.231 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.231 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.231 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.231 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.231 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.231 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.232 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.232 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.232 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.232 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.232 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.232 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.232 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.233 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.233 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.233 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.233 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.233 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.233 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.234 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.234 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.234 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.234 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.234 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.234 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.234 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.234 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.235 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.235 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.235 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.235 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.235 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.235 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.235 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.235 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.236 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.236 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.236 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.236 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.236 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.236 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.236 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.236 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.237 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.237 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.237 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.237 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.237 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.237 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.237 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.238 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.238 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.238 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.238 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.238 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.238 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.238 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.238 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.239 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.239 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.239 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.239 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.239 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.239 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.239 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.240 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.240 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.240 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.240 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.240 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.240 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.240 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.240 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.241 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.241 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.241 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.241 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.241 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.241 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.242 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.242 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.242 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.242 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.242 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.242 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.243 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.243 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.243 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.243 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.243 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.243 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.244 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.244 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.244 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.244 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.244 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.245 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.245 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.245 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.245 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.245 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.245 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.246 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.246 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.246 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.246 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.246 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.246 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.247 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.247 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.247 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.247 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.247 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.247 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.247 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.248 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.248 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.248 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.248 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.248 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.248 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.248 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.249 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.249 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.249 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.249 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.249 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.249 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.249 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.249 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.250 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.250 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.250 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.250 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.250 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.250 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.250 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.250 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.251 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.251 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.251 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.251 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.251 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.251 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.251 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.251 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.252 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.252 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.252 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.252 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.252 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.252 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.252 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.253 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.253 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.253 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.253 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.253 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.253 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.253 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.253 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.254 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.254 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.254 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.254 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.254 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.254 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.254 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.254 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.255 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.255 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.255 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.255 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.255 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.255 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.255 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.255 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.256 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.256 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.256 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.256 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.256 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.256 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.256 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.256 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.257 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.257 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.257 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.257 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.257 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.257 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.257 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.258 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.258 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.258 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.258 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.258 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.258 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.258 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.258 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.259 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.259 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.259 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.259 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.259 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.259 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.259 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.259 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.260 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.260 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.260 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.260 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.260 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.260 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.260 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.261 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.261 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.261 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.261 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.261 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.261 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.261 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.261 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.262 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.262 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.262 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.262 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.262 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.262 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.262 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.262 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.263 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.263 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.263 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.263 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.263 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.263 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.263 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.264 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.264 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.264 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.264 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.264 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.264 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.265 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.265 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.265 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.265 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.265 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.265 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.265 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.266 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.266 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.266 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.266 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.266 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.266 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.266 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.267 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.267 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.267 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.267 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.267 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.267 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:33:40.267 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.270 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.271 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.271 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.271 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.271 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.271 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.272 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.272 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.272 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.272 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.272 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.273 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.273 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.273 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.273 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.273 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.273 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.273 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.273 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.274 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.275 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.276 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.276 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.276 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.276 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.276 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.276 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.276 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.276 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.277 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.277 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.277 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.277 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.277 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.277 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: pgmap v376: 97 pgs: 97 active+clean; 496 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.6 MiB/s wr, 722 op/s 2023-07-18T03:33:40.277 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.278 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.278 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.278 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.278 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.278 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.278 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.278 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.278 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.279 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.279 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.279 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.279 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.279 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.279 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.279 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.279 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.280 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.280 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.280 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.280 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.280 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.280 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.280 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.281 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.282 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.282 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.282 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.282 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.282 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.282 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.282 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.283 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.283 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.283 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.283 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.283 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.283 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.283 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.283 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.284 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.285 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.286 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.287 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.287 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.287 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.287 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.287 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.287 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.287 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.287 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.288 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.288 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.288 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.288 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.288 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.288 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.288 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.288 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.289 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.289 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.289 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.289 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.289 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.289 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.289 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.290 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.291 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.291 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.291 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.291 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.291 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.291 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.292 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.292 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.292 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.292 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.292 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.292 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.292 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.293 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.293 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.293 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.293 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.293 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.293 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.294 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.294 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.294 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.294 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.294 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.294 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.295 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.295 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.295 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.295 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.295 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.295 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.295 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.296 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.296 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.296 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.296 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.296 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.296 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.296 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.296 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.297 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.297 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.297 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.297 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.297 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.297 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.297 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.298 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.299 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.299 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.299 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.299 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.299 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.299 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.299 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.299 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.300 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.300 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.300 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.300 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.300 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.301 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.301 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.301 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.301 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.301 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.301 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.302 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.302 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.302 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.302 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.302 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.302 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.303 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.303 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.303 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.303 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.303 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.304 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.304 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.304 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.304 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.304 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.304 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.304 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.305 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.305 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.305 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.305 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.305 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.305 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.305 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.306 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.306 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.306 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.306 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.306 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.306 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.306 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.307 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.307 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.307 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.307 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.307 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.307 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.307 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.308 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.308 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.308 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.308 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.308 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.308 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.308 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.308 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.309 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.309 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.309 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.309 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.309 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.309 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.309 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.309 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.310 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.310 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.310 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.310 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.310 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.310 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.310 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.311 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.311 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.311 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.311 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.311 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.311 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.311 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.311 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.312 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.312 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.312 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.312 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.312 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.312 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.312 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.314 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.314 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.314 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.314 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.314 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.314 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.314 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.315 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.315 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.315 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.315 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.315 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.315 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.315 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.316 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.316 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.316 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.316 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.316 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.316 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.316 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.316 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.317 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.317 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.317 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.317 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.317 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.317 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.317 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.317 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.318 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.318 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.318 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.318 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.318 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.318 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.318 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.319 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.319 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.319 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.319 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.319 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.319 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.319 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.319 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.320 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.320 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.320 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.320 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.320 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.320 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.320 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.320 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.321 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.321 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.321 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.321 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.321 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.321 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.321 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.322 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.322 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.322 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.322 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.322 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.322 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.323 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.323 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.323 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.323 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.323 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.323 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.324 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.324 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.324 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.324 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.324 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.324 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.324 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.325 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.325 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.325 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.325 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.325 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.325 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.325 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.326 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.326 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.326 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.326 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.326 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.326 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.326 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.327 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.327 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.327 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.327 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.327 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.327 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.328 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.328 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.328 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.328 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.328 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.328 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.329 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.329 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.329 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.329 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.329 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.330 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.330 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.330 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.330 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.330 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.331 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.331 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.331 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.331 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.331 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.331 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.331 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.332 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.332 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.332 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.332 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.332 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.332 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.332 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.333 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.333 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.333 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.333 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.333 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.333 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.333 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.334 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.334 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.334 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.334 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.334 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.334 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.334 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.335 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.335 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.335 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.335 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.335 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.335 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.335 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.335 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.336 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.337 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.338 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.339 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.339 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.339 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.339 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.339 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.339 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.339 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.340 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.340 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.340 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.340 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.340 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.340 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.340 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.340 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.341 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.341 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.341 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.341 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.341 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.341 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.341 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.341 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.342 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.342 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.342 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.342 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.342 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.342 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.342 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.342 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.343 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.343 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.343 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.343 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.343 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.343 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.343 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.344 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.344 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.344 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.344 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.344 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.344 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.344 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.345 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.345 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.345 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.345 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.345 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.345 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.345 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.346 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.346 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.346 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.346 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.346 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.346 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.346 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.346 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.354 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.355 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.356 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.356 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.356 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.356 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.356 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.356 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.356 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.357 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.357 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.357 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.357 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.357 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.357 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.357 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.357 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.358 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.358 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.358 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.358 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.358 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.358 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.359 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.359 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.359 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.359 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.359 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.360 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.360 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.360 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.360 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.360 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.360 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.361 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.361 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.361 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.361 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.361 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.361 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.362 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.362 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.362 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.362 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.362 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.362 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.363 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.363 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.363 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.363 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.363 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.363 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.363 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.363 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.364 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.364 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.364 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.364 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.364 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.364 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.365 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.365 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.365 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.365 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.365 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.365 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.365 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.366 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.366 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.366 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.366 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.366 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.367 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.367 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.367 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.367 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.367 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.367 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.367 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.367 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.368 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.368 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.368 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.368 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.368 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.368 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.368 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.369 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.369 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.369 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.369 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.369 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.369 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.369 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.369 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.370 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.370 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.370 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.370 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.370 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.370 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.370 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.370 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.371 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.371 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.371 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.371 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.371 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.371 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.371 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.371 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.372 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.372 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.372 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.372 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.372 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.372 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.372 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.373 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.373 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.373 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.373 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.373 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.373 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.373 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.373 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.374 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.374 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.374 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.374 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.374 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.374 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.374 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.374 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.375 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.375 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.375 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.375 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.375 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.375 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.375 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.375 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.376 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.376 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.376 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.376 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.376 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.376 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.376 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.376 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.377 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.377 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.377 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.377 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.377 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.377 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.377 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.377 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.378 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.378 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.378 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.378 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.378 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.378 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.378 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.379 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.380 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.380 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.380 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.380 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.380 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.380 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.380 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.380 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.381 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.381 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.381 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.381 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.381 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.381 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.381 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.381 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.382 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.382 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.382 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.382 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.382 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.382 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.382 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.382 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.383 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.383 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.383 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.383 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.383 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.383 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.384 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.384 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.384 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.384 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.384 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.384 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.384 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:40.385 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:33:40.385 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:33:42.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:41 smithi134 ceph-mon[98216]: pgmap v377: 97 pgs: 97 active+clean; 503 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.5 MiB/s wr, 729 op/s 2023-07-18T03:33:42.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:41 smithi040 ceph-mon[96439]: pgmap v377: 97 pgs: 97 active+clean; 503 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.5 MiB/s wr, 729 op/s 2023-07-18T03:33:42.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:41 smithi064 ceph-mon[98059]: pgmap v377: 97 pgs: 97 active+clean; 503 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.5 MiB/s wr, 729 op/s 2023-07-18T03:33:43.150 INFO:tasks.workunit.client.0.smithi040.stdout: 140 272337 323 198905 206 49947 25 2023-07-18T03:33:44.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:43 smithi134 ceph-mon[98216]: pgmap v378: 97 pgs: 97 active+clean; 543 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 910 op/s 2023-07-18T03:33:44.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:43 smithi040 ceph-mon[96439]: pgmap v378: 97 pgs: 97 active+clean; 543 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 910 op/s 2023-07-18T03:33:44.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:43 smithi064 ceph-mon[98059]: pgmap v378: 97 pgs: 97 active+clean; 543 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 13 MiB/s wr, 910 op/s 2023-07-18T03:33:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.990+0000 7f5256d21700 1 -- 172.21.15.40:0/868255466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5250071850 msgr2=0x7f525010b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.990+0000 7f5256d21700 1 --2- 172.21.15.40:0/868255466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5250071850 0x7f525010b7d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f52400098f0 tx=0x7f5240009c00 comp rx=0 tx=0).stop 2023-07-18T03:33:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.991+0000 7f5256d21700 1 -- 172.21.15.40:0/868255466 shutdown_connections 2023-07-18T03:33:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.991+0000 7f5256d21700 1 --2- 172.21.15.40:0/868255466 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f525010bd60 0x7f525010e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.991+0000 7f5256d21700 1 --2- 172.21.15.40:0/868255466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5250070f00 0x7f5250071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.991+0000 7f5256d21700 1 --2- 172.21.15.40:0/868255466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5250071850 0x7f525010b7d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.991+0000 7f5256d21700 1 -- 172.21.15.40:0/868255466 >> 172.21.15.40:0/868255466 conn(0x7f525006ca50 msgr2=0x7f525006eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.991+0000 7f5256d21700 1 -- 172.21.15.40:0/868255466 shutdown_connections 2023-07-18T03:33:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.991+0000 7f5256d21700 1 -- 172.21.15.40:0/868255466 wait complete. 2023-07-18T03:33:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.992+0000 7f5256d21700 1 Processor -- start 2023-07-18T03:33:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.992+0000 7f5256d21700 1 -- start start 2023-07-18T03:33:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.992+0000 7f5256d21700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5250070f00 0x7f52501139e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.992+0000 7f5256d21700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5250071850 0x7f5250113f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.992+0000 7f5256d21700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f525010bd60 0x7f5250114df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.992+0000 7f5256d21700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5250115450 con 0x7f5250071850 2023-07-18T03:33:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.992+0000 7f5256d21700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52501155c0 con 0x7f5250070f00 2023-07-18T03:33:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.992+0000 7f5256d21700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5250115780 con 0x7f525010bd60 2023-07-18T03:33:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.993+0000 7f5254abd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5250070f00 0x7f52501139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.993+0000 7f5254abd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5250070f00 0x7f52501139e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42940/0 (socket says 172.21.15.40:42940) 2023-07-18T03:33:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.993+0000 7f5254abd700 1 -- 172.21.15.40:0/717433107 learned_addr learned my addr 172.21.15.40:0/717433107 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.993+0000 7f524ffff700 1 --2- 172.21.15.40:0/717433107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5250071850 0x7f5250113f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.994+0000 7f524ffff700 1 -- 172.21.15.40:0/717433107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f525010bd60 msgr2=0x7f5250114df0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:33:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.994+0000 7f524ffff700 1 --2- 172.21.15.40:0/717433107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f525010bd60 0x7f5250114df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.994+0000 7f524ffff700 1 -- 172.21.15.40:0/717433107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5250070f00 msgr2=0x7f52501139e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.994+0000 7f524ffff700 1 --2- 172.21.15.40:0/717433107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5250070f00 0x7f52501139e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.994+0000 7f524ffff700 1 -- 172.21.15.40:0/717433107 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5240009520 con 0x7f5250071850 2023-07-18T03:33:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:45.994+0000 7f524ffff700 1 --2- 172.21.15.40:0/717433107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5250071850 0x7f5250113f20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f524000b2e0 tx=0x7f524000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.004+0000 7f524dffb700 1 -- 172.21.15.40:0/717433107 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524001c070 con 0x7f5250071850 2023-07-18T03:33:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.004+0000 7f524dffb700 1 -- 172.21.15.40:0/717433107 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5240005530 con 0x7f5250071850 2023-07-18T03:33:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.004+0000 7f524dffb700 1 -- 172.21.15.40:0/717433107 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240003e80 con 0x7f5250071850 2023-07-18T03:33:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.004+0000 7f5256d21700 1 -- 172.21.15.40:0/717433107 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5250115a00 con 0x7f5250071850 2023-07-18T03:33:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.004+0000 7f5256d21700 1 -- 172.21.15.40:0/717433107 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5250116210 con 0x7f5250071850 2023-07-18T03:33:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.010+0000 7f524dffb700 1 -- 172.21.15.40:0/717433107 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5240003680 con 0x7f5250071850 2023-07-18T03:33:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.010+0000 7f5262ea7b80 1 Processor -- start 2023-07-18T03:33:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.011+0000 7f5262ea7b80 1 -- start start 2023-07-18T03:33:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.011+0000 7f524dffb700 1 --2- 172.21.15.40:0/717433107 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f523c0a1350 0x7f523c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.011+0000 7f5262ea7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf7ed6b0 0x561baf991300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.011+0000 7f5262ea7b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf991840 0x561baf9921a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.011+0000 7f5262ea7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf9926e0 0x561baf992b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.011+0000 7f5262ea7b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561baf896c50 con 0x561baf991840 2023-07-18T03:33:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.011+0000 7f5262ea7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561baf8e99e0 con 0x561baf9926e0 2023-07-18T03:33:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.011+0000 7f5262ea7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561baf8e2e70 con 0x561baf7ed6b0 2023-07-18T03:33:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.011+0000 7f524dffb700 1 -- 172.21.15.40:0/717433107 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f52400cdaf0 con 0x7f5250071850 2023-07-18T03:33:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.012+0000 7f5254abd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf7ed6b0 0x561baf991300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.012+0000 7f5254abd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf7ed6b0 0x561baf991300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48646/0 (socket says 172.21.15.40:48646) 2023-07-18T03:33:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.012+0000 7f5254abd700 1 -- 172.21.15.40:0/1060913651 learned_addr learned my addr 172.21.15.40:0/1060913651 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.012+0000 7f524ffff700 1 --2- 172.21.15.40:0/1060913651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf991840 0x561baf9921a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.013+0000 7f5254abd700 1 -- 172.21.15.40:0/1060913651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf9926e0 msgr2=0x561baf992b80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:33:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.013+0000 7f5254abd700 1 --2- 172.21.15.40:0/1060913651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf9926e0 0x561baf992b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.013+0000 7f5254abd700 1 -- 172.21.15.40:0/1060913651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf991840 msgr2=0x561baf9921a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.013+0000 7f5254abd700 1 --2- 172.21.15.40:0/1060913651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf991840 0x561baf9921a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.013+0000 7f5254abd700 1 -- 172.21.15.40:0/1060913651 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561baf710d80 con 0x561baf7ed6b0 2023-07-18T03:33:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.013+0000 7f5254abd700 1 --2- 172.21.15.40:0/1060913651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf7ed6b0 0x561baf991300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f52480098a0 tx=0x7f5248009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55f21dc93d3c1781 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.019+0000 7f52552be700 1 --2- 172.21.15.40:0/717433107 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f523c0a1350 0x7f523c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.020+0000 7f52552be700 1 --2- 172.21.15.40:0/717433107 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f523c0a1350 0x7f523c0a3800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5238005b60 tx=0x7f5238005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.022+0000 7f5246ffd700 1 -- 172.21.15.40:0/1060913651 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248007cd0 con 0x561baf7ed6b0 2023-07-18T03:33:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.022+0000 7f5246ffd700 1 -- 172.21.15.40:0/1060913651 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f524800f050 con 0x561baf7ed6b0 2023-07-18T03:33:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.022+0000 7f5246ffd700 1 -- 172.21.15.40:0/1060913651 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248014770 con 0x561baf7ed6b0 2023-07-18T03:33:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.024+0000 7f5262ea7b80 1 -- 172.21.15.40:0/1060913651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf7ed6b0 msgr2=0x561baf991300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.024+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/1060913651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf7ed6b0 0x561baf991300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f52480098a0 tx=0x7f5248009bb0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.024+0000 7f5262ea7b80 1 -- 172.21.15.40:0/1060913651 shutdown_connections 2023-07-18T03:33:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.024+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/1060913651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf9926e0 0x561baf992b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.024+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/1060913651 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf7ed6b0 0x561baf991300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.024+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/1060913651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf991840 0x561baf9921a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.024+0000 7f5262ea7b80 1 -- 172.21.15.40:0/1060913651 >> 172.21.15.40:0/1060913651 conn(0x561baf986be0 msgr2=0x561baf98bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.024+0000 7f5262ea7b80 1 -- 172.21.15.40:0/1060913651 shutdown_connections 2023-07-18T03:33:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.024+0000 7f5262ea7b80 1 -- 172.21.15.40:0/1060913651 wait complete. 2023-07-18T03:33:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.025+0000 7f5262ea7b80 1 -- start start 2023-07-18T03:33:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.025+0000 7f5262ea7b80 1 Processor -- start 2023-07-18T03:33:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.025+0000 7f5262ea7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf7ed6b0 0x561baf99a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.025+0000 7f5262ea7b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf991840 0x561baf99ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.025+0000 7f5262ea7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf9926e0 0x561baf9a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.025+0000 7f5262ea7b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561baf882a60 con 0x561baf991840 2023-07-18T03:33:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.025+0000 7f5262ea7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561baf882760 con 0x561baf7ed6b0 2023-07-18T03:33:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.025+0000 7f5262ea7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561baf7668c0 con 0x561baf9926e0 2023-07-18T03:33:46.025 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.025+0000 7f5262ea7b80 10 client.0 _ll_register_callbacks cb 0x561baf880700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.026+0000 7f524ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf991840 0x561baf99ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.026+0000 7f524ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf991840 0x561baf99ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59626/0 (socket says 172.21.15.40:59626) 2023-07-18T03:33:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.026+0000 7f524ffff700 1 -- 172.21.15.40:0/1674016803 learned_addr learned my addr 172.21.15.40:0/1674016803 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.026+0000 7f52552be700 1 --2- 172.21.15.40:0/1674016803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf9926e0 0x561baf9a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.026+0000 7f524ffff700 1 -- 172.21.15.40:0/1674016803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf9926e0 msgr2=0x561baf9a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.026+0000 7f524ffff700 1 --2- 172.21.15.40:0/1674016803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf9926e0 0x561baf9a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.026+0000 7f524ffff700 1 -- 172.21.15.40:0/1674016803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf7ed6b0 msgr2=0x561baf99a910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:33:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.026+0000 7f524ffff700 1 --2- 172.21.15.40:0/1674016803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf7ed6b0 0x561baf99a910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.026+0000 7f524ffff700 1 -- 172.21.15.40:0/1674016803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5248003c10 con 0x561baf991840 2023-07-18T03:33:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.027+0000 7f524ffff700 1 --2- 172.21.15.40:0/1674016803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf991840 0x561baf99ae50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f524000fbd0 tx=0x7f524009afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0a2afb04f20d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.031+0000 7f5237fff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:33:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.031+0000 7f5237fff700 1 -- 172.21.15.40:0/1674016803 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52400a32d0 con 0x561baf991840 2023-07-18T03:33:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.031+0000 7f5237fff700 1 -- 172.21.15.40:0/1674016803 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f524009a400 con 0x561baf991840 2023-07-18T03:33:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.031+0000 7f5262ea7b80 1 -- 172.21.15.40:0/1674016803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561baf83c490 con 0x561baf991840 2023-07-18T03:33:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.031+0000 7f5237fff700 1 -- 172.21.15.40:0/1674016803 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52400b9250 con 0x561baf991840 2023-07-18T03:33:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.031+0000 7f5262ea7b80 10 client.16143 fetch_fsmap learned FSMap version 32 2023-07-18T03:33:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.032+0000 7f5262ea7b80 1 -- 172.21.15.40:0/1674016803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561baf6f9c00 con 0x561baf991840 2023-07-18T03:33:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.032+0000 7f5237fff700 1 -- 172.21.15.40:0/1674016803 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561baf83c490 con 0x561baf991840 2023-07-18T03:33:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.032+0000 7f5237fff700 1 -- 172.21.15.40:0/1674016803 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f52400aeb80 con 0x561baf991840 2023-07-18T03:33:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.032+0000 7f5262ea7b80 10 client.16143 fetch_fsmap finished waiting for FSMap version 32 2023-07-18T03:33:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.032+0000 7f5262ea7b80 10 client.16143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:33:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.032+0000 7f5262ea7b80 20 client.16143 populate_metadata read hostname 'smithi040' 2023-07-18T03:33:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.032+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/1674016803 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561baf9a6300 0x561baf9a8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.032+0000 7f5262ea7b80 4 client.16143 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:33:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.032+0000 7f5262ea7b80 1 -- 172.21.15.40:0/1674016803 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561baf8dbe50 con 0x561baf9a6300 2023-07-18T03:33:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.033+0000 7f5254abd700 1 --2- 172.21.15.40:0/1674016803 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561baf9a6300 0x561baf9a8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.039+0000 7f5237fff700 10 client.16143 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.039+0000 7f5254abd700 1 --2- 172.21.15.40:0/1674016803 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561baf9a6300 0x561baf9a8710 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3a96606fe30ceaf server_cookie=a2bd4f9cea78ccdb in_seq=0 out_seq=0 2023-07-18T03:33:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.040+0000 7f5237fff700 1 -- 172.21.15.40:0/1674016803 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561baf8dbe50 con 0x561baf9a6300 2023-07-18T03:33:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.040+0000 7f5237fff700 10 client.16143 handle_command_reply: tid=0 2023-07-18T03:33:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.040+0000 7f5262ea7b80 1 client.16143 shutdown 2023-07-18T03:33:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.040+0000 7f5262ea7b80 1 -- 172.21.15.40:0/1674016803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf991840 msgr2=0x561baf99ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.040+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/1674016803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf991840 0x561baf99ae50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f524000fbd0 tx=0x7f524009afc0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.040+0000 7f5262ea7b80 1 -- 172.21.15.40:0/1674016803 shutdown_connections 2023-07-18T03:33:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.040+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/1674016803 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561baf9a6300 0x561baf9a8710 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.040+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/1674016803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf9926e0 0x561baf9a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.040+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/1674016803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf7ed6b0 0x561baf99a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.040+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/1674016803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf991840 0x561baf99ae50 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.040+0000 7f5262ea7b80 1 -- 172.21.15.40:0/1674016803 >> 172.21.15.40:0/1674016803 conn(0x561baf986be0 msgr2=0x561baf98b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.040+0000 7f5237fff700 0 client.16143 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.040+0000 7f5262ea7b80 1 -- 172.21.15.40:0/1674016803 shutdown_connections 2023-07-18T03:33:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.040+0000 7f5262ea7b80 1 -- 172.21.15.40:0/1674016803 wait complete. 2023-07-18T03:33:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.041+0000 7f5262ea7b80 20 client.16143 trim_cache size 0 max 16384 2023-07-18T03:33:46.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:45 smithi134 ceph-mon[98216]: pgmap v379: 97 pgs: 97 active+clean; 543 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 902 op/s 2023-07-18T03:33:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.074+0000 7f5262ea7b80 1 Processor -- start 2023-07-18T03:33:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.074+0000 7f5262ea7b80 1 -- start start 2023-07-18T03:33:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.074+0000 7f5262ea7b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf7ed6b0 0x561baf9955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.074+0000 7f5262ea7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf991840 0x561baf9a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.074+0000 7f5262ea7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf9926e0 0x561baf995b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.074+0000 7f5262ea7b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561baf8dbe50 con 0x561baf7ed6b0 2023-07-18T03:33:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.074+0000 7f5262ea7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561baf83c490 con 0x561baf991840 2023-07-18T03:33:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.074+0000 7f5262ea7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561baf957060 con 0x561baf9926e0 2023-07-18T03:33:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.074+0000 7f5254abd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf7ed6b0 0x561baf9955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.074+0000 7f524ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf991840 0x561baf9a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.074+0000 7f5254abd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf7ed6b0 0x561baf9955f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59632/0 (socket says 172.21.15.40:59632) 2023-07-18T03:33:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.074+0000 7f5254abd700 1 -- 172.21.15.40:0/2651414833 learned_addr learned my addr 172.21.15.40:0/2651414833 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.074+0000 7f52552be700 1 --2- 172.21.15.40:0/2651414833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf9926e0 0x561baf995b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.075+0000 7f52552be700 1 -- 172.21.15.40:0/2651414833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf991840 msgr2=0x561baf9a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.075+0000 7f52552be700 1 --2- 172.21.15.40:0/2651414833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf991840 0x561baf9a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.075+0000 7f52552be700 1 -- 172.21.15.40:0/2651414833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf7ed6b0 msgr2=0x561baf9955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.075+0000 7f52552be700 1 --2- 172.21.15.40:0/2651414833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf7ed6b0 0x561baf9955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.075+0000 7f52552be700 1 -- 172.21.15.40:0/2651414833 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561baf7371f0 con 0x561baf9926e0 2023-07-18T03:33:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.075+0000 7f524ffff700 1 --2- 172.21.15.40:0/2651414833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf991840 0x561baf9a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:33:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.075+0000 7f5254abd700 1 --2- 172.21.15.40:0/2651414833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf7ed6b0 0x561baf9955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:33:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.075+0000 7f52552be700 1 --2- 172.21.15.40:0/2651414833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf9926e0 0x561baf995b30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f523800e5d0 tx=0x7f5238011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1de6252969c5d2b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.076+0000 7f52477fe700 1 -- 172.21.15.40:0/2651414833 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238011900 con 0x561baf9926e0 2023-07-18T03:33:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.076+0000 7f52477fe700 1 -- 172.21.15.40:0/2651414833 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f523801f420 con 0x561baf9926e0 2023-07-18T03:33:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.076+0000 7f52477fe700 1 -- 172.21.15.40:0/2651414833 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238010630 con 0x561baf9926e0 2023-07-18T03:33:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.077+0000 7f5262ea7b80 1 -- 172.21.15.40:0/2651414833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf9926e0 msgr2=0x561baf995b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.077+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/2651414833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf9926e0 0x561baf995b30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f523800e5d0 tx=0x7f5238011000 comp rx=0 tx=0).stop 2023-07-18T03:33:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.077+0000 7f5262ea7b80 1 -- 172.21.15.40:0/2651414833 shutdown_connections 2023-07-18T03:33:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.077+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/2651414833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf9926e0 0x561baf995b30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.077+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/2651414833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf7ed6b0 0x561baf9955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.077+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/2651414833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf991840 0x561baf9a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.077+0000 7f5262ea7b80 1 -- 172.21.15.40:0/2651414833 >> 172.21.15.40:0/2651414833 conn(0x561baf986be0 msgr2=0x561baf998b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.077+0000 7f5262ea7b80 1 -- 172.21.15.40:0/2651414833 shutdown_connections 2023-07-18T03:33:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.077+0000 7f5262ea7b80 1 -- 172.21.15.40:0/2651414833 wait complete. 2023-07-18T03:33:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.077+0000 7f5262ea7b80 1 -- start start 2023-07-18T03:33:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.078+0000 7f5262ea7b80 1 Processor -- start 2023-07-18T03:33:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.078+0000 7f5262ea7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf7ed6b0 0x561baf99d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.078+0000 7f5262ea7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf991840 0x561baf99dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.078+0000 7f5262ea7b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf9926e0 0x561baf9a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.078+0000 7f5262ea7b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561baf9a1400 con 0x561baf9926e0 2023-07-18T03:33:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.078+0000 7f5262ea7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x561baf9a1570 con 0x561baf991840 2023-07-18T03:33:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.078+0000 7f5262ea7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x561baf9a1730 con 0x561baf7ed6b0 2023-07-18T03:33:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.078+0000 7f5262ea7b80 10 client.0 _ll_register_callbacks cb 0x7f524009a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.078+0000 7f524ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf991840 0x561baf99dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.078+0000 7f5254abd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf7ed6b0 0x561baf99d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.078+0000 7f524ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf991840 0x561baf99dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:42964/0 (socket says 172.21.15.40:42964) 2023-07-18T03:33:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.078+0000 7f524ffff700 1 -- 172.21.15.40:0/4069136316 learned_addr learned my addr 172.21.15.40:0/4069136316 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.079+0000 7f52552be700 1 --2- 172.21.15.40:0/4069136316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf9926e0 0x561baf9a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.079+0000 7f524ffff700 1 -- 172.21.15.40:0/4069136316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf7ed6b0 msgr2=0x561baf99d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.079+0000 7f524ffff700 1 --2- 172.21.15.40:0/4069136316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf7ed6b0 0x561baf99d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.079+0000 7f524ffff700 1 -- 172.21.15.40:0/4069136316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf9926e0 msgr2=0x561baf9a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.079+0000 7f524ffff700 1 --2- 172.21.15.40:0/4069136316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf9926e0 0x561baf9a0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.079+0000 7f524ffff700 1 -- 172.21.15.40:0/4069136316 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561baf710d80 con 0x561baf991840 2023-07-18T03:33:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.079+0000 7f524ffff700 1 --2- 172.21.15.40:0/4069136316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf991840 0x561baf99dea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f52400a2da0 tx=0x7f52400a31d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ec821ebaf36a7fb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.079+0000 7f5237fff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:33:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.079+0000 7f5237fff700 1 -- 172.21.15.40:0/4069136316 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52400aed80 con 0x561baf991840 2023-07-18T03:33:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.079+0000 7f5237fff700 1 -- 172.21.15.40:0/4069136316 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f52400b9250 con 0x561baf991840 2023-07-18T03:33:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.079+0000 7f5262ea7b80 1 -- 172.21.15.40:0/4069136316 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f524800f050 con 0x561baf991840 2023-07-18T03:33:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.080+0000 7f5237fff700 1 -- 172.21.15.40:0/4069136316 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52400a3010 con 0x561baf991840 2023-07-18T03:33:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.080+0000 7f5262ea7b80 10 client.25778 fetch_fsmap learned FSMap version 32 2023-07-18T03:33:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.080+0000 7f5262ea7b80 1 -- 172.21.15.40:0/4069136316 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f523800e840 con 0x561baf991840 2023-07-18T03:33:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.080+0000 7f5237fff700 1 -- 172.21.15.40:0/4069136316 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f524800f050 con 0x561baf991840 2023-07-18T03:33:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.080+0000 7f5237fff700 1 -- 172.21.15.40:0/4069136316 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f52400050a0 con 0x561baf991840 2023-07-18T03:33:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.080+0000 7f5262ea7b80 10 client.25778 fetch_fsmap finished waiting for FSMap version 32 2023-07-18T03:33:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.080+0000 7f5262ea7b80 10 client.25778 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:33:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.080+0000 7f5262ea7b80 20 client.25778 populate_metadata read hostname 'smithi040' 2023-07-18T03:33:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.081+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/4069136316 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561baf9fabf0 0x561baf9fd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.081+0000 7f5262ea7b80 4 client.25778 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:33:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.081+0000 7f5262ea7b80 1 -- 172.21.15.40:0/4069136316 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561baf89dff0 con 0x561baf9fabf0 2023-07-18T03:33:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.081+0000 7f5254abd700 1 --2- 172.21.15.40:0/4069136316 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561baf9fabf0 0x561baf9fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.081+0000 7f5237fff700 10 client.25778 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.081+0000 7f5254abd700 1 --2- 172.21.15.40:0/4069136316 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561baf9fabf0 0x561baf9fd000 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da95d8f75d5acebf server_cookie=3aab572e2324be6c in_seq=0 out_seq=0 2023-07-18T03:33:46.084 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:45 smithi040 ceph-mon[96439]: pgmap v379: 97 pgs: 97 active+clean; 543 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 902 op/s 2023-07-18T03:33:46.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:45 smithi064 ceph-mon[98059]: pgmap v379: 97 pgs: 97 active+clean; 543 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 902 op/s 2023-07-18T03:33:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.371+0000 7f5237fff700 1 -- 172.21.15.40:0/4069136316 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561baf89dff0 con 0x561baf9fabf0 2023-07-18T03:33:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.371+0000 7f5237fff700 10 client.25778 handle_command_reply: tid=0 2023-07-18T03:33:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.372+0000 7f5262ea7b80 1 client.25778 shutdown 2023-07-18T03:33:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.372+0000 7f5262ea7b80 1 -- 172.21.15.40:0/4069136316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf991840 msgr2=0x561baf99dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.372+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/4069136316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf991840 0x561baf99dea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f52400a2da0 tx=0x7f52400a31d0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.374+0000 7f5262ea7b80 1 -- 172.21.15.40:0/4069136316 shutdown_connections 2023-07-18T03:33:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.374+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/4069136316 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x561baf9fabf0 0x561baf9fd000 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.374+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/4069136316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561baf9926e0 0x561baf9a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.374+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/4069136316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561baf7ed6b0 0x561baf99d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.374+0000 7f5262ea7b80 1 --2- 172.21.15.40:0/4069136316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561baf991840 0x561baf99dea0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.374+0000 7f5262ea7b80 1 -- 172.21.15.40:0/4069136316 >> 172.21.15.40:0/4069136316 conn(0x561baf986be0 msgr2=0x561baf9955f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.374+0000 7f5237fff700 0 client.25778 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.374+0000 7f5262ea7b80 1 -- 172.21.15.40:0/4069136316 shutdown_connections 2023-07-18T03:33:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:33:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:33:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:33:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:33:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.381+0000 7f5262ea7b80 1 -- 172.21.15.40:0/4069136316 wait complete. 2023-07-18T03:33:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.381+0000 7f5262ea7b80 20 client.25778 trim_cache size 0 max 16384 2023-07-18T03:33:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.397+0000 7f5256d21700 1 -- 172.21.15.40:0/717433107 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f523c0a1350 msgr2=0x7f523c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.397+0000 7f5256d21700 1 --2- 172.21.15.40:0/717433107 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f523c0a1350 0x7f523c0a3800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5238005b60 tx=0x7f5238005f30 comp rx=0 tx=0).stop 2023-07-18T03:33:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.397+0000 7f5256d21700 1 -- 172.21.15.40:0/717433107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5250071850 msgr2=0x7f5250113f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.397+0000 7f5256d21700 1 --2- 172.21.15.40:0/717433107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5250071850 0x7f5250113f20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f524000b2e0 tx=0x7f524000b8d0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.398+0000 7f5256d21700 1 -- 172.21.15.40:0/717433107 shutdown_connections 2023-07-18T03:33:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.398+0000 7f5256d21700 1 --2- 172.21.15.40:0/717433107 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f523c0a1350 0x7f523c0a3800 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.398+0000 7f5256d21700 1 --2- 172.21.15.40:0/717433107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f525010bd60 0x7f5250114df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.398+0000 7f5256d21700 1 --2- 172.21.15.40:0/717433107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5250070f00 0x7f52501139e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.398+0000 7f5256d21700 1 --2- 172.21.15.40:0/717433107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5250071850 0x7f5250113f20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.398+0000 7f5256d21700 1 -- 172.21.15.40:0/717433107 >> 172.21.15.40:0/717433107 conn(0x7f525006ca50 msgr2=0x7f525006db50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.398+0000 7f5256d21700 1 -- 172.21.15.40:0/717433107 shutdown_connections 2023-07-18T03:33:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:46.398+0000 7f5256d21700 1 -- 172.21.15.40:0/717433107 wait complete. 2023-07-18T03:33:48.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:47 smithi134 ceph-mon[98216]: pgmap v380: 97 pgs: 97 active+clean; 543 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 857 op/s 2023-07-18T03:33:48.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:47 smithi040 ceph-mon[96439]: pgmap v380: 97 pgs: 97 active+clean; 543 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 857 op/s 2023-07-18T03:33:48.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:47 smithi064 ceph-mon[98059]: pgmap v380: 97 pgs: 97 active+clean; 543 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 857 op/s 2023-07-18T03:33:50.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:49 smithi134 ceph-mon[98216]: pgmap v381: 97 pgs: 97 active+clean; 546 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 11 MiB/s wr, 745 op/s 2023-07-18T03:33:50.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:49 smithi040 ceph-mon[96439]: pgmap v381: 97 pgs: 97 active+clean; 546 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 11 MiB/s wr, 745 op/s 2023-07-18T03:33:50.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:49 smithi064 ceph-mon[98059]: pgmap v381: 97 pgs: 97 active+clean; 546 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 11 MiB/s wr, 745 op/s 2023-07-18T03:33:52.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:51 smithi134 ceph-mon[98216]: pgmap v382: 97 pgs: 97 active+clean; 546 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 4.6 MiB/s wr, 295 op/s 2023-07-18T03:33:52.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:51 smithi040 ceph-mon[96439]: pgmap v382: 97 pgs: 97 active+clean; 546 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 4.6 MiB/s wr, 295 op/s 2023-07-18T03:33:52.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:51 smithi064 ceph-mon[98059]: pgmap v382: 97 pgs: 97 active+clean; 546 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 4.6 MiB/s wr, 295 op/s 2023-07-18T03:33:53.150 INFO:tasks.workunit.client.0.smithi040.stdout: 156 1512499 819 1099375 959 243159 23 2023-07-18T03:33:54.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:53 smithi134 ceph-mon[98216]: pgmap v383: 97 pgs: 97 active+clean; 552 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 309 op/s 2023-07-18T03:33:54.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:53 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:33:54.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:53 smithi040 ceph-mon[96439]: pgmap v383: 97 pgs: 97 active+clean; 552 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 309 op/s 2023-07-18T03:33:54.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:53 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:33:54.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:53 smithi064 ceph-mon[98059]: pgmap v383: 97 pgs: 97 active+clean; 552 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 309 op/s 2023-07-18T03:33:54.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:53 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:33:56.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:55 smithi134 ceph-mon[98216]: pgmap v384: 97 pgs: 97 active+clean; 556 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2023-07-18T03:33:56.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:55 smithi040 ceph-mon[96439]: pgmap v384: 97 pgs: 97 active+clean; 556 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2023-07-18T03:33:56.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:55 smithi064 ceph-mon[98059]: pgmap v384: 97 pgs: 97 active+clean; 556 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.4 MiB/s wr, 139 op/s 2023-07-18T03:33:58.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:57 smithi134 ceph-mon[98216]: pgmap v385: 97 pgs: 97 active+clean; 560 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 134 op/s 2023-07-18T03:33:58.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:57 smithi040 ceph-mon[96439]: pgmap v385: 97 pgs: 97 active+clean; 560 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 134 op/s 2023-07-18T03:33:58.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:57 smithi064 ceph-mon[98059]: pgmap v385: 97 pgs: 97 active+clean; 560 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 134 op/s 2023-07-18T03:33:59.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:58 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:33:59.083 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:58 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:33:59.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:58 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:33:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.532+0000 7f5d640c4700 1 -- 172.21.15.40:0/3564863232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c0ffc90 msgr2=0x7f5d5c100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.532+0000 7f5d640c4700 1 --2- 172.21.15.40:0/3564863232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c0ffc90 0x7f5d5c100100 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009990 tx=0x7f5d4c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.532+0000 7f5d640c4700 1 -- 172.21.15.40:0/3564863232 shutdown_connections 2023-07-18T03:33:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.532+0000 7f5d640c4700 1 --2- 172.21.15.40:0/3564863232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c100640 0x7f5d5c106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.532+0000 7f5d640c4700 1 --2- 172.21.15.40:0/3564863232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d5c0fea00 0x7f5d5c0fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.532+0000 7f5d640c4700 1 --2- 172.21.15.40:0/3564863232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c0ffc90 0x7f5d5c100100 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.532+0000 7f5d640c4700 1 -- 172.21.15.40:0/3564863232 >> 172.21.15.40:0/3564863232 conn(0x7f5d5c0fa000 msgr2=0x7f5d5c0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.533+0000 7f5d640c4700 1 -- 172.21.15.40:0/3564863232 shutdown_connections 2023-07-18T03:33:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.533+0000 7f5d640c4700 1 -- 172.21.15.40:0/3564863232 wait complete. 2023-07-18T03:33:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.533+0000 7f5d640c4700 1 Processor -- start 2023-07-18T03:33:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.533+0000 7f5d640c4700 1 -- start start 2023-07-18T03:33:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.533+0000 7f5d640c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d5c0fea00 0x7f5d5c194870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.533+0000 7f5d640c4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c0ffc90 0x7f5d5c194db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.533+0000 7f5d640c4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c100640 0x7f5d5c195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.533+0000 7f5d640c4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c196290 con 0x7f5d5c100640 2023-07-18T03:33:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.533+0000 7f5d640c4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c196400 con 0x7f5d5c0fea00 2023-07-18T03:33:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.534+0000 7f5d640c4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c1965c0 con 0x7f5d5c0ffc90 2023-07-18T03:33:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.534+0000 7f5d62661700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c100640 0x7f5d5c195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.534+0000 7f5d61e60700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d5c0fea00 0x7f5d5c194870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.534+0000 7f5d62661700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c100640 0x7f5d5c195c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59658/0 (socket says 172.21.15.40:59658) 2023-07-18T03:33:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.534+0000 7f5d62661700 1 -- 172.21.15.40:0/2156969678 learned_addr learned my addr 172.21.15.40:0/2156969678 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.534+0000 7f5d6165f700 1 --2- 172.21.15.40:0/2156969678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c0ffc90 0x7f5d5c194db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.534+0000 7f5d62661700 1 -- 172.21.15.40:0/2156969678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c0ffc90 msgr2=0x7f5d5c194db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.534+0000 7f5d62661700 1 --2- 172.21.15.40:0/2156969678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c0ffc90 0x7f5d5c194db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.534+0000 7f5d62661700 1 -- 172.21.15.40:0/2156969678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d5c0fea00 msgr2=0x7f5d5c194870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.534+0000 7f5d62661700 1 --2- 172.21.15.40:0/2156969678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d5c0fea00 0x7f5d5c194870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.534+0000 7f5d62661700 1 -- 172.21.15.40:0/2156969678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c0095f0 con 0x7f5d5c100640 2023-07-18T03:33:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.534+0000 7f5d62661700 1 --2- 172.21.15.40:0/2156969678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c100640 0x7f5d5c195c30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400ef30 tx=0x7f5d5400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.534+0000 7f5d52ffd700 1 -- 172.21.15.40:0/2156969678 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54018070 con 0x7f5d5c100640 2023-07-18T03:33:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.534+0000 7f5d52ffd700 1 -- 172.21.15.40:0/2156969678 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d5400f040 con 0x7f5d5c100640 2023-07-18T03:33:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.534+0000 7f5d52ffd700 1 -- 172.21.15.40:0/2156969678 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54007ba0 con 0x7f5d5c100640 2023-07-18T03:33:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.534+0000 7f5d640c4700 1 -- 172.21.15.40:0/2156969678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c196840 con 0x7f5d5c100640 2023-07-18T03:33:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.534+0000 7f5d640c4700 1 -- 172.21.15.40:0/2156969678 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c067ae0 con 0x7f5d5c100640 2023-07-18T03:33:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.536+0000 7f5d52ffd700 1 -- 172.21.15.40:0/2156969678 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5d5400c940 con 0x7f5d5c100640 2023-07-18T03:33:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.537+0000 7f5d52ffd700 1 --2- 172.21.15.40:0/2156969678 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5d480a1150 0x7f5d480a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.537+0000 7f5d52ffd700 1 -- 172.21.15.40:0/2156969678 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5d540cca30 con 0x7f5d5c100640 2023-07-18T03:33:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.537+0000 7f5d61e60700 1 --2- 172.21.15.40:0/2156969678 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5d480a1150 0x7f5d480a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.538+0000 7f5d61e60700 1 --2- 172.21.15.40:0/2156969678 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5d480a1150 0x7f5d480a3600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009c60 tx=0x7f5d58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.538+0000 7f5d7024ab80 1 Processor -- start 2023-07-18T03:33:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.538+0000 7f5d7024ab80 1 -- start start 2023-07-18T03:33:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.538+0000 7f5d7024ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 0x56212c5fc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.538+0000 7f5d7024ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fc840 0x56212c5fd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.538+0000 7f5d7024ab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fd6e0 0x56212c5fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.538+0000 7f5d7024ab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212c501c50 con 0x56212c5fd6e0 2023-07-18T03:33:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.538+0000 7f5d7024ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212c5549e0 con 0x56212c5fc840 2023-07-18T03:33:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.538+0000 7f5d7024ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212c54de70 con 0x56212c4586b0 2023-07-18T03:33:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.539+0000 7f5d61e60700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 0x56212c5fc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.539+0000 7f5d61e60700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 0x56212c5fc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48690/0 (socket says 172.21.15.40:48690) 2023-07-18T03:33:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.539+0000 7f5d62661700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fd6e0 0x56212c5fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.539+0000 7f5d61e60700 1 -- 172.21.15.40:0/394704527 learned_addr learned my addr 172.21.15.40:0/394704527 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.539+0000 7f5d6165f700 1 --2- 172.21.15.40:0/394704527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fc840 0x56212c5fd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.539+0000 7f5d62661700 1 -- 172.21.15.40:0/394704527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 msgr2=0x56212c5fc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.539+0000 7f5d62661700 1 --2- 172.21.15.40:0/394704527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 0x56212c5fc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.539+0000 7f5d62661700 1 -- 172.21.15.40:0/394704527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fc840 msgr2=0x56212c5fd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.539+0000 7f5d62661700 1 --2- 172.21.15.40:0/394704527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fc840 0x56212c5fd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.539+0000 7f5d62661700 1 -- 172.21.15.40:0/394704527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56212c37bd80 con 0x56212c5fd6e0 2023-07-18T03:33:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.539+0000 7f5d62661700 1 --2- 172.21.15.40:0/394704527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fd6e0 0x56212c5fdb80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5d540ca560 tx=0x7f5d54096320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d2496f7461180b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.539+0000 7f5d47fff700 1 -- 172.21.15.40:0/394704527 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54099eb0 con 0x56212c5fd6e0 2023-07-18T03:33:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.539+0000 7f5d47fff700 1 -- 172.21.15.40:0/394704527 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d5409f040 con 0x56212c5fd6e0 2023-07-18T03:33:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.539+0000 7f5d47fff700 1 -- 172.21.15.40:0/394704527 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5409a110 con 0x56212c5fd6e0 2023-07-18T03:33:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.540+0000 7f5d7024ab80 1 -- 172.21.15.40:0/394704527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fd6e0 msgr2=0x56212c5fdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.540+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/394704527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fd6e0 0x56212c5fdb80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5d540ca560 tx=0x7f5d54096320 comp rx=0 tx=0).stop 2023-07-18T03:33:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.540+0000 7f5d7024ab80 1 -- 172.21.15.40:0/394704527 shutdown_connections 2023-07-18T03:33:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.540+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/394704527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fd6e0 0x56212c5fdb80 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.540+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/394704527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 0x56212c5fc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.540+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/394704527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fc840 0x56212c5fd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.540+0000 7f5d7024ab80 1 -- 172.21.15.40:0/394704527 >> 172.21.15.40:0/394704527 conn(0x56212c5f1be0 msgr2=0x56212c5f6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.540+0000 7f5d7024ab80 1 -- 172.21.15.40:0/394704527 shutdown_connections 2023-07-18T03:33:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.540+0000 7f5d7024ab80 1 -- 172.21.15.40:0/394704527 wait complete. 2023-07-18T03:33:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.540+0000 7f5d7024ab80 1 -- start start 2023-07-18T03:33:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.541+0000 7f5d7024ab80 1 Processor -- start 2023-07-18T03:33:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.541+0000 7f5d7024ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 0x56212c6058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.541+0000 7f5d7024ab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fc840 0x56212c605e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.541+0000 7f5d7024ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fd6e0 0x56212c60ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.541+0000 7f5d7024ab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212c4eda60 con 0x56212c5fc840 2023-07-18T03:33:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.541+0000 7f5d7024ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212c4ed760 con 0x56212c5fd6e0 2023-07-18T03:33:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.541+0000 7f5d7024ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212c3d18c0 con 0x56212c4586b0 2023-07-18T03:33:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.541+0000 7f5d7024ab80 10 client.0 _ll_register_callbacks cb 0x56212c4eb700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.541+0000 7f5d6165f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fc840 0x56212c605e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.541+0000 7f5d61e60700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 0x56212c6058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.541+0000 7f5d61e60700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 0x56212c6058d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48696/0 (socket says 172.21.15.40:48696) 2023-07-18T03:33:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.541+0000 7f5d6165f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fc840 0x56212c605e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59670/0 (socket says 172.21.15.40:59670) 2023-07-18T03:33:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.541+0000 7f5d61e60700 1 -- 172.21.15.40:0/3307270576 learned_addr learned my addr 172.21.15.40:0/3307270576 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.541+0000 7f5d62661700 1 --2- 172.21.15.40:0/3307270576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fd6e0 0x56212c60ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.542+0000 7f5d61e60700 1 -- 172.21.15.40:0/3307270576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fd6e0 msgr2=0x56212c60ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.542+0000 7f5d61e60700 1 --2- 172.21.15.40:0/3307270576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fd6e0 0x56212c60ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.542+0000 7f5d61e60700 1 -- 172.21.15.40:0/3307270576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fc840 msgr2=0x56212c605e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.542+0000 7f5d61e60700 1 --2- 172.21.15.40:0/3307270576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fc840 0x56212c605e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.542+0000 7f5d61e60700 1 -- 172.21.15.40:0/3307270576 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c067ae0 con 0x56212c4586b0 2023-07-18T03:33:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.542+0000 7f5d61e60700 1 --2- 172.21.15.40:0/3307270576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 0x56212c6058d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5d58003900 tx=0x7f5d580047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e62c5d8f3cb6f692 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.542+0000 7f5d457fa700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:33:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.542+0000 7f5d457fa700 1 -- 172.21.15.40:0/3307270576 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58022070 con 0x56212c4586b0 2023-07-18T03:33:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.542+0000 7f5d457fa700 1 -- 172.21.15.40:0/3307270576 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5d5801d6e0 con 0x56212c4586b0 2023-07-18T03:33:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.542+0000 7f5d7024ab80 1 -- 172.21.15.40:0/3307270576 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56212c4a7490 con 0x56212c4586b0 2023-07-18T03:33:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.542+0000 7f5d457fa700 1 -- 172.21.15.40:0/3307270576 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d5801d8a0 con 0x56212c4586b0 2023-07-18T03:33:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.542+0000 7f5d7024ab80 10 client.25801 fetch_fsmap learned FSMap version 34 2023-07-18T03:33:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.543+0000 7f5d7024ab80 1 -- 172.21.15.40:0/3307270576 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56212c364c00 con 0x56212c4586b0 2023-07-18T03:33:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.544+0000 7f5d457fa700 1 -- 172.21.15.40:0/3307270576 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56212c4a7490 con 0x56212c4586b0 2023-07-18T03:33:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.544+0000 7f5d457fa700 1 -- 172.21.15.40:0/3307270576 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5d5802f020 con 0x56212c4586b0 2023-07-18T03:33:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.544+0000 7f5d7024ab80 10 client.25801 fetch_fsmap finished waiting for FSMap version 34 2023-07-18T03:33:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.544+0000 7f5d7024ab80 10 client.25801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:33:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.544+0000 7f5d7024ab80 20 client.25801 populate_metadata read hostname 'smithi040' 2023-07-18T03:33:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.544+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/3307270576 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56212c6116e0 0x56212c613af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.544+0000 7f5d7024ab80 4 client.25801 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:33:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.544+0000 7f5d7024ab80 1 -- 172.21.15.40:0/3307270576 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56212c546e50 con 0x56212c6116e0 2023-07-18T03:33:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.544+0000 7f5d6165f700 1 --2- 172.21.15.40:0/3307270576 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56212c6116e0 0x56212c613af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.545+0000 7f5d457fa700 10 client.25801 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.545+0000 7f5d6165f700 1 --2- 172.21.15.40:0/3307270576 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56212c6116e0 0x56212c613af0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=548a00330dff850e server_cookie=e91a02d0b0f94302 in_seq=0 out_seq=0 2023-07-18T03:33:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.545+0000 7f5d457fa700 1 -- 172.21.15.40:0/3307270576 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56212c546e50 con 0x56212c6116e0 2023-07-18T03:33:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.545+0000 7f5d457fa700 10 client.25801 handle_command_reply: tid=0 2023-07-18T03:33:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.545+0000 7f5d7024ab80 1 client.25801 shutdown 2023-07-18T03:33:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.546+0000 7f5d7024ab80 1 -- 172.21.15.40:0/3307270576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 msgr2=0x56212c6058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.546+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/3307270576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 0x56212c6058d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5d58003900 tx=0x7f5d580047c0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.546+0000 7f5d7024ab80 1 -- 172.21.15.40:0/3307270576 shutdown_connections 2023-07-18T03:33:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.546+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/3307270576 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56212c6116e0 0x56212c613af0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.546+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/3307270576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fd6e0 0x56212c60ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.546+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/3307270576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 0x56212c6058d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.546+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/3307270576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fc840 0x56212c605e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.546+0000 7f5d457fa700 0 client.25801 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.546+0000 7f5d7024ab80 1 -- 172.21.15.40:0/3307270576 >> 172.21.15.40:0/3307270576 conn(0x56212c5f1be0 msgr2=0x56212c5f64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.546+0000 7f5d7024ab80 1 -- 172.21.15.40:0/3307270576 shutdown_connections 2023-07-18T03:33:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.546+0000 7f5d7024ab80 1 -- 172.21.15.40:0/3307270576 wait complete. 2023-07-18T03:33:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.546+0000 7f5d7024ab80 20 client.25801 trim_cache size 0 max 16384 2023-07-18T03:33:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.561+0000 7f5d7024ab80 1 Processor -- start 2023-07-18T03:33:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.561+0000 7f5d7024ab80 1 -- start start 2023-07-18T03:33:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.561+0000 7f5d7024ab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c4586b0 0x56212c5fdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.561+0000 7f5d7024ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fc840 0x56212c60ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.561+0000 7f5d7024ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c5fd6e0 0x56212c600630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.561+0000 7f5d7024ab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212c5eec00 con 0x56212c4586b0 2023-07-18T03:33:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.561+0000 7f5d7024ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212c604d80 con 0x56212c5fc840 2023-07-18T03:33:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.561+0000 7f5d7024ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212c605440 con 0x56212c5fd6e0 2023-07-18T03:33:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.561+0000 7f5d61e60700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c4586b0 0x56212c5fdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.561+0000 7f5d61e60700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c4586b0 0x56212c5fdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59676/0 (socket says 172.21.15.40:59676) 2023-07-18T03:33:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.561+0000 7f5d62661700 1 --2- 172.21.15.40:0/1831748967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c5fd6e0 0x56212c600630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.561+0000 7f5d61e60700 1 -- 172.21.15.40:0/1831748967 learned_addr learned my addr 172.21.15.40:0/1831748967 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.561+0000 7f5d6165f700 1 --2- 172.21.15.40:0/1831748967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fc840 0x56212c60ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.561+0000 7f5d61e60700 1 -- 172.21.15.40:0/1831748967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c5fd6e0 msgr2=0x56212c600630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.561+0000 7f5d61e60700 1 --2- 172.21.15.40:0/1831748967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c5fd6e0 0x56212c600630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.561+0000 7f5d61e60700 1 -- 172.21.15.40:0/1831748967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fc840 msgr2=0x56212c60ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.562+0000 7f5d61e60700 1 --2- 172.21.15.40:0/1831748967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fc840 0x56212c60ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.562+0000 7f5d61e60700 1 -- 172.21.15.40:0/1831748967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56212c3a21f0 con 0x56212c4586b0 2023-07-18T03:33:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.562+0000 7f5d61e60700 1 --2- 172.21.15.40:0/1831748967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c4586b0 0x56212c5fdb20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5d58031d10 tx=0x7f5d58011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3a98f4e5396aae server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.562+0000 7f5d50ff9700 1 -- 172.21.15.40:0/1831748967 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58022070 con 0x56212c4586b0 2023-07-18T03:33:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.562+0000 7f5d50ff9700 1 -- 172.21.15.40:0/1831748967 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d580318e0 con 0x56212c4586b0 2023-07-18T03:33:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.562+0000 7f5d50ff9700 1 -- 172.21.15.40:0/1831748967 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5801dba0 con 0x56212c4586b0 2023-07-18T03:33:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.563+0000 7f5d7024ab80 1 -- 172.21.15.40:0/1831748967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c4586b0 msgr2=0x56212c5fdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.563+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/1831748967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c4586b0 0x56212c5fdb20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5d58031d10 tx=0x7f5d58011b70 comp rx=0 tx=0).stop 2023-07-18T03:33:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.563+0000 7f5d7024ab80 1 -- 172.21.15.40:0/1831748967 shutdown_connections 2023-07-18T03:33:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.563+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/1831748967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c5fd6e0 0x56212c600630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.563+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/1831748967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c4586b0 0x56212c5fdb20 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.563+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/1831748967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fc840 0x56212c60ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.563+0000 7f5d7024ab80 1 -- 172.21.15.40:0/1831748967 >> 172.21.15.40:0/1831748967 conn(0x56212c5f1be0 msgr2=0x56212c6045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.563+0000 7f5d7024ab80 1 -- 172.21.15.40:0/1831748967 shutdown_connections 2023-07-18T03:33:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.563+0000 7f5d7024ab80 1 -- 172.21.15.40:0/1831748967 wait complete. 2023-07-18T03:33:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.563+0000 7f5d7024ab80 1 -- start start 2023-07-18T03:33:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.563+0000 7f5d7024ab80 1 Processor -- start 2023-07-18T03:33:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.563+0000 7f5d7024ab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 0x56212c608ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.563+0000 7f5d7024ab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fc840 0x56212c608ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.563+0000 7f5d7024ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fd6e0 0x56212c60c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.563+0000 7f5d7024ab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212c60c550 con 0x56212c5fc840 2023-07-18T03:33:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.563+0000 7f5d7024ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212c60c6c0 con 0x56212c5fd6e0 2023-07-18T03:33:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.563+0000 7f5d7024ab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212c60c880 con 0x56212c4586b0 2023-07-18T03:33:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d7024ab80 10 client.0 _ll_register_callbacks cb 0x7f5d580044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:33:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d6165f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fc840 0x56212c608ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d6165f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fc840 0x56212c608ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59684/0 (socket says 172.21.15.40:59684) 2023-07-18T03:33:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d6165f700 1 -- 172.21.15.40:0/4035645001 learned_addr learned my addr 172.21.15.40:0/4035645001 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:33:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d61e60700 1 --2- 172.21.15.40:0/4035645001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 0x56212c608ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d62661700 1 --2- 172.21.15.40:0/4035645001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fd6e0 0x56212c60c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d6165f700 1 -- 172.21.15.40:0/4035645001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 msgr2=0x56212c608ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d6165f700 1 --2- 172.21.15.40:0/4035645001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 0x56212c608ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d6165f700 1 -- 172.21.15.40:0/4035645001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fd6e0 msgr2=0x56212c60c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:33:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d6165f700 1 --2- 172.21.15.40:0/4035645001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fd6e0 0x56212c60c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d6165f700 1 -- 172.21.15.40:0/4035645001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56212c364c00 con 0x56212c5fc840 2023-07-18T03:33:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d6165f700 1 --2- 172.21.15.40:0/4035645001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fc840 0x56212c608ff0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0095c0 tx=0x7f5d4c02d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe41e2f13668eca server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:33:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d457fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:33:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d457fa700 1 -- 172.21.15.40:0/4035645001 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c01d040 con 0x56212c5fc840 2023-07-18T03:33:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d457fa700 1 -- 172.21.15.40:0/4035645001 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5d4c02db60 con 0x56212c5fc840 2023-07-18T03:33:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d7024ab80 1 -- 172.21.15.40:0/4035645001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d5409f040 con 0x56212c5fc840 2023-07-18T03:33:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d457fa700 1 -- 172.21.15.40:0/4035645001 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d4c02dd20 con 0x56212c5fc840 2023-07-18T03:33:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d7024ab80 10 client.16164 fetch_fsmap learned FSMap version 34 2023-07-18T03:33:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.564+0000 7f5d7024ab80 1 -- 172.21.15.40:0/4035645001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56212c37a250 con 0x56212c5fc840 2023-07-18T03:33:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.565+0000 7f5d457fa700 1 -- 172.21.15.40:0/4035645001 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d4c03c020 con 0x56212c5fc840 2023-07-18T03:33:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.565+0000 7f5d457fa700 1 -- 172.21.15.40:0/4035645001 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 34) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5d4c0228c0 con 0x56212c5fc840 2023-07-18T03:33:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.565+0000 7f5d7024ab80 10 client.16164 fetch_fsmap finished waiting for FSMap version 34 2023-07-18T03:33:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.565+0000 7f5d7024ab80 10 client.16164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:33:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.565+0000 7f5d7024ab80 20 client.16164 populate_metadata read hostname 'smithi040' 2023-07-18T03:33:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.565+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/4035645001 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56212c6663e0 0x56212c6687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:33:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.565+0000 7f5d7024ab80 4 client.16164 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:33:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.565+0000 7f5d7024ab80 1 -- 172.21.15.40:0/4035645001 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56212c508ff0 con 0x56212c6663e0 2023-07-18T03:33:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.565+0000 7f5d61e60700 1 --2- 172.21.15.40:0/4035645001 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56212c6663e0 0x56212c6687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:33:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.565+0000 7f5d457fa700 10 client.16164 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.565+0000 7f5d61e60700 1 --2- 172.21.15.40:0/4035645001 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56212c6663e0 0x56212c6687f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=672628055783d27e server_cookie=9662c64f040a25a3 in_seq=0 out_seq=0 2023-07-18T03:33:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.796+0000 7f5d457fa700 1 -- 172.21.15.40:0/4035645001 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56212c508ff0 con 0x56212c6663e0 2023-07-18T03:33:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.796+0000 7f5d457fa700 10 client.16164 handle_command_reply: tid=0 2023-07-18T03:33:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.796+0000 7f5d7024ab80 1 client.16164 shutdown 2023-07-18T03:33:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.796+0000 7f5d7024ab80 1 -- 172.21.15.40:0/4035645001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fc840 msgr2=0x56212c608ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.796+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/4035645001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fc840 0x56212c608ff0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0095c0 tx=0x7f5d4c02d420 comp rx=0 tx=0).stop 2023-07-18T03:33:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.796+0000 7f5d7024ab80 1 -- 172.21.15.40:0/4035645001 shutdown_connections 2023-07-18T03:33:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.796+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/4035645001 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56212c6663e0 0x56212c6687f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.796+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/4035645001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56212c5fd6e0 0x56212c60c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.796+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/4035645001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56212c4586b0 0x56212c608ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.796+0000 7f5d7024ab80 1 --2- 172.21.15.40:0/4035645001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56212c5fc840 0x56212c608ff0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.796+0000 7f5d7024ab80 1 -- 172.21.15.40:0/4035645001 >> 172.21.15.40:0/4035645001 conn(0x56212c5f1be0 msgr2=0x56212c600630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.797+0000 7f5d457fa700 0 client.16164 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:33:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.797+0000 7f5d7024ab80 1 -- 172.21.15.40:0/4035645001 shutdown_connections 2023-07-18T03:33:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.797+0000 7f5d7024ab80 1 -- 172.21.15.40:0/4035645001 wait complete. 2023-07-18T03:33:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.797+0000 7f5d7024ab80 20 client.16164 trim_cache size 0 max 16384 2023-07-18T03:33:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:33:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:33:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:33:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:33:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.797+0000 7f5d640c4700 1 -- 172.21.15.40:0/2156969678 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5d480a1150 msgr2=0x7f5d480a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.797+0000 7f5d640c4700 1 --2- 172.21.15.40:0/2156969678 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5d480a1150 0x7f5d480a3600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009c60 tx=0x7f5d58009260 comp rx=0 tx=0).stop 2023-07-18T03:33:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.797+0000 7f5d640c4700 1 -- 172.21.15.40:0/2156969678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c100640 msgr2=0x7f5d5c195c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:33:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.797+0000 7f5d640c4700 1 --2- 172.21.15.40:0/2156969678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c100640 0x7f5d5c195c30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400ef30 tx=0x7f5d5400c3a0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.797+0000 7f5d640c4700 1 -- 172.21.15.40:0/2156969678 shutdown_connections 2023-07-18T03:33:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.797+0000 7f5d640c4700 1 --2- 172.21.15.40:0/2156969678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c100640 0x7f5d5c195c30 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.797+0000 7f5d640c4700 1 --2- 172.21.15.40:0/2156969678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d5c0fea00 0x7f5d5c194870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.797+0000 7f5d640c4700 1 --2- 172.21.15.40:0/2156969678 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5d480a1150 0x7f5d480a3600 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.797+0000 7f5d640c4700 1 --2- 172.21.15.40:0/2156969678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d5c0ffc90 0x7f5d5c194db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:33:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.797+0000 7f5d640c4700 1 -- 172.21.15.40:0/2156969678 >> 172.21.15.40:0/2156969678 conn(0x7f5d5c0fa000 msgr2=0x7f5d5c0fc210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:33:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.797+0000 7f5d640c4700 1 -- 172.21.15.40:0/2156969678 shutdown_connections 2023-07-18T03:33:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:33:59.797+0000 7f5d640c4700 1 -- 172.21.15.40:0/2156969678 wait complete. 2023-07-18T03:34:00.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:33:59 smithi040 ceph-mon[96439]: pgmap v386: 97 pgs: 97 active+clean; 569 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 158 op/s 2023-07-18T03:34:00.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:33:59 smithi064 ceph-mon[98059]: pgmap v386: 97 pgs: 97 active+clean; 569 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 158 op/s 2023-07-18T03:34:00.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:33:59 smithi134 ceph-mon[98216]: pgmap v386: 97 pgs: 97 active+clean; 569 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 158 op/s 2023-07-18T03:34:00.601 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:34:02.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.185+0000 7fb153da1700 1 -- 172.21.15.40:0/3362808382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb14c071830 msgr2=0x7fb14c10b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:02.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.185+0000 7fb153da1700 1 --2- 172.21.15.40:0/3362808382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb14c071830 0x7fb14c10b7d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb14400b0a0 tx=0x7fb14400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.186+0000 7fb153da1700 1 -- 172.21.15.40:0/3362808382 shutdown_connections 2023-07-18T03:34:02.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.186+0000 7fb153da1700 1 --2- 172.21.15.40:0/3362808382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14c10bd60 0x7fb14c10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.186+0000 7fb153da1700 1 --2- 172.21.15.40:0/3362808382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb14c070ee0 0x7fb14c0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.186+0000 7fb153da1700 1 --2- 172.21.15.40:0/3362808382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb14c071830 0x7fb14c10b7d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.186+0000 7fb153da1700 1 -- 172.21.15.40:0/3362808382 >> 172.21.15.40:0/3362808382 conn(0x7fb14c06ca90 msgr2=0x7fb14c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:02.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.194+0000 7fb153da1700 1 -- 172.21.15.40:0/3362808382 shutdown_connections 2023-07-18T03:34:02.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.194+0000 7fb153da1700 1 -- 172.21.15.40:0/3362808382 wait complete. 2023-07-18T03:34:02.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.195+0000 7fb153da1700 1 Processor -- start 2023-07-18T03:34:02.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.195+0000 7fb153da1700 1 -- start start 2023-07-18T03:34:02.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.195+0000 7fb153da1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14c070ee0 0x7fb14c113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.195+0000 7fb153da1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb14c071830 0x7fb14c114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.195+0000 7fb153da1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb14c10bd60 0x7fb14c1b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.195+0000 7fb153da1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c1170e0 con 0x7fb14c070ee0 2023-07-18T03:34:02.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.195+0000 7fb153da1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c117250 con 0x7fb14c071830 2023-07-18T03:34:02.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.195+0000 7fb153da1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c1b4ef0 con 0x7fb14c10bd60 2023-07-18T03:34:02.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.196+0000 7fb15233e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb14c10bd60 0x7fb14c1b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:02.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.196+0000 7fb15233e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb14c10bd60 0x7fb14c1b49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48726/0 (socket says 172.21.15.40:48726) 2023-07-18T03:34:02.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.196+0000 7fb15233e700 1 -- 172.21.15.40:0/4163642423 learned_addr learned my addr 172.21.15.40:0/4163642423 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:02.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.196+0000 7fb15133c700 1 --2- 172.21.15.40:0/4163642423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb14c071830 0x7fb14c114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:02.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.196+0000 7fb151b3d700 1 --2- 172.21.15.40:0/4163642423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14c070ee0 0x7fb14c113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:02.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.196+0000 7fb15133c700 1 -- 172.21.15.40:0/4163642423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb14c10bd60 msgr2=0x7fb14c1b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:02.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.196+0000 7fb15133c700 1 --2- 172.21.15.40:0/4163642423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb14c10bd60 0x7fb14c1b49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.196+0000 7fb15133c700 1 -- 172.21.15.40:0/4163642423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14c070ee0 msgr2=0x7fb14c113ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:02.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.196+0000 7fb15133c700 1 --2- 172.21.15.40:0/4163642423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14c070ee0 0x7fb14c113ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.196+0000 7fb15133c700 1 -- 172.21.15.40:0/4163642423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb144009d00 con 0x7fb14c071830 2023-07-18T03:34:02.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.199+0000 7fb15133c700 1 --2- 172.21.15.40:0/4163642423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb14c071830 0x7fb14c114000 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb144007bd0 tx=0x7fb1440096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:02.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.214+0000 7fb142ffd700 1 -- 172.21.15.40:0/4163642423 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14400e040 con 0x7fb14c071830 2023-07-18T03:34:02.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.214+0000 7fb142ffd700 1 -- 172.21.15.40:0/4163642423 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb144012cf0 con 0x7fb14c071830 2023-07-18T03:34:02.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.214+0000 7fb142ffd700 1 -- 172.21.15.40:0/4163642423 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb144005680 con 0x7fb14c071830 2023-07-18T03:34:02.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.214+0000 7fb153da1700 1 -- 172.21.15.40:0/4163642423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14c1b50f0 con 0x7fb14c071830 2023-07-18T03:34:02.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.214+0000 7fb153da1700 1 -- 172.21.15.40:0/4163642423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14c1b5540 con 0x7fb14c071830 2023-07-18T03:34:02.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.221+0000 7fb15ff27b80 1 Processor -- start 2023-07-18T03:34:02.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.222+0000 7fb15ff27b80 1 -- start start 2023-07-18T03:34:02.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.222+0000 7fb15ff27b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a24d0 0x564c1d5a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.222+0000 7fb15ff27b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a2e20 0x564c1d5a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.222+0000 7fb15ff27b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a3c70 0x564c1d5a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.222+0000 7fb15ff27b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d4a7cb0 con 0x564c1d5a2e20 2023-07-18T03:34:02.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.222+0000 7fb15ff27b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d4faa40 con 0x564c1d5a3c70 2023-07-18T03:34:02.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.222+0000 7fb15ff27b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d4f3ed0 con 0x564c1d5a24d0 2023-07-18T03:34:02.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.223+0000 7fb151b3d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a2e20 0x564c1d5a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:02.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.223+0000 7fb15233e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a24d0 0x564c1d5a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:02.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.223+0000 7fb15233e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a24d0 0x564c1d5a28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48730/0 (socket says 172.21.15.40:48730) 2023-07-18T03:34:02.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.223+0000 7fb15233e700 1 -- 172.21.15.40:0/1404251346 learned_addr learned my addr 172.21.15.40:0/1404251346 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:02.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.225+0000 7fb15133c700 1 --2- 172.21.15.40:0/1404251346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a3c70 0x564c1d5a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:02.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.226+0000 7fb15233e700 1 -- 172.21.15.40:0/1404251346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a3c70 msgr2=0x564c1d5a40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:02.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.226+0000 7fb15233e700 1 --2- 172.21.15.40:0/1404251346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a3c70 0x564c1d5a40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.226+0000 7fb15233e700 1 -- 172.21.15.40:0/1404251346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a2e20 msgr2=0x564c1d5a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:02.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.226+0000 7fb15233e700 1 --2- 172.21.15.40:0/1404251346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a2e20 0x564c1d5a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.226+0000 7fb15233e700 1 -- 172.21.15.40:0/1404251346 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c1d321de0 con 0x564c1d5a24d0 2023-07-18T03:34:02.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.226+0000 7fb15233e700 1 --2- 172.21.15.40:0/1404251346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a24d0 0x564c1d5a28e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb13c009b30 tx=0x7fb13c009e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9fb7b3472c14d3b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:02.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.226+0000 7fb142ffd700 1 -- 172.21.15.40:0/4163642423 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb14401b450 con 0x7fb14c071830 2023-07-18T03:34:02.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.227+0000 7fb142ffd700 1 --2- 172.21.15.40:0/4163642423 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb1380a1350 0x7fb1380a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.228+0000 7fb142ffd700 1 -- 172.21.15.40:0/4163642423 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb1440ce7f0 con 0x7fb14c071830 2023-07-18T03:34:02.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.234+0000 7fb151b3d700 1 --2- 172.21.15.40:0/4163642423 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb1380a1350 0x7fb1380a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:02.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.245+0000 7fb137fff700 1 -- 172.21.15.40:0/1404251346 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c009630 con 0x564c1d5a24d0 2023-07-18T03:34:02.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.245+0000 7fb137fff700 1 -- 172.21.15.40:0/1404251346 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb13c00ee50 con 0x564c1d5a24d0 2023-07-18T03:34:02.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.245+0000 7fb137fff700 1 -- 172.21.15.40:0/1404251346 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c0123d0 con 0x564c1d5a24d0 2023-07-18T03:34:02.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.246+0000 7fb15ff27b80 1 -- 172.21.15.40:0/1404251346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a24d0 msgr2=0x564c1d5a28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:02.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.246+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/1404251346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a24d0 0x564c1d5a28e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb13c009b30 tx=0x7fb13c009e40 comp rx=0 tx=0).stop 2023-07-18T03:34:02.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.248+0000 7fb151b3d700 1 --2- 172.21.15.40:0/4163642423 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb1380a1350 0x7fb1380a3800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb14800bad0 tx=0x7fb148008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:02.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.249+0000 7fb15ff27b80 1 -- 172.21.15.40:0/1404251346 shutdown_connections 2023-07-18T03:34:02.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.249+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/1404251346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a3c70 0x564c1d5a40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.249+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/1404251346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a24d0 0x564c1d5a28e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.249+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/1404251346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a2e20 0x564c1d5a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.249+0000 7fb15ff27b80 1 -- 172.21.15.40:0/1404251346 >> 172.21.15.40:0/1404251346 conn(0x564c1d597da0 msgr2=0x564c1d59d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:02.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.249+0000 7fb15ff27b80 1 -- 172.21.15.40:0/1404251346 shutdown_connections 2023-07-18T03:34:02.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.249+0000 7fb15ff27b80 1 -- 172.21.15.40:0/1404251346 wait complete. 2023-07-18T03:34:02.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.250+0000 7fb15ff27b80 1 -- start start 2023-07-18T03:34:02.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.250+0000 7fb15ff27b80 1 Processor -- start 2023-07-18T03:34:02.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.251+0000 7fb15ff27b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a24d0 0x564c1d5ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.251+0000 7fb15ff27b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a2e20 0x564c1d5ac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.251+0000 7fb15ff27b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a3c70 0x564c1d5b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.251+0000 7fb15ff27b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d493ac0 con 0x564c1d5a24d0 2023-07-18T03:34:02.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.251+0000 7fb15ff27b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d4937c0 con 0x564c1d5a2e20 2023-07-18T03:34:02.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.251+0000 7fb15ff27b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d377920 con 0x564c1d5a3c70 2023-07-18T03:34:02.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.251+0000 7fb15ff27b80 10 client.0 _ll_register_callbacks cb 0x564c1d491760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:34:02.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.261+0000 7fb151b3d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a24d0 0x564c1d5ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:02.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.261+0000 7fb151b3d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a24d0 0x564c1d5ac1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59708/0 (socket says 172.21.15.40:59708) 2023-07-18T03:34:02.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.261+0000 7fb151b3d700 1 -- 172.21.15.40:0/2488213238 learned_addr learned my addr 172.21.15.40:0/2488213238 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:02.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.269+0000 7fb15133c700 1 --2- 172.21.15.40:0/2488213238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a2e20 0x564c1d5ac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:02.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.269+0000 7fb15233e700 1 --2- 172.21.15.40:0/2488213238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a3c70 0x564c1d5b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:02.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.270+0000 7fb15133c700 1 -- 172.21.15.40:0/2488213238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a3c70 msgr2=0x564c1d5b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:02.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.270+0000 7fb15133c700 1 --2- 172.21.15.40:0/2488213238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a3c70 0x564c1d5b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.270+0000 7fb15133c700 1 -- 172.21.15.40:0/2488213238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a24d0 msgr2=0x564c1d5ac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:02.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.270+0000 7fb15133c700 1 --2- 172.21.15.40:0/2488213238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a24d0 0x564c1d5ac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.270+0000 7fb15133c700 1 -- 172.21.15.40:0/2488213238 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13c009790 con 0x564c1d5a2e20 2023-07-18T03:34:02.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.270+0000 7fb15133c700 1 --2- 172.21.15.40:0/2488213238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a2e20 0x564c1d5ac6f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb144098680 tx=0x7fb14409b7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e9e35330ef57eeb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:02.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:01 smithi134 ceph-mon[98216]: pgmap v387: 97 pgs: 97 active+clean; 569 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 105 op/s 2023-07-18T03:34:02.336 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:01 smithi040 ceph-mon[96439]: pgmap v387: 97 pgs: 97 active+clean; 569 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 105 op/s 2023-07-18T03:34:02.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.332+0000 7fb1357fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:34:02.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.332+0000 7fb1357fa700 1 -- 172.21.15.40:0/2488213238 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14409fea0 con 0x564c1d5a2e20 2023-07-18T03:34:02.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.333+0000 7fb1357fa700 1 -- 172.21.15.40:0/2488213238 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb14409bc10 con 0x564c1d5a2e20 2023-07-18T03:34:02.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.333+0000 7fb15ff27b80 1 -- 172.21.15.40:0/2488213238 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c1d44d4f0 con 0x564c1d5a2e20 2023-07-18T03:34:02.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.349+0000 7fb1357fa700 1 -- 172.21.15.40:0/2488213238 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb14409fc40 con 0x564c1d5a2e20 2023-07-18T03:34:02.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.349+0000 7fb15ff27b80 10 client.25805 fetch_fsmap learned FSMap version 34 2023-07-18T03:34:02.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.349+0000 7fb15ff27b80 1 -- 172.21.15.40:0/2488213238 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564c1d30ac60 con 0x564c1d5a2e20 2023-07-18T03:34:02.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:01 smithi064 ceph-mon[98059]: pgmap v387: 97 pgs: 97 active+clean; 569 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 105 op/s 2023-07-18T03:34:02.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.360+0000 7fb1357fa700 1 -- 172.21.15.40:0/2488213238 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564c1d44d4f0 con 0x564c1d5a2e20 2023-07-18T03:34:02.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.360+0000 7fb1357fa700 1 -- 172.21.15.40:0/2488213238 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fb144003c00 con 0x564c1d5a2e20 2023-07-18T03:34:02.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.361+0000 7fb15ff27b80 10 client.25805 fetch_fsmap finished waiting for FSMap version 34 2023-07-18T03:34:02.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.361+0000 7fb15ff27b80 10 client.25805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:34:02.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.361+0000 7fb15ff27b80 20 client.25805 populate_metadata read hostname 'smithi040' 2023-07-18T03:34:02.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.361+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/2488213238 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564c1d5b7dc0 0x564c1d5ba1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.361+0000 7fb15ff27b80 4 client.25805 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:34:02.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.361+0000 7fb15ff27b80 1 -- 172.21.15.40:0/2488213238 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c1d4eceb0 con 0x564c1d5b7dc0 2023-07-18T03:34:02.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.364+0000 7fb151b3d700 1 --2- 172.21.15.40:0/2488213238 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564c1d5b7dc0 0x564c1d5ba1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:02.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.418+0000 7fb151b3d700 1 --2- 172.21.15.40:0/2488213238 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564c1d5b7dc0 0x564c1d5ba1d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0cdb3fb5e938257 server_cookie=c308c190db983dd0 in_seq=0 out_seq=0 2023-07-18T03:34:02.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.429+0000 7fb1357fa700 10 client.25805 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:02.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.429+0000 7fb1357fa700 1 -- 172.21.15.40:0/2488213238 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564c1d4eceb0 con 0x564c1d5b7dc0 2023-07-18T03:34:02.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.429+0000 7fb1357fa700 10 client.25805 handle_command_reply: tid=0 2023-07-18T03:34:02.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.429+0000 7fb15ff27b80 1 client.25805 shutdown 2023-07-18T03:34:02.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.431+0000 7fb15ff27b80 1 -- 172.21.15.40:0/2488213238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a2e20 msgr2=0x564c1d5ac6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:02.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.431+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/2488213238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a2e20 0x564c1d5ac6f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb144098680 tx=0x7fb14409b7d0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.431+0000 7fb15ff27b80 1 -- 172.21.15.40:0/2488213238 shutdown_connections 2023-07-18T03:34:02.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.431+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/2488213238 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564c1d5b7dc0 0x564c1d5ba1d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.431+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/2488213238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a3c70 0x564c1d5b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.431+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/2488213238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a24d0 0x564c1d5ac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.431+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/2488213238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a2e20 0x564c1d5ac6f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.431+0000 7fb15ff27b80 1 -- 172.21.15.40:0/2488213238 >> 172.21.15.40:0/2488213238 conn(0x564c1d597da0 msgr2=0x564c1d59c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:02.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.432+0000 7fb1357fa700 0 client.25805 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:02.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.436+0000 7fb15ff27b80 1 -- 172.21.15.40:0/2488213238 shutdown_connections 2023-07-18T03:34:02.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.440+0000 7fb15ff27b80 1 -- 172.21.15.40:0/2488213238 wait complete. 2023-07-18T03:34:02.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.441+0000 7fb15ff27b80 20 client.25805 trim_cache size 0 max 16384 2023-07-18T03:34:02.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.514+0000 7fb15ff27b80 1 Processor -- start 2023-07-18T03:34:02.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.523+0000 7fb15ff27b80 1 -- start start 2023-07-18T03:34:02.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.523+0000 7fb15ff27b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a24d0 0x564c1d59c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.523+0000 7fb15ff27b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a2e20 0x564c1d5b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.524+0000 7fb15ff27b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a3c70 0x564c1d5b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.524+0000 7fb15ff27b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d5ab4f0 con 0x564c1d5a2e20 2023-07-18T03:34:02.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.524+0000 7fb15ff27b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d59ed60 con 0x564c1d5a24d0 2023-07-18T03:34:02.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.524+0000 7fb15ff27b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d4eceb0 con 0x564c1d5a3c70 2023-07-18T03:34:02.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.524+0000 7fb15233e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a3c70 0x564c1d5b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:02.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.524+0000 7fb15233e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a3c70 0x564c1d5b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48746/0 (socket says 172.21.15.40:48746) 2023-07-18T03:34:02.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.524+0000 7fb15233e700 1 -- 172.21.15.40:0/4101993892 learned_addr learned my addr 172.21.15.40:0/4101993892 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:02.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.527+0000 7fb15233e700 1 -- 172.21.15.40:0/4101993892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a24d0 msgr2=0x564c1d59c8b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:34:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.527+0000 7fb15233e700 1 --2- 172.21.15.40:0/4101993892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a24d0 0x564c1d59c8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.527+0000 7fb15233e700 1 -- 172.21.15.40:0/4101993892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a2e20 msgr2=0x564c1d5b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.527+0000 7fb15233e700 1 --2- 172.21.15.40:0/4101993892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a2e20 0x564c1d5b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.527+0000 7fb15233e700 1 -- 172.21.15.40:0/4101993892 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c1d348250 con 0x564c1d5a3c70 2023-07-18T03:34:02.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.533+0000 7fb15233e700 1 --2- 172.21.15.40:0/4101993892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a3c70 0x564c1d5b3f60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb13c009b30 tx=0x7fb13c015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b5536ee81be4a21 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:02.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.572+0000 7fb140ff9700 1 -- 172.21.15.40:0/4101993892 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c028070 con 0x564c1d5a3c70 2023-07-18T03:34:02.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.572+0000 7fb140ff9700 1 -- 172.21.15.40:0/4101993892 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb13c005a50 con 0x564c1d5a3c70 2023-07-18T03:34:02.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.572+0000 7fb140ff9700 1 -- 172.21.15.40:0/4101993892 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c013ad0 con 0x564c1d5a3c70 2023-07-18T03:34:02.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.578+0000 7fb15ff27b80 1 -- 172.21.15.40:0/4101993892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a3c70 msgr2=0x564c1d5b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:02.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.578+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/4101993892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a3c70 0x564c1d5b3f60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb13c009b30 tx=0x7fb13c015040 comp rx=0 tx=0).stop 2023-07-18T03:34:02.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.579+0000 7fb15ff27b80 1 -- 172.21.15.40:0/4101993892 shutdown_connections 2023-07-18T03:34:02.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.579+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/4101993892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a3c70 0x564c1d5b3f60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.579+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/4101993892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a24d0 0x564c1d59c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.579+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/4101993892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a2e20 0x564c1d5b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.579+0000 7fb15ff27b80 1 -- 172.21.15.40:0/4101993892 >> 172.21.15.40:0/4101993892 conn(0x564c1d597da0 msgr2=0x564c1d5aade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:02.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.579+0000 7fb15ff27b80 1 -- 172.21.15.40:0/4101993892 shutdown_connections 2023-07-18T03:34:02.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.579+0000 7fb15ff27b80 1 -- 172.21.15.40:0/4101993892 wait complete. 2023-07-18T03:34:02.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.580+0000 7fb15ff27b80 1 -- start start 2023-07-18T03:34:02.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.580+0000 7fb15ff27b80 1 Processor -- start 2023-07-18T03:34:02.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.581+0000 7fb15ff27b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a24d0 0x564c1d5b1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.581+0000 7fb15ff27b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a2e20 0x564c1d5b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.581+0000 7fb15ff27b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a3c70 0x564c1d5b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.581+0000 7fb15ff27b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d5b2e10 con 0x564c1d5a2e20 2023-07-18T03:34:02.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.581+0000 7fb15ff27b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d5b2f80 con 0x564c1d5a3c70 2023-07-18T03:34:02.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.581+0000 7fb15ff27b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d5b3140 con 0x564c1d5a24d0 2023-07-18T03:34:02.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.581+0000 7fb15ff27b80 10 client.0 _ll_register_callbacks cb 0x564c1d491760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:34:02.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.589+0000 7fb151b3d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a24d0 0x564c1d5b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:02.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.589+0000 7fb151b3d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a24d0 0x564c1d5b1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48750/0 (socket says 172.21.15.40:48750) 2023-07-18T03:34:02.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.589+0000 7fb151b3d700 1 -- 172.21.15.40:0/2414051488 learned_addr learned my addr 172.21.15.40:0/2414051488 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:02.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.589+0000 7fb15133c700 1 --2- 172.21.15.40:0/2414051488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a2e20 0x564c1d5b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:02.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.589+0000 7fb15233e700 1 --2- 172.21.15.40:0/2414051488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a3c70 0x564c1d5b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:02.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.589+0000 7fb151b3d700 1 -- 172.21.15.40:0/2414051488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a3c70 msgr2=0x564c1d5b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:02.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.589+0000 7fb151b3d700 1 --2- 172.21.15.40:0/2414051488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a3c70 0x564c1d5b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.589+0000 7fb151b3d700 1 -- 172.21.15.40:0/2414051488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a2e20 msgr2=0x564c1d5b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:02.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.589+0000 7fb151b3d700 1 --2- 172.21.15.40:0/2414051488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a2e20 0x564c1d5b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.589+0000 7fb151b3d700 1 -- 172.21.15.40:0/2414051488 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c1d321de0 con 0x564c1d5a24d0 2023-07-18T03:34:02.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.590+0000 7fb151b3d700 1 --2- 172.21.15.40:0/2414051488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a24d0 0x564c1d5b1380 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x564c1d5baa70 tx=0x7fb148004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5cab9bb29a7184e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:02.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.594+0000 7fb1357fa700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:34:02.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.594+0000 7fb1357fa700 1 -- 172.21.15.40:0/2414051488 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb148005e30 con 0x564c1d5a24d0 2023-07-18T03:34:02.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.594+0000 7fb1357fa700 1 -- 172.21.15.40:0/2414051488 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb1480147c0 con 0x564c1d5a24d0 2023-07-18T03:34:02.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.608+0000 7fb15ff27b80 1 -- 172.21.15.40:0/2414051488 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb13c00ee50 con 0x564c1d5a24d0 2023-07-18T03:34:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.617+0000 7fb1357fa700 1 -- 172.21.15.40:0/2414051488 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb148014980 con 0x564c1d5a24d0 2023-07-18T03:34:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.629+0000 7fb15ff27b80 10 client.25825 fetch_fsmap learned FSMap version 34 2023-07-18T03:34:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.629+0000 7fb15ff27b80 1 -- 172.21.15.40:0/2414051488 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564c1d3202b0 con 0x564c1d5a24d0 2023-07-18T03:34:02.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.642+0000 7fb1357fa700 1 -- 172.21.15.40:0/2414051488 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb13c00ee50 con 0x564c1d5a24d0 2023-07-18T03:34:02.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.642+0000 7fb1357fa700 1 -- 172.21.15.40:0/2414051488 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fb148022430 con 0x564c1d5a24d0 2023-07-18T03:34:02.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.652+0000 7fb15ff27b80 10 client.25825 fetch_fsmap finished waiting for FSMap version 34 2023-07-18T03:34:02.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.652+0000 7fb15ff27b80 10 client.25825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:34:02.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.653+0000 7fb15ff27b80 20 client.25825 populate_metadata read hostname 'smithi040' 2023-07-18T03:34:02.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.653+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/2414051488 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564c1d60cc20 0x564c1d60f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:02.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.653+0000 7fb15ff27b80 4 client.25825 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:34:02.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.653+0000 7fb15ff27b80 1 -- 172.21.15.40:0/2414051488 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564c1d4af050 con 0x564c1d60cc20 2023-07-18T03:34:02.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.662+0000 7fb15133c700 1 --2- 172.21.15.40:0/2414051488 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564c1d60cc20 0x564c1d60f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.688+0000 7fb15133c700 1 --2- 172.21.15.40:0/2414051488 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564c1d60cc20 0x564c1d60f030 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47b2622481ca5c31 server_cookie=b9b978c8986e4f27 in_seq=0 out_seq=0 2023-07-18T03:34:02.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.688+0000 7fb1357fa700 10 client.25825 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:02.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.699+0000 7fb1357fa700 1 -- 172.21.15.40:0/2414051488 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x564c1d4af050 con 0x564c1d60cc20 2023-07-18T03:34:02.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.699+0000 7fb1357fa700 10 client.25825 handle_command_reply: tid=0 2023-07-18T03:34:02.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.699+0000 7fb15ff27b80 1 client.25825 shutdown 2023-07-18T03:34:02.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.699+0000 7fb15ff27b80 1 -- 172.21.15.40:0/2414051488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a24d0 msgr2=0x564c1d5b1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:02.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.699+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/2414051488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a24d0 0x564c1d5b1380 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x564c1d5baa70 tx=0x7fb148004790 comp rx=0 tx=0).stop 2023-07-18T03:34:02.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.700+0000 7fb15ff27b80 1 -- 172.21.15.40:0/2414051488 shutdown_connections 2023-07-18T03:34:02.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.700+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/2414051488 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564c1d60cc20 0x564c1d60f030 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.700+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/2414051488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564c1d5a3c70 0x564c1d5b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.700+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/2414051488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564c1d5a24d0 0x564c1d5b1380 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.700+0000 7fb15ff27b80 1 --2- 172.21.15.40:0/2414051488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564c1d5a2e20 0x564c1d5b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.700+0000 7fb15ff27b80 1 -- 172.21.15.40:0/2414051488 >> 172.21.15.40:0/2414051488 conn(0x564c1d597da0 msgr2=0x564c1d59c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:02.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.700+0000 7fb1357fa700 0 client.25825 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:02.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.700+0000 7fb15ff27b80 1 -- 172.21.15.40:0/2414051488 shutdown_connections 2023-07-18T03:34:02.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.700+0000 7fb15ff27b80 1 -- 172.21.15.40:0/2414051488 wait complete. 2023-07-18T03:34:02.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.701+0000 7fb15ff27b80 20 client.25825 trim_cache size 0 max 16384 2023-07-18T03:34:02.700 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:34:02.701 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-18T03:34:02.701 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:34:02.701 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:34:02.701 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:34:02.701 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:34:02.701 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:34:02.702 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:34:02.702 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:34:02.702 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:34:02.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.725+0000 7fb153da1700 1 -- 172.21.15.40:0/4163642423 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb1380a1350 msgr2=0x7fb1380a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:02.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.725+0000 7fb153da1700 1 --2- 172.21.15.40:0/4163642423 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb1380a1350 0x7fb1380a3800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb14800bad0 tx=0x7fb148008040 comp rx=0 tx=0).stop 2023-07-18T03:34:02.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.725+0000 7fb153da1700 1 -- 172.21.15.40:0/4163642423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb14c071830 msgr2=0x7fb14c114000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:02.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.725+0000 7fb153da1700 1 --2- 172.21.15.40:0/4163642423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb14c071830 0x7fb14c114000 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb144007bd0 tx=0x7fb1440096e0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.725+0000 7fb153da1700 1 -- 172.21.15.40:0/4163642423 shutdown_connections 2023-07-18T03:34:02.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.725+0000 7fb153da1700 1 --2- 172.21.15.40:0/4163642423 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb1380a1350 0x7fb1380a3800 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.725+0000 7fb153da1700 1 --2- 172.21.15.40:0/4163642423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb14c10bd60 0x7fb14c1b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.725+0000 7fb153da1700 1 --2- 172.21.15.40:0/4163642423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb14c070ee0 0x7fb14c113ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.725+0000 7fb153da1700 1 --2- 172.21.15.40:0/4163642423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb14c071830 0x7fb14c114000 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:02.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.725+0000 7fb153da1700 1 -- 172.21.15.40:0/4163642423 >> 172.21.15.40:0/4163642423 conn(0x7fb14c06ca90 msgr2=0x7fb14c06ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:02.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.726+0000 7fb153da1700 1 -- 172.21.15.40:0/4163642423 shutdown_connections 2023-07-18T03:34:02.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:02.726+0000 7fb153da1700 1 -- 172.21.15.40:0/4163642423 wait complete. 2023-07-18T03:34:02.765 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:34:03.159 INFO:tasks.workunit.client.0.smithi040.stdout: 173 817562 807 600673 983 111073 194 2023-07-18T03:34:04.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:03 smithi134 ceph-mon[98216]: pgmap v388: 97 pgs: 97 active+clean; 565 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 202 op/s 2023-07-18T03:34:04.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:03 smithi040 ceph-mon[96439]: pgmap v388: 97 pgs: 97 active+clean; 565 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 202 op/s 2023-07-18T03:34:04.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:03 smithi064 ceph-mon[98059]: pgmap v388: 97 pgs: 97 active+clean; 565 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 202 op/s 2023-07-18T03:34:06.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:05 smithi134 ceph-mon[98216]: pgmap v389: 97 pgs: 97 active+clean; 566 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2023-07-18T03:34:06.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:05 smithi040 ceph-mon[96439]: pgmap v389: 97 pgs: 97 active+clean; 566 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2023-07-18T03:34:06.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:05 smithi064 ceph-mon[98059]: pgmap v389: 97 pgs: 97 active+clean; 566 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2023-07-18T03:34:07.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:06 smithi134 ceph-mon[98216]: pgmap v390: 97 pgs: 97 active+clean; 571 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2023-07-18T03:34:07.343 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:06 smithi040 ceph-mon[96439]: pgmap v390: 97 pgs: 97 active+clean; 571 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2023-07-18T03:34:07.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:06 smithi064 ceph-mon[98059]: pgmap v390: 97 pgs: 97 active+clean; 571 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 191 op/s 2023-07-18T03:34:08.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:08 smithi134 ceph-mon[98216]: pgmap v391: 97 pgs: 97 active+clean; 622 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 364 op/s 2023-07-18T03:34:08.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:08 smithi040 ceph-mon[96439]: pgmap v391: 97 pgs: 97 active+clean; 622 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 364 op/s 2023-07-18T03:34:08.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:08 smithi064 ceph-mon[98059]: pgmap v391: 97 pgs: 97 active+clean; 622 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 364 op/s 2023-07-18T03:34:10.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:10 smithi134 ceph-mon[98216]: pgmap v392: 97 pgs: 97 active+clean; 629 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 5.6 MiB/s wr, 358 op/s 2023-07-18T03:34:10.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:10 smithi040 ceph-mon[96439]: pgmap v392: 97 pgs: 97 active+clean; 629 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 5.6 MiB/s wr, 358 op/s 2023-07-18T03:34:10.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:10 smithi064 ceph-mon[98059]: pgmap v392: 97 pgs: 97 active+clean; 629 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 5.6 MiB/s wr, 358 op/s 2023-07-18T03:34:13.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:12 smithi134 ceph-mon[98216]: pgmap v393: 97 pgs: 97 active+clean; 685 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 557 op/s 2023-07-18T03:34:13.087 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:12 smithi040 ceph-mon[96439]: pgmap v393: 97 pgs: 97 active+clean; 685 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 557 op/s 2023-07-18T03:34:13.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:12 smithi064 ceph-mon[98059]: pgmap v393: 97 pgs: 97 active+clean; 685 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 557 op/s 2023-07-18T03:34:13.157 INFO:tasks.workunit.client.0.smithi040.stdout: 184 2360154 588 1708853 447 368648 433 2023-07-18T03:34:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.855+0000 7f181a4c6700 1 -- 172.21.15.40:0/2344670567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1814071850 msgr2=0x7f181410b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.855+0000 7f181a4c6700 1 --2- 172.21.15.40:0/2344670567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1814071850 0x7f181410b7d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1804009990 tx=0x7f1804009ca0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.856+0000 7f181a4c6700 1 -- 172.21.15.40:0/2344670567 shutdown_connections 2023-07-18T03:34:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.856+0000 7f181a4c6700 1 --2- 172.21.15.40:0/2344670567 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f181410bd60 0x7f181410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.856+0000 7f181a4c6700 1 --2- 172.21.15.40:0/2344670567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1814070f00 0x7f1814071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.856+0000 7f181a4c6700 1 --2- 172.21.15.40:0/2344670567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1814071850 0x7f181410b7d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.856+0000 7f181a4c6700 1 -- 172.21.15.40:0/2344670567 >> 172.21.15.40:0/2344670567 conn(0x7f181406ca50 msgr2=0x7f181406eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.860+0000 7f181a4c6700 1 -- 172.21.15.40:0/2344670567 shutdown_connections 2023-07-18T03:34:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.865+0000 7f181a4c6700 1 -- 172.21.15.40:0/2344670567 wait complete. 2023-07-18T03:34:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.866+0000 7f181a4c6700 1 Processor -- start 2023-07-18T03:34:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.867+0000 7f181a4c6700 1 -- start start 2023-07-18T03:34:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.867+0000 7f181a4c6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1814070f00 0x7f1814113c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.867+0000 7f181a4c6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1814071850 0x7f1814114140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.867+0000 7f181a4c6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f181410bd60 0x7f1814115010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.867+0000 7f181a4c6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1814115670 con 0x7f1814071850 2023-07-18T03:34:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.867+0000 7f181a4c6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18141157e0 con 0x7f181410bd60 2023-07-18T03:34:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.867+0000 7f181a4c6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18141159a0 con 0x7f1814070f00 2023-07-18T03:34:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.870+0000 7f18137fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1814071850 0x7f1814114140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.870+0000 7f18137fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1814071850 0x7f1814114140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59736/0 (socket says 172.21.15.40:59736) 2023-07-18T03:34:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.870+0000 7f18137fe700 1 -- 172.21.15.40:0/1520494876 learned_addr learned my addr 172.21.15.40:0/1520494876 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.871+0000 7f18137fe700 1 -- 172.21.15.40:0/1520494876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1814070f00 msgr2=0x7f1814113c00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:34:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.871+0000 7f18137fe700 1 --2- 172.21.15.40:0/1520494876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1814070f00 0x7f1814113c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.871+0000 7f18137fe700 1 -- 172.21.15.40:0/1520494876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f181410bd60 msgr2=0x7f1814115010 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:34:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.871+0000 7f18137fe700 1 --2- 172.21.15.40:0/1520494876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f181410bd60 0x7f1814115010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.871+0000 7f18137fe700 1 -- 172.21.15.40:0/1520494876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18040095f0 con 0x7f1814071850 2023-07-18T03:34:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.871+0000 7f18137fe700 1 --2- 172.21.15.40:0/1520494876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1814071850 0x7f1814114140 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f18040036f0 tx=0x7f1804003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.872+0000 7f18117fa700 1 -- 172.21.15.40:0/1520494876 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180401d070 con 0x7f1814071850 2023-07-18T03:34:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.872+0000 7f18117fa700 1 -- 172.21.15.40:0/1520494876 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f180400f460 con 0x7f1814071850 2023-07-18T03:34:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.872+0000 7f18117fa700 1 -- 172.21.15.40:0/1520494876 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804021670 con 0x7f1814071850 2023-07-18T03:34:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.872+0000 7f181a4c6700 1 -- 172.21.15.40:0/1520494876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1814115c20 con 0x7f1814071850 2023-07-18T03:34:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.872+0000 7f181a4c6700 1 -- 172.21.15.40:0/1520494876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1814116430 con 0x7f1814071850 2023-07-18T03:34:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.878+0000 7f182664cb80 1 Processor -- start 2023-07-18T03:34:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.878+0000 7f182664cb80 1 -- start start 2023-07-18T03:34:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.878+0000 7f182664cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d01386b0 0x55d2d02dc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.878+0000 7f182664cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d02dc840 0x55d2d02dd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.878+0000 7f182664cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d02dd6e0 0x55d2d02ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.878+0000 7f182664cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d01e1c50 con 0x55d2d01386b0 2023-07-18T03:34:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.878+0000 7f182664cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d02349e0 con 0x55d2d02dd6e0 2023-07-18T03:34:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.878+0000 7f182664cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d022de70 con 0x55d2d02dc840 2023-07-18T03:34:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.883+0000 7f18137fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d02dd6e0 0x55d2d02ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.884+0000 7f18137fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d02dd6e0 0x55d2d02ddb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43068/0 (socket says 172.21.15.40:43068) 2023-07-18T03:34:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.884+0000 7f18137fe700 1 -- 172.21.15.40:0/1824745018 learned_addr learned my addr 172.21.15.40:0/1824745018 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.886+0000 7f1818a63700 1 --2- 172.21.15.40:0/1824745018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d01386b0 0x55d2d02dc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.888+0000 7f18137fe700 1 -- 172.21.15.40:0/1824745018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d02dc840 msgr2=0x55d2d02dd1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:34:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.888+0000 7f18137fe700 1 --2- 172.21.15.40:0/1824745018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d02dc840 0x55d2d02dd1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.888+0000 7f18137fe700 1 -- 172.21.15.40:0/1824745018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d01386b0 msgr2=0x55d2d02dc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.888+0000 7f18137fe700 1 --2- 172.21.15.40:0/1824745018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d01386b0 0x55d2d02dc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.888+0000 7f18137fe700 1 -- 172.21.15.40:0/1824745018 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2d005bd80 con 0x55d2d02dd6e0 2023-07-18T03:34:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.888+0000 7f18117fa700 1 -- 172.21.15.40:0/1520494876 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1804021930 con 0x7f1814071850 2023-07-18T03:34:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.889+0000 7f18117fa700 1 --2- 172.21.15.40:0/1520494876 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f17fc0a1320 0x7f17fc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.890+0000 7f18117fa700 1 -- 172.21.15.40:0/1520494876 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f18040d5a30 con 0x7f1814071850 2023-07-18T03:34:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.890+0000 7f18137fe700 1 --2- 172.21.15.40:0/1824745018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d02dd6e0 0x55d2d02ddb80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f18040d3450 tx=0x7f18040d6290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9114ce57959ccc27 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.890+0000 7f1818a63700 1 --2- 172.21.15.40:0/1520494876 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f17fc0a1320 0x7f17fc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.891+0000 7f17fa7fc700 1 -- 172.21.15.40:0/1824745018 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180409f070 con 0x55d2d02dd6e0 2023-07-18T03:34:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.891+0000 7f17fa7fc700 1 -- 172.21.15.40:0/1824745018 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18040a2b90 con 0x55d2d02dd6e0 2023-07-18T03:34:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.891+0000 7f17fa7fc700 1 -- 172.21.15.40:0/1824745018 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18040a6e60 con 0x55d2d02dd6e0 2023-07-18T03:34:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.892+0000 7f182664cb80 1 -- 172.21.15.40:0/1824745018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d02dd6e0 msgr2=0x55d2d02ddb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.892+0000 7f182664cb80 1 --2- 172.21.15.40:0/1824745018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d02dd6e0 0x55d2d02ddb80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f18040d3450 tx=0x7f18040d6290 comp rx=0 tx=0).stop 2023-07-18T03:34:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.892+0000 7f1818a63700 1 --2- 172.21.15.40:0/1520494876 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f17fc0a1320 0x7f17fc0a37d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55d2d02dc060 tx=0x7f180800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.893+0000 7f182664cb80 1 -- 172.21.15.40:0/1824745018 shutdown_connections 2023-07-18T03:34:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.893+0000 7f182664cb80 1 --2- 172.21.15.40:0/1824745018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d02dd6e0 0x55d2d02ddb80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.893+0000 7f182664cb80 1 --2- 172.21.15.40:0/1824745018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d01386b0 0x55d2d02dc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.893+0000 7f182664cb80 1 --2- 172.21.15.40:0/1824745018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d02dc840 0x55d2d02dd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.893+0000 7f182664cb80 1 -- 172.21.15.40:0/1824745018 >> 172.21.15.40:0/1824745018 conn(0x55d2d02d1be0 msgr2=0x55d2d02d6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.902+0000 7f182664cb80 1 -- 172.21.15.40:0/1824745018 shutdown_connections 2023-07-18T03:34:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.902+0000 7f182664cb80 1 -- 172.21.15.40:0/1824745018 wait complete. 2023-07-18T03:34:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.903+0000 7f182664cb80 1 -- start start 2023-07-18T03:34:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.903+0000 7f182664cb80 1 Processor -- start 2023-07-18T03:34:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.903+0000 7f182664cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d01386b0 0x55d2d03075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.903+0000 7f182664cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d02dd6e0 0x55d2d0307b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.903+0000 7f182664cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 0x55d2d030ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.903+0000 7f182664cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d01cda60 con 0x55d2d02dd6e0 2023-07-18T03:34:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.903+0000 7f182664cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d01cd760 con 0x55d2d030cb50 2023-07-18T03:34:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.903+0000 7f182664cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d00b18c0 con 0x55d2d01386b0 2023-07-18T03:34:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.903+0000 7f182664cb80 10 client.0 _ll_register_callbacks cb 0x55d2d01cb700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:34:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.905+0000 7f1818a63700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 0x55d2d030ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.905+0000 7f1818a63700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 0x55d2d030ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43076/0 (socket says 172.21.15.40:43076) 2023-07-18T03:34:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.905+0000 7f1818a63700 1 -- 172.21.15.40:0/1919435119 learned_addr learned my addr 172.21.15.40:0/1919435119 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.905+0000 7f18137fe700 1 --2- 172.21.15.40:0/1919435119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d02dd6e0 0x55d2d0307b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.905+0000 7f1818a63700 1 -- 172.21.15.40:0/1919435119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d01386b0 msgr2=0x55d2d03075e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:34:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.905+0000 7f1818a63700 1 --2- 172.21.15.40:0/1919435119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d01386b0 0x55d2d03075e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.905+0000 7f1818a63700 1 -- 172.21.15.40:0/1919435119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d02dd6e0 msgr2=0x55d2d0307b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.905+0000 7f1818a63700 1 --2- 172.21.15.40:0/1919435119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d02dd6e0 0x55d2d0307b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.905+0000 7f1818a63700 1 -- 172.21.15.40:0/1919435119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1814116430 con 0x55d2d030cb50 2023-07-18T03:34:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.905+0000 7f18137fe700 1 --2- 172.21.15.40:0/1919435119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d02dd6e0 0x55d2d0307b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:34:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.910+0000 7f1818a63700 1 --2- 172.21.15.40:0/1919435119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 0x55d2d030ef30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1808016bd0 tx=0x7f18080049f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ccd26594def4ec1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.914+0000 7f17f7ff7700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:34:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.919+0000 7f17f7ff7700 1 -- 172.21.15.40:0/1919435119 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1808010040 con 0x55d2d030cb50 2023-07-18T03:34:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.919+0000 7f17f7ff7700 1 -- 172.21.15.40:0/1919435119 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1808015b90 con 0x55d2d030cb50 2023-07-18T03:34:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.919+0000 7f182664cb80 1 -- 172.21.15.40:0/1919435119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2d0187490 con 0x55d2d030cb50 2023-07-18T03:34:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.919+0000 7f17f7ff7700 1 -- 172.21.15.40:0/1919435119 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1808015d50 con 0x55d2d030cb50 2023-07-18T03:34:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.920+0000 7f182664cb80 10 client.25814 fetch_fsmap learned FSMap version 34 2023-07-18T03:34:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.920+0000 7f182664cb80 1 -- 172.21.15.40:0/1919435119 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d2d0044c00 con 0x55d2d030cb50 2023-07-18T03:34:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.920+0000 7f17f7ff7700 1 -- 172.21.15.40:0/1919435119 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f180802f020 con 0x55d2d030cb50 2023-07-18T03:34:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.921+0000 7f17f7ff7700 1 -- 172.21.15.40:0/1919435119 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f180801f070 con 0x55d2d030cb50 2023-07-18T03:34:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.922+0000 7f182664cb80 10 client.25814 fetch_fsmap finished waiting for FSMap version 34 2023-07-18T03:34:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.922+0000 7f182664cb80 10 client.25814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:34:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.922+0000 7f182664cb80 20 client.25814 populate_metadata read hostname 'smithi040' 2023-07-18T03:34:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.922+0000 7f182664cb80 1 --2- 172.21.15.40:0/1919435119 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d2d03138d0 0x55d2d0315ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.922+0000 7f182664cb80 4 client.25814 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:34:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.922+0000 7f182664cb80 1 -- 172.21.15.40:0/1919435119 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2d0226e50 con 0x55d2d03138d0 2023-07-18T03:34:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.922+0000 7f1813fff700 1 --2- 172.21.15.40:0/1919435119 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d2d03138d0 0x55d2d0315ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.923+0000 7f17f7ff7700 10 client.25814 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.923+0000 7f1813fff700 1 --2- 172.21.15.40:0/1919435119 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d2d03138d0 0x55d2d0315ce0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f8a8f1245459deb server_cookie=853a9db301a682f2 in_seq=0 out_seq=0 2023-07-18T03:34:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.925+0000 7f17f7ff7700 1 -- 172.21.15.40:0/1919435119 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d2d0226e50 con 0x55d2d03138d0 2023-07-18T03:34:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.925+0000 7f17f7ff7700 10 client.25814 handle_command_reply: tid=0 2023-07-18T03:34:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.925+0000 7f182664cb80 1 client.25814 shutdown 2023-07-18T03:34:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.935+0000 7f182664cb80 1 -- 172.21.15.40:0/1919435119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 msgr2=0x55d2d030ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.935+0000 7f182664cb80 1 --2- 172.21.15.40:0/1919435119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 0x55d2d030ef30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1808016bd0 tx=0x7f18080049f0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.936+0000 7f182664cb80 1 -- 172.21.15.40:0/1919435119 shutdown_connections 2023-07-18T03:34:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.936+0000 7f182664cb80 1 --2- 172.21.15.40:0/1919435119 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d2d03138d0 0x55d2d0315ce0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.936+0000 7f182664cb80 1 --2- 172.21.15.40:0/1919435119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 0x55d2d030ef30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.936+0000 7f182664cb80 1 --2- 172.21.15.40:0/1919435119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d01386b0 0x55d2d03075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.936+0000 7f17f7ff7700 0 client.25814 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.936+0000 7f182664cb80 1 --2- 172.21.15.40:0/1919435119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d02dd6e0 0x55d2d0307b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.936+0000 7f182664cb80 1 -- 172.21.15.40:0/1919435119 >> 172.21.15.40:0/1919435119 conn(0x55d2d02d1be0 msgr2=0x55d2d02d64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.936+0000 7f182664cb80 1 -- 172.21.15.40:0/1919435119 shutdown_connections 2023-07-18T03:34:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.936+0000 7f182664cb80 1 -- 172.21.15.40:0/1919435119 wait complete. 2023-07-18T03:34:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.936+0000 7f182664cb80 20 client.25814 trim_cache size 0 max 16384 2023-07-18T03:34:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.967+0000 7f182664cb80 1 Processor -- start 2023-07-18T03:34:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.968+0000 7f182664cb80 1 -- start start 2023-07-18T03:34:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.968+0000 7f182664cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d01386b0 0x55d2d0306c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.968+0000 7f182664cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d02dd6e0 0x55d2d0323580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.968+0000 7f182664cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 0x55d2d0306400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.968+0000 7f182664cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d01e1c50 con 0x55d2d02dd6e0 2023-07-18T03:34:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.968+0000 7f182664cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d0310c00 con 0x55d2d030cb50 2023-07-18T03:34:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.968+0000 7f182664cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d03087d0 con 0x55d2d01386b0 2023-07-18T03:34:13.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.968+0000 7f1813fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d01386b0 0x55d2d0306c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.968+0000 7f1813fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d01386b0 0x55d2d0306c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48784/0 (socket says 172.21.15.40:48784) 2023-07-18T03:34:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.968+0000 7f1813fff700 1 -- 172.21.15.40:0/2604464741 learned_addr learned my addr 172.21.15.40:0/2604464741 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.968+0000 7f1818a63700 1 --2- 172.21.15.40:0/2604464741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 0x55d2d0306400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.969+0000 7f18137fe700 1 --2- 172.21.15.40:0/2604464741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d02dd6e0 0x55d2d0323580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.969+0000 7f1818a63700 1 -- 172.21.15.40:0/2604464741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d01386b0 msgr2=0x55d2d0306c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.969+0000 7f1818a63700 1 --2- 172.21.15.40:0/2604464741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d01386b0 0x55d2d0306c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.969+0000 7f1818a63700 1 -- 172.21.15.40:0/2604464741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d02dd6e0 msgr2=0x55d2d0323580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.969+0000 7f1818a63700 1 --2- 172.21.15.40:0/2604464741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d02dd6e0 0x55d2d0323580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.969+0000 7f1818a63700 1 -- 172.21.15.40:0/2604464741 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2d00821f0 con 0x55d2d030cb50 2023-07-18T03:34:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.970+0000 7f1818a63700 1 --2- 172.21.15.40:0/2604464741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 0x55d2d0306400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1808020580 tx=0x7f1808009d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a77470a77d50ffe0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.971+0000 7f17faffd700 1 -- 172.21.15.40:0/2604464741 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1808010040 con 0x55d2d030cb50 2023-07-18T03:34:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.971+0000 7f17faffd700 1 -- 172.21.15.40:0/2604464741 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f180802f030 con 0x55d2d030cb50 2023-07-18T03:34:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.971+0000 7f17faffd700 1 -- 172.21.15.40:0/2604464741 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18080346e0 con 0x55d2d030cb50 2023-07-18T03:34:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.972+0000 7f182664cb80 1 -- 172.21.15.40:0/2604464741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 msgr2=0x55d2d0306400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.972+0000 7f182664cb80 1 --2- 172.21.15.40:0/2604464741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 0x55d2d0306400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1808020580 tx=0x7f1808009d80 comp rx=0 tx=0).stop 2023-07-18T03:34:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.980+0000 7f182664cb80 1 -- 172.21.15.40:0/2604464741 shutdown_connections 2023-07-18T03:34:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.980+0000 7f182664cb80 1 --2- 172.21.15.40:0/2604464741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 0x55d2d0306400 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.980+0000 7f182664cb80 1 --2- 172.21.15.40:0/2604464741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d01386b0 0x55d2d0306c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.980+0000 7f182664cb80 1 --2- 172.21.15.40:0/2604464741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d02dd6e0 0x55d2d0323580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.981+0000 7f182664cb80 1 -- 172.21.15.40:0/2604464741 >> 172.21.15.40:0/2604464741 conn(0x55d2d02d1be0 msgr2=0x55d2d02dcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:13.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.988+0000 7f182664cb80 1 -- 172.21.15.40:0/2604464741 shutdown_connections 2023-07-18T03:34:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.988+0000 7f182664cb80 1 -- 172.21.15.40:0/2604464741 wait complete. 2023-07-18T03:34:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.988+0000 7f182664cb80 1 -- start start 2023-07-18T03:34:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.988+0000 7f182664cb80 1 Processor -- start 2023-07-18T03:34:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.989+0000 7f182664cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d01386b0 0x55d2d0320a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.989+0000 7f182664cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d02dd6e0 0x55d2d0320f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.989+0000 7f182664cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 0x55d2d0321f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.989+0000 7f182664cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d03224a0 con 0x55d2d02dd6e0 2023-07-18T03:34:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.989+0000 7f182664cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d0322610 con 0x55d2d030cb50 2023-07-18T03:34:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.989+0000 7f182664cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d03227d0 con 0x55d2d01386b0 2023-07-18T03:34:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.989+0000 7f182664cb80 10 client.0 _ll_register_callbacks cb 0x7f18080167b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:34:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.989+0000 7f1813fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d01386b0 0x55d2d0320a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.989+0000 7f1813fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d01386b0 0x55d2d0320a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48790/0 (socket says 172.21.15.40:48790) 2023-07-18T03:34:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.989+0000 7f1813fff700 1 -- 172.21.15.40:0/3129421035 learned_addr learned my addr 172.21.15.40:0/3129421035 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.989+0000 7f1818a63700 1 --2- 172.21.15.40:0/3129421035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 0x55d2d0321f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.990+0000 7f1813fff700 1 -- 172.21.15.40:0/3129421035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 msgr2=0x55d2d0321f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.990+0000 7f1813fff700 1 --2- 172.21.15.40:0/3129421035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 0x55d2d0321f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.990+0000 7f1813fff700 1 -- 172.21.15.40:0/3129421035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d02dd6e0 msgr2=0x55d2d0320f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.990+0000 7f18137fe700 1 --2- 172.21.15.40:0/3129421035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d02dd6e0 0x55d2d0320f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.990+0000 7f1813fff700 1 --2- 172.21.15.40:0/3129421035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d02dd6e0 0x55d2d0320f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.990+0000 7f1813fff700 1 -- 172.21.15.40:0/3129421035 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2d0044c00 con 0x55d2d01386b0 2023-07-18T03:34:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.990+0000 7f18137fe700 1 --2- 172.21.15.40:0/3129421035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d02dd6e0 0x55d2d0320f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:34:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.990+0000 7f1813fff700 1 --2- 172.21.15.40:0/3129421035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d01386b0 0x55d2d0320a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1814113970 tx=0x7f180c00ad80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=471ab5c2b8bc6141 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.990+0000 7f17f7ff7700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:34:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.990+0000 7f17f7ff7700 1 -- 172.21.15.40:0/3129421035 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c013030 con 0x55d2d01386b0 2023-07-18T03:34:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.990+0000 7f17f7ff7700 1 -- 172.21.15.40:0/3129421035 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f180c00d830 con 0x55d2d01386b0 2023-07-18T03:34:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.990+0000 7f182664cb80 1 -- 172.21.15.40:0/3129421035 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18040a2b90 con 0x55d2d01386b0 2023-07-18T03:34:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.991+0000 7f17f7ff7700 1 -- 172.21.15.40:0/3129421035 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f180c00d9f0 con 0x55d2d01386b0 2023-07-18T03:34:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.991+0000 7f182664cb80 10 client.25831 fetch_fsmap learned FSMap version 34 2023-07-18T03:34:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.991+0000 7f182664cb80 1 -- 172.21.15.40:0/3129421035 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d2d005a250 con 0x55d2d01386b0 2023-07-18T03:34:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.991+0000 7f17f7ff7700 1 -- 172.21.15.40:0/3129421035 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18040a2b90 con 0x55d2d01386b0 2023-07-18T03:34:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.991+0000 7f17f7ff7700 1 -- 172.21.15.40:0/3129421035 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f180c0293d0 con 0x55d2d01386b0 2023-07-18T03:34:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.991+0000 7f182664cb80 10 client.25831 fetch_fsmap finished waiting for FSMap version 34 2023-07-18T03:34:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.991+0000 7f182664cb80 10 client.25831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:34:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.991+0000 7f182664cb80 20 client.25831 populate_metadata read hostname 'smithi040' 2023-07-18T03:34:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.991+0000 7f182664cb80 1 --2- 172.21.15.40:0/3129421035 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d2d0368410 0x55d2d036a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.991+0000 7f182664cb80 4 client.25831 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:34:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.991+0000 7f182664cb80 1 -- 172.21.15.40:0/3129421035 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d2d01e8ff0 con 0x55d2d0368410 2023-07-18T03:34:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.992+0000 7f18137fe700 1 --2- 172.21.15.40:0/3129421035 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d2d0368410 0x55d2d036a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.992+0000 7f17f7ff7700 10 client.25831 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:13.992+0000 7f18137fe700 1 --2- 172.21.15.40:0/3129421035 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d2d0368410 0x55d2d036a820 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b80f8ca929839f5 server_cookie=22c6f1b15b301722 in_seq=0 out_seq=0 2023-07-18T03:34:14.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:14 smithi134 ceph-mon[98216]: pgmap v394: 97 pgs: 97 active+clean; 692 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 485 op/s 2023-07-18T03:34:14.335 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:14 smithi040 ceph-mon[96439]: pgmap v394: 97 pgs: 97 active+clean; 692 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 485 op/s 2023-07-18T03:34:14.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:14 smithi064 ceph-mon[98059]: pgmap v394: 97 pgs: 97 active+clean; 692 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 485 op/s 2023-07-18T03:34:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:34:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:34:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:34:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:34:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.395+0000 7f17f7ff7700 1 -- 172.21.15.40:0/3129421035 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d2d01e8ff0 con 0x55d2d0368410 2023-07-18T03:34:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.395+0000 7f17f7ff7700 10 client.25831 handle_command_reply: tid=0 2023-07-18T03:34:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.399+0000 7f182664cb80 1 client.25831 shutdown 2023-07-18T03:34:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.399+0000 7f182664cb80 1 -- 172.21.15.40:0/3129421035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d01386b0 msgr2=0x55d2d0320a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.399+0000 7f182664cb80 1 --2- 172.21.15.40:0/3129421035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d01386b0 0x55d2d0320a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1814113970 tx=0x7f180c00ad80 comp rx=0 tx=0).stop 2023-07-18T03:34:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.400+0000 7f182664cb80 1 -- 172.21.15.40:0/3129421035 shutdown_connections 2023-07-18T03:34:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.400+0000 7f182664cb80 1 --2- 172.21.15.40:0/3129421035 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d2d0368410 0x55d2d036a820 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.400+0000 7f182664cb80 1 --2- 172.21.15.40:0/3129421035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2d030cb50 0x55d2d0321f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.400+0000 7f182664cb80 1 --2- 172.21.15.40:0/3129421035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d2d01386b0 0x55d2d0320a10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.400+0000 7f182664cb80 1 --2- 172.21.15.40:0/3129421035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d2d02dd6e0 0x55d2d0320f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.400+0000 7f182664cb80 1 -- 172.21.15.40:0/3129421035 >> 172.21.15.40:0/3129421035 conn(0x55d2d02d1be0 msgr2=0x55d2d02e0420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.400+0000 7f17f7ff7700 0 client.25831 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.400+0000 7f182664cb80 1 -- 172.21.15.40:0/3129421035 shutdown_connections 2023-07-18T03:34:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.400+0000 7f182664cb80 1 -- 172.21.15.40:0/3129421035 wait complete. 2023-07-18T03:34:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.400+0000 7f182664cb80 20 client.25831 trim_cache size 0 max 16384 2023-07-18T03:34:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.560+0000 7f181a4c6700 1 -- 172.21.15.40:0/1520494876 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f17fc0a1320 msgr2=0x7f17fc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.560+0000 7f181a4c6700 1 --2- 172.21.15.40:0/1520494876 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f17fc0a1320 0x7f17fc0a37d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55d2d02dc060 tx=0x7f180800d040 comp rx=0 tx=0).stop 2023-07-18T03:34:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.560+0000 7f181a4c6700 1 -- 172.21.15.40:0/1520494876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1814071850 msgr2=0x7f1814114140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.560+0000 7f181a4c6700 1 --2- 172.21.15.40:0/1520494876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1814071850 0x7f1814114140 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f18040036f0 tx=0x7f1804003d80 comp rx=0 tx=0).stop 2023-07-18T03:34:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.562+0000 7f181a4c6700 1 -- 172.21.15.40:0/1520494876 shutdown_connections 2023-07-18T03:34:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.562+0000 7f181a4c6700 1 --2- 172.21.15.40:0/1520494876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f181410bd60 0x7f1814115010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.562+0000 7f181a4c6700 1 --2- 172.21.15.40:0/1520494876 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f17fc0a1320 0x7f17fc0a37d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.562+0000 7f181a4c6700 1 --2- 172.21.15.40:0/1520494876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1814070f00 0x7f1814113c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.562+0000 7f181a4c6700 1 --2- 172.21.15.40:0/1520494876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1814071850 0x7f1814114140 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:15.559 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.562+0000 7f181a4c6700 1 -- 172.21.15.40:0/1520494876 >> 172.21.15.40:0/1520494876 conn(0x7f181406ca50 msgr2=0x7f181406e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.567+0000 7f181a4c6700 1 -- 172.21.15.40:0/1520494876 shutdown_connections 2023-07-18T03:34:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:15.567+0000 7f181a4c6700 1 -- 172.21.15.40:0/1520494876 wait complete. 2023-07-18T03:34:16.334 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:16 smithi040 ceph-mon[96439]: pgmap v395: 97 pgs: 97 active+clean; 690 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 481 op/s 2023-07-18T03:34:16.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:16 smithi064 ceph-mon[98059]: pgmap v395: 97 pgs: 97 active+clean; 690 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 481 op/s 2023-07-18T03:34:16.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:16 smithi134 ceph-mon[98216]: pgmap v395: 97 pgs: 97 active+clean; 690 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 481 op/s 2023-07-18T03:34:18.343 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:18 smithi040 ceph-mon[96439]: pgmap v396: 97 pgs: 97 active+clean; 694 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 550 op/s 2023-07-18T03:34:18.353 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:18 smithi064 ceph-mon[98059]: pgmap v396: 97 pgs: 97 active+clean; 694 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 550 op/s 2023-07-18T03:34:18.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:18 smithi134 ceph-mon[98216]: pgmap v396: 97 pgs: 97 active+clean; 694 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 550 op/s 2023-07-18T03:34:20.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:20 smithi134 ceph-mon[98216]: pgmap v397: 97 pgs: 97 active+clean; 694 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.8 MiB/s wr, 375 op/s 2023-07-18T03:34:20.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:20 smithi040 ceph-mon[96439]: pgmap v397: 97 pgs: 97 active+clean; 694 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.8 MiB/s wr, 375 op/s 2023-07-18T03:34:20.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:20 smithi064 ceph-mon[98059]: pgmap v397: 97 pgs: 97 active+clean; 694 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.8 MiB/s wr, 375 op/s 2023-07-18T03:34:23.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:22 smithi134 ceph-mon[98216]: pgmap v398: 97 pgs: 97 active+clean; 699 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.8 MiB/s wr, 402 op/s 2023-07-18T03:34:23.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:22 smithi040 ceph-mon[96439]: pgmap v398: 97 pgs: 97 active+clean; 699 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.8 MiB/s wr, 402 op/s 2023-07-18T03:34:23.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:22 smithi064 ceph-mon[98059]: pgmap v398: 97 pgs: 97 active+clean; 699 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.8 MiB/s wr, 402 op/s 2023-07-18T03:34:23.165 INFO:tasks.workunit.client.0.smithi040.stdout: 197 2349239 737 1717715 706 407993 811 2023-07-18T03:34:24.345 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:24 smithi040 ceph-mon[96439]: pgmap v399: 97 pgs: 97 active+clean; 699 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 216 op/s 2023-07-18T03:34:24.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:24 smithi064 ceph-mon[98059]: pgmap v399: 97 pgs: 97 active+clean; 699 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 216 op/s 2023-07-18T03:34:24.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:24 smithi134 ceph-mon[98216]: pgmap v399: 97 pgs: 97 active+clean; 699 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 216 op/s 2023-07-18T03:34:26.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:26 smithi064 ceph-mon[98059]: pgmap v400: 97 pgs: 97 active+clean; 703 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 208 op/s 2023-07-18T03:34:26.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:26 smithi134 ceph-mon[98216]: pgmap v400: 97 pgs: 97 active+clean; 703 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 208 op/s 2023-07-18T03:34:26.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:26 smithi040 ceph-mon[96439]: pgmap v400: 97 pgs: 97 active+clean; 703 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 208 op/s 2023-07-18T03:34:28.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:28 smithi134 ceph-mon[98216]: pgmap v401: 97 pgs: 97 active+clean; 705 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2023-07-18T03:34:28.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:28 smithi040 ceph-mon[96439]: pgmap v401: 97 pgs: 97 active+clean; 705 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2023-07-18T03:34:28.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:28 smithi064 ceph-mon[98059]: pgmap v401: 97 pgs: 97 active+clean; 705 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2023-07-18T03:34:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.296+0000 7f4469ac9700 1 -- 172.21.15.40:0/2842215812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4464100600 msgr2=0x7f44640687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.296+0000 7f4469ac9700 1 --2- 172.21.15.40:0/2842215812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4464100600 0x7f44640687d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f445c00b0a0 tx=0x7f445c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.296+0000 7f4469ac9700 1 -- 172.21.15.40:0/2842215812 shutdown_connections 2023-07-18T03:34:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.296+0000 7f4469ac9700 1 --2- 172.21.15.40:0/2842215812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4464100600 0x7f44640687d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.296+0000 7f4469ac9700 1 --2- 172.21.15.40:0/2842215812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44640fea80 0x7f44640fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.296+0000 7f4469ac9700 1 --2- 172.21.15.40:0/2842215812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44640ffc70 0x7f44641000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.296+0000 7f4469ac9700 1 -- 172.21.15.40:0/2842215812 >> 172.21.15.40:0/2842215812 conn(0x7f44640fa0c0 msgr2=0x7f44640fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.296+0000 7f4469ac9700 1 -- 172.21.15.40:0/2842215812 shutdown_connections 2023-07-18T03:34:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.296+0000 7f4469ac9700 1 -- 172.21.15.40:0/2842215812 wait complete. 2023-07-18T03:34:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.297+0000 7f4469ac9700 1 Processor -- start 2023-07-18T03:34:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.298+0000 7f4469ac9700 1 -- start start 2023-07-18T03:34:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.298+0000 7f4469ac9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44640fea80 0x7f4464197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.298+0000 7f4469ac9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44640ffc70 0x7f4464198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.298+0000 7f4469ac9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4464100600 0x7f4464198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.298+0000 7f4469ac9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4464199620 con 0x7f44640ffc70 2023-07-18T03:34:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.298+0000 7f4469ac9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4464199790 con 0x7f44640fea80 2023-07-18T03:34:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.298+0000 7f4469ac9700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4464199950 con 0x7f4464100600 2023-07-18T03:34:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.298+0000 7f44637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44640fea80 0x7f4464197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.298+0000 7f44637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44640fea80 0x7f4464197c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43104/0 (socket says 172.21.15.40:43104) 2023-07-18T03:34:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.298+0000 7f44637fe700 1 -- 172.21.15.40:0/707027665 learned_addr learned my addr 172.21.15.40:0/707027665 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.298+0000 7f4463fff700 1 --2- 172.21.15.40:0/707027665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4464100600 0x7f4464198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.298+0000 7f4462ffd700 1 --2- 172.21.15.40:0/707027665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44640ffc70 0x7f4464198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.298+0000 7f44637fe700 1 -- 172.21.15.40:0/707027665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4464100600 msgr2=0x7f4464198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.298+0000 7f44637fe700 1 --2- 172.21.15.40:0/707027665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4464100600 0x7f4464198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.298+0000 7f44637fe700 1 -- 172.21.15.40:0/707027665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44640ffc70 msgr2=0x7f4464198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.298+0000 7f44637fe700 1 --2- 172.21.15.40:0/707027665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44640ffc70 0x7f4464198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.298+0000 7f44637fe700 1 -- 172.21.15.40:0/707027665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c009d00 con 0x7f44640fea80 2023-07-18T03:34:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.299+0000 7f4462ffd700 1 --2- 172.21.15.40:0/707027665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44640ffc70 0x7f4464198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:34:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.299+0000 7f44637fe700 1 --2- 172.21.15.40:0/707027665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44640fea80 0x7f4464197c00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f445800d980 tx=0x7f445800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.300+0000 7f4460ff9700 1 -- 172.21.15.40:0/707027665 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4458004020 con 0x7f44640fea80 2023-07-18T03:34:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.300+0000 7f4460ff9700 1 -- 172.21.15.40:0/707027665 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4458009730 con 0x7f44640fea80 2023-07-18T03:34:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.300+0000 7f4460ff9700 1 -- 172.21.15.40:0/707027665 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4458003a50 con 0x7f44640fea80 2023-07-18T03:34:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.300+0000 7f4469ac9700 1 -- 172.21.15.40:0/707027665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4464199c30 con 0x7f44640fea80 2023-07-18T03:34:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.300+0000 7f4469ac9700 1 -- 172.21.15.40:0/707027665 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44641a03f0 con 0x7f44640fea80 2023-07-18T03:34:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.304+0000 7f4460ff9700 1 -- 172.21.15.40:0/707027665 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4458009b60 con 0x7f44640fea80 2023-07-18T03:34:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.305+0000 7f4460ff9700 1 --2- 172.21.15.40:0/707027665 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f444c0a12a0 0x7f444c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.305+0000 7f4460ff9700 1 -- 172.21.15.40:0/707027665 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f44580cc3e0 con 0x7f44640fea80 2023-07-18T03:34:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.305+0000 7f4462ffd700 1 --2- 172.21.15.40:0/707027665 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f444c0a12a0 0x7f444c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.307+0000 7f4475c4fb80 1 Processor -- start 2023-07-18T03:34:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.307+0000 7f4475c4fb80 1 -- start start 2023-07-18T03:34:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.307+0000 7f4475c4fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f51f6b0 0x55a78f6c3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.307+0000 7f4475c4fb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c3840 0x55a78f6c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.307+0000 7f4475c4fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a78f6c46e0 0x55a78f6c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.307+0000 7f4475c4fb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78f5c8c50 con 0x55a78f6c3840 2023-07-18T03:34:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.307+0000 7f4475c4fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78f61b9e0 con 0x55a78f6c46e0 2023-07-18T03:34:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.307+0000 7f4475c4fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78f614e70 con 0x55a78f51f6b0 2023-07-18T03:34:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.307+0000 7f44637fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f51f6b0 0x55a78f6c3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.307+0000 7f44637fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f51f6b0 0x55a78f6c3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48814/0 (socket says 172.21.15.40:48814) 2023-07-18T03:34:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.307+0000 7f44637fe700 1 -- 172.21.15.40:0/2486037573 learned_addr learned my addr 172.21.15.40:0/2486037573 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.308+0000 7f4462ffd700 1 --2- 172.21.15.40:0/2486037573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c3840 0x55a78f6c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.308+0000 7f44637fe700 1 -- 172.21.15.40:0/2486037573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a78f6c46e0 msgr2=0x55a78f6c4b80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:34:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.308+0000 7f44637fe700 1 --2- 172.21.15.40:0/2486037573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a78f6c46e0 0x55a78f6c4b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.308+0000 7f44637fe700 1 -- 172.21.15.40:0/2486037573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c3840 msgr2=0x55a78f6c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.308+0000 7f44637fe700 1 --2- 172.21.15.40:0/2486037573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c3840 0x55a78f6c41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.308+0000 7f44637fe700 1 -- 172.21.15.40:0/2486037573 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a78f442d80 con 0x55a78f51f6b0 2023-07-18T03:34:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.309+0000 7f4462ffd700 1 --2- 172.21.15.40:0/707027665 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f444c0a12a0 0x7f444c0a3750 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4454010450 tx=0x7f4454009a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.311+0000 7f44637fe700 1 --2- 172.21.15.40:0/2486037573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f51f6b0 0x55a78f6c3300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f44580a16f0 tx=0x7f44580993e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3be250b51eb8f5b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.311+0000 7f4449ffb700 1 -- 172.21.15.40:0/2486037573 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44580a1c20 con 0x55a78f51f6b0 2023-07-18T03:34:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.311+0000 7f4449ffb700 1 -- 172.21.15.40:0/2486037573 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4458097910 con 0x55a78f51f6b0 2023-07-18T03:34:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.311+0000 7f4449ffb700 1 -- 172.21.15.40:0/2486037573 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4458099a30 con 0x55a78f51f6b0 2023-07-18T03:34:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.312+0000 7f4475c4fb80 1 -- 172.21.15.40:0/2486037573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f51f6b0 msgr2=0x55a78f6c3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.312+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/2486037573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f51f6b0 0x55a78f6c3300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f44580a16f0 tx=0x7f44580993e0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.313+0000 7f4475c4fb80 1 -- 172.21.15.40:0/2486037573 shutdown_connections 2023-07-18T03:34:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.313+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/2486037573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a78f6c46e0 0x55a78f6c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.313+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/2486037573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f51f6b0 0x55a78f6c3300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.313+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/2486037573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c3840 0x55a78f6c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.313+0000 7f4475c4fb80 1 -- 172.21.15.40:0/2486037573 >> 172.21.15.40:0/2486037573 conn(0x55a78f6b8be0 msgr2=0x55a78f6bdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.313+0000 7f4475c4fb80 1 -- 172.21.15.40:0/2486037573 shutdown_connections 2023-07-18T03:34:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.313+0000 7f4475c4fb80 1 -- 172.21.15.40:0/2486037573 wait complete. 2023-07-18T03:34:29.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.313+0000 7f4475c4fb80 1 -- start start 2023-07-18T03:34:29.316 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.314+0000 7f4475c4fb80 1 Processor -- start 2023-07-18T03:34:29.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.314+0000 7f4475c4fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a78f51f6b0 0x55a78f6dd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.314+0000 7f4475c4fb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c3840 0x55a78f6ddce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.314+0000 7f4475c4fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f6c46e0 0x55a78f6e4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.314+0000 7f4475c4fb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78f5b4a60 con 0x55a78f6c3840 2023-07-18T03:34:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.314+0000 7f4475c4fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78f5b4760 con 0x55a78f51f6b0 2023-07-18T03:34:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.314+0000 7f4475c4fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78f4988c0 con 0x55a78f6c46e0 2023-07-18T03:34:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.314+0000 7f4475c4fb80 10 client.0 _ll_register_callbacks cb 0x55a78f5b2700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:34:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.314+0000 7f4463fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f6c46e0 0x55a78f6e4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.314+0000 7f4463fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f6c46e0 0x55a78f6e4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48822/0 (socket says 172.21.15.40:48822) 2023-07-18T03:34:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.314+0000 7f4463fff700 1 -- 172.21.15.40:0/4034167035 learned_addr learned my addr 172.21.15.40:0/4034167035 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.315+0000 7f4463fff700 1 -- 172.21.15.40:0/4034167035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a78f51f6b0 msgr2=0x55a78f6dd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.315+0000 7f4463fff700 1 --2- 172.21.15.40:0/4034167035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a78f51f6b0 0x55a78f6dd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.315+0000 7f4463fff700 1 -- 172.21.15.40:0/4034167035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c3840 msgr2=0x55a78f6ddce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.315+0000 7f4462ffd700 1 --2- 172.21.15.40:0/4034167035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c3840 0x55a78f6ddce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.315+0000 7f4463fff700 1 --2- 172.21.15.40:0/4034167035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c3840 0x55a78f6ddce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.315+0000 7f4463fff700 1 -- 172.21.15.40:0/4034167035 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44641a03f0 con 0x55a78f6c46e0 2023-07-18T03:34:29.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.315+0000 7f4462ffd700 1 --2- 172.21.15.40:0/4034167035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c3840 0x55a78f6ddce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:34:29.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.315+0000 7f4463fff700 1 --2- 172.21.15.40:0/4034167035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f6c46e0 0x55a78f6e4d20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f445c007850 tx=0x7f445c007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cbd54a5710cf089 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:29.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.315+0000 7f4442ffd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:34:29.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.315+0000 7f4442ffd700 1 -- 172.21.15.40:0/4034167035 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c00e040 con 0x55a78f6c46e0 2023-07-18T03:34:29.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.315+0000 7f4442ffd700 1 -- 172.21.15.40:0/4034167035 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f445c003c00 con 0x55a78f6c46e0 2023-07-18T03:34:29.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.315+0000 7f4475c4fb80 1 -- 172.21.15.40:0/4034167035 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a78f56e490 con 0x55a78f6c46e0 2023-07-18T03:34:29.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.316+0000 7f4442ffd700 1 -- 172.21.15.40:0/4034167035 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f445c00bea0 con 0x55a78f6c46e0 2023-07-18T03:34:29.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.316+0000 7f4475c4fb80 10 client.25843 fetch_fsmap learned FSMap version 34 2023-07-18T03:34:29.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.316+0000 7f4475c4fb80 1 -- 172.21.15.40:0/4034167035 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a78f42bc00 con 0x55a78f6c46e0 2023-07-18T03:34:29.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.316+0000 7f4442ffd700 1 -- 172.21.15.40:0/4034167035 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a78f56e490 con 0x55a78f6c46e0 2023-07-18T03:34:29.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.316+0000 7f4442ffd700 1 -- 172.21.15.40:0/4034167035 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f445c019030 con 0x55a78f6c46e0 2023-07-18T03:34:29.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.316+0000 7f4475c4fb80 10 client.25843 fetch_fsmap finished waiting for FSMap version 34 2023-07-18T03:34:29.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.316+0000 7f4475c4fb80 10 client.25843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:34:29.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.316+0000 7f4475c4fb80 20 client.25843 populate_metadata read hostname 'smithi040' 2023-07-18T03:34:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.316+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/4034167035 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55a78f6f1e60 0x55a78f6f4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.316+0000 7f4475c4fb80 4 client.25843 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:34:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.316+0000 7f4475c4fb80 1 -- 172.21.15.40:0/4034167035 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a78f60de50 con 0x55a78f6f1e60 2023-07-18T03:34:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.316+0000 7f44637fe700 1 --2- 172.21.15.40:0/4034167035 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55a78f6f1e60 0x55a78f6f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.317+0000 7f4442ffd700 10 client.25843 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.317+0000 7f44637fe700 1 --2- 172.21.15.40:0/4034167035 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55a78f6f1e60 0x55a78f6f4270 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60c633a5e5ba49d9 server_cookie=7d662ccb8876de32 in_seq=0 out_seq=0 2023-07-18T03:34:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.318+0000 7f4442ffd700 1 -- 172.21.15.40:0/4034167035 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a78f60de50 con 0x55a78f6f1e60 2023-07-18T03:34:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.318+0000 7f4442ffd700 10 client.25843 handle_command_reply: tid=0 2023-07-18T03:34:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.318+0000 7f4475c4fb80 1 client.25843 shutdown 2023-07-18T03:34:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.318+0000 7f4475c4fb80 1 -- 172.21.15.40:0/4034167035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f6c46e0 msgr2=0x55a78f6e4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.318+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/4034167035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f6c46e0 0x55a78f6e4d20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f445c007850 tx=0x7f445c007930 comp rx=0 tx=0).stop 2023-07-18T03:34:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.318+0000 7f4475c4fb80 1 -- 172.21.15.40:0/4034167035 shutdown_connections 2023-07-18T03:34:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.318+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/4034167035 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55a78f6f1e60 0x55a78f6f4270 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.318+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/4034167035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f6c46e0 0x55a78f6e4d20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.318+0000 7f4442ffd700 0 client.25843 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.318+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/4034167035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a78f51f6b0 0x55a78f6dd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.318+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/4034167035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c3840 0x55a78f6ddce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.318+0000 7f4475c4fb80 1 -- 172.21.15.40:0/4034167035 >> 172.21.15.40:0/4034167035 conn(0x55a78f6b8be0 msgr2=0x55a78f6bd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.318+0000 7f4475c4fb80 1 -- 172.21.15.40:0/4034167035 shutdown_connections 2023-07-18T03:34:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.318+0000 7f4475c4fb80 1 -- 172.21.15.40:0/4034167035 wait complete. 2023-07-18T03:34:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.319+0000 7f4475c4fb80 20 client.25843 trim_cache size 0 max 16384 2023-07-18T03:34:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.353+0000 7f4475c4fb80 1 Processor -- start 2023-07-18T03:34:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.353+0000 7f4475c4fb80 1 -- start start 2023-07-18T03:34:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.353+0000 7f4475c4fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a78f51f6b0 0x55a78f6c7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.353+0000 7f4475c4fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f6c3840 0x55a78f6c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.353+0000 7f4475c4fb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c46e0 0x55a78f6e4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.353+0000 7f4475c4fb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78f60de50 con 0x55a78f6c46e0 2023-07-18T03:34:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.353+0000 7f4475c4fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78f56e490 con 0x55a78f51f6b0 2023-07-18T03:34:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.353+0000 7f4475c4fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78f6de990 con 0x55a78f6c3840 2023-07-18T03:34:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.353+0000 7f4463fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c46e0 0x55a78f6e4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.353+0000 7f4463fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c46e0 0x55a78f6e4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59802/0 (socket says 172.21.15.40:59802) 2023-07-18T03:34:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.353+0000 7f4463fff700 1 -- 172.21.15.40:0/4156834824 learned_addr learned my addr 172.21.15.40:0/4156834824 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.354+0000 7f4462ffd700 1 --2- 172.21.15.40:0/4156834824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f6c3840 0x55a78f6c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.354+0000 7f4463fff700 1 -- 172.21.15.40:0/4156834824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f6c3840 msgr2=0x55a78f6c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.354+0000 7f4463fff700 1 --2- 172.21.15.40:0/4156834824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f6c3840 0x55a78f6c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.354+0000 7f4463fff700 1 -- 172.21.15.40:0/4156834824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a78f51f6b0 msgr2=0x55a78f6c7550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:34:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.354+0000 7f4463fff700 1 --2- 172.21.15.40:0/4156834824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a78f51f6b0 0x55a78f6c7550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.354+0000 7f4463fff700 1 -- 172.21.15.40:0/4156834824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a78f4691f0 con 0x55a78f6c46e0 2023-07-18T03:34:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.354+0000 7f4463fff700 1 --2- 172.21.15.40:0/4156834824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c46e0 0x55a78f6e4c40 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f445c00b3f0 tx=0x7f445c009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25414bd92eb0a13d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.355+0000 7f444a7fc700 1 -- 172.21.15.40:0/4156834824 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c00e040 con 0x55a78f6c46e0 2023-07-18T03:34:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.355+0000 7f444a7fc700 1 -- 172.21.15.40:0/4156834824 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f445c005a70 con 0x55a78f6c46e0 2023-07-18T03:34:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.355+0000 7f444a7fc700 1 -- 172.21.15.40:0/4156834824 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c01c970 con 0x55a78f6c46e0 2023-07-18T03:34:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.356+0000 7f4475c4fb80 1 -- 172.21.15.40:0/4156834824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c46e0 msgr2=0x55a78f6e4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.356+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/4156834824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c46e0 0x55a78f6e4c40 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f445c00b3f0 tx=0x7f445c009630 comp rx=0 tx=0).stop 2023-07-18T03:34:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.356+0000 7f4475c4fb80 1 -- 172.21.15.40:0/4156834824 shutdown_connections 2023-07-18T03:34:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.356+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/4156834824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c46e0 0x55a78f6e4c40 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.356+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/4156834824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a78f51f6b0 0x55a78f6c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.356+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/4156834824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f6c3840 0x55a78f6c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.356+0000 7f4475c4fb80 1 -- 172.21.15.40:0/4156834824 >> 172.21.15.40:0/4156834824 conn(0x55a78f6b8be0 msgr2=0x55a78f6dd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.356+0000 7f4475c4fb80 1 -- 172.21.15.40:0/4156834824 shutdown_connections 2023-07-18T03:34:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.357+0000 7f4475c4fb80 1 -- 172.21.15.40:0/4156834824 wait complete. 2023-07-18T03:34:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.357+0000 7f4475c4fb80 1 -- start start 2023-07-18T03:34:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.357+0000 7f4475c4fb80 1 Processor -- start 2023-07-18T03:34:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.358+0000 7f4475c4fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f51f6b0 0x55a78f6e0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.358+0000 7f4475c4fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a78f6c3840 0x55a78f6e1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.358+0000 7f4475c4fb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c46e0 0x55a78f6e4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.358+0000 7f4475c4fb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78f6e45b0 con 0x55a78f6c46e0 2023-07-18T03:34:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.358+0000 7f4475c4fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78f6e4720 con 0x55a78f6c3840 2023-07-18T03:34:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.358+0000 7f4475c4fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78f6e4890 con 0x55a78f51f6b0 2023-07-18T03:34:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.358+0000 7f4475c4fb80 10 client.0 _ll_register_callbacks cb 0x7f445c009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:34:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.358+0000 7f44637fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f51f6b0 0x55a78f6e0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.358+0000 7f44637fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f51f6b0 0x55a78f6e0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48832/0 (socket says 172.21.15.40:48832) 2023-07-18T03:34:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.358+0000 7f44637fe700 1 -- 172.21.15.40:0/452726026 learned_addr learned my addr 172.21.15.40:0/452726026 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.359+0000 7f44637fe700 1 -- 172.21.15.40:0/452726026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a78f6c3840 msgr2=0x55a78f6e1050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:34:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.359+0000 7f44637fe700 1 --2- 172.21.15.40:0/452726026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a78f6c3840 0x55a78f6e1050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.359+0000 7f44637fe700 1 -- 172.21.15.40:0/452726026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c46e0 msgr2=0x55a78f6e4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.359+0000 7f44637fe700 1 --2- 172.21.15.40:0/452726026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c46e0 0x55a78f6e4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.359+0000 7f44637fe700 1 -- 172.21.15.40:0/452726026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a78f42bc00 con 0x55a78f51f6b0 2023-07-18T03:34:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.359+0000 7f44637fe700 1 --2- 172.21.15.40:0/452726026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f51f6b0 0x55a78f6e0b10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4458009520 tx=0x7f4458095c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63727d88d556d135 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.359+0000 7f4442ffd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:34:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.359+0000 7f4442ffd700 1 -- 172.21.15.40:0/452726026 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44580098a0 con 0x55a78f51f6b0 2023-07-18T03:34:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.359+0000 7f4442ffd700 1 -- 172.21.15.40:0/452726026 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f44580b90b0 con 0x55a78f51f6b0 2023-07-18T03:34:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.359+0000 7f4475c4fb80 1 -- 172.21.15.40:0/452726026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4458097910 con 0x55a78f51f6b0 2023-07-18T03:34:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.359+0000 7f4442ffd700 1 -- 172.21.15.40:0/452726026 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44580b9270 con 0x55a78f51f6b0 2023-07-18T03:34:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.359+0000 7f4475c4fb80 10 client.25849 fetch_fsmap learned FSMap version 34 2023-07-18T03:34:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.360+0000 7f4475c4fb80 1 -- 172.21.15.40:0/452726026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a78f441250 con 0x55a78f51f6b0 2023-07-18T03:34:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.360+0000 7f4442ffd700 1 -- 172.21.15.40:0/452726026 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44580bd080 con 0x55a78f51f6b0 2023-07-18T03:34:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.361+0000 7f4442ffd700 1 -- 172.21.15.40:0/452726026 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 34) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f44580a1790 con 0x55a78f51f6b0 2023-07-18T03:34:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.361+0000 7f4475c4fb80 10 client.25849 fetch_fsmap finished waiting for FSMap version 34 2023-07-18T03:34:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.361+0000 7f4475c4fb80 10 client.25849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:34:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.361+0000 7f4475c4fb80 20 client.25849 populate_metadata read hostname 'smithi040' 2023-07-18T03:34:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.361+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/452726026 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55a78f6e6f90 0x55a78f6e93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.361+0000 7f4475c4fb80 4 client.25849 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:34:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.361+0000 7f4475c4fb80 1 -- 172.21.15.40:0/452726026 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a78f5cfff0 con 0x55a78f6e6f90 2023-07-18T03:34:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.364+0000 7f4462ffd700 1 --2- 172.21.15.40:0/452726026 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55a78f6e6f90 0x55a78f6e93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.365+0000 7f4462ffd700 1 --2- 172.21.15.40:0/452726026 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55a78f6e6f90 0x55a78f6e93a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf76ef1c34bf8553 server_cookie=57b8f5a081298279 in_seq=0 out_seq=0 2023-07-18T03:34:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.371+0000 7f4442ffd700 10 client.25849 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.943+0000 7f4442ffd700 1 -- 172.21.15.40:0/452726026 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a78f5cfff0 con 0x55a78f6e6f90 2023-07-18T03:34:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.943+0000 7f4442ffd700 10 client.25849 handle_command_reply: tid=0 2023-07-18T03:34:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.943+0000 7f4475c4fb80 1 client.25849 shutdown 2023-07-18T03:34:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.953+0000 7f4475c4fb80 1 -- 172.21.15.40:0/452726026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f51f6b0 msgr2=0x55a78f6e0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.953+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/452726026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f51f6b0 0x55a78f6e0b10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4458009520 tx=0x7f4458095c40 comp rx=0 tx=0).stop 2023-07-18T03:34:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.958+0000 7f4475c4fb80 1 -- 172.21.15.40:0/452726026 shutdown_connections 2023-07-18T03:34:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.958+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/452726026 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55a78f6e6f90 0x55a78f6e93a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.958+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/452726026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55a78f6c46e0 0x55a78f6e4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.958+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/452726026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a78f51f6b0 0x55a78f6e0b10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.958+0000 7f4475c4fb80 1 --2- 172.21.15.40:0/452726026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a78f6c3840 0x55a78f6e1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.958+0000 7f4475c4fb80 1 -- 172.21.15.40:0/452726026 >> 172.21.15.40:0/452726026 conn(0x55a78f6b8be0 msgr2=0x55a78f6dd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.958+0000 7f4442ffd700 0 client.25849 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.958+0000 7f4475c4fb80 1 -- 172.21.15.40:0/452726026 shutdown_connections 2023-07-18T03:34:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.958+0000 7f4475c4fb80 1 -- 172.21.15.40:0/452726026 wait complete. 2023-07-18T03:34:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:29.959+0000 7f4475c4fb80 20 client.25849 trim_cache size 0 max 16384 2023-07-18T03:34:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:34:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:34:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:34:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:34:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:30.143+0000 7f4469ac9700 1 -- 172.21.15.40:0/707027665 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f444c0a12a0 msgr2=0x7f444c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:30.143+0000 7f4469ac9700 1 --2- 172.21.15.40:0/707027665 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f444c0a12a0 0x7f444c0a3750 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4454010450 tx=0x7f4454009a40 comp rx=0 tx=0).stop 2023-07-18T03:34:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:30.143+0000 7f4469ac9700 1 -- 172.21.15.40:0/707027665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44640fea80 msgr2=0x7f4464197c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:30.143+0000 7f4469ac9700 1 --2- 172.21.15.40:0/707027665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44640fea80 0x7f4464197c00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f445800d980 tx=0x7f445800dc90 comp rx=0 tx=0).stop 2023-07-18T03:34:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:30.143+0000 7f4469ac9700 1 -- 172.21.15.40:0/707027665 shutdown_connections 2023-07-18T03:34:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:30.143+0000 7f4469ac9700 1 --2- 172.21.15.40:0/707027665 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f444c0a12a0 0x7f444c0a3750 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:30.143+0000 7f4469ac9700 1 --2- 172.21.15.40:0/707027665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4464100600 0x7f4464198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:30.143+0000 7f4469ac9700 1 --2- 172.21.15.40:0/707027665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44640fea80 0x7f4464197c00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:30.143+0000 7f4469ac9700 1 --2- 172.21.15.40:0/707027665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44640ffc70 0x7f4464198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:30.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:30.143+0000 7f4469ac9700 1 -- 172.21.15.40:0/707027665 >> 172.21.15.40:0/707027665 conn(0x7f44640fa0c0 msgr2=0x7f44640fbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:30.144+0000 7f4469ac9700 1 -- 172.21.15.40:0/707027665 shutdown_connections 2023-07-18T03:34:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:30.144+0000 7f4469ac9700 1 -- 172.21.15.40:0/707027665 wait complete. 2023-07-18T03:34:30.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:30 smithi064 ceph-mon[98059]: pgmap v402: 97 pgs: 97 active+clean; 705 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 117 op/s 2023-07-18T03:34:30.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:30 smithi134 ceph-mon[98216]: pgmap v402: 97 pgs: 97 active+clean; 705 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 117 op/s 2023-07-18T03:34:30.585 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:30 smithi040 ceph-mon[96439]: pgmap v402: 97 pgs: 97 active+clean; 705 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 117 op/s 2023-07-18T03:34:32.766 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:34:33.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:32 smithi134 ceph-mon[98216]: pgmap v403: 97 pgs: 97 active+clean; 707 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2023-07-18T03:34:33.080 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:32 smithi040 ceph-mon[96439]: pgmap v403: 97 pgs: 97 active+clean; 707 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2023-07-18T03:34:33.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:32 smithi064 ceph-mon[98059]: pgmap v403: 97 pgs: 97 active+clean; 707 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2023-07-18T03:34:33.166 INFO:tasks.workunit.client.0.smithi040.stdout: 206 2477615 598 1761088 493 470487 617 2023-07-18T03:34:34.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:33 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:34:34.093 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:33 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:34:34.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:33 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:34:35.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:34 smithi064 ceph-mon[98059]: pgmap v404: 97 pgs: 97 active+clean; 707 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-07-18T03:34:35.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:34 smithi134 ceph-mon[98216]: pgmap v404: 97 pgs: 97 active+clean; 707 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-07-18T03:34:35.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:34 smithi040 ceph-mon[96439]: pgmap v404: 97 pgs: 97 active+clean; 707 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2023-07-18T03:34:36.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.059+0000 7fde9aef2700 1 -- 172.21.15.40:0/1280308521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde94070ee0 msgr2=0x7fde940712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:36.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.059+0000 7fde9aef2700 1 --2- 172.21.15.40:0/1280308521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde94070ee0 0x7fde940712f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fde84009990 tx=0x7fde84009ca0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.060+0000 7fde9aef2700 1 -- 172.21.15.40:0/1280308521 shutdown_connections 2023-07-18T03:34:36.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.060+0000 7fde9aef2700 1 --2- 172.21.15.40:0/1280308521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde9410bd60 0x7fde9410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.060+0000 7fde9aef2700 1 --2- 172.21.15.40:0/1280308521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde94070ee0 0x7fde940712f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.060+0000 7fde9aef2700 1 --2- 172.21.15.40:0/1280308521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde94071830 0x7fde9410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.060+0000 7fde9aef2700 1 -- 172.21.15.40:0/1280308521 >> 172.21.15.40:0/1280308521 conn(0x7fde9406ca90 msgr2=0x7fde9406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:36.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.060+0000 7fde9aef2700 1 -- 172.21.15.40:0/1280308521 shutdown_connections 2023-07-18T03:34:36.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.060+0000 7fde9aef2700 1 -- 172.21.15.40:0/1280308521 wait complete. 2023-07-18T03:34:36.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.062+0000 7fde9aef2700 1 Processor -- start 2023-07-18T03:34:36.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.062+0000 7fde9aef2700 1 -- start start 2023-07-18T03:34:36.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.062+0000 7fde9aef2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde94070ee0 0x7fde941a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.062+0000 7fde9aef2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde94071830 0x7fde941a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.062+0000 7fde9aef2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde9410bd60 0x7fde941abca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.062+0000 7fde9aef2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde941ac270 con 0x7fde94071830 2023-07-18T03:34:36.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.062+0000 7fde9aef2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9410ec40 con 0x7fde9410bd60 2023-07-18T03:34:36.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.062+0000 7fde9aef2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9410edd0 con 0x7fde94070ee0 2023-07-18T03:34:36.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.063+0000 7fde9948f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde9410bd60 0x7fde941abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:36.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.063+0000 7fde9948f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde9410bd60 0x7fde941abca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43146/0 (socket says 172.21.15.40:43146) 2023-07-18T03:34:36.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.063+0000 7fde9948f700 1 -- 172.21.15.40:0/17267540 learned_addr learned my addr 172.21.15.40:0/17267540 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:36.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.063+0000 7fde98c8e700 1 --2- 172.21.15.40:0/17267540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde94070ee0 0x7fde941a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:36.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.063+0000 7fde9948f700 1 -- 172.21.15.40:0/17267540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde94070ee0 msgr2=0x7fde941a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:36.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.063+0000 7fde9948f700 1 --2- 172.21.15.40:0/17267540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde94070ee0 0x7fde941a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.063+0000 7fde9948f700 1 -- 172.21.15.40:0/17267540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde94071830 msgr2=0x7fde941a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:36.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.063+0000 7fde9948f700 1 --2- 172.21.15.40:0/17267540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde94071830 0x7fde941a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.063+0000 7fde9948f700 1 -- 172.21.15.40:0/17267540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde840095f0 con 0x7fde9410bd60 2023-07-18T03:34:36.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.066+0000 7fde9948f700 1 --2- 172.21.15.40:0/17267540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde9410bd60 0x7fde941abca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fde8c007ee0 tx=0x7fde8c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:36.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.105+0000 7fde91ffb700 1 -- 172.21.15.40:0/17267540 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c017070 con 0x7fde9410bd60 2023-07-18T03:34:36.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.105+0000 7fde91ffb700 1 -- 172.21.15.40:0/17267540 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde8c00f040 con 0x7fde9410bd60 2023-07-18T03:34:36.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.105+0000 7fde91ffb700 1 -- 172.21.15.40:0/17267540 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c005740 con 0x7fde9410bd60 2023-07-18T03:34:36.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.105+0000 7fde9aef2700 1 -- 172.21.15.40:0/17267540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9410f0b0 con 0x7fde9410bd60 2023-07-18T03:34:36.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.105+0000 7fde9aef2700 1 -- 172.21.15.40:0/17267540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9410f570 con 0x7fde9410bd60 2023-07-18T03:34:36.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.111+0000 7fdea7078b80 1 Processor -- start 2023-07-18T03:34:36.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.112+0000 7fdea7078b80 1 -- start start 2023-07-18T03:34:36.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.113+0000 7fdea7078b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bd4d0 0x5595682bd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.113+0000 7fdea7078b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bde20 0x5595682be6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.113+0000 7fdea7078b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bec70 0x5595682bf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.113+0000 7fdea7078b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595681c2cb0 con 0x5595682bec70 2023-07-18T03:34:36.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.113+0000 7fdea7078b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559568215a40 con 0x5595682bde20 2023-07-18T03:34:36.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.113+0000 7fdea7078b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55956820eed0 con 0x5595682bd4d0 2023-07-18T03:34:36.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.113+0000 7fde98c8e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bde20 0x5595682be6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:36.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.113+0000 7fde98c8e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bde20 0x5595682be6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43154/0 (socket says 172.21.15.40:43154) 2023-07-18T03:34:36.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.113+0000 7fde98c8e700 1 -- 172.21.15.40:0/2461601705 learned_addr learned my addr 172.21.15.40:0/2461601705 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:36.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.114+0000 7fde93fff700 1 --2- 172.21.15.40:0/2461601705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bd4d0 0x5595682bd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:36.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.114+0000 7fde98c8e700 1 -- 172.21.15.40:0/2461601705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bd4d0 msgr2=0x5595682bd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:36.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.114+0000 7fde98c8e700 1 --2- 172.21.15.40:0/2461601705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bd4d0 0x5595682bd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.114+0000 7fde98c8e700 1 -- 172.21.15.40:0/2461601705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bec70 msgr2=0x5595682bf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:36.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.114+0000 7fde98c8e700 1 --2- 172.21.15.40:0/2461601705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bec70 0x5595682bf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.114+0000 7fde98c8e700 1 -- 172.21.15.40:0/2461601705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55956803cde0 con 0x5595682bde20 2023-07-18T03:34:36.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.122+0000 7fde98c8e700 1 --2- 172.21.15.40:0/2461601705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bde20 0x5595682be6e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fde84009f00 tx=0x7fde84009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8c371522b7f682f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:36.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.124+0000 7fde91ffb700 1 -- 172.21.15.40:0/17267540 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fde8c0058a0 con 0x7fde9410bd60 2023-07-18T03:34:36.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.127+0000 7fde91ffb700 1 --2- 172.21.15.40:0/17267540 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fde7c0a1350 0x7fde7c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.127+0000 7fde91ffb700 1 -- 172.21.15.40:0/17267540 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fde8c0cd020 con 0x7fde9410bd60 2023-07-18T03:34:36.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.131+0000 7fde93fff700 1 --2- 172.21.15.40:0/17267540 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fde7c0a1350 0x7fde7c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:36.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.159+0000 7fde7affd700 1 -- 172.21.15.40:0/2461601705 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8400b9d0 con 0x5595682bde20 2023-07-18T03:34:36.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.159+0000 7fde7affd700 1 -- 172.21.15.40:0/2461601705 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde8401e040 con 0x5595682bde20 2023-07-18T03:34:36.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.159+0000 7fde7affd700 1 -- 172.21.15.40:0/2461601705 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84004e50 con 0x5595682bde20 2023-07-18T03:34:36.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.160+0000 7fdea7078b80 1 -- 172.21.15.40:0/2461601705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bde20 msgr2=0x5595682be6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:36.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.160+0000 7fdea7078b80 1 --2- 172.21.15.40:0/2461601705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bde20 0x5595682be6e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fde84009f00 tx=0x7fde84009f30 comp rx=0 tx=0).stop 2023-07-18T03:34:36.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.162+0000 7fdea7078b80 1 -- 172.21.15.40:0/2461601705 shutdown_connections 2023-07-18T03:34:36.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.162+0000 7fdea7078b80 1 --2- 172.21.15.40:0/2461601705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bec70 0x5595682bf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.162+0000 7fdea7078b80 1 --2- 172.21.15.40:0/2461601705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bd4d0 0x5595682bd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.162+0000 7fdea7078b80 1 --2- 172.21.15.40:0/2461601705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bde20 0x5595682be6e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.162+0000 7fdea7078b80 1 -- 172.21.15.40:0/2461601705 >> 172.21.15.40:0/2461601705 conn(0x5595682b2da0 msgr2=0x5595682b81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:36.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.163+0000 7fdea7078b80 1 -- 172.21.15.40:0/2461601705 shutdown_connections 2023-07-18T03:34:36.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.163+0000 7fdea7078b80 1 -- 172.21.15.40:0/2461601705 wait complete. 2023-07-18T03:34:36.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.164+0000 7fdea7078b80 1 -- start start 2023-07-18T03:34:36.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.164+0000 7fdea7078b80 1 Processor -- start 2023-07-18T03:34:36.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.164+0000 7fdea7078b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bd4d0 0x5595682c7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.164+0000 7fdea7078b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bde20 0x5595682c7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.164+0000 7fdea7078b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bec70 0x5595682ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.164+0000 7fdea7078b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595681aeac0 con 0x5595682bde20 2023-07-18T03:34:36.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.165+0000 7fdea7078b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595681ae7c0 con 0x5595682bd4d0 2023-07-18T03:34:36.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.165+0000 7fdea7078b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559568092920 con 0x5595682bec70 2023-07-18T03:34:36.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.165+0000 7fdea7078b80 10 client.0 _ll_register_callbacks cb 0x5595681ac760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:34:36.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.166+0000 7fde9948f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bec70 0x5595682ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:36.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.166+0000 7fde9948f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bec70 0x5595682ce6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48862/0 (socket says 172.21.15.40:48862) 2023-07-18T03:34:36.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.166+0000 7fde9948f700 1 -- 172.21.15.40:0/1123551028 learned_addr learned my addr 172.21.15.40:0/1123551028 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:36.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.166+0000 7fde93fff700 1 --2- 172.21.15.40:0/17267540 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fde7c0a1350 0x7fde7c0a3800 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fde94072540 tx=0x7fde88009d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:36.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.166+0000 7fde98c8e700 1 --2- 172.21.15.40:0/1123551028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bd4d0 0x5595682c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:36.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.177+0000 7fde93fff700 1 --2- 172.21.15.40:0/1123551028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bde20 0x5595682c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:36.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.177+0000 7fde9948f700 1 -- 172.21.15.40:0/1123551028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bd4d0 msgr2=0x5595682c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:36.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.177+0000 7fde9948f700 1 --2- 172.21.15.40:0/1123551028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bd4d0 0x5595682c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.177+0000 7fde9948f700 1 -- 172.21.15.40:0/1123551028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bde20 msgr2=0x5595682c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:36.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.177+0000 7fde9948f700 1 --2- 172.21.15.40:0/1123551028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bde20 0x5595682c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.177+0000 7fde9948f700 1 -- 172.21.15.40:0/1123551028 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde94117640 con 0x5595682bec70 2023-07-18T03:34:36.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.180+0000 7fde9948f700 1 --2- 172.21.15.40:0/1123551028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bec70 0x5595682ce6a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009510 tx=0x7fde8c09a4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2faa6342f80cae8f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:36.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.227+0000 7fde73fff700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:34:36.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.227+0000 7fde73fff700 1 -- 172.21.15.40:0/1123551028 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c09e1b0 con 0x5595682bec70 2023-07-18T03:34:36.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.228+0000 7fde73fff700 1 -- 172.21.15.40:0/1123551028 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fde8c0a2800 con 0x5595682bec70 2023-07-18T03:34:36.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.228+0000 7fdea7078b80 1 -- 172.21.15.40:0/1123551028 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595681684f0 con 0x5595682bec70 2023-07-18T03:34:36.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.230+0000 7fde73fff700 1 -- 172.21.15.40:0/1123551028 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde8c0998a0 con 0x5595682bec70 2023-07-18T03:34:36.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.230+0000 7fdea7078b80 10 client.25858 fetch_fsmap learned FSMap version 36 2023-07-18T03:34:36.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.230+0000 7fdea7078b80 1 -- 172.21.15.40:0/1123551028 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559568025c60 con 0x5595682bec70 2023-07-18T03:34:36.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.237+0000 7fde73fff700 1 -- 172.21.15.40:0/1123551028 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5595681684f0 con 0x5595682bec70 2023-07-18T03:34:36.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.237+0000 7fde73fff700 1 -- 172.21.15.40:0/1123551028 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 36) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fde8c00dce0 con 0x5595682bec70 2023-07-18T03:34:36.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.237+0000 7fdea7078b80 10 client.25858 fetch_fsmap finished waiting for FSMap version 36 2023-07-18T03:34:36.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.237+0000 7fdea7078b80 10 client.25858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:34:36.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.237+0000 7fdea7078b80 20 client.25858 populate_metadata read hostname 'smithi040' 2023-07-18T03:34:36.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.237+0000 7fdea7078b80 1 --2- 172.21.15.40:0/1123551028 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5595682d2f30 0x5595682d5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.237+0000 7fdea7078b80 4 client.25858 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:34:36.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.237+0000 7fdea7078b80 1 -- 172.21.15.40:0/1123551028 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559568207eb0 con 0x5595682d2f30 2023-07-18T03:34:36.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.238+0000 7fde98c8e700 1 --2- 172.21.15.40:0/1123551028 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5595682d2f30 0x5595682d5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:36.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.255+0000 7fde98c8e700 1 --2- 172.21.15.40:0/1123551028 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5595682d2f30 0x5595682d5340 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b86bc8f47acc28 server_cookie=2a437ddc40eaa43e in_seq=0 out_seq=0 2023-07-18T03:34:36.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.255+0000 7fde73fff700 10 client.25858 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:36.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.260+0000 7fde73fff700 1 -- 172.21.15.40:0/1123551028 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559568207eb0 con 0x5595682d2f30 2023-07-18T03:34:36.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.260+0000 7fde73fff700 10 client.25858 handle_command_reply: tid=0 2023-07-18T03:34:36.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.260+0000 7fdea7078b80 1 client.25858 shutdown 2023-07-18T03:34:36.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.261+0000 7fdea7078b80 1 -- 172.21.15.40:0/1123551028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bec70 msgr2=0x5595682ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:36.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.261+0000 7fdea7078b80 1 --2- 172.21.15.40:0/1123551028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bec70 0x5595682ce6a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009510 tx=0x7fde8c09a4a0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.261+0000 7fdea7078b80 1 -- 172.21.15.40:0/1123551028 shutdown_connections 2023-07-18T03:34:36.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.261+0000 7fdea7078b80 1 --2- 172.21.15.40:0/1123551028 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5595682d2f30 0x5595682d5340 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.261+0000 7fdea7078b80 1 --2- 172.21.15.40:0/1123551028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bec70 0x5595682ce6a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.261+0000 7fdea7078b80 1 --2- 172.21.15.40:0/1123551028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bd4d0 0x5595682c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.261+0000 7fdea7078b80 1 --2- 172.21.15.40:0/1123551028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bde20 0x5595682c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.261+0000 7fdea7078b80 1 -- 172.21.15.40:0/1123551028 >> 172.21.15.40:0/1123551028 conn(0x5595682b2da0 msgr2=0x5595682b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:36.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.261+0000 7fde73fff700 0 client.25858 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:36.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.262+0000 7fdea7078b80 1 -- 172.21.15.40:0/1123551028 shutdown_connections 2023-07-18T03:34:36.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.262+0000 7fdea7078b80 1 -- 172.21.15.40:0/1123551028 wait complete. 2023-07-18T03:34:36.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.262+0000 7fdea7078b80 20 client.25858 trim_cache size 0 max 16384 2023-07-18T03:34:36.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.405+0000 7fdea7078b80 1 Processor -- start 2023-07-18T03:34:36.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.412+0000 7fdea7078b80 1 -- start start 2023-07-18T03:34:36.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.413+0000 7fdea7078b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bd4d0 0x5595682c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.413+0000 7fdea7078b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bde20 0x5595682ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.413+0000 7fdea7078b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bec70 0x5595682d4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.413+0000 7fdea7078b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595682c4dd0 con 0x5595682bde20 2023-07-18T03:34:36.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.413+0000 7fdea7078b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595682c6ed0 con 0x5595682bec70 2023-07-18T03:34:36.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.413+0000 7fdea7078b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595682d2c50 con 0x5595682bd4d0 2023-07-18T03:34:36.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.414+0000 7fde98c8e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bd4d0 0x5595682c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:36.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.414+0000 7fde98c8e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bd4d0 0x5595682c5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48868/0 (socket says 172.21.15.40:48868) 2023-07-18T03:34:36.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.414+0000 7fde98c8e700 1 -- 172.21.15.40:0/7065555 learned_addr learned my addr 172.21.15.40:0/7065555 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:36.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.414+0000 7fde9948f700 1 --2- 172.21.15.40:0/7065555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bec70 0x5595682d4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:36.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.415+0000 7fde9948f700 1 -- 172.21.15.40:0/7065555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bd4d0 msgr2=0x5595682c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:36.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.415+0000 7fde9948f700 1 --2- 172.21.15.40:0/7065555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bd4d0 0x5595682c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.415+0000 7fde9948f700 1 -- 172.21.15.40:0/7065555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bde20 msgr2=0x5595682ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:36.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.415+0000 7fde9948f700 1 --2- 172.21.15.40:0/7065555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bde20 0x5595682ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.415+0000 7fde9948f700 1 -- 172.21.15.40:0/7065555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559568063250 con 0x5595682bec70 2023-07-18T03:34:36.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.423+0000 7fde9948f700 1 --2- 172.21.15.40:0/7065555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bec70 0x5595682d4f40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fde8c098700 tx=0x7fde8c0967e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7889044774cd970d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.425+0000 7fde7b7fe700 1 -- 172.21.15.40:0/7065555 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c0a23c0 con 0x5595682bec70 2023-07-18T03:34:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.425+0000 7fde7b7fe700 1 -- 172.21.15.40:0/7065555 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde8c0a25d0 con 0x5595682bec70 2023-07-18T03:34:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.425+0000 7fde7b7fe700 1 -- 172.21.15.40:0/7065555 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c00de10 con 0x5595682bec70 2023-07-18T03:34:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.434+0000 7fdea7078b80 1 -- 172.21.15.40:0/7065555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bec70 msgr2=0x5595682d4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.434+0000 7fdea7078b80 1 --2- 172.21.15.40:0/7065555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bec70 0x5595682d4f40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fde8c098700 tx=0x7fde8c0967e0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.588+0000 7fdea7078b80 1 -- 172.21.15.40:0/7065555 shutdown_connections 2023-07-18T03:34:36.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.588+0000 7fdea7078b80 1 --2- 172.21.15.40:0/7065555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bec70 0x5595682d4f40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.588+0000 7fdea7078b80 1 --2- 172.21.15.40:0/7065555 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bd4d0 0x5595682c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.588+0000 7fdea7078b80 1 --2- 172.21.15.40:0/7065555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bde20 0x5595682ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.588+0000 7fdea7078b80 1 -- 172.21.15.40:0/7065555 >> 172.21.15.40:0/7065555 conn(0x5595682b2da0 msgr2=0x5595682cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.589+0000 7fdea7078b80 1 -- 172.21.15.40:0/7065555 shutdown_connections 2023-07-18T03:34:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.589+0000 7fdea7078b80 1 -- 172.21.15.40:0/7065555 wait complete. 2023-07-18T03:34:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.589+0000 7fdea7078b80 1 -- start start 2023-07-18T03:34:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.590+0000 7fdea7078b80 1 Processor -- start 2023-07-18T03:34:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.591+0000 7fdea7078b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bd4d0 0x5595682ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.591+0000 7fdea7078b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bde20 0x5595682ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.591+0000 7fdea7078b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bec70 0x5595682cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.591+0000 7fdea7078b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595682cdf50 con 0x5595682bde20 2023-07-18T03:34:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.591+0000 7fdea7078b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595682ce0c0 con 0x5595682bec70 2023-07-18T03:34:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.591+0000 7fdea7078b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595682ce280 con 0x5595682bd4d0 2023-07-18T03:34:36.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.591+0000 7fdea7078b80 10 client.0 _ll_register_callbacks cb 0x7fde8c099d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:34:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.600+0000 7fde98c8e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bd4d0 0x5595682ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.600+0000 7fde98c8e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bd4d0 0x5595682ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48876/0 (socket says 172.21.15.40:48876) 2023-07-18T03:34:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.600+0000 7fde98c8e700 1 -- 172.21.15.40:0/3116108488 learned_addr learned my addr 172.21.15.40:0/3116108488 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.600+0000 7fde9948f700 1 --2- 172.21.15.40:0/3116108488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bec70 0x5595682cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:36.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.600+0000 7fde93fff700 1 --2- 172.21.15.40:0/3116108488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bde20 0x5595682ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:36.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.601+0000 7fde9948f700 1 -- 172.21.15.40:0/3116108488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bd4d0 msgr2=0x5595682ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:36.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.601+0000 7fde9948f700 1 --2- 172.21.15.40:0/3116108488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bd4d0 0x5595682ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.601+0000 7fde9948f700 1 -- 172.21.15.40:0/3116108488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bde20 msgr2=0x5595682ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:36.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.601+0000 7fde9948f700 1 --2- 172.21.15.40:0/3116108488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bde20 0x5595682ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.601+0000 7fde9948f700 1 -- 172.21.15.40:0/3116108488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559568025c60 con 0x5595682bec70 2023-07-18T03:34:36.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.604+0000 7fde9948f700 1 --2- 172.21.15.40:0/3116108488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bec70 0x5595682cda10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fde8c098700 tx=0x7fde8c00de00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4d2cb2d02e97844 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:36.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.624+0000 7fde73fff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:34:36.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.624+0000 7fde73fff700 1 -- 172.21.15.40:0/3116108488 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c0a23c0 con 0x5595682bec70 2023-07-18T03:34:36.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.624+0000 7fde73fff700 1 -- 172.21.15.40:0/3116108488 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fde8c0a2800 con 0x5595682bec70 2023-07-18T03:34:36.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.624+0000 7fdea7078b80 1 -- 172.21.15.40:0/3116108488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde8401e040 con 0x5595682bec70 2023-07-18T03:34:36.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.631+0000 7fde73fff700 1 -- 172.21.15.40:0/3116108488 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde8c004f90 con 0x5595682bec70 2023-07-18T03:34:36.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.631+0000 7fdea7078b80 10 client.25844 fetch_fsmap learned FSMap version 36 2023-07-18T03:34:36.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.631+0000 7fdea7078b80 1 -- 172.21.15.40:0/3116108488 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55956803b2b0 con 0x5595682bec70 2023-07-18T03:34:36.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.660+0000 7fde73fff700 1 -- 172.21.15.40:0/3116108488 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde8401e040 con 0x5595682bec70 2023-07-18T03:34:36.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.660+0000 7fde73fff700 1 -- 172.21.15.40:0/3116108488 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 36) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fde8c0b8b80 con 0x5595682bec70 2023-07-18T03:34:36.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.660+0000 7fdea7078b80 10 client.25844 fetch_fsmap finished waiting for FSMap version 36 2023-07-18T03:34:36.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.660+0000 7fdea7078b80 10 client.25844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:34:36.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.660+0000 7fdea7078b80 20 client.25844 populate_metadata read hostname 'smithi040' 2023-07-18T03:34:36.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.660+0000 7fdea7078b80 1 --2- 172.21.15.40:0/3116108488 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559568327780 0x559568329b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:36.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.660+0000 7fdea7078b80 4 client.25844 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:34:36.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.660+0000 7fdea7078b80 1 -- 172.21.15.40:0/3116108488 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5595681ca050 con 0x559568327780 2023-07-18T03:34:36.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.660+0000 7fde98c8e700 1 --2- 172.21.15.40:0/3116108488 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559568327780 0x559568329b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:36.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.679+0000 7fde98c8e700 1 --2- 172.21.15.40:0/3116108488 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559568327780 0x559568329b60 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3397dff8a1839f8d server_cookie=a4b070466476fc5a in_seq=0 out_seq=0 2023-07-18T03:34:36.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.679+0000 7fde73fff700 10 client.25844 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:36.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.687+0000 7fde73fff700 1 -- 172.21.15.40:0/3116108488 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5595681ca050 con 0x559568327780 2023-07-18T03:34:36.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.687+0000 7fde73fff700 10 client.25844 handle_command_reply: tid=0 2023-07-18T03:34:36.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.687+0000 7fdea7078b80 1 client.25844 shutdown 2023-07-18T03:34:36.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.757+0000 7fdea7078b80 1 -- 172.21.15.40:0/3116108488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bec70 msgr2=0x5595682cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:36.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.757+0000 7fdea7078b80 1 --2- 172.21.15.40:0/3116108488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bec70 0x5595682cda10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fde8c098700 tx=0x7fde8c00de00 comp rx=0 tx=0).stop 2023-07-18T03:34:36.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.758+0000 7fdea7078b80 1 -- 172.21.15.40:0/3116108488 shutdown_connections 2023-07-18T03:34:36.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.758+0000 7fdea7078b80 1 --2- 172.21.15.40:0/3116108488 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559568327780 0x559568329b60 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.758+0000 7fdea7078b80 1 --2- 172.21.15.40:0/3116108488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5595682bec70 0x5595682cda10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.758+0000 7fdea7078b80 1 --2- 172.21.15.40:0/3116108488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595682bd4d0 0x5595682ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.758+0000 7fdea7078b80 1 --2- 172.21.15.40:0/3116108488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5595682bde20 0x5595682ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.758+0000 7fdea7078b80 1 -- 172.21.15.40:0/3116108488 >> 172.21.15.40:0/3116108488 conn(0x5595682b2da0 msgr2=0x5595682d4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:36.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.758+0000 7fde73fff700 0 client.25844 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:36.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.758+0000 7fdea7078b80 1 -- 172.21.15.40:0/3116108488 shutdown_connections 2023-07-18T03:34:36.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.758+0000 7fdea7078b80 1 -- 172.21.15.40:0/3116108488 wait complete. 2023-07-18T03:34:36.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.759+0000 7fdea7078b80 20 client.25844 trim_cache size 0 max 16384 2023-07-18T03:34:36.758 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:34:36.758 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-18T03:34:36.759 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:34:36.759 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:34:36.759 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:34:36.759 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:34:36.760 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:34:36.760 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:34:36.760 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:34:36.760 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:34:36.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.944+0000 7fde9aef2700 1 -- 172.21.15.40:0/17267540 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fde7c0a1350 msgr2=0x7fde7c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:36.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.944+0000 7fde9aef2700 1 --2- 172.21.15.40:0/17267540 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fde7c0a1350 0x7fde7c0a3800 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fde94072540 tx=0x7fde88009d90 comp rx=0 tx=0).stop 2023-07-18T03:34:36.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.944+0000 7fde9aef2700 1 -- 172.21.15.40:0/17267540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde9410bd60 msgr2=0x7fde941abca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:36.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.944+0000 7fde9aef2700 1 --2- 172.21.15.40:0/17267540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde9410bd60 0x7fde941abca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fde8c007ee0 tx=0x7fde8c00d3b0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.947+0000 7fde9aef2700 1 -- 172.21.15.40:0/17267540 shutdown_connections 2023-07-18T03:34:36.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.947+0000 7fde9aef2700 1 --2- 172.21.15.40:0/17267540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde9410bd60 0x7fde941abca0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.947+0000 7fde9aef2700 1 --2- 172.21.15.40:0/17267540 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fde7c0a1350 0x7fde7c0a3800 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.947+0000 7fde9aef2700 1 --2- 172.21.15.40:0/17267540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde94070ee0 0x7fde941a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.947+0000 7fde9aef2700 1 --2- 172.21.15.40:0/17267540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde94071830 0x7fde941a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:36.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.947+0000 7fde9aef2700 1 -- 172.21.15.40:0/17267540 >> 172.21.15.40:0/17267540 conn(0x7fde9406ca90 msgr2=0x7fde9406dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:36.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.947+0000 7fde9aef2700 1 -- 172.21.15.40:0/17267540 shutdown_connections 2023-07-18T03:34:36.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:34:36.947+0000 7fde9aef2700 1 -- 172.21.15.40:0/17267540 wait complete. 2023-07-18T03:34:37.018 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:34:37.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:36 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:34:37.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:36 smithi134 ceph-mon[98216]: pgmap v405: 97 pgs: 97 active+clean; 708 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2023-07-18T03:34:37.339 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:36 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:34:37.339 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:36 smithi040 ceph-mon[96439]: pgmap v405: 97 pgs: 97 active+clean; 708 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2023-07-18T03:34:37.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:36 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:34:37.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:36 smithi064 ceph-mon[98059]: pgmap v405: 97 pgs: 97 active+clean; 708 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 194 op/s 2023-07-18T03:34:38.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:38 smithi134 ceph-mon[98216]: pgmap v406: 97 pgs: 97 active+clean; 713 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 213 op/s 2023-07-18T03:34:38.587 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:38 smithi040 ceph-mon[96439]: pgmap v406: 97 pgs: 97 active+clean; 713 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 213 op/s 2023-07-18T03:34:38.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:38 smithi064 ceph-mon[98059]: pgmap v406: 97 pgs: 97 active+clean; 713 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 213 op/s 2023-07-18T03:34:39.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:34:39.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:34:39.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:34:39.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:34:39.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:39 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:34:39.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:34:39.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:34:39.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:34:39.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:34:39.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:39 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:34:39.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:34:39.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:34:39.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:34:39.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:34:39.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:39 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:34:40.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:40 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:34:40.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:40 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:34:40.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:40 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:34:40.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:40 smithi134 ceph-mon[98216]: pgmap v407: 97 pgs: 97 active+clean; 713 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2023-07-18T03:34:40.585 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:40 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:34:40.586 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:40 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:34:40.586 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:40 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:34:40.586 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:40 smithi040 ceph-mon[96439]: pgmap v407: 97 pgs: 97 active+clean; 713 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2023-07-18T03:34:40.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:40 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:34:40.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:40 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:34:40.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:40 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:34:40.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:40 smithi064 ceph-mon[98059]: pgmap v407: 97 pgs: 97 active+clean; 713 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2023-07-18T03:34:43.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:42 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:34:43.067 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:42 smithi134 ceph-mon[98216]: pgmap v408: 97 pgs: 97 active+clean; 717 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 221 op/s 2023-07-18T03:34:43.089 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:42 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:34:43.089 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:42 smithi040 ceph-mon[96439]: pgmap v408: 97 pgs: 97 active+clean; 717 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 221 op/s 2023-07-18T03:34:43.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:42 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:34:43.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:42 smithi064 ceph-mon[98059]: pgmap v408: 97 pgs: 97 active+clean; 717 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 221 op/s 2023-07-18T03:34:43.167 INFO:tasks.workunit.client.0.smithi040.stdout: 217 2170548 578 1536139 689 447568 512 2023-07-18T03:34:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.591+0000 7f6cdaf9b700 1 -- 172.21.15.40:0/4210259140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cd40fea00 msgr2=0x7f6cd40fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.591+0000 7f6cdaf9b700 1 --2- 172.21.15.40:0/4210259140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cd40fea00 0x7f6cd40fee10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8009990 tx=0x7f6cc8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.591+0000 7f6cdaf9b700 1 -- 172.21.15.40:0/4210259140 shutdown_connections 2023-07-18T03:34:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.591+0000 7f6cdaf9b700 1 --2- 172.21.15.40:0/4210259140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd4100640 0x7f6cd4106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.591+0000 7f6cdaf9b700 1 --2- 172.21.15.40:0/4210259140 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cd40fea00 0x7f6cd40fee10 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.591+0000 7f6cdaf9b700 1 --2- 172.21.15.40:0/4210259140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cd40ffc90 0x7f6cd4100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.591+0000 7f6cdaf9b700 1 -- 172.21.15.40:0/4210259140 >> 172.21.15.40:0/4210259140 conn(0x7f6cd40fa000 msgr2=0x7f6cd40fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.591+0000 7f6cdaf9b700 1 -- 172.21.15.40:0/4210259140 shutdown_connections 2023-07-18T03:34:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.591+0000 7f6cdaf9b700 1 -- 172.21.15.40:0/4210259140 wait complete. 2023-07-18T03:34:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.591+0000 7f6cdaf9b700 1 Processor -- start 2023-07-18T03:34:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.591+0000 7f6cdaf9b700 1 -- start start 2023-07-18T03:34:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cdaf9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cd40fea00 0x7f6cd4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cdaf9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd40ffc90 0x7f6cd419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cdaf9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cd4100640 0x7f6cd419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cdaf9b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd419d5e0 con 0x7f6cd40fea00 2023-07-18T03:34:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cdaf9b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd419d720 con 0x7f6cd40ffc90 2023-07-18T03:34:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cdaf9b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd419d8e0 con 0x7f6cd4100640 2023-07-18T03:34:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cd8d37700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cd40fea00 0x7f6cd4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cd8d37700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cd40fea00 0x7f6cd4198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59858/0 (socket says 172.21.15.40:59858) 2023-07-18T03:34:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cd8d37700 1 -- 172.21.15.40:0/1487400784 learned_addr learned my addr 172.21.15.40:0/1487400784 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cd3fff700 1 --2- 172.21.15.40:0/1487400784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd40ffc90 0x7f6cd419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cd9538700 1 --2- 172.21.15.40:0/1487400784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cd4100640 0x7f6cd419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cd8d37700 1 -- 172.21.15.40:0/1487400784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cd4100640 msgr2=0x7f6cd419d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cd8d37700 1 --2- 172.21.15.40:0/1487400784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cd4100640 0x7f6cd419d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cd8d37700 1 -- 172.21.15.40:0/1487400784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd40ffc90 msgr2=0x7f6cd419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cd8d37700 1 --2- 172.21.15.40:0/1487400784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd40ffc90 0x7f6cd419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cd8d37700 1 -- 172.21.15.40:0/1487400784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc80095f0 con 0x7f6cd40fea00 2023-07-18T03:34:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cd8d37700 1 --2- 172.21.15.40:0/1487400784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cd40fea00 0x7f6cd4198570 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80059d0 tx=0x7f6cc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cd1ffb700 1 -- 172.21.15.40:0/1487400784 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc801d070 con 0x7f6cd40fea00 2023-07-18T03:34:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cd1ffb700 1 -- 172.21.15.40:0/1487400784 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc800fb90 con 0x7f6cd40fea00 2023-07-18T03:34:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cdaf9b700 1 -- 172.21.15.40:0/1487400784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd419db60 con 0x7f6cd40fea00 2023-07-18T03:34:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.592+0000 7f6cd1ffb700 1 -- 172.21.15.40:0/1487400784 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc80173f0 con 0x7f6cd40fea00 2023-07-18T03:34:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.593+0000 7f6cdaf9b700 1 -- 172.21.15.40:0/1487400784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd419df80 con 0x7f6cd40fea00 2023-07-18T03:34:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.595+0000 7f6cd1ffb700 1 -- 172.21.15.40:0/1487400784 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6cc800fd00 con 0x7f6cd40fea00 2023-07-18T03:34:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.595+0000 7f6cd1ffb700 1 --2- 172.21.15.40:0/1487400784 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6cc40a1150 0x7f6cc40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.596+0000 7f6cd1ffb700 1 -- 172.21.15.40:0/1487400784 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6cc80cdd30 con 0x7f6cd40fea00 2023-07-18T03:34:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.596+0000 7f6cd3fff700 1 --2- 172.21.15.40:0/1487400784 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6cc40a1150 0x7f6cc40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.596+0000 7f6cd3fff700 1 --2- 172.21.15.40:0/1487400784 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6cc40a1150 0x7f6cc40a3600 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4067db0 tx=0x7f6cc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6ce7121b80 1 Processor -- start 2023-07-18T03:34:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6ce7121b80 1 -- start start 2023-07-18T03:34:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6ce7121b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576505dd6b0 0x557650781300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6ce7121b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557650781840 0x5576507821a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6ce7121b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576507826e0 0x557650782b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6ce7121b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557650686c50 con 0x557650781840 2023-07-18T03:34:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6ce7121b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576506d99e0 con 0x5576505dd6b0 2023-07-18T03:34:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6ce7121b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576506d2e70 con 0x5576507826e0 2023-07-18T03:34:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6cd3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557650781840 0x5576507821a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6cd3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557650781840 0x5576507821a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59868/0 (socket says 172.21.15.40:59868) 2023-07-18T03:34:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6cd3fff700 1 -- 172.21.15.40:0/2011559694 learned_addr learned my addr 172.21.15.40:0/2011559694 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6cd9538700 1 --2- 172.21.15.40:0/2011559694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576507826e0 0x557650782b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:43.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6cd8d37700 1 --2- 172.21.15.40:0/2011559694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576505dd6b0 0x557650781300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6cd3fff700 1 -- 172.21.15.40:0/2011559694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576507826e0 msgr2=0x557650782b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6cd3fff700 1 --2- 172.21.15.40:0/2011559694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576507826e0 0x557650782b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6cd3fff700 1 -- 172.21.15.40:0/2011559694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576505dd6b0 msgr2=0x557650781300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6cd3fff700 1 --2- 172.21.15.40:0/2011559694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576505dd6b0 0x557650781300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.597+0000 7f6cd3fff700 1 -- 172.21.15.40:0/2011559694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557650500d80 con 0x557650781840 2023-07-18T03:34:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.598+0000 7f6cd3fff700 1 --2- 172.21.15.40:0/2011559694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557650781840 0x5576507821a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0011d50 tx=0x7f6cc0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d40a4acc8f5e9bb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.598+0000 7f6cbeffd700 1 -- 172.21.15.40:0/2011559694 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc000eb20 con 0x557650781840 2023-07-18T03:34:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.598+0000 7f6cbeffd700 1 -- 172.21.15.40:0/2011559694 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc0021070 con 0x557650781840 2023-07-18T03:34:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.598+0000 7f6cbeffd700 1 -- 172.21.15.40:0/2011559694 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0003be0 con 0x557650781840 2023-07-18T03:34:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.598+0000 7f6ce7121b80 1 -- 172.21.15.40:0/2011559694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557650781840 msgr2=0x5576507821a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.598+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/2011559694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557650781840 0x5576507821a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0011d50 tx=0x7f6cc0014000 comp rx=0 tx=0).stop 2023-07-18T03:34:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.598+0000 7f6ce7121b80 1 -- 172.21.15.40:0/2011559694 shutdown_connections 2023-07-18T03:34:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.598+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/2011559694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576507826e0 0x557650782b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.598+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/2011559694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576505dd6b0 0x557650781300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.598+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/2011559694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557650781840 0x5576507821a0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.598+0000 7f6ce7121b80 1 -- 172.21.15.40:0/2011559694 >> 172.21.15.40:0/2011559694 conn(0x557650776be0 msgr2=0x55765077bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.599+0000 7f6ce7121b80 1 -- 172.21.15.40:0/2011559694 shutdown_connections 2023-07-18T03:34:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.599+0000 7f6ce7121b80 1 -- 172.21.15.40:0/2011559694 wait complete. 2023-07-18T03:34:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.599+0000 7f6ce7121b80 1 -- start start 2023-07-18T03:34:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.599+0000 7f6ce7121b80 1 Processor -- start 2023-07-18T03:34:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.599+0000 7f6ce7121b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5576505dd6b0 0x55765078a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.599+0000 7f6ce7121b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557650781840 0x55765078ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.599+0000 7f6ce7121b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576507826e0 0x557650791dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.599+0000 7f6ce7121b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557650672a60 con 0x5576505dd6b0 2023-07-18T03:34:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.599+0000 7f6ce7121b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557650672760 con 0x5576507826e0 2023-07-18T03:34:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.599+0000 7f6ce7121b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576505568c0 con 0x557650781840 2023-07-18T03:34:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.599+0000 7f6ce7121b80 10 client.0 _ll_register_callbacks cb 0x557650670700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:34:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6cd8d37700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5576505dd6b0 0x55765078a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:43.603 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6cd8d37700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5576505dd6b0 0x55765078a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59872/0 (socket says 172.21.15.40:59872) 2023-07-18T03:34:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6cd8d37700 1 -- 172.21.15.40:0/2980178823 learned_addr learned my addr 172.21.15.40:0/2980178823 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6cd3fff700 1 --2- 172.21.15.40:0/2980178823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557650781840 0x55765078ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6cd9538700 1 --2- 172.21.15.40:0/2980178823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576507826e0 0x557650791dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6cd8d37700 1 -- 172.21.15.40:0/2980178823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557650781840 msgr2=0x55765078ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6cd8d37700 1 --2- 172.21.15.40:0/2980178823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557650781840 0x55765078ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6cd8d37700 1 -- 172.21.15.40:0/2980178823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576507826e0 msgr2=0x557650791dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6cd8d37700 1 --2- 172.21.15.40:0/2980178823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576507826e0 0x557650791dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6cd8d37700 1 -- 172.21.15.40:0/2980178823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc0017070 con 0x5576505dd6b0 2023-07-18T03:34:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6cd8d37700 1 --2- 172.21.15.40:0/2980178823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5576505dd6b0 0x55765078a840 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80cb810 tx=0x7f6cc809ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3436ce6823779aeb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6cb3fff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:34:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6cb3fff700 1 -- 172.21.15.40:0/2980178823 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc80a10d0 con 0x5576505dd6b0 2023-07-18T03:34:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6cb3fff700 1 -- 172.21.15.40:0/2980178823 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6cc80a3570 con 0x5576505dd6b0 2023-07-18T03:34:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6ce7121b80 1 -- 172.21.15.40:0/2980178823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55765062c490 con 0x5576505dd6b0 2023-07-18T03:34:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6cb3fff700 1 -- 172.21.15.40:0/2980178823 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cc80a32e0 con 0x5576505dd6b0 2023-07-18T03:34:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6ce7121b80 10 client.16221 fetch_fsmap learned FSMap version 37 2023-07-18T03:34:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.600+0000 7f6ce7121b80 1 -- 172.21.15.40:0/2980178823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5576504e9c00 con 0x5576505dd6b0 2023-07-18T03:34:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.601+0000 7f6cb3fff700 1 -- 172.21.15.40:0/2980178823 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55765062c490 con 0x5576505dd6b0 2023-07-18T03:34:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.601+0000 7f6cb3fff700 1 -- 172.21.15.40:0/2980178823 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 37) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f6cc80970a0 con 0x5576505dd6b0 2023-07-18T03:34:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.601+0000 7f6ce7121b80 10 client.16221 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:34:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.601+0000 7f6ce7121b80 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:34:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.601+0000 7f6ce7121b80 20 client.16221 populate_metadata read hostname 'smithi040' 2023-07-18T03:34:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.601+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/2980178823 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5576507962e0 0x5576507986f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.601+0000 7f6ce7121b80 4 client.16221 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:34:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.601+0000 7f6ce7121b80 1 -- 172.21.15.40:0/2980178823 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576506cbe50 con 0x5576507962e0 2023-07-18T03:34:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.601+0000 7f6cd3fff700 1 --2- 172.21.15.40:0/2980178823 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5576507962e0 0x5576507986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.602+0000 7f6cb3fff700 10 client.16221 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.602+0000 7f6cd3fff700 1 --2- 172.21.15.40:0/2980178823 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5576507962e0 0x5576507986f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6f29fafe9dbf150 server_cookie=66578d9462c35a00 in_seq=0 out_seq=0 2023-07-18T03:34:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.604+0000 7f6cb3fff700 1 -- 172.21.15.40:0/2980178823 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5576506cbe50 con 0x5576507962e0 2023-07-18T03:34:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.604+0000 7f6cb3fff700 10 client.16221 handle_command_reply: tid=0 2023-07-18T03:34:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.604+0000 7f6ce7121b80 1 client.16221 shutdown 2023-07-18T03:34:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.604+0000 7f6ce7121b80 1 -- 172.21.15.40:0/2980178823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5576505dd6b0 msgr2=0x55765078a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.604+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/2980178823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5576505dd6b0 0x55765078a840 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80cb810 tx=0x7f6cc809ac40 comp rx=0 tx=0).stop 2023-07-18T03:34:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.604+0000 7f6ce7121b80 1 -- 172.21.15.40:0/2980178823 shutdown_connections 2023-07-18T03:34:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.604+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/2980178823 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5576507962e0 0x5576507986f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.605+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/2980178823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576507826e0 0x557650791dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.605+0000 7f6cb3fff700 0 client.16221 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.605+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/2980178823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5576505dd6b0 0x55765078a840 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.605+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/2980178823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557650781840 0x55765078ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.605+0000 7f6ce7121b80 1 -- 172.21.15.40:0/2980178823 >> 172.21.15.40:0/2980178823 conn(0x557650776be0 msgr2=0x55765077b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.605+0000 7f6ce7121b80 1 -- 172.21.15.40:0/2980178823 shutdown_connections 2023-07-18T03:34:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.605+0000 7f6ce7121b80 1 -- 172.21.15.40:0/2980178823 wait complete. 2023-07-18T03:34:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.605+0000 7f6ce7121b80 20 client.16221 trim_cache size 0 max 16384 2023-07-18T03:34:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.623+0000 7f6ce7121b80 1 Processor -- start 2023-07-18T03:34:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.623+0000 7f6ce7121b80 1 -- start start 2023-07-18T03:34:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.623+0000 7f6ce7121b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576505dd6b0 0x55765079d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.623+0000 7f6ce7121b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557650781840 0x557650791d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.623+0000 7f6ce7121b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576507826e0 0x5576507855b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.623+0000 7f6ce7121b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557650789cf0 con 0x557650781840 2023-07-18T03:34:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.623+0000 7f6ce7121b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765078a3b0 con 0x5576505dd6b0 2023-07-18T03:34:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.623+0000 7f6ce7121b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557650778d00 con 0x5576507826e0 2023-07-18T03:34:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.623+0000 7f6cd3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557650781840 0x557650791d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.623+0000 7f6cd9538700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576507826e0 0x5576507855b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.623+0000 7f6cd9538700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576507826e0 0x5576507855b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48912/0 (socket says 172.21.15.40:48912) 2023-07-18T03:34:43.622 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.623+0000 7f6cd3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557650781840 0x557650791d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59882/0 (socket says 172.21.15.40:59882) 2023-07-18T03:34:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.623+0000 7f6cd9538700 1 -- 172.21.15.40:0/1424899270 learned_addr learned my addr 172.21.15.40:0/1424899270 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.623+0000 7f6cd8d37700 1 --2- 172.21.15.40:0/1424899270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576505dd6b0 0x55765079d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.624+0000 7f6cd9538700 1 -- 172.21.15.40:0/1424899270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576505dd6b0 msgr2=0x55765079d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.624+0000 7f6cd9538700 1 --2- 172.21.15.40:0/1424899270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576505dd6b0 0x55765079d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.624+0000 7f6cd9538700 1 -- 172.21.15.40:0/1424899270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557650781840 msgr2=0x557650791d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.624+0000 7f6cd9538700 1 --2- 172.21.15.40:0/1424899270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557650781840 0x557650791d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.624+0000 7f6cd9538700 1 -- 172.21.15.40:0/1424899270 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576505271f0 con 0x5576507826e0 2023-07-18T03:34:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.624+0000 7f6cd3fff700 1 --2- 172.21.15.40:0/1424899270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557650781840 0x557650791d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:34:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.624+0000 7f6cd9538700 1 --2- 172.21.15.40:0/1424899270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576507826e0 0x5576507855b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc011750 tx=0x7f6ccc009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d37e8024de31a51 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.624+0000 7f6cbf7fe700 1 -- 172.21.15.40:0/1424899270 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc00e040 con 0x5576507826e0 2023-07-18T03:34:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.624+0000 7f6cbf7fe700 1 -- 172.21.15.40:0/1424899270 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ccc00f040 con 0x5576507826e0 2023-07-18T03:34:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.624+0000 7f6cbf7fe700 1 -- 172.21.15.40:0/1424899270 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0208a0 con 0x5576507826e0 2023-07-18T03:34:43.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.625+0000 7f6ce7121b80 1 -- 172.21.15.40:0/1424899270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576507826e0 msgr2=0x5576507855b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:43.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.625+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/1424899270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576507826e0 0x5576507855b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc011750 tx=0x7f6ccc009920 comp rx=0 tx=0).stop 2023-07-18T03:34:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.625+0000 7f6ce7121b80 1 -- 172.21.15.40:0/1424899270 shutdown_connections 2023-07-18T03:34:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.625+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/1424899270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576507826e0 0x5576507855b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.625+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/1424899270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576505dd6b0 0x55765079d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.625+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/1424899270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557650781840 0x557650791d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.625+0000 7f6ce7121b80 1 -- 172.21.15.40:0/1424899270 >> 172.21.15.40:0/1424899270 conn(0x557650776be0 msgr2=0x5576507895a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.625+0000 7f6ce7121b80 1 -- 172.21.15.40:0/1424899270 shutdown_connections 2023-07-18T03:34:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.625+0000 7f6ce7121b80 1 -- 172.21.15.40:0/1424899270 wait complete. 2023-07-18T03:34:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.625+0000 7f6ce7121b80 1 -- start start 2023-07-18T03:34:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.626+0000 7f6ce7121b80 1 Processor -- start 2023-07-18T03:34:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.626+0000 7f6ce7121b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576505dd6b0 0x55765078dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.626+0000 7f6ce7121b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557650781840 0x55765078e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.626+0000 7f6ce7121b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5576507826e0 0x5576507910b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.626+0000 7f6ce7121b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576507915f0 con 0x5576507826e0 2023-07-18T03:34:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.626+0000 7f6ce7121b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557650791760 con 0x557650781840 2023-07-18T03:34:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.626+0000 7f6ce7121b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557650791920 con 0x5576505dd6b0 2023-07-18T03:34:43.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.626+0000 7f6ce7121b80 10 client.0 _ll_register_callbacks cb 0x557650670700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:34:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.626+0000 7f6cd8d37700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576505dd6b0 0x55765078dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.626+0000 7f6cd9538700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5576507826e0 0x5576507910b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.626+0000 7f6cd8d37700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576505dd6b0 0x55765078dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48914/0 (socket says 172.21.15.40:48914) 2023-07-18T03:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.626+0000 7f6cd9538700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5576507826e0 0x5576507910b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59888/0 (socket says 172.21.15.40:59888) 2023-07-18T03:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.626+0000 7f6cd8d37700 1 -- 172.21.15.40:0/913417426 learned_addr learned my addr 172.21.15.40:0/913417426 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.627+0000 7f6cd8d37700 1 -- 172.21.15.40:0/913417426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557650781840 msgr2=0x55765078e4f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:34:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.627+0000 7f6cd8d37700 1 --2- 172.21.15.40:0/913417426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557650781840 0x55765078e4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.627+0000 7f6cd8d37700 1 -- 172.21.15.40:0/913417426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5576507826e0 msgr2=0x5576507910b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.627+0000 7f6cd8d37700 1 --2- 172.21.15.40:0/913417426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5576507826e0 0x5576507910b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.627+0000 7f6cd8d37700 1 -- 172.21.15.40:0/913417426 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557650500d80 con 0x5576505dd6b0 2023-07-18T03:34:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.627+0000 7f6cd9538700 1 --2- 172.21.15.40:0/913417426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5576507826e0 0x5576507910b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:34:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.627+0000 7f6cd8d37700 1 --2- 172.21.15.40:0/913417426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576505dd6b0 0x55765078dfb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6cc809b140 tx=0x7f6cc809b0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ac795723f2bdc41 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.627+0000 7f6cb3fff700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:34:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.627+0000 7f6cb3fff700 1 -- 172.21.15.40:0/913417426 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc80a10d0 con 0x5576505dd6b0 2023-07-18T03:34:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.627+0000 7f6cb3fff700 1 -- 172.21.15.40:0/913417426 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6cc80a32f0 con 0x5576505dd6b0 2023-07-18T03:34:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.634+0000 7f6ce7121b80 1 -- 172.21.15.40:0/913417426 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cc0021070 con 0x5576505dd6b0 2023-07-18T03:34:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.634+0000 7f6cb3fff700 1 -- 172.21.15.40:0/913417426 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cc80a3570 con 0x5576505dd6b0 2023-07-18T03:34:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.634+0000 7f6ce7121b80 10 client.25882 fetch_fsmap learned FSMap version 37 2023-07-18T03:34:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.634+0000 7f6ce7121b80 1 -- 172.21.15.40:0/913417426 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7f6ccc005e70 con 0x5576505dd6b0 2023-07-18T03:34:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.634+0000 7f6cb3fff700 1 -- 172.21.15.40:0/913417426 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cc0021070 con 0x5576505dd6b0 2023-07-18T03:34:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.634+0000 7f6cb3fff700 1 -- 172.21.15.40:0/913417426 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 37) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f6cc80b9240 con 0x5576505dd6b0 2023-07-18T03:34:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.634+0000 7f6ce7121b80 10 client.25882 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:34:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.634+0000 7f6ce7121b80 10 client.25882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:34:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.634+0000 7f6ce7121b80 20 client.25882 populate_metadata read hostname 'smithi040' 2023-07-18T03:34:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.634+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/913417426 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5576507962e0 0x5576507ecf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.634+0000 7f6ce7121b80 4 client.25882 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:34:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.634+0000 7f6ce7121b80 1 -- 172.21.15.40:0/913417426 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55765068dff0 con 0x5576507962e0 2023-07-18T03:34:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.635+0000 7f6cd3fff700 1 --2- 172.21.15.40:0/913417426 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5576507962e0 0x5576507ecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.635+0000 7f6cb3fff700 10 client.25882 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:43.635+0000 7f6cd3fff700 1 --2- 172.21.15.40:0/913417426 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5576507962e0 0x5576507ecf10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71b186fcb565f75a server_cookie=19013f61ddcbd3a1 in_seq=0 out_seq=0 2023-07-18T03:34:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.251+0000 7f6cb3fff700 1 -- 172.21.15.40:0/913417426 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55765068dff0 con 0x5576507962e0 2023-07-18T03:34:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.251+0000 7f6cb3fff700 10 client.25882 handle_command_reply: tid=0 2023-07-18T03:34:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.251+0000 7f6ce7121b80 1 client.25882 shutdown 2023-07-18T03:34:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.251+0000 7f6ce7121b80 1 -- 172.21.15.40:0/913417426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576505dd6b0 msgr2=0x55765078dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.251+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/913417426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576505dd6b0 0x55765078dfb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6cc809b140 tx=0x7f6cc809b0b0 comp rx=0 tx=0).stop 2023-07-18T03:34:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:34:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:34:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:34:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:34:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.255+0000 7f6ce7121b80 1 -- 172.21.15.40:0/913417426 shutdown_connections 2023-07-18T03:34:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.255+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/913417426 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5576507962e0 0x5576507ecf10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.255+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/913417426 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5576507826e0 0x5576507910b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.255+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/913417426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5576505dd6b0 0x55765078dfb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:44.255 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.255+0000 7f6ce7121b80 1 --2- 172.21.15.40:0/913417426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557650781840 0x55765078e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.255+0000 7f6ce7121b80 1 -- 172.21.15.40:0/913417426 >> 172.21.15.40:0/913417426 conn(0x557650776be0 msgr2=0x5576507855b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.255+0000 7f6cb3fff700 0 client.25882 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.255+0000 7f6ce7121b80 1 -- 172.21.15.40:0/913417426 shutdown_connections 2023-07-18T03:34:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.255+0000 7f6ce7121b80 1 -- 172.21.15.40:0/913417426 wait complete. 2023-07-18T03:34:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.256+0000 7f6ce7121b80 20 client.25882 trim_cache size 0 max 16384 2023-07-18T03:34:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.333+0000 7f6cdaf9b700 1 -- 172.21.15.40:0/1487400784 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6cc40a1150 msgr2=0x7f6cc40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.333+0000 7f6cdaf9b700 1 --2- 172.21.15.40:0/1487400784 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6cc40a1150 0x7f6cc40a3600 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4067db0 tx=0x7f6cc0009260 comp rx=0 tx=0).stop 2023-07-18T03:34:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.333+0000 7f6cdaf9b700 1 -- 172.21.15.40:0/1487400784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cd40fea00 msgr2=0x7f6cd4198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.333+0000 7f6cdaf9b700 1 --2- 172.21.15.40:0/1487400784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cd40fea00 0x7f6cd4198570 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80059d0 tx=0x7f6cc8004730 comp rx=0 tx=0).stop 2023-07-18T03:34:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.336+0000 7f6cdaf9b700 1 -- 172.21.15.40:0/1487400784 shutdown_connections 2023-07-18T03:34:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.336+0000 7f6cdaf9b700 1 --2- 172.21.15.40:0/1487400784 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6cc40a1150 0x7f6cc40a3600 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.336+0000 7f6cdaf9b700 1 --2- 172.21.15.40:0/1487400784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cd4100640 0x7f6cd419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.336+0000 7f6cdaf9b700 1 --2- 172.21.15.40:0/1487400784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6cd40fea00 0x7f6cd4198570 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.336+0000 7f6cdaf9b700 1 --2- 172.21.15.40:0/1487400784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cd40ffc90 0x7f6cd419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.336+0000 7f6cdaf9b700 1 -- 172.21.15.40:0/1487400784 >> 172.21.15.40:0/1487400784 conn(0x7f6cd40fa000 msgr2=0x7f6cd40fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.337+0000 7f6cdaf9b700 1 -- 172.21.15.40:0/1487400784 shutdown_connections 2023-07-18T03:34:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:44.337+0000 7f6cdaf9b700 1 -- 172.21.15.40:0/1487400784 wait complete. 2023-07-18T03:34:45.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:45 smithi134 ceph-mon[98216]: pgmap v409: 97 pgs: 97 active+clean; 746 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 247 op/s 2023-07-18T03:34:45.340 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:45 smithi040 ceph-mon[96439]: pgmap v409: 97 pgs: 97 active+clean; 746 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 247 op/s 2023-07-18T03:34:45.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:45 smithi064 ceph-mon[98059]: pgmap v409: 97 pgs: 97 active+clean; 746 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 247 op/s 2023-07-18T03:34:46.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:46 smithi134 ceph-mon[98216]: pgmap v410: 97 pgs: 97 active+clean; 793 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.7 MiB/s wr, 463 op/s 2023-07-18T03:34:46.586 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:46 smithi040 ceph-mon[96439]: pgmap v410: 97 pgs: 97 active+clean; 793 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.7 MiB/s wr, 463 op/s 2023-07-18T03:34:46.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:46 smithi064 ceph-mon[98059]: pgmap v410: 97 pgs: 97 active+clean; 793 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.7 MiB/s wr, 463 op/s 2023-07-18T03:34:49.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:49 smithi134 ceph-mon[98216]: pgmap v411: 97 pgs: 97 active+clean; 852 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 MiB/s wr, 752 op/s 2023-07-18T03:34:49.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:49 smithi040 ceph-mon[96439]: pgmap v411: 97 pgs: 97 active+clean; 852 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 MiB/s wr, 752 op/s 2023-07-18T03:34:49.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:49 smithi064 ceph-mon[98059]: pgmap v411: 97 pgs: 97 active+clean; 852 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 13 MiB/s wr, 752 op/s 2023-07-18T03:34:50.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:50 smithi134 ceph-mon[98216]: pgmap v412: 97 pgs: 97 active+clean; 852 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 712 op/s 2023-07-18T03:34:50.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:50 smithi040 ceph-mon[96439]: pgmap v412: 97 pgs: 97 active+clean; 852 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 712 op/s 2023-07-18T03:34:50.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:50 smithi064 ceph-mon[98059]: pgmap v412: 97 pgs: 97 active+clean; 852 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 712 op/s 2023-07-18T03:34:53.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:52 smithi134 ceph-mon[98216]: pgmap v413: 97 pgs: 97 active+clean; 853 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 723 op/s 2023-07-18T03:34:53.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:52 smithi040 ceph-mon[96439]: pgmap v413: 97 pgs: 97 active+clean; 853 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 723 op/s 2023-07-18T03:34:53.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:52 smithi064 ceph-mon[98059]: pgmap v413: 97 pgs: 97 active+clean; 853 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 723 op/s 2023-07-18T03:34:53.169 INFO:tasks.workunit.client.0.smithi040.stdout: 226 2283401 466 1613353 408 478542 410 2023-07-18T03:34:55.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:55 smithi134 ceph-mon[98216]: pgmap v414: 97 pgs: 97 active+clean; 856 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 712 op/s 2023-07-18T03:34:55.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:55 smithi040 ceph-mon[96439]: pgmap v414: 97 pgs: 97 active+clean; 856 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 712 op/s 2023-07-18T03:34:55.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:55 smithi064 ceph-mon[98059]: pgmap v414: 97 pgs: 97 active+clean; 856 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 712 op/s 2023-07-18T03:34:56.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:56 smithi134 ceph-mon[98216]: pgmap v415: 97 pgs: 97 active+clean; 856 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.8 MiB/s wr, 590 op/s 2023-07-18T03:34:56.588 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:56 smithi040 ceph-mon[96439]: pgmap v415: 97 pgs: 97 active+clean; 856 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.8 MiB/s wr, 590 op/s 2023-07-18T03:34:56.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:56 smithi064 ceph-mon[98059]: pgmap v415: 97 pgs: 97 active+clean; 856 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.8 MiB/s wr, 590 op/s 2023-07-18T03:34:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.650+0000 7f4097e20700 1 -- 172.21.15.40:0/2984149559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40900fddc0 msgr2=0x7f40900fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.650+0000 7f4097e20700 1 --2- 172.21.15.40:0/2984149559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40900fddc0 0x7f40900fe1d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f408c0098f0 tx=0x7f408c009c00 comp rx=0 tx=0).stop 2023-07-18T03:34:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.650+0000 7f4097e20700 1 -- 172.21.15.40:0/2984149559 shutdown_connections 2023-07-18T03:34:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.650+0000 7f4097e20700 1 --2- 172.21.15.40:0/2984149559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40900ff4d0 0x7f4090103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.650+0000 7f4097e20700 1 --2- 172.21.15.40:0/2984149559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40900fddc0 0x7f40900fe1d0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.650+0000 7f4097e20700 1 --2- 172.21.15.40:0/2984149559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40900feb20 0x7f40900fef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.650+0000 7f4097e20700 1 -- 172.21.15.40:0/2984149559 >> 172.21.15.40:0/2984149559 conn(0x7f40900f9780 msgr2=0x7f40900fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.650+0000 7f4097e20700 1 -- 172.21.15.40:0/2984149559 shutdown_connections 2023-07-18T03:34:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.650+0000 7f4097e20700 1 -- 172.21.15.40:0/2984149559 wait complete. 2023-07-18T03:34:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.651+0000 7f4097e20700 1 Processor -- start 2023-07-18T03:34:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.651+0000 7f4097e20700 1 -- start start 2023-07-18T03:34:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.651+0000 7f4097e20700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40900fddc0 0x7f409019bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.651+0000 7f4097e20700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40900feb20 0x7f409019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.651+0000 7f4097e20700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40900ff4d0 0x7f409019d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.651+0000 7f4097e20700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409019d9d0 con 0x7f40900ff4d0 2023-07-18T03:34:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.651+0000 7f4097e20700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409019db40 con 0x7f40900feb20 2023-07-18T03:34:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.651+0000 7f4097e20700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409019dd00 con 0x7f40900fddc0 2023-07-18T03:34:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.651+0000 7f40963bd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40900ff4d0 0x7f409019d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.651+0000 7f40963bd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40900ff4d0 0x7f409019d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59898/0 (socket says 172.21.15.40:59898) 2023-07-18T03:34:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.651+0000 7f4095bbc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40900fddc0 0x7f409019bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.651+0000 7f40963bd700 1 -- 172.21.15.40:0/1015178623 learned_addr learned my addr 172.21.15.40:0/1015178623 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.652+0000 7f40953bb700 1 --2- 172.21.15.40:0/1015178623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40900feb20 0x7f409019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.652+0000 7f40963bd700 1 -- 172.21.15.40:0/1015178623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40900fddc0 msgr2=0x7f409019bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.652+0000 7f40963bd700 1 --2- 172.21.15.40:0/1015178623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40900fddc0 0x7f409019bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.652+0000 7f40963bd700 1 -- 172.21.15.40:0/1015178623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40900feb20 msgr2=0x7f409019c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.652+0000 7f40963bd700 1 --2- 172.21.15.40:0/1015178623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40900feb20 0x7f409019c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.652+0000 7f40963bd700 1 -- 172.21.15.40:0/1015178623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c009520 con 0x7f40900ff4d0 2023-07-18T03:34:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.652+0000 7f4095bbc700 1 --2- 172.21.15.40:0/1015178623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40900fddc0 0x7f409019bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:34:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.652+0000 7f40963bd700 1 --2- 172.21.15.40:0/1015178623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40900ff4d0 0x7f409019d370 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f408800c110 tx=0x7f408800c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.652+0000 7f4086ffd700 1 -- 172.21.15.40:0/1015178623 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4088017070 con 0x7f40900ff4d0 2023-07-18T03:34:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.652+0000 7f4086ffd700 1 -- 172.21.15.40:0/1015178623 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f408800e050 con 0x7f40900ff4d0 2023-07-18T03:34:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.652+0000 7f4086ffd700 1 -- 172.21.15.40:0/1015178623 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4088007940 con 0x7f40900ff4d0 2023-07-18T03:34:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.652+0000 7f4097e20700 1 -- 172.21.15.40:0/1015178623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409019df80 con 0x7f40900ff4d0 2023-07-18T03:34:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.653+0000 7f4097e20700 1 -- 172.21.15.40:0/1015178623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40901a47a0 con 0x7f40900ff4d0 2023-07-18T03:34:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.657+0000 7f4086ffd700 1 -- 172.21.15.40:0/1015178623 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4088004ad0 con 0x7f40900ff4d0 2023-07-18T03:34:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.657+0000 7f4086ffd700 1 --2- 172.21.15.40:0/1015178623 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f407c0a1150 0x7f407c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.658+0000 7f4086ffd700 1 -- 172.21.15.40:0/1015178623 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f40880cf170 con 0x7f40900ff4d0 2023-07-18T03:34:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.659+0000 7f40a3fa6b80 1 Processor -- start 2023-07-18T03:34:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.659+0000 7f40a3fa6b80 1 -- start start 2023-07-18T03:34:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.659+0000 7f40a3fa6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe281300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.659+0000 7f40a3fa6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe281840 0x55fdfe2821a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.659+0000 7f40a3fa6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe2826e0 0x55fdfe282b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.659+0000 7f40a3fa6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfe186c50 con 0x55fdfe281840 2023-07-18T03:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.659+0000 7f40a3fa6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfe1d99e0 con 0x55fdfe0dd6b0 2023-07-18T03:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.659+0000 7f40a3fa6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfe1d2e70 con 0x55fdfe2826e0 2023-07-18T03:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.659+0000 7f4095bbc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe281300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.659+0000 7f4095bbc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe281300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43232/0 (socket says 172.21.15.40:43232) 2023-07-18T03:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.659+0000 7f4095bbc700 1 -- 172.21.15.40:0/4056302636 learned_addr learned my addr 172.21.15.40:0/4056302636 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.659+0000 7f40963bd700 1 --2- 172.21.15.40:0/4056302636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe2826e0 0x55fdfe282b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.660+0000 7f4095bbc700 1 -- 172.21.15.40:0/4056302636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe2826e0 msgr2=0x55fdfe282b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.660+0000 7f4095bbc700 1 --2- 172.21.15.40:0/4056302636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe2826e0 0x55fdfe282b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.660+0000 7f4095bbc700 1 -- 172.21.15.40:0/4056302636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe281840 msgr2=0x55fdfe2821a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.660+0000 7f4095bbc700 1 --2- 172.21.15.40:0/4056302636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe281840 0x55fdfe2821a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.660+0000 7f4095bbc700 1 -- 172.21.15.40:0/4056302636 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdfe000d80 con 0x55fdfe0dd6b0 2023-07-18T03:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.660+0000 7f40963bd700 1 --2- 172.21.15.40:0/4056302636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe2826e0 0x55fdfe282b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.660+0000 7f4095bbc700 1 --2- 172.21.15.40:0/1015178623 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f407c0a1150 0x7f407c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.660+0000 7f4095bbc700 1 --2- 172.21.15.40:0/4056302636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe281300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f408c009fd0 tx=0x7f408c00bfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bed038a4352f5df8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.660+0000 7f407bfff700 1 -- 172.21.15.40:0/4056302636 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c00f780 con 0x55fdfe0dd6b0 2023-07-18T03:34:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.660+0000 7f407bfff700 1 -- 172.21.15.40:0/4056302636 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f408c00bbd0 con 0x55fdfe0dd6b0 2023-07-18T03:34:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.660+0000 7f407bfff700 1 -- 172.21.15.40:0/4056302636 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c004910 con 0x55fdfe0dd6b0 2023-07-18T03:34:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.661+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/4056302636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe0dd6b0 msgr2=0x55fdfe281300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.661+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/4056302636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe281300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f408c009fd0 tx=0x7f408c00bfb0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.661+0000 7f4095bbc700 1 --2- 172.21.15.40:0/1015178623 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f407c0a1150 0x7f407c0a3600 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f408c011040 tx=0x7f408c026000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.661+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/4056302636 shutdown_connections 2023-07-18T03:34:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.661+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/4056302636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe2826e0 0x55fdfe282b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.661+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/4056302636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe281300 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.661+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/4056302636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe281840 0x55fdfe2821a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.661+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/4056302636 >> 172.21.15.40:0/4056302636 conn(0x55fdfe276be0 msgr2=0x55fdfe27bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.661+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/4056302636 shutdown_connections 2023-07-18T03:34:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.661+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/4056302636 wait complete. 2023-07-18T03:34:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.662+0000 7f40a3fa6b80 1 -- start start 2023-07-18T03:34:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.662+0000 7f40a3fa6b80 1 Processor -- start 2023-07-18T03:34:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.663+0000 7f40a3fa6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe28a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.663+0000 7f40a3fa6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe281840 0x55fdfe28ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.663+0000 7f40a3fa6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe2826e0 0x55fdfe291dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.663+0000 7f40a3fa6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfe172a60 con 0x55fdfe0dd6b0 2023-07-18T03:34:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.663+0000 7f40a3fa6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfe172760 con 0x55fdfe281840 2023-07-18T03:34:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.663+0000 7f40a3fa6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfe0568c0 con 0x55fdfe2826e0 2023-07-18T03:34:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.663+0000 7f40a3fa6b80 10 client.0 _ll_register_callbacks cb 0x55fdfe170700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:34:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.663+0000 7f40953bb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe281840 0x55fdfe28ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.663+0000 7f40953bb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe281840 0x55fdfe28ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43240/0 (socket says 172.21.15.40:43240) 2023-07-18T03:34:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.663+0000 7f40953bb700 1 -- 172.21.15.40:0/371345663 learned_addr learned my addr 172.21.15.40:0/371345663 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.663+0000 7f4095bbc700 1 --2- 172.21.15.40:0/371345663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe28a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.663+0000 7f40953bb700 1 -- 172.21.15.40:0/371345663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe2826e0 msgr2=0x55fdfe291dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:34:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.663+0000 7f40953bb700 1 --2- 172.21.15.40:0/371345663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe2826e0 0x55fdfe291dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.663+0000 7f40953bb700 1 -- 172.21.15.40:0/371345663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe0dd6b0 msgr2=0x55fdfe28a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.663+0000 7f40953bb700 1 --2- 172.21.15.40:0/371345663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe28a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.663+0000 7f40953bb700 1 -- 172.21.15.40:0/371345663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40901047b0 con 0x55fdfe281840 2023-07-18T03:34:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.664+0000 7f40953bb700 1 --2- 172.21.15.40:0/371345663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe281840 0x55fdfe28ad90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f40800116c0 tx=0x7f40800119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e738ac44568c56f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.664+0000 7f40797fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:34:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.664+0000 7f40797fa700 1 -- 172.21.15.40:0/371345663 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408000e5c0 con 0x55fdfe281840 2023-07-18T03:34:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.664+0000 7f40797fa700 1 -- 172.21.15.40:0/371345663 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f408000e720 con 0x55fdfe281840 2023-07-18T03:34:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.664+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/371345663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fdfe12c490 con 0x55fdfe281840 2023-07-18T03:34:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.664+0000 7f40797fa700 1 -- 172.21.15.40:0/371345663 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f408000e920 con 0x55fdfe281840 2023-07-18T03:34:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.664+0000 7f40a3fa6b80 10 client.25871 fetch_fsmap learned FSMap version 37 2023-07-18T03:34:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.664+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/371345663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55fdfdfe9c00 con 0x55fdfe281840 2023-07-18T03:34:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.664+0000 7f40797fa700 1 -- 172.21.15.40:0/371345663 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4080028080 con 0x55fdfe281840 2023-07-18T03:34:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.665+0000 7f40797fa700 1 -- 172.21.15.40:0/371345663 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 37) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f4080022030 con 0x55fdfe281840 2023-07-18T03:34:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.665+0000 7f40a3fa6b80 10 client.25871 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:34:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.665+0000 7f40a3fa6b80 10 client.25871 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:34:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.665+0000 7f40a3fa6b80 20 client.25871 populate_metadata read hostname 'smithi040' 2023-07-18T03:34:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.665+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/371345663 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fdfe296720 0x55fdfe298b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.665+0000 7f40a3fa6b80 4 client.25871 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:34:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.665+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/371345663 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fdfe1cbe50 con 0x55fdfe296720 2023-07-18T03:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.665+0000 7f4095bbc700 1 --2- 172.21.15.40:0/371345663 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fdfe296720 0x55fdfe298b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.665+0000 7f40797fa700 10 client.25871 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.666+0000 7f4095bbc700 1 --2- 172.21.15.40:0/371345663 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fdfe296720 0x55fdfe298b30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fd9f9285f837a2f server_cookie=701e798180fe1f2f in_seq=0 out_seq=0 2023-07-18T03:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.667+0000 7f40797fa700 1 -- 172.21.15.40:0/371345663 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fdfe1cbe50 con 0x55fdfe296720 2023-07-18T03:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.667+0000 7f40797fa700 10 client.25871 handle_command_reply: tid=0 2023-07-18T03:34:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.667+0000 7f40a3fa6b80 1 client.25871 shutdown 2023-07-18T03:34:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.667+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/371345663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe281840 msgr2=0x55fdfe28ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.667+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/371345663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe281840 0x55fdfe28ad90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f40800116c0 tx=0x7f40800119d0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.667+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/371345663 shutdown_connections 2023-07-18T03:34:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.667+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/371345663 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fdfe296720 0x55fdfe298b30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.667+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/371345663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe2826e0 0x55fdfe291dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.667+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/371345663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe28a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.667+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/371345663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe281840 0x55fdfe28ad90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.667+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/371345663 >> 172.21.15.40:0/371345663 conn(0x55fdfe276be0 msgr2=0x55fdfe27b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.667+0000 7f40797fa700 0 client.25871 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.668+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/371345663 shutdown_connections 2023-07-18T03:34:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.668+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/371345663 wait complete. 2023-07-18T03:34:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.668+0000 7f40a3fa6b80 20 client.25871 trim_cache size 0 max 16384 2023-07-18T03:34:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.700+0000 7f40a3fa6b80 1 Processor -- start 2023-07-18T03:34:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.700+0000 7f40a3fa6b80 1 -- start start 2023-07-18T03:34:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.700+0000 7f40a3fa6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe282ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.700+0000 7f40a3fa6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe281840 0x55fdfe291d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.700+0000 7f40a3fa6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe2826e0 0x55fdfe288a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.700+0000 7f40a3fa6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfe28b9a0 con 0x55fdfe281840 2023-07-18T03:34:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.700+0000 7f40a3fa6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfe2939d0 con 0x55fdfe2826e0 2023-07-18T03:34:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.700+0000 7f40a3fa6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfe28b4d0 con 0x55fdfe0dd6b0 2023-07-18T03:34:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.701+0000 7f40953bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe281840 0x55fdfe291d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.701+0000 7f4095bbc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe282ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.701+0000 7f40963bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe2826e0 0x55fdfe288a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.701+0000 7f40953bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe281840 0x55fdfe291d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59920/0 (socket says 172.21.15.40:59920) 2023-07-18T03:34:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.701+0000 7f40963bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe2826e0 0x55fdfe288a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43248/0 (socket says 172.21.15.40:43248) 2023-07-18T03:34:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.701+0000 7f40953bb700 1 -- 172.21.15.40:0/1724883129 learned_addr learned my addr 172.21.15.40:0/1724883129 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.701+0000 7f4095bbc700 1 -- 172.21.15.40:0/1724883129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe2826e0 msgr2=0x55fdfe288a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.701+0000 7f4095bbc700 1 --2- 172.21.15.40:0/1724883129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe2826e0 0x55fdfe288a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.701+0000 7f4095bbc700 1 -- 172.21.15.40:0/1724883129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe281840 msgr2=0x55fdfe291d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.701+0000 7f4095bbc700 1 --2- 172.21.15.40:0/1724883129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe281840 0x55fdfe291d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.701+0000 7f4095bbc700 1 -- 172.21.15.40:0/1724883129 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdfe0271f0 con 0x55fdfe0dd6b0 2023-07-18T03:34:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.701+0000 7f4095bbc700 1 --2- 172.21.15.40:0/1724883129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe282ab0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f408c010e30 tx=0x7f408c019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c56d53c00378d080 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.701+0000 7f4084ff9700 1 -- 172.21.15.40:0/1724883129 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c0048d0 con 0x55fdfe0dd6b0 2023-07-18T03:34:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.702+0000 7f4084ff9700 1 -- 172.21.15.40:0/1724883129 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f408c01b040 con 0x55fdfe0dd6b0 2023-07-18T03:34:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.702+0000 7f4084ff9700 1 -- 172.21.15.40:0/1724883129 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c03bd40 con 0x55fdfe0dd6b0 2023-07-18T03:34:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.702+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/1724883129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe0dd6b0 msgr2=0x55fdfe282ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.702+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/1724883129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe282ab0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f408c010e30 tx=0x7f408c019040 comp rx=0 tx=0).stop 2023-07-18T03:34:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.702+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/1724883129 shutdown_connections 2023-07-18T03:34:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.702+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/1724883129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe2826e0 0x55fdfe288a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.703 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.702+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/1724883129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe282ab0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.702+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/1724883129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe281840 0x55fdfe291d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.702+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/1724883129 >> 172.21.15.40:0/1724883129 conn(0x55fdfe276be0 msgr2=0x55fdfe29d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.703+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/1724883129 shutdown_connections 2023-07-18T03:34:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.703+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/1724883129 wait complete. 2023-07-18T03:34:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.703+0000 7f40a3fa6b80 1 -- start start 2023-07-18T03:34:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.703+0000 7f40a3fa6b80 1 Processor -- start 2023-07-18T03:34:57.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.703+0000 7f40a3fa6b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe28d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.703+0000 7f40a3fa6b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe281840 0x55fdfe28dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.703+0000 7f40a3fa6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe2826e0 0x55fdfe290c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.703+0000 7f40a3fa6b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfe2911d0 con 0x55fdfe281840 2023-07-18T03:34:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.703+0000 7f40a3fa6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfe291340 con 0x55fdfe2826e0 2023-07-18T03:34:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.703+0000 7f40a3fa6b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdfe291500 con 0x55fdfe0dd6b0 2023-07-18T03:34:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.703+0000 7f40a3fa6b80 10 client.0 _ll_register_callbacks cb 0x7f4080009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:34:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.704+0000 7f40953bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe281840 0x55fdfe28dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.704+0000 7f40963bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe2826e0 0x55fdfe290c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.704+0000 7f40963bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe2826e0 0x55fdfe290c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43254/0 (socket says 172.21.15.40:43254) 2023-07-18T03:34:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.704+0000 7f40953bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe281840 0x55fdfe28dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59924/0 (socket says 172.21.15.40:59924) 2023-07-18T03:34:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.704+0000 7f40963bd700 1 -- 172.21.15.40:0/2790837300 learned_addr learned my addr 172.21.15.40:0/2790837300 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:34:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.704+0000 7f40963bd700 1 -- 172.21.15.40:0/2790837300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe0dd6b0 msgr2=0x55fdfe28d730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:34:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.704+0000 7f40963bd700 1 --2- 172.21.15.40:0/2790837300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe28d730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.704+0000 7f40963bd700 1 -- 172.21.15.40:0/2790837300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe281840 msgr2=0x55fdfe28dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:34:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.704+0000 7f40963bd700 1 --2- 172.21.15.40:0/2790837300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe281840 0x55fdfe28dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.704+0000 7f40963bd700 1 -- 172.21.15.40:0/2790837300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408c033040 con 0x55fdfe2826e0 2023-07-18T03:34:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.704+0000 7f40953bb700 1 --2- 172.21.15.40:0/2790837300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe281840 0x55fdfe28dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:34:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.704+0000 7f40963bd700 1 --2- 172.21.15.40:0/2790837300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe2826e0 0x55fdfe290c90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4088009fd0 tx=0x7f40880994f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbacc9852b145eca server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:34:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.704+0000 7f40797fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:34:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.704+0000 7f40797fa700 1 -- 172.21.15.40:0/2790837300 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40880a1170 con 0x55fdfe2826e0 2023-07-18T03:34:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.704+0000 7f40797fa700 1 -- 172.21.15.40:0/2790837300 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4088098fb0 con 0x55fdfe2826e0 2023-07-18T03:34:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.704+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/2790837300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f408c00bbd0 con 0x55fdfe2826e0 2023-07-18T03:34:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.705+0000 7f40797fa700 1 -- 172.21.15.40:0/2790837300 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f408809cfe0 con 0x55fdfe2826e0 2023-07-18T03:34:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.705+0000 7f40a3fa6b80 10 client.25877 fetch_fsmap learned FSMap version 37 2023-07-18T03:34:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.705+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/2790837300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55fdfdfff250 con 0x55fdfe2826e0 2023-07-18T03:34:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.705+0000 7f40797fa700 1 -- 172.21.15.40:0/2790837300 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40880c7080 con 0x55fdfe2826e0 2023-07-18T03:34:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.705+0000 7f40797fa700 1 -- 172.21.15.40:0/2790837300 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 37) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f408800cad0 con 0x55fdfe2826e0 2023-07-18T03:34:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.705+0000 7f40a3fa6b80 10 client.25877 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:34:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.705+0000 7f40a3fa6b80 10 client.25877 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:34:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.705+0000 7f40a3fa6b80 20 client.25877 populate_metadata read hostname 'smithi040' 2023-07-18T03:34:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.705+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/2790837300 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fdfe2fc2e0 0x55fdfe2fe6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:34:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.705+0000 7f40a3fa6b80 4 client.25877 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:34:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.705+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/2790837300 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fdfe18dff0 con 0x55fdfe2fc2e0 2023-07-18T03:34:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.705+0000 7f4095bbc700 1 --2- 172.21.15.40:0/2790837300 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fdfe2fc2e0 0x55fdfe2fe6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:34:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.706+0000 7f40797fa700 10 client.25877 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:57.706+0000 7f4095bbc700 1 --2- 172.21.15.40:0/2790837300 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fdfe2fc2e0 0x55fdfe2fe6f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64f4a9b66192e068 server_cookie=758d6efcdaaf4cf5 in_seq=0 out_seq=0 2023-07-18T03:34:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.259+0000 7f40797fa700 1 -- 172.21.15.40:0/2790837300 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fdfe18dff0 con 0x55fdfe2fc2e0 2023-07-18T03:34:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.259+0000 7f40797fa700 10 client.25877 handle_command_reply: tid=0 2023-07-18T03:34:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.259+0000 7f40a3fa6b80 1 client.25877 shutdown 2023-07-18T03:34:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.259+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/2790837300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe2826e0 msgr2=0x55fdfe290c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.259+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/2790837300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe2826e0 0x55fdfe290c90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4088009fd0 tx=0x7f40880994f0 comp rx=0 tx=0).stop 2023-07-18T03:34:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.259+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/2790837300 shutdown_connections 2023-07-18T03:34:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.259+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/2790837300 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fdfe2fc2e0 0x55fdfe2fe6f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.259+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/2790837300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fdfe2826e0 0x55fdfe290c90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.259+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/2790837300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdfe0dd6b0 0x55fdfe28d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.259+0000 7f40a3fa6b80 1 --2- 172.21.15.40:0/2790837300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fdfe281840 0x55fdfe28dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.259+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/2790837300 >> 172.21.15.40:0/2790837300 conn(0x55fdfe276be0 msgr2=0x55fdfe288a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.260+0000 7f40797fa700 0 client.25877 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:34:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.260+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/2790837300 shutdown_connections 2023-07-18T03:34:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.260+0000 7f40a3fa6b80 1 -- 172.21.15.40:0/2790837300 wait complete. 2023-07-18T03:34:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.260+0000 7f40a3fa6b80 20 client.25877 trim_cache size 0 max 16384 2023-07-18T03:34:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:34:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:34:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:34:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:34:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.330+0000 7f4097e20700 1 -- 172.21.15.40:0/1015178623 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f407c0a1150 msgr2=0x7f407c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.330+0000 7f4097e20700 1 --2- 172.21.15.40:0/1015178623 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f407c0a1150 0x7f407c0a3600 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f408c011040 tx=0x7f408c026000 comp rx=0 tx=0).stop 2023-07-18T03:34:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.330+0000 7f4097e20700 1 -- 172.21.15.40:0/1015178623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40900ff4d0 msgr2=0x7f409019d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:34:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.330+0000 7f4097e20700 1 --2- 172.21.15.40:0/1015178623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40900ff4d0 0x7f409019d370 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f408800c110 tx=0x7f408800c420 comp rx=0 tx=0).stop 2023-07-18T03:34:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.331+0000 7f4097e20700 1 -- 172.21.15.40:0/1015178623 shutdown_connections 2023-07-18T03:34:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.331+0000 7f4097e20700 1 --2- 172.21.15.40:0/1015178623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40900ff4d0 0x7f409019d370 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.331+0000 7f4097e20700 1 --2- 172.21.15.40:0/1015178623 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f407c0a1150 0x7f407c0a3600 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.331+0000 7f4097e20700 1 --2- 172.21.15.40:0/1015178623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40900fddc0 0x7f409019bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.331+0000 7f4097e20700 1 --2- 172.21.15.40:0/1015178623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40900feb20 0x7f409019c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:34:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.331+0000 7f4097e20700 1 -- 172.21.15.40:0/1015178623 >> 172.21.15.40:0/1015178623 conn(0x7f40900f9780 msgr2=0x7f40900fa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:34:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.332+0000 7f4097e20700 1 -- 172.21.15.40:0/1015178623 shutdown_connections 2023-07-18T03:34:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:34:58.332+0000 7f4097e20700 1 -- 172.21.15.40:0/1015178623 wait complete. 2023-07-18T03:34:59.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:34:59 smithi134 ceph-mon[98216]: pgmap v416: 97 pgs: 97 active+clean; 862 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 406 op/s 2023-07-18T03:34:59.586 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:34:59 smithi040 ceph-mon[96439]: pgmap v416: 97 pgs: 97 active+clean; 862 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 406 op/s 2023-07-18T03:34:59.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:34:59 smithi064 ceph-mon[98059]: pgmap v416: 97 pgs: 97 active+clean; 862 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 406 op/s 2023-07-18T03:35:00.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:00 smithi134 ceph-mon[98216]: pgmap v417: 97 pgs: 97 active+clean; 862 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 129 op/s 2023-07-18T03:35:00.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:00 smithi040 ceph-mon[96439]: pgmap v417: 97 pgs: 97 active+clean; 862 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 129 op/s 2023-07-18T03:35:00.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:00 smithi064 ceph-mon[98059]: pgmap v417: 97 pgs: 97 active+clean; 862 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 129 op/s 2023-07-18T03:35:03.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:02 smithi134 ceph-mon[98216]: pgmap v418: 97 pgs: 97 active+clean; 865 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 271 op/s 2023-07-18T03:35:03.082 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:02 smithi040 ceph-mon[96439]: pgmap v418: 97 pgs: 97 active+clean; 865 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 271 op/s 2023-07-18T03:35:03.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:02 smithi064 ceph-mon[98059]: pgmap v418: 97 pgs: 97 active+clean; 865 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 271 op/s 2023-07-18T03:35:03.170 INFO:tasks.workunit.client.0.smithi040.stdout: 240 2466671 681 1736417 629 522299 873 2023-07-18T03:35:05.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:05 smithi134 ceph-mon[98216]: pgmap v419: 97 pgs: 97 active+clean; 863 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 331 op/s 2023-07-18T03:35:05.587 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:05 smithi040 ceph-mon[96439]: pgmap v419: 97 pgs: 97 active+clean; 863 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 331 op/s 2023-07-18T03:35:05.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:05 smithi064 ceph-mon[98059]: pgmap v419: 97 pgs: 97 active+clean; 863 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 331 op/s 2023-07-18T03:35:06.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:06 smithi134 ceph-mon[98216]: pgmap v420: 97 pgs: 97 active+clean; 864 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 365 op/s 2023-07-18T03:35:06.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:06 smithi064 ceph-mon[98059]: pgmap v420: 97 pgs: 97 active+clean; 864 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 365 op/s 2023-07-18T03:35:06.601 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:06 smithi040 ceph-mon[96439]: pgmap v420: 97 pgs: 97 active+clean; 864 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 365 op/s 2023-07-18T03:35:07.019 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:35:08.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.679+0000 7f726f41e700 1 -- 172.21.15.40:0/1456087881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7268070ee0 msgr2=0x7f72680712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:08.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.679+0000 7f726f41e700 1 --2- 172.21.15.40:0/1456087881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7268070ee0 0x7f72680712f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f72640098f0 tx=0x7f7264009c00 comp rx=0 tx=0).stop 2023-07-18T03:35:08.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.695+0000 7f726f41e700 1 -- 172.21.15.40:0/1456087881 shutdown_connections 2023-07-18T03:35:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.695+0000 7f726f41e700 1 --2- 172.21.15.40:0/1456087881 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f726810bd60 0x7f726810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.695+0000 7f726f41e700 1 --2- 172.21.15.40:0/1456087881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7268070ee0 0x7f72680712f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.695+0000 7f726f41e700 1 --2- 172.21.15.40:0/1456087881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7268071830 0x7f726810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.695+0000 7f726f41e700 1 -- 172.21.15.40:0/1456087881 >> 172.21.15.40:0/1456087881 conn(0x7f726806ca90 msgr2=0x7f726806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:08.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.700+0000 7f726f41e700 1 -- 172.21.15.40:0/1456087881 shutdown_connections 2023-07-18T03:35:08.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.700+0000 7f726f41e700 1 -- 172.21.15.40:0/1456087881 wait complete. 2023-07-18T03:35:08.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.702+0000 7f726f41e700 1 Processor -- start 2023-07-18T03:35:08.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.702+0000 7f726f41e700 1 -- start start 2023-07-18T03:35:08.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.702+0000 7f726f41e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7268070ee0 0x7f72681a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.702+0000 7f726f41e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7268071830 0x7f72681a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.702+0000 7f726f41e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f726810bd60 0x7f72681abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.702+0000 7f726f41e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72681ac2e0 con 0x7f7268070ee0 2023-07-18T03:35:08.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.702+0000 7f726f41e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726810ec40 con 0x7f7268071830 2023-07-18T03:35:08.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.702+0000 7f726f41e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726810edd0 con 0x7f726810bd60 2023-07-18T03:35:08.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.703+0000 7f726d1ba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7268070ee0 0x7f72681a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:08.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.703+0000 7f726d1ba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7268070ee0 0x7f72681a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59938/0 (socket says 172.21.15.40:59938) 2023-07-18T03:35:08.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.703+0000 7f726d1ba700 1 -- 172.21.15.40:0/4056196519 learned_addr learned my addr 172.21.15.40:0/4056196519 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:08.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.711+0000 7f726d9bb700 1 --2- 172.21.15.40:0/4056196519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f726810bd60 0x7f72681abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:08.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.711+0000 7f726c9b9700 1 --2- 172.21.15.40:0/4056196519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7268071830 0x7f72681a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:08.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.713+0000 7f726d1ba700 1 -- 172.21.15.40:0/4056196519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f726810bd60 msgr2=0x7f72681abd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:08.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.713+0000 7f726d1ba700 1 --2- 172.21.15.40:0/4056196519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f726810bd60 0x7f72681abd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.713+0000 7f726d1ba700 1 -- 172.21.15.40:0/4056196519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7268071830 msgr2=0x7f72681a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:08.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.713+0000 7f726d1ba700 1 --2- 172.21.15.40:0/4056196519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7268071830 0x7f72681a4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.713+0000 7f726d1ba700 1 -- 172.21.15.40:0/4056196519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7264009520 con 0x7f7268070ee0 2023-07-18T03:35:08.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.722+0000 7f726d1ba700 1 --2- 172.21.15.40:0/4056196519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7268070ee0 0x7f72681a4760 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f726400b790 tx=0x7f726400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:08.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.734+0000 7f725e7fc700 1 -- 172.21.15.40:0/4056196519 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726401c070 con 0x7f7268070ee0 2023-07-18T03:35:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.734+0000 7f725e7fc700 1 -- 172.21.15.40:0/4056196519 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f726400f7d0 con 0x7f7268070ee0 2023-07-18T03:35:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.734+0000 7f725e7fc700 1 -- 172.21.15.40:0/4056196519 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7264004a80 con 0x7f7268070ee0 2023-07-18T03:35:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.736+0000 7f726f41e700 1 -- 172.21.15.40:0/4056196519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726810f050 con 0x7f7268070ee0 2023-07-18T03:35:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.736+0000 7f726f41e700 1 -- 172.21.15.40:0/4056196519 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f726810f4b0 con 0x7f7268070ee0 2023-07-18T03:35:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.743+0000 7f727b5a4b80 1 Processor -- start 2023-07-18T03:35:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.743+0000 7f727b5a4b80 1 -- start start 2023-07-18T03:35:08.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.744+0000 7f727b5a4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10e4d0 0x5620df10e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.744+0000 7f727b5a4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10ee20 0x5620df10f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.744+0000 7f727b5a4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10fc70 0x5620df1100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.744+0000 7f727b5a4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620df013cb0 con 0x5620df10fc70 2023-07-18T03:35:08.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.744+0000 7f727b5a4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620df066a40 con 0x5620df10ee20 2023-07-18T03:35:08.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.744+0000 7f727b5a4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620df05fed0 con 0x5620df10e4d0 2023-07-18T03:35:08.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.745+0000 7f726d9bb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10e4d0 0x5620df10e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:08.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.745+0000 7f726c9b9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10fc70 0x5620df1100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:08.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.745+0000 7f726c9b9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10fc70 0x5620df1100e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59944/0 (socket says 172.21.15.40:59944) 2023-07-18T03:35:08.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.746+0000 7f726c9b9700 1 -- 172.21.15.40:0/3898938919 learned_addr learned my addr 172.21.15.40:0/3898938919 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:08.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.746+0000 7f726d1ba700 1 --2- 172.21.15.40:0/3898938919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10ee20 0x5620df10f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:08.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.753+0000 7f725e7fc700 1 -- 172.21.15.40:0/4056196519 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7264020070 con 0x7f7268070ee0 2023-07-18T03:35:08.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.754+0000 7f725e7fc700 1 --2- 172.21.15.40:0/4056196519 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72540a1350 0x7f72540a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.755+0000 7f725e7fc700 1 -- 172.21.15.40:0/4056196519 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f72640d2c70 con 0x7f7268070ee0 2023-07-18T03:35:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.755+0000 7f726d1ba700 1 -- 172.21.15.40:0/3898938919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10e4d0 msgr2=0x5620df10e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.755+0000 7f726d1ba700 1 --2- 172.21.15.40:0/3898938919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10e4d0 0x5620df10e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.755+0000 7f726d1ba700 1 -- 172.21.15.40:0/3898938919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10fc70 msgr2=0x5620df1100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:08.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.755+0000 7f726d1ba700 1 --2- 172.21.15.40:0/3898938919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10fc70 0x5620df1100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.755+0000 7f726d1ba700 1 -- 172.21.15.40:0/3898938919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620dee8dde0 con 0x5620df10ee20 2023-07-18T03:35:08.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.758+0000 7f726d1ba700 1 --2- 172.21.15.40:0/3898938919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10ee20 0x5620df10f6e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f72640d0690 tx=0x7f726409c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99839732366af51c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:08.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.765+0000 7f72537fe700 1 -- 172.21.15.40:0/3898938919 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726409f960 con 0x5620df10ee20 2023-07-18T03:35:08.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.765+0000 7f72537fe700 1 -- 172.21.15.40:0/3898938919 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72640a0000 con 0x5620df10ee20 2023-07-18T03:35:08.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.765+0000 7f72537fe700 1 -- 172.21.15.40:0/3898938919 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72640d0900 con 0x5620df10ee20 2023-07-18T03:35:08.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.767+0000 7f727b5a4b80 1 -- 172.21.15.40:0/3898938919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10ee20 msgr2=0x5620df10f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:08.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.767+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/3898938919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10ee20 0x5620df10f6e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f72640d0690 tx=0x7f726409c340 comp rx=0 tx=0).stop 2023-07-18T03:35:08.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.767+0000 7f726d9bb700 1 --2- 172.21.15.40:0/4056196519 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72540a1350 0x7f72540a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:08.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.778+0000 7f726d9bb700 1 --2- 172.21.15.40:0/4056196519 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72540a1350 0x7f72540a3800 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7260009fd0 tx=0x7f7260009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:08.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.785+0000 7f727b5a4b80 1 -- 172.21.15.40:0/3898938919 shutdown_connections 2023-07-18T03:35:08.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.785+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/3898938919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10fc70 0x5620df1100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.785+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/3898938919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10e4d0 0x5620df10e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.785+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/3898938919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10ee20 0x5620df10f6e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.785+0000 7f727b5a4b80 1 -- 172.21.15.40:0/3898938919 >> 172.21.15.40:0/3898938919 conn(0x5620df103da0 msgr2=0x5620df1091d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:08.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.790+0000 7f727b5a4b80 1 -- 172.21.15.40:0/3898938919 shutdown_connections 2023-07-18T03:35:08.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.806+0000 7f727b5a4b80 1 -- 172.21.15.40:0/3898938919 wait complete. 2023-07-18T03:35:08.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.807+0000 7f727b5a4b80 1 -- start start 2023-07-18T03:35:08.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.818+0000 7f727b5a4b80 1 Processor -- start 2023-07-18T03:35:08.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.819+0000 7f727b5a4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10e4d0 0x5620df139e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.819+0000 7f727b5a4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10ee20 0x5620df13a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.819+0000 7f727b5a4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10fc70 0x5620df1413c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.819+0000 7f727b5a4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620defffac0 con 0x5620df10e4d0 2023-07-18T03:35:08.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.819+0000 7f727b5a4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620defff7c0 con 0x5620df10fc70 2023-07-18T03:35:08.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.819+0000 7f727b5a4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620deee3920 con 0x5620df10ee20 2023-07-18T03:35:08.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.819+0000 7f727b5a4b80 10 client.0 _ll_register_callbacks cb 0x5620deffd760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:35:08.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.822+0000 7f726c9b9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10ee20 0x5620df13a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:08.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.822+0000 7f726c9b9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10ee20 0x5620df13a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48984/0 (socket says 172.21.15.40:48984) 2023-07-18T03:35:08.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.822+0000 7f726c9b9700 1 -- 172.21.15.40:0/187536043 learned_addr learned my addr 172.21.15.40:0/187536043 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:08.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.822+0000 7f726c9b9700 1 -- 172.21.15.40:0/187536043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10fc70 msgr2=0x5620df1413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:08.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.822+0000 7f726c9b9700 1 --2- 172.21.15.40:0/187536043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10fc70 0x5620df1413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.822+0000 7f726c9b9700 1 -- 172.21.15.40:0/187536043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10e4d0 msgr2=0x5620df139e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:08.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.822+0000 7f726c9b9700 1 --2- 172.21.15.40:0/187536043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10e4d0 0x5620df139e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.822+0000 7f726c9b9700 1 -- 172.21.15.40:0/187536043 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7264030040 con 0x5620df10ee20 2023-07-18T03:35:08.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.823+0000 7f726c9b9700 1 --2- 172.21.15.40:0/187536043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10ee20 0x5620df13a380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7258013690 tx=0x7f7258013a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=527c97bb19a9e843 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:08.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.831+0000 7f7250ff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:35:08.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.831+0000 7f7250ff9700 1 -- 172.21.15.40:0/187536043 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258004240 con 0x5620df10ee20 2023-07-18T03:35:08.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.831+0000 7f7250ff9700 1 -- 172.21.15.40:0/187536043 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f72580043a0 con 0x5620df10ee20 2023-07-18T03:35:08.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.831+0000 7f727b5a4b80 1 -- 172.21.15.40:0/187536043 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620defb94f0 con 0x5620df10ee20 2023-07-18T03:35:08.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.833+0000 7f7250ff9700 1 -- 172.21.15.40:0/187536043 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7258004560 con 0x5620df10ee20 2023-07-18T03:35:08.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.833+0000 7f727b5a4b80 10 client.25903 fetch_fsmap learned FSMap version 37 2023-07-18T03:35:08.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.833+0000 7f727b5a4b80 1 -- 172.21.15.40:0/187536043 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5620dee76c60 con 0x5620df10ee20 2023-07-18T03:35:08.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.834+0000 7f7250ff9700 1 -- 172.21.15.40:0/187536043 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620defb94f0 con 0x5620df10ee20 2023-07-18T03:35:08.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.834+0000 7f7250ff9700 1 -- 172.21.15.40:0/187536043 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 37) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f72580119f0 con 0x5620df10ee20 2023-07-18T03:35:08.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.834+0000 7f727b5a4b80 10 client.25903 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:35:08.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.834+0000 7f727b5a4b80 10 client.25903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:35:08.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.834+0000 7f727b5a4b80 20 client.25903 populate_metadata read hostname 'smithi040' 2023-07-18T03:35:08.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.834+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/187536043 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5620df145cb0 0x5620df1480c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.834+0000 7f727b5a4b80 4 client.25903 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:35:08.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.834+0000 7f727b5a4b80 1 -- 172.21.15.40:0/187536043 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620df058eb0 con 0x5620df145cb0 2023-07-18T03:35:08.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.834+0000 7f726d1ba700 1 --2- 172.21.15.40:0/187536043 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5620df145cb0 0x5620df1480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:08.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.835+0000 7f726d1ba700 1 --2- 172.21.15.40:0/187536043 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5620df145cb0 0x5620df1480c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5bc6f97b59675ab server_cookie=2cee02c4c8380baf in_seq=0 out_seq=0 2023-07-18T03:35:08.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.835+0000 7f7250ff9700 10 client.25903 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:08.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.839+0000 7f7250ff9700 1 -- 172.21.15.40:0/187536043 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5620df058eb0 con 0x5620df145cb0 2023-07-18T03:35:08.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.839+0000 7f7250ff9700 10 client.25903 handle_command_reply: tid=0 2023-07-18T03:35:08.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.840+0000 7f727b5a4b80 1 client.25903 shutdown 2023-07-18T03:35:08.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.848+0000 7f727b5a4b80 1 -- 172.21.15.40:0/187536043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10ee20 msgr2=0x5620df13a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:08.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.848+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/187536043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10ee20 0x5620df13a380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7258013690 tx=0x7f7258013a50 comp rx=0 tx=0).stop 2023-07-18T03:35:08.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.849+0000 7f727b5a4b80 1 -- 172.21.15.40:0/187536043 shutdown_connections 2023-07-18T03:35:08.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.849+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/187536043 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5620df145cb0 0x5620df1480c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.849+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/187536043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10fc70 0x5620df1413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.849+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/187536043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10e4d0 0x5620df139e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.849+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/187536043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10ee20 0x5620df13a380 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.849+0000 7f727b5a4b80 1 -- 172.21.15.40:0/187536043 >> 172.21.15.40:0/187536043 conn(0x5620df103da0 msgr2=0x5620df1088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:08.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.849+0000 7f7250ff9700 0 client.25903 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:08.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.849+0000 7f727b5a4b80 1 -- 172.21.15.40:0/187536043 shutdown_connections 2023-07-18T03:35:08.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.849+0000 7f727b5a4b80 1 -- 172.21.15.40:0/187536043 wait complete. 2023-07-18T03:35:08.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.850+0000 7f727b5a4b80 20 client.25903 trim_cache size 0 max 16384 2023-07-18T03:35:08.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.931+0000 7f727b5a4b80 1 Processor -- start 2023-07-18T03:35:08.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.934+0000 7f727b5a4b80 1 -- start start 2023-07-18T03:35:08.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.935+0000 7f727b5a4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10e4d0 0x5620df141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.935+0000 7f727b5a4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10ee20 0x5620df1412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.935+0000 7f727b5a4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10fc70 0x5620df138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.935+0000 7f727b5a4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620df139a10 con 0x5620df10ee20 2023-07-18T03:35:08.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.935+0000 7f727b5a4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620df143080 con 0x5620df10fc70 2023-07-18T03:35:08.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.935+0000 7f727b5a4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620df1392f0 con 0x5620df10e4d0 2023-07-18T03:35:08.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.935+0000 7f726d1ba700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10e4d0 0x5620df141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:08.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.935+0000 7f726d1ba700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10e4d0 0x5620df141cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:48988/0 (socket says 172.21.15.40:48988) 2023-07-18T03:35:08.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.935+0000 7f726d1ba700 1 -- 172.21.15.40:0/2944903207 learned_addr learned my addr 172.21.15.40:0/2944903207 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:08.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.935+0000 7f726d1ba700 1 -- 172.21.15.40:0/2944903207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10fc70 msgr2=0x5620df138000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:35:08.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.935+0000 7f726d1ba700 1 --2- 172.21.15.40:0/2944903207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10fc70 0x5620df138000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.935+0000 7f726d1ba700 1 -- 172.21.15.40:0/2944903207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10ee20 msgr2=0x5620df1412e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:35:08.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.935+0000 7f726d1ba700 1 --2- 172.21.15.40:0/2944903207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10ee20 0x5620df1412e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.935+0000 7f726d1ba700 1 -- 172.21.15.40:0/2944903207 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620deeb4250 con 0x5620df10e4d0 2023-07-18T03:35:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.936+0000 7f726d1ba700 1 --2- 172.21.15.40:0/2944903207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10e4d0 0x5620df141cb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7264005920 tx=0x7f726400f940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2abd8f395b168243 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.936+0000 7f7253fff700 1 -- 172.21.15.40:0/2944903207 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726409e4c0 con 0x5620df10e4d0 2023-07-18T03:35:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.936+0000 7f7253fff700 1 -- 172.21.15.40:0/2944903207 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72640a4030 con 0x5620df10e4d0 2023-07-18T03:35:08.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.936+0000 7f7253fff700 1 -- 172.21.15.40:0/2944903207 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7264004e40 con 0x5620df10e4d0 2023-07-18T03:35:08.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.937+0000 7f727b5a4b80 1 -- 172.21.15.40:0/2944903207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10e4d0 msgr2=0x5620df141cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:08.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.937+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/2944903207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10e4d0 0x5620df141cb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7264005920 tx=0x7f726400f940 comp rx=0 tx=0).stop 2023-07-18T03:35:08.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.947+0000 7f727b5a4b80 1 -- 172.21.15.40:0/2944903207 shutdown_connections 2023-07-18T03:35:08.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.947+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/2944903207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10fc70 0x5620df138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.947+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/2944903207 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10e4d0 0x5620df141cb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.947+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/2944903207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10ee20 0x5620df1412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.947+0000 7f727b5a4b80 1 -- 172.21.15.40:0/2944903207 >> 172.21.15.40:0/2944903207 conn(0x5620df103da0 msgr2=0x5620df14ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:08.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.950+0000 7f727b5a4b80 1 -- 172.21.15.40:0/2944903207 shutdown_connections 2023-07-18T03:35:08.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.952+0000 7f727b5a4b80 1 -- 172.21.15.40:0/2944903207 wait complete. 2023-07-18T03:35:08.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.952+0000 7f727b5a4b80 1 -- start start 2023-07-18T03:35:08.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.952+0000 7f727b5a4b80 1 Processor -- start 2023-07-18T03:35:08.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.953+0000 7f727b5a4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10e4d0 0x5620df13cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.953+0000 7f727b5a4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10ee20 0x5620df13d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.953+0000 7f727b5a4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10fc70 0x5620df140260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.953+0000 7f727b5a4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620df1407a0 con 0x5620df10e4d0 2023-07-18T03:35:08.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.953+0000 7f727b5a4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620df140910 con 0x5620df10ee20 2023-07-18T03:35:08.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.953+0000 7f727b5a4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620df140ad0 con 0x5620df10fc70 2023-07-18T03:35:08.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.953+0000 7f727b5a4b80 10 client.0 _ll_register_callbacks cb 0x7f7258009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:35:08.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.953+0000 7f726d1ba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10e4d0 0x5620df13cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:08.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.953+0000 7f726d1ba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10e4d0 0x5620df13cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59962/0 (socket says 172.21.15.40:59962) 2023-07-18T03:35:08.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.953+0000 7f726d1ba700 1 -- 172.21.15.40:0/3298535619 learned_addr learned my addr 172.21.15.40:0/3298535619 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:08.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.953+0000 7f726d9bb700 1 --2- 172.21.15.40:0/3298535619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10fc70 0x5620df140260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:08.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.953+0000 7f726c9b9700 1 --2- 172.21.15.40:0/3298535619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10ee20 0x5620df13d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:08.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.954+0000 7f726d9bb700 1 -- 172.21.15.40:0/3298535619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10ee20 msgr2=0x5620df13d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:08.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.954+0000 7f726d9bb700 1 --2- 172.21.15.40:0/3298535619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10ee20 0x5620df13d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.954+0000 7f726d9bb700 1 -- 172.21.15.40:0/3298535619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10e4d0 msgr2=0x5620df13cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:08.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.954+0000 7f726d9bb700 1 --2- 172.21.15.40:0/3298535619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10e4d0 0x5620df13cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.954+0000 7f726d9bb700 1 -- 172.21.15.40:0/3298535619 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620dee8dde0 con 0x5620df10fc70 2023-07-18T03:35:08.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.954+0000 7f726d9bb700 1 --2- 172.21.15.40:0/3298535619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10fc70 0x5620df140260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7260018940 tx=0x7f7260009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91ab442e3f67e1e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:08.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.954+0000 7f7250ff9700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:35:08.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.954+0000 7f7250ff9700 1 -- 172.21.15.40:0/3298535619 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726001c040 con 0x5620df10fc70 2023-07-18T03:35:08.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.954+0000 7f7250ff9700 1 -- 172.21.15.40:0/3298535619 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7260016a00 con 0x5620df10fc70 2023-07-18T03:35:08.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.954+0000 7f727b5a4b80 1 -- 172.21.15.40:0/3298535619 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72640a0000 con 0x5620df10fc70 2023-07-18T03:35:08.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.955+0000 7f7250ff9700 1 -- 172.21.15.40:0/3298535619 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f726000e040 con 0x5620df10fc70 2023-07-18T03:35:08.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.955+0000 7f727b5a4b80 10 client.25909 fetch_fsmap learned FSMap version 37 2023-07-18T03:35:08.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.955+0000 7f727b5a4b80 1 -- 172.21.15.40:0/3298535619 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5620dee8c2b0 con 0x5620df10fc70 2023-07-18T03:35:08.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.955+0000 7f7250ff9700 1 -- 172.21.15.40:0/3298535619 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7260031020 con 0x5620df10fc70 2023-07-18T03:35:08.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.955+0000 7f7250ff9700 1 -- 172.21.15.40:0/3298535619 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 37) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f726002b400 con 0x5620df10fc70 2023-07-18T03:35:08.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.955+0000 7f727b5a4b80 10 client.25909 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:35:08.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.955+0000 7f727b5a4b80 10 client.25909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:35:08.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.955+0000 7f727b5a4b80 20 client.25909 populate_metadata read hostname 'smithi040' 2023-07-18T03:35:08.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.955+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/3298535619 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5620df19a980 0x5620df19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:08.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.955+0000 7f727b5a4b80 4 client.25909 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:35:08.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.955+0000 7f727b5a4b80 1 -- 172.21.15.40:0/3298535619 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5620df01b050 con 0x5620df19a980 2023-07-18T03:35:08.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.962+0000 7f726d1ba700 1 --2- 172.21.15.40:0/3298535619 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5620df19a980 0x5620df19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:08.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.962+0000 7f7250ff9700 10 client.25909 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:08.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.962+0000 7f726d1ba700 1 --2- 172.21.15.40:0/3298535619 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5620df19a980 0x5620df19cd90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a06b6ba5573524b server_cookie=ecbe866a6ddf1f43 in_seq=0 out_seq=0 2023-07-18T03:35:08.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.964+0000 7f7250ff9700 1 -- 172.21.15.40:0/3298535619 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5620df01b050 con 0x5620df19a980 2023-07-18T03:35:08.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.964+0000 7f7250ff9700 10 client.25909 handle_command_reply: tid=0 2023-07-18T03:35:08.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.964+0000 7f727b5a4b80 1 client.25909 shutdown 2023-07-18T03:35:08.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.964+0000 7f727b5a4b80 1 -- 172.21.15.40:0/3298535619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10fc70 msgr2=0x5620df140260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:08.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.964+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/3298535619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10fc70 0x5620df140260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7260018940 tx=0x7f7260009920 comp rx=0 tx=0).stop 2023-07-18T03:35:08.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.964+0000 7f727b5a4b80 1 -- 172.21.15.40:0/3298535619 shutdown_connections 2023-07-18T03:35:08.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.964+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/3298535619 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5620df19a980 0x5620df19cd90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.964+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/3298535619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620df10fc70 0x5620df140260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.964+0000 7f7250ff9700 0 client.25909 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:08.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.964+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/3298535619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5620df10e4d0 0x5620df13cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.964+0000 7f727b5a4b80 1 --2- 172.21.15.40:0/3298535619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620df10ee20 0x5620df13d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.964+0000 7f727b5a4b80 1 -- 172.21.15.40:0/3298535619 >> 172.21.15.40:0/3298535619 conn(0x5620df103da0 msgr2=0x5620df141cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.964+0000 7f727b5a4b80 1 -- 172.21.15.40:0/3298535619 shutdown_connections 2023-07-18T03:35:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.964+0000 7f727b5a4b80 1 -- 172.21.15.40:0/3298535619 wait complete. 2023-07-18T03:35:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.965+0000 7f727b5a4b80 20 client.25909 trim_cache size 0 max 16384 2023-07-18T03:35:08.966 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:35:08.967 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-18T03:35:08.967 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:35:08.967 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:35:08.967 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:35:08.967 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:35:08.967 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:35:08.967 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:35:08.968 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:35:08.968 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:35:08.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.968+0000 7f726f41e700 1 -- 172.21.15.40:0/4056196519 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72540a1350 msgr2=0x7f72540a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:08.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.968+0000 7f726f41e700 1 --2- 172.21.15.40:0/4056196519 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72540a1350 0x7f72540a3800 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7260009fd0 tx=0x7f7260009040 comp rx=0 tx=0).stop 2023-07-18T03:35:08.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.968+0000 7f726f41e700 1 -- 172.21.15.40:0/4056196519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7268070ee0 msgr2=0x7f72681a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:08.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.968+0000 7f726f41e700 1 --2- 172.21.15.40:0/4056196519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7268070ee0 0x7f72681a4760 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f726400b790 tx=0x7f726400b870 comp rx=0 tx=0).stop 2023-07-18T03:35:08.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.968+0000 7f726f41e700 1 -- 172.21.15.40:0/4056196519 shutdown_connections 2023-07-18T03:35:08.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.968+0000 7f726f41e700 1 --2- 172.21.15.40:0/4056196519 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72540a1350 0x7f72540a3800 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.968+0000 7f726f41e700 1 --2- 172.21.15.40:0/4056196519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f726810bd60 0x7f72681abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.968+0000 7f726f41e700 1 --2- 172.21.15.40:0/4056196519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7268070ee0 0x7f72681a4760 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.968+0000 7f726f41e700 1 --2- 172.21.15.40:0/4056196519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7268071830 0x7f72681a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:08.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.968+0000 7f726f41e700 1 -- 172.21.15.40:0/4056196519 >> 172.21.15.40:0/4056196519 conn(0x7f726806ca90 msgr2=0x7f726806e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:08.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.968+0000 7f726f41e700 1 -- 172.21.15.40:0/4056196519 shutdown_connections 2023-07-18T03:35:08.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:08.968+0000 7f726f41e700 1 -- 172.21.15.40:0/4056196519 wait complete. 2023-07-18T03:35:08.996 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:35:09.341 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:09 smithi040 ceph-mon[96439]: pgmap v421: 97 pgs: 97 active+clean; 870 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 415 op/s 2023-07-18T03:35:09.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:09 smithi134 ceph-mon[98216]: pgmap v421: 97 pgs: 97 active+clean; 870 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 415 op/s 2023-07-18T03:35:09.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:09 smithi064 ceph-mon[98059]: pgmap v421: 97 pgs: 97 active+clean; 870 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 415 op/s 2023-07-18T03:35:11.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:11 smithi134 ceph-mon[98216]: pgmap v422: 97 pgs: 97 active+clean; 870 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 350 op/s 2023-07-18T03:35:11.587 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:11 smithi040 ceph-mon[96439]: pgmap v422: 97 pgs: 97 active+clean; 870 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 350 op/s 2023-07-18T03:35:11.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:11 smithi064 ceph-mon[98059]: pgmap v422: 97 pgs: 97 active+clean; 870 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 350 op/s 2023-07-18T03:35:13.170 INFO:tasks.workunit.client.0.smithi040.stdout: 251 2431852 660 1685773 519 574456 1301 2023-07-18T03:35:13.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:12 smithi134 ceph-mon[98216]: pgmap v423: 97 pgs: 97 active+clean; 875 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 367 op/s 2023-07-18T03:35:13.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:12 smithi064 ceph-mon[98059]: pgmap v423: 97 pgs: 97 active+clean; 875 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 367 op/s 2023-07-18T03:35:13.355 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:12 smithi040 ceph-mon[96439]: pgmap v423: 97 pgs: 97 active+clean; 875 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 367 op/s 2023-07-18T03:35:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.826+0000 7f6d7f59e700 1 -- 172.21.15.40:0/2008351396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d80076cf0 msgr2=0x7f6d800791a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.826+0000 7f6d7f59e700 1 --2- 172.21.15.40:0/2008351396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d80076cf0 0x7f6d800791a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6d740098f0 tx=0x7f6d74009c00 comp rx=0 tx=0).stop 2023-07-18T03:35:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.841+0000 7f6d7f59e700 1 -- 172.21.15.40:0/2008351396 shutdown_connections 2023-07-18T03:35:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.841+0000 7f6d7f59e700 1 --2- 172.21.15.40:0/2008351396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d80076cf0 0x7f6d800791a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.841+0000 7f6d7f59e700 1 --2- 172.21.15.40:0/2008351396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d800710c0 0x7f6d800714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.841+0000 7f6d7f59e700 1 --2- 172.21.15.40:0/2008351396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d80071a10 0x7f6d80076680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.841+0000 7f6d7f59e700 1 -- 172.21.15.40:0/2008351396 >> 172.21.15.40:0/2008351396 conn(0x7f6d8006c910 msgr2=0x7f6d8006ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.846+0000 7f6d7f59e700 1 -- 172.21.15.40:0/2008351396 shutdown_connections 2023-07-18T03:35:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.857+0000 7f6d7f59e700 1 -- 172.21.15.40:0/2008351396 wait complete. 2023-07-18T03:35:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.858+0000 7f6d7f59e700 1 Processor -- start 2023-07-18T03:35:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.858+0000 7f6d7f59e700 1 -- start start 2023-07-18T03:35:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.858+0000 7f6d7f59e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d800710c0 0x7f6d80080e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.858+0000 7f6d7f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d80071a10 0x7f6d800813c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.858+0000 7f6d7f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d80076cf0 0x7f6d80082400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.858+0000 7f6d7f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d800829d0 con 0x7f6d80071a10 2023-07-18T03:35:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.858+0000 7f6d7f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d80082b40 con 0x7f6d80076cf0 2023-07-18T03:35:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.858+0000 7f6d7f59e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d80082d00 con 0x7f6d800710c0 2023-07-18T03:35:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.872+0000 7f6d7e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d800710c0 0x7f6d80080e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.872+0000 7f6d7ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d80076cf0 0x7f6d80082400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.872+0000 7f6d7dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d80071a10 0x7f6d800813c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.872+0000 7f6d7dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d80071a10 0x7f6d800813c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59978/0 (socket says 172.21.15.40:59978) 2023-07-18T03:35:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.872+0000 7f6d7dd9b700 1 -- 172.21.15.40:0/2885716002 learned_addr learned my addr 172.21.15.40:0/2885716002 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.879+0000 7f6d7e59c700 1 -- 172.21.15.40:0/2885716002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d80076cf0 msgr2=0x7f6d80082400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.879+0000 7f6d7e59c700 1 --2- 172.21.15.40:0/2885716002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d80076cf0 0x7f6d80082400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.879+0000 7f6d7e59c700 1 -- 172.21.15.40:0/2885716002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d80071a10 msgr2=0x7f6d800813c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.879+0000 7f6d7e59c700 1 --2- 172.21.15.40:0/2885716002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d80071a10 0x7f6d800813c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.879+0000 7f6d7e59c700 1 -- 172.21.15.40:0/2885716002 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d74009520 con 0x7f6d800710c0 2023-07-18T03:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.886+0000 7f6d7e59c700 1 --2- 172.21.15.40:0/2885716002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d800710c0 0x7f6d80080e80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6d70009940 tx=0x7f6d70009c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.890+0000 7f6d6f7fe700 1 -- 172.21.15.40:0/2885716002 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d70009e90 con 0x7f6d800710c0 2023-07-18T03:35:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.890+0000 7f6d6f7fe700 1 -- 172.21.15.40:0/2885716002 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d70010040 con 0x7f6d800710c0 2023-07-18T03:35:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.890+0000 7f6d6f7fe700 1 -- 172.21.15.40:0/2885716002 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d700038e0 con 0x7f6d800710c0 2023-07-18T03:35:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.890+0000 7f6d7f59e700 1 -- 172.21.15.40:0/2885716002 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d80082f80 con 0x7f6d800710c0 2023-07-18T03:35:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.890+0000 7f6d7f59e700 1 -- 172.21.15.40:0/2885716002 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d80083440 con 0x7f6d800710c0 2023-07-18T03:35:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.895+0000 7f6d90d84b80 1 Processor -- start 2023-07-18T03:35:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.895+0000 7f6d90d84b80 1 -- start start 2023-07-18T03:35:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.896+0000 7f6d90d84b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8180550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.896+0000 7f6d90d84b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa8180a90 0x55ffa81812a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.896+0000 7f6d90d84b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa81817e0 0x55ffa8181c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.896+0000 7f6d90d84b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa8085c50 con 0x55ffa7fdc6b0 2023-07-18T03:35:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.896+0000 7f6d90d84b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa80d89e0 con 0x55ffa81817e0 2023-07-18T03:35:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.896+0000 7f6d90d84b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa80d1e70 con 0x55ffa8180a90 2023-07-18T03:35:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.898+0000 7f6d7dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa81817e0 0x55ffa8181c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.898+0000 7f6d7dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa81817e0 0x55ffa8181c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43308/0 (socket says 172.21.15.40:43308) 2023-07-18T03:35:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.898+0000 7f6d7dd9b700 1 -- 172.21.15.40:0/2500853261 learned_addr learned my addr 172.21.15.40:0/2500853261 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.898+0000 7f6d7e59c700 1 --2- 172.21.15.40:0/2500853261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa8180a90 0x55ffa81812a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.904+0000 7f6d6f7fe700 1 -- 172.21.15.40:0/2885716002 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6d7000e860 con 0x7f6d800710c0 2023-07-18T03:35:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.906+0000 7f6d6f7fe700 1 --2- 172.21.15.40:0/2885716002 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d680a1350 0x7f6d680a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.906+0000 7f6d7ed9d700 1 --2- 172.21.15.40:0/2500853261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8180550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.906+0000 7f6d6f7fe700 1 -- 172.21.15.40:0/2885716002 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6d700d1820 con 0x7f6d800710c0 2023-07-18T03:35:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.907+0000 7f6d7e59c700 1 -- 172.21.15.40:0/2500853261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa81817e0 msgr2=0x55ffa8181c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.907+0000 7f6d7e59c700 1 --2- 172.21.15.40:0/2500853261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa81817e0 0x55ffa8181c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.907+0000 7f6d7e59c700 1 -- 172.21.15.40:0/2500853261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa7fdc6b0 msgr2=0x55ffa8180550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.907+0000 7f6d7e59c700 1 --2- 172.21.15.40:0/2500853261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8180550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.907+0000 7f6d7e59c700 1 -- 172.21.15.40:0/2500853261 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffa7effd80 con 0x55ffa8180a90 2023-07-18T03:35:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.908+0000 7f6d7ed9d700 1 --2- 172.21.15.40:0/2885716002 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d680a1350 0x7f6d680a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:13.917 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.919+0000 7f6d7ed9d700 1 --2- 172.21.15.40:0/2885716002 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d680a1350 0x7f6d680a3800 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6d74005c10 tx=0x7f6d74009c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.929+0000 7f6d7e59c700 1 --2- 172.21.15.40:0/2500853261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa8180a90 0x55ffa81812a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d700cf240 tx=0x7f6d7000ae00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16ad222ba688bda2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.940+0000 7f6d6cff9700 1 -- 172.21.15.40:0/2500853261 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7009ae60 con 0x55ffa8180a90 2023-07-18T03:35:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.940+0000 7f6d6cff9700 1 -- 172.21.15.40:0/2500853261 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d7009e980 con 0x55ffa8180a90 2023-07-18T03:35:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.940+0000 7f6d6cff9700 1 -- 172.21.15.40:0/2500853261 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d700a7090 con 0x55ffa8180a90 2023-07-18T03:35:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.947+0000 7f6d90d84b80 1 -- 172.21.15.40:0/2500853261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa8180a90 msgr2=0x55ffa81812a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.947+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/2500853261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa8180a90 0x55ffa81812a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d700cf240 tx=0x7f6d7000ae00 comp rx=0 tx=0).stop 2023-07-18T03:35:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.952+0000 7f6d90d84b80 1 -- 172.21.15.40:0/2500853261 shutdown_connections 2023-07-18T03:35:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.952+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/2500853261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa81817e0 0x55ffa8181c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.952+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/2500853261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8180550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.952+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/2500853261 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa8180a90 0x55ffa81812a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.952+0000 7f6d90d84b80 1 -- 172.21.15.40:0/2500853261 >> 172.21.15.40:0/2500853261 conn(0x55ffa8175d40 msgr2=0x55ffa817ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.952+0000 7f6d90d84b80 1 -- 172.21.15.40:0/2500853261 shutdown_connections 2023-07-18T03:35:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.952+0000 7f6d90d84b80 1 -- 172.21.15.40:0/2500853261 wait complete. 2023-07-18T03:35:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.953+0000 7f6d90d84b80 1 -- start start 2023-07-18T03:35:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.953+0000 7f6d90d84b80 1 Processor -- start 2023-07-18T03:35:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.953+0000 7f6d90d84b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8189ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.953+0000 7f6d90d84b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa8180a90 0x55ffa8189ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.953+0000 7f6d90d84b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa81817e0 0x55ffa8191030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.953+0000 7f6d90d84b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa8071a60 con 0x55ffa81817e0 2023-07-18T03:35:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.953+0000 7f6d90d84b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa8071760 con 0x55ffa8180a90 2023-07-18T03:35:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.953+0000 7f6d90d84b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa7f558c0 con 0x55ffa7fdc6b0 2023-07-18T03:35:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.953+0000 7f6d90d84b80 10 client.0 _ll_register_callbacks cb 0x55ffa806f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:35:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.954+0000 7f6d7e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8189ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.954+0000 7f6d7e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8189ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49018/0 (socket says 172.21.15.40:49018) 2023-07-18T03:35:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.954+0000 7f6d7e59c700 1 -- 172.21.15.40:0/3528267852 learned_addr learned my addr 172.21.15.40:0/3528267852 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.966+0000 7f6d7dd9b700 1 --2- 172.21.15.40:0/3528267852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa8180a90 0x55ffa8189ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.967+0000 7f6d7e59c700 1 -- 172.21.15.40:0/3528267852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa8180a90 msgr2=0x55ffa8189ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.967+0000 7f6d7e59c700 1 --2- 172.21.15.40:0/3528267852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa8180a90 0x55ffa8189ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.967+0000 7f6d7e59c700 1 -- 172.21.15.40:0/3528267852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa81817e0 msgr2=0x55ffa8191030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.967+0000 7f6d7e59c700 1 --2- 172.21.15.40:0/3528267852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa81817e0 0x55ffa8191030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.967+0000 7f6d7e59c700 1 -- 172.21.15.40:0/3528267852 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d80083440 con 0x55ffa7fdc6b0 2023-07-18T03:35:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.972+0000 7f6d7e59c700 1 --2- 172.21.15.40:0/3528267852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8189ab0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6d70000c00 tx=0x7f6d7009a7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d65eae16ea778407 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.989+0000 7f6d65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:35:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.989+0000 7f6d65ffb700 1 -- 172.21.15.40:0/3528267852 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7009e510 con 0x55ffa7fdc6b0 2023-07-18T03:35:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.989+0000 7f6d65ffb700 1 -- 172.21.15.40:0/3528267852 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6d7009ebb0 con 0x55ffa7fdc6b0 2023-07-18T03:35:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.989+0000 7f6d90d84b80 1 -- 172.21.15.40:0/3528267852 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffa802b490 con 0x55ffa7fdc6b0 2023-07-18T03:35:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.997+0000 7f6d65ffb700 1 -- 172.21.15.40:0/3528267852 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d7009cd90 con 0x55ffa7fdc6b0 2023-07-18T03:35:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.997+0000 7f6d90d84b80 10 client.25921 fetch_fsmap learned FSMap version 37 2023-07-18T03:35:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:13.997+0000 7f6d90d84b80 1 -- 172.21.15.40:0/3528267852 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55ffa7ee8c00 con 0x55ffa7fdc6b0 2023-07-18T03:35:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.005+0000 7f6d65ffb700 1 -- 172.21.15.40:0/3528267852 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ffa802b490 con 0x55ffa7fdc6b0 2023-07-18T03:35:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.005+0000 7f6d65ffb700 1 -- 172.21.15.40:0/3528267852 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 37) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f6d700d1bc0 con 0x55ffa7fdc6b0 2023-07-18T03:35:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.005+0000 7f6d90d84b80 10 client.25921 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:35:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.005+0000 7f6d90d84b80 10 client.25921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:35:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.005+0000 7f6d90d84b80 20 client.25921 populate_metadata read hostname 'smithi040' 2023-07-18T03:35:14.003 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.005+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/3528267852 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ffa81955d0 0x55ffa81979e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.005+0000 7f6d90d84b80 4 client.25921 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:35:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.005+0000 7f6d90d84b80 1 -- 172.21.15.40:0/3528267852 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffa80cae50 con 0x55ffa81955d0 2023-07-18T03:35:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.005+0000 7f6d7dd9b700 1 --2- 172.21.15.40:0/3528267852 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ffa81955d0 0x55ffa81979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.028+0000 7f6d7dd9b700 1 --2- 172.21.15.40:0/3528267852 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ffa81955d0 0x55ffa81979e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1521c6c5e47b1e90 server_cookie=3cc5d1d94b91b702 in_seq=0 out_seq=0 2023-07-18T03:35:14.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.028+0000 7f6d65ffb700 10 client.25921 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.039+0000 7f6d65ffb700 1 -- 172.21.15.40:0/3528267852 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ffa80cae50 con 0x55ffa81955d0 2023-07-18T03:35:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.039+0000 7f6d65ffb700 10 client.25921 handle_command_reply: tid=0 2023-07-18T03:35:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.039+0000 7f6d90d84b80 1 client.25921 shutdown 2023-07-18T03:35:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.039+0000 7f6d90d84b80 1 -- 172.21.15.40:0/3528267852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa7fdc6b0 msgr2=0x55ffa8189ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.039+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/3528267852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8189ab0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6d70000c00 tx=0x7f6d7009a7f0 comp rx=0 tx=0).stop 2023-07-18T03:35:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.040+0000 7f6d90d84b80 1 -- 172.21.15.40:0/3528267852 shutdown_connections 2023-07-18T03:35:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.040+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/3528267852 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ffa81955d0 0x55ffa81979e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.040+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/3528267852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa81817e0 0x55ffa8191030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.040+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/3528267852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8189ab0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.040+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/3528267852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa8180a90 0x55ffa8189ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.040+0000 7f6d90d84b80 1 -- 172.21.15.40:0/3528267852 >> 172.21.15.40:0/3528267852 conn(0x55ffa8175d40 msgr2=0x55ffa817a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.040+0000 7f6d65ffb700 0 client.25921 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.040+0000 7f6d90d84b80 1 -- 172.21.15.40:0/3528267852 shutdown_connections 2023-07-18T03:35:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.040+0000 7f6d90d84b80 1 -- 172.21.15.40:0/3528267852 wait complete. 2023-07-18T03:35:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.040+0000 7f6d90d84b80 20 client.25921 trim_cache size 0 max 16384 2023-07-18T03:35:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.087+0000 7f6d90d84b80 1 Processor -- start 2023-07-18T03:35:14.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.090+0000 7f6d90d84b80 1 -- start start 2023-07-18T03:35:14.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.090+0000 7f6d90d84b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8189860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:14.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.090+0000 7f6d90d84b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa8180a90 0x55ffa8190ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.090+0000 7f6d90d84b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa81817e0 0x55ffa8189da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.090+0000 7f6d90d84b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa817bd50 con 0x55ffa81817e0 2023-07-18T03:35:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.090+0000 7f6d90d84b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa8195160 con 0x55ffa8180a90 2023-07-18T03:35:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.090+0000 7f6d90d84b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa817c6a0 con 0x55ffa7fdc6b0 2023-07-18T03:35:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.090+0000 7f6d7dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa8180a90 0x55ffa8190ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.091+0000 7f6d7e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8189860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.091+0000 7f6d7dd9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa8180a90 0x55ffa8190ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43326/0 (socket says 172.21.15.40:43326) 2023-07-18T03:35:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.091+0000 7f6d7dd9b700 1 -- 172.21.15.40:0/1742898932 learned_addr learned my addr 172.21.15.40:0/1742898932 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.098+0000 7f6d7ed9d700 1 --2- 172.21.15.40:0/1742898932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa81817e0 0x55ffa8189da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.110+0000 7f6d7dd9b700 1 -- 172.21.15.40:0/1742898932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa7fdc6b0 msgr2=0x55ffa8189860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.110+0000 7f6d7dd9b700 1 --2- 172.21.15.40:0/1742898932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8189860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.110+0000 7f6d7dd9b700 1 -- 172.21.15.40:0/1742898932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa81817e0 msgr2=0x55ffa8189da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.110+0000 7f6d7dd9b700 1 --2- 172.21.15.40:0/1742898932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa81817e0 0x55ffa8189da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:14.108 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.110+0000 7f6d7dd9b700 1 -- 172.21.15.40:0/1742898932 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffa7f261f0 con 0x55ffa8180a90 2023-07-18T03:35:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.121+0000 7f6d7dd9b700 1 --2- 172.21.15.40:0/1742898932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa8180a90 0x55ffa8190ff0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009c10 tx=0x7f6d78004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8efd93c1182406a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.123+0000 7f6d6d7fa700 1 -- 172.21.15.40:0/1742898932 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7801b040 con 0x55ffa8180a90 2023-07-18T03:35:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.123+0000 7f6d6d7fa700 1 -- 172.21.15.40:0/1742898932 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d7800f040 con 0x55ffa8180a90 2023-07-18T03:35:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.123+0000 7f6d6d7fa700 1 -- 172.21.15.40:0/1742898932 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7800e040 con 0x55ffa8180a90 2023-07-18T03:35:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.124+0000 7f6d90d84b80 1 -- 172.21.15.40:0/1742898932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa8180a90 msgr2=0x55ffa8190ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.124+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/1742898932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa8180a90 0x55ffa8190ff0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009c10 tx=0x7f6d78004ad0 comp rx=0 tx=0).stop 2023-07-18T03:35:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.124+0000 7f6d90d84b80 1 -- 172.21.15.40:0/1742898932 shutdown_connections 2023-07-18T03:35:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.125+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/1742898932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa81817e0 0x55ffa8189da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.125+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/1742898932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8189860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.125+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/1742898932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa8180a90 0x55ffa8190ff0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.125+0000 7f6d90d84b80 1 -- 172.21.15.40:0/1742898932 >> 172.21.15.40:0/1742898932 conn(0x55ffa8175d40 msgr2=0x55ffa8187b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.125+0000 7f6d90d84b80 1 -- 172.21.15.40:0/1742898932 shutdown_connections 2023-07-18T03:35:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.125+0000 7f6d90d84b80 1 -- 172.21.15.40:0/1742898932 wait complete. 2023-07-18T03:35:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.125+0000 7f6d90d84b80 1 -- start start 2023-07-18T03:35:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.125+0000 7f6d90d84b80 1 Processor -- start 2023-07-18T03:35:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.126+0000 7f6d90d84b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8190a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.126+0000 7f6d90d84b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa8180a90 0x55ffa8190f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.126+0000 7f6d90d84b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa81817e0 0x55ffa818d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.126+0000 7f6d90d84b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa818dc80 con 0x55ffa8180a90 2023-07-18T03:35:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.126+0000 7f6d90d84b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa818ddf0 con 0x55ffa7fdc6b0 2023-07-18T03:35:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.126+0000 7f6d90d84b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa818dfb0 con 0x55ffa81817e0 2023-07-18T03:35:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.126+0000 7f6d90d84b80 10 client.0 _ll_register_callbacks cb 0x7f6d78009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:35:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.135+0000 7f6d7dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa8180a90 0x55ffa8190f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.135+0000 7f6d7dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa8180a90 0x55ffa8190f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60008/0 (socket says 172.21.15.40:60008) 2023-07-18T03:35:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.135+0000 7f6d7dd9b700 1 -- 172.21.15.40:0/3197098066 learned_addr learned my addr 172.21.15.40:0/3197098066 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.145+0000 7f6d7ed9d700 1 --2- 172.21.15.40:0/3197098066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa81817e0 0x55ffa818d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.145+0000 7f6d7e59c700 1 --2- 172.21.15.40:0/3197098066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8190a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:14.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.149+0000 7f6d7dd9b700 1 -- 172.21.15.40:0/3197098066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa81817e0 msgr2=0x55ffa818d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.149+0000 7f6d7dd9b700 1 --2- 172.21.15.40:0/3197098066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa81817e0 0x55ffa818d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.149+0000 7f6d7dd9b700 1 -- 172.21.15.40:0/3197098066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa7fdc6b0 msgr2=0x55ffa8190a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.149+0000 7f6d7dd9b700 1 --2- 172.21.15.40:0/3197098066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8190a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.149+0000 7f6d7dd9b700 1 -- 172.21.15.40:0/3197098066 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d78009350 con 0x55ffa8180a90 2023-07-18T03:35:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.150+0000 7f6d7dd9b700 1 --2- 172.21.15.40:0/3197098066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa8180a90 0x55ffa8190f40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6d78016610 tx=0x7f6d7801d6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35651b9a2c8e4fa server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.150+0000 7f6d65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:35:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.151+0000 7f6d65ffb700 1 -- 172.21.15.40:0/3197098066 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7801b070 con 0x55ffa8180a90 2023-07-18T03:35:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.151+0000 7f6d65ffb700 1 -- 172.21.15.40:0/3197098066 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6d7801d980 con 0x55ffa8180a90 2023-07-18T03:35:14.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.151+0000 7f6d90d84b80 1 -- 172.21.15.40:0/3197098066 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d7009e980 con 0x55ffa8180a90 2023-07-18T03:35:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.151+0000 7f6d65ffb700 1 -- 172.21.15.40:0/3197098066 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d7800e040 con 0x55ffa8180a90 2023-07-18T03:35:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.151+0000 7f6d90d84b80 10 client.16269 fetch_fsmap learned FSMap version 37 2023-07-18T03:35:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.151+0000 7f6d90d84b80 1 -- 172.21.15.40:0/3197098066 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55ffa7efe250 con 0x55ffa8180a90 2023-07-18T03:35:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.152+0000 7f6d65ffb700 1 -- 172.21.15.40:0/3197098066 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d7009e980 con 0x55ffa8180a90 2023-07-18T03:35:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.152+0000 7f6d65ffb700 1 -- 172.21.15.40:0/3197098066 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 37) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f6d78005110 con 0x55ffa8180a90 2023-07-18T03:35:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.153+0000 7f6d90d84b80 10 client.16269 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:35:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.153+0000 7f6d90d84b80 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:35:14.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.153+0000 7f6d90d84b80 20 client.16269 populate_metadata read hostname 'smithi040' 2023-07-18T03:35:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.153+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/3197098066 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ffa81eab10 0x55ffa81ecf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.153+0000 7f6d90d84b80 4 client.16269 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:35:14.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.153+0000 7f6d90d84b80 1 -- 172.21.15.40:0/3197098066 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ffa808cff0 con 0x55ffa81eab10 2023-07-18T03:35:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.190+0000 7f6d7e59c700 1 --2- 172.21.15.40:0/3197098066 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ffa81eab10 0x55ffa81ecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.219+0000 7f6d7e59c700 1 --2- 172.21.15.40:0/3197098066 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ffa81eab10 0x55ffa81ecf20 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dd40ecdc2e02235 server_cookie=8dcac2793e227ef6 in_seq=0 out_seq=0 2023-07-18T03:35:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:14.219+0000 7f6d65ffb700 10 client.16269 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:15.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:15 smithi134 ceph-mon[98216]: pgmap v424: 97 pgs: 97 active+clean; 889 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 303 op/s 2023-07-18T03:35:15.585 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:15 smithi040 ceph-mon[96439]: pgmap v424: 97 pgs: 97 active+clean; 889 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 303 op/s 2023-07-18T03:35:15.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:15 smithi064 ceph-mon[98059]: pgmap v424: 97 pgs: 97 active+clean; 889 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 303 op/s 2023-07-18T03:35:16.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:16 smithi134 ceph-mon[98216]: pgmap v425: 97 pgs: 97 active+clean; 904 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 347 op/s 2023-07-18T03:35:16.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:16 smithi064 ceph-mon[98059]: pgmap v425: 97 pgs: 97 active+clean; 904 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 347 op/s 2023-07-18T03:35:16.641 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:16 smithi040 ceph-mon[96439]: pgmap v425: 97 pgs: 97 active+clean; 904 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 347 op/s 2023-07-18T03:35:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.085+0000 7f6d65ffb700 1 -- 172.21.15.40:0/3197098066 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ffa808cff0 con 0x55ffa81eab10 2023-07-18T03:35:17.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.085+0000 7f6d65ffb700 10 client.16269 handle_command_reply: tid=0 2023-07-18T03:35:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.085+0000 7f6d90d84b80 1 client.16269 shutdown 2023-07-18T03:35:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.089+0000 7f6d90d84b80 1 -- 172.21.15.40:0/3197098066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa8180a90 msgr2=0x55ffa8190f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.089+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/3197098066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa8180a90 0x55ffa8190f40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6d78016610 tx=0x7f6d7801d6e0 comp rx=0 tx=0).stop 2023-07-18T03:35:17.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.089+0000 7f6d90d84b80 1 -- 172.21.15.40:0/3197098066 shutdown_connections 2023-07-18T03:35:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.089+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/3197098066 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55ffa81eab10 0x55ffa81ecf20 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.090+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/3197098066 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ffa81817e0 0x55ffa818d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.090+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/3197098066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ffa7fdc6b0 0x55ffa8190a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.090+0000 7f6d90d84b80 1 --2- 172.21.15.40:0/3197098066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55ffa8180a90 0x55ffa8190f40 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.090+0000 7f6d90d84b80 1 -- 172.21.15.40:0/3197098066 >> 172.21.15.40:0/3197098066 conn(0x55ffa8175d40 msgr2=0x55ffa8189860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.090+0000 7f6d65ffb700 0 client.16269 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.090+0000 7f6d90d84b80 1 -- 172.21.15.40:0/3197098066 shutdown_connections 2023-07-18T03:35:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.090+0000 7f6d90d84b80 1 -- 172.21.15.40:0/3197098066 wait complete. 2023-07-18T03:35:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.090+0000 7f6d90d84b80 20 client.16269 trim_cache size 0 max 16384 2023-07-18T03:35:17.090 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:35:17.090 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:35:17.090 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:35:17.090 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:35:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.220+0000 7f6d7f59e700 1 -- 172.21.15.40:0/2885716002 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d680a1350 msgr2=0x7f6d680a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.221+0000 7f6d7f59e700 1 --2- 172.21.15.40:0/2885716002 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d680a1350 0x7f6d680a3800 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6d74005c10 tx=0x7f6d74009c00 comp rx=0 tx=0).stop 2023-07-18T03:35:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.221+0000 7f6d7f59e700 1 -- 172.21.15.40:0/2885716002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d800710c0 msgr2=0x7f6d80080e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.221+0000 7f6d7f59e700 1 --2- 172.21.15.40:0/2885716002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d800710c0 0x7f6d80080e80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6d70009940 tx=0x7f6d70009c50 comp rx=0 tx=0).stop 2023-07-18T03:35:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.221+0000 7f6d7f59e700 1 -- 172.21.15.40:0/2885716002 shutdown_connections 2023-07-18T03:35:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.221+0000 7f6d7f59e700 1 --2- 172.21.15.40:0/2885716002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d80076cf0 0x7f6d80082400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.221+0000 7f6d7f59e700 1 --2- 172.21.15.40:0/2885716002 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6d680a1350 0x7f6d680a3800 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.221+0000 7f6d7f59e700 1 --2- 172.21.15.40:0/2885716002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d800710c0 0x7f6d80080e80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.221+0000 7f6d7f59e700 1 --2- 172.21.15.40:0/2885716002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d80071a10 0x7f6d800813c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.221+0000 7f6d7f59e700 1 -- 172.21.15.40:0/2885716002 >> 172.21.15.40:0/2885716002 conn(0x7f6d8006c910 msgr2=0x7f6d8006e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.233+0000 7f6d7f59e700 1 -- 172.21.15.40:0/2885716002 shutdown_connections 2023-07-18T03:35:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:17.233+0000 7f6d7f59e700 1 -- 172.21.15.40:0/2885716002 wait complete. 2023-07-18T03:35:19.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:19 smithi134 ceph-mon[98216]: pgmap v426: 97 pgs: 97 active+clean; 972 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 742 op/s 2023-07-18T03:35:19.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:19 smithi040 ceph-mon[96439]: pgmap v426: 97 pgs: 97 active+clean; 972 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 742 op/s 2023-07-18T03:35:19.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:19 smithi064 ceph-mon[98059]: pgmap v426: 97 pgs: 97 active+clean; 972 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 742 op/s 2023-07-18T03:35:21.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:21 smithi134 ceph-mon[98216]: pgmap v427: 97 pgs: 97 active+clean; 978 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 729 op/s 2023-07-18T03:35:21.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:21 smithi040 ceph-mon[96439]: pgmap v427: 97 pgs: 97 active+clean; 978 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 729 op/s 2023-07-18T03:35:21.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:21 smithi064 ceph-mon[98059]: pgmap v427: 97 pgs: 97 active+clean; 978 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 729 op/s 2023-07-18T03:35:23.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:22 smithi134 ceph-mon[98216]: pgmap v428: 97 pgs: 97 active+clean; 986 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 829 op/s 2023-07-18T03:35:23.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:22 smithi040 ceph-mon[96439]: pgmap v428: 97 pgs: 97 active+clean; 986 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 829 op/s 2023-07-18T03:35:23.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:22 smithi064 ceph-mon[98059]: pgmap v428: 97 pgs: 97 active+clean; 986 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 829 op/s 2023-07-18T03:35:23.170 INFO:tasks.workunit.client.0.smithi040.stdout: 261 1674454 633 1161201 681 458128 1270 2023-07-18T03:35:25.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:25 smithi134 ceph-mon[98216]: pgmap v429: 97 pgs: 97 active+clean; 995 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 843 op/s 2023-07-18T03:35:25.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:25 smithi040 ceph-mon[96439]: pgmap v429: 97 pgs: 97 active+clean; 995 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 843 op/s 2023-07-18T03:35:25.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:25 smithi064 ceph-mon[98059]: pgmap v429: 97 pgs: 97 active+clean; 995 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 843 op/s 2023-07-18T03:35:27.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:27 smithi134 ceph-mon[98216]: pgmap v430: 97 pgs: 97 active+clean; 996 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 780 op/s 2023-07-18T03:35:27.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:27 smithi040 ceph-mon[96439]: pgmap v430: 97 pgs: 97 active+clean; 996 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 780 op/s 2023-07-18T03:35:27.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:27 smithi064 ceph-mon[98059]: pgmap v430: 97 pgs: 97 active+clean; 996 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 780 op/s 2023-07-18T03:35:29.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:29 smithi134 ceph-mon[98216]: pgmap v431: 97 pgs: 97 active+clean; 1006 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.8 MiB/s wr, 717 op/s 2023-07-18T03:35:29.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:29 smithi040 ceph-mon[96439]: pgmap v431: 97 pgs: 97 active+clean; 1006 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.8 MiB/s wr, 717 op/s 2023-07-18T03:35:29.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:29 smithi064 ceph-mon[98059]: pgmap v431: 97 pgs: 97 active+clean; 1006 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.8 MiB/s wr, 717 op/s 2023-07-18T03:35:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.502+0000 7f5f538b6700 1 -- 172.21.15.40:0/3539735758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4c0ffba0 msgr2=0x7f5f4c0fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.502+0000 7f5f538b6700 1 --2- 172.21.15.40:0/3539735758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4c0ffba0 0x7f5f4c0fffb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009990 tx=0x7f5f48009ca0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.502+0000 7f5f538b6700 1 -- 172.21.15.40:0/3539735758 shutdown_connections 2023-07-18T03:35:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.502+0000 7f5f538b6700 1 --2- 172.21.15.40:0/3539735758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4c101720 0x7f5f4c107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.502+0000 7f5f538b6700 1 --2- 172.21.15.40:0/3539735758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4c0ffba0 0x7f5f4c0fffb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.502+0000 7f5f538b6700 1 --2- 172.21.15.40:0/3539735758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f4c100d90 0x7f5f4c1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.502+0000 7f5f538b6700 1 -- 172.21.15.40:0/3539735758 >> 172.21.15.40:0/3539735758 conn(0x7f5f4c0fb1c0 msgr2=0x7f5f4c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:30.499 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.502+0000 7f5f538b6700 1 -- 172.21.15.40:0/3539735758 shutdown_connections 2023-07-18T03:35:30.500 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.502+0000 7f5f538b6700 1 -- 172.21.15.40:0/3539735758 wait complete. 2023-07-18T03:35:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f538b6700 1 Processor -- start 2023-07-18T03:35:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f538b6700 1 -- start start 2023-07-18T03:35:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f538b6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4c0ffba0 0x7f5f4c195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f538b6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4c100d90 0x7f5f4c195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f538b6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f4c101720 0x7f5f4c196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f538b6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c197350 con 0x7f5f4c101720 2023-07-18T03:35:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f538b6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c1974c0 con 0x7f5f4c0ffba0 2023-07-18T03:35:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f50e51700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4c100d90 0x7f5f4c195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f51e53700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f4c101720 0x7f5f4c196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f51652700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4c0ffba0 0x7f5f4c195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f50e51700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4c100d90 0x7f5f4c195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49050/0 (socket says 172.21.15.40:49050) 2023-07-18T03:35:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f50e51700 1 -- 172.21.15.40:0/805817972 learned_addr learned my addr 172.21.15.40:0/805817972 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f538b6700 1 -- 172.21.15.40:0/805817972 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c197680 con 0x7f5f4c100d90 2023-07-18T03:35:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f50e51700 1 -- 172.21.15.40:0/805817972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4c0ffba0 msgr2=0x7f5f4c195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f50e51700 1 --2- 172.21.15.40:0/805817972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4c0ffba0 0x7f5f4c195930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f50e51700 1 -- 172.21.15.40:0/805817972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f4c101720 msgr2=0x7f5f4c196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f50e51700 1 --2- 172.21.15.40:0/805817972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f4c101720 0x7f5f4c196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f50e51700 1 -- 172.21.15.40:0/805817972 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f480095f0 con 0x7f5f4c100d90 2023-07-18T03:35:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.503+0000 7f5f51e53700 1 --2- 172.21.15.40:0/805817972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f4c101720 0x7f5f4c196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:35:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.504+0000 7f5f50e51700 1 --2- 172.21.15.40:0/805817972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4c100d90 0x7f5f4c195e70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00e9a0 tx=0x7f5f3c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.504+0000 7f5f427fc700 1 -- 172.21.15.40:0/805817972 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c009790 con 0x7f5f4c100d90 2023-07-18T03:35:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.504+0000 7f5f538b6700 1 -- 172.21.15.40:0/805817972 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4c197960 con 0x7f5f4c100d90 2023-07-18T03:35:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.504+0000 7f5f427fc700 1 -- 172.21.15.40:0/805817972 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f3c010430 con 0x7f5f4c100d90 2023-07-18T03:35:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.504+0000 7f5f427fc700 1 -- 172.21.15.40:0/805817972 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c018680 con 0x7f5f4c100d90 2023-07-18T03:35:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.504+0000 7f5f538b6700 1 -- 172.21.15.40:0/805817972 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c19e120 con 0x7f5f4c100d90 2023-07-18T03:35:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.506+0000 7f5f427fc700 1 -- 172.21.15.40:0/805817972 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5f3c0105a0 con 0x7f5f4c100d90 2023-07-18T03:35:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.507+0000 7f5f427fc700 1 --2- 172.21.15.40:0/805817972 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5f380a1150 0x7f5f380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.507+0000 7f5f51652700 1 --2- 172.21.15.40:0/805817972 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5f380a1150 0x7f5f380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.508+0000 7f5f427fc700 1 -- 172.21.15.40:0/805817972 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5f3c014070 con 0x7f5f4c100d90 2023-07-18T03:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.508+0000 7f5f5fa3cb80 1 Processor -- start 2023-07-18T03:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.508+0000 7f5f5fa3cb80 1 -- start start 2023-07-18T03:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.508+0000 7f5f51652700 1 --2- 172.21.15.40:0/805817972 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5f380a1150 0x7f5f380a3600 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800b2e0 tx=0x7f5f48005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.508+0000 7f5f5fa3cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259cde6b0 0x564259e82300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.508+0000 7f5f5fa3cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e82840 0x564259e831a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.508+0000 7f5f5fa3cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259e836e0 0x564259e83b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.508+0000 7f5f5fa3cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564259d87c50 con 0x564259e82840 2023-07-18T03:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.508+0000 7f5f5fa3cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564259dda9e0 con 0x564259e836e0 2023-07-18T03:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.508+0000 7f5f5fa3cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564259dd3e70 con 0x564259cde6b0 2023-07-18T03:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.508+0000 7f5f50e51700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e82840 0x564259e831a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.508+0000 7f5f50e51700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e82840 0x564259e831a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60030/0 (socket says 172.21.15.40:60030) 2023-07-18T03:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.508+0000 7f5f50e51700 1 -- 172.21.15.40:0/112125341 learned_addr learned my addr 172.21.15.40:0/112125341 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.508+0000 7f5f51652700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259cde6b0 0x564259e82300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.508+0000 7f5f51e53700 1 --2- 172.21.15.40:0/112125341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259e836e0 0x564259e83b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.509+0000 7f5f50e51700 1 -- 172.21.15.40:0/112125341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259cde6b0 msgr2=0x564259e82300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.509+0000 7f5f50e51700 1 --2- 172.21.15.40:0/112125341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259cde6b0 0x564259e82300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.509+0000 7f5f50e51700 1 -- 172.21.15.40:0/112125341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259e836e0 msgr2=0x564259e83b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.509+0000 7f5f50e51700 1 --2- 172.21.15.40:0/112125341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259e836e0 0x564259e83b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.509+0000 7f5f50e51700 1 -- 172.21.15.40:0/112125341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564259c01d80 con 0x564259e82840 2023-07-18T03:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.509+0000 7f5f50e51700 1 --2- 172.21.15.40:0/112125341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e82840 0x564259e831a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c099e60 tx=0x7f5f3c099930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885741f98ffe1604 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.509+0000 7f5f377fe700 1 -- 172.21.15.40:0/112125341 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0a2320 con 0x564259e82840 2023-07-18T03:35:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.509+0000 7f5f377fe700 1 -- 172.21.15.40:0/112125341 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f3c098270 con 0x564259e82840 2023-07-18T03:35:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.509+0000 7f5f377fe700 1 -- 172.21.15.40:0/112125341 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0a20d0 con 0x564259e82840 2023-07-18T03:35:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.510+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/112125341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e82840 msgr2=0x564259e831a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.510+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/112125341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e82840 0x564259e831a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c099e60 tx=0x7f5f3c099930 comp rx=0 tx=0).stop 2023-07-18T03:35:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.510+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/112125341 shutdown_connections 2023-07-18T03:35:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.510+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/112125341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259e836e0 0x564259e83b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.510+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/112125341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259cde6b0 0x564259e82300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.510+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/112125341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e82840 0x564259e831a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.510+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/112125341 >> 172.21.15.40:0/112125341 conn(0x564259e77be0 msgr2=0x564259e7ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.510+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/112125341 shutdown_connections 2023-07-18T03:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.510+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/112125341 wait complete. 2023-07-18T03:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.510+0000 7f5f5fa3cb80 1 -- start start 2023-07-18T03:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.510+0000 7f5f5fa3cb80 1 Processor -- start 2023-07-18T03:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.510+0000 7f5f5fa3cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259cde6b0 0x564259e8b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.510+0000 7f5f5fa3cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259e82840 0x564259e8be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f5fa3cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e836e0 0x564259e92e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f5fa3cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564259d73a60 con 0x564259e836e0 2023-07-18T03:35:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f5fa3cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564259d73760 con 0x564259e82840 2023-07-18T03:35:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f5fa3cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564259c578c0 con 0x564259cde6b0 2023-07-18T03:35:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f5fa3cb80 10 client.0 _ll_register_callbacks cb 0x564259d71700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:35:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f51e53700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e836e0 0x564259e92e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f51652700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259cde6b0 0x564259e8b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f51e53700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e836e0 0x564259e92e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60038/0 (socket says 172.21.15.40:60038) 2023-07-18T03:35:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f51e53700 1 -- 172.21.15.40:0/3558225029 learned_addr learned my addr 172.21.15.40:0/3558225029 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f50e51700 1 --2- 172.21.15.40:0/3558225029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259e82840 0x564259e8be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f51e53700 1 -- 172.21.15.40:0/3558225029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259cde6b0 msgr2=0x564259e8b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f51e53700 1 --2- 172.21.15.40:0/3558225029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259cde6b0 0x564259e8b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f51e53700 1 -- 172.21.15.40:0/3558225029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259e82840 msgr2=0x564259e8be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f51e53700 1 --2- 172.21.15.40:0/3558225029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259e82840 0x564259e8be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f51e53700 1 -- 172.21.15.40:0/3558225029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c19e120 con 0x564259e836e0 2023-07-18T03:35:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f51e53700 1 --2- 172.21.15.40:0/3558225029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e836e0 0x564259e92e70 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5f44004710 tx=0x7f5f44004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9773acb7fb132b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:35:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f34ff9700 1 -- 172.21.15.40:0/3558225029 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4400e9c0 con 0x564259e836e0 2023-07-18T03:35:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f34ff9700 1 -- 172.21.15.40:0/3558225029 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5f4400eb20 con 0x564259e836e0 2023-07-18T03:35:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/3558225029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564259d2d490 con 0x564259e836e0 2023-07-18T03:35:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f34ff9700 1 -- 172.21.15.40:0/3558225029 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f4400ece0 con 0x564259e836e0 2023-07-18T03:35:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.511+0000 7f5f5fa3cb80 10 client.16281 fetch_fsmap learned FSMap version 37 2023-07-18T03:35:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.512+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/3558225029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x564259beac00 con 0x564259e836e0 2023-07-18T03:35:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.512+0000 7f5f34ff9700 1 -- 172.21.15.40:0/3558225029 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564259d2d490 con 0x564259e836e0 2023-07-18T03:35:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.512+0000 7f5f34ff9700 1 -- 172.21.15.40:0/3558225029 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 37) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f5f4401c020 con 0x564259e836e0 2023-07-18T03:35:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.512+0000 7f5f5fa3cb80 10 client.16281 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:35:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.512+0000 7f5f5fa3cb80 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:35:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.512+0000 7f5f5fa3cb80 20 client.16281 populate_metadata read hostname 'smithi040' 2023-07-18T03:35:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.512+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/3558225029 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564259e97880 0x564259e99c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.512+0000 7f5f5fa3cb80 4 client.16281 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:35:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.512+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/3558225029 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564259dcce50 con 0x564259e97880 2023-07-18T03:35:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.512+0000 7f5f51652700 1 --2- 172.21.15.40:0/3558225029 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564259e97880 0x564259e99c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.513+0000 7f5f34ff9700 10 client.16281 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.513+0000 7f5f51652700 1 --2- 172.21.15.40:0/3558225029 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564259e97880 0x564259e99c90 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9e6bfdab0e406b8 server_cookie=8dad8abd422b2e80 in_seq=0 out_seq=0 2023-07-18T03:35:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.513+0000 7f5f34ff9700 1 -- 172.21.15.40:0/3558225029 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564259dcce50 con 0x564259e97880 2023-07-18T03:35:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.513+0000 7f5f34ff9700 10 client.16281 handle_command_reply: tid=0 2023-07-18T03:35:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.513+0000 7f5f5fa3cb80 1 client.16281 shutdown 2023-07-18T03:35:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.513+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/3558225029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e836e0 msgr2=0x564259e92e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.514+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/3558225029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e836e0 0x564259e92e70 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5f44004710 tx=0x7f5f44004a20 comp rx=0 tx=0).stop 2023-07-18T03:35:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.514+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/3558225029 shutdown_connections 2023-07-18T03:35:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.514+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/3558225029 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564259e97880 0x564259e99c90 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.514+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/3558225029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e836e0 0x564259e92e70 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.514+0000 7f5f34ff9700 0 client.16281 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.514+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/3558225029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259cde6b0 0x564259e8b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.514+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/3558225029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259e82840 0x564259e8be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.514+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/3558225029 >> 172.21.15.40:0/3558225029 conn(0x564259e77be0 msgr2=0x564259e7c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.514+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/3558225029 shutdown_connections 2023-07-18T03:35:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.514+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/3558225029 wait complete. 2023-07-18T03:35:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.514+0000 7f5f5fa3cb80 20 client.16281 trim_cache size 0 max 16384 2023-07-18T03:35:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f5fa3cb80 1 Processor -- start 2023-07-18T03:35:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f5fa3cb80 1 -- start start 2023-07-18T03:35:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f5fa3cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259cde6b0 0x564259e8b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f5fa3cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259e82840 0x564259e92d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f5fa3cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259e836e0 0x564259e8bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f5fa3cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564259e8c360 con 0x564259cde6b0 2023-07-18T03:35:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f5fa3cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564259e97440 con 0x564259e836e0 2023-07-18T03:35:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f5fa3cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564259e79d00 con 0x564259e82840 2023-07-18T03:35:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f51652700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259cde6b0 0x564259e8b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f51652700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259cde6b0 0x564259e8b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60042/0 (socket says 172.21.15.40:60042) 2023-07-18T03:35:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f51652700 1 -- 172.21.15.40:0/2605242302 learned_addr learned my addr 172.21.15.40:0/2605242302 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f51e53700 1 --2- 172.21.15.40:0/2605242302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259e836e0 0x564259e8bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f51652700 1 -- 172.21.15.40:0/2605242302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259e82840 msgr2=0x564259e92d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f51652700 1 --2- 172.21.15.40:0/2605242302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259e82840 0x564259e92d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f51652700 1 -- 172.21.15.40:0/2605242302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259e836e0 msgr2=0x564259e8bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f51652700 1 --2- 172.21.15.40:0/2605242302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259e836e0 0x564259e8bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f51652700 1 -- 172.21.15.40:0/2605242302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564259c281f0 con 0x564259cde6b0 2023-07-18T03:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.530+0000 7f5f51e53700 1 --2- 172.21.15.40:0/2605242302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259e836e0 0x564259e8bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.531+0000 7f5f51652700 1 --2- 172.21.15.40:0/2605242302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259cde6b0 0x564259e8b8b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c100bf0 tx=0x7f5f48020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1dad23be2d83cf8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.531+0000 7f5f37fff700 1 -- 172.21.15.40:0/2605242302 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4802b040 con 0x564259cde6b0 2023-07-18T03:35:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.531+0000 7f5f37fff700 1 -- 172.21.15.40:0/2605242302 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f4802c040 con 0x564259cde6b0 2023-07-18T03:35:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.531+0000 7f5f37fff700 1 -- 172.21.15.40:0/2605242302 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f48004e20 con 0x564259cde6b0 2023-07-18T03:35:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/2605242302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259cde6b0 msgr2=0x564259e8b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/2605242302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259cde6b0 0x564259e8b8b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c100bf0 tx=0x7f5f48020000 comp rx=0 tx=0).stop 2023-07-18T03:35:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/2605242302 shutdown_connections 2023-07-18T03:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/2605242302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259e836e0 0x564259e8bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/2605242302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259cde6b0 0x564259e8b8b0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/2605242302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259e82840 0x564259e92d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/2605242302 >> 172.21.15.40:0/2605242302 conn(0x564259e77be0 msgr2=0x564259e86550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/2605242302 shutdown_connections 2023-07-18T03:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/2605242302 wait complete. 2023-07-18T03:35:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 1 -- start start 2023-07-18T03:35:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 1 Processor -- start 2023-07-18T03:35:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259cde6b0 0x564259e8ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259e82840 0x564259e8f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e836e0 0x564259e92230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564259e92770 con 0x564259e836e0 2023-07-18T03:35:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564259e928e0 con 0x564259cde6b0 2023-07-18T03:35:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564259e92aa0 con 0x564259e82840 2023-07-18T03:35:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.532+0000 7f5f5fa3cb80 10 client.0 _ll_register_callbacks cb 0x7f5f44004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:35:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f51e53700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e836e0 0x564259e92230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f50e51700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259e82840 0x564259e8f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f51652700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259cde6b0 0x564259e8ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f51e53700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e836e0 0x564259e92230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60052/0 (socket says 172.21.15.40:60052) 2023-07-18T03:35:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f51e53700 1 -- 172.21.15.40:0/3928299161 learned_addr learned my addr 172.21.15.40:0/3928299161 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f51e53700 1 -- 172.21.15.40:0/3928299161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259e82840 msgr2=0x564259e8f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f51e53700 1 --2- 172.21.15.40:0/3928299161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259e82840 0x564259e8f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f51e53700 1 -- 172.21.15.40:0/3928299161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259cde6b0 msgr2=0x564259e8ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f51e53700 1 --2- 172.21.15.40:0/3928299161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259cde6b0 0x564259e8ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f51e53700 1 -- 172.21.15.40:0/3928299161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c108910 con 0x564259e836e0 2023-07-18T03:35:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f51e53700 1 --2- 172.21.15.40:0/3928299161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e836e0 0x564259e92230 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5f440046e0 tx=0x7f5f440060f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42f059ba578f776 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:35:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f34ff9700 1 -- 172.21.15.40:0/3928299161 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44004ea0 con 0x564259e836e0 2023-07-18T03:35:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f34ff9700 1 -- 172.21.15.40:0/3928299161 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5f4401e9e0 con 0x564259e836e0 2023-07-18T03:35:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/3928299161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f3c098270 con 0x564259e836e0 2023-07-18T03:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f34ff9700 1 -- 172.21.15.40:0/3928299161 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f4401e820 con 0x564259e836e0 2023-07-18T03:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f5fa3cb80 10 client.16287 fetch_fsmap learned FSMap version 37 2023-07-18T03:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.533+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/3928299161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x564259c00250 con 0x564259e836e0 2023-07-18T03:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.534+0000 7f5f34ff9700 1 -- 172.21.15.40:0/3928299161 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f3c098270 con 0x564259e836e0 2023-07-18T03:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.534+0000 7f5f34ff9700 1 -- 172.21.15.40:0/3928299161 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 37) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f5f4401c070 con 0x564259e836e0 2023-07-18T03:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.534+0000 7f5f5fa3cb80 10 client.16287 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.534+0000 7f5f5fa3cb80 10 client.16287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:35:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.534+0000 7f5f5fa3cb80 20 client.16287 populate_metadata read hostname 'smithi040' 2023-07-18T03:35:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.534+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/3928299161 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564259eec790 0x564259eeeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.534+0000 7f5f5fa3cb80 4 client.16287 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:35:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.534+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/3928299161 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564259d8eff0 con 0x564259eec790 2023-07-18T03:35:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.534+0000 7f5f51652700 1 --2- 172.21.15.40:0/3928299161 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564259eec790 0x564259eeeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.534+0000 7f5f34ff9700 10 client.16287 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:30.534+0000 7f5f51652700 1 --2- 172.21.15.40:0/3928299161 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564259eec790 0x564259eeeba0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e702639affd28d4 server_cookie=3bfce8a9c22afff9 in_seq=0 out_seq=0 2023-07-18T03:35:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.027+0000 7f5f34ff9700 1 -- 172.21.15.40:0/3928299161 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564259d8eff0 con 0x564259eec790 2023-07-18T03:35:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.027+0000 7f5f34ff9700 10 client.16287 handle_command_reply: tid=0 2023-07-18T03:35:31.025 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.027+0000 7f5f5fa3cb80 1 client.16287 shutdown 2023-07-18T03:35:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.028+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/3928299161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e836e0 msgr2=0x564259e92230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.028+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/3928299161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e836e0 0x564259e92230 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5f440046e0 tx=0x7f5f440060f0 comp rx=0 tx=0).stop 2023-07-18T03:35:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.028+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/3928299161 shutdown_connections 2023-07-18T03:35:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.028+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/3928299161 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564259eec790 0x564259eeeba0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.028+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/3928299161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564259e836e0 0x564259e92230 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.028+0000 7f5f34ff9700 0 client.16287 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.028+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/3928299161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564259cde6b0 0x564259e8ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.028+0000 7f5f5fa3cb80 1 --2- 172.21.15.40:0/3928299161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564259e82840 0x564259e8f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.028+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/3928299161 >> 172.21.15.40:0/3928299161 conn(0x564259e77be0 msgr2=0x564259e86550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.028+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/3928299161 shutdown_connections 2023-07-18T03:35:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.028+0000 7f5f5fa3cb80 1 -- 172.21.15.40:0/3928299161 wait complete. 2023-07-18T03:35:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.028+0000 7f5f5fa3cb80 20 client.16287 trim_cache size 0 max 16384 2023-07-18T03:35:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:35:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:35:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:35:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:35:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.028+0000 7f5f538b6700 1 -- 172.21.15.40:0/805817972 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5f380a1150 msgr2=0x7f5f380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.028+0000 7f5f538b6700 1 --2- 172.21.15.40:0/805817972 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5f380a1150 0x7f5f380a3600 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800b2e0 tx=0x7f5f48005fb0 comp rx=0 tx=0).stop 2023-07-18T03:35:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.028+0000 7f5f538b6700 1 -- 172.21.15.40:0/805817972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4c100d90 msgr2=0x7f5f4c195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.028+0000 7f5f538b6700 1 --2- 172.21.15.40:0/805817972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4c100d90 0x7f5f4c195e70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00e9a0 tx=0x7f5f3c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:35:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.029+0000 7f5f538b6700 1 -- 172.21.15.40:0/805817972 shutdown_connections 2023-07-18T03:35:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.029+0000 7f5f538b6700 1 --2- 172.21.15.40:0/805817972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f4c101720 0x7f5f4c196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.029+0000 7f5f538b6700 1 --2- 172.21.15.40:0/805817972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4c0ffba0 0x7f5f4c195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.029+0000 7f5f538b6700 1 --2- 172.21.15.40:0/805817972 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5f380a1150 0x7f5f380a3600 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.029+0000 7f5f538b6700 1 --2- 172.21.15.40:0/805817972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f4c100d90 0x7f5f4c195e70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.029+0000 7f5f538b6700 1 -- 172.21.15.40:0/805817972 >> 172.21.15.40:0/805817972 conn(0x7f5f4c0fb1c0 msgr2=0x7f5f4c1042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.029+0000 7f5f538b6700 1 -- 172.21.15.40:0/805817972 shutdown_connections 2023-07-18T03:35:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:31.029+0000 7f5f538b6700 1 -- 172.21.15.40:0/805817972 wait complete. 2023-07-18T03:35:31.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:31 smithi134 ceph-mon[98216]: pgmap v432: 97 pgs: 97 active+clean; 1006 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 275 op/s 2023-07-18T03:35:31.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:31 smithi040 ceph-mon[96439]: pgmap v432: 97 pgs: 97 active+clean; 1006 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 275 op/s 2023-07-18T03:35:31.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:31 smithi064 ceph-mon[98059]: pgmap v432: 97 pgs: 97 active+clean; 1006 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 275 op/s 2023-07-18T03:35:33.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:32 smithi134 ceph-mon[98216]: pgmap v433: 97 pgs: 97 active+clean; 1012 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 364 op/s 2023-07-18T03:35:33.079 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:32 smithi040 ceph-mon[96439]: pgmap v433: 97 pgs: 97 active+clean; 1012 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 364 op/s 2023-07-18T03:35:33.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:32 smithi064 ceph-mon[98059]: pgmap v433: 97 pgs: 97 active+clean; 1012 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 364 op/s 2023-07-18T03:35:33.170 INFO:tasks.workunit.client.0.smithi040.stdout: 279 119163 858 83434 1108 32375 614 2023-07-18T03:35:35.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:35 smithi134 ceph-mon[98216]: pgmap v434: 97 pgs: 97 active+clean; 1011 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 340 op/s 2023-07-18T03:35:35.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:35 smithi040 ceph-mon[96439]: pgmap v434: 97 pgs: 97 active+clean; 1011 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 340 op/s 2023-07-18T03:35:35.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:35 smithi064 ceph-mon[98059]: pgmap v434: 97 pgs: 97 active+clean; 1011 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 340 op/s 2023-07-18T03:35:37.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:37 smithi134 ceph-mon[98216]: pgmap v435: 97 pgs: 97 active+clean; 1012 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 307 op/s 2023-07-18T03:35:37.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:37 smithi040 ceph-mon[96439]: pgmap v435: 97 pgs: 97 active+clean; 1012 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 307 op/s 2023-07-18T03:35:37.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:37 smithi064 ceph-mon[98059]: pgmap v435: 97 pgs: 97 active+clean; 1012 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 307 op/s 2023-07-18T03:35:38.997 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:35:39.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:39 smithi134 ceph-mon[98216]: pgmap v436: 97 pgs: 97 active+clean; 1015 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 307 op/s 2023-07-18T03:35:39.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:39 smithi040 ceph-mon[96439]: pgmap v436: 97 pgs: 97 active+clean; 1015 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 307 op/s 2023-07-18T03:35:39.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:39 smithi064 ceph-mon[98059]: pgmap v436: 97 pgs: 97 active+clean; 1015 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 307 op/s 2023-07-18T03:35:40.252 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:40 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:35:40.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:40 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:35:40.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:40 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:35:41.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.301+0000 7f9b4ffff700 1 -- 172.21.15.40:0/369314068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b500711d0 msgr2=0x7f9b500715e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:41.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.301+0000 7f9b4ffff700 1 --2- 172.21.15.40:0/369314068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b500711d0 0x7f9b500715e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009990 tx=0x7f9b40009ca0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.302+0000 7f9b4ffff700 1 -- 172.21.15.40:0/369314068 shutdown_connections 2023-07-18T03:35:41.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.302+0000 7f9b4ffff700 1 --2- 172.21.15.40:0/369314068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b5010bba0 0x7f9b5010e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.302+0000 7f9b4ffff700 1 --2- 172.21.15.40:0/369314068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b500711d0 0x7f9b500715e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.302+0000 7f9b4ffff700 1 --2- 172.21.15.40:0/369314068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b50071b20 0x7f9b5010b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.302+0000 7f9b4ffff700 1 -- 172.21.15.40:0/369314068 >> 172.21.15.40:0/369314068 conn(0x7f9b5006c8b0 msgr2=0x7f9b5006ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:41.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.302+0000 7f9b4ffff700 1 -- 172.21.15.40:0/369314068 shutdown_connections 2023-07-18T03:35:41.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.302+0000 7f9b4ffff700 1 -- 172.21.15.40:0/369314068 wait complete. 2023-07-18T03:35:41.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.303+0000 7f9b4ffff700 1 Processor -- start 2023-07-18T03:35:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.304+0000 7f9b4ffff700 1 -- start start 2023-07-18T03:35:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.304+0000 7f9b4ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b500711d0 0x7f9b501139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.304+0000 7f9b4ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b50071b20 0x7f9b50113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.304+0000 7f9b4ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b5010bba0 0x7f9b501b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.304+0000 7f9b4ffff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b50117010 con 0x7f9b50071b20 2023-07-18T03:35:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.304+0000 7f9b4ffff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b50117180 con 0x7f9b5010bba0 2023-07-18T03:35:41.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.304+0000 7f9b4ffff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b501b4c60 con 0x7f9b500711d0 2023-07-18T03:35:41.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.313+0000 7f9b4e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b50071b20 0x7f9b50113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:41.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.313+0000 7f9b4e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b50071b20 0x7f9b50113f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60064/0 (socket says 172.21.15.40:60064) 2023-07-18T03:35:41.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.313+0000 7f9b4e7fc700 1 -- 172.21.15.40:0/3262949410 learned_addr learned my addr 172.21.15.40:0/3262949410 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:41.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.313+0000 7f9b4effd700 1 --2- 172.21.15.40:0/3262949410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b500711d0 0x7f9b501139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:41.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.313+0000 7f9b4f7fe700 1 --2- 172.21.15.40:0/3262949410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b5010bba0 0x7f9b501b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:41.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.314+0000 7f9b4f7fe700 1 -- 172.21.15.40:0/3262949410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b500711d0 msgr2=0x7f9b501139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:41.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.314+0000 7f9b4f7fe700 1 --2- 172.21.15.40:0/3262949410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b500711d0 0x7f9b501139f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.314+0000 7f9b4f7fe700 1 -- 172.21.15.40:0/3262949410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b50071b20 msgr2=0x7f9b50113f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:41.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.314+0000 7f9b4f7fe700 1 --2- 172.21.15.40:0/3262949410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b50071b20 0x7f9b50113f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.314+0000 7f9b4f7fe700 1 -- 172.21.15.40:0/3262949410 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b400095f0 con 0x7f9b5010bba0 2023-07-18T03:35:41.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.315+0000 7f9b4e7fc700 1 --2- 172.21.15.40:0/3262949410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b50071b20 0x7f9b50113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:35:41.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.321+0000 7f9b4f7fe700 1 --2- 172.21.15.40:0/3262949410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b5010bba0 0x7f9b501b4720 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800ab40 tx=0x7f9b4800af00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:41.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.332+0000 7f9b549fe700 1 -- 172.21.15.40:0/3262949410 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4800f070 con 0x7f9b5010bba0 2023-07-18T03:35:41.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.333+0000 7f9b549fe700 1 -- 172.21.15.40:0/3262949410 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b48007a70 con 0x7f9b5010bba0 2023-07-18T03:35:41.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.333+0000 7f9b549fe700 1 -- 172.21.15.40:0/3262949410 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b48013b30 con 0x7f9b5010bba0 2023-07-18T03:35:41.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.333+0000 7f9b4ffff700 1 -- 172.21.15.40:0/3262949410 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b501b4e00 con 0x7f9b5010bba0 2023-07-18T03:35:41.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.333+0000 7f9b4ffff700 1 -- 172.21.15.40:0/3262949410 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b501b5250 con 0x7f9b5010bba0 2023-07-18T03:35:41.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.339+0000 7f9b61b86b80 1 Processor -- start 2023-07-18T03:35:41.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.340+0000 7f9b61b86b80 1 -- start start 2023-07-18T03:35:41.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.340+0000 7f9b61b86b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656f9720 0x5566656f9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.340+0000 7f9b61b86b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fa070 0x5566656fa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.340+0000 7f9b61b86b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656fae30 0x5566656fb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.340+0000 7f9b61b86b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566655fecb0 con 0x5566656fae30 2023-07-18T03:35:41.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.340+0000 7f9b61b86b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556665651a40 con 0x5566656f9720 2023-07-18T03:35:41.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.340+0000 7f9b61b86b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666564aed0 con 0x5566656fa070 2023-07-18T03:35:41.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.340+0000 7f9b4effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fa070 0x5566656fa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:41.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.340+0000 7f9b4effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fa070 0x5566656fa8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49100/0 (socket says 172.21.15.40:49100) 2023-07-18T03:35:41.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.340+0000 7f9b4effd700 1 -- 172.21.15.40:0/3011330458 learned_addr learned my addr 172.21.15.40:0/3011330458 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:41.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.341+0000 7f9b4e7fc700 1 --2- 172.21.15.40:0/3011330458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656f9720 0x5566656f9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:41.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.341+0000 7f9b4e7fc700 1 --2- 172.21.15.40:0/3011330458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656fae30 0x5566656fb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:41.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.343+0000 7f9b4effd700 1 -- 172.21.15.40:0/3011330458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656f9720 msgr2=0x5566656f9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:41.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.343+0000 7f9b4effd700 1 --2- 172.21.15.40:0/3011330458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656f9720 0x5566656f9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.343+0000 7f9b4effd700 1 -- 172.21.15.40:0/3011330458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656fae30 msgr2=0x5566656fb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:41.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.343+0000 7f9b4effd700 1 --2- 172.21.15.40:0/3011330458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656fae30 0x5566656fb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.343+0000 7f9b4effd700 1 -- 172.21.15.40:0/3011330458 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556665478de0 con 0x5566656fa070 2023-07-18T03:35:41.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.343+0000 7f9b549fe700 1 -- 172.21.15.40:0/3262949410 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9b48013c90 con 0x7f9b5010bba0 2023-07-18T03:35:41.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.345+0000 7f9b549fe700 1 --2- 172.21.15.40:0/3262949410 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9b380a1430 0x7f9b380a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.345+0000 7f9b549fe700 1 -- 172.21.15.40:0/3262949410 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9b480ccf90 con 0x7f9b5010bba0 2023-07-18T03:35:41.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.345+0000 7f9b4effd700 1 --2- 172.21.15.40:0/3011330458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fa070 0x5566656fa8a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9b40000c00 tx=0x7f9b40009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f41ce9e75c2a018 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.347+0000 7f9b4e7fc700 1 --2- 172.21.15.40:0/3262949410 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9b380a1430 0x7f9b380a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:41.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.352+0000 7f9b35ffb700 1 -- 172.21.15.40:0/3011330458 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4001d070 con 0x5566656fa070 2023-07-18T03:35:41.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.352+0000 7f9b35ffb700 1 -- 172.21.15.40:0/3011330458 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b4000be80 con 0x5566656fa070 2023-07-18T03:35:41.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.352+0000 7f9b35ffb700 1 -- 172.21.15.40:0/3011330458 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40005580 con 0x5566656fa070 2023-07-18T03:35:41.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.359+0000 7f9b61b86b80 1 -- 172.21.15.40:0/3011330458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fa070 msgr2=0x5566656fa8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:41.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.359+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/3011330458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fa070 0x5566656fa8a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9b40000c00 tx=0x7f9b40009ee0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.360+0000 7f9b61b86b80 1 -- 172.21.15.40:0/3011330458 shutdown_connections 2023-07-18T03:35:41.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.360+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/3011330458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656fae30 0x5566656fb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.360+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/3011330458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656f9720 0x5566656f9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.360+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/3011330458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fa070 0x5566656fa8a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.360+0000 7f9b61b86b80 1 -- 172.21.15.40:0/3011330458 >> 172.21.15.40:0/3011330458 conn(0x5566656eeff0 msgr2=0x5566656f4460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:41.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.360+0000 7f9b4e7fc700 1 --2- 172.21.15.40:0/3262949410 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9b380a1430 0x7f9b380a38e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9b44006f00 tx=0x7f9b44006e90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:41.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.361+0000 7f9b61b86b80 1 -- 172.21.15.40:0/3011330458 shutdown_connections 2023-07-18T03:35:41.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.361+0000 7f9b61b86b80 1 -- 172.21.15.40:0/3011330458 wait complete. 2023-07-18T03:35:41.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.361+0000 7f9b61b86b80 1 -- start start 2023-07-18T03:35:41.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.361+0000 7f9b61b86b80 1 Processor -- start 2023-07-18T03:35:41.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.362+0000 7f9b61b86b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656f9720 0x5566657032a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.362+0000 7f9b61b86b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656fa070 0x5566657037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.362+0000 7f9b61b86b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656fae30 0x55666570a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.362+0000 7f9b61b86b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566655eaac0 con 0x5566656fae30 2023-07-18T03:35:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.362+0000 7f9b61b86b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566655ea7c0 con 0x5566656fa070 2023-07-18T03:35:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.362+0000 7f9b61b86b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566654ce920 con 0x5566656f9720 2023-07-18T03:35:41.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.362+0000 7f9b61b86b80 10 client.0 _ll_register_callbacks cb 0x5566655e8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:35:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.364+0000 7f9b4effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656f9720 0x5566657032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.364+0000 7f9b4effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656f9720 0x5566657032a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49106/0 (socket says 172.21.15.40:49106) 2023-07-18T03:35:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.364+0000 7f9b4effd700 1 -- 172.21.15.40:0/1949838703 learned_addr learned my addr 172.21.15.40:0/1949838703 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.364+0000 7f9b4e7fc700 1 --2- 172.21.15.40:0/1949838703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656fa070 0x5566657037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:41.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.364+0000 7f9b4effd700 1 -- 172.21.15.40:0/1949838703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656fa070 msgr2=0x5566657037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:41.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.364+0000 7f9b4effd700 1 --2- 172.21.15.40:0/1949838703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656fa070 0x5566657037e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.364+0000 7f9b4effd700 1 -- 172.21.15.40:0/1949838703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656fae30 msgr2=0x55666570a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:41.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.364+0000 7f9b4effd700 1 --2- 172.21.15.40:0/1949838703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656fae30 0x55666570a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.364+0000 7f9b4effd700 1 -- 172.21.15.40:0/1949838703 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b50117590 con 0x5566656f9720 2023-07-18T03:35:41.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.370+0000 7f9b4effd700 1 --2- 172.21.15.40:0/1949838703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656f9720 0x5566657032a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009ce0 tx=0x7f9b400055e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb1be9175e5c3b01 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:41.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.377+0000 7f9b337f6700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:35:41.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.377+0000 7f9b337f6700 1 -- 172.21.15.40:0/1949838703 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4001d070 con 0x5566656f9720 2023-07-18T03:35:41.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.377+0000 7f9b337f6700 1 -- 172.21.15.40:0/1949838703 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9b40003d70 con 0x5566656f9720 2023-07-18T03:35:41.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.377+0000 7f9b61b86b80 1 -- 172.21.15.40:0/1949838703 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566655a44f0 con 0x5566656f9720 2023-07-18T03:35:41.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.391+0000 7f9b337f6700 1 -- 172.21.15.40:0/1949838703 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b4000f460 con 0x5566656f9720 2023-07-18T03:35:41.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.391+0000 7f9b61b86b80 10 client.25948 fetch_fsmap learned FSMap version 37 2023-07-18T03:35:41.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.391+0000 7f9b61b86b80 1 -- 172.21.15.40:0/1949838703 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x556665461c60 con 0x5566656f9720 2023-07-18T03:35:41.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.395+0000 7f9b337f6700 1 -- 172.21.15.40:0/1949838703 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5566655a44f0 con 0x5566656f9720 2023-07-18T03:35:41.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.395+0000 7f9b337f6700 1 -- 172.21.15.40:0/1949838703 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 37) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f9b40029030 con 0x5566656f9720 2023-07-18T03:35:41.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.395+0000 7f9b61b86b80 10 client.25948 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:35:41.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.395+0000 7f9b61b86b80 10 client.25948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:35:41.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.395+0000 7f9b61b86b80 20 client.25948 populate_metadata read hostname 'smithi040' 2023-07-18T03:35:41.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.395+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/1949838703 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55666570ec70 0x556665711080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.395+0000 7f9b61b86b80 4 client.25948 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:35:41.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.395+0000 7f9b61b86b80 1 -- 172.21.15.40:0/1949838703 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556665643eb0 con 0x55666570ec70 2023-07-18T03:35:41.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.399+0000 7f9b4e7fc700 1 --2- 172.21.15.40:0/1949838703 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55666570ec70 0x556665711080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:41.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.423+0000 7f9b4e7fc700 1 --2- 172.21.15.40:0/1949838703 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55666570ec70 0x556665711080 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91cca875df62cab server_cookie=55710c400bf3a8bd in_seq=0 out_seq=0 2023-07-18T03:35:41.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.423+0000 7f9b337f6700 10 client.25948 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:41.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.424+0000 7f9b337f6700 1 -- 172.21.15.40:0/1949838703 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556665643eb0 con 0x55666570ec70 2023-07-18T03:35:41.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.424+0000 7f9b337f6700 10 client.25948 handle_command_reply: tid=0 2023-07-18T03:35:41.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.424+0000 7f9b61b86b80 1 client.25948 shutdown 2023-07-18T03:35:41.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.433+0000 7f9b61b86b80 1 -- 172.21.15.40:0/1949838703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656f9720 msgr2=0x5566657032a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:41.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.433+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/1949838703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656f9720 0x5566657032a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009ce0 tx=0x7f9b400055e0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.433+0000 7f9b61b86b80 1 -- 172.21.15.40:0/1949838703 shutdown_connections 2023-07-18T03:35:41.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.433+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/1949838703 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55666570ec70 0x556665711080 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.433+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/1949838703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656fae30 0x55666570a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.433+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/1949838703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656f9720 0x5566657032a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.433+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/1949838703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656fa070 0x5566657037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.433+0000 7f9b61b86b80 1 -- 172.21.15.40:0/1949838703 >> 172.21.15.40:0/1949838703 conn(0x5566656eeff0 msgr2=0x5566656f3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:41.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.433+0000 7f9b337f6700 0 client.25948 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:41.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.434+0000 7f9b61b86b80 1 -- 172.21.15.40:0/1949838703 shutdown_connections 2023-07-18T03:35:41.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.434+0000 7f9b61b86b80 1 -- 172.21.15.40:0/1949838703 wait complete. 2023-07-18T03:35:41.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.434+0000 7f9b61b86b80 20 client.25948 trim_cache size 0 max 16384 2023-07-18T03:35:41.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.510+0000 7f9b61b86b80 1 Processor -- start 2023-07-18T03:35:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.510+0000 7f9b61b86b80 1 -- start start 2023-07-18T03:35:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.511+0000 7f9b61b86b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656f9720 0x556665701140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.511+0000 7f9b61b86b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656fa070 0x556665701680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.511+0000 7f9b61b86b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fae30 0x55666570a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.512+0000 7f9b4e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656fa070 0x556665701680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.512+0000 7f9b4e7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656fa070 0x556665701680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43412/0 (socket says 172.21.15.40:43412) 2023-07-18T03:35:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.512+0000 7f9b4e7fc700 1 -- 172.21.15.40:0/2355794206 learned_addr learned my addr 172.21.15.40:0/2355794206 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.512+0000 7f9b4f7fe700 1 --2- 172.21.15.40:0/2355794206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fae30 0x55666570a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.512+0000 7f9b61b86b80 1 -- 172.21.15.40:0/2355794206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x556665643eb0 con 0x5566656f9720 2023-07-18T03:35:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.512+0000 7f9b61b86b80 1 -- 172.21.15.40:0/2355794206 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566655a44f0 con 0x5566656fa070 2023-07-18T03:35:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.512+0000 7f9b61b86b80 1 -- 172.21.15.40:0/2355794206 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566657042f0 con 0x5566656fae30 2023-07-18T03:35:41.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.522+0000 7f9b4effd700 1 --2- 172.21.15.40:0/2355794206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656f9720 0x556665701140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:41.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.522+0000 7f9b4f7fe700 1 -- 172.21.15.40:0/2355794206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656fa070 msgr2=0x556665701680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:41.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.522+0000 7f9b4f7fe700 1 --2- 172.21.15.40:0/2355794206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656fa070 0x556665701680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.522+0000 7f9b4f7fe700 1 -- 172.21.15.40:0/2355794206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656f9720 msgr2=0x556665701140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:41.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.522+0000 7f9b4f7fe700 1 --2- 172.21.15.40:0/2355794206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656f9720 0x556665701140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.522+0000 7f9b4f7fe700 1 -- 172.21.15.40:0/2355794206 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55666549f250 con 0x5566656fae30 2023-07-18T03:35:41.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.522+0000 7f9b4f7fe700 1 --2- 172.21.15.40:0/2355794206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fae30 0x55666570a7b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9b4809a0b0 tx=0x7f9b4809a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad7e7bcc5fec2dc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:41.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:41 smithi134 ceph-mon[98216]: pgmap v437: 97 pgs: 97 active+clean; 1018 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2023-07-18T03:35:41.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:41 smithi040 ceph-mon[96439]: pgmap v437: 97 pgs: 97 active+clean; 1018 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2023-07-18T03:35:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.595+0000 7f9b367fc700 1 -- 172.21.15.40:0/2355794206 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4809a3b0 con 0x5566656fae30 2023-07-18T03:35:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.595+0000 7f9b367fc700 1 -- 172.21.15.40:0/2355794206 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b4809e3e0 con 0x5566656fae30 2023-07-18T03:35:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.595+0000 7f9b367fc700 1 -- 172.21.15.40:0/2355794206 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b480a00b0 con 0x5566656fae30 2023-07-18T03:35:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.597+0000 7f9b61b86b80 1 -- 172.21.15.40:0/2355794206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fae30 msgr2=0x55666570a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.597+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/2355794206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fae30 0x55666570a7b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9b4809a0b0 tx=0x7f9b4809a200 comp rx=0 tx=0).stop 2023-07-18T03:35:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.598+0000 7f9b61b86b80 1 -- 172.21.15.40:0/2355794206 shutdown_connections 2023-07-18T03:35:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.598+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/2355794206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fae30 0x55666570a7b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.598+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/2355794206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656f9720 0x556665701140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.598+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/2355794206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656fa070 0x556665701680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.598+0000 7f9b61b86b80 1 -- 172.21.15.40:0/2355794206 >> 172.21.15.40:0/2355794206 conn(0x5566656eeff0 msgr2=0x556665710c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.598+0000 7f9b61b86b80 1 -- 172.21.15.40:0/2355794206 shutdown_connections 2023-07-18T03:35:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.598+0000 7f9b61b86b80 1 -- 172.21.15.40:0/2355794206 wait complete. 2023-07-18T03:35:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.599+0000 7f9b61b86b80 1 -- start start 2023-07-18T03:35:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.599+0000 7f9b61b86b80 1 Processor -- start 2023-07-18T03:35:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.599+0000 7f9b61b86b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656f9720 0x556665706840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.599+0000 7f9b61b86b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fa070 0x556665706d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.599+0000 7f9b61b86b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656fae30 0x556665709da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.599+0000 7f9b61b86b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666570a2e0 con 0x5566656fae30 2023-07-18T03:35:41.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.599+0000 7f9b61b86b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666570a450 con 0x5566656f9720 2023-07-18T03:35:41.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.599+0000 7f9b61b86b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666570a610 con 0x5566656fa070 2023-07-18T03:35:41.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.599+0000 7f9b61b86b80 10 client.0 _ll_register_callbacks cb 0x7f9b44009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:35:41.606 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:41 smithi064 ceph-mon[98059]: pgmap v437: 97 pgs: 97 active+clean; 1018 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2023-07-18T03:35:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.607+0000 7f9b4e7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fa070 0x556665706d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.607+0000 7f9b4e7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fa070 0x556665706d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49120/0 (socket says 172.21.15.40:49120) 2023-07-18T03:35:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.607+0000 7f9b4e7fc700 1 -- 172.21.15.40:0/3159695716 learned_addr learned my addr 172.21.15.40:0/3159695716 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.607+0000 7f9b4effd700 1 --2- 172.21.15.40:0/3159695716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656f9720 0x556665706840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:41.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.608+0000 7f9b4f7fe700 1 --2- 172.21.15.40:0/3159695716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656fae30 0x556665709da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:41.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.618+0000 7f9b4e7fc700 1 -- 172.21.15.40:0/3159695716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656f9720 msgr2=0x556665706840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:41.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.618+0000 7f9b4e7fc700 1 --2- 172.21.15.40:0/3159695716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656f9720 0x556665706840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.618+0000 7f9b4e7fc700 1 -- 172.21.15.40:0/3159695716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656fae30 msgr2=0x556665709da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:41.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.618+0000 7f9b4e7fc700 1 --2- 172.21.15.40:0/3159695716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656fae30 0x556665709da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.618+0000 7f9b4e7fc700 1 -- 172.21.15.40:0/3159695716 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b501b5250 con 0x5566656fa070 2023-07-18T03:35:41.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.618+0000 7f9b4f7fe700 1 --2- 172.21.15.40:0/3159695716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656fae30 0x556665709da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:35:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.628+0000 7f9b4e7fc700 1 --2- 172.21.15.40:0/3159695716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fa070 0x556665706d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b44009a60 tx=0x7f9b44009380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e100a1aa5cec7212 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.629+0000 7f9b337f6700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:35:41.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.629+0000 7f9b337f6700 1 -- 172.21.15.40:0/3159695716 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b44017660 con 0x5566656fa070 2023-07-18T03:35:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.629+0000 7f9b337f6700 1 -- 172.21.15.40:0/3159695716 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9b4401d040 con 0x5566656fa070 2023-07-18T03:35:41.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.629+0000 7f9b61b86b80 1 -- 172.21.15.40:0/3159695716 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b4000be80 con 0x5566656fa070 2023-07-18T03:35:41.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.629+0000 7f9b337f6700 1 -- 172.21.15.40:0/3159695716 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b44024e90 con 0x5566656fa070 2023-07-18T03:35:41.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.640+0000 7f9b61b86b80 10 client.25954 fetch_fsmap learned FSMap version 37 2023-07-18T03:35:41.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.640+0000 7f9b61b86b80 1 -- 172.21.15.40:0/3159695716 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5566654772b0 con 0x5566656fa070 2023-07-18T03:35:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.651+0000 7f9b337f6700 1 -- 172.21.15.40:0/3159695716 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b4000be80 con 0x5566656fa070 2023-07-18T03:35:41.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.651+0000 7f9b337f6700 1 -- 172.21.15.40:0/3159695716 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 37) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f9b44033850 con 0x5566656fa070 2023-07-18T03:35:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.652+0000 7f9b61b86b80 10 client.25954 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:35:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.652+0000 7f9b61b86b80 10 client.25954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:35:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.652+0000 7f9b61b86b80 20 client.25954 populate_metadata read hostname 'smithi040' 2023-07-18T03:35:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.652+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/3159695716 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556665763d70 0x556665766180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.652+0000 7f9b61b86b80 4 client.25954 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:35:41.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.652+0000 7f9b61b86b80 1 -- 172.21.15.40:0/3159695716 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556665606050 con 0x556665763d70 2023-07-18T03:35:41.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.668+0000 7f9b4effd700 1 --2- 172.21.15.40:0/3159695716 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556665763d70 0x556665766180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:41.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.678+0000 7f9b4effd700 1 --2- 172.21.15.40:0/3159695716 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556665763d70 0x556665766180 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bfa5e27148e80ce server_cookie=4152a5ca2cbb93fe in_seq=0 out_seq=0 2023-07-18T03:35:41.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.678+0000 7f9b337f6700 10 client.25954 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:41.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.679+0000 7f9b337f6700 1 -- 172.21.15.40:0/3159695716 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x556665606050 con 0x556665763d70 2023-07-18T03:35:41.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.679+0000 7f9b337f6700 10 client.25954 handle_command_reply: tid=0 2023-07-18T03:35:41.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.679+0000 7f9b61b86b80 1 client.25954 shutdown 2023-07-18T03:35:41.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.679+0000 7f9b61b86b80 1 -- 172.21.15.40:0/3159695716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fa070 msgr2=0x556665706d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:41.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.679+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/3159695716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fa070 0x556665706d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b44009a60 tx=0x7f9b44009380 comp rx=0 tx=0).stop 2023-07-18T03:35:41.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.694+0000 7f9b61b86b80 1 -- 172.21.15.40:0/3159695716 shutdown_connections 2023-07-18T03:35:41.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.694+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/3159695716 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556665763d70 0x556665766180 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.694+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/3159695716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5566656fae30 0x556665709da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.694+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/3159695716 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566656f9720 0x556665706840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.694+0000 7f9b61b86b80 1 --2- 172.21.15.40:0/3159695716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5566656fa070 0x556665706d80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.694+0000 7f9b61b86b80 1 -- 172.21.15.40:0/3159695716 >> 172.21.15.40:0/3159695716 conn(0x5566656eeff0 msgr2=0x556665701140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:41.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.694+0000 7f9b337f6700 0 client.25954 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:41.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.705+0000 7f9b61b86b80 1 -- 172.21.15.40:0/3159695716 shutdown_connections 2023-07-18T03:35:41.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.705+0000 7f9b61b86b80 1 -- 172.21.15.40:0/3159695716 wait complete. 2023-07-18T03:35:41.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.705+0000 7f9b61b86b80 20 client.25954 trim_cache size 0 max 16384 2023-07-18T03:35:41.702 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:35:41.703 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-18T03:35:41.703 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:35:41.703 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:35:41.703 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:35:41.703 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:35:41.703 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:35:41.703 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:35:41.703 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:35:41.704 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:35:41.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.779+0000 7f9b4ffff700 1 -- 172.21.15.40:0/3262949410 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9b380a1430 msgr2=0x7f9b380a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:41.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.779+0000 7f9b4ffff700 1 --2- 172.21.15.40:0/3262949410 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9b380a1430 0x7f9b380a38e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9b44006f00 tx=0x7f9b44006e90 comp rx=0 tx=0).stop 2023-07-18T03:35:41.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.779+0000 7f9b4ffff700 1 -- 172.21.15.40:0/3262949410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b5010bba0 msgr2=0x7f9b501b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:41.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.779+0000 7f9b4ffff700 1 --2- 172.21.15.40:0/3262949410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b5010bba0 0x7f9b501b4720 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800ab40 tx=0x7f9b4800af00 comp rx=0 tx=0).stop 2023-07-18T03:35:41.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.779+0000 7f9b4ffff700 1 -- 172.21.15.40:0/3262949410 shutdown_connections 2023-07-18T03:35:41.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.779+0000 7f9b4ffff700 1 --2- 172.21.15.40:0/3262949410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b5010bba0 0x7f9b501b4720 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.779+0000 7f9b4ffff700 1 --2- 172.21.15.40:0/3262949410 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9b380a1430 0x7f9b380a38e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.779+0000 7f9b4ffff700 1 --2- 172.21.15.40:0/3262949410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b500711d0 0x7f9b501139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.779+0000 7f9b4ffff700 1 --2- 172.21.15.40:0/3262949410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b50071b20 0x7f9b50113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:41.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.779+0000 7f9b4ffff700 1 -- 172.21.15.40:0/3262949410 >> 172.21.15.40:0/3262949410 conn(0x7f9b5006c8b0 msgr2=0x7f9b5006db50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:41.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.780+0000 7f9b4ffff700 1 -- 172.21.15.40:0/3262949410 shutdown_connections 2023-07-18T03:35:41.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:35:41.780+0000 7f9b4ffff700 1 -- 172.21.15.40:0/3262949410 wait complete. 2023-07-18T03:35:41.845 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:35:43.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:42 smithi064 ceph-mon[98059]: pgmap v438: 97 pgs: 97 active+clean; 1022 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2023-07-18T03:35:43.174 INFO:tasks.workunit.client.0.smithi040.stdout: 291 1731942 722 1215721 677 468682 430 2023-07-18T03:35:43.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:42 smithi134 ceph-mon[98216]: pgmap v438: 97 pgs: 97 active+clean; 1022 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2023-07-18T03:35:43.347 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:42 smithi040 ceph-mon[96439]: pgmap v438: 97 pgs: 97 active+clean; 1022 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2023-07-18T03:35:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.586+0000 7fd5f336a700 1 -- 172.21.15.40:0/2997926749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5ec070f00 msgr2=0x7fd5ec071310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.586+0000 7fd5f336a700 1 --2- 172.21.15.40:0/2997926749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5ec070f00 0x7fd5ec071310 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009990 tx=0x7fd5e8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.586+0000 7fd5f336a700 1 -- 172.21.15.40:0/2997926749 shutdown_connections 2023-07-18T03:35:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.586+0000 7fd5f336a700 1 --2- 172.21.15.40:0/2997926749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5ec10bd60 0x7fd5ec10e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.586+0000 7fd5f336a700 1 --2- 172.21.15.40:0/2997926749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5ec070f00 0x7fd5ec071310 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.586+0000 7fd5f336a700 1 --2- 172.21.15.40:0/2997926749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5ec071850 0x7fd5ec10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.586+0000 7fd5f336a700 1 -- 172.21.15.40:0/2997926749 >> 172.21.15.40:0/2997926749 conn(0x7fd5ec06ca50 msgr2=0x7fd5ec06eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.587+0000 7fd5f336a700 1 -- 172.21.15.40:0/2997926749 shutdown_connections 2023-07-18T03:35:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.587+0000 7fd5f336a700 1 -- 172.21.15.40:0/2997926749 wait complete. 2023-07-18T03:35:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.587+0000 7fd5f336a700 1 Processor -- start 2023-07-18T03:35:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.588+0000 7fd5f336a700 1 -- start start 2023-07-18T03:35:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.588+0000 7fd5f336a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5ec070f00 0x7fd5ec1a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.588+0000 7fd5f336a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5ec071850 0x7fd5ec1a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.588+0000 7fd5f336a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5ec10bd60 0x7fd5ec1a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.588+0000 7fd5f336a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec1a6210 con 0x7fd5ec071850 2023-07-18T03:35:44.585 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.588+0000 7fd5f336a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec1a6380 con 0x7fd5ec10bd60 2023-07-18T03:35:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.588+0000 7fd5f336a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec1a6540 con 0x7fd5ec070f00 2023-07-18T03:35:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.588+0000 7fd5f1907700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5ec10bd60 0x7fd5ec1a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.588+0000 7fd5f1907700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5ec10bd60 0x7fd5ec1a5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43432/0 (socket says 172.21.15.40:43432) 2023-07-18T03:35:44.586 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.588+0000 7fd5f1907700 1 -- 172.21.15.40:0/642967307 learned_addr learned my addr 172.21.15.40:0/642967307 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.589+0000 7fd5f1907700 1 -- 172.21.15.40:0/642967307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5ec070f00 msgr2=0x7fd5ec1a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.589+0000 7fd5f1907700 1 --2- 172.21.15.40:0/642967307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5ec070f00 0x7fd5ec1a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.589+0000 7fd5f1907700 1 -- 172.21.15.40:0/642967307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5ec071850 msgr2=0x7fd5ec1a4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.589+0000 7fd5f1907700 1 --2- 172.21.15.40:0/642967307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5ec071850 0x7fd5ec1a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.589+0000 7fd5f1907700 1 -- 172.21.15.40:0/642967307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e80095f0 con 0x7fd5ec10bd60 2023-07-18T03:35:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.594+0000 7fd5f1907700 1 --2- 172.21.15.40:0/642967307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5ec10bd60 0x7fd5ec1a5bb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00e990 tx=0x7fd5dc00ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.665+0000 7fd5e27fc700 1 -- 172.21.15.40:0/642967307 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc004c40 con 0x7fd5ec10bd60 2023-07-18T03:35:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.665+0000 7fd5e27fc700 1 -- 172.21.15.40:0/642967307 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5dc00cdd0 con 0x7fd5ec10bd60 2023-07-18T03:35:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.665+0000 7fd5e27fc700 1 -- 172.21.15.40:0/642967307 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0058f0 con 0x7fd5ec10bd60 2023-07-18T03:35:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.675+0000 7fd5f336a700 1 -- 172.21.15.40:0/642967307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ec1a6820 con 0x7fd5ec10bd60 2023-07-18T03:35:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.675+0000 7fd5f336a700 1 -- 172.21.15.40:0/642967307 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ec10ee20 con 0x7fd5ec10bd60 2023-07-18T03:35:44.680 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.682+0000 7fd5ff4f0b80 1 Processor -- start 2023-07-18T03:35:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.705+0000 7fd5e27fc700 1 -- 172.21.15.40:0/642967307 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd5dc00c910 con 0x7fd5ec10bd60 2023-07-18T03:35:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.706+0000 7fd5e27fc700 1 --2- 172.21.15.40:0/642967307 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd5d80a1350 0x7fd5d80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.707+0000 7fd5e27fc700 1 -- 172.21.15.40:0/642967307 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd5dc0ccc10 con 0x7fd5ec10bd60 2023-07-18T03:35:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.708+0000 7fd5ff4f0b80 1 -- start start 2023-07-18T03:35:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.708+0000 7fd5ff4f0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1b676b0 0x5615b1d0b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.708+0000 7fd5ff4f0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1d0b840 0x5615b1d0c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.708+0000 7fd5ff4f0b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1d0c6e0 0x5615b1d0cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.708+0000 7fd5ff4f0b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b1c10c50 con 0x5615b1d0c6e0 2023-07-18T03:35:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.708+0000 7fd5ff4f0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b1c639e0 con 0x5615b1b676b0 2023-07-18T03:35:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.708+0000 7fd5ff4f0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b1c5ce70 con 0x5615b1d0b840 2023-07-18T03:35:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.709+0000 7fd5f0905700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1d0b840 0x5615b1d0c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.709+0000 7fd5f0905700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1d0b840 0x5615b1d0c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49142/0 (socket says 172.21.15.40:49142) 2023-07-18T03:35:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.709+0000 7fd5f0905700 1 -- 172.21.15.40:0/715942491 learned_addr learned my addr 172.21.15.40:0/715942491 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.709+0000 7fd5f1106700 1 --2- 172.21.15.40:0/715942491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1b676b0 0x5615b1d0b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.709+0000 7fd5f1106700 1 -- 172.21.15.40:0/715942491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1d0b840 msgr2=0x5615b1d0c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.709+0000 7fd5f1106700 1 --2- 172.21.15.40:0/715942491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1d0b840 0x5615b1d0c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.709+0000 7fd5f1106700 1 -- 172.21.15.40:0/715942491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1d0c6e0 msgr2=0x5615b1d0cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.709+0000 7fd5f1106700 1 --2- 172.21.15.40:0/715942491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1d0c6e0 0x5615b1d0cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.709 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.709+0000 7fd5f1106700 1 -- 172.21.15.40:0/715942491 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615b1a8ad80 con 0x5615b1b676b0 2023-07-18T03:35:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.729+0000 7fd5f0905700 1 --2- 172.21.15.40:0/642967307 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd5d80a1350 0x7fd5d80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:44.726 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.729+0000 7fd5f1106700 1 --2- 172.21.15.40:0/715942491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1b676b0 0x5615b1d0b300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009990 tx=0x7fd5e8009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3de98c36e8436fb4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.775+0000 7fd5d77fe700 1 -- 172.21.15.40:0/715942491 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e801d070 con 0x5615b1b676b0 2023-07-18T03:35:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.775+0000 7fd5d77fe700 1 -- 172.21.15.40:0/715942491 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5e800be00 con 0x5615b1b676b0 2023-07-18T03:35:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.775+0000 7fd5d77fe700 1 -- 172.21.15.40:0/715942491 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8004970 con 0x5615b1b676b0 2023-07-18T03:35:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.776+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/715942491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1b676b0 msgr2=0x5615b1d0b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.776+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/715942491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1b676b0 0x5615b1d0b300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009990 tx=0x7fd5e8009510 comp rx=0 tx=0).stop 2023-07-18T03:35:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.776+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/715942491 shutdown_connections 2023-07-18T03:35:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.776+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/715942491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1d0c6e0 0x5615b1d0cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.776+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/715942491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1b676b0 0x5615b1d0b300 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.776+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/715942491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1d0b840 0x5615b1d0c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.776+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/715942491 >> 172.21.15.40:0/715942491 conn(0x5615b1d00be0 msgr2=0x5615b1d05cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.780+0000 7fd5f0905700 1 --2- 172.21.15.40:0/642967307 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd5d80a1350 0x7fd5d80a3800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009d50 tx=0x7fd5e4006d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.787+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/715942491 shutdown_connections 2023-07-18T03:35:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.798+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/715942491 wait complete. 2023-07-18T03:35:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.798+0000 7fd5ff4f0b80 1 -- start start 2023-07-18T03:35:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.798+0000 7fd5ff4f0b80 1 Processor -- start 2023-07-18T03:35:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.799+0000 7fd5ff4f0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1b676b0 0x5615b1d14900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.799+0000 7fd5ff4f0b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1d0b840 0x5615b1d14e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.799+0000 7fd5ff4f0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1d0c6e0 0x5615b1d1be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.799+0000 7fd5ff4f0b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b1bfca60 con 0x5615b1d0b840 2023-07-18T03:35:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.799+0000 7fd5ff4f0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b1bfc760 con 0x5615b1d0c6e0 2023-07-18T03:35:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.799+0000 7fd5ff4f0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b1ae08c0 con 0x5615b1b676b0 2023-07-18T03:35:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.799+0000 7fd5ff4f0b80 10 client.0 _ll_register_callbacks cb 0x5615b1bfa700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:35:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.799+0000 7fd5f1106700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1b676b0 0x5615b1d14900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.799+0000 7fd5f1907700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1d0c6e0 0x5615b1d1be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.800+0000 7fd5f1907700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1d0c6e0 0x5615b1d1be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43446/0 (socket says 172.21.15.40:43446) 2023-07-18T03:35:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.800+0000 7fd5f1907700 1 -- 172.21.15.40:0/14377760 learned_addr learned my addr 172.21.15.40:0/14377760 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.804+0000 7fd5f1907700 1 -- 172.21.15.40:0/14377760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1b676b0 msgr2=0x5615b1d14900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.804+0000 7fd5f1907700 1 --2- 172.21.15.40:0/14377760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1b676b0 0x5615b1d14900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.804+0000 7fd5f1907700 1 -- 172.21.15.40:0/14377760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1d0b840 msgr2=0x5615b1d14e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.804+0000 7fd5f1907700 1 --2- 172.21.15.40:0/14377760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1d0b840 0x5615b1d14e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.804+0000 7fd5f1907700 1 -- 172.21.15.40:0/14377760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ec117640 con 0x5615b1d0c6e0 2023-07-18T03:35:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.805+0000 7fd5f1907700 1 --2- 172.21.15.40:0/14377760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1d0c6e0 0x5615b1d1be80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0a1f40 tx=0x7fd5dc021380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be2364d712d83d05 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.806+0000 7fd5d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:35:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.806+0000 7fd5d4ff9700 1 -- 172.21.15.40:0/14377760 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc09a020 con 0x5615b1d0c6e0 2023-07-18T03:35:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.806+0000 7fd5d4ff9700 1 -- 172.21.15.40:0/14377760 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd5dc09e050 con 0x5615b1d0c6e0 2023-07-18T03:35:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.806+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/14377760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615b1bb6490 con 0x5615b1d0c6e0 2023-07-18T03:35:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.809+0000 7fd5d4ff9700 1 -- 172.21.15.40:0/14377760 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5dc0a2470 con 0x5615b1d0c6e0 2023-07-18T03:35:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.809+0000 7fd5ff4f0b80 10 client.25946 fetch_fsmap learned FSMap version 37 2023-07-18T03:35:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.809+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/14377760 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5615b1a73c00 con 0x5615b1d0c6e0 2023-07-18T03:35:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.813+0000 7fd5d4ff9700 1 -- 172.21.15.40:0/14377760 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615b1bb6490 con 0x5615b1d0c6e0 2023-07-18T03:35:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.813+0000 7fd5d4ff9700 1 -- 172.21.15.40:0/14377760 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 37) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fd5dc0add20 con 0x5615b1d0c6e0 2023-07-18T03:35:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.820+0000 7fd5ff4f0b80 10 client.25946 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:35:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.820+0000 7fd5ff4f0b80 10 client.25946 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:35:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.820+0000 7fd5ff4f0b80 20 client.25946 populate_metadata read hostname 'smithi040' 2023-07-18T03:35:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.820+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/14377760 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5615b1d20640 0x5615b1d22a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.820+0000 7fd5ff4f0b80 4 client.25946 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:35:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.820+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/14377760 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615b1c55e50 con 0x5615b1d20640 2023-07-18T03:35:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.832+0000 7fd5f1106700 1 --2- 172.21.15.40:0/14377760 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5615b1d20640 0x5615b1d22a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:44.856 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.860+0000 7fd5f1106700 1 --2- 172.21.15.40:0/14377760 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5615b1d20640 0x5615b1d22a50 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bc6ca6da10de07a server_cookie=5ca352a15a7fecce in_seq=0 out_seq=0 2023-07-18T03:35:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.873+0000 7fd5d4ff9700 10 client.25946 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.873+0000 7fd5d4ff9700 1 -- 172.21.15.40:0/14377760 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5615b1c55e50 con 0x5615b1d20640 2023-07-18T03:35:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.873+0000 7fd5d4ff9700 10 client.25946 handle_command_reply: tid=0 2023-07-18T03:35:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.873+0000 7fd5ff4f0b80 1 client.25946 shutdown 2023-07-18T03:35:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.885+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/14377760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1d0c6e0 msgr2=0x5615b1d1be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.885+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/14377760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1d0c6e0 0x5615b1d1be80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0a1f40 tx=0x7fd5dc021380 comp rx=0 tx=0).stop 2023-07-18T03:35:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.885+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/14377760 shutdown_connections 2023-07-18T03:35:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.885+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/14377760 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5615b1d20640 0x5615b1d22a50 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.885+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/14377760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1d0c6e0 0x5615b1d1be80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.885+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/14377760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1b676b0 0x5615b1d14900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.885+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/14377760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1d0b840 0x5615b1d14e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.885+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/14377760 >> 172.21.15.40:0/14377760 conn(0x5615b1d00be0 msgr2=0x5615b1d054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.885+0000 7fd5d4ff9700 0 client.25946 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.885+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/14377760 shutdown_connections 2023-07-18T03:35:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.885+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/14377760 wait complete. 2023-07-18T03:35:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.917+0000 7fd5ff4f0b80 20 client.25946 trim_cache size 0 max 16384 2023-07-18T03:35:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.954+0000 7fd5ff4f0b80 1 Processor -- start 2023-07-18T03:35:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.954+0000 7fd5ff4f0b80 1 -- start start 2023-07-18T03:35:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.955+0000 7fd5ff4f0b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1b676b0 0x5615b1d12b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.955+0000 7fd5ff4f0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1d0b840 0x5615b1d1bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.955+0000 7fd5ff4f0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1d0c6e0 0x5615b1d135a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.955+0000 7fd5ff4f0b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b1cfde40 con 0x5615b1b676b0 2023-07-18T03:35:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.955+0000 7fd5ff4f0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b1c55e50 con 0x5615b1d0c6e0 2023-07-18T03:35:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.955+0000 7fd5ff4f0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b1bb6490 con 0x5615b1d0b840 2023-07-18T03:35:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.956+0000 7fd5f1106700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1b676b0 0x5615b1d12b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.956+0000 7fd5f1106700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1b676b0 0x5615b1d12b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60126/0 (socket says 172.21.15.40:60126) 2023-07-18T03:35:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.956+0000 7fd5f1106700 1 -- 172.21.15.40:0/2623446643 learned_addr learned my addr 172.21.15.40:0/2623446643 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.957+0000 7fd5f1106700 1 -- 172.21.15.40:0/2623446643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1d0b840 msgr2=0x5615b1d1bda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:35:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.957+0000 7fd5f1106700 1 --2- 172.21.15.40:0/2623446643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1d0b840 0x5615b1d1bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.957+0000 7fd5f1106700 1 -- 172.21.15.40:0/2623446643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1d0c6e0 msgr2=0x5615b1d135a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:35:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.957+0000 7fd5f1106700 1 --2- 172.21.15.40:0/2623446643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1d0c6e0 0x5615b1d135a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.957+0000 7fd5f1106700 1 -- 172.21.15.40:0/2623446643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615b1ab11f0 con 0x5615b1b676b0 2023-07-18T03:35:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.960+0000 7fd5f1106700 1 --2- 172.21.15.40:0/2623446643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1b676b0 0x5615b1d12b40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80095c0 tx=0x7fd5e801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce5a5351deef0eb server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.961+0000 7fd5d7fff700 1 -- 172.21.15.40:0/2623446643 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8053070 con 0x5615b1b676b0 2023-07-18T03:35:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.961+0000 7fd5d7fff700 1 -- 172.21.15.40:0/2623446643 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5e801e040 con 0x5615b1b676b0 2023-07-18T03:35:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.961+0000 7fd5d7fff700 1 -- 172.21.15.40:0/2623446643 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e801d040 con 0x5615b1b676b0 2023-07-18T03:35:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.962+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/2623446643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1b676b0 msgr2=0x5615b1d12b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.962+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/2623446643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1b676b0 0x5615b1d12b40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80095c0 tx=0x7fd5e801a040 comp rx=0 tx=0).stop 2023-07-18T03:35:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.968+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/2623446643 shutdown_connections 2023-07-18T03:35:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.968+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/2623446643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1d0c6e0 0x5615b1d135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.968+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/2623446643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1b676b0 0x5615b1d12b40 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.968+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/2623446643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1d0b840 0x5615b1d1bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.968+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/2623446643 >> 172.21.15.40:0/2623446643 conn(0x5615b1d00be0 msgr2=0x5615b1d273b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.969+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/2623446643 shutdown_connections 2023-07-18T03:35:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.969+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/2623446643 wait complete. 2023-07-18T03:35:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.969+0000 7fd5ff4f0b80 1 -- start start 2023-07-18T03:35:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.969+0000 7fd5ff4f0b80 1 Processor -- start 2023-07-18T03:35:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.970+0000 7fd5ff4f0b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1b676b0 0x5615b1d19580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.970+0000 7fd5ff4f0b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1d0b840 0x5615b1d19ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.970+0000 7fd5ff4f0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1d0c6e0 0x5615b1d1aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.970+0000 7fd5ff4f0b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b1d1b010 con 0x5615b1b676b0 2023-07-18T03:35:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.970+0000 7fd5ff4f0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b1d1b180 con 0x5615b1d0c6e0 2023-07-18T03:35:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.970+0000 7fd5ff4f0b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615b1d1b340 con 0x5615b1d0b840 2023-07-18T03:35:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.970+0000 7fd5ff4f0b80 10 client.0 _ll_register_callbacks cb 0x7fd5dc099b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:35:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.971+0000 7fd5f0905700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1d0b840 0x5615b1d19ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.971+0000 7fd5f0905700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1d0b840 0x5615b1d19ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49162/0 (socket says 172.21.15.40:49162) 2023-07-18T03:35:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.971+0000 7fd5f0905700 1 -- 172.21.15.40:0/884465575 learned_addr learned my addr 172.21.15.40:0/884465575 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.971+0000 7fd5f1907700 1 --2- 172.21.15.40:0/884465575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1d0c6e0 0x5615b1d1aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.987+0000 7fd5f1106700 1 --2- 172.21.15.40:0/884465575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1b676b0 0x5615b1d19580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:44.983 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.987+0000 7fd5f1907700 1 -- 172.21.15.40:0/884465575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1d0b840 msgr2=0x5615b1d19ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.987+0000 7fd5f1907700 1 --2- 172.21.15.40:0/884465575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1d0b840 0x5615b1d19ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.987+0000 7fd5f1907700 1 -- 172.21.15.40:0/884465575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1b676b0 msgr2=0x5615b1d19580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.987+0000 7fd5f1907700 1 --2- 172.21.15.40:0/884465575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1b676b0 0x5615b1d19580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:44.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.987+0000 7fd5f1907700 1 -- 172.21.15.40:0/884465575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615b1a8ad80 con 0x5615b1d0c6e0 2023-07-18T03:35:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.998+0000 7fd5f1907700 1 --2- 172.21.15.40:0/884465575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1d0c6e0 0x5615b1d1aad0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc096520 tx=0x7fd5dc004ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b850a32d0ad1fc42 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.999+0000 7fd5d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:35:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.999+0000 7fd5d4ff9700 1 -- 172.21.15.40:0/884465575 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc099510 con 0x5615b1d0c6e0 2023-07-18T03:35:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.999+0000 7fd5d4ff9700 1 -- 172.21.15.40:0/884465575 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fd5dc09e050 con 0x5615b1d0c6e0 2023-07-18T03:35:44.996 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:44.999+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/884465575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5e800be00 con 0x5615b1d0c6e0 2023-07-18T03:35:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:45.010+0000 7fd5d4ff9700 1 -- 172.21.15.40:0/884465575 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5dc0a2270 con 0x5615b1d0c6e0 2023-07-18T03:35:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:45.010+0000 7fd5ff4f0b80 10 client.25949 fetch_fsmap learned FSMap version 37 2023-07-18T03:35:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:45.010+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/884465575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5615b1a89250 con 0x5615b1d0c6e0 2023-07-18T03:35:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:45.019+0000 7fd5d4ff9700 1 -- 172.21.15.40:0/884465575 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5e800be00 con 0x5615b1d0c6e0 2023-07-18T03:35:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:45.019+0000 7fd5d4ff9700 1 -- 172.21.15.40:0/884465575 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 37) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fd5dc018030 con 0x5615b1d0c6e0 2023-07-18T03:35:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:45.020+0000 7fd5ff4f0b80 10 client.25949 fetch_fsmap finished waiting for FSMap version 37 2023-07-18T03:35:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:45.020+0000 7fd5ff4f0b80 10 client.25949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:35:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:45.020+0000 7fd5ff4f0b80 20 client.25949 populate_metadata read hostname 'smithi040' 2023-07-18T03:35:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:45.020+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/884465575 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5615b1d74b40 0x5615b1d76f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:45.020+0000 7fd5ff4f0b80 4 client.25949 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:35:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:45.020+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/884465575 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5615b1c17ff0 con 0x5615b1d74b40 2023-07-18T03:35:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:45.021+0000 7fd5f1106700 1 --2- 172.21.15.40:0/884465575 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5615b1d74b40 0x5615b1d76f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:45.047+0000 7fd5f1106700 1 --2- 172.21.15.40:0/884465575 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5615b1d74b40 0x5615b1d76f50 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4817b09cf6729c5 server_cookie=6ad6d32850283765 in_seq=0 out_seq=0 2023-07-18T03:35:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:45.047+0000 7fd5d4ff9700 10 client.25949 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:45.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:45 smithi134 ceph-mon[98216]: pgmap v439: 97 pgs: 97 active+clean; 1022 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 140 op/s 2023-07-18T03:35:45.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:45 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:35:45.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:45 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:35:45.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:45 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:35:45.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:45 smithi040 ceph-mon[96439]: pgmap v439: 97 pgs: 97 active+clean; 1022 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 140 op/s 2023-07-18T03:35:45.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:45 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:35:45.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:45 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:35:45.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:45 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:35:45.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:45 smithi064 ceph-mon[98059]: pgmap v439: 97 pgs: 97 active+clean; 1022 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 140 op/s 2023-07-18T03:35:45.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:45 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:35:45.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:45 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:35:45.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:45 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:35:46.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:46 smithi064 ceph-mon[98059]: pgmap v440: 97 pgs: 97 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 147 op/s 2023-07-18T03:35:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.757+0000 7fd5d4ff9700 1 -- 172.21.15.40:0/884465575 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5615b1c17ff0 con 0x5615b1d74b40 2023-07-18T03:35:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.757+0000 7fd5d4ff9700 10 client.25949 handle_command_reply: tid=0 2023-07-18T03:35:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.757+0000 7fd5ff4f0b80 1 client.25949 shutdown 2023-07-18T03:35:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.757+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/884465575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1d0c6e0 msgr2=0x5615b1d1aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.757+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/884465575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1d0c6e0 0x5615b1d1aad0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc096520 tx=0x7fd5dc004ee0 comp rx=0 tx=0).stop 2023-07-18T03:35:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.759+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/884465575 shutdown_connections 2023-07-18T03:35:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.759+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/884465575 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5615b1d74b40 0x5615b1d76f50 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.759+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/884465575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615b1d0c6e0 0x5615b1d1aad0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.759+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/884465575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5615b1b676b0 0x5615b1d19580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.759+0000 7fd5ff4f0b80 1 --2- 172.21.15.40:0/884465575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615b1d0b840 0x5615b1d19ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.759+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/884465575 >> 172.21.15.40:0/884465575 conn(0x5615b1d00be0 msgr2=0x5615b1d12b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.759+0000 7fd5d4ff9700 0 client.25949 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.761+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/884465575 shutdown_connections 2023-07-18T03:35:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.761+0000 7fd5ff4f0b80 1 -- 172.21.15.40:0/884465575 wait complete. 2023-07-18T03:35:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.761+0000 7fd5ff4f0b80 20 client.25949 trim_cache size 0 max 16384 2023-07-18T03:35:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:35:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:35:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:35:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:35:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.762+0000 7fd5f336a700 1 -- 172.21.15.40:0/642967307 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd5d80a1350 msgr2=0x7fd5d80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.762+0000 7fd5f336a700 1 --2- 172.21.15.40:0/642967307 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd5d80a1350 0x7fd5d80a3800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009d50 tx=0x7fd5e4006d00 comp rx=0 tx=0).stop 2023-07-18T03:35:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.762+0000 7fd5f336a700 1 -- 172.21.15.40:0/642967307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5ec10bd60 msgr2=0x7fd5ec1a5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.762+0000 7fd5f336a700 1 --2- 172.21.15.40:0/642967307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5ec10bd60 0x7fd5ec1a5bb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00e990 tx=0x7fd5dc00ed50 comp rx=0 tx=0).stop 2023-07-18T03:35:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.762+0000 7fd5f336a700 1 -- 172.21.15.40:0/642967307 shutdown_connections 2023-07-18T03:35:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.762+0000 7fd5f336a700 1 --2- 172.21.15.40:0/642967307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5ec10bd60 0x7fd5ec1a5bb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.762+0000 7fd5f336a700 1 --2- 172.21.15.40:0/642967307 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd5d80a1350 0x7fd5d80a3800 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.762+0000 7fd5f336a700 1 --2- 172.21.15.40:0/642967307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5ec070f00 0x7fd5ec1a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.762+0000 7fd5f336a700 1 --2- 172.21.15.40:0/642967307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5ec071850 0x7fd5ec1a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.762+0000 7fd5f336a700 1 -- 172.21.15.40:0/642967307 >> 172.21.15.40:0/642967307 conn(0x7fd5ec06ca50 msgr2=0x7fd5ec06e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.762+0000 7fd5f336a700 1 -- 172.21.15.40:0/642967307 shutdown_connections 2023-07-18T03:35:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:46.762+0000 7fd5f336a700 1 -- 172.21.15.40:0/642967307 wait complete. 2023-07-18T03:35:46.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:46 smithi134 ceph-mon[98216]: pgmap v440: 97 pgs: 97 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 147 op/s 2023-07-18T03:35:46.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:46 smithi040 ceph-mon[96439]: pgmap v440: 97 pgs: 97 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 147 op/s 2023-07-18T03:35:47.316 INFO:journalctl@ceph.mds.c.smithi134.stdout:Jul 18 03:35:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-c[124209]: ignoring --setuser ceph since I am not root 2023-07-18T03:35:47.316 INFO:journalctl@ceph.mds.c.smithi134.stdout:Jul 18 03:35:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-c[124209]: ignoring --setgroup ceph since I am not root 2023-07-18T03:35:47.317 INFO:journalctl@ceph.mds.c.smithi134.stdout:Jul 18 03:35:46 smithi134 ceph-mds[124233]: -- 172.21.15.134:0/2807126915 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e52315d1e0 con 0x55e523132c00 2023-07-18T03:35:47.317 INFO:journalctl@ceph.mds.c.smithi134.stdout:Jul 18 03:35:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-c[124209]: starting mds.c at 2023-07-18T03:35:48.066 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:47 smithi134 ceph-mon[98216]: daemon mds.c restarted 2023-07-18T03:35:48.090 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:47 smithi040 ceph-mon[96439]: daemon mds.c restarted 2023-07-18T03:35:48.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:47 smithi064 ceph-mon[98059]: daemon mds.c restarted 2023-07-18T03:35:49.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:48 smithi064 ceph-mon[98059]: osdmap e84: 12 total, 12 up, 12 in 2023-07-18T03:35:49.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:48 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.134:6836/3831344228,v1:172.21.15.134:6837/3831344228] up:boot 2023-07-18T03:35:49.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:48 smithi064 ceph-mon[98059]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-18T03:35:49.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:48 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:35:49.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:48 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:35:49.101 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:48 smithi064 ceph-mon[98059]: pgmap v442: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 11 MiB/s wr, 573 op/s 2023-07-18T03:35:49.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:48 smithi134 ceph-mon[98216]: osdmap e84: 12 total, 12 up, 12 in 2023-07-18T03:35:49.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:48 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.134:6836/3831344228,v1:172.21.15.134:6837/3831344228] up:boot 2023-07-18T03:35:49.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:48 smithi134 ceph-mon[98216]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-18T03:35:49.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:48 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:35:49.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:48 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:35:49.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:48 smithi134 ceph-mon[98216]: pgmap v442: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 11 MiB/s wr, 573 op/s 2023-07-18T03:35:49.338 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:48 smithi040 ceph-mon[96439]: osdmap e84: 12 total, 12 up, 12 in 2023-07-18T03:35:49.338 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:48 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.134:6836/3831344228,v1:172.21.15.134:6837/3831344228] up:boot 2023-07-18T03:35:49.338 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:48 smithi040 ceph-mon[96439]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-07-18T03:35:49.339 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:48 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:35:49.339 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:48 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:35:49.339 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:48 smithi040 ceph-mon[96439]: pgmap v442: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 11 MiB/s wr, 573 op/s 2023-07-18T03:35:51.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:51 smithi134 ceph-mon[98216]: pgmap v443: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 620 op/s 2023-07-18T03:35:51.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:51 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:35:51.588 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:51 smithi040 ceph-mon[96439]: pgmap v443: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 620 op/s 2023-07-18T03:35:51.588 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:51 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:35:51.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:51 smithi064 ceph-mon[98059]: pgmap v443: 97 pgs: 97 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 620 op/s 2023-07-18T03:35:51.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:51 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:35:53.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:52 smithi064 ceph-mon[98059]: pgmap v444: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 14 MiB/s wr, 782 op/s 2023-07-18T03:35:53.179 INFO:tasks.workunit.client.0.smithi040.stdout: 300 1920968 393 1351025 632 537275 917 2023-07-18T03:35:53.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:52 smithi134 ceph-mon[98216]: pgmap v444: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 14 MiB/s wr, 782 op/s 2023-07-18T03:35:53.336 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:52 smithi040 ceph-mon[96439]: pgmap v444: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 14 MiB/s wr, 782 op/s 2023-07-18T03:35:55.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:55 smithi134 ceph-mon[98216]: pgmap v445: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 16 MiB/s wr, 876 op/s 2023-07-18T03:35:55.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:55 smithi040 ceph-mon[96439]: pgmap v445: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 16 MiB/s wr, 876 op/s 2023-07-18T03:35:55.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:55 smithi064 ceph-mon[98059]: pgmap v445: 97 pgs: 97 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 16 MiB/s wr, 876 op/s 2023-07-18T03:35:56.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:56 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:35:56.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:56 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:35:56.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:56 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:35:57.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:57 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:35:57.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:57 smithi134 ceph-mon[98216]: pgmap v446: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 14 MiB/s wr, 807 op/s 2023-07-18T03:35:57.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:57 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:35:57.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:57 smithi040 ceph-mon[96439]: pgmap v446: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 14 MiB/s wr, 807 op/s 2023-07-18T03:35:57.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:57 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:35:57.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:57 smithi064 ceph-mon[98059]: pgmap v446: 97 pgs: 97 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 14 MiB/s wr, 807 op/s 2023-07-18T03:35:59.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:35:59 smithi134 ceph-mon[98216]: pgmap v447: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.3 MiB/s wr, 505 op/s 2023-07-18T03:35:59.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:35:59 smithi040 ceph-mon[96439]: pgmap v447: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.3 MiB/s wr, 505 op/s 2023-07-18T03:35:59.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:35:59 smithi064 ceph-mon[98059]: pgmap v447: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.3 MiB/s wr, 505 op/s 2023-07-18T03:35:59.892 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.896+0000 7fa05bb8e700 1 -- 172.21.15.40:0/2959584531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0540ffc90 msgr2=0x7fa054100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:59.893 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.896+0000 7fa05bb8e700 1 --2- 172.21.15.40:0/2959584531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0540ffc90 0x7fa054100100 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa044009990 tx=0x7fa044009ca0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.893 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.896+0000 7fa05bb8e700 1 -- 172.21.15.40:0/2959584531 shutdown_connections 2023-07-18T03:35:59.893 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.896+0000 7fa05bb8e700 1 --2- 172.21.15.40:0/2959584531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa054100640 0x7fa054106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.893 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.896+0000 7fa05bb8e700 1 --2- 172.21.15.40:0/2959584531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0540fea00 0x7fa0540fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.893 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.896+0000 7fa05bb8e700 1 --2- 172.21.15.40:0/2959584531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0540ffc90 0x7fa054100100 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.896+0000 7fa05bb8e700 1 -- 172.21.15.40:0/2959584531 >> 172.21.15.40:0/2959584531 conn(0x7fa0540fa000 msgr2=0x7fa0540fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.896+0000 7fa05bb8e700 1 -- 172.21.15.40:0/2959584531 shutdown_connections 2023-07-18T03:35:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.896+0000 7fa05bb8e700 1 -- 172.21.15.40:0/2959584531 wait complete. 2023-07-18T03:35:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.897+0000 7fa05bb8e700 1 Processor -- start 2023-07-18T03:35:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.897+0000 7fa05bb8e700 1 -- start start 2023-07-18T03:35:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.897+0000 7fa05bb8e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0540fea00 0x7fa05419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.897+0000 7fa05bb8e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0540ffc90 0x7fa05419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.897+0000 7fa05bb8e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa054100640 0x7fa05419f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.897+0000 7fa05bb8e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa054068850 con 0x7fa054100640 2023-07-18T03:35:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.897+0000 7fa05bb8e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05419f970 con 0x7fa0540fea00 2023-07-18T03:35:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.897+0000 7fa05bb8e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05419fae0 con 0x7fa0540ffc90 2023-07-18T03:35:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.897+0000 7fa05a12b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa054100640 0x7fa05419f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.897+0000 7fa05992a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0540fea00 0x7fa05419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.897+0000 7fa059129700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0540ffc90 0x7fa05419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.897+0000 7fa059129700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0540ffc90 0x7fa05419cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49176/0 (socket says 172.21.15.40:49176) 2023-07-18T03:35:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.897+0000 7fa059129700 1 -- 172.21.15.40:0/1285227436 learned_addr learned my addr 172.21.15.40:0/1285227436 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.897+0000 7fa05992a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0540fea00 0x7fa05419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43472/0 (socket says 172.21.15.40:43472) 2023-07-18T03:35:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.898+0000 7fa05992a700 1 -- 172.21.15.40:0/1285227436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0540ffc90 msgr2=0x7fa05419cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.898+0000 7fa05992a700 1 --2- 172.21.15.40:0/1285227436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0540ffc90 0x7fa05419cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.898+0000 7fa05992a700 1 -- 172.21.15.40:0/1285227436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa054100640 msgr2=0x7fa05419f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.898+0000 7fa05992a700 1 --2- 172.21.15.40:0/1285227436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa054100640 0x7fa05419f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.898+0000 7fa05992a700 1 -- 172.21.15.40:0/1285227436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0440095f0 con 0x7fa0540fea00 2023-07-18T03:35:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.898+0000 7fa059129700 1 --2- 172.21.15.40:0/1285227436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0540ffc90 0x7fa05419cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:35:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.898+0000 7fa05a12b700 1 --2- 172.21.15.40:0/1285227436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa054100640 0x7fa05419f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:35:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.898+0000 7fa05992a700 1 --2- 172.21.15.40:0/1285227436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0540fea00 0x7fa05419a990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa05000e9f0 tx=0x7fa05000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.898+0000 7fa04affd700 1 -- 172.21.15.40:0/1285227436 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05000c9c0 con 0x7fa0540fea00 2023-07-18T03:35:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.898+0000 7fa05bb8e700 1 -- 172.21.15.40:0/1285227436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05419fdc0 con 0x7fa0540fea00 2023-07-18T03:35:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.898+0000 7fa05bb8e700 1 -- 172.21.15.40:0/1285227436 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0541a01f0 con 0x7fa0540fea00 2023-07-18T03:35:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.898+0000 7fa04affd700 1 -- 172.21.15.40:0/1285227436 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa050010430 con 0x7fa0540fea00 2023-07-18T03:35:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.898+0000 7fa04affd700 1 -- 172.21.15.40:0/1285227436 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa050018680 con 0x7fa0540fea00 2023-07-18T03:35:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.901+0000 7fa04affd700 1 -- 172.21.15.40:0/1285227436 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa0500187e0 con 0x7fa0540fea00 2023-07-18T03:35:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.901+0000 7fa04affd700 1 --2- 172.21.15.40:0/1285227436 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0400a1150 0x7fa0400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.901+0000 7fa059129700 1 --2- 172.21.15.40:0/1285227436 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0400a1150 0x7fa0400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.901+0000 7fa04affd700 1 -- 172.21.15.40:0/1285227436 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa050014070 con 0x7fa0540fea00 2023-07-18T03:35:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.902+0000 7fa067d14b80 1 Processor -- start 2023-07-18T03:35:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.902+0000 7fa067d14b80 1 -- start start 2023-07-18T03:35:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.902+0000 7fa059129700 1 --2- 172.21.15.40:0/1285227436 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0400a1150 0x7fa0400a3600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa054067db0 tx=0x7fa044005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.902+0000 7fa067d14b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d14ee6b0 0x55c0d1692300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.902+0000 7fa067d14b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d1692840 0x55c0d16931a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.902+0000 7fa067d14b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d16936e0 0x55c0d1693b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.902+0000 7fa067d14b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d1597c50 con 0x55c0d14ee6b0 2023-07-18T03:35:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.902+0000 7fa067d14b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d15ea9e0 con 0x55c0d1692840 2023-07-18T03:35:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.902+0000 7fa067d14b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d15e3e70 con 0x55c0d16936e0 2023-07-18T03:35:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.902+0000 7fa05992a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d14ee6b0 0x55c0d1692300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.902+0000 7fa05992a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d14ee6b0 0x55c0d1692300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60154/0 (socket says 172.21.15.40:60154) 2023-07-18T03:35:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.902+0000 7fa05992a700 1 -- 172.21.15.40:0/4278081364 learned_addr learned my addr 172.21.15.40:0/4278081364 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.902+0000 7fa059129700 1 --2- 172.21.15.40:0/4278081364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d1692840 0x55c0d16931a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.902+0000 7fa05a12b700 1 --2- 172.21.15.40:0/4278081364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d16936e0 0x55c0d1693b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.903+0000 7fa05992a700 1 -- 172.21.15.40:0/4278081364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d16936e0 msgr2=0x55c0d1693b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.903+0000 7fa05992a700 1 --2- 172.21.15.40:0/4278081364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d16936e0 0x55c0d1693b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.903+0000 7fa05992a700 1 -- 172.21.15.40:0/4278081364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d1692840 msgr2=0x55c0d16931a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.903+0000 7fa05992a700 1 --2- 172.21.15.40:0/4278081364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d1692840 0x55c0d16931a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.903+0000 7fa05992a700 1 -- 172.21.15.40:0/4278081364 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0d1411d80 con 0x55c0d14ee6b0 2023-07-18T03:35:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.903+0000 7fa05992a700 1 --2- 172.21.15.40:0/4278081364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d14ee6b0 0x55c0d1692300 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa05009ad30 tx=0x7fa05009a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c2312b754cc52d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.903+0000 7fa03ffff700 1 -- 172.21.15.40:0/4278081364 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05009ae00 con 0x55c0d14ee6b0 2023-07-18T03:35:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.903+0000 7fa03ffff700 1 -- 172.21.15.40:0/4278081364 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa05009edd0 con 0x55c0d14ee6b0 2023-07-18T03:35:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.903+0000 7fa03ffff700 1 -- 172.21.15.40:0/4278081364 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0500a2fc0 con 0x55c0d14ee6b0 2023-07-18T03:35:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.904+0000 7fa067d14b80 1 -- 172.21.15.40:0/4278081364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d14ee6b0 msgr2=0x55c0d1692300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.904+0000 7fa067d14b80 1 --2- 172.21.15.40:0/4278081364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d14ee6b0 0x55c0d1692300 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa05009ad30 tx=0x7fa05009a700 comp rx=0 tx=0).stop 2023-07-18T03:35:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.904+0000 7fa067d14b80 1 -- 172.21.15.40:0/4278081364 shutdown_connections 2023-07-18T03:35:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.904+0000 7fa067d14b80 1 --2- 172.21.15.40:0/4278081364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d16936e0 0x55c0d1693b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.904+0000 7fa067d14b80 1 --2- 172.21.15.40:0/4278081364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d14ee6b0 0x55c0d1692300 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.904+0000 7fa067d14b80 1 --2- 172.21.15.40:0/4278081364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d1692840 0x55c0d16931a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.904+0000 7fa067d14b80 1 -- 172.21.15.40:0/4278081364 >> 172.21.15.40:0/4278081364 conn(0x55c0d1687be0 msgr2=0x55c0d168ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.904+0000 7fa067d14b80 1 -- 172.21.15.40:0/4278081364 shutdown_connections 2023-07-18T03:35:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.904+0000 7fa067d14b80 1 -- 172.21.15.40:0/4278081364 wait complete. 2023-07-18T03:35:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.904+0000 7fa067d14b80 1 -- start start 2023-07-18T03:35:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.904+0000 7fa067d14b80 1 Processor -- start 2023-07-18T03:35:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.905+0000 7fa067d14b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d14ee6b0 0x55c0d169b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.905+0000 7fa067d14b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d1692840 0x55c0d169be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.905+0000 7fa067d14b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d16936e0 0x55c0d16a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.905+0000 7fa067d14b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d1583a60 con 0x55c0d1692840 2023-07-18T03:35:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.905+0000 7fa067d14b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d1583760 con 0x55c0d16936e0 2023-07-18T03:35:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.905+0000 7fa067d14b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d14678c0 con 0x55c0d14ee6b0 2023-07-18T03:35:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.905+0000 7fa067d14b80 10 client.0 _ll_register_callbacks cb 0x55c0d1581700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:35:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.905+0000 7fa059129700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d1692840 0x55c0d169be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.905+0000 7fa05992a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d14ee6b0 0x55c0d169b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.905+0000 7fa059129700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d1692840 0x55c0d169be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60162/0 (socket says 172.21.15.40:60162) 2023-07-18T03:35:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.905+0000 7fa059129700 1 -- 172.21.15.40:0/1637652722 learned_addr learned my addr 172.21.15.40:0/1637652722 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.905+0000 7fa05a12b700 1 --2- 172.21.15.40:0/1637652722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d16936e0 0x55c0d16a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.905+0000 7fa059129700 1 -- 172.21.15.40:0/1637652722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d14ee6b0 msgr2=0x55c0d169b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.905+0000 7fa059129700 1 --2- 172.21.15.40:0/1637652722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d14ee6b0 0x55c0d169b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.905+0000 7fa059129700 1 -- 172.21.15.40:0/1637652722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d16936e0 msgr2=0x55c0d16a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.905+0000 7fa059129700 1 --2- 172.21.15.40:0/1637652722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d16936e0 0x55c0d16a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.905+0000 7fa059129700 1 -- 172.21.15.40:0/1637652722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0541a01f0 con 0x55c0d1692840 2023-07-18T03:35:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.906+0000 7fa059129700 1 --2- 172.21.15.40:0/1637652722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d1692840 0x55c0d169be20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa044005910 tx=0x7fa04401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e58f85b0cd066cc server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.906+0000 7fa03d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:35:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.906+0000 7fa03d7fa700 1 -- 172.21.15.40:0/1637652722 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044034070 con 0x55c0d1692840 2023-07-18T03:35:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.906+0000 7fa03d7fa700 1 -- 172.21.15.40:0/1637652722 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa044026af0 con 0x55c0d1692840 2023-07-18T03:35:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.906+0000 7fa067d14b80 1 -- 172.21.15.40:0/1637652722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0d153d490 con 0x55c0d1692840 2023-07-18T03:35:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.906+0000 7fa03d7fa700 1 -- 172.21.15.40:0/1637652722 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa04402b050 con 0x55c0d1692840 2023-07-18T03:35:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.906+0000 7fa067d14b80 10 client.16323 fetch_fsmap learned FSMap version 41 2023-07-18T03:35:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.906+0000 7fa067d14b80 1 -- 172.21.15.40:0/1637652722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c0d13fac00 con 0x55c0d1692840 2023-07-18T03:35:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.906+0000 7fa03d7fa700 1 -- 172.21.15.40:0/1637652722 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c0d153d490 con 0x55c0d1692840 2023-07-18T03:35:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.906+0000 7fa03d7fa700 1 -- 172.21.15.40:0/1637652722 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 41) v1 ==== 7518+0+0 (secure 0 0 0) 0x7fa044037030 con 0x55c0d1692840 2023-07-18T03:35:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.906+0000 7fa067d14b80 10 client.16323 fetch_fsmap finished waiting for FSMap version 41 2023-07-18T03:35:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.906+0000 7fa067d14b80 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:35:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.906+0000 7fa067d14b80 20 client.16323 populate_metadata read hostname 'smithi040' 2023-07-18T03:35:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.906+0000 7fa067d14b80 1 --2- 172.21.15.40:0/1637652722 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c0d16a7840 0x55c0d16a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.906+0000 7fa067d14b80 4 client.16323 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:35:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.906+0000 7fa067d14b80 1 -- 172.21.15.40:0/1637652722 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0d15dce50 con 0x55c0d16a7840 2023-07-18T03:35:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.907+0000 7fa05992a700 1 --2- 172.21.15.40:0/1637652722 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c0d16a7840 0x55c0d16a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.907+0000 7fa03d7fa700 10 client.16323 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.907+0000 7fa05992a700 1 --2- 172.21.15.40:0/1637652722 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c0d16a7840 0x55c0d16a9c50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd7e4633646f74f3 server_cookie=723ef45d3d4038f6 in_seq=0 out_seq=0 2023-07-18T03:35:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.908+0000 7fa03d7fa700 1 -- 172.21.15.40:0/1637652722 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c0d15dce50 con 0x55c0d16a7840 2023-07-18T03:35:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.908+0000 7fa03d7fa700 10 client.16323 handle_command_reply: tid=0 2023-07-18T03:35:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.908+0000 7fa067d14b80 1 client.16323 shutdown 2023-07-18T03:35:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.908+0000 7fa067d14b80 1 -- 172.21.15.40:0/1637652722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d1692840 msgr2=0x55c0d169be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.908+0000 7fa067d14b80 1 --2- 172.21.15.40:0/1637652722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d1692840 0x55c0d169be20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa044005910 tx=0x7fa04401a050 comp rx=0 tx=0).stop 2023-07-18T03:35:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.908+0000 7fa067d14b80 1 -- 172.21.15.40:0/1637652722 shutdown_connections 2023-07-18T03:35:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.908+0000 7fa067d14b80 1 --2- 172.21.15.40:0/1637652722 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c0d16a7840 0x55c0d16a9c50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.908+0000 7fa067d14b80 1 --2- 172.21.15.40:0/1637652722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d16936e0 0x55c0d16a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.908+0000 7fa067d14b80 1 --2- 172.21.15.40:0/1637652722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d14ee6b0 0x55c0d169b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.908+0000 7fa067d14b80 1 --2- 172.21.15.40:0/1637652722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d1692840 0x55c0d169be20 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.908+0000 7fa03d7fa700 0 client.16323 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.908+0000 7fa067d14b80 1 -- 172.21.15.40:0/1637652722 >> 172.21.15.40:0/1637652722 conn(0x55c0d1687be0 msgr2=0x55c0d168c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.908+0000 7fa067d14b80 1 -- 172.21.15.40:0/1637652722 shutdown_connections 2023-07-18T03:35:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.908+0000 7fa067d14b80 1 -- 172.21.15.40:0/1637652722 wait complete. 2023-07-18T03:35:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.908+0000 7fa067d14b80 20 client.16323 trim_cache size 0 max 16384 2023-07-18T03:35:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.923+0000 7fa067d14b80 1 Processor -- start 2023-07-18T03:35:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.923+0000 7fa067d14b80 1 -- start start 2023-07-18T03:35:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.923+0000 7fa067d14b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d14ee6b0 0x55c0d169b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.923+0000 7fa067d14b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d1692840 0x55c0d169be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.923+0000 7fa067d14b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d16936e0 0x55c0d16a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.923+0000 7fa067d14b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d169c7e0 con 0x55c0d1692840 2023-07-18T03:35:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.923+0000 7fa067d14b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d169b450 con 0x55c0d14ee6b0 2023-07-18T03:35:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.924+0000 7fa067d14b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d1692f30 con 0x55c0d16936e0 2023-07-18T03:35:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.924+0000 7fa059129700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d1692840 0x55c0d169be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.924+0000 7fa059129700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d1692840 0x55c0d169be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60170/0 (socket says 172.21.15.40:60170) 2023-07-18T03:35:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.924+0000 7fa059129700 1 -- 172.21.15.40:0/2971187967 learned_addr learned my addr 172.21.15.40:0/2971187967 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.924+0000 7fa05992a700 1 --2- 172.21.15.40:0/2971187967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d14ee6b0 0x55c0d169b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.924+0000 7fa05a12b700 1 --2- 172.21.15.40:0/2971187967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d16936e0 0x55c0d16a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.924+0000 7fa059129700 1 -- 172.21.15.40:0/2971187967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d16936e0 msgr2=0x55c0d16a2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.924+0000 7fa059129700 1 --2- 172.21.15.40:0/2971187967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d16936e0 0x55c0d16a2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.924+0000 7fa059129700 1 -- 172.21.15.40:0/2971187967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d14ee6b0 msgr2=0x55c0d169b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.924+0000 7fa059129700 1 --2- 172.21.15.40:0/2971187967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d14ee6b0 0x55c0d169b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.924+0000 7fa059129700 1 -- 172.21.15.40:0/2971187967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0d14381f0 con 0x55c0d1692840 2023-07-18T03:35:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.924+0000 7fa05a12b700 1 --2- 172.21.15.40:0/2971187967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d16936e0 0x55c0d16a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:35:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.924+0000 7fa05992a700 1 --2- 172.21.15.40:0/2971187967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d14ee6b0 0x55c0d169b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:35:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.924+0000 7fa059129700 1 --2- 172.21.15.40:0/2971187967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d1692840 0x55c0d169be20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa0440092c0 tx=0x7fa0440092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee04fe3d5b9774a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.924+0000 7fa048dac700 1 -- 172.21.15.40:0/2971187967 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044034070 con 0x55c0d1692840 2023-07-18T03:35:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.924+0000 7fa048dac700 1 -- 172.21.15.40:0/2971187967 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa04402c040 con 0x55c0d1692840 2023-07-18T03:35:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.924+0000 7fa048dac700 1 -- 172.21.15.40:0/2971187967 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04402b070 con 0x55c0d1692840 2023-07-18T03:35:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.925+0000 7fa067d14b80 1 -- 172.21.15.40:0/2971187967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d1692840 msgr2=0x55c0d169be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:35:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.925+0000 7fa067d14b80 1 --2- 172.21.15.40:0/2971187967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d1692840 0x55c0d169be20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa0440092c0 tx=0x7fa0440092f0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.925+0000 7fa067d14b80 1 -- 172.21.15.40:0/2971187967 shutdown_connections 2023-07-18T03:35:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.925+0000 7fa067d14b80 1 --2- 172.21.15.40:0/2971187967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d16936e0 0x55c0d16a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.925+0000 7fa067d14b80 1 --2- 172.21.15.40:0/2971187967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d14ee6b0 0x55c0d169b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.925+0000 7fa067d14b80 1 --2- 172.21.15.40:0/2971187967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d1692840 0x55c0d169be20 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.925+0000 7fa067d14b80 1 -- 172.21.15.40:0/2971187967 >> 172.21.15.40:0/2971187967 conn(0x55c0d1687be0 msgr2=0x55c0d1699ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:35:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.925+0000 7fa067d14b80 1 -- 172.21.15.40:0/2971187967 shutdown_connections 2023-07-18T03:35:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.925+0000 7fa067d14b80 1 -- 172.21.15.40:0/2971187967 wait complete. 2023-07-18T03:35:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.925+0000 7fa067d14b80 1 -- start start 2023-07-18T03:35:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.925+0000 7fa067d14b80 1 Processor -- start 2023-07-18T03:35:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.926+0000 7fa067d14b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d14ee6b0 0x55c0d16a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.926+0000 7fa067d14b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d1692840 0x55c0d16f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.926+0000 7fa067d14b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d16936e0 0x55c0d16f9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.926+0000 7fa067d14b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d16fa340 con 0x55c0d16936e0 2023-07-18T03:35:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.926+0000 7fa067d14b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d16fa4b0 con 0x55c0d14ee6b0 2023-07-18T03:35:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.926+0000 7fa067d14b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0d16fa670 con 0x55c0d1692840 2023-07-18T03:35:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.926+0000 7fa067d14b80 10 client.0 _ll_register_callbacks cb 0x55c0d1581700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:35:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.926+0000 7fa05a12b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d16936e0 0x55c0d16f9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.926+0000 7fa05992a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d14ee6b0 0x55c0d16a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.926+0000 7fa059129700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d1692840 0x55c0d16f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.926+0000 7fa05992a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d14ee6b0 0x55c0d16a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43500/0 (socket says 172.21.15.40:43500) 2023-07-18T03:35:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.926+0000 7fa05992a700 1 -- 172.21.15.40:0/1804368993 learned_addr learned my addr 172.21.15.40:0/1804368993 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:35:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.926+0000 7fa05a12b700 1 -- 172.21.15.40:0/1804368993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d1692840 msgr2=0x55c0d16f76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.926+0000 7fa05a12b700 1 --2- 172.21.15.40:0/1804368993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d1692840 0x55c0d16f76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.926+0000 7fa05a12b700 1 -- 172.21.15.40:0/1804368993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d14ee6b0 msgr2=0x55c0d16a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:35:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.926+0000 7fa05a12b700 1 --2- 172.21.15.40:0/1804368993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d14ee6b0 0x55c0d16a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:35:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.926+0000 7fa05a12b700 1 -- 172.21.15.40:0/1804368993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0d1411d80 con 0x55c0d16936e0 2023-07-18T03:35:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.927+0000 7fa05992a700 1 --2- 172.21.15.40:0/1804368993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d14ee6b0 0x55c0d16a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:35:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.927+0000 7fa05a12b700 1 --2- 172.21.15.40:0/1804368993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d16936e0 0x55c0d16f9c20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x55c0d16a3440 tx=0x7fa04c018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a032a18e375b508 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:35:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.927+0000 7fa03d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:35:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.927+0000 7fa03d7fa700 1 -- 172.21.15.40:0/1804368993 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c021440 con 0x55c0d16936e0 2023-07-18T03:35:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.927+0000 7fa03d7fa700 1 -- 172.21.15.40:0/1804368993 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa04c016430 con 0x55c0d16936e0 2023-07-18T03:35:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.927+0000 7fa067d14b80 1 -- 172.21.15.40:0/1804368993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa05009edd0 con 0x55c0d16936e0 2023-07-18T03:35:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.927+0000 7fa03d7fa700 1 -- 172.21.15.40:0/1804368993 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa04c0165f0 con 0x55c0d16936e0 2023-07-18T03:35:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.927+0000 7fa067d14b80 10 client.16329 fetch_fsmap learned FSMap version 41 2023-07-18T03:35:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.927+0000 7fa067d14b80 1 -- 172.21.15.40:0/1804368993 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c0d13fac00 con 0x55c0d16936e0 2023-07-18T03:35:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.927+0000 7fa03d7fa700 1 -- 172.21.15.40:0/1804368993 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa05009edd0 con 0x55c0d16936e0 2023-07-18T03:35:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.927+0000 7fa03d7fa700 1 -- 172.21.15.40:0/1804368993 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 41) v1 ==== 7518+0+0 (secure 0 0 0) 0x7fa04c029020 con 0x55c0d16936e0 2023-07-18T03:35:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.927+0000 7fa067d14b80 10 client.16329 fetch_fsmap finished waiting for FSMap version 41 2023-07-18T03:35:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.927+0000 7fa067d14b80 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:35:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.927+0000 7fa067d14b80 20 client.16329 populate_metadata read hostname 'smithi040' 2023-07-18T03:35:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.927+0000 7fa067d14b80 1 --2- 172.21.15.40:0/1804368993 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c0d16fc120 0x55c0d16fe530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:35:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.927+0000 7fa067d14b80 4 client.16329 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:35:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.927+0000 7fa067d14b80 1 -- 172.21.15.40:0/1804368993 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c0d159eff0 con 0x55c0d16fc120 2023-07-18T03:35:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.928+0000 7fa05992a700 1 --2- 172.21.15.40:0/1804368993 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c0d16fc120 0x55c0d16fe530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:35:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.928+0000 7fa03d7fa700 10 client.16329 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:35:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:35:59.928+0000 7fa05992a700 1 --2- 172.21.15.40:0/1804368993 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c0d16fc120 0x55c0d16fe530 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d357ebdf1520b4bc server_cookie=ea80cce9321a3907 in_seq=0 out_seq=0 2023-07-18T03:36:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.154+0000 7fa03d7fa700 1 -- 172.21.15.40:0/1804368993 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c0d159eff0 con 0x55c0d16fc120 2023-07-18T03:36:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.154+0000 7fa03d7fa700 10 client.16329 handle_command_reply: tid=0 2023-07-18T03:36:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.154+0000 7fa067d14b80 1 client.16329 shutdown 2023-07-18T03:36:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.154+0000 7fa067d14b80 1 -- 172.21.15.40:0/1804368993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d16936e0 msgr2=0x55c0d16f9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.154+0000 7fa067d14b80 1 --2- 172.21.15.40:0/1804368993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d16936e0 0x55c0d16f9c20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x55c0d16a3440 tx=0x7fa04c018420 comp rx=0 tx=0).stop 2023-07-18T03:36:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.154+0000 7fa067d14b80 1 -- 172.21.15.40:0/1804368993 shutdown_connections 2023-07-18T03:36:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.154+0000 7fa067d14b80 1 --2- 172.21.15.40:0/1804368993 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c0d16fc120 0x55c0d16fe530 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.154+0000 7fa067d14b80 1 --2- 172.21.15.40:0/1804368993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c0d16936e0 0x55c0d16f9c20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.154+0000 7fa067d14b80 1 --2- 172.21.15.40:0/1804368993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d14ee6b0 0x55c0d16a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.154+0000 7fa067d14b80 1 --2- 172.21.15.40:0/1804368993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c0d1692840 0x55c0d16f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.154+0000 7fa067d14b80 1 -- 172.21.15.40:0/1804368993 >> 172.21.15.40:0/1804368993 conn(0x55c0d1687be0 msgr2=0x55c0d1696630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.154+0000 7fa03d7fa700 0 client.16329 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.154+0000 7fa067d14b80 1 -- 172.21.15.40:0/1804368993 shutdown_connections 2023-07-18T03:36:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.154+0000 7fa067d14b80 1 -- 172.21.15.40:0/1804368993 wait complete. 2023-07-18T03:36:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.154+0000 7fa067d14b80 20 client.16329 trim_cache size 0 max 16384 2023-07-18T03:36:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:36:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:36:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:36:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:36:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.155+0000 7fa05bb8e700 1 -- 172.21.15.40:0/1285227436 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0400a1150 msgr2=0x7fa0400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.155+0000 7fa05bb8e700 1 --2- 172.21.15.40:0/1285227436 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0400a1150 0x7fa0400a3600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa054067db0 tx=0x7fa044005960 comp rx=0 tx=0).stop 2023-07-18T03:36:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.155+0000 7fa05bb8e700 1 -- 172.21.15.40:0/1285227436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0540fea00 msgr2=0x7fa05419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.155+0000 7fa05bb8e700 1 --2- 172.21.15.40:0/1285227436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0540fea00 0x7fa05419a990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa05000e9f0 tx=0x7fa05000ed00 comp rx=0 tx=0).stop 2023-07-18T03:36:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.155+0000 7fa05bb8e700 1 -- 172.21.15.40:0/1285227436 shutdown_connections 2023-07-18T03:36:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.155+0000 7fa05bb8e700 1 --2- 172.21.15.40:0/1285227436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa054100640 0x7fa05419f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.155+0000 7fa05bb8e700 1 --2- 172.21.15.40:0/1285227436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0540fea00 0x7fa05419a990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.155+0000 7fa05bb8e700 1 --2- 172.21.15.40:0/1285227436 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa0400a1150 0x7fa0400a3600 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.155+0000 7fa05bb8e700 1 --2- 172.21.15.40:0/1285227436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0540ffc90 0x7fa05419cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.155+0000 7fa05bb8e700 1 -- 172.21.15.40:0/1285227436 >> 172.21.15.40:0/1285227436 conn(0x7fa0540fa000 msgr2=0x7fa0540fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:01.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.156+0000 7fa05bb8e700 1 -- 172.21.15.40:0/1285227436 shutdown_connections 2023-07-18T03:36:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:01.156+0000 7fa05bb8e700 1 -- 172.21.15.40:0/1285227436 wait complete. 2023-07-18T03:36:01.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:01 smithi134 ceph-mon[98216]: pgmap v448: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 379 op/s 2023-07-18T03:36:01.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:01 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:36:01.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:01 smithi040 ceph-mon[96439]: pgmap v448: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 379 op/s 2023-07-18T03:36:01.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:01 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:36:01.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:01 smithi064 ceph-mon[98059]: pgmap v448: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 379 op/s 2023-07-18T03:36:01.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:01 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:36:03.100 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:02 smithi064 ceph-mon[98059]: pgmap v449: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.8 MiB/s wr, 668 op/s 2023-07-18T03:36:03.182 INFO:tasks.workunit.client.0.smithi040.stdout: 316 1334626 999 941480 859 382413 1100 2023-07-18T03:36:03.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:02 smithi134 ceph-mon[98216]: pgmap v449: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.8 MiB/s wr, 668 op/s 2023-07-18T03:36:03.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:02 smithi040 ceph-mon[96439]: pgmap v449: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.8 MiB/s wr, 668 op/s 2023-07-18T03:36:05.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:05 smithi134 ceph-mon[98216]: pgmap v450: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 664 op/s 2023-07-18T03:36:05.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:05 smithi040 ceph-mon[96439]: pgmap v450: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 664 op/s 2023-07-18T03:36:05.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:05 smithi064 ceph-mon[98059]: pgmap v450: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 664 op/s 2023-07-18T03:36:07.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:07 smithi134 ceph-mon[98216]: pgmap v451: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 690 op/s 2023-07-18T03:36:07.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:07 smithi040 ceph-mon[96439]: pgmap v451: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 690 op/s 2023-07-18T03:36:07.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:07 smithi064 ceph-mon[98059]: pgmap v451: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 690 op/s 2023-07-18T03:36:09.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:09 smithi134 ceph-mon[98216]: pgmap v452: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 725 op/s 2023-07-18T03:36:09.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:09 smithi040 ceph-mon[96439]: pgmap v452: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 725 op/s 2023-07-18T03:36:09.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:09 smithi064 ceph-mon[98059]: pgmap v452: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 725 op/s 2023-07-18T03:36:11.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:11 smithi134 ceph-mon[98216]: pgmap v453: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 672 op/s 2023-07-18T03:36:11.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:11 smithi040 ceph-mon[96439]: pgmap v453: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 672 op/s 2023-07-18T03:36:11.846 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:36:11.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:11 smithi064 ceph-mon[98059]: pgmap v453: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 672 op/s 2023-07-18T03:36:13.191 INFO:tasks.workunit.client.0.smithi040.stdout: 324 2410259 539 1700756 451 701664 952 2023-07-18T03:36:13.320 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:12 smithi134 ceph-mon[98216]: pgmap v454: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 684 op/s 2023-07-18T03:36:13.338 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:12 smithi040 ceph-mon[96439]: pgmap v454: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 684 op/s 2023-07-18T03:36:13.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:12 smithi064 ceph-mon[98059]: pgmap v454: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 684 op/s 2023-07-18T03:36:15.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:15 smithi134 ceph-mon[98216]: pgmap v455: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 361 op/s 2023-07-18T03:36:15.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:15 smithi040 ceph-mon[96439]: pgmap v455: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 361 op/s 2023-07-18T03:36:15.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:15 smithi064 ceph-mon[98059]: pgmap v455: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 361 op/s 2023-07-18T03:36:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.126+0000 7fab60499700 1 -- 172.21.15.40:0/2229733031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab58107c80 msgr2=0x7fab58117230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.126+0000 7fab60499700 1 --2- 172.21.15.40:0/2229733031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab58107c80 0x7fab58117230 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fab5000b0a0 tx=0x7fab5000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.126+0000 7fab60499700 1 -- 172.21.15.40:0/2229733031 shutdown_connections 2023-07-18T03:36:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.126+0000 7fab60499700 1 --2- 172.21.15.40:0/2229733031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab58107c80 0x7fab58117230 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.126+0000 7fab60499700 1 --2- 172.21.15.40:0/2229733031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab580713c0 0x7fab580717d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.126+0000 7fab60499700 1 --2- 172.21.15.40:0/2229733031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab581072f0 0x7fab58107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.126 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.126+0000 7fab60499700 1 -- 172.21.15.40:0/2229733031 >> 172.21.15.40:0/2229733031 conn(0x7fab5806ca40 msgr2=0x7fab5806ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.129+0000 7fab60499700 1 -- 172.21.15.40:0/2229733031 shutdown_connections 2023-07-18T03:36:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.129+0000 7fab60499700 1 -- 172.21.15.40:0/2229733031 wait complete. 2023-07-18T03:36:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.130+0000 7fab60499700 1 Processor -- start 2023-07-18T03:36:16.127 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.130+0000 7fab60499700 1 -- start start 2023-07-18T03:36:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.130+0000 7fab60499700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab580713c0 0x7fab581a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.130+0000 7fab60499700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab581072f0 0x7fab581a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.130+0000 7fab60499700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab58107c80 0x7fab581abba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.130+0000 7fab60499700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab581ac170 con 0x7fab581072f0 2023-07-18T03:36:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.130+0000 7fab60499700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5810ea20 con 0x7fab58107c80 2023-07-18T03:36:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.130+0000 7fab60499700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5810ebb0 con 0x7fab580713c0 2023-07-18T03:36:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.136+0000 7fab5e235700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab580713c0 0x7fab581a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.136+0000 7fab5e235700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab580713c0 0x7fab581a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49224/0 (socket says 172.21.15.40:49224) 2023-07-18T03:36:16.133 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.136+0000 7fab5e235700 1 -- 172.21.15.40:0/2213727181 learned_addr learned my addr 172.21.15.40:0/2213727181 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.141+0000 7fab5ea36700 1 --2- 172.21.15.40:0/2213727181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab58107c80 0x7fab581abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:16.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.144+0000 7fab5e235700 1 -- 172.21.15.40:0/2213727181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab58107c80 msgr2=0x7fab581abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.144+0000 7fab5e235700 1 --2- 172.21.15.40:0/2213727181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab58107c80 0x7fab581abba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.144+0000 7fab5e235700 1 -- 172.21.15.40:0/2213727181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab581072f0 msgr2=0x7fab581a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.144+0000 7fab5e235700 1 --2- 172.21.15.40:0/2213727181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab581072f0 0x7fab581a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.144+0000 7fab5e235700 1 -- 172.21.15.40:0/2213727181 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab50009d00 con 0x7fab580713c0 2023-07-18T03:36:16.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.144+0000 7fab5e235700 1 --2- 172.21.15.40:0/2213727181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab580713c0 0x7fab581a45f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fab5400e970 tx=0x7fab5400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.202+0000 7fab4f7fe700 1 -- 172.21.15.40:0/2213727181 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5400cbd0 con 0x7fab580713c0 2023-07-18T03:36:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.202+0000 7fab4f7fe700 1 -- 172.21.15.40:0/2213727181 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab5400eec0 con 0x7fab580713c0 2023-07-18T03:36:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.202+0000 7fab4f7fe700 1 -- 172.21.15.40:0/2213727181 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54003d10 con 0x7fab580713c0 2023-07-18T03:36:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.202+0000 7fab60499700 1 -- 172.21.15.40:0/2213727181 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5810ee90 con 0x7fab580713c0 2023-07-18T03:36:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.202+0000 7fab60499700 1 -- 172.21.15.40:0/2213727181 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5810f350 con 0x7fab580713c0 2023-07-18T03:36:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.210+0000 7fab6c61fb80 1 Processor -- start 2023-07-18T03:36:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.213+0000 7fab6c61fb80 1 -- start start 2023-07-18T03:36:16.211 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.214+0000 7fab6c61fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cbdc6b0 0x56080cd80300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.214+0000 7fab6c61fb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd80840 0x56080cd811a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.214+0000 7fab6c61fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cd816e0 0x56080cd81b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.214+0000 7fab6c61fb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080cc85c50 con 0x56080cd80840 2023-07-18T03:36:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.214+0000 7fab6c61fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080ccd89e0 con 0x56080cd816e0 2023-07-18T03:36:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.214+0000 7fab6c61fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080ccd1e70 con 0x56080cbdc6b0 2023-07-18T03:36:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.214+0000 7fab5da34700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cd816e0 0x56080cd81b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.214+0000 7fab5ea36700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cbdc6b0 0x56080cd80300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.214+0000 7fab5ea36700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cbdc6b0 0x56080cd80300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49228/0 (socket says 172.21.15.40:49228) 2023-07-18T03:36:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.214+0000 7fab5ea36700 1 -- 172.21.15.40:0/2985962780 learned_addr learned my addr 172.21.15.40:0/2985962780 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.214+0000 7fab5ea36700 1 -- 172.21.15.40:0/2985962780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cd816e0 msgr2=0x56080cd81b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.214+0000 7fab5ea36700 1 --2- 172.21.15.40:0/2985962780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cd816e0 0x56080cd81b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.214+0000 7fab5ea36700 1 -- 172.21.15.40:0/2985962780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd80840 msgr2=0x56080cd811a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:36:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.214+0000 7fab5ea36700 1 --2- 172.21.15.40:0/2985962780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd80840 0x56080cd811a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.214+0000 7fab5ea36700 1 -- 172.21.15.40:0/2985962780 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56080caffd80 con 0x56080cbdc6b0 2023-07-18T03:36:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.215+0000 7fab4f7fe700 1 -- 172.21.15.40:0/2213727181 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fab5400cd30 con 0x7fab580713c0 2023-07-18T03:36:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.216+0000 7fab4f7fe700 1 --2- 172.21.15.40:0/2213727181 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fab440a1350 0x7fab440a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:16.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.217+0000 7fab4f7fe700 1 -- 172.21.15.40:0/2213727181 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fab54014070 con 0x7fab580713c0 2023-07-18T03:36:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.220+0000 7fab5e235700 1 --2- 172.21.15.40:0/2213727181 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fab440a1350 0x7fab440a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.227+0000 7fab5ea36700 1 --2- 172.21.15.40:0/2985962780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cbdc6b0 0x56080cd80300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fab5000b0a0 tx=0x7fab50009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64d2d0cd05cb33a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.249+0000 7fab5e235700 1 --2- 172.21.15.40:0/2213727181 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fab440a1350 0x7fab440a3800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fab540a1e40 tx=0x7fab540036b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.320+0000 7fab4cdac700 1 -- 172.21.15.40:0/2985962780 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000e040 con 0x56080cbdc6b0 2023-07-18T03:36:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.320+0000 7fab4cdac700 1 -- 172.21.15.40:0/2985962780 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab5000bdf0 con 0x56080cbdc6b0 2023-07-18T03:36:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.320+0000 7fab4cdac700 1 -- 172.21.15.40:0/2985962780 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab500053b0 con 0x56080cbdc6b0 2023-07-18T03:36:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.321+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2985962780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cbdc6b0 msgr2=0x56080cd80300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.321+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/2985962780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cbdc6b0 0x56080cd80300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fab5000b0a0 tx=0x7fab50009520 comp rx=0 tx=0).stop 2023-07-18T03:36:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.360+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2985962780 shutdown_connections 2023-07-18T03:36:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.360+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/2985962780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cd816e0 0x56080cd81b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.360+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/2985962780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cbdc6b0 0x56080cd80300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.360+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/2985962780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd80840 0x56080cd811a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.360+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2985962780 >> 172.21.15.40:0/2985962780 conn(0x56080cd75be0 msgr2=0x56080cd7acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:16.373 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.376+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2985962780 shutdown_connections 2023-07-18T03:36:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.376+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2985962780 wait complete. 2023-07-18T03:36:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.376+0000 7fab6c61fb80 1 -- start start 2023-07-18T03:36:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.377+0000 7fab6c61fb80 1 Processor -- start 2023-07-18T03:36:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.377+0000 7fab6c61fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cbdc6b0 0x56080cd89930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.377+0000 7fab6c61fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cd80840 0x56080cd89e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.377+0000 7fab6c61fb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd816e0 0x56080cd90eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.377+0000 7fab6c61fb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080cc71a60 con 0x56080cd816e0 2023-07-18T03:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.377+0000 7fab6c61fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080cc71760 con 0x56080cd80840 2023-07-18T03:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.377+0000 7fab6c61fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080cb558c0 con 0x56080cbdc6b0 2023-07-18T03:36:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.377+0000 7fab6c61fb80 10 client.0 _ll_register_callbacks cb 0x56080cc6f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:36:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.382+0000 7fab5da34700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cd80840 0x56080cd89e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.382+0000 7fab5da34700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cd80840 0x56080cd89e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43532/0 (socket says 172.21.15.40:43532) 2023-07-18T03:36:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.382+0000 7fab5da34700 1 -- 172.21.15.40:0/2528578581 learned_addr learned my addr 172.21.15.40:0/2528578581 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.382+0000 7fab5ea36700 1 --2- 172.21.15.40:0/2528578581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cbdc6b0 0x56080cd89930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.382+0000 7fab5ea36700 1 --2- 172.21.15.40:0/2528578581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd816e0 0x56080cd90eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.383+0000 7fab5ea36700 1 -- 172.21.15.40:0/2528578581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cd80840 msgr2=0x56080cd89e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.383+0000 7fab5ea36700 1 --2- 172.21.15.40:0/2528578581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cd80840 0x56080cd89e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.383+0000 7fab5ea36700 1 -- 172.21.15.40:0/2528578581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd816e0 msgr2=0x56080cd90eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.383+0000 7fab5ea36700 1 --2- 172.21.15.40:0/2528578581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd816e0 0x56080cd90eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.383+0000 7fab5ea36700 1 -- 172.21.15.40:0/2528578581 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab58117770 con 0x56080cbdc6b0 2023-07-18T03:36:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.388+0000 7fab5ea36700 1 --2- 172.21.15.40:0/2528578581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cbdc6b0 0x56080cd89930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fab5000b070 tx=0x7fab5000bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=661048b855ecde1f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.400+0000 7fab41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:36:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.400+0000 7fab41ffb700 1 -- 172.21.15.40:0/2528578581 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000e040 con 0x56080cbdc6b0 2023-07-18T03:36:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.401+0000 7fab41ffb700 1 -- 172.21.15.40:0/2528578581 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fab50005510 con 0x56080cbdc6b0 2023-07-18T03:36:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.401+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2528578581 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56080cc2b490 con 0x56080cbdc6b0 2023-07-18T03:36:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.405+0000 7fab41ffb700 1 -- 172.21.15.40:0/2528578581 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab500056f0 con 0x56080cbdc6b0 2023-07-18T03:36:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.405+0000 7fab6c61fb80 10 client.26002 fetch_fsmap learned FSMap version 42 2023-07-18T03:36:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.405+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2528578581 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x56080cae8c00 con 0x56080cbdc6b0 2023-07-18T03:36:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.412+0000 7fab41ffb700 1 -- 172.21.15.40:0/2528578581 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56080cc2b490 con 0x56080cbdc6b0 2023-07-18T03:36:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.412+0000 7fab41ffb700 1 -- 172.21.15.40:0/2528578581 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 42) v1 ==== 7522+0+0 (secure 0 0 0) 0x7fab50019070 con 0x56080cbdc6b0 2023-07-18T03:36:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.412+0000 7fab6c61fb80 10 client.26002 fetch_fsmap finished waiting for FSMap version 42 2023-07-18T03:36:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.412+0000 7fab6c61fb80 10 client.26002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:36:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.412+0000 7fab6c61fb80 20 client.26002 populate_metadata read hostname 'smithi040' 2023-07-18T03:36:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.412+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/2528578581 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56080cd95300 0x56080cd97710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.412+0000 7fab6c61fb80 4 client.26002 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:36:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.412+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2528578581 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56080cccae50 con 0x56080cd95300 2023-07-18T03:36:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.414+0000 7fab5da34700 1 --2- 172.21.15.40:0/2528578581 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56080cd95300 0x56080cd97710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.437+0000 7fab5da34700 1 --2- 172.21.15.40:0/2528578581 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56080cd95300 0x56080cd97710 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6869d687d7610d03 server_cookie=308c8eacf5a47a1b in_seq=0 out_seq=0 2023-07-18T03:36:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.437+0000 7fab41ffb700 10 client.26002 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.442+0000 7fab41ffb700 1 -- 172.21.15.40:0/2528578581 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56080cccae50 con 0x56080cd95300 2023-07-18T03:36:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.442+0000 7fab41ffb700 10 client.26002 handle_command_reply: tid=0 2023-07-18T03:36:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.443+0000 7fab6c61fb80 1 client.26002 shutdown 2023-07-18T03:36:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.443+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2528578581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cbdc6b0 msgr2=0x56080cd89930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.443+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/2528578581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cbdc6b0 0x56080cd89930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fab5000b070 tx=0x7fab5000bf60 comp rx=0 tx=0).stop 2023-07-18T03:36:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.486+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2528578581 shutdown_connections 2023-07-18T03:36:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.486+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/2528578581 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56080cd95300 0x56080cd97710 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.486+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/2528578581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd816e0 0x56080cd90eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.486+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/2528578581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cbdc6b0 0x56080cd89930 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.487+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/2528578581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cd80840 0x56080cd89e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.487+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2528578581 >> 172.21.15.40:0/2528578581 conn(0x56080cd75be0 msgr2=0x56080cd7a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.487+0000 7fab41ffb700 0 client.26002 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.487+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2528578581 shutdown_connections 2023-07-18T03:36:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.487+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2528578581 wait complete. 2023-07-18T03:36:16.494 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.497+0000 7fab6c61fb80 20 client.26002 trim_cache size 0 max 16384 2023-07-18T03:36:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.766+0000 7fab6c61fb80 1 Processor -- start 2023-07-18T03:36:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.785+0000 7fab6c61fb80 1 -- start start 2023-07-18T03:36:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.785+0000 7fab6c61fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cbdc6b0 0x56080cd87b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.786+0000 7fab6c61fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cd80840 0x56080cd90e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.786+0000 7fab6c61fb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd816e0 0x56080cd885a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.786+0000 7fab6c61fb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080cd46090 con 0x56080cd816e0 2023-07-18T03:36:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.786+0000 7fab6c61fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080cd72be0 con 0x56080cbdc6b0 2023-07-18T03:36:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.786+0000 7fab6c61fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080cd8a720 con 0x56080cd80840 2023-07-18T03:36:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.787+0000 7fab5ea36700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd816e0 0x56080cd885a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.787+0000 7fab5ea36700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd816e0 0x56080cd885a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60216/0 (socket says 172.21.15.40:60216) 2023-07-18T03:36:16.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.787+0000 7fab5ea36700 1 -- 172.21.15.40:0/124307297 learned_addr learned my addr 172.21.15.40:0/124307297 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.794+0000 7fab5ea36700 1 --2- 172.21.15.40:0/124307297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cbdc6b0 0x56080cd87b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.794+0000 7fab5ea36700 1 -- 172.21.15.40:0/124307297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cd80840 msgr2=0x56080cd90e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:36:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.794+0000 7fab5ea36700 1 --2- 172.21.15.40:0/124307297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cd80840 0x56080cd90e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.794+0000 7fab5ea36700 1 -- 172.21.15.40:0/124307297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd816e0 msgr2=0x56080cd885a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.794+0000 7fab5ea36700 1 --2- 172.21.15.40:0/124307297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd816e0 0x56080cd885a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.794+0000 7fab5ea36700 1 -- 172.21.15.40:0/124307297 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56080cb261f0 con 0x56080cbdc6b0 2023-07-18T03:36:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:16.795+0000 7fab5ea36700 1 --2- 172.21.15.40:0/124307297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cbdc6b0 0x56080cd87b70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fab50002230 tx=0x7fab500099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68f99fc394eaecef server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.012+0000 7fab4d5ad700 1 -- 172.21.15.40:0/124307297 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000e040 con 0x56080cbdc6b0 2023-07-18T03:36:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.012+0000 7fab4d5ad700 1 -- 172.21.15.40:0/124307297 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab5001c510 con 0x56080cbdc6b0 2023-07-18T03:36:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.012+0000 7fab4d5ad700 1 -- 172.21.15.40:0/124307297 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50023770 con 0x56080cbdc6b0 2023-07-18T03:36:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.013+0000 7fab6c61fb80 1 -- 172.21.15.40:0/124307297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cbdc6b0 msgr2=0x56080cd87b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.014+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/124307297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cbdc6b0 0x56080cd87b70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fab50002230 tx=0x7fab500099e0 comp rx=0 tx=0).stop 2023-07-18T03:36:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.014+0000 7fab6c61fb80 1 -- 172.21.15.40:0/124307297 shutdown_connections 2023-07-18T03:36:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.014+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/124307297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd816e0 0x56080cd885a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.014+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/124307297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cbdc6b0 0x56080cd87b70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.014+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/124307297 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cd80840 0x56080cd90e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.014+0000 7fab6c61fb80 1 -- 172.21.15.40:0/124307297 >> 172.21.15.40:0/124307297 conn(0x56080cd75be0 msgr2=0x56080cd7ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.014+0000 7fab6c61fb80 1 -- 172.21.15.40:0/124307297 shutdown_connections 2023-07-18T03:36:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.014+0000 7fab6c61fb80 1 -- 172.21.15.40:0/124307297 wait complete. 2023-07-18T03:36:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.014+0000 7fab6c61fb80 1 -- start start 2023-07-18T03:36:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.015+0000 7fab6c61fb80 1 Processor -- start 2023-07-18T03:36:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.015+0000 7fab6c61fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cbdc6b0 0x56080cd8c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.015+0000 7fab6c61fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cd80840 0x56080cd8cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.015+0000 7fab6c61fb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd816e0 0x56080cd8fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.015+0000 7fab6c61fb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080cd90220 con 0x56080cd816e0 2023-07-18T03:36:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.015+0000 7fab6c61fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080cd90390 con 0x56080cbdc6b0 2023-07-18T03:36:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.015+0000 7fab6c61fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56080cd90550 con 0x56080cd80840 2023-07-18T03:36:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.015+0000 7fab6c61fb80 10 client.0 _ll_register_callbacks cb 0x7fab4800b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:36:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.015+0000 7fab5ea36700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cbdc6b0 0x56080cd8c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.022+0000 7fab5da34700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cd80840 0x56080cd8cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.022+0000 7fab5da34700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cd80840 0x56080cd8cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49248/0 (socket says 172.21.15.40:49248) 2023-07-18T03:36:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.022+0000 7fab5da34700 1 -- 172.21.15.40:0/2959252874 learned_addr learned my addr 172.21.15.40:0/2959252874 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.022+0000 7fab5ea36700 1 --2- 172.21.15.40:0/2959252874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd816e0 0x56080cd8fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.025+0000 7fab5da34700 1 -- 172.21.15.40:0/2959252874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cbdc6b0 msgr2=0x56080cd8c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.025+0000 7fab5da34700 1 --2- 172.21.15.40:0/2959252874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cbdc6b0 0x56080cd8c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.025+0000 7fab5da34700 1 -- 172.21.15.40:0/2959252874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd816e0 msgr2=0x56080cd8fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.025+0000 7fab5da34700 1 --2- 172.21.15.40:0/2959252874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd816e0 0x56080cd8fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:17.033 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.025+0000 7fab5da34700 1 -- 172.21.15.40:0/2959252874 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56080cae8c00 con 0x56080cd80840 2023-07-18T03:36:17.042 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.044+0000 7fab5da34700 1 --2- 172.21.15.40:0/2959252874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cd80840 0x56080cd8cd60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fab48005d60 tx=0x7fab48019b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a979de26d9eaa5a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.199+0000 7fab41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:36:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.199+0000 7fab41ffb700 1 -- 172.21.15.40:0/2959252874 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48017040 con 0x56080cd80840 2023-07-18T03:36:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.199+0000 7fab41ffb700 1 -- 172.21.15.40:0/2959252874 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fab4800b740 con 0x56080cd80840 2023-07-18T03:36:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.203+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2959252874 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab5000bdf0 con 0x56080cd80840 2023-07-18T03:36:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.216+0000 7fab41ffb700 1 -- 172.21.15.40:0/2959252874 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab4800b900 con 0x56080cd80840 2023-07-18T03:36:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.216+0000 7fab6c61fb80 10 client.26005 fetch_fsmap learned FSMap version 42 2023-07-18T03:36:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.216+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2959252874 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x56080cafe250 con 0x56080cd80840 2023-07-18T03:36:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.221+0000 7fab41ffb700 1 -- 172.21.15.40:0/2959252874 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab5000bdf0 con 0x56080cd80840 2023-07-18T03:36:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.221+0000 7fab41ffb700 1 -- 172.21.15.40:0/2959252874 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 42) v1 ==== 7522+0+0 (secure 0 0 0) 0x7fab48011440 con 0x56080cd80840 2023-07-18T03:36:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.222+0000 7fab6c61fb80 10 client.26005 fetch_fsmap finished waiting for FSMap version 42 2023-07-18T03:36:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.222+0000 7fab6c61fb80 10 client.26005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:36:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.222+0000 7fab6c61fb80 20 client.26005 populate_metadata read hostname 'smithi040' 2023-07-18T03:36:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.222+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/2959252874 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56080cde9e10 0x56080cdec220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.222+0000 7fab6c61fb80 4 client.26005 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:36:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.222+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2959252874 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56080cc8cff0 con 0x56080cde9e10 2023-07-18T03:36:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.223+0000 7fab5ea36700 1 --2- 172.21.15.40:0/2959252874 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56080cde9e10 0x56080cdec220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.224+0000 7fab5ea36700 1 --2- 172.21.15.40:0/2959252874 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56080cde9e10 0x56080cdec220 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92b3a8bf574b27e1 server_cookie=c28a5d8b09e08338 in_seq=0 out_seq=0 2023-07-18T03:36:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.235+0000 7fab41ffb700 10 client.26005 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.568+0000 7fab41ffb700 1 -- 172.21.15.40:0/2959252874 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56080cc8cff0 con 0x56080cde9e10 2023-07-18T03:36:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.568+0000 7fab41ffb700 10 client.26005 handle_command_reply: tid=0 2023-07-18T03:36:17.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:17 smithi134 ceph-mon[98216]: pgmap v456: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2023-07-18T03:36:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.568+0000 7fab6c61fb80 1 client.26005 shutdown 2023-07-18T03:36:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.568+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2959252874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cd80840 msgr2=0x56080cd8cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.568+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/2959252874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cd80840 0x56080cd8cd60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fab48005d60 tx=0x7fab48019b40 comp rx=0 tx=0).stop 2023-07-18T03:36:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.569+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2959252874 shutdown_connections 2023-07-18T03:36:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.569+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/2959252874 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56080cde9e10 0x56080cdec220 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.569+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/2959252874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56080cd816e0 0x56080cd8fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.569+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/2959252874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56080cbdc6b0 0x56080cd8c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.569+0000 7fab6c61fb80 1 --2- 172.21.15.40:0/2959252874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56080cd80840 0x56080cd8cd60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.569+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2959252874 >> 172.21.15.40:0/2959252874 conn(0x56080cd75be0 msgr2=0x56080cd896e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.569+0000 7fab41ffb700 0 client.26005 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.569+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2959252874 shutdown_connections 2023-07-18T03:36:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.569+0000 7fab6c61fb80 1 -- 172.21.15.40:0/2959252874 wait complete. 2023-07-18T03:36:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.569+0000 7fab6c61fb80 20 client.26005 trim_cache size 0 max 16384 2023-07-18T03:36:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:36:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:36:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:36:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:36:17.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:17 smithi040 ceph-mon[96439]: pgmap v456: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2023-07-18T03:36:17.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:17 smithi064 ceph-mon[98059]: pgmap v456: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2023-07-18T03:36:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.768+0000 7fab60499700 1 -- 172.21.15.40:0/2213727181 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fab440a1350 msgr2=0x7fab440a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.768+0000 7fab60499700 1 --2- 172.21.15.40:0/2213727181 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fab440a1350 0x7fab440a3800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fab540a1e40 tx=0x7fab540036b0 comp rx=0 tx=0).stop 2023-07-18T03:36:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.768+0000 7fab60499700 1 -- 172.21.15.40:0/2213727181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab580713c0 msgr2=0x7fab581a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.768+0000 7fab60499700 1 --2- 172.21.15.40:0/2213727181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab580713c0 0x7fab581a45f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fab5400e970 tx=0x7fab5400ec80 comp rx=0 tx=0).stop 2023-07-18T03:36:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.769+0000 7fab60499700 1 -- 172.21.15.40:0/2213727181 shutdown_connections 2023-07-18T03:36:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.769+0000 7fab60499700 1 --2- 172.21.15.40:0/2213727181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab58107c80 0x7fab581abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.769+0000 7fab60499700 1 --2- 172.21.15.40:0/2213727181 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fab440a1350 0x7fab440a3800 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.769+0000 7fab60499700 1 --2- 172.21.15.40:0/2213727181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab580713c0 0x7fab581a45f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.769+0000 7fab60499700 1 --2- 172.21.15.40:0/2213727181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab581072f0 0x7fab581a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.769+0000 7fab60499700 1 -- 172.21.15.40:0/2213727181 >> 172.21.15.40:0/2213727181 conn(0x7fab5806ca40 msgr2=0x7fab5806f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.769+0000 7fab60499700 1 -- 172.21.15.40:0/2213727181 shutdown_connections 2023-07-18T03:36:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:17.769+0000 7fab60499700 1 -- 172.21.15.40:0/2213727181 wait complete. 2023-07-18T03:36:18.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.616+0000 7f37baf1e700 1 -- 172.21.15.40:0/4141683054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b410bd60 msgr2=0x7f37b410e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:18.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.616+0000 7f37baf1e700 1 --2- 172.21.15.40:0/4141683054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b410bd60 0x7f37b410e1f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00b0a0 tx=0x7f37ac00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:36:18.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.617+0000 7f37baf1e700 1 -- 172.21.15.40:0/4141683054 shutdown_connections 2023-07-18T03:36:18.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.617+0000 7f37baf1e700 1 --2- 172.21.15.40:0/4141683054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b410bd60 0x7f37b410e1f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:18.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.617+0000 7f37baf1e700 1 --2- 172.21.15.40:0/4141683054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37b4070ee0 0x7f37b40712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:18.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.617+0000 7f37baf1e700 1 --2- 172.21.15.40:0/4141683054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37b4071830 0x7f37b410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:18.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.617+0000 7f37baf1e700 1 -- 172.21.15.40:0/4141683054 >> 172.21.15.40:0/4141683054 conn(0x7f37b406ca90 msgr2=0x7f37b406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:18.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.618+0000 7f37baf1e700 1 -- 172.21.15.40:0/4141683054 shutdown_connections 2023-07-18T03:36:18.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.618+0000 7f37baf1e700 1 -- 172.21.15.40:0/4141683054 wait complete. 2023-07-18T03:36:18.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.619+0000 7f37baf1e700 1 Processor -- start 2023-07-18T03:36:18.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.619+0000 7f37baf1e700 1 -- start start 2023-07-18T03:36:18.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.619+0000 7f37baf1e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b4070ee0 0x7f37b4113bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:18.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.619+0000 7f37baf1e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37b4071830 0x7f37b4114110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:18.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.619+0000 7f37baf1e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37b410bd60 0x7f37b41b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:18.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.619+0000 7f37baf1e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b4114720 con 0x7f37b410bd60 2023-07-18T03:36:18.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.619+0000 7f37baf1e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b41b4d90 con 0x7f37b4070ee0 2023-07-18T03:36:18.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.619+0000 7f37baf1e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b41b4ed0 con 0x7f37b4071830 2023-07-18T03:36:18.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.620+0000 7f37b3fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37b4071830 0x7f37b4114110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:18.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.620+0000 7f37b3fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37b4071830 0x7f37b4114110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49262/0 (socket says 172.21.15.40:49262) 2023-07-18T03:36:18.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.620+0000 7f37b3fff700 1 -- 172.21.15.40:0/768142118 learned_addr learned my addr 172.21.15.40:0/768142118 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:18.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.620+0000 7f37b8cba700 1 --2- 172.21.15.40:0/768142118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b4070ee0 0x7f37b4113bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:18.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.620+0000 7f37b3fff700 1 -- 172.21.15.40:0/768142118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b4070ee0 msgr2=0x7f37b4113bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:18.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.620+0000 7f37b3fff700 1 --2- 172.21.15.40:0/768142118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b4070ee0 0x7f37b4113bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:18.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.620+0000 7f37b3fff700 1 -- 172.21.15.40:0/768142118 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37b410bd60 msgr2=0x7f37b41b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:18.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.620+0000 7f37b3fff700 1 --2- 172.21.15.40:0/768142118 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37b410bd60 0x7f37b41b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:18.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.621+0000 7f37b3fff700 1 -- 172.21.15.40:0/768142118 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac009d00 con 0x7f37b4071830 2023-07-18T03:36:18.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.633+0000 7f37b3fff700 1 --2- 172.21.15.40:0/768142118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37b4071830 0x7f37b4114110 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f37a800e970 tx=0x7f37a800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.692+0000 7f37b1ffb700 1 -- 172.21.15.40:0/768142118 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8009790 con 0x7f37b4071830 2023-07-18T03:36:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.692+0000 7f37b1ffb700 1 -- 172.21.15.40:0/768142118 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37a80044e0 con 0x7f37b4071830 2023-07-18T03:36:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.692+0000 7f37b1ffb700 1 -- 172.21.15.40:0/768142118 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8003d10 con 0x7f37b4071830 2023-07-18T03:36:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.692+0000 7f37baf1e700 1 -- 172.21.15.40:0/768142118 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b41b5130 con 0x7f37b4071830 2023-07-18T03:36:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.692+0000 7f37baf1e700 1 -- 172.21.15.40:0/768142118 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b41b55c0 con 0x7f37b4071830 2023-07-18T03:36:18.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.698+0000 7f37b1ffb700 1 -- 172.21.15.40:0/768142118 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f37a8018430 con 0x7f37b4071830 2023-07-18T03:36:18.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.699+0000 7f37b1ffb700 1 --2- 172.21.15.40:0/768142118 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f379c0a1350 0x7f379c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:18.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.699+0000 7f37b1ffb700 1 -- 172.21.15.40:0/768142118 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f37a8014070 con 0x7f37b4071830 2023-07-18T03:36:18.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.710+0000 7f37b8cba700 1 --2- 172.21.15.40:0/768142118 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f379c0a1350 0x7f379c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:18.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.724+0000 7f37c70a4b80 1 Processor -- start 2023-07-18T03:36:18.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.725+0000 7f37c70a4b80 1 -- start start 2023-07-18T03:36:18.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.725+0000 7f37c70a4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bd4d0 0x55cb984bd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:18.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.725+0000 7f37c70a4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bde20 0x55cb984be6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:18.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.725+0000 7f37c70a4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bec70 0x55cb984bf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:18.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.725+0000 7f37c70a4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb983c2cb0 con 0x55cb984bec70 2023-07-18T03:36:18.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.725+0000 7f37c70a4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb98415a40 con 0x55cb984bd4d0 2023-07-18T03:36:18.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.725+0000 7f37c70a4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb9840eed0 con 0x55cb984bde20 2023-07-18T03:36:18.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.727+0000 7f37b3fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bde20 0x55cb984be6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:18.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.727+0000 7f37b3fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bde20 0x55cb984be6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49272/0 (socket says 172.21.15.40:49272) 2023-07-18T03:36:18.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.727+0000 7f37b3fff700 1 -- 172.21.15.40:0/42955303 learned_addr learned my addr 172.21.15.40:0/42955303 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:18.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.729+0000 7f37b8cba700 1 --2- 172.21.15.40:0/42955303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bd4d0 0x55cb984bd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:18.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.735+0000 7f37b94bb700 1 --2- 172.21.15.40:0/42955303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bec70 0x55cb984bf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:18.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.738+0000 7f37b8cba700 1 --2- 172.21.15.40:0/768142118 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f379c0a1350 0x7f379c0a3800 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009bf0 tx=0x7f37a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:18.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.738+0000 7f37b3fff700 1 -- 172.21.15.40:0/42955303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bd4d0 msgr2=0x55cb984bd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:18.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.738+0000 7f37b3fff700 1 --2- 172.21.15.40:0/42955303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bd4d0 0x55cb984bd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:18.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.738+0000 7f37b3fff700 1 -- 172.21.15.40:0/42955303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bec70 msgr2=0x55cb984bf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:18.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.738+0000 7f37b3fff700 1 --2- 172.21.15.40:0/42955303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bec70 0x55cb984bf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:18.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.738+0000 7f37b3fff700 1 -- 172.21.15.40:0/42955303 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb9823cde0 con 0x55cb984bde20 2023-07-18T03:36:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:18.748+0000 7f37b3fff700 1 --2- 172.21.15.40:0/42955303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bde20 0x55cb984be6e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f37a80a1e00 tx=0x7f37a8099af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc29407f80308d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.129+0000 7f37a2ffd700 1 -- 172.21.15.40:0/42955303 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8099ee0 con 0x55cb984bde20 2023-07-18T03:36:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.129+0000 7f37a2ffd700 1 -- 172.21.15.40:0/42955303 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37a809df10 con 0x55cb984bde20 2023-07-18T03:36:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.129+0000 7f37a2ffd700 1 -- 172.21.15.40:0/42955303 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a809a140 con 0x55cb984bde20 2023-07-18T03:36:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.130+0000 7f37c70a4b80 1 -- 172.21.15.40:0/42955303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bde20 msgr2=0x55cb984be6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:19.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.130+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/42955303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bde20 0x55cb984be6e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f37a80a1e00 tx=0x7f37a8099af0 comp rx=0 tx=0).stop 2023-07-18T03:36:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.143+0000 7f37c70a4b80 1 -- 172.21.15.40:0/42955303 shutdown_connections 2023-07-18T03:36:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.143+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/42955303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bec70 0x55cb984bf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.143+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/42955303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bd4d0 0x55cb984bd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.143+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/42955303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bde20 0x55cb984be6e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:19.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.143+0000 7f37c70a4b80 1 -- 172.21.15.40:0/42955303 >> 172.21.15.40:0/42955303 conn(0x55cb984b2da0 msgr2=0x55cb984b81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:19.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.163+0000 7f37c70a4b80 1 -- 172.21.15.40:0/42955303 shutdown_connections 2023-07-18T03:36:19.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.163+0000 7f37c70a4b80 1 -- 172.21.15.40:0/42955303 wait complete. 2023-07-18T03:36:19.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.164+0000 7f37c70a4b80 1 -- start start 2023-07-18T03:36:19.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.164+0000 7f37c70a4b80 1 Processor -- start 2023-07-18T03:36:19.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.165+0000 7f37c70a4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bd4d0 0x55cb984c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:19.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.165+0000 7f37c70a4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bde20 0x55cb984c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:19.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.165+0000 7f37c70a4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bec70 0x55cb984ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:19.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.165+0000 7f37c70a4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb983aeac0 con 0x55cb984bd4d0 2023-07-18T03:36:19.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.165+0000 7f37c70a4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb983ae7c0 con 0x55cb984bde20 2023-07-18T03:36:19.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.165+0000 7f37c70a4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb98292920 con 0x55cb984bec70 2023-07-18T03:36:19.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.165+0000 7f37c70a4b80 10 client.0 _ll_register_callbacks cb 0x55cb983ac760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:36:19.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.166+0000 7f37b94bb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bec70 0x55cb984ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:19.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.166+0000 7f37b94bb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bec70 0x55cb984ce730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49278/0 (socket says 172.21.15.40:49278) 2023-07-18T03:36:19.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.166+0000 7f37b94bb700 1 -- 172.21.15.40:0/3830767860 learned_addr learned my addr 172.21.15.40:0/3830767860 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:19.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.172+0000 7f37b8cba700 1 --2- 172.21.15.40:0/3830767860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bd4d0 0x55cb984c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:19.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.177+0000 7f37b3fff700 1 --2- 172.21.15.40:0/3830767860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bde20 0x55cb984c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:19.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.178+0000 7f37b8cba700 1 -- 172.21.15.40:0/3830767860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bec70 msgr2=0x55cb984ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:19.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.178+0000 7f37b8cba700 1 --2- 172.21.15.40:0/3830767860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bec70 0x55cb984ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.178+0000 7f37b8cba700 1 -- 172.21.15.40:0/3830767860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bde20 msgr2=0x55cb984c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.178+0000 7f37b8cba700 1 --2- 172.21.15.40:0/3830767860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bde20 0x55cb984c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.178+0000 7f37b8cba700 1 -- 172.21.15.40:0/3830767860 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b41b55c0 con 0x55cb984bd4d0 2023-07-18T03:36:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.185+0000 7f37b8cba700 1 --2- 172.21.15.40:0/3830767860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bd4d0 0x55cb984c71b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f37a4015000 tx=0x7f37a40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9319f07ccce837 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:19.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:19 smithi134 ceph-mon[98216]: pgmap v457: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 162 op/s 2023-07-18T03:36:19.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:19 smithi064 ceph-mon[98059]: pgmap v457: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 162 op/s 2023-07-18T03:36:19.602 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:19 smithi040 ceph-mon[96439]: pgmap v457: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 162 op/s 2023-07-18T03:36:19.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.627+0000 7f3797fff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:36:19.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.627+0000 7f3797fff700 1 -- 172.21.15.40:0/3830767860 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4004220 con 0x55cb984bd4d0 2023-07-18T03:36:19.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.627+0000 7f3797fff700 1 -- 172.21.15.40:0/3830767860 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f37a4018070 con 0x55cb984bd4d0 2023-07-18T03:36:19.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.627+0000 7f37c70a4b80 1 -- 172.21.15.40:0/3830767860 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb983684f0 con 0x55cb984bd4d0 2023-07-18T03:36:19.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.634+0000 7f3797fff700 1 -- 172.21.15.40:0/3830767860 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37a4004380 con 0x55cb984bd4d0 2023-07-18T03:36:19.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.635+0000 7f37c70a4b80 10 client.16350 fetch_fsmap learned FSMap version 42 2023-07-18T03:36:19.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.635+0000 7f37c70a4b80 1 -- 172.21.15.40:0/3830767860 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55cb98225c60 con 0x55cb984bd4d0 2023-07-18T03:36:19.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.636+0000 7f3797fff700 1 -- 172.21.15.40:0/3830767860 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cb983684f0 con 0x55cb984bd4d0 2023-07-18T03:36:19.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.636+0000 7f3797fff700 1 -- 172.21.15.40:0/3830767860 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 42) v1 ==== 7522+0+0 (secure 0 0 0) 0x7f37a401f720 con 0x55cb984bd4d0 2023-07-18T03:36:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.638+0000 7f37c70a4b80 10 client.16350 fetch_fsmap finished waiting for FSMap version 42 2023-07-18T03:36:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.638+0000 7f37c70a4b80 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:36:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.638+0000 7f37c70a4b80 20 client.16350 populate_metadata read hostname 'smithi040' 2023-07-18T03:36:19.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.638+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/3830767860 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55cb984d3050 0x55cb984d5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:19.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.638+0000 7f37c70a4b80 4 client.16350 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:36:19.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.638+0000 7f37c70a4b80 1 -- 172.21.15.40:0/3830767860 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb98407eb0 con 0x55cb984d3050 2023-07-18T03:36:19.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.646+0000 7f37b3fff700 1 --2- 172.21.15.40:0/3830767860 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55cb984d3050 0x55cb984d5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:19.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.663+0000 7f37b3fff700 1 --2- 172.21.15.40:0/3830767860 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55cb984d3050 0x55cb984d5460 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21d0c7e09ca24e61 server_cookie=5314fdbfc1ebf552 in_seq=0 out_seq=0 2023-07-18T03:36:19.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.663+0000 7f3797fff700 10 client.16350 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:19.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.685+0000 7f3797fff700 1 -- 172.21.15.40:0/3830767860 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cb98407eb0 con 0x55cb984d3050 2023-07-18T03:36:19.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.685+0000 7f3797fff700 10 client.16350 handle_command_reply: tid=0 2023-07-18T03:36:19.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.686+0000 7f37c70a4b80 1 client.16350 shutdown 2023-07-18T03:36:19.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.707+0000 7f37c70a4b80 1 -- 172.21.15.40:0/3830767860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bd4d0 msgr2=0x55cb984c71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:19.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.707+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/3830767860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bd4d0 0x55cb984c71b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f37a4015000 tx=0x7f37a40047c0 comp rx=0 tx=0).stop 2023-07-18T03:36:19.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.707+0000 7f37c70a4b80 1 -- 172.21.15.40:0/3830767860 shutdown_connections 2023-07-18T03:36:19.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.707+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/3830767860 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55cb984d3050 0x55cb984d5460 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:19.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.707+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/3830767860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bec70 0x55cb984ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:19.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.707+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/3830767860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bd4d0 0x55cb984c71b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:19.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.708+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/3830767860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bde20 0x55cb984c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:19.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.708+0000 7f37c70a4b80 1 -- 172.21.15.40:0/3830767860 >> 172.21.15.40:0/3830767860 conn(0x55cb984b2da0 msgr2=0x55cb984b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:19.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.729+0000 7f3797fff700 0 client.16350 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:19.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.734+0000 7f37c70a4b80 1 -- 172.21.15.40:0/3830767860 shutdown_connections 2023-07-18T03:36:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.734+0000 7f37c70a4b80 1 -- 172.21.15.40:0/3830767860 wait complete. 2023-07-18T03:36:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:19.734+0000 7f37c70a4b80 20 client.16350 trim_cache size 0 max 16384 2023-07-18T03:36:20.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.272+0000 7f37c70a4b80 1 Processor -- start 2023-07-18T03:36:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.276+0000 7f37c70a4b80 1 -- start start 2023-07-18T03:36:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.276+0000 7f37c70a4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bd4d0 0x55cb984b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.276+0000 7f37c70a4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bde20 0x55cb984ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.276+0000 7f37c70a4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bec70 0x55cb984d5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.276+0000 7f37c70a4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb984c6d80 con 0x55cb984bec70 2023-07-18T03:36:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.276+0000 7f37c70a4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb984be1f0 con 0x55cb984bde20 2023-07-18T03:36:20.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.276+0000 7f37c70a4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb984d03a0 con 0x55cb984bd4d0 2023-07-18T03:36:20.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.280+0000 7f37b94bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bec70 0x55cb984d5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.280+0000 7f37b94bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bec70 0x55cb984d5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60262/0 (socket says 172.21.15.40:60262) 2023-07-18T03:36:20.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.280+0000 7f37b94bb700 1 -- 172.21.15.40:0/968893097 learned_addr learned my addr 172.21.15.40:0/968893097 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:20.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.281+0000 7f37b94bb700 1 -- 172.21.15.40:0/968893097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bd4d0 msgr2=0x55cb984b78b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:36:20.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.281+0000 7f37b94bb700 1 --2- 172.21.15.40:0/968893097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bd4d0 0x55cb984b78b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.281+0000 7f37b94bb700 1 -- 172.21.15.40:0/968893097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bde20 msgr2=0x55cb984ce6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:36:20.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.281+0000 7f37b94bb700 1 --2- 172.21.15.40:0/968893097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bde20 0x55cb984ce6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.281+0000 7f37b94bb700 1 -- 172.21.15.40:0/968893097 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb984bf310 con 0x55cb984bec70 2023-07-18T03:36:20.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.293+0000 7f37b94bb700 1 --2- 172.21.15.40:0/968893097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bec70 0x55cb984d5060 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f37ac007a60 tx=0x7f37ac0094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16bf35a2ebb8431a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:20.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.304+0000 7f37a37fe700 1 -- 172.21.15.40:0/968893097 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00e040 con 0x55cb984bec70 2023-07-18T03:36:20.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.304+0000 7f37a37fe700 1 -- 172.21.15.40:0/968893097 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37ac007760 con 0x55cb984bec70 2023-07-18T03:36:20.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.304+0000 7f37a37fe700 1 -- 172.21.15.40:0/968893097 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac005330 con 0x55cb984bec70 2023-07-18T03:36:20.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.305+0000 7f37c70a4b80 1 -- 172.21.15.40:0/968893097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bec70 msgr2=0x55cb984d5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:20.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.305+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/968893097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bec70 0x55cb984d5060 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f37ac007a60 tx=0x7f37ac0094d0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.346+0000 7f37c70a4b80 1 -- 172.21.15.40:0/968893097 shutdown_connections 2023-07-18T03:36:20.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.346+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/968893097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bec70 0x55cb984d5060 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.346+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/968893097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bd4d0 0x55cb984b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.346+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/968893097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bde20 0x55cb984ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.346+0000 7f37c70a4b80 1 -- 172.21.15.40:0/968893097 >> 172.21.15.40:0/968893097 conn(0x55cb984b2da0 msgr2=0x55cb984c5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:20.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.356+0000 7f37c70a4b80 1 -- 172.21.15.40:0/968893097 shutdown_connections 2023-07-18T03:36:20.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.357+0000 7f37c70a4b80 1 -- 172.21.15.40:0/968893097 wait complete. 2023-07-18T03:36:20.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.357+0000 7f37c70a4b80 1 -- start start 2023-07-18T03:36:20.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.357+0000 7f37c70a4b80 1 Processor -- start 2023-07-18T03:36:20.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.358+0000 7f37c70a4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bd4d0 0x55cb984ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.358+0000 7f37c70a4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bde20 0x55cb984cad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.358+0000 7f37c70a4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bec70 0x55cb984cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.358+0000 7f37c70a4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb984ce260 con 0x55cb984bde20 2023-07-18T03:36:20.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.358+0000 7f37c70a4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb984ce3d0 con 0x55cb984bd4d0 2023-07-18T03:36:20.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.358+0000 7f37c70a4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb984ce590 con 0x55cb984bec70 2023-07-18T03:36:20.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.358+0000 7f37c70a4b80 10 client.0 _ll_register_callbacks cb 0x55cb983ac760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:36:20.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.366+0000 7f37b3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bde20 0x55cb984cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.366+0000 7f37b3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bde20 0x55cb984cad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60266/0 (socket says 172.21.15.40:60266) 2023-07-18T03:36:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.366+0000 7f37b3fff700 1 -- 172.21.15.40:0/3091963027 learned_addr learned my addr 172.21.15.40:0/3091963027 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.366+0000 7f37b94bb700 1 --2- 172.21.15.40:0/3091963027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bec70 0x55cb984cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.366+0000 7f37b8cba700 1 --2- 172.21.15.40:0/3091963027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bd4d0 0x55cb984ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.367+0000 7f37b8cba700 1 -- 172.21.15.40:0/3091963027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bec70 msgr2=0x55cb984cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.367+0000 7f37b8cba700 1 --2- 172.21.15.40:0/3091963027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bec70 0x55cb984cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.367+0000 7f37b8cba700 1 -- 172.21.15.40:0/3091963027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bde20 msgr2=0x55cb984cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.367+0000 7f37b8cba700 1 --2- 172.21.15.40:0/3091963027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bde20 0x55cb984cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.367+0000 7f37b8cba700 1 -- 172.21.15.40:0/3091963027 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b4117640 con 0x55cb984bd4d0 2023-07-18T03:36:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.367+0000 7f37b8cba700 1 --2- 172.21.15.40:0/3091963027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bd4d0 0x55cb984ca7c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f37a401fab0 tx=0x7f37a40326d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59e82950ca41c73c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:20.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.369+0000 7f3797fff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:36:20.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.369+0000 7f3797fff700 1 -- 172.21.15.40:0/3091963027 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4037070 con 0x55cb984bd4d0 2023-07-18T03:36:20.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.369+0000 7f3797fff700 1 -- 172.21.15.40:0/3091963027 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f37a4018040 con 0x55cb984bd4d0 2023-07-18T03:36:20.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.369+0000 7f37c70a4b80 1 -- 172.21.15.40:0/3091963027 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37a809df10 con 0x55cb984bd4d0 2023-07-18T03:36:20.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.386+0000 7f3797fff700 1 -- 172.21.15.40:0/3091963027 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37a401fc00 con 0x55cb984bd4d0 2023-07-18T03:36:20.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.386+0000 7f37c70a4b80 10 client.26003 fetch_fsmap learned FSMap version 42 2023-07-18T03:36:20.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.386+0000 7f37c70a4b80 1 -- 172.21.15.40:0/3091963027 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55cb98263250 con 0x55cb984bd4d0 2023-07-18T03:36:20.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.406+0000 7f3797fff700 1 -- 172.21.15.40:0/3091963027 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37a809df10 con 0x55cb984bd4d0 2023-07-18T03:36:20.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.406+0000 7f3797fff700 1 -- 172.21.15.40:0/3091963027 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 42) v1 ==== 7522+0+0 (secure 0 0 0) 0x7f37a4028860 con 0x55cb984bd4d0 2023-07-18T03:36:20.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.407+0000 7f37c70a4b80 10 client.26003 fetch_fsmap finished waiting for FSMap version 42 2023-07-18T03:36:20.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.407+0000 7f37c70a4b80 10 client.26003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:36:20.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.407+0000 7f37c70a4b80 20 client.26003 populate_metadata read hostname 'smithi040' 2023-07-18T03:36:20.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.407+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/3091963027 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55cb985281a0 0x55cb9852a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:20.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.407+0000 7f37c70a4b80 4 client.26003 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:36:20.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.407+0000 7f37c70a4b80 1 -- 172.21.15.40:0/3091963027 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cb983ca050 con 0x55cb985281a0 2023-07-18T03:36:20.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.415+0000 7f37b3fff700 1 --2- 172.21.15.40:0/3091963027 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55cb985281a0 0x55cb9852a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:20.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.436+0000 7f37b3fff700 1 --2- 172.21.15.40:0/3091963027 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55cb985281a0 0x55cb9852a5b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c048c32ad106733 server_cookie=2bbe67f6943de0ec in_seq=0 out_seq=0 2023-07-18T03:36:20.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.436+0000 7f3797fff700 10 client.26003 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:20.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.437+0000 7f3797fff700 1 -- 172.21.15.40:0/3091963027 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55cb983ca050 con 0x55cb985281a0 2023-07-18T03:36:20.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.437+0000 7f3797fff700 10 client.26003 handle_command_reply: tid=0 2023-07-18T03:36:20.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.437+0000 7f37c70a4b80 1 client.26003 shutdown 2023-07-18T03:36:20.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.437+0000 7f37c70a4b80 1 -- 172.21.15.40:0/3091963027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bd4d0 msgr2=0x55cb984ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:20.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.437+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/3091963027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bd4d0 0x55cb984ca7c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f37a401fab0 tx=0x7f37a40326d0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.437+0000 7f37c70a4b80 1 -- 172.21.15.40:0/3091963027 shutdown_connections 2023-07-18T03:36:20.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.437+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/3091963027 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55cb985281a0 0x55cb9852a5b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.437+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/3091963027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb984bec70 0x55cb984cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.437+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/3091963027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cb984bd4d0 0x55cb984ca7c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.437+0000 7f37c70a4b80 1 --2- 172.21.15.40:0/3091963027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55cb984bde20 0x55cb984cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.437+0000 7f37c70a4b80 1 -- 172.21.15.40:0/3091963027 >> 172.21.15.40:0/3091963027 conn(0x55cb984b2da0 msgr2=0x55cb984b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:20.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.437+0000 7f3797fff700 0 client.26003 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:20.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.438+0000 7f37c70a4b80 1 -- 172.21.15.40:0/3091963027 shutdown_connections 2023-07-18T03:36:20.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.438+0000 7f37c70a4b80 1 -- 172.21.15.40:0/3091963027 wait complete. 2023-07-18T03:36:20.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.439+0000 7f37c70a4b80 20 client.26003 trim_cache size 0 max 16384 2023-07-18T03:36:20.438 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:36:20.439 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-18T03:36:20.439 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:36:20.439 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:36:20.439 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:36:20.439 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:36:20.439 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:36:20.439 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:36:20.440 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:36:20.440 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:36:20.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.595+0000 7f37baf1e700 1 -- 172.21.15.40:0/768142118 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f379c0a1350 msgr2=0x7f379c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:20.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.595+0000 7f37baf1e700 1 --2- 172.21.15.40:0/768142118 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f379c0a1350 0x7f379c0a3800 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009bf0 tx=0x7f37a4009260 comp rx=0 tx=0).stop 2023-07-18T03:36:20.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.595+0000 7f37baf1e700 1 -- 172.21.15.40:0/768142118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37b4071830 msgr2=0x7f37b4114110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:20.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.595+0000 7f37baf1e700 1 --2- 172.21.15.40:0/768142118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37b4071830 0x7f37b4114110 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f37a800e970 tx=0x7f37a800ed30 comp rx=0 tx=0).stop 2023-07-18T03:36:20.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.595+0000 7f37baf1e700 1 -- 172.21.15.40:0/768142118 shutdown_connections 2023-07-18T03:36:20.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.595+0000 7f37baf1e700 1 --2- 172.21.15.40:0/768142118 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37b410bd60 0x7f37b41b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.595+0000 7f37baf1e700 1 --2- 172.21.15.40:0/768142118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37b4070ee0 0x7f37b4113bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.595+0000 7f37baf1e700 1 --2- 172.21.15.40:0/768142118 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f379c0a1350 0x7f379c0a3800 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.595+0000 7f37baf1e700 1 --2- 172.21.15.40:0/768142118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37b4071830 0x7f37b4114110 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:20.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.595+0000 7f37baf1e700 1 -- 172.21.15.40:0/768142118 >> 172.21.15.40:0/768142118 conn(0x7f37b406ca90 msgr2=0x7f37b406dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:20.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.597+0000 7f37baf1e700 1 -- 172.21.15.40:0/768142118 shutdown_connections 2023-07-18T03:36:20.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:20.597+0000 7f37baf1e700 1 -- 172.21.15.40:0/768142118 wait complete. 2023-07-18T03:36:20.764 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:36:21.583 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:21 smithi040 ceph-mon[96439]: pgmap v458: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 114 op/s 2023-07-18T03:36:21.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:21 smithi064 ceph-mon[98059]: pgmap v458: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 114 op/s 2023-07-18T03:36:21.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:21 smithi134 ceph-mon[98216]: pgmap v458: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 114 op/s 2023-07-18T03:36:23.191 INFO:tasks.workunit.client.0.smithi040.stdout: 336 2355032 369 1671436 661 710712 1003 2023-07-18T03:36:23.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:22 smithi134 ceph-mon[98216]: pgmap v459: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 225 op/s 2023-07-18T03:36:23.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:22 smithi040 ceph-mon[96439]: pgmap v459: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 225 op/s 2023-07-18T03:36:23.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:22 smithi064 ceph-mon[98059]: pgmap v459: 97 pgs: 97 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 225 op/s 2023-07-18T03:36:24.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:23 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:36:24.343 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:23 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:36:24.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:23 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:36:25.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:24 smithi134 ceph-mon[98216]: pgmap v460: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2023-07-18T03:36:25.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:24 smithi040 ceph-mon[96439]: pgmap v460: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2023-07-18T03:36:25.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:24 smithi064 ceph-mon[98059]: pgmap v460: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 244 op/s 2023-07-18T03:36:27.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:27 smithi134 ceph-mon[98216]: pgmap v461: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 265 op/s 2023-07-18T03:36:27.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:27 smithi040 ceph-mon[96439]: pgmap v461: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 265 op/s 2023-07-18T03:36:27.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:27 smithi064 ceph-mon[98059]: pgmap v461: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 265 op/s 2023-07-18T03:36:29.567 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:29 smithi134 ceph-mon[98216]: pgmap v462: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.3 MiB/s wr, 378 op/s 2023-07-18T03:36:29.586 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:29 smithi040 ceph-mon[96439]: pgmap v462: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.3 MiB/s wr, 378 op/s 2023-07-18T03:36:29.601 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:29 smithi064 ceph-mon[98059]: pgmap v462: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.3 MiB/s wr, 378 op/s 2023-07-18T03:36:31.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:31 smithi134 ceph-mon[98216]: pgmap v463: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.7 MiB/s wr, 334 op/s 2023-07-18T03:36:31.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:31 smithi040 ceph-mon[96439]: pgmap v463: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.7 MiB/s wr, 334 op/s 2023-07-18T03:36:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.589+0000 7feaea25e700 1 -- 172.21.15.40:0/2502538150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae4071850 msgr2=0x7feae410b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.589+0000 7feaea25e700 1 --2- 172.21.15.40:0/2502538150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae4071850 0x7feae410b7d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fead80098f0 tx=0x7fead8009c00 comp rx=0 tx=0).stop 2023-07-18T03:36:31.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:31 smithi064 ceph-mon[98059]: pgmap v463: 97 pgs: 97 active+clean; 1.2 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.7 MiB/s wr, 334 op/s 2023-07-18T03:36:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.608+0000 7feaea25e700 1 -- 172.21.15.40:0/2502538150 shutdown_connections 2023-07-18T03:36:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.608+0000 7feaea25e700 1 --2- 172.21.15.40:0/2502538150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae410bd60 0x7feae410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.608+0000 7feaea25e700 1 --2- 172.21.15.40:0/2502538150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feae4070f00 0x7feae4071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.608+0000 7feaea25e700 1 --2- 172.21.15.40:0/2502538150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae4071850 0x7feae410b7d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.608+0000 7feaea25e700 1 -- 172.21.15.40:0/2502538150 >> 172.21.15.40:0/2502538150 conn(0x7feae406ca50 msgr2=0x7feae406eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.610+0000 7feaea25e700 1 -- 172.21.15.40:0/2502538150 shutdown_connections 2023-07-18T03:36:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.610+0000 7feaea25e700 1 -- 172.21.15.40:0/2502538150 wait complete. 2023-07-18T03:36:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.611+0000 7feaea25e700 1 Processor -- start 2023-07-18T03:36:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.612+0000 7feaea25e700 1 -- start start 2023-07-18T03:36:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.612+0000 7feaea25e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae4070f00 0x7feae41a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.612+0000 7feaea25e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae4071850 0x7feae41a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.612+0000 7feaea25e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feae410bd60 0x7feae41a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.612+0000 7feaea25e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae41a61b0 con 0x7feae4071850 2023-07-18T03:36:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.612+0000 7feaea25e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae41a6320 con 0x7feae4070f00 2023-07-18T03:36:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.612+0000 7feaea25e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae41a64e0 con 0x7feae410bd60 2023-07-18T03:36:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.613+0000 7feae37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae4070f00 0x7feae41a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.613+0000 7feae37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae4070f00 0x7feae41a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43608/0 (socket says 172.21.15.40:43608) 2023-07-18T03:36:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.613+0000 7feae37fe700 1 -- 172.21.15.40:0/2934296702 learned_addr learned my addr 172.21.15.40:0/2934296702 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.615+0000 7feae2ffd700 1 --2- 172.21.15.40:0/2934296702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae4071850 0x7feae41a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:31.620 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.623+0000 7feae3fff700 1 --2- 172.21.15.40:0/2934296702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feae410bd60 0x7feae41a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.624+0000 7feae37fe700 1 -- 172.21.15.40:0/2934296702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feae410bd60 msgr2=0x7feae41a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.624+0000 7feae37fe700 1 --2- 172.21.15.40:0/2934296702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feae410bd60 0x7feae41a5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.624+0000 7feae37fe700 1 -- 172.21.15.40:0/2934296702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae4071850 msgr2=0x7feae41a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.624+0000 7feae37fe700 1 --2- 172.21.15.40:0/2934296702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae4071850 0x7feae41a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.624+0000 7feae37fe700 1 -- 172.21.15.40:0/2934296702 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead8009520 con 0x7feae4070f00 2023-07-18T03:36:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.638+0000 7feae37fe700 1 --2- 172.21.15.40:0/2934296702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae4070f00 0x7feae41a4790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fead400c9b0 tx=0x7fead400ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:31.648 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.650+0000 7feae0ff9700 1 -- 172.21.15.40:0/2934296702 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4007b00 con 0x7feae4070f00 2023-07-18T03:36:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.650+0000 7feae0ff9700 1 -- 172.21.15.40:0/2934296702 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fead4007c60 con 0x7feae4070f00 2023-07-18T03:36:31.649 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.650+0000 7feae0ff9700 1 -- 172.21.15.40:0/2934296702 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4018660 con 0x7feae4070f00 2023-07-18T03:36:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.650+0000 7feaea25e700 1 -- 172.21.15.40:0/2934296702 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae41a67c0 con 0x7feae4070f00 2023-07-18T03:36:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.650+0000 7feaea25e700 1 -- 172.21.15.40:0/2934296702 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae410ee20 con 0x7feae4070f00 2023-07-18T03:36:31.655 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.657+0000 7feaf63e4b80 1 Processor -- start 2023-07-18T03:36:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.658+0000 7feaf63e4b80 1 -- start start 2023-07-18T03:36:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.658+0000 7feaf63e4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dbb16b0 0x55990dd55300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.658+0000 7feaf63e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dd55840 0x55990dd561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:31.656 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.658+0000 7feaf63e4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd566e0 0x55990dd56b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.658+0000 7feaf63e4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990dc5ac50 con 0x55990dd566e0 2023-07-18T03:36:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.658+0000 7feaf63e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990dcad9e0 con 0x55990dd55840 2023-07-18T03:36:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.658+0000 7feaf63e4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990dca6e70 con 0x55990dbb16b0 2023-07-18T03:36:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.659+0000 7feae2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd566e0 0x55990dd56b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.659+0000 7feae2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd566e0 0x55990dd56b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60284/0 (socket says 172.21.15.40:60284) 2023-07-18T03:36:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.659+0000 7feae2ffd700 1 -- 172.21.15.40:0/4045062995 learned_addr learned my addr 172.21.15.40:0/4045062995 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:31.658 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.661+0000 7feae3fff700 1 --2- 172.21.15.40:0/4045062995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dbb16b0 0x55990dd55300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:31.660 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.662+0000 7feae37fe700 1 --2- 172.21.15.40:0/4045062995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dd55840 0x55990dd561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.664+0000 7feae3fff700 1 -- 172.21.15.40:0/4045062995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dd55840 msgr2=0x55990dd561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.664+0000 7feae3fff700 1 --2- 172.21.15.40:0/4045062995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dd55840 0x55990dd561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.664+0000 7feae3fff700 1 -- 172.21.15.40:0/4045062995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd566e0 msgr2=0x55990dd56b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.664+0000 7feae3fff700 1 --2- 172.21.15.40:0/4045062995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd566e0 0x55990dd56b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.664+0000 7feae3fff700 1 -- 172.21.15.40:0/4045062995 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55990dad4d80 con 0x55990dbb16b0 2023-07-18T03:36:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.668+0000 7feae0ff9700 1 -- 172.21.15.40:0/2934296702 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fead400f450 con 0x7feae4070f00 2023-07-18T03:36:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.670+0000 7feae0ff9700 1 --2- 172.21.15.40:0/2934296702 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7feacc0a1350 0x7feacc0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:31.667 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.670+0000 7feae0ff9700 1 -- 172.21.15.40:0/2934296702 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fead40cfae0 con 0x7feae4070f00 2023-07-18T03:36:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.673+0000 7feae37fe700 1 --2- 172.21.15.40:0/2934296702 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7feacc0a1350 0x7feacc0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:31.676 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.675+0000 7feae3fff700 1 --2- 172.21.15.40:0/4045062995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dbb16b0 0x55990dd55300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feadc009ae0 tx=0x7feadc009df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=333a42a91ef477b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.693+0000 7feae37fe700 1 --2- 172.21.15.40:0/2934296702 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7feacc0a1350 0x7feacc0a3800 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fead40998e0 tx=0x7fead40cd430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:31.698 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.701+0000 7fead1ffb700 1 -- 172.21.15.40:0/4045062995 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc014410 con 0x55990dbb16b0 2023-07-18T03:36:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.701+0000 7fead1ffb700 1 -- 172.21.15.40:0/4045062995 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feadc00f050 con 0x55990dbb16b0 2023-07-18T03:36:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.701+0000 7fead1ffb700 1 -- 172.21.15.40:0/4045062995 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc013760 con 0x55990dbb16b0 2023-07-18T03:36:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.702+0000 7feaf63e4b80 1 -- 172.21.15.40:0/4045062995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dbb16b0 msgr2=0x55990dd55300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:31.699 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.702+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/4045062995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dbb16b0 0x55990dd55300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feadc009ae0 tx=0x7feadc009df0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.702+0000 7feaf63e4b80 1 -- 172.21.15.40:0/4045062995 shutdown_connections 2023-07-18T03:36:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.702+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/4045062995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd566e0 0x55990dd56b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.702+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/4045062995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dbb16b0 0x55990dd55300 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.702+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/4045062995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dd55840 0x55990dd561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.702+0000 7feaf63e4b80 1 -- 172.21.15.40:0/4045062995 >> 172.21.15.40:0/4045062995 conn(0x55990dd4abe0 msgr2=0x55990dd4fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.703+0000 7feaf63e4b80 1 -- 172.21.15.40:0/4045062995 shutdown_connections 2023-07-18T03:36:31.702 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.703+0000 7feaf63e4b80 1 -- 172.21.15.40:0/4045062995 wait complete. 2023-07-18T03:36:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.703+0000 7feaf63e4b80 1 -- start start 2023-07-18T03:36:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.704+0000 7feaf63e4b80 1 Processor -- start 2023-07-18T03:36:31.703 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.704+0000 7feaf63e4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dbb16b0 0x55990dd5e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.704+0000 7feaf63e4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd55840 0x55990dd5ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.704+0000 7feaf63e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dd566e0 0x55990dd65eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.704+0000 7feaf63e4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990dc46a60 con 0x55990dd55840 2023-07-18T03:36:31.704 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.704+0000 7feaf63e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990dc46760 con 0x55990dd566e0 2023-07-18T03:36:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.704+0000 7feaf63e4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990db2a8c0 con 0x55990dbb16b0 2023-07-18T03:36:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.704+0000 7feaf63e4b80 10 client.0 _ll_register_callbacks cb 0x55990dc44700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:36:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.706+0000 7feae2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd55840 0x55990dd5ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.706+0000 7feae2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd55840 0x55990dd5ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60292/0 (socket says 172.21.15.40:60292) 2023-07-18T03:36:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.706+0000 7feae2ffd700 1 -- 172.21.15.40:0/2892539852 learned_addr learned my addr 172.21.15.40:0/2892539852 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.707+0000 7feae3fff700 1 --2- 172.21.15.40:0/2892539852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dbb16b0 0x55990dd5e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.707+0000 7feae3fff700 1 --2- 172.21.15.40:0/2892539852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dd566e0 0x55990dd65eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.707+0000 7feae2ffd700 1 -- 172.21.15.40:0/2892539852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dbb16b0 msgr2=0x55990dd5e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.707+0000 7feae2ffd700 1 --2- 172.21.15.40:0/2892539852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dbb16b0 0x55990dd5e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.706 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.707+0000 7feae2ffd700 1 -- 172.21.15.40:0/2892539852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dd566e0 msgr2=0x55990dd65eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.707+0000 7feae2ffd700 1 --2- 172.21.15.40:0/2892539852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dd566e0 0x55990dd65eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.707+0000 7feae2ffd700 1 -- 172.21.15.40:0/2892539852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feadc005e70 con 0x55990dd55840 2023-07-18T03:36:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.709+0000 7feae2ffd700 1 --2- 172.21.15.40:0/2892539852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd55840 0x55990dd5ee70 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fead800b2e0 tx=0x7fead8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e47218dc6bbce2b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.729+0000 7feac6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:36:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.729+0000 7feac6ffd700 1 -- 172.21.15.40:0/2892539852 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead801c040 con 0x55990dd55840 2023-07-18T03:36:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.729+0000 7feac6ffd700 1 -- 172.21.15.40:0/2892539852 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fead80055a0 con 0x55990dd55840 2023-07-18T03:36:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.729+0000 7feaf63e4b80 1 -- 172.21.15.40:0/2892539852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55990dc00490 con 0x55990dd55840 2023-07-18T03:36:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.733+0000 7feac6ffd700 1 -- 172.21.15.40:0/2892539852 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fead800b610 con 0x55990dd55840 2023-07-18T03:36:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.734+0000 7feaf63e4b80 10 client.16368 fetch_fsmap learned FSMap version 43 2023-07-18T03:36:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.734+0000 7feaf63e4b80 1 -- 172.21.15.40:0/2892539852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55990dabdc00 con 0x55990dd55840 2023-07-18T03:36:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.743+0000 7feac6ffd700 1 -- 172.21.15.40:0/2892539852 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55990dc00490 con 0x55990dd55840 2023-07-18T03:36:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.743+0000 7feac6ffd700 1 -- 172.21.15.40:0/2892539852 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 43) v1 ==== 7518+0+0 (secure 0 0 0) 0x7fead8015020 con 0x55990dd55840 2023-07-18T03:36:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.743+0000 7feaf63e4b80 10 client.16368 fetch_fsmap finished waiting for FSMap version 43 2023-07-18T03:36:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.743+0000 7feaf63e4b80 10 client.16368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:36:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.743+0000 7feaf63e4b80 20 client.16368 populate_metadata read hostname 'smithi040' 2023-07-18T03:36:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.743+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/2892539852 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55990dd6a8a0 0x55990dd6ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.743+0000 7feaf63e4b80 4 client.16368 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:36:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.743+0000 7feaf63e4b80 1 -- 172.21.15.40:0/2892539852 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55990dc9fe50 con 0x55990dd6a8a0 2023-07-18T03:36:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.745+0000 7feae3fff700 1 --2- 172.21.15.40:0/2892539852 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55990dd6a8a0 0x55990dd6ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.786+0000 7feae3fff700 1 --2- 172.21.15.40:0/2892539852 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55990dd6a8a0 0x55990dd6ccb0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f67ee8f431fcc71 server_cookie=eb05d047e948185 in_seq=0 out_seq=0 2023-07-18T03:36:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.786+0000 7feac6ffd700 10 client.16368 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.841+0000 7feac6ffd700 1 -- 172.21.15.40:0/2892539852 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55990dc9fe50 con 0x55990dd6a8a0 2023-07-18T03:36:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.841+0000 7feac6ffd700 10 client.16368 handle_command_reply: tid=0 2023-07-18T03:36:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.841+0000 7feaf63e4b80 1 client.16368 shutdown 2023-07-18T03:36:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.842+0000 7feaf63e4b80 1 -- 172.21.15.40:0/2892539852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd55840 msgr2=0x55990dd5ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:31.839 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.842+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/2892539852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd55840 0x55990dd5ee70 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fead800b2e0 tx=0x7fead8005780 comp rx=0 tx=0).stop 2023-07-18T03:36:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.842+0000 7feaf63e4b80 1 -- 172.21.15.40:0/2892539852 shutdown_connections 2023-07-18T03:36:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.842+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/2892539852 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55990dd6a8a0 0x55990dd6ccb0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.842+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/2892539852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dd566e0 0x55990dd65eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.842+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/2892539852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dbb16b0 0x55990dd5e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.842+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/2892539852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd55840 0x55990dd5ee70 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.840 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.842+0000 7feaf63e4b80 1 -- 172.21.15.40:0/2892539852 >> 172.21.15.40:0/2892539852 conn(0x55990dd4abe0 msgr2=0x55990dd4fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.842+0000 7feac6ffd700 0 client.16368 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.843+0000 7feaf63e4b80 1 -- 172.21.15.40:0/2892539852 shutdown_connections 2023-07-18T03:36:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.843+0000 7feaf63e4b80 1 -- 172.21.15.40:0/2892539852 wait complete. 2023-07-18T03:36:31.841 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.843+0000 7feaf63e4b80 20 client.16368 trim_cache size 0 max 16384 2023-07-18T03:36:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.977+0000 7feaf63e4b80 1 Processor -- start 2023-07-18T03:36:31.974 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.977+0000 7feaf63e4b80 1 -- start start 2023-07-18T03:36:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.977+0000 7feaf63e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dbb16b0 0x55990dd5d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:31.975 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.978+0000 7feaf63e4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd55840 0x55990dd65dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.978+0000 7feaf63e4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dd566e0 0x55990dd5e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.978+0000 7feaf63e4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990dd6a470 con 0x55990dd55840 2023-07-18T03:36:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.978+0000 7feaf63e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990dd5e4a0 con 0x55990dbb16b0 2023-07-18T03:36:31.976 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.978+0000 7feaf63e4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990dd66340 con 0x55990dd566e0 2023-07-18T03:36:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.982+0000 7feae2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd55840 0x55990dd65dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.983+0000 7feae3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dbb16b0 0x55990dd5d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.986+0000 7feae3fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dd566e0 0x55990dd5e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.987+0000 7feae3fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dd566e0 0x55990dd5e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49332/0 (socket says 172.21.15.40:49332) 2023-07-18T03:36:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.987+0000 7feae3fff700 1 -- 172.21.15.40:0/1070206846 learned_addr learned my addr 172.21.15.40:0/1070206846 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.987+0000 7feae3fff700 1 -- 172.21.15.40:0/1070206846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dbb16b0 msgr2=0x55990dd5d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.987+0000 7feae3fff700 1 --2- 172.21.15.40:0/1070206846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dbb16b0 0x55990dd5d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.987+0000 7feae3fff700 1 -- 172.21.15.40:0/1070206846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd55840 msgr2=0x55990dd65dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.994+0000 7feae3fff700 1 --2- 172.21.15.40:0/1070206846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd55840 0x55990dd65dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.994+0000 7feae3fff700 1 -- 172.21.15.40:0/1070206846 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55990dafb1f0 con 0x55990dd566e0 2023-07-18T03:36:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.994+0000 7feae2ffd700 1 --2- 172.21.15.40:0/1070206846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd55840 0x55990dd65dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:36:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.994+0000 7feae3fff700 1 --2- 172.21.15.40:0/1070206846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dd566e0 0x55990dd5e930 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55990dd55060 tx=0x7feadc009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20a720cc761883c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.999+0000 7fead27fc700 1 -- 172.21.15.40:0/1070206846 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc026040 con 0x55990dd566e0 2023-07-18T03:36:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.999+0000 7fead27fc700 1 -- 172.21.15.40:0/1070206846 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feadc029d90 con 0x55990dd566e0 2023-07-18T03:36:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:31.999+0000 7fead27fc700 1 -- 172.21.15.40:0/1070206846 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc014d10 con 0x55990dd566e0 2023-07-18T03:36:32.099 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.084+0000 7feaf63e4b80 1 -- 172.21.15.40:0/1070206846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dd566e0 msgr2=0x55990dd5e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.084+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/1070206846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dd566e0 0x55990dd5e930 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55990dd55060 tx=0x7feadc009f70 comp rx=0 tx=0).stop 2023-07-18T03:36:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.112+0000 7feaf63e4b80 1 -- 172.21.15.40:0/1070206846 shutdown_connections 2023-07-18T03:36:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.112+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/1070206846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dd566e0 0x55990dd5e930 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.112+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/1070206846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dbb16b0 0x55990dd5d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.112+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/1070206846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd55840 0x55990dd65dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.112+0000 7feaf63e4b80 1 -- 172.21.15.40:0/1070206846 >> 172.21.15.40:0/1070206846 conn(0x55990dd4abe0 msgr2=0x55990dd715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.113+0000 7feaf63e4b80 1 -- 172.21.15.40:0/1070206846 shutdown_connections 2023-07-18T03:36:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.113+0000 7feaf63e4b80 1 -- 172.21.15.40:0/1070206846 wait complete. 2023-07-18T03:36:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.113+0000 7feaf63e4b80 1 -- start start 2023-07-18T03:36:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.114+0000 7feaf63e4b80 1 Processor -- start 2023-07-18T03:36:32.112 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.114+0000 7feaf63e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dbb16b0 0x55990dd61d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.114+0000 7feaf63e4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd55840 0x55990dd622d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.114+0000 7feaf63e4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dd566e0 0x55990dd652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.114+0000 7feaf63e4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990dd65830 con 0x55990dd55840 2023-07-18T03:36:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.114+0000 7feaf63e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990dd659a0 con 0x55990dbb16b0 2023-07-18T03:36:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.114+0000 7feaf63e4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990dd65b60 con 0x55990dd566e0 2023-07-18T03:36:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.114+0000 7feaf63e4b80 10 client.0 _ll_register_callbacks cb 0x7fead800f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:36:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.115+0000 7feae3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dbb16b0 0x55990dd61d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.115+0000 7feae3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dbb16b0 0x55990dd61d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43634/0 (socket says 172.21.15.40:43634) 2023-07-18T03:36:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.115+0000 7feae3fff700 1 -- 172.21.15.40:0/1867751350 learned_addr learned my addr 172.21.15.40:0/1867751350 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.115+0000 7feae3fff700 1 --2- 172.21.15.40:0/1867751350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dd566e0 0x55990dd652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.115+0000 7feae3fff700 1 -- 172.21.15.40:0/1867751350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dd566e0 msgr2=0x55990dd652f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.115+0000 7feae3fff700 1 --2- 172.21.15.40:0/1867751350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dd566e0 0x55990dd652f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.115+0000 7feae3fff700 1 -- 172.21.15.40:0/1867751350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd55840 msgr2=0x55990dd622d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.115+0000 7feae3fff700 1 --2- 172.21.15.40:0/1867751350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd55840 0x55990dd622d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.115+0000 7feae3fff700 1 -- 172.21.15.40:0/1867751350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55990dad4d80 con 0x55990dbb16b0 2023-07-18T03:36:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.116+0000 7feae3fff700 1 --2- 172.21.15.40:0/1867751350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dbb16b0 0x55990dd61d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7feadc014850 tx=0x7feadc023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51888c04f12062ad server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.230+0000 7feac6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:36:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.230+0000 7feac6ffd700 1 -- 172.21.15.40:0/1867751350 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc029620 con 0x55990dbb16b0 2023-07-18T03:36:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.230+0000 7feac6ffd700 1 -- 172.21.15.40:0/1867751350 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7feadc029ad0 con 0x55990dbb16b0 2023-07-18T03:36:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.230+0000 7feaf63e4b80 1 -- 172.21.15.40:0/1867751350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feadc00f050 con 0x55990dbb16b0 2023-07-18T03:36:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.232+0000 7feac6ffd700 1 -- 172.21.15.40:0/1867751350 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7feadc032460 con 0x55990dbb16b0 2023-07-18T03:36:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.233+0000 7feaf63e4b80 10 client.26018 fetch_fsmap learned FSMap version 43 2023-07-18T03:36:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.233+0000 7feaf63e4b80 1 -- 172.21.15.40:0/1867751350 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55990dad3250 con 0x55990dbb16b0 2023-07-18T03:36:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.243+0000 7feac6ffd700 1 -- 172.21.15.40:0/1867751350 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feadc00f050 con 0x55990dbb16b0 2023-07-18T03:36:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.243+0000 7feac6ffd700 1 -- 172.21.15.40:0/1867751350 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 43) v1 ==== 7518+0+0 (secure 0 0 0) 0x7feadc009b10 con 0x55990dbb16b0 2023-07-18T03:36:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.243+0000 7feaf63e4b80 10 client.26018 fetch_fsmap finished waiting for FSMap version 43 2023-07-18T03:36:32.241 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.243+0000 7feaf63e4b80 10 client.26018 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:36:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.243+0000 7feaf63e4b80 20 client.26018 populate_metadata read hostname 'smithi040' 2023-07-18T03:36:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.243+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/1867751350 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55990ddbf490 0x55990ddc18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.243+0000 7feaf63e4b80 4 client.26018 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:36:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.243+0000 7feaf63e4b80 1 -- 172.21.15.40:0/1867751350 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55990dc61ff0 con 0x55990ddbf490 2023-07-18T03:36:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.246+0000 7feae2ffd700 1 --2- 172.21.15.40:0/1867751350 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55990ddbf490 0x55990ddc18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.261+0000 7feae2ffd700 1 --2- 172.21.15.40:0/1867751350 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55990ddbf490 0x55990ddc18a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4ac711eb7befa9a server_cookie=9baa8ef6c678239e in_seq=0 out_seq=0 2023-07-18T03:36:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.262+0000 7feac6ffd700 10 client.26018 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.965+0000 7feac6ffd700 1 -- 172.21.15.40:0/1867751350 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55990dc61ff0 con 0x55990ddbf490 2023-07-18T03:36:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.965+0000 7feac6ffd700 10 client.26018 handle_command_reply: tid=0 2023-07-18T03:36:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.965+0000 7feaf63e4b80 1 client.26018 shutdown 2023-07-18T03:36:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.966+0000 7feaf63e4b80 1 -- 172.21.15.40:0/1867751350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dbb16b0 msgr2=0x55990dd61d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.966+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/1867751350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dbb16b0 0x55990dd61d90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7feadc014850 tx=0x7feadc023040 comp rx=0 tx=0).stop 2023-07-18T03:36:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.966+0000 7feaf63e4b80 1 -- 172.21.15.40:0/1867751350 shutdown_connections 2023-07-18T03:36:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.966+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/1867751350 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55990ddbf490 0x55990ddc18a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.966+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/1867751350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55990dd566e0 0x55990dd652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.966+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/1867751350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990dbb16b0 0x55990dd61d90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.966+0000 7feaf63e4b80 1 --2- 172.21.15.40:0/1867751350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55990dd55840 0x55990dd622d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.966+0000 7feaf63e4b80 1 -- 172.21.15.40:0/1867751350 >> 172.21.15.40:0/1867751350 conn(0x55990dd4abe0 msgr2=0x55990dd5d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.966+0000 7feac6ffd700 0 client.26018 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.967+0000 7feaf63e4b80 1 -- 172.21.15.40:0/1867751350 shutdown_connections 2023-07-18T03:36:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.967+0000 7feaf63e4b80 1 -- 172.21.15.40:0/1867751350 wait complete. 2023-07-18T03:36:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:32.967+0000 7feaf63e4b80 20 client.26018 trim_cache size 0 max 16384 2023-07-18T03:36:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:36:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:36:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:36:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:36:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:33.150+0000 7feaea25e700 1 -- 172.21.15.40:0/2934296702 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7feacc0a1350 msgr2=0x7feacc0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:33.150+0000 7feaea25e700 1 --2- 172.21.15.40:0/2934296702 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7feacc0a1350 0x7feacc0a3800 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fead40998e0 tx=0x7fead40cd430 comp rx=0 tx=0).stop 2023-07-18T03:36:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:33.150+0000 7feaea25e700 1 -- 172.21.15.40:0/2934296702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae4070f00 msgr2=0x7feae41a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:33.150+0000 7feaea25e700 1 --2- 172.21.15.40:0/2934296702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae4070f00 0x7feae41a4790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fead400c9b0 tx=0x7fead400ccc0 comp rx=0 tx=0).stop 2023-07-18T03:36:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:33.151+0000 7feaea25e700 1 -- 172.21.15.40:0/2934296702 shutdown_connections 2023-07-18T03:36:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:33.151+0000 7feaea25e700 1 --2- 172.21.15.40:0/2934296702 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7feacc0a1350 0x7feacc0a3800 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:33.151+0000 7feaea25e700 1 --2- 172.21.15.40:0/2934296702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feae410bd60 0x7feae41a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:33.151+0000 7feaea25e700 1 --2- 172.21.15.40:0/2934296702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feae4070f00 0x7feae41a4790 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:33.151+0000 7feaea25e700 1 --2- 172.21.15.40:0/2934296702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feae4071850 0x7feae41a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:33.151+0000 7feaea25e700 1 -- 172.21.15.40:0/2934296702 >> 172.21.15.40:0/2934296702 conn(0x7feae406ca50 msgr2=0x7feae406e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:33.151+0000 7feaea25e700 1 -- 172.21.15.40:0/2934296702 shutdown_connections 2023-07-18T03:36:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:33.151+0000 7feaea25e700 1 -- 172.21.15.40:0/2934296702 wait complete. 2023-07-18T03:36:33.200 INFO:tasks.workunit.client.0.smithi040.stdout: 343 2227310 566 1577849 394 688653 1034 2023-07-18T03:36:33.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:32 smithi134 ceph-mon[98216]: pgmap v464: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 574 op/s 2023-07-18T03:36:33.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:32 smithi040 ceph-mon[96439]: pgmap v464: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 574 op/s 2023-07-18T03:36:33.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:32 smithi064 ceph-mon[98059]: pgmap v464: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 574 op/s 2023-07-18T03:36:35.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:35 smithi134 ceph-mon[98216]: pgmap v465: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 571 op/s 2023-07-18T03:36:35.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:35 smithi040 ceph-mon[96439]: pgmap v465: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 571 op/s 2023-07-18T03:36:35.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:35 smithi064 ceph-mon[98059]: pgmap v465: 97 pgs: 97 active+clean; 1.3 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.7 MiB/s wr, 571 op/s 2023-07-18T03:36:37.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:37 smithi134 ceph-mon[98216]: pgmap v466: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.8 MiB/s wr, 618 op/s 2023-07-18T03:36:37.582 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:37 smithi040 ceph-mon[96439]: pgmap v466: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.8 MiB/s wr, 618 op/s 2023-07-18T03:36:37.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:37 smithi064 ceph-mon[98059]: pgmap v466: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.8 MiB/s wr, 618 op/s 2023-07-18T03:36:39.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:39 smithi040 ceph-mon[96439]: pgmap v467: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.1 MiB/s wr, 789 op/s 2023-07-18T03:36:39.602 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:39 smithi064 ceph-mon[98059]: pgmap v467: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.1 MiB/s wr, 789 op/s 2023-07-18T03:36:39.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:39 smithi134 ceph-mon[98216]: pgmap v467: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 8.1 MiB/s wr, 789 op/s 2023-07-18T03:36:41.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:41 smithi134 ceph-mon[98216]: pgmap v468: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 662 op/s 2023-07-18T03:36:41.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:41 smithi040 ceph-mon[96439]: pgmap v468: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 662 op/s 2023-07-18T03:36:41.849 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:41 smithi064 ceph-mon[98059]: pgmap v468: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 662 op/s 2023-07-18T03:36:43.208 INFO:tasks.workunit.client.0.smithi040.stdout: 353 2338869 494 1657107 483 751578 774 2023-07-18T03:36:43.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:42 smithi134 ceph-mon[98216]: pgmap v469: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.3 MiB/s wr, 751 op/s 2023-07-18T03:36:43.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:42 smithi040 ceph-mon[96439]: pgmap v469: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.3 MiB/s wr, 751 op/s 2023-07-18T03:36:43.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:42 smithi064 ceph-mon[98059]: pgmap v469: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.3 MiB/s wr, 751 op/s 2023-07-18T03:36:45.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:45 smithi134 ceph-mon[98216]: pgmap v470: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 536 op/s 2023-07-18T03:36:45.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:45 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:36:45.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:45 smithi040 ceph-mon[96439]: pgmap v470: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 536 op/s 2023-07-18T03:36:45.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:45 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:36:45.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:45 smithi064 ceph-mon[98059]: pgmap v470: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 536 op/s 2023-07-18T03:36:45.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:45 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:36:47.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:47 smithi040 ceph-mon[96439]: pgmap v471: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 453 op/s 2023-07-18T03:36:47.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:47 smithi064 ceph-mon[98059]: pgmap v471: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 453 op/s 2023-07-18T03:36:47.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:47 smithi134 ceph-mon[98216]: pgmap v471: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 453 op/s 2023-07-18T03:36:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.868+0000 7f9aa188f700 1 -- 172.21.15.40:0/2767520470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a9c10bd60 msgr2=0x7f9a9c10e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.868+0000 7f9aa188f700 1 --2- 172.21.15.40:0/2767520470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a9c10bd60 0x7f9a9c10e210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009e00 tx=0x7f9a9400b330 comp rx=0 tx=0).stop 2023-07-18T03:36:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.868+0000 7f9aa188f700 1 -- 172.21.15.40:0/2767520470 shutdown_connections 2023-07-18T03:36:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.868+0000 7f9aa188f700 1 --2- 172.21.15.40:0/2767520470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a9c10bd60 0x7f9a9c10e210 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.868+0000 7f9aa188f700 1 --2- 172.21.15.40:0/2767520470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a9c070f00 0x7f9a9c071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.868+0000 7f9aa188f700 1 --2- 172.21.15.40:0/2767520470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a9c071850 0x7f9a9c10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.868+0000 7f9aa188f700 1 -- 172.21.15.40:0/2767520470 >> 172.21.15.40:0/2767520470 conn(0x7f9a9c06ca50 msgr2=0x7f9a9c06eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.868+0000 7f9aa188f700 1 -- 172.21.15.40:0/2767520470 shutdown_connections 2023-07-18T03:36:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.868+0000 7f9aa188f700 1 -- 172.21.15.40:0/2767520470 wait complete. 2023-07-18T03:36:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.869+0000 7f9aa188f700 1 Processor -- start 2023-07-18T03:36:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.869+0000 7f9aa188f700 1 -- start start 2023-07-18T03:36:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.870+0000 7f9aa188f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a9c070f00 0x7f9a9c1163d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.870+0000 7f9aa188f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a9c071850 0x7f9a9c116910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.870+0000 7f9aa188f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a9c10bd60 0x7f9a9c114af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.870+0000 7f9aa188f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c1150f0 con 0x7f9a9c071850 2023-07-18T03:36:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.870+0000 7f9aa188f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c115260 con 0x7f9a9c10bd60 2023-07-18T03:36:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.870+0000 7f9aa188f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c1153d0 con 0x7f9a9c070f00 2023-07-18T03:36:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.870+0000 7f9a9a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a9c071850 0x7f9a9c116910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.870+0000 7f9a9a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a9c071850 0x7f9a9c116910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60320/0 (socket says 172.21.15.40:60320) 2023-07-18T03:36:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.870+0000 7f9a9a7fc700 1 -- 172.21.15.40:0/4059434074 learned_addr learned my addr 172.21.15.40:0/4059434074 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.871+0000 7f9a9affd700 1 --2- 172.21.15.40:0/4059434074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a9c070f00 0x7f9a9c1163d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.871+0000 7f9a9b7fe700 1 --2- 172.21.15.40:0/4059434074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a9c10bd60 0x7f9a9c114af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.872+0000 7f9a9affd700 1 -- 172.21.15.40:0/4059434074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a9c10bd60 msgr2=0x7f9a9c114af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.872+0000 7f9a9affd700 1 --2- 172.21.15.40:0/4059434074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a9c10bd60 0x7f9a9c114af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.872+0000 7f9a9affd700 1 -- 172.21.15.40:0/4059434074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a9c071850 msgr2=0x7f9a9c116910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.872+0000 7f9a9affd700 1 --2- 172.21.15.40:0/4059434074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a9c071850 0x7f9a9c116910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.872+0000 7f9a9affd700 1 -- 172.21.15.40:0/4059434074 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a94005e70 con 0x7f9a9c070f00 2023-07-18T03:36:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.873+0000 7f9a9affd700 1 --2- 172.21.15.40:0/4059434074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a9c070f00 0x7f9a9c1163d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00e9f0 tx=0x7f9a8c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.947+0000 7f9aa088d700 1 -- 172.21.15.40:0/4059434074 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c00c9c0 con 0x7f9a9c070f00 2023-07-18T03:36:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.947+0000 7f9aa088d700 1 -- 172.21.15.40:0/4059434074 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a8c010430 con 0x7f9a9c070f00 2023-07-18T03:36:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.947+0000 7f9aa088d700 1 -- 172.21.15.40:0/4059434074 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c018680 con 0x7f9a9c070f00 2023-07-18T03:36:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.947+0000 7f9aa188f700 1 -- 172.21.15.40:0/4059434074 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9c1156b0 con 0x7f9a9c070f00 2023-07-18T03:36:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.947+0000 7f9aa188f700 1 -- 172.21.15.40:0/4059434074 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9c1b58f0 con 0x7f9a9c070f00 2023-07-18T03:36:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.960+0000 7f9aada15b80 1 Processor -- start 2023-07-18T03:36:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.967+0000 7f9aada15b80 1 -- start start 2023-07-18T03:36:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.968+0000 7f9aada15b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c39b6b0 0x56196c53f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.968+0000 7f9aada15b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c53f840 0x56196c5401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.968+0000 7f9aada15b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c5406e0 0x56196c540b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.968+0000 7f9aada15b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c444c50 con 0x56196c5406e0 2023-07-18T03:36:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.968+0000 7f9aada15b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c4979e0 con 0x56196c39b6b0 2023-07-18T03:36:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.968+0000 7f9aada15b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c490e70 con 0x56196c53f840 2023-07-18T03:36:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.968+0000 7f9a9b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c39b6b0 0x56196c53f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.968+0000 7f9a9b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c39b6b0 0x56196c53f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43654/0 (socket says 172.21.15.40:43654) 2023-07-18T03:36:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.968+0000 7f9a9b7fe700 1 -- 172.21.15.40:0/2172484081 learned_addr learned my addr 172.21.15.40:0/2172484081 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.970+0000 7f9a9affd700 1 --2- 172.21.15.40:0/2172484081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c53f840 0x56196c5401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.970+0000 7f9a9b7fe700 1 -- 172.21.15.40:0/2172484081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c53f840 msgr2=0x56196c5401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.970+0000 7f9a9b7fe700 1 --2- 172.21.15.40:0/2172484081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c53f840 0x56196c5401a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.970+0000 7f9a9b7fe700 1 -- 172.21.15.40:0/2172484081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c5406e0 msgr2=0x56196c540b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.970+0000 7f9a9b7fe700 1 --2- 172.21.15.40:0/2172484081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c5406e0 0x56196c540b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.970+0000 7f9a9b7fe700 1 -- 172.21.15.40:0/2172484081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56196c2bed80 con 0x56196c39b6b0 2023-07-18T03:36:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.971+0000 7f9a9b7fe700 1 --2- 172.21.15.40:0/2172484081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c39b6b0 0x56196c53f300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a940074f0 tx=0x7f9a94009ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19f35cf788ebe250 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.998+0000 7f9aa088d700 1 -- 172.21.15.40:0/4059434074 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9a8c00cb20 con 0x7f9a9c070f00 2023-07-18T03:36:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:48.999+0000 7f9aa088d700 1 --2- 172.21.15.40:0/4059434074 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9a840a1350 0x7f9a840a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.000+0000 7f9aa088d700 1 -- 172.21.15.40:0/4059434074 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9a8c014070 con 0x7f9a9c070f00 2023-07-18T03:36:49.001 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.006+0000 7f9a9affd700 1 --2- 172.21.15.40:0/4059434074 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9a840a1350 0x7f9a840a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:49.022 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.027+0000 7f9a9affd700 1 --2- 172.21.15.40:0/4059434074 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9a840a1350 0x7f9a840a3800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c012020 tx=0x7f9a8c0ccb10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.197+0000 7f9a81ffb700 1 -- 172.21.15.40:0/2172484081 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9400e040 con 0x56196c39b6b0 2023-07-18T03:36:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.197+0000 7f9a81ffb700 1 -- 172.21.15.40:0/2172484081 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a94007e80 con 0x56196c39b6b0 2023-07-18T03:36:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.197+0000 7f9a81ffb700 1 -- 172.21.15.40:0/2172484081 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9401c640 con 0x56196c39b6b0 2023-07-18T03:36:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.198+0000 7f9aada15b80 1 -- 172.21.15.40:0/2172484081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c39b6b0 msgr2=0x56196c53f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:49.195 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.198+0000 7f9aada15b80 1 --2- 172.21.15.40:0/2172484081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c39b6b0 0x56196c53f300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a940074f0 tx=0x7f9a94009ac0 comp rx=0 tx=0).stop 2023-07-18T03:36:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.199+0000 7f9aada15b80 1 -- 172.21.15.40:0/2172484081 shutdown_connections 2023-07-18T03:36:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.199+0000 7f9aada15b80 1 --2- 172.21.15.40:0/2172484081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c5406e0 0x56196c540b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.199+0000 7f9aada15b80 1 --2- 172.21.15.40:0/2172484081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c39b6b0 0x56196c53f300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.199+0000 7f9aada15b80 1 --2- 172.21.15.40:0/2172484081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c53f840 0x56196c5401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:49.196 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.199+0000 7f9aada15b80 1 -- 172.21.15.40:0/2172484081 >> 172.21.15.40:0/2172484081 conn(0x56196c534be0 msgr2=0x56196c539cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.206+0000 7f9aada15b80 1 -- 172.21.15.40:0/2172484081 shutdown_connections 2023-07-18T03:36:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.206+0000 7f9aada15b80 1 -- 172.21.15.40:0/2172484081 wait complete. 2023-07-18T03:36:49.203 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.207+0000 7f9aada15b80 1 -- start start 2023-07-18T03:36:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.207+0000 7f9aada15b80 1 Processor -- start 2023-07-18T03:36:49.204 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.208+0000 7f9aada15b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c39b6b0 0x56196c5488a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.208+0000 7f9aada15b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c53f840 0x56196c548de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.208+0000 7f9aada15b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c5406e0 0x56196c54fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:49.205 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.208+0000 7f9aada15b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c430a60 con 0x56196c39b6b0 2023-07-18T03:36:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.208+0000 7f9aada15b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c430760 con 0x56196c53f840 2023-07-18T03:36:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.208+0000 7f9aada15b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c3148c0 con 0x56196c5406e0 2023-07-18T03:36:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.208+0000 7f9aada15b80 10 client.0 _ll_register_callbacks cb 0x56196c42e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:36:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.208+0000 7f9a9a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c53f840 0x56196c548de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.208+0000 7f9a9b7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c5406e0 0x56196c54fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.208+0000 7f9a9a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c53f840 0x56196c548de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43660/0 (socket says 172.21.15.40:43660) 2023-07-18T03:36:49.207 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.208+0000 7f9a9a7fc700 1 -- 172.21.15.40:0/1547841152 learned_addr learned my addr 172.21.15.40:0/1547841152 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.218+0000 7f9a9b7fe700 1 -- 172.21.15.40:0/1547841152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c53f840 msgr2=0x56196c548de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.218+0000 7f9a9b7fe700 1 --2- 172.21.15.40:0/1547841152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c53f840 0x56196c548de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.218+0000 7f9a9b7fe700 1 -- 172.21.15.40:0/1547841152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c39b6b0 msgr2=0x56196c5488a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.218+0000 7f9a9b7fe700 1 --2- 172.21.15.40:0/1547841152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c39b6b0 0x56196c5488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.218+0000 7f9a9b7fe700 1 -- 172.21.15.40:0/1547841152 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9c117640 con 0x56196c5406e0 2023-07-18T03:36:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.219+0000 7f9a9b7fe700 1 --2- 172.21.15.40:0/1547841152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c5406e0 0x56196c54fe20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9a940074f0 tx=0x7f9a9400bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96ca71d4aec74cca server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.355+0000 7f9a7affd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:36:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.355+0000 7f9a7affd700 1 -- 172.21.15.40:0/1547841152 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9400e040 con 0x56196c5406e0 2023-07-18T03:36:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.355+0000 7f9a7affd700 1 -- 172.21.15.40:0/1547841152 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9a9401c700 con 0x56196c5406e0 2023-07-18T03:36:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.355+0000 7f9aada15b80 1 -- 172.21.15.40:0/1547841152 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56196c489e50 con 0x56196c5406e0 2023-07-18T03:36:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.357+0000 7f9a7affd700 1 -- 172.21.15.40:0/1547841152 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a9401c8c0 con 0x56196c5406e0 2023-07-18T03:36:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.357+0000 7f9aada15b80 10 client.26041 fetch_fsmap learned FSMap version 43 2023-07-18T03:36:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.357+0000 7f9aada15b80 1 -- 172.21.15.40:0/1547841152 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x56196c2a7c00 con 0x56196c5406e0 2023-07-18T03:36:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.369+0000 7f9a7affd700 1 -- 172.21.15.40:0/1547841152 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56196c489e50 con 0x56196c5406e0 2023-07-18T03:36:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.369+0000 7f9a7affd700 1 -- 172.21.15.40:0/1547841152 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 43) v1 ==== 7518+0+0 (secure 0 0 0) 0x7f9a94019070 con 0x56196c5406e0 2023-07-18T03:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.370+0000 7f9aada15b80 10 client.26041 fetch_fsmap finished waiting for FSMap version 43 2023-07-18T03:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.370+0000 7f9aada15b80 10 client.26041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.370+0000 7f9aada15b80 20 client.26041 populate_metadata read hostname 'smithi040' 2023-07-18T03:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.370+0000 7f9aada15b80 1 --2- 172.21.15.40:0/1547841152 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56196c5541f0 0x56196c556600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.370+0000 7f9aada15b80 4 client.26041 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:36:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.370+0000 7f9aada15b80 1 -- 172.21.15.40:0/1547841152 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56196c44bff0 con 0x56196c5541f0 2023-07-18T03:36:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.371+0000 7f9a9a7fc700 1 --2- 172.21.15.40:0/1547841152 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56196c5541f0 0x56196c556600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.408+0000 7f9a9a7fc700 1 --2- 172.21.15.40:0/1547841152 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56196c5541f0 0x56196c556600 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=296305b0a289a445 server_cookie=e6544b99b7112544 in_seq=0 out_seq=0 2023-07-18T03:36:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.408+0000 7f9a7affd700 10 client.26041 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.410+0000 7f9a7affd700 1 -- 172.21.15.40:0/1547841152 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56196c44bff0 con 0x56196c5541f0 2023-07-18T03:36:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.410+0000 7f9a7affd700 10 client.26041 handle_command_reply: tid=0 2023-07-18T03:36:49.406 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.410+0000 7f9aada15b80 1 client.26041 shutdown 2023-07-18T03:36:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.411+0000 7f9aada15b80 1 -- 172.21.15.40:0/1547841152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c5406e0 msgr2=0x56196c54fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.411+0000 7f9aada15b80 1 --2- 172.21.15.40:0/1547841152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c5406e0 0x56196c54fe20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9a940074f0 tx=0x7f9a9400bc40 comp rx=0 tx=0).stop 2023-07-18T03:36:49.407 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.411+0000 7f9aada15b80 1 -- 172.21.15.40:0/1547841152 shutdown_connections 2023-07-18T03:36:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.411+0000 7f9aada15b80 1 --2- 172.21.15.40:0/1547841152 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56196c5541f0 0x56196c556600 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.411+0000 7f9aada15b80 1 --2- 172.21.15.40:0/1547841152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c5406e0 0x56196c54fe20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.411+0000 7f9aada15b80 1 --2- 172.21.15.40:0/1547841152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c39b6b0 0x56196c5488a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.411+0000 7f9aada15b80 1 --2- 172.21.15.40:0/1547841152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c53f840 0x56196c548de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.411+0000 7f9aada15b80 1 -- 172.21.15.40:0/1547841152 >> 172.21.15.40:0/1547841152 conn(0x56196c534be0 msgr2=0x56196c5394e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.411+0000 7f9a7affd700 0 client.26041 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.412+0000 7f9aada15b80 1 -- 172.21.15.40:0/1547841152 shutdown_connections 2023-07-18T03:36:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.412+0000 7f9aada15b80 1 -- 172.21.15.40:0/1547841152 wait complete. 2023-07-18T03:36:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.412+0000 7f9aada15b80 20 client.26041 trim_cache size 0 max 16384 2023-07-18T03:36:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.525+0000 7f9aada15b80 1 Processor -- start 2023-07-18T03:36:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.525+0000 7f9aada15b80 1 -- start start 2023-07-18T03:36:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.526+0000 7f9aada15b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c39b6b0 0x56196c546ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:49.522 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.526+0000 7f9aada15b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c53f840 0x56196c54fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.526+0000 7f9aada15b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c5406e0 0x56196c543630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.526+0000 7f9a9a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c53f840 0x56196c54fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.526+0000 7f9a9a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c53f840 0x56196c54fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43670/0 (socket says 172.21.15.40:43670) 2023-07-18T03:36:49.523 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.527+0000 7f9a9a7fc700 1 -- 172.21.15.40:0/63590359 learned_addr learned my addr 172.21.15.40:0/63590359 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.527+0000 7f9aada15b80 1 -- 172.21.15.40:0/63590359 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c548470 con 0x56196c39b6b0 2023-07-18T03:36:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.527+0000 7f9a9b7fe700 1 --2- 172.21.15.40:0/63590359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c5406e0 0x56196c543630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.527+0000 7f9aada15b80 1 -- 172.21.15.40:0/63590359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c505060 con 0x56196c53f840 2023-07-18T03:36:49.524 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.527+0000 7f9aada15b80 1 -- 172.21.15.40:0/63590359 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c547d50 con 0x56196c5406e0 2023-07-18T03:36:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.530+0000 7f9a9b7fe700 1 --2- 172.21.15.40:0/63590359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c39b6b0 0x56196c546ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:49.526 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.530+0000 7f9a9a7fc700 1 -- 172.21.15.40:0/63590359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c5406e0 msgr2=0x56196c543630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.530+0000 7f9a9a7fc700 1 --2- 172.21.15.40:0/63590359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c5406e0 0x56196c543630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.530+0000 7f9a9a7fc700 1 -- 172.21.15.40:0/63590359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c39b6b0 msgr2=0x56196c546ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.530+0000 7f9a9a7fc700 1 --2- 172.21.15.40:0/63590359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c39b6b0 0x56196c546ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.530+0000 7f9a9a7fc700 1 -- 172.21.15.40:0/63590359 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56196c540d80 con 0x56196c53f840 2023-07-18T03:36:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.531+0000 7f9a9a7fc700 1 --2- 172.21.15.40:0/63590359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c53f840 0x56196c54fdb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009740 tx=0x7f9a9001c590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd6e7a46ca847d06 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.601+0000 7f9a827fc700 1 -- 172.21.15.40:0/63590359 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90017040 con 0x56196c53f840 2023-07-18T03:36:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.601+0000 7f9a827fc700 1 -- 172.21.15.40:0/63590359 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a900044f0 con 0x56196c53f840 2023-07-18T03:36:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.601+0000 7f9a827fc700 1 -- 172.21.15.40:0/63590359 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90005280 con 0x56196c53f840 2023-07-18T03:36:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.602+0000 7f9aada15b80 1 -- 172.21.15.40:0/63590359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c53f840 msgr2=0x56196c54fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.602+0000 7f9aada15b80 1 --2- 172.21.15.40:0/63590359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c53f840 0x56196c54fdb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009740 tx=0x7f9a9001c590 comp rx=0 tx=0).stop 2023-07-18T03:36:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.603+0000 7f9aada15b80 1 -- 172.21.15.40:0/63590359 shutdown_connections 2023-07-18T03:36:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.603+0000 7f9aada15b80 1 --2- 172.21.15.40:0/63590359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c5406e0 0x56196c543630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.603+0000 7f9aada15b80 1 --2- 172.21.15.40:0/63590359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c39b6b0 0x56196c546ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.603+0000 7f9aada15b80 1 --2- 172.21.15.40:0/63590359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c53f840 0x56196c54fdb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.603+0000 7f9aada15b80 1 -- 172.21.15.40:0/63590359 >> 172.21.15.40:0/63590359 conn(0x56196c534be0 msgr2=0x56196c5475a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.625+0000 7f9aada15b80 1 -- 172.21.15.40:0/63590359 shutdown_connections 2023-07-18T03:36:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.625+0000 7f9aada15b80 1 -- 172.21.15.40:0/63590359 wait complete. 2023-07-18T03:36:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.625+0000 7f9aada15b80 1 -- start start 2023-07-18T03:36:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.626+0000 7f9aada15b80 1 Processor -- start 2023-07-18T03:36:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.655+0000 7f9aada15b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c39b6b0 0x56196c54dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.655+0000 7f9aada15b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c53f840 0x56196c5a4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.655+0000 7f9aada15b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c5406e0 0x56196c5a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.655+0000 7f9aada15b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c5a6d30 con 0x56196c39b6b0 2023-07-18T03:36:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.655+0000 7f9aada15b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c5a6ea0 con 0x56196c5406e0 2023-07-18T03:36:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.655+0000 7f9aada15b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x56196c5a7060 con 0x56196c53f840 2023-07-18T03:36:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.655+0000 7f9aada15b80 10 client.0 _ll_register_callbacks cb 0x7f9a94009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:36:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.663+0000 7f9a9a7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c53f840 0x56196c5a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.663+0000 7f9a9a7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c53f840 0x56196c5a4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49380/0 (socket says 172.21.15.40:49380) 2023-07-18T03:36:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.663+0000 7f9a9a7fc700 1 -- 172.21.15.40:0/486709743 learned_addr learned my addr 172.21.15.40:0/486709743 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.663+0000 7f9a9b7fe700 1 --2- 172.21.15.40:0/486709743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c5406e0 0x56196c5a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.664+0000 7f9a9b7fe700 1 --2- 172.21.15.40:0/486709743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c39b6b0 0x56196c54dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.664+0000 7f9a9a7fc700 1 -- 172.21.15.40:0/486709743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c5406e0 msgr2=0x56196c5a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.664+0000 7f9a9a7fc700 1 --2- 172.21.15.40:0/486709743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c5406e0 0x56196c5a65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.664+0000 7f9a9a7fc700 1 -- 172.21.15.40:0/486709743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c39b6b0 msgr2=0x56196c54dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.664+0000 7f9a9a7fc700 1 --2- 172.21.15.40:0/486709743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c39b6b0 0x56196c54dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.664+0000 7f9a9a7fc700 1 -- 172.21.15.40:0/486709743 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a90009ca0 con 0x56196c53f840 2023-07-18T03:36:49.671 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.675+0000 7f9a9a7fc700 1 --2- 172.21.15.40:0/486709743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c53f840 0x56196c5a4080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009350 tx=0x7f9a90004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dafb849e01b8045 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:49.679 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.683+0000 7f9a7affd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:36:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.690+0000 7f9a7affd700 1 -- 172.21.15.40:0/486709743 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90017040 con 0x56196c53f840 2023-07-18T03:36:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.690+0000 7f9a7affd700 1 -- 172.21.15.40:0/486709743 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9a90005220 con 0x56196c53f840 2023-07-18T03:36:49.687 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.691+0000 7f9aada15b80 1 -- 172.21.15.40:0/486709743 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56196c3ea490 con 0x56196c53f840 2023-07-18T03:36:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.695+0000 7f9a7affd700 1 -- 172.21.15.40:0/486709743 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a900053e0 con 0x56196c53f840 2023-07-18T03:36:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.695+0000 7f9aada15b80 10 client.26047 fetch_fsmap learned FSMap version 43 2023-07-18T03:36:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.695+0000 7f9aada15b80 1 -- 172.21.15.40:0/486709743 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x56196c2e51f0 con 0x56196c53f840 2023-07-18T03:36:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.697+0000 7f9a7affd700 1 -- 172.21.15.40:0/486709743 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56196c3ea490 con 0x56196c53f840 2023-07-18T03:36:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.697+0000 7f9a7affd700 1 -- 172.21.15.40:0/486709743 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 43) v1 ==== 7518+0+0 (secure 0 0 0) 0x7f9a90025040 con 0x56196c53f840 2023-07-18T03:36:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.697+0000 7f9aada15b80 10 client.26047 fetch_fsmap finished waiting for FSMap version 43 2023-07-18T03:36:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.697+0000 7f9aada15b80 10 client.26047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:36:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.697+0000 7f9aada15b80 20 client.26047 populate_metadata read hostname 'smithi040' 2023-07-18T03:36:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.697+0000 7f9aada15b80 1 --2- 172.21.15.40:0/486709743 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56196c5a89a0 0x56196c5aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.697+0000 7f9aada15b80 4 client.26047 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:36:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.697+0000 7f9aada15b80 1 -- 172.21.15.40:0/486709743 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56196c2dff10 con 0x56196c5a89a0 2023-07-18T03:36:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.698+0000 7f9a9b7fe700 1 --2- 172.21.15.40:0/486709743 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56196c5a89a0 0x56196c5aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.718+0000 7f9a9b7fe700 1 --2- 172.21.15.40:0/486709743 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56196c5a89a0 0x56196c5aadb0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d28ba27853ee3d server_cookie=a9024e6188c9473c in_seq=0 out_seq=0 2023-07-18T03:36:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:49.718+0000 7f9a7affd700 10 client.26047 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:49.821 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:49 smithi134 ceph-mon[98216]: pgmap v472: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 453 op/s 2023-07-18T03:36:49.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:49 smithi040 ceph-mon[96439]: pgmap v472: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 453 op/s 2023-07-18T03:36:49.852 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:49 smithi064 ceph-mon[98059]: pgmap v472: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 453 op/s 2023-07-18T03:36:50.766 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:36:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.177+0000 7f9a7affd700 1 -- 172.21.15.40:0/486709743 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56196c2dff10 con 0x56196c5a89a0 2023-07-18T03:36:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.177+0000 7f9a7affd700 10 client.26047 handle_command_reply: tid=0 2023-07-18T03:36:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.177+0000 7f9aada15b80 1 client.26047 shutdown 2023-07-18T03:36:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.180+0000 7f9aada15b80 1 -- 172.21.15.40:0/486709743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c53f840 msgr2=0x56196c5a4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.181+0000 7f9aada15b80 1 --2- 172.21.15.40:0/486709743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c53f840 0x56196c5a4080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009350 tx=0x7f9a90004750 comp rx=0 tx=0).stop 2023-07-18T03:36:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.181+0000 7f9aada15b80 1 -- 172.21.15.40:0/486709743 shutdown_connections 2023-07-18T03:36:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.181+0000 7f9aada15b80 1 --2- 172.21.15.40:0/486709743 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56196c5a89a0 0x56196c5aadb0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.181+0000 7f9aada15b80 1 --2- 172.21.15.40:0/486709743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56196c5406e0 0x56196c5a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.181+0000 7f9aada15b80 1 --2- 172.21.15.40:0/486709743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56196c39b6b0 0x56196c54dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.181+0000 7f9aada15b80 1 --2- 172.21.15.40:0/486709743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56196c53f840 0x56196c5a4080 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.181+0000 7f9aada15b80 1 -- 172.21.15.40:0/486709743 >> 172.21.15.40:0/486709743 conn(0x56196c534be0 msgr2=0x56196c546ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.181+0000 7f9a7affd700 0 client.26047 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.185+0000 7f9aada15b80 1 -- 172.21.15.40:0/486709743 shutdown_connections 2023-07-18T03:36:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.185+0000 7f9aada15b80 1 -- 172.21.15.40:0/486709743 wait complete. 2023-07-18T03:36:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.185+0000 7f9aada15b80 20 client.26047 trim_cache size 0 max 16384 2023-07-18T03:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:36:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:36:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.266+0000 7f9aa188f700 1 -- 172.21.15.40:0/4059434074 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9a840a1350 msgr2=0x7f9a840a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.266+0000 7f9aa188f700 1 --2- 172.21.15.40:0/4059434074 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9a840a1350 0x7f9a840a3800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c012020 tx=0x7f9a8c0ccb10 comp rx=0 tx=0).stop 2023-07-18T03:36:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.266+0000 7f9aa188f700 1 -- 172.21.15.40:0/4059434074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a9c070f00 msgr2=0x7f9a9c1163d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.266+0000 7f9aa188f700 1 --2- 172.21.15.40:0/4059434074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a9c070f00 0x7f9a9c1163d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00e9f0 tx=0x7f9a8c00ed00 comp rx=0 tx=0).stop 2023-07-18T03:36:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.267+0000 7f9aa188f700 1 -- 172.21.15.40:0/4059434074 shutdown_connections 2023-07-18T03:36:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.267+0000 7f9aa188f700 1 --2- 172.21.15.40:0/4059434074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a9c10bd60 0x7f9a9c114af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.267+0000 7f9aa188f700 1 --2- 172.21.15.40:0/4059434074 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9a840a1350 0x7f9a840a3800 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.267+0000 7f9aa188f700 1 --2- 172.21.15.40:0/4059434074 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9a9c070f00 0x7f9a9c1163d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.267+0000 7f9aa188f700 1 --2- 172.21.15.40:0/4059434074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a9c071850 0x7f9a9c116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.267+0000 7f9aa188f700 1 -- 172.21.15.40:0/4059434074 >> 172.21.15.40:0/4059434074 conn(0x7f9a9c06ca50 msgr2=0x7f9a9c06e830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.267+0000 7f9aa188f700 1 -- 172.21.15.40:0/4059434074 shutdown_connections 2023-07-18T03:36:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:36:51.267+0000 7f9aa188f700 1 -- 172.21.15.40:0/4059434074 wait complete. 2023-07-18T03:36:51.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.674+0000 7f7be162d700 1 -- 172.21.15.40:0/986356774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bdc10bd60 msgr2=0x7f7bdc10e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:51.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.674+0000 7f7be162d700 1 --2- 172.21.15.40:0/986356774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bdc10bd60 0x7f7bdc10e1f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0098f0 tx=0x7f7bcc009c00 comp rx=0 tx=0).stop 2023-07-18T03:36:51.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.675+0000 7f7be162d700 1 -- 172.21.15.40:0/986356774 shutdown_connections 2023-07-18T03:36:51.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.675+0000 7f7be162d700 1 --2- 172.21.15.40:0/986356774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bdc10bd60 0x7f7bdc10e1f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.675+0000 7f7be162d700 1 --2- 172.21.15.40:0/986356774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bdc070ee0 0x7f7bdc0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.675+0000 7f7be162d700 1 --2- 172.21.15.40:0/986356774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bdc071830 0x7f7bdc10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.675+0000 7f7be162d700 1 -- 172.21.15.40:0/986356774 >> 172.21.15.40:0/986356774 conn(0x7f7bdc06ca90 msgr2=0x7f7bdc06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:51.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.675+0000 7f7be162d700 1 -- 172.21.15.40:0/986356774 shutdown_connections 2023-07-18T03:36:51.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.675+0000 7f7be162d700 1 -- 172.21.15.40:0/986356774 wait complete. 2023-07-18T03:36:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.676+0000 7f7be162d700 1 Processor -- start 2023-07-18T03:36:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.677+0000 7f7be162d700 1 -- start start 2023-07-18T03:36:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.677+0000 7f7be162d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bdc070ee0 0x7f7bdc1a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:51.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.677+0000 7f7be162d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bdc071830 0x7f7bdc1a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:51.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.677+0000 7f7be162d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bdc10bd60 0x7f7bdc1abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:51.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.677+0000 7f7be162d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc1ac1a0 con 0x7f7bdc070ee0 2023-07-18T03:36:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.677+0000 7f7be162d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc10ec40 con 0x7f7bdc10bd60 2023-07-18T03:36:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.677+0000 7f7be162d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc10edd0 con 0x7f7bdc071830 2023-07-18T03:36:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.678+0000 7f7bda7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bdc071830 0x7f7bdc1a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:51.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.678+0000 7f7bda7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bdc071830 0x7f7bdc1a4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49392/0 (socket says 172.21.15.40:49392) 2023-07-18T03:36:51.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.678+0000 7f7bda7fc700 1 -- 172.21.15.40:0/2422400247 learned_addr learned my addr 172.21.15.40:0/2422400247 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.680+0000 7f7bdaffd700 1 --2- 172.21.15.40:0/2422400247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bdc070ee0 0x7f7bdc1a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.680+0000 7f7bda7fc700 1 -- 172.21.15.40:0/2422400247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bdc10bd60 msgr2=0x7f7bdc1abbd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-18T03:36:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.680+0000 7f7bda7fc700 1 --2- 172.21.15.40:0/2422400247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bdc10bd60 0x7f7bdc1abbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.680+0000 7f7bda7fc700 1 -- 172.21.15.40:0/2422400247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bdc070ee0 msgr2=0x7f7bdc1a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.680+0000 7f7bda7fc700 1 --2- 172.21.15.40:0/2422400247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bdc070ee0 0x7f7bdc1a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.680+0000 7f7bda7fc700 1 -- 172.21.15.40:0/2422400247 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc009520 con 0x7f7bdc071830 2023-07-18T03:36:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.686+0000 7f7bda7fc700 1 --2- 172.21.15.40:0/2422400247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bdc071830 0x7f7bdc1a4b60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000d980 tx=0x7f7bd000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:51.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.697+0000 7f7bc3fff700 1 -- 172.21.15.40:0/2422400247 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd000de60 con 0x7f7bdc071830 2023-07-18T03:36:51.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.697+0000 7f7bc3fff700 1 -- 172.21.15.40:0/2422400247 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bd00044e0 con 0x7f7bdc071830 2023-07-18T03:36:51.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.697+0000 7f7bc3fff700 1 -- 172.21.15.40:0/2422400247 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0003b10 con 0x7f7bdc071830 2023-07-18T03:36:51.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.697+0000 7f7be162d700 1 -- 172.21.15.40:0/2422400247 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bdc10f0b0 con 0x7f7bdc071830 2023-07-18T03:36:51.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.697+0000 7f7be162d700 1 -- 172.21.15.40:0/2422400247 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bdc10f570 con 0x7f7bdc071830 2023-07-18T03:36:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.708+0000 7f7bed7b3b80 1 Processor -- start 2023-07-18T03:36:51.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.708+0000 7f7bed7b3b80 1 -- start start 2023-07-18T03:36:51.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.709+0000 7f7bed7b3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f24d0 0x5635843f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:51.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.709+0000 7f7bed7b3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f2e20 0x5635843f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.709+0000 7f7bed7b3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f3c70 0x5635843f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.709+0000 7f7bed7b3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635842f7cb0 con 0x5635843f2e20 2023-07-18T03:36:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.709+0000 7f7bed7b3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56358434aa40 con 0x5635843f3c70 2023-07-18T03:36:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.709+0000 7f7bed7b3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563584343ed0 con 0x5635843f24d0 2023-07-18T03:36:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.709+0000 7f7bdb7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f24d0 0x5635843f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.709+0000 7f7bdb7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f24d0 0x5635843f28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49398/0 (socket says 172.21.15.40:49398) 2023-07-18T03:36:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.709+0000 7f7bdb7fe700 1 -- 172.21.15.40:0/3844496075 learned_addr learned my addr 172.21.15.40:0/3844496075 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.710+0000 7f7bdb7fe700 1 -- 172.21.15.40:0/3844496075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f3c70 msgr2=0x5635843f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:51.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.710+0000 7f7bdb7fe700 1 --2- 172.21.15.40:0/3844496075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f3c70 0x5635843f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.710+0000 7f7bdb7fe700 1 -- 172.21.15.40:0/3844496075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f2e20 msgr2=0x5635843f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:51.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.710+0000 7f7bdb7fe700 1 --2- 172.21.15.40:0/3844496075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f2e20 0x5635843f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.710+0000 7f7bdb7fe700 1 -- 172.21.15.40:0/3844496075 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563584171de0 con 0x5635843f24d0 2023-07-18T03:36:51.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.757+0000 7f7bdb7fe700 1 --2- 172.21.15.40:0/3844496075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f24d0 0x5635843f28e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00fa40 tx=0x7f7bcc00bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=841a7bc8bd01b481 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:51.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.757+0000 7f7bc3fff700 1 -- 172.21.15.40:0/2422400247 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7bd0003c70 con 0x7f7bdc071830 2023-07-18T03:36:51.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.758+0000 7f7bc3fff700 1 --2- 172.21.15.40:0/2422400247 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7bc40a1350 0x7f7bc40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:51.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.758+0000 7f7bc3fff700 1 -- 172.21.15.40:0/2422400247 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7bd00cd4f0 con 0x7f7bdc071830 2023-07-18T03:36:51.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.770+0000 7f7bc17fa700 1 -- 172.21.15.40:0/3844496075 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc01c070 con 0x5635843f24d0 2023-07-18T03:36:51.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.770+0000 7f7bc17fa700 1 -- 172.21.15.40:0/3844496075 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bcc010ca0 con 0x5635843f24d0 2023-07-18T03:36:51.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.770+0000 7f7bc17fa700 1 -- 172.21.15.40:0/3844496075 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0052a0 con 0x5635843f24d0 2023-07-18T03:36:51.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.771+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/3844496075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f24d0 msgr2=0x5635843f28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:51.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.771+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/3844496075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f24d0 0x5635843f28e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00fa40 tx=0x7f7bcc00bef0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.771+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/3844496075 shutdown_connections 2023-07-18T03:36:51.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.771+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/3844496075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f3c70 0x5635843f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.772+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/3844496075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f24d0 0x5635843f28e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.772+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/3844496075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f2e20 0x5635843f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.772+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/3844496075 >> 172.21.15.40:0/3844496075 conn(0x5635843e7da0 msgr2=0x5635843ed1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:51.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.772+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/3844496075 shutdown_connections 2023-07-18T03:36:51.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.772+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/3844496075 wait complete. 2023-07-18T03:36:51.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.772+0000 7f7bed7b3b80 1 -- start start 2023-07-18T03:36:51.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.773+0000 7f7bed7b3b80 1 Processor -- start 2023-07-18T03:36:51.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.773+0000 7f7bed7b3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f24d0 0x5635843fc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:51.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.773+0000 7f7bed7b3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f2e20 0x5635843fc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:51.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.773+0000 7f7bed7b3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f3c70 0x5635844037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:51.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.773+0000 7f7bed7b3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635842e3ac0 con 0x5635843f24d0 2023-07-18T03:36:51.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.773+0000 7f7bed7b3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635842e37c0 con 0x5635843f3c70 2023-07-18T03:36:51.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.773+0000 7f7bed7b3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635841c7920 con 0x5635843f2e20 2023-07-18T03:36:51.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.773+0000 7f7bed7b3b80 10 client.0 _ll_register_callbacks cb 0x5635842e1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:36:51.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.774+0000 7f7bda7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f2e20 0x5635843fc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:51.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.774+0000 7f7bda7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f2e20 0x5635843fc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49406/0 (socket says 172.21.15.40:49406) 2023-07-18T03:36:51.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.774+0000 7f7bda7fc700 1 -- 172.21.15.40:0/2422724839 learned_addr learned my addr 172.21.15.40:0/2422724839 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:51.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.774+0000 7f7bdb7fe700 1 --2- 172.21.15.40:0/2422724839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f3c70 0x5635844037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:51.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.784+0000 7f7bdb7fe700 1 -- 172.21.15.40:0/2422724839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f2e20 msgr2=0x5635843fc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:51.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.784+0000 7f7bdb7fe700 1 --2- 172.21.15.40:0/2422724839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f2e20 0x5635843fc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.784+0000 7f7bdb7fe700 1 -- 172.21.15.40:0/2422724839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f24d0 msgr2=0x5635843fc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:51.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.784+0000 7f7bdb7fe700 1 --2- 172.21.15.40:0/2422724839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f24d0 0x5635843fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.784+0000 7f7bdb7fe700 1 -- 172.21.15.40:0/2422724839 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bdc117640 con 0x5635843f3c70 2023-07-18T03:36:51.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.788+0000 7f7bdb7fe700 1 --2- 172.21.15.40:0/2422724839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f3c70 0x5635844037e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00c010 tx=0x7f7bcc010e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5d14d776927f6db server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:51.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.788+0000 7f7bbeff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:36:51.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.788+0000 7f7bbeff5700 1 -- 172.21.15.40:0/2422724839 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc01c070 con 0x5635843f3c70 2023-07-18T03:36:51.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.788+0000 7f7bbeff5700 1 -- 172.21.15.40:0/2422724839 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7bcc00f660 con 0x5635843f3c70 2023-07-18T03:36:51.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.788+0000 7f7bdaffd700 1 --2- 172.21.15.40:0/2422400247 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7bc40a1350 0x7f7bc40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:51.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.789+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/2422724839 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56358429d4f0 con 0x5635843f3c70 2023-07-18T03:36:51.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.795+0000 7f7bbeff5700 1 -- 172.21.15.40:0/2422724839 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bcc00f820 con 0x5635843f3c70 2023-07-18T03:36:51.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.795+0000 7f7bed7b3b80 10 client.26042 fetch_fsmap learned FSMap version 44 2023-07-18T03:36:51.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.795+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/2422724839 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x56358415ac60 con 0x5635843f3c70 2023-07-18T03:36:51.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.804+0000 7f7bbeff5700 1 -- 172.21.15.40:0/2422724839 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56358429d4f0 con 0x5635843f3c70 2023-07-18T03:36:51.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.804+0000 7f7bbeff5700 1 -- 172.21.15.40:0/2422724839 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 44) v1 ==== 7514+0+0 (secure 0 0 0) 0x7f7bcc032050 con 0x5635843f3c70 2023-07-18T03:36:51.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.806+0000 7f7bed7b3b80 10 client.26042 fetch_fsmap finished waiting for FSMap version 44 2023-07-18T03:36:51.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.806+0000 7f7bed7b3b80 10 client.26042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:36:51.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.806+0000 7f7bed7b3b80 20 client.26042 populate_metadata read hostname 'smithi040' 2023-07-18T03:36:51.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.806+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/2422724839 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563584407b10 0x563584409f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:51.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.806+0000 7f7bed7b3b80 4 client.26042 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:36:51.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.806+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/2422724839 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56358433ceb0 con 0x563584407b10 2023-07-18T03:36:51.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:51 smithi134 ceph-mon[98216]: pgmap v473: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 250 op/s 2023-07-18T03:36:51.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.815+0000 7f7bdaffd700 1 --2- 172.21.15.40:0/2422724839 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563584407b10 0x563584409f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:51.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.815+0000 7f7bdaffd700 1 --2- 172.21.15.40:0/2422724839 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563584407b10 0x563584409f20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=536f20f47bfe005b server_cookie=69e36658f91cd596 in_seq=0 out_seq=0 2023-07-18T03:36:51.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:51 smithi064 ceph-mon[98059]: pgmap v473: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 250 op/s 2023-07-18T03:36:51.860 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:51 smithi040 ceph-mon[96439]: pgmap v473: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 250 op/s 2023-07-18T03:36:51.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.820+0000 7f7bbeff5700 10 client.26042 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:51.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.820+0000 7f7bbeff5700 1 -- 172.21.15.40:0/2422724839 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56358433ceb0 con 0x563584407b10 2023-07-18T03:36:51.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.820+0000 7f7bbeff5700 10 client.26042 handle_command_reply: tid=0 2023-07-18T03:36:51.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.821+0000 7f7bed7b3b80 1 client.26042 shutdown 2023-07-18T03:36:51.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.831+0000 7f7bdaffd700 1 --2- 172.21.15.40:0/2422400247 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7bc40a1350 0x7f7bc40a3800 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7bd40062a0 tx=0x7f7bd4004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:51.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.879+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/2422724839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f3c70 msgr2=0x5635844037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:51.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.879+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/2422724839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f3c70 0x5635844037e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00c010 tx=0x7f7bcc010e80 comp rx=0 tx=0).stop 2023-07-18T03:36:51.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.880+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/2422724839 shutdown_connections 2023-07-18T03:36:51.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.880+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/2422724839 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563584407b10 0x563584409f20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.880+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/2422724839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f3c70 0x5635844037e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.880+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/2422724839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f24d0 0x5635843fc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.880+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/2422724839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f2e20 0x5635843fc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:51.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.880+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/2422724839 >> 172.21.15.40:0/2422724839 conn(0x5635843e7da0 msgr2=0x5635843ed1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:51.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.880+0000 7f7bbeff5700 0 client.26042 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:51.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.880+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/2422724839 shutdown_connections 2023-07-18T03:36:51.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.880+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/2422724839 wait complete. 2023-07-18T03:36:51.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:51.881+0000 7f7bed7b3b80 20 client.26042 trim_cache size 0 max 16384 2023-07-18T03:36:52.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.127+0000 7f7bed7b3b80 1 Processor -- start 2023-07-18T03:36:52.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.131+0000 7f7bed7b3b80 1 -- start start 2023-07-18T03:36:52.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.131+0000 7f7bed7b3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f24d0 0x5635844040d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:52.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.131+0000 7f7bed7b3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f2e20 0x563584403770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:52.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.131+0000 7f7bed7b3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f3c70 0x5635843fae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:52.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.131+0000 7f7bed7b3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635843fb710 con 0x5635843f24d0 2023-07-18T03:36:52.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.131+0000 7f7bed7b3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635843fbdd0 con 0x5635843f3c70 2023-07-18T03:36:52.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.131+0000 7f7bed7b3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635843eed00 con 0x5635843f2e20 2023-07-18T03:36:52.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.131+0000 7f7bda7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f2e20 0x563584403770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:52.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.131+0000 7f7bda7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f2e20 0x563584403770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49414/0 (socket says 172.21.15.40:49414) 2023-07-18T03:36:52.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.131+0000 7f7bda7fc700 1 -- 172.21.15.40:0/586838027 learned_addr learned my addr 172.21.15.40:0/586838027 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:52.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.132+0000 7f7bdb7fe700 1 --2- 172.21.15.40:0/586838027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f3c70 0x5635843fae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:52.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.136+0000 7f7bdaffd700 1 --2- 172.21.15.40:0/586838027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f24d0 0x5635844040d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:52.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.136+0000 7f7bda7fc700 1 -- 172.21.15.40:0/586838027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f3c70 msgr2=0x5635843fae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:52.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.136+0000 7f7bda7fc700 1 --2- 172.21.15.40:0/586838027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f3c70 0x5635843fae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:52.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.136+0000 7f7bda7fc700 1 -- 172.21.15.40:0/586838027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f24d0 msgr2=0x5635844040d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:52.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.136+0000 7f7bda7fc700 1 --2- 172.21.15.40:0/586838027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f24d0 0x5635844040d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:52.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.136+0000 7f7bda7fc700 1 -- 172.21.15.40:0/586838027 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563584198250 con 0x5635843f2e20 2023-07-18T03:36:52.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.146+0000 7f7bda7fc700 1 --2- 172.21.15.40:0/586838027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f2e20 0x563584403770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7bd009a580 tx=0x7f7bd009a2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b60bce2ca9ed67ef server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:52.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.174+0000 7f7bc1ffb700 1 -- 172.21.15.40:0/586838027 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd00a2cd0 con 0x5635843f2e20 2023-07-18T03:36:52.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.174+0000 7f7bc1ffb700 1 -- 172.21.15.40:0/586838027 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bd00cb080 con 0x5635843f2e20 2023-07-18T03:36:52.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.174+0000 7f7bc1ffb700 1 -- 172.21.15.40:0/586838027 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0099fd0 con 0x5635843f2e20 2023-07-18T03:36:52.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.180+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/586838027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f2e20 msgr2=0x563584403770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.180+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/586838027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f2e20 0x563584403770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7bd009a580 tx=0x7f7bd009a2e0 comp rx=0 tx=0).stop 2023-07-18T03:36:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.181+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/586838027 shutdown_connections 2023-07-18T03:36:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.181+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/586838027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f3c70 0x5635843fae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.181+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/586838027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f24d0 0x5635844040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.181+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/586838027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f2e20 0x563584403770 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.181+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/586838027 >> 172.21.15.40:0/586838027 conn(0x5635843e7da0 msgr2=0x56358440e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:52.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.193+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/586838027 shutdown_connections 2023-07-18T03:36:52.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.200+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/586838027 wait complete. 2023-07-18T03:36:52.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.200+0000 7f7bed7b3b80 1 -- start start 2023-07-18T03:36:52.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.200+0000 7f7bed7b3b80 1 Processor -- start 2023-07-18T03:36:52.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.201+0000 7f7bed7b3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f24d0 0x563584402fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:52.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.201+0000 7f7bed7b3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f2e20 0x563584403510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:52.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.201+0000 7f7bed7b3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f3c70 0x5635843ffa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:52.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.201+0000 7f7bed7b3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563584400040 con 0x5635843f3c70 2023-07-18T03:36:52.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.201+0000 7f7bed7b3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635844001b0 con 0x5635843f2e20 2023-07-18T03:36:52.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.201+0000 7f7bed7b3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563584400370 con 0x5635843f24d0 2023-07-18T03:36:52.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.201+0000 7f7bed7b3b80 10 client.0 _ll_register_callbacks cb 0x7f7bd4009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:36:52.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.201+0000 7f7bdb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f3c70 0x5635843ffa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:52.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.201+0000 7f7bdb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f3c70 0x5635843ffa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60394/0 (socket says 172.21.15.40:60394) 2023-07-18T03:36:52.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.201+0000 7f7bdb7fe700 1 -- 172.21.15.40:0/1957601828 learned_addr learned my addr 172.21.15.40:0/1957601828 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:36:52.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.202+0000 7f7bdaffd700 1 --2- 172.21.15.40:0/1957601828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f24d0 0x563584402fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:52.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.202+0000 7f7bda7fc700 1 --2- 172.21.15.40:0/1957601828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f2e20 0x563584403510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:52.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.202+0000 7f7bdb7fe700 1 -- 172.21.15.40:0/1957601828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f24d0 msgr2=0x563584402fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:52.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.202+0000 7f7bdb7fe700 1 --2- 172.21.15.40:0/1957601828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f24d0 0x563584402fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:52.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.202+0000 7f7bdb7fe700 1 -- 172.21.15.40:0/1957601828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f2e20 msgr2=0x563584403510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:36:52.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.202+0000 7f7bdb7fe700 1 --2- 172.21.15.40:0/1957601828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f2e20 0x563584403510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:52.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.202+0000 7f7bdb7fe700 1 -- 172.21.15.40:0/1957601828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bdc10f570 con 0x5635843f3c70 2023-07-18T03:36:52.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.203+0000 7f7bdaffd700 1 --2- 172.21.15.40:0/1957601828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f24d0 0x563584402fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:36:52.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.203+0000 7f7bdb7fe700 1 --2- 172.21.15.40:0/1957601828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f3c70 0x5635843ffa70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x5635843e9fd0 tx=0x7f7bcc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897acc71941f85df server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:36:52.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.203+0000 7f7bbeff5700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:36:52.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.204+0000 7f7bbeff5700 1 -- 172.21.15.40:0/1957601828 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc01c040 con 0x5635843f3c70 2023-07-18T03:36:52.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.204+0000 7f7bbeff5700 1 -- 172.21.15.40:0/1957601828 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7bcc000e40 con 0x5635843f3c70 2023-07-18T03:36:52.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.207+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/1957601828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bcc010ca0 con 0x5635843f3c70 2023-07-18T03:36:52.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.228+0000 7f7bbeff5700 1 -- 172.21.15.40:0/1957601828 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bcc005b00 con 0x5635843f3c70 2023-07-18T03:36:52.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.228+0000 7f7bed7b3b80 10 client.16392 fetch_fsmap learned FSMap version 44 2023-07-18T03:36:52.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.228+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/1957601828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5635841702b0 con 0x5635843f3c70 2023-07-18T03:36:52.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.235+0000 7f7bbeff5700 1 -- 172.21.15.40:0/1957601828 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bcc010ca0 con 0x5635843f3c70 2023-07-18T03:36:52.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.235+0000 7f7bbeff5700 1 -- 172.21.15.40:0/1957601828 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 44) v1 ==== 7514+0+0 (secure 0 0 0) 0x7f7bcc009920 con 0x5635843f3c70 2023-07-18T03:36:52.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.237+0000 7f7bed7b3b80 10 client.16392 fetch_fsmap finished waiting for FSMap version 44 2023-07-18T03:36:52.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.237+0000 7f7bed7b3b80 10 client.16392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:36:52.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.237+0000 7f7bed7b3b80 20 client.16392 populate_metadata read hostname 'smithi040' 2023-07-18T03:36:52.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.237+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/1957601828 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56358445c9d0 0x56358445ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:36:52.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.237+0000 7f7bed7b3b80 4 client.16392 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:36:52.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.237+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/1957601828 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5635842ff050 con 0x56358445c9d0 2023-07-18T03:36:52.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.246+0000 7f7bdaffd700 1 --2- 172.21.15.40:0/1957601828 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56358445c9d0 0x56358445ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:36:52.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.252+0000 7f7bdaffd700 1 --2- 172.21.15.40:0/1957601828 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56358445c9d0 0x56358445ede0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9c9dff581f8293a server_cookie=15f23c6d4cf1d851 in_seq=0 out_seq=0 2023-07-18T03:36:52.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.263+0000 7f7bbeff5700 10 client.16392 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:52.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.264+0000 7f7bbeff5700 1 -- 172.21.15.40:0/1957601828 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5635842ff050 con 0x56358445c9d0 2023-07-18T03:36:52.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.264+0000 7f7bbeff5700 10 client.16392 handle_command_reply: tid=0 2023-07-18T03:36:52.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.264+0000 7f7bed7b3b80 1 client.16392 shutdown 2023-07-18T03:36:52.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.269+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/1957601828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f3c70 msgr2=0x5635843ffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:52.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.269+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/1957601828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f3c70 0x5635843ffa70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x5635843e9fd0 tx=0x7f7bcc02a680 comp rx=0 tx=0).stop 2023-07-18T03:36:52.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.275+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/1957601828 shutdown_connections 2023-07-18T03:36:52.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.275+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/1957601828 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x56358445c9d0 0x56358445ede0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:52.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.275+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/1957601828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5635843f3c70 0x5635843ffa70 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:52.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.275+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/1957601828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5635843f24d0 0x563584402fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:52.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.275+0000 7f7bed7b3b80 1 --2- 172.21.15.40:0/1957601828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5635843f2e20 0x563584403510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:52.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.275+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/1957601828 >> 172.21.15.40:0/1957601828 conn(0x5635843e7da0 msgr2=0x5635843fae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:52.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.275+0000 7f7bbeff5700 0 client.16392 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:36:52.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.275+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/1957601828 shutdown_connections 2023-07-18T03:36:52.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.275+0000 7f7bed7b3b80 1 -- 172.21.15.40:0/1957601828 wait complete. 2023-07-18T03:36:52.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.276+0000 7f7bed7b3b80 20 client.16392 trim_cache size 0 max 16384 2023-07-18T03:36:52.275 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:36:52.275 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-18T03:36:52.275 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:36:52.276 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:36:52.276 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:36:52.276 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:36:52.276 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:36:52.277 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:36:52.277 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:36:52.277 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:36:52.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.299+0000 7f7be162d700 1 -- 172.21.15.40:0/2422400247 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7bc40a1350 msgr2=0x7f7bc40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:52.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.299+0000 7f7be162d700 1 --2- 172.21.15.40:0/2422400247 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7bc40a1350 0x7f7bc40a3800 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7bd40062a0 tx=0x7f7bd4004ad0 comp rx=0 tx=0).stop 2023-07-18T03:36:52.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.299+0000 7f7be162d700 1 -- 172.21.15.40:0/2422400247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bdc071830 msgr2=0x7f7bdc1a4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:36:52.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.299+0000 7f7be162d700 1 --2- 172.21.15.40:0/2422400247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bdc071830 0x7f7bdc1a4b60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000d980 tx=0x7f7bd000dd40 comp rx=0 tx=0).stop 2023-07-18T03:36:52.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.299+0000 7f7be162d700 1 -- 172.21.15.40:0/2422400247 shutdown_connections 2023-07-18T03:36:52.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.299+0000 7f7be162d700 1 --2- 172.21.15.40:0/2422400247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bdc10bd60 0x7f7bdc1abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:52.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.299+0000 7f7be162d700 1 --2- 172.21.15.40:0/2422400247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bdc070ee0 0x7f7bdc1a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:52.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.299+0000 7f7be162d700 1 --2- 172.21.15.40:0/2422400247 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7bc40a1350 0x7f7bc40a3800 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:52.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.299+0000 7f7be162d700 1 --2- 172.21.15.40:0/2422400247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7bdc071830 0x7f7bdc1a4b60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:36:52.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.299+0000 7f7be162d700 1 -- 172.21.15.40:0/2422400247 >> 172.21.15.40:0/2422400247 conn(0x7f7bdc06ca90 msgr2=0x7f7bdc06e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:36:52.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.310+0000 7f7be162d700 1 -- 172.21.15.40:0/2422400247 shutdown_connections 2023-07-18T03:36:52.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:36:52.310+0000 7f7be162d700 1 -- 172.21.15.40:0/2422400247 wait complete. 2023-07-18T03:36:52.329 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:36:52.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:52 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:36:52.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:52 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:36:52.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:52 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:36:52.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:52 smithi040 ceph-mon[96439]: pgmap v474: 97 pgs: 97 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 509 op/s 2023-07-18T03:36:52.580 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:52 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:36:52.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:52 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:36:52.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:52 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:36:52.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:52 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:36:52.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:52 smithi134 ceph-mon[98216]: pgmap v474: 97 pgs: 97 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 509 op/s 2023-07-18T03:36:52.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:52 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:36:52.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:52 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:36:52.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:52 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:36:52.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:52 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:36:52.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:52 smithi064 ceph-mon[98059]: pgmap v474: 97 pgs: 97 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 509 op/s 2023-07-18T03:36:52.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:52 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:36:53.208 INFO:tasks.workunit.client.0.smithi040.stdout: 361 2115475 612 1507783 540 698389 833 2023-07-18T03:36:55.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:55 smithi134 ceph-mon[98216]: pgmap v475: 97 pgs: 97 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 511 op/s 2023-07-18T03:36:55.584 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:55 smithi040 ceph-mon[96439]: pgmap v475: 97 pgs: 97 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 511 op/s 2023-07-18T03:36:55.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:55 smithi064 ceph-mon[98059]: pgmap v475: 97 pgs: 97 active+clean; 1.3 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 511 op/s 2023-07-18T03:36:57.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:57 smithi134 ceph-mon[98216]: pgmap v476: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 529 op/s 2023-07-18T03:36:57.581 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:57 smithi040 ceph-mon[96439]: pgmap v476: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 529 op/s 2023-07-18T03:36:57.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:57 smithi064 ceph-mon[98059]: pgmap v476: 97 pgs: 97 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 529 op/s 2023-07-18T03:36:59.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:36:59 smithi134 ceph-mon[98216]: pgmap v477: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.4 MiB/s wr, 601 op/s 2023-07-18T03:36:59.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:36:59 smithi040 ceph-mon[96439]: pgmap v477: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.4 MiB/s wr, 601 op/s 2023-07-18T03:36:59.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:36:59 smithi064 ceph-mon[98059]: pgmap v477: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.4 MiB/s wr, 601 op/s 2023-07-18T03:37:01.589 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:01 smithi040 ceph-mon[96439]: pgmap v478: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 512 op/s 2023-07-18T03:37:01.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:01 smithi064 ceph-mon[98059]: pgmap v478: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 512 op/s 2023-07-18T03:37:01.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:01 smithi134 ceph-mon[98216]: pgmap v478: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 512 op/s 2023-07-18T03:37:02.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:02 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:37:02.836 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:02 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:37:02.849 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:02 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:37:03.211 INFO:tasks.workunit.client.0.smithi040.stdout: 369 1610773 542 1149790 516 536603 1096 2023-07-18T03:37:03.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:03 smithi134 ceph-mon[98216]: pgmap v479: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 618 op/s 2023-07-18T03:37:03.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:03 smithi040 ceph-mon[96439]: pgmap v479: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 618 op/s 2023-07-18T03:37:03.849 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:03 smithi064 ceph-mon[98059]: pgmap v479: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 618 op/s 2023-07-18T03:37:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.492+0000 7fccb2ce6700 1 -- 172.21.15.40:0/3970620593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccac0ffc10 msgr2=0x7fccac100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.492+0000 7fccb2ce6700 1 --2- 172.21.15.40:0/3970620593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccac0ffc10 0x7fccac100060 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009990 tx=0x7fcc98009ca0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.492+0000 7fccb2ce6700 1 -- 172.21.15.40:0/3970620593 shutdown_connections 2023-07-18T03:37:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.492+0000 7fccb2ce6700 1 --2- 172.21.15.40:0/3970620593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccac1005a0 0x7fccac106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.492+0000 7fccb2ce6700 1 --2- 172.21.15.40:0/3970620593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccac0fea20 0x7fccac0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.492+0000 7fccb2ce6700 1 --2- 172.21.15.40:0/3970620593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccac0ffc10 0x7fccac100060 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.492+0000 7fccb2ce6700 1 -- 172.21.15.40:0/3970620593 >> 172.21.15.40:0/3970620593 conn(0x7fccac0fa020 msgr2=0x7fccac0fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.492+0000 7fccb2ce6700 1 -- 172.21.15.40:0/3970620593 shutdown_connections 2023-07-18T03:37:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.492+0000 7fccb2ce6700 1 -- 172.21.15.40:0/3970620593 wait complete. 2023-07-18T03:37:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.493+0000 7fccb2ce6700 1 Processor -- start 2023-07-18T03:37:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.493+0000 7fccb2ce6700 1 -- start start 2023-07-18T03:37:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.493+0000 7fccb2ce6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccac0fea20 0x7fccac197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.493+0000 7fccb2ce6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccac0ffc10 0x7fccac1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.493+0000 7fccb2ce6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccac1005a0 0x7fccac198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.493+0000 7fccb2ce6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac199590 con 0x7fccac0fea20 2023-07-18T03:37:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.493+0000 7fccb2ce6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac199700 con 0x7fccac0ffc10 2023-07-18T03:37:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.493+0000 7fccb2ce6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac1998c0 con 0x7fccac1005a0 2023-07-18T03:37:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.493+0000 7fccb0a82700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccac0fea20 0x7fccac197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.493+0000 7fccb0a82700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccac0fea20 0x7fccac197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60404/0 (socket says 172.21.15.40:60404) 2023-07-18T03:37:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.493+0000 7fccb0a82700 1 -- 172.21.15.40:0/2478022316 learned_addr learned my addr 172.21.15.40:0/2478022316 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.493+0000 7fccabfff700 1 --2- 172.21.15.40:0/2478022316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccac0ffc10 0x7fccac1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.493+0000 7fccb1283700 1 --2- 172.21.15.40:0/2478022316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccac1005a0 0x7fccac198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.494+0000 7fccb0a82700 1 -- 172.21.15.40:0/2478022316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccac1005a0 msgr2=0x7fccac198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.494+0000 7fccb0a82700 1 --2- 172.21.15.40:0/2478022316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccac1005a0 0x7fccac198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.494+0000 7fccb0a82700 1 -- 172.21.15.40:0/2478022316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccac0ffc10 msgr2=0x7fccac1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.494+0000 7fccb0a82700 1 --2- 172.21.15.40:0/2478022316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccac0ffc10 0x7fccac1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.494+0000 7fccb0a82700 1 -- 172.21.15.40:0/2478022316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc980095f0 con 0x7fccac0fea20 2023-07-18T03:37:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.494+0000 7fccb0a82700 1 --2- 172.21.15.40:0/2478022316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccac0fea20 0x7fccac197b70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcca000e970 tx=0x7fcca000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.494+0000 7fcca9ffb700 1 -- 172.21.15.40:0/2478022316 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0009790 con 0x7fccac0fea20 2023-07-18T03:37:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.494+0000 7fcca9ffb700 1 -- 172.21.15.40:0/2478022316 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcca000c7e0 con 0x7fccac0fea20 2023-07-18T03:37:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.494+0000 7fccb2ce6700 1 -- 172.21.15.40:0/2478022316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccac199ba0 con 0x7fccac0fea20 2023-07-18T03:37:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.494+0000 7fcca9ffb700 1 -- 172.21.15.40:0/2478022316 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0003c70 con 0x7fccac0fea20 2023-07-18T03:37:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.494+0000 7fccb2ce6700 1 -- 172.21.15.40:0/2478022316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccac1a0360 con 0x7fccac0fea20 2023-07-18T03:37:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.496+0000 7fcca9ffb700 1 -- 172.21.15.40:0/2478022316 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcca000cc50 con 0x7fccac0fea20 2023-07-18T03:37:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.497+0000 7fcca9ffb700 1 --2- 172.21.15.40:0/2478022316 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcc9c0a1150 0x7fcc9c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.497+0000 7fcca9ffb700 1 -- 172.21.15.40:0/2478022316 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcca0014070 con 0x7fccac0fea20 2023-07-18T03:37:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.497+0000 7fccabfff700 1 --2- 172.21.15.40:0/2478022316 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcc9c0a1150 0x7fcc9c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.497+0000 7fccabfff700 1 --2- 172.21.15.40:0/2478022316 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcc9c0a1150 0x7fcc9c0a3600 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800b2e0 tx=0x7fcc98005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccbee6cb80 1 Processor -- start 2023-07-18T03:37:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccbee6cb80 1 -- start start 2023-07-18T03:37:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccbee6cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df0186b0 0x55b5df1bc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccbee6cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bc840 0x55b5df1bd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccbee6cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df1bd6e0 0x55b5df1bdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccbee6cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5df0c1c50 con 0x55b5df1bc840 2023-07-18T03:37:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccbee6cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5df1149e0 con 0x55b5df1bd6e0 2023-07-18T03:37:04.502 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccbee6cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5df10de70 con 0x55b5df0186b0 2023-07-18T03:37:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccabfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bc840 0x55b5df1bd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccabfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bc840 0x55b5df1bd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60414/0 (socket says 172.21.15.40:60414) 2023-07-18T03:37:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccabfff700 1 -- 172.21.15.40:0/3979132548 learned_addr learned my addr 172.21.15.40:0/3979132548 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccb1283700 1 --2- 172.21.15.40:0/3979132548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df1bd6e0 0x55b5df1bdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccb0a82700 1 --2- 172.21.15.40:0/3979132548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df0186b0 0x55b5df1bc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccabfff700 1 -- 172.21.15.40:0/3979132548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df0186b0 msgr2=0x55b5df1bc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccabfff700 1 --2- 172.21.15.40:0/3979132548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df0186b0 0x55b5df1bc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccabfff700 1 -- 172.21.15.40:0/3979132548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df1bd6e0 msgr2=0x55b5df1bdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccabfff700 1 --2- 172.21.15.40:0/3979132548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df1bd6e0 0x55b5df1bdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccabfff700 1 -- 172.21.15.40:0/3979132548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5def3bd80 con 0x55b5df1bc840 2023-07-18T03:37:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.498+0000 7fccb1283700 1 --2- 172.21.15.40:0/3979132548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df1bd6e0 0x55b5df1bdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.499+0000 7fccabfff700 1 --2- 172.21.15.40:0/3979132548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bc840 0x55b5df1bd1a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fcc98015000 tx=0x7fcc98020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca62ea9a9ea8403f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.499+0000 7fcc96ffd700 1 -- 172.21.15.40:0/3979132548 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9800f660 con 0x55b5df1bc840 2023-07-18T03:37:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.499+0000 7fcc96ffd700 1 -- 172.21.15.40:0/3979132548 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc9801e040 con 0x55b5df1bc840 2023-07-18T03:37:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.499+0000 7fcc96ffd700 1 -- 172.21.15.40:0/3979132548 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9802fae0 con 0x55b5df1bc840 2023-07-18T03:37:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.499+0000 7fccbee6cb80 1 -- 172.21.15.40:0/3979132548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bc840 msgr2=0x55b5df1bd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.499+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/3979132548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bc840 0x55b5df1bd1a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fcc98015000 tx=0x7fcc98020000 comp rx=0 tx=0).stop 2023-07-18T03:37:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.499+0000 7fccbee6cb80 1 -- 172.21.15.40:0/3979132548 shutdown_connections 2023-07-18T03:37:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.499+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/3979132548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df1bd6e0 0x55b5df1bdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.499+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/3979132548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df0186b0 0x55b5df1bc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.499+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/3979132548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bc840 0x55b5df1bd1a0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.499+0000 7fccbee6cb80 1 -- 172.21.15.40:0/3979132548 >> 172.21.15.40:0/3979132548 conn(0x55b5df1b1be0 msgr2=0x55b5df1b6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.499+0000 7fccbee6cb80 1 -- 172.21.15.40:0/3979132548 shutdown_connections 2023-07-18T03:37:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.499+0000 7fccbee6cb80 1 -- 172.21.15.40:0/3979132548 wait complete. 2023-07-18T03:37:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.500+0000 7fccbee6cb80 1 -- start start 2023-07-18T03:37:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.500+0000 7fccbee6cb80 1 Processor -- start 2023-07-18T03:37:04.506 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.500+0000 7fccbee6cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df0186b0 0x55b5df1c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.500+0000 7fccbee6cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df1bc840 0x55b5df1c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.500+0000 7fccbee6cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bd6e0 0x55b5df1ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.500+0000 7fccbee6cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5df0ada60 con 0x55b5df1bd6e0 2023-07-18T03:37:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.500+0000 7fccbee6cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5df0ad760 con 0x55b5df1bc840 2023-07-18T03:37:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.500+0000 7fccbee6cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5def918c0 con 0x55b5df0186b0 2023-07-18T03:37:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.500+0000 7fccbee6cb80 10 client.0 _ll_register_callbacks cb 0x55b5df0ab700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.500+0000 7fccb1283700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bd6e0 0x55b5df1ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.500+0000 7fccabfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df1bc840 0x55b5df1c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.500+0000 7fccb0a82700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df0186b0 0x55b5df1c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.500+0000 7fccb1283700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bd6e0 0x55b5df1ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60422/0 (socket says 172.21.15.40:60422) 2023-07-18T03:37:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.500+0000 7fccabfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df1bc840 0x55b5df1c5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43746/0 (socket says 172.21.15.40:43746) 2023-07-18T03:37:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.500+0000 7fccb1283700 1 -- 172.21.15.40:0/4077208952 learned_addr learned my addr 172.21.15.40:0/4077208952 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.501+0000 7fccb0a82700 1 -- 172.21.15.40:0/4077208952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df1bc840 msgr2=0x55b5df1c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.501+0000 7fccb0a82700 1 --2- 172.21.15.40:0/4077208952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df1bc840 0x55b5df1c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.501+0000 7fccb0a82700 1 -- 172.21.15.40:0/4077208952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bd6e0 msgr2=0x55b5df1ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.501+0000 7fccb0a82700 1 --2- 172.21.15.40:0/4077208952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bd6e0 0x55b5df1ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.501+0000 7fccb0a82700 1 -- 172.21.15.40:0/4077208952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca4011230 con 0x55b5df0186b0 2023-07-18T03:37:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.501+0000 7fccb1283700 1 --2- 172.21.15.40:0/4077208952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bd6e0 0x55b5df1ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.501+0000 7fccb0a82700 1 --2- 172.21.15.40:0/4077208952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df0186b0 0x55b5df1c5820 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcca0099bd0 tx=0x7fcca0003dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=334392a4e5938a94 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.501+0000 7fcc8bfff700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:37:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.501+0000 7fcc8bfff700 1 -- 172.21.15.40:0/4077208952 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca009a330 con 0x55b5df0186b0 2023-07-18T03:37:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.501+0000 7fcc8bfff700 1 -- 172.21.15.40:0/4077208952 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcca00a2320 con 0x55b5df0186b0 2023-07-18T03:37:04.509 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.501+0000 7fccbee6cb80 1 -- 172.21.15.40:0/4077208952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5df067490 con 0x55b5df0186b0 2023-07-18T03:37:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.501+0000 7fcc8bfff700 1 -- 172.21.15.40:0/4077208952 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcca00b8250 con 0x55b5df0186b0 2023-07-18T03:37:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.501+0000 7fccbee6cb80 10 client.26074 fetch_fsmap learned FSMap version 45 2023-07-18T03:37:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.501+0000 7fccbee6cb80 1 -- 172.21.15.40:0/4077208952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7fccac107790 con 0x55b5df0186b0 2023-07-18T03:37:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.502+0000 7fcc8bfff700 1 -- 172.21.15.40:0/4077208952 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b5df067490 con 0x55b5df0186b0 2023-07-18T03:37:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.502+0000 7fcc8bfff700 1 -- 172.21.15.40:0/4077208952 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 45) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fcca00adbd0 con 0x55b5df0186b0 2023-07-18T03:37:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.502+0000 7fccbee6cb80 10 client.26074 fetch_fsmap finished waiting for FSMap version 45 2023-07-18T03:37:04.510 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.502+0000 7fccbee6cb80 10 client.26074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:37:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.502+0000 7fccbee6cb80 20 client.26074 populate_metadata read hostname 'smithi040' 2023-07-18T03:37:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.502+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/4077208952 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55b5df1d1590 0x55b5df1d39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.502+0000 7fccbee6cb80 4 client.26074 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:37:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.502+0000 7fccbee6cb80 1 -- 172.21.15.40:0/4077208952 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5df106e50 con 0x55b5df1d1590 2023-07-18T03:37:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.502+0000 7fccabfff700 1 --2- 172.21.15.40:0/4077208952 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55b5df1d1590 0x55b5df1d39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.502+0000 7fcc8bfff700 10 client.26074 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.502+0000 7fccabfff700 1 --2- 172.21.15.40:0/4077208952 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55b5df1d1590 0x55b5df1d39a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f72099779df852ab server_cookie=de264c7ed03a8e12 in_seq=0 out_seq=0 2023-07-18T03:37:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.503+0000 7fcc8bfff700 1 -- 172.21.15.40:0/4077208952 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b5df106e50 con 0x55b5df1d1590 2023-07-18T03:37:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.503+0000 7fcc8bfff700 10 client.26074 handle_command_reply: tid=0 2023-07-18T03:37:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.503+0000 7fccbee6cb80 1 client.26074 shutdown 2023-07-18T03:37:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.503+0000 7fccbee6cb80 1 -- 172.21.15.40:0/4077208952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df0186b0 msgr2=0x55b5df1c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.503+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/4077208952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df0186b0 0x55b5df1c5820 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcca0099bd0 tx=0x7fcca0003dd0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.503+0000 7fccbee6cb80 1 -- 172.21.15.40:0/4077208952 shutdown_connections 2023-07-18T03:37:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.503+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/4077208952 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55b5df1d1590 0x55b5df1d39a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.512 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.503+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/4077208952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bd6e0 0x55b5df1ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.503+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/4077208952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df0186b0 0x55b5df1c5820 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.503+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/4077208952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df1bc840 0x55b5df1c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.503+0000 7fcc8bfff700 0 client.26074 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.503+0000 7fccbee6cb80 1 -- 172.21.15.40:0/4077208952 >> 172.21.15.40:0/4077208952 conn(0x55b5df1b1be0 msgr2=0x55b5df1b64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.503+0000 7fccbee6cb80 1 -- 172.21.15.40:0/4077208952 shutdown_connections 2023-07-18T03:37:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.504+0000 7fccbee6cb80 1 -- 172.21.15.40:0/4077208952 wait complete. 2023-07-18T03:37:04.513 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.504+0000 7fccbee6cb80 20 client.26074 trim_cache size 0 max 16384 2023-07-18T03:37:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.518+0000 7fccbee6cb80 1 Processor -- start 2023-07-18T03:37:04.518 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.518+0000 7fccbee6cb80 1 -- start start 2023-07-18T03:37:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.518+0000 7fccbee6cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df0186b0 0x55b5df1c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.518+0000 7fccbee6cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bc840 0x55b5df1ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.518+0000 7fccbee6cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df1bd6e0 0x55b5df1c0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.518+0000 7fccbee6cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5df1c53f0 con 0x55b5df1bc840 2023-07-18T03:37:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.518+0000 7fccbee6cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5df1c4cd0 con 0x55b5df0186b0 2023-07-18T03:37:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.518+0000 7fccbee6cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5df1bcec0 con 0x55b5df1bd6e0 2023-07-18T03:37:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.519+0000 7fccabfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bc840 0x55b5df1ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.519 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.519+0000 7fccabfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bc840 0x55b5df1ccd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60428/0 (socket says 172.21.15.40:60428) 2023-07-18T03:37:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.519+0000 7fccabfff700 1 -- 172.21.15.40:0/3217690579 learned_addr learned my addr 172.21.15.40:0/3217690579 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.519+0000 7fccb0a82700 1 --2- 172.21.15.40:0/3217690579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df0186b0 0x55b5df1c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.519+0000 7fccb1283700 1 --2- 172.21.15.40:0/3217690579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df1bd6e0 0x55b5df1c0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.519+0000 7fccabfff700 1 -- 172.21.15.40:0/3217690579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df1bd6e0 msgr2=0x55b5df1c0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.519+0000 7fccabfff700 1 --2- 172.21.15.40:0/3217690579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df1bd6e0 0x55b5df1c0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.519+0000 7fccabfff700 1 -- 172.21.15.40:0/3217690579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df0186b0 msgr2=0x55b5df1c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:04.520 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.519+0000 7fccabfff700 1 --2- 172.21.15.40:0/3217690579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df0186b0 0x55b5df1c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.519+0000 7fccabfff700 1 -- 172.21.15.40:0/3217690579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5def24c00 con 0x55b5df1bc840 2023-07-18T03:37:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.519+0000 7fccabfff700 1 --2- 172.21.15.40:0/3217690579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bc840 0x55b5df1ccd60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009990 tx=0x7fcc98027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f36b923e6914eda server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.519+0000 7fcc977fe700 1 -- 172.21.15.40:0/3217690579 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9802fb60 con 0x55b5df1bc840 2023-07-18T03:37:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.519+0000 7fcc977fe700 1 -- 172.21.15.40:0/3217690579 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc9803eb90 con 0x55b5df1bc840 2023-07-18T03:37:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.519+0000 7fcc977fe700 1 -- 172.21.15.40:0/3217690579 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9803a6c0 con 0x55b5df1bc840 2023-07-18T03:37:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.520+0000 7fccbee6cb80 1 -- 172.21.15.40:0/3217690579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bc840 msgr2=0x55b5df1ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.520+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/3217690579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bc840 0x55b5df1ccd60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009990 tx=0x7fcc98027040 comp rx=0 tx=0).stop 2023-07-18T03:37:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.520+0000 7fccbee6cb80 1 -- 172.21.15.40:0/3217690579 shutdown_connections 2023-07-18T03:37:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.520+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/3217690579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df1bd6e0 0x55b5df1c0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.520+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/3217690579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df0186b0 0x55b5df1c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.520+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/3217690579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bc840 0x55b5df1ccd60 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.520+0000 7fccbee6cb80 1 -- 172.21.15.40:0/3217690579 >> 172.21.15.40:0/3217690579 conn(0x55b5df1b1be0 msgr2=0x55b5df1d8300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.520+0000 7fccbee6cb80 1 -- 172.21.15.40:0/3217690579 shutdown_connections 2023-07-18T03:37:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.520+0000 7fccbee6cb80 1 -- 172.21.15.40:0/3217690579 wait complete. 2023-07-18T03:37:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.520+0000 7fccbee6cb80 1 -- start start 2023-07-18T03:37:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.520+0000 7fccbee6cb80 1 Processor -- start 2023-07-18T03:37:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccbee6cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df0186b0 0x55b5df1cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccbee6cb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df1bc840 0x55b5df221420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccbee6cb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bd6e0 0x55b5df223970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccbee6cb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5df223eb0 con 0x55b5df1bd6e0 2023-07-18T03:37:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccbee6cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5df223ff0 con 0x55b5df0186b0 2023-07-18T03:37:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccbee6cb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5df224180 con 0x55b5df1bc840 2023-07-18T03:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccbee6cb80 10 client.0 _ll_register_callbacks cb 0x55b5df0ab700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccb0a82700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df0186b0 0x55b5df1cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccb1283700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bd6e0 0x55b5df223970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccabfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df1bc840 0x55b5df221420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccb0a82700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df0186b0 0x55b5df1cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43758/0 (socket says 172.21.15.40:43758) 2023-07-18T03:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccb0a82700 1 -- 172.21.15.40:0/719703780 learned_addr learned my addr 172.21.15.40:0/719703780 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:04.524 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccb0a82700 1 -- 172.21.15.40:0/719703780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df1bc840 msgr2=0x55b5df221420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccb0a82700 1 --2- 172.21.15.40:0/719703780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df1bc840 0x55b5df221420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccb0a82700 1 -- 172.21.15.40:0/719703780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bd6e0 msgr2=0x55b5df223970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccb0a82700 1 --2- 172.21.15.40:0/719703780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bd6e0 0x55b5df223970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccb0a82700 1 -- 172.21.15.40:0/719703780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5def3bd80 con 0x55b5df0186b0 2023-07-18T03:37:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccb1283700 1 --2- 172.21.15.40:0/719703780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bd6e0 0x55b5df223970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:04.525 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccb0a82700 1 --2- 172.21.15.40:0/719703780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df0186b0 0x55b5df1cad20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcca0003750 tx=0x7fcca009e080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d00a72c57bdd06b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fcc8bfff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:37:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fcc8bfff700 1 -- 172.21.15.40:0/719703780 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca00ca490 con 0x55b5df0186b0 2023-07-18T03:37:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fccbee6cb80 1 -- 172.21.15.40:0/719703780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc9801e040 con 0x55b5df0186b0 2023-07-18T03:37:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.521+0000 7fcc8bfff700 1 -- 172.21.15.40:0/719703780 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcca00a2320 con 0x55b5df0186b0 2023-07-18T03:37:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.522+0000 7fcc8bfff700 1 -- 172.21.15.40:0/719703780 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcca00a1f60 con 0x55b5df0186b0 2023-07-18T03:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.522+0000 7fccbee6cb80 10 client.26066 fetch_fsmap learned FSMap version 45 2023-07-18T03:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.522+0000 7fccbee6cb80 1 -- 172.21.15.40:0/719703780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55b5def621f0 con 0x55b5df0186b0 2023-07-18T03:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.522+0000 7fcc8bfff700 1 -- 172.21.15.40:0/719703780 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcca007e070 con 0x55b5df0186b0 2023-07-18T03:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.522+0000 7fcc8bfff700 1 -- 172.21.15.40:0/719703780 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fcca009a130 con 0x55b5df0186b0 2023-07-18T03:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.522+0000 7fccbee6cb80 10 client.26066 fetch_fsmap finished waiting for FSMap version 45 2023-07-18T03:37:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.522+0000 7fccbee6cb80 10 client.26066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:37:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.522+0000 7fccbee6cb80 20 client.26066 populate_metadata read hostname 'smithi040' 2023-07-18T03:37:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.522+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/719703780 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55b5df2257a0 0x55b5df227bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.522+0000 7fccbee6cb80 4 client.26066 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:37:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.522+0000 7fccbee6cb80 1 -- 172.21.15.40:0/719703780 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b5df0c8ff0 con 0x55b5df2257a0 2023-07-18T03:37:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.522+0000 7fccabfff700 1 --2- 172.21.15.40:0/719703780 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55b5df2257a0 0x55b5df227bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.523+0000 7fcc8bfff700 10 client.26066 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.523+0000 7fccabfff700 1 --2- 172.21.15.40:0/719703780 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55b5df2257a0 0x55b5df227bb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44b4edb3f1f11f1b server_cookie=a4170c00b9d06f26 in_seq=0 out_seq=0 2023-07-18T03:37:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.653+0000 7fcc8bfff700 1 -- 172.21.15.40:0/719703780 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b5df0c8ff0 con 0x55b5df2257a0 2023-07-18T03:37:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.653+0000 7fcc8bfff700 10 client.26066 handle_command_reply: tid=0 2023-07-18T03:37:04.650 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.654+0000 7fccbee6cb80 1 client.26066 shutdown 2023-07-18T03:37:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.654+0000 7fccbee6cb80 1 -- 172.21.15.40:0/719703780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df0186b0 msgr2=0x55b5df1cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.654+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/719703780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df0186b0 0x55b5df1cad20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcca0003750 tx=0x7fcca009e080 comp rx=0 tx=0).stop 2023-07-18T03:37:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.654+0000 7fccbee6cb80 1 -- 172.21.15.40:0/719703780 shutdown_connections 2023-07-18T03:37:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.654+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/719703780 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55b5df2257a0 0x55b5df227bb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.654+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/719703780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b5df1bd6e0 0x55b5df223970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.654+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/719703780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5df0186b0 0x55b5df1cad20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.654+0000 7fccbee6cb80 1 --2- 172.21.15.40:0/719703780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b5df1bc840 0x55b5df221420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.654+0000 7fcc8bfff700 0 client.26066 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.654+0000 7fccbee6cb80 1 -- 172.21.15.40:0/719703780 >> 172.21.15.40:0/719703780 conn(0x55b5df1b1be0 msgr2=0x55b5df1c44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.654+0000 7fccbee6cb80 1 -- 172.21.15.40:0/719703780 shutdown_connections 2023-07-18T03:37:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.654+0000 7fccbee6cb80 1 -- 172.21.15.40:0/719703780 wait complete. 2023-07-18T03:37:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.654+0000 7fccbee6cb80 20 client.26066 trim_cache size 0 max 16384 2023-07-18T03:37:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:37:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:37:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:37:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:37:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.655+0000 7fccb2ce6700 1 -- 172.21.15.40:0/2478022316 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcc9c0a1150 msgr2=0x7fcc9c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.655+0000 7fccb2ce6700 1 --2- 172.21.15.40:0/2478022316 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcc9c0a1150 0x7fcc9c0a3600 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800b2e0 tx=0x7fcc98005fb0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.655+0000 7fccb2ce6700 1 -- 172.21.15.40:0/2478022316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccac0fea20 msgr2=0x7fccac197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.655+0000 7fccb2ce6700 1 --2- 172.21.15.40:0/2478022316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccac0fea20 0x7fccac197b70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcca000e970 tx=0x7fcca000ec80 comp rx=0 tx=0).stop 2023-07-18T03:37:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.655+0000 7fccb2ce6700 1 -- 172.21.15.40:0/2478022316 shutdown_connections 2023-07-18T03:37:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.655+0000 7fccb2ce6700 1 --2- 172.21.15.40:0/2478022316 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcc9c0a1150 0x7fcc9c0a3600 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.655+0000 7fccb2ce6700 1 --2- 172.21.15.40:0/2478022316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fccac1005a0 0x7fccac198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.655+0000 7fccb2ce6700 1 --2- 172.21.15.40:0/2478022316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccac0fea20 0x7fccac197b70 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.655+0000 7fccb2ce6700 1 --2- 172.21.15.40:0/2478022316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccac0ffc10 0x7fccac1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.655+0000 7fccb2ce6700 1 -- 172.21.15.40:0/2478022316 >> 172.21.15.40:0/2478022316 conn(0x7fccac0fa020 msgr2=0x7fccac0684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.655+0000 7fccb2ce6700 1 -- 172.21.15.40:0/2478022316 shutdown_connections 2023-07-18T03:37:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:04.655+0000 7fccb2ce6700 1 -- 172.21.15.40:0/2478022316 wait complete. 2023-07-18T03:37:05.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:05 smithi134 ceph-mon[98216]: pgmap v480: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.5 MiB/s wr, 395 op/s 2023-07-18T03:37:05.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:05 smithi040 ceph-mon[96439]: pgmap v480: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.5 MiB/s wr, 395 op/s 2023-07-18T03:37:05.849 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:05 smithi064 ceph-mon[98059]: pgmap v480: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.5 MiB/s wr, 395 op/s 2023-07-18T03:37:07.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:07 smithi134 ceph-mon[98216]: pgmap v481: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.3 MiB/s wr, 361 op/s 2023-07-18T03:37:07.816 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:07 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:37:07.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:07 smithi040 ceph-mon[96439]: pgmap v481: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.3 MiB/s wr, 361 op/s 2023-07-18T03:37:07.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:07 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:37:07.849 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:07 smithi064 ceph-mon[98059]: pgmap v481: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.3 MiB/s wr, 361 op/s 2023-07-18T03:37:07.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:07 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:37:09.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:09 smithi134 ceph-mon[98216]: pgmap v482: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 427 op/s 2023-07-18T03:37:09.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:09 smithi040 ceph-mon[96439]: pgmap v482: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 427 op/s 2023-07-18T03:37:09.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:09 smithi064 ceph-mon[98059]: pgmap v482: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 427 op/s 2023-07-18T03:37:11.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:11 smithi134 ceph-mon[98216]: pgmap v483: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 321 op/s 2023-07-18T03:37:11.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:11 smithi040 ceph-mon[96439]: pgmap v483: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 321 op/s 2023-07-18T03:37:11.849 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:11 smithi064 ceph-mon[98059]: pgmap v483: 97 pgs: 97 active+clean; 1.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 321 op/s 2023-07-18T03:37:12.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:12 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:37:12.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:12 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:37:12.849 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:12 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:37:13.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:13 smithi134 ceph-mon[98216]: pgmap v484: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 401 op/s 2023-07-18T03:37:13.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:13 smithi040 ceph-mon[96439]: pgmap v484: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 401 op/s 2023-07-18T03:37:13.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:13 smithi064 ceph-mon[98059]: pgmap v484: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.2 MiB/s wr, 401 op/s 2023-07-18T03:37:15.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:15 smithi134 ceph-mon[98216]: pgmap v485: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 4.0 MiB/s wr, 321 op/s 2023-07-18T03:37:15.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:15 smithi040 ceph-mon[96439]: pgmap v485: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 4.0 MiB/s wr, 321 op/s 2023-07-18T03:37:15.849 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:15 smithi064 ceph-mon[98059]: pgmap v485: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 4.0 MiB/s wr, 321 op/s 2023-07-18T03:37:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.777+0000 7efdee7ba700 1 -- 172.21.15.40:0/420415037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efde8100640 msgr2=0x7efde8106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.777+0000 7efdee7ba700 1 --2- 172.21.15.40:0/420415037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efde8100640 0x7efde8106ee0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7efde0009ec0 tx=0x7efde000b330 comp rx=0 tx=0).stop 2023-07-18T03:37:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.777+0000 7efdee7ba700 1 -- 172.21.15.40:0/420415037 shutdown_connections 2023-07-18T03:37:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.777+0000 7efdee7ba700 1 --2- 172.21.15.40:0/420415037 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efde8100640 0x7efde8106ee0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.777+0000 7efdee7ba700 1 --2- 172.21.15.40:0/420415037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efde80fea00 0x7efde80fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.777+0000 7efdee7ba700 1 --2- 172.21.15.40:0/420415037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde80ffc90 0x7efde8100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.777+0000 7efdee7ba700 1 -- 172.21.15.40:0/420415037 >> 172.21.15.40:0/420415037 conn(0x7efde80fa000 msgr2=0x7efde80fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.777+0000 7efdee7ba700 1 -- 172.21.15.40:0/420415037 shutdown_connections 2023-07-18T03:37:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.777+0000 7efdee7ba700 1 -- 172.21.15.40:0/420415037 wait complete. 2023-07-18T03:37:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.778+0000 7efdee7ba700 1 Processor -- start 2023-07-18T03:37:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.778+0000 7efdee7ba700 1 -- start start 2023-07-18T03:37:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.778+0000 7efdee7ba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efde80fea00 0x7efde8198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.778+0000 7efdee7ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efde80ffc90 0x7efde819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.778+0000 7efdee7ba700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde8100640 0x7efde819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.778+0000 7efdee7ba700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde819d5c0 con 0x7efde80fea00 2023-07-18T03:37:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.778+0000 7efdee7ba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde819d700 con 0x7efde80ffc90 2023-07-18T03:37:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.778+0000 7efdee7ba700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde819d8c0 con 0x7efde8100640 2023-07-18T03:37:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.778+0000 7efde7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efde80fea00 0x7efde8198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.778+0000 7efde7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efde80fea00 0x7efde8198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60444/0 (socket says 172.21.15.40:60444) 2023-07-18T03:37:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.778+0000 7efde77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efde80ffc90 0x7efde819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.778+0000 7efde7fff700 1 -- 172.21.15.40:0/1953237973 learned_addr learned my addr 172.21.15.40:0/1953237973 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.778+0000 7efdecd57700 1 --2- 172.21.15.40:0/1953237973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde8100640 0x7efde819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.779+0000 7efde77fe700 1 -- 172.21.15.40:0/1953237973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde8100640 msgr2=0x7efde819cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.779+0000 7efde77fe700 1 --2- 172.21.15.40:0/1953237973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde8100640 0x7efde819cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.779+0000 7efde77fe700 1 -- 172.21.15.40:0/1953237973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efde80fea00 msgr2=0x7efde8198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.779+0000 7efde77fe700 1 --2- 172.21.15.40:0/1953237973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efde80fea00 0x7efde8198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.779+0000 7efde77fe700 1 -- 172.21.15.40:0/1953237973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde0005e70 con 0x7efde80ffc90 2023-07-18T03:37:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.779+0000 7efde7fff700 1 --2- 172.21.15.40:0/1953237973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efde80fea00 0x7efde8198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.779+0000 7efde77fe700 1 --2- 172.21.15.40:0/1953237973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efde80ffc90 0x7efde819aaa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efddc00e9f0 tx=0x7efddc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.779+0000 7efde57fa700 1 -- 172.21.15.40:0/1953237973 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc00c9c0 con 0x7efde80ffc90 2023-07-18T03:37:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.779+0000 7efdee7ba700 1 -- 172.21.15.40:0/1953237973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde819dba0 con 0x7efde80ffc90 2023-07-18T03:37:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.779+0000 7efde57fa700 1 -- 172.21.15.40:0/1953237973 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efddc010430 con 0x7efde80ffc90 2023-07-18T03:37:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.779+0000 7efdee7ba700 1 -- 172.21.15.40:0/1953237973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde819e100 con 0x7efde80ffc90 2023-07-18T03:37:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.779+0000 7efde57fa700 1 -- 172.21.15.40:0/1953237973 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc018620 con 0x7efde80ffc90 2023-07-18T03:37:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.781+0000 7efde57fa700 1 -- 172.21.15.40:0/1953237973 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efddc010860 con 0x7efde80ffc90 2023-07-18T03:37:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.782+0000 7efde57fa700 1 --2- 172.21.15.40:0/1953237973 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efdd00a1150 0x7efdd00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.783+0000 7efde7fff700 1 --2- 172.21.15.40:0/1953237973 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efdd00a1150 0x7efdd00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.783+0000 7efde57fa700 1 -- 172.21.15.40:0/1953237973 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7efddc014070 con 0x7efde80ffc90 2023-07-18T03:37:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.783+0000 7efdfa940b80 1 Processor -- start 2023-07-18T03:37:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.783+0000 7efde7fff700 1 --2- 172.21.15.40:0/1953237973 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efdd00a1150 0x7efdd00a3600 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efde80683e0 tx=0x7efdd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.783+0000 7efdfa940b80 1 -- start start 2023-07-18T03:37:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.783+0000 7efdfa940b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576d606b0 0x55d576f04300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.783+0000 7efdfa940b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576f04840 0x55d576f051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.783+0000 7efdfa940b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576f056e0 0x55d576f05b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.783+0000 7efdfa940b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d576e09c50 con 0x55d576d606b0 2023-07-18T03:37:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.783+0000 7efdfa940b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d576e5c9e0 con 0x55d576f056e0 2023-07-18T03:37:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.783+0000 7efdfa940b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d576e55e70 con 0x55d576f04840 2023-07-18T03:37:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.783+0000 7efde7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576d606b0 0x55d576f04300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.783+0000 7efde7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576d606b0 0x55d576f04300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60452/0 (socket says 172.21.15.40:60452) 2023-07-18T03:37:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.783+0000 7efde7fff700 1 -- 172.21.15.40:0/3941949691 learned_addr learned my addr 172.21.15.40:0/3941949691 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.783+0000 7efde77fe700 1 --2- 172.21.15.40:0/3941949691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576f04840 0x55d576f051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.783+0000 7efdecd57700 1 --2- 172.21.15.40:0/3941949691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576f056e0 0x55d576f05b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.784+0000 7efde77fe700 1 -- 172.21.15.40:0/3941949691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576f056e0 msgr2=0x55d576f05b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.784+0000 7efde77fe700 1 --2- 172.21.15.40:0/3941949691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576f056e0 0x55d576f05b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.784+0000 7efde77fe700 1 -- 172.21.15.40:0/3941949691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576d606b0 msgr2=0x55d576f04300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.784+0000 7efde77fe700 1 --2- 172.21.15.40:0/3941949691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576d606b0 0x55d576f04300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.784+0000 7efde77fe700 1 -- 172.21.15.40:0/3941949691 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d576c83d80 con 0x55d576f04840 2023-07-18T03:37:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.784+0000 7efde77fe700 1 --2- 172.21.15.40:0/3941949691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576f04840 0x55d576f051a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efddc00c5b0 tx=0x7efddc0999d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aa26bed102e38f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.784+0000 7efdce7fc700 1 -- 172.21.15.40:0/3941949691 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc09de40 con 0x55d576f04840 2023-07-18T03:37:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.784+0000 7efdce7fc700 1 -- 172.21.15.40:0/3941949691 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efddc0a2260 con 0x55d576f04840 2023-07-18T03:37:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.784+0000 7efdce7fc700 1 -- 172.21.15.40:0/3941949691 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc0a1dd0 con 0x55d576f04840 2023-07-18T03:37:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.785+0000 7efdfa940b80 1 -- 172.21.15.40:0/3941949691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576f04840 msgr2=0x55d576f051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.785+0000 7efdfa940b80 1 --2- 172.21.15.40:0/3941949691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576f04840 0x55d576f051a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efddc00c5b0 tx=0x7efddc0999d0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.785+0000 7efdfa940b80 1 -- 172.21.15.40:0/3941949691 shutdown_connections 2023-07-18T03:37:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.785+0000 7efdfa940b80 1 --2- 172.21.15.40:0/3941949691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576f056e0 0x55d576f05b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.785+0000 7efdfa940b80 1 --2- 172.21.15.40:0/3941949691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576d606b0 0x55d576f04300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.785+0000 7efdfa940b80 1 --2- 172.21.15.40:0/3941949691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576f04840 0x55d576f051a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.785+0000 7efdfa940b80 1 -- 172.21.15.40:0/3941949691 >> 172.21.15.40:0/3941949691 conn(0x55d576ef9be0 msgr2=0x55d576efecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.785+0000 7efdfa940b80 1 -- 172.21.15.40:0/3941949691 shutdown_connections 2023-07-18T03:37:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.785+0000 7efdfa940b80 1 -- 172.21.15.40:0/3941949691 wait complete. 2023-07-18T03:37:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.785+0000 7efdfa940b80 1 -- start start 2023-07-18T03:37:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdfa940b80 1 Processor -- start 2023-07-18T03:37:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdfa940b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576d606b0 0x55d576f0d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdfa940b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576f04840 0x55d576f0dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdfa940b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f14de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdfa940b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d576df5a60 con 0x55d576f056e0 2023-07-18T03:37:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdfa940b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d576df5760 con 0x55d576f04840 2023-07-18T03:37:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdfa940b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d576cd98c0 con 0x55d576d606b0 2023-07-18T03:37:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdfa940b80 10 client.0 _ll_register_callbacks cb 0x55d576df3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdecd57700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f14de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdecd57700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f14de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60462/0 (socket says 172.21.15.40:60462) 2023-07-18T03:37:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdecd57700 1 -- 172.21.15.40:0/2170218717 learned_addr learned my addr 172.21.15.40:0/2170218717 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efde7fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576d606b0 0x55d576f0d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efde77fe700 1 --2- 172.21.15.40:0/2170218717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576f04840 0x55d576f0dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdecd57700 1 -- 172.21.15.40:0/2170218717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576d606b0 msgr2=0x55d576f0d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdecd57700 1 --2- 172.21.15.40:0/2170218717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576d606b0 0x55d576f0d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdecd57700 1 -- 172.21.15.40:0/2170218717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576f04840 msgr2=0x55d576f0dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdecd57700 1 --2- 172.21.15.40:0/2170218717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576f04840 0x55d576f0dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdecd57700 1 -- 172.21.15.40:0/2170218717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde819e100 con 0x55d576f056e0 2023-07-18T03:37:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdecd57700 1 --2- 172.21.15.40:0/2170218717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f14de0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efde00095a0 tx=0x7efde00095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25918a61d5e28e84 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.786+0000 7efdc77fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:37:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.787+0000 7efdc77fe700 1 -- 172.21.15.40:0/2170218717 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde000e040 con 0x55d576f056e0 2023-07-18T03:37:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.787+0000 7efdfa940b80 1 -- 172.21.15.40:0/2170218717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d576daf490 con 0x55d576f056e0 2023-07-18T03:37:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.787+0000 7efdc77fe700 1 -- 172.21.15.40:0/2170218717 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7efde0007a40 con 0x55d576f056e0 2023-07-18T03:37:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.787+0000 7efdc77fe700 1 -- 172.21.15.40:0/2170218717 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7efde0007c00 con 0x55d576f056e0 2023-07-18T03:37:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.787+0000 7efdfa940b80 10 client.16422 fetch_fsmap learned FSMap version 47 2023-07-18T03:37:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.787+0000 7efdfa940b80 1 -- 172.21.15.40:0/2170218717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55d576c6cc00 con 0x55d576f056e0 2023-07-18T03:37:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.787+0000 7efdc77fe700 1 -- 172.21.15.40:0/2170218717 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d576daf490 con 0x55d576f056e0 2023-07-18T03:37:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.787+0000 7efdc77fe700 1 -- 172.21.15.40:0/2170218717 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 47) v1 ==== 7522+0+0 (secure 0 0 0) 0x7efde000b050 con 0x55d576f056e0 2023-07-18T03:37:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.787+0000 7efdfa940b80 10 client.16422 fetch_fsmap finished waiting for FSMap version 47 2023-07-18T03:37:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.787+0000 7efdfa940b80 10 client.16422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:37:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.787+0000 7efdfa940b80 20 client.16422 populate_metadata read hostname 'smithi040' 2023-07-18T03:37:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.787+0000 7efdfa940b80 1 --2- 172.21.15.40:0/2170218717 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d576f19770 0x55d576f1bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.787+0000 7efdfa940b80 4 client.16422 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:37:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.787+0000 7efdfa940b80 1 -- 172.21.15.40:0/2170218717 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d576e4ee50 con 0x55d576f19770 2023-07-18T03:37:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.788+0000 7efde7fff700 1 --2- 172.21.15.40:0/2170218717 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d576f19770 0x55d576f1bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.788+0000 7efdc77fe700 10 client.16422 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.788+0000 7efde7fff700 1 --2- 172.21.15.40:0/2170218717 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d576f19770 0x55d576f1bb80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2bc95668afa5dd7 server_cookie=96d0755844509b53 in_seq=0 out_seq=0 2023-07-18T03:37:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.789+0000 7efdc77fe700 1 -- 172.21.15.40:0/2170218717 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d576e4ee50 con 0x55d576f19770 2023-07-18T03:37:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.789+0000 7efdc77fe700 10 client.16422 handle_command_reply: tid=0 2023-07-18T03:37:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.789+0000 7efdfa940b80 1 client.16422 shutdown 2023-07-18T03:37:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.789+0000 7efdfa940b80 1 -- 172.21.15.40:0/2170218717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 msgr2=0x55d576f14de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.789+0000 7efdfa940b80 1 --2- 172.21.15.40:0/2170218717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f14de0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efde00095a0 tx=0x7efde00095d0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.789+0000 7efdfa940b80 1 -- 172.21.15.40:0/2170218717 shutdown_connections 2023-07-18T03:37:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.789+0000 7efdfa940b80 1 --2- 172.21.15.40:0/2170218717 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d576f19770 0x55d576f1bb80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.789+0000 7efdfa940b80 1 --2- 172.21.15.40:0/2170218717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f14de0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.789+0000 7efdfa940b80 1 --2- 172.21.15.40:0/2170218717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576d606b0 0x55d576f0d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.789+0000 7efdc77fe700 0 client.16422 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.789+0000 7efdfa940b80 1 --2- 172.21.15.40:0/2170218717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576f04840 0x55d576f0dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.789+0000 7efdfa940b80 1 -- 172.21.15.40:0/2170218717 >> 172.21.15.40:0/2170218717 conn(0x55d576ef9be0 msgr2=0x55d576efe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.789+0000 7efdfa940b80 1 -- 172.21.15.40:0/2170218717 shutdown_connections 2023-07-18T03:37:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.789+0000 7efdfa940b80 1 -- 172.21.15.40:0/2170218717 wait complete. 2023-07-18T03:37:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.790+0000 7efdfa940b80 20 client.16422 trim_cache size 0 max 16384 2023-07-18T03:37:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdfa940b80 1 Processor -- start 2023-07-18T03:37:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdfa940b80 1 -- start start 2023-07-18T03:37:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdfa940b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576d606b0 0x55d576f0d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdfa940b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576f04840 0x55d576f14d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdfa940b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f0db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdfa940b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d576efbd00 con 0x55d576f056e0 2023-07-18T03:37:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdfa940b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d576f15240 con 0x55d576d606b0 2023-07-18T03:37:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdfa940b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d576f004e0 con 0x55d576f04840 2023-07-18T03:37:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdecd57700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f0db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdecd57700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f0db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60470/0 (socket says 172.21.15.40:60470) 2023-07-18T03:37:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdecd57700 1 -- 172.21.15.40:0/3442930355 learned_addr learned my addr 172.21.15.40:0/3442930355 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efde7fff700 1 --2- 172.21.15.40:0/3442930355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576d606b0 0x55d576f0d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efde77fe700 1 --2- 172.21.15.40:0/3442930355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576f04840 0x55d576f14d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdecd57700 1 -- 172.21.15.40:0/3442930355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576f04840 msgr2=0x55d576f14d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdecd57700 1 --2- 172.21.15.40:0/3442930355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576f04840 0x55d576f14d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdecd57700 1 -- 172.21.15.40:0/3442930355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576d606b0 msgr2=0x55d576f0d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdecd57700 1 --2- 172.21.15.40:0/3442930355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576d606b0 0x55d576f0d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdecd57700 1 -- 172.21.15.40:0/3442930355 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d576caa1f0 con 0x55d576f056e0 2023-07-18T03:37:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdecd57700 1 --2- 172.21.15.40:0/3442930355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f0db50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efde0002440 tx=0x7efde0025490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5614d7400c2d5765 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.805+0000 7efdceffd700 1 -- 172.21.15.40:0/3442930355 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde000e070 con 0x55d576f056e0 2023-07-18T03:37:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.806+0000 7efdceffd700 1 -- 172.21.15.40:0/3442930355 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efde000b070 con 0x55d576f056e0 2023-07-18T03:37:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.806+0000 7efdceffd700 1 -- 172.21.15.40:0/3442930355 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde0030480 con 0x55d576f056e0 2023-07-18T03:37:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.806+0000 7efdfa940b80 1 -- 172.21.15.40:0/3442930355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 msgr2=0x55d576f0db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.806+0000 7efdfa940b80 1 --2- 172.21.15.40:0/3442930355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f0db50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efde0002440 tx=0x7efde0025490 comp rx=0 tx=0).stop 2023-07-18T03:37:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.806+0000 7efdfa940b80 1 -- 172.21.15.40:0/3442930355 shutdown_connections 2023-07-18T03:37:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.806+0000 7efdfa940b80 1 --2- 172.21.15.40:0/3442930355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f0db50 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.806+0000 7efdfa940b80 1 --2- 172.21.15.40:0/3442930355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576d606b0 0x55d576f0d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.806+0000 7efdfa940b80 1 --2- 172.21.15.40:0/3442930355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576f04840 0x55d576f14d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.806+0000 7efdfa940b80 1 -- 172.21.15.40:0/3442930355 >> 172.21.15.40:0/3442930355 conn(0x55d576ef9be0 msgr2=0x55d576f08590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.806+0000 7efdfa940b80 1 -- 172.21.15.40:0/3442930355 shutdown_connections 2023-07-18T03:37:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.806+0000 7efdfa940b80 1 -- 172.21.15.40:0/3442930355 wait complete. 2023-07-18T03:37:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.806+0000 7efdfa940b80 1 -- start start 2023-07-18T03:37:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efdfa940b80 1 Processor -- start 2023-07-18T03:37:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efdfa940b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576d606b0 0x55d576f10f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efdfa940b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576f04840 0x55d576f11470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efdfa940b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f13ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efdfa940b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d576f143e0 con 0x55d576f056e0 2023-07-18T03:37:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efdfa940b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d576f14550 con 0x55d576f04840 2023-07-18T03:37:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efdfa940b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d576f14710 con 0x55d576d606b0 2023-07-18T03:37:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efdfa940b80 10 client.0 _ll_register_callbacks cb 0x7efde0009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efde7fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576d606b0 0x55d576f10f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efdecd57700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f13ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efde77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576f04840 0x55d576f11470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efde77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576f04840 0x55d576f11470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43800/0 (socket says 172.21.15.40:43800) 2023-07-18T03:37:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efde7fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576d606b0 0x55d576f10f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49500/0 (socket says 172.21.15.40:49500) 2023-07-18T03:37:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efdecd57700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f13ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60476/0 (socket says 172.21.15.40:60476) 2023-07-18T03:37:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efde77fe700 1 -- 172.21.15.40:0/4070765444 learned_addr learned my addr 172.21.15.40:0/4070765444 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efde7fff700 1 -- 172.21.15.40:0/4070765444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576f04840 msgr2=0x55d576f11470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efde7fff700 1 --2- 172.21.15.40:0/4070765444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576f04840 0x55d576f11470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efde7fff700 1 -- 172.21.15.40:0/4070765444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 msgr2=0x55d576f13ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efde7fff700 1 --2- 172.21.15.40:0/4070765444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f13ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efde7fff700 1 -- 172.21.15.40:0/4070765444 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d576c6cc00 con 0x55d576d606b0 2023-07-18T03:37:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efde77fe700 1 --2- 172.21.15.40:0/4070765444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576f04840 0x55d576f11470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.807+0000 7efdecd57700 1 --2- 172.21.15.40:0/4070765444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f13ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.808+0000 7efde7fff700 1 --2- 172.21.15.40:0/4070765444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576d606b0 0x55d576f10f30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55d576f1c3f0 tx=0x7efdd80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2ed100587bc36d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.808+0000 7efdc77fe700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:37:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.808+0000 7efdc77fe700 1 -- 172.21.15.40:0/4070765444 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd8024040 con 0x55d576d606b0 2023-07-18T03:37:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.808+0000 7efdfa940b80 1 -- 172.21.15.40:0/4070765444 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efddc0a2260 con 0x55d576d606b0 2023-07-18T03:37:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.808+0000 7efdc77fe700 1 -- 172.21.15.40:0/4070765444 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7efdd8029370 con 0x55d576d606b0 2023-07-18T03:37:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.808+0000 7efdc77fe700 1 -- 172.21.15.40:0/4070765444 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdd8003ed0 con 0x55d576d606b0 2023-07-18T03:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.808+0000 7efdfa940b80 10 client.26098 fetch_fsmap learned FSMap version 47 2023-07-18T03:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.808+0000 7efdfa940b80 1 -- 172.21.15.40:0/4070765444 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55d576c82250 con 0x55d576d606b0 2023-07-18T03:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.808+0000 7efdc77fe700 1 -- 172.21.15.40:0/4070765444 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efddc0a2260 con 0x55d576d606b0 2023-07-18T03:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.808+0000 7efdc77fe700 1 -- 172.21.15.40:0/4070765444 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 47) v1 ==== 7522+0+0 (secure 0 0 0) 0x7efdd801eca0 con 0x55d576d606b0 2023-07-18T03:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.809+0000 7efdfa940b80 10 client.26098 fetch_fsmap finished waiting for FSMap version 47 2023-07-18T03:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.809+0000 7efdfa940b80 10 client.26098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:37:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.809+0000 7efdfa940b80 20 client.26098 populate_metadata read hostname 'smithi040' 2023-07-18T03:37:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.809+0000 7efdfa940b80 1 --2- 172.21.15.40:0/4070765444 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d576f6e530 0x55d576f70940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.809+0000 7efdfa940b80 4 client.26098 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:37:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.809+0000 7efdfa940b80 1 -- 172.21.15.40:0/4070765444 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d576e10ff0 con 0x55d576f6e530 2023-07-18T03:37:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.809+0000 7efde77fe700 1 --2- 172.21.15.40:0/4070765444 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d576f6e530 0x55d576f70940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.809+0000 7efdc77fe700 10 client.26098 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:17.809+0000 7efde77fe700 1 --2- 172.21.15.40:0/4070765444 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d576f6e530 0x55d576f70940 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caf646a23158b181 server_cookie=bc460cabc4be7c08 in_seq=0 out_seq=0 2023-07-18T03:37:17.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:17 smithi134 ceph-mon[98216]: pgmap v486: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 3.8 MiB/s wr, 305 op/s 2023-07-18T03:37:17.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:17 smithi040 ceph-mon[96439]: pgmap v486: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 3.8 MiB/s wr, 305 op/s 2023-07-18T03:37:17.856 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:17 smithi064 ceph-mon[98059]: pgmap v486: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 3.8 MiB/s wr, 305 op/s 2023-07-18T03:37:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.098+0000 7efdc77fe700 1 -- 172.21.15.40:0/4070765444 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d576e10ff0 con 0x55d576f6e530 2023-07-18T03:37:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.098+0000 7efdc77fe700 10 client.26098 handle_command_reply: tid=0 2023-07-18T03:37:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.098+0000 7efdfa940b80 1 client.26098 shutdown 2023-07-18T03:37:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.098+0000 7efdfa940b80 1 -- 172.21.15.40:0/4070765444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576d606b0 msgr2=0x55d576f10f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.098+0000 7efdfa940b80 1 --2- 172.21.15.40:0/4070765444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576d606b0 0x55d576f10f30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55d576f1c3f0 tx=0x7efdd80047c0 comp rx=0 tx=0).stop 2023-07-18T03:37:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.099+0000 7efdfa940b80 1 -- 172.21.15.40:0/4070765444 shutdown_connections 2023-07-18T03:37:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.099+0000 7efdfa940b80 1 --2- 172.21.15.40:0/4070765444 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d576f6e530 0x55d576f70940 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.099+0000 7efdfa940b80 1 --2- 172.21.15.40:0/4070765444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d576f056e0 0x55d576f13ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.099+0000 7efdc77fe700 0 client.26098 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.099+0000 7efdfa940b80 1 --2- 172.21.15.40:0/4070765444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d576d606b0 0x55d576f10f30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.099+0000 7efdfa940b80 1 --2- 172.21.15.40:0/4070765444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d576f04840 0x55d576f11470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.099+0000 7efdfa940b80 1 -- 172.21.15.40:0/4070765444 >> 172.21.15.40:0/4070765444 conn(0x55d576ef9be0 msgr2=0x55d576f0d610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.099+0000 7efdfa940b80 1 -- 172.21.15.40:0/4070765444 shutdown_connections 2023-07-18T03:37:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.099+0000 7efdfa940b80 1 -- 172.21.15.40:0/4070765444 wait complete. 2023-07-18T03:37:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.099+0000 7efdfa940b80 20 client.26098 trim_cache size 0 max 16384 2023-07-18T03:37:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:37:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:37:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:37:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:37:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.100+0000 7efdee7ba700 1 -- 172.21.15.40:0/1953237973 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efdd00a1150 msgr2=0x7efdd00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.100+0000 7efdee7ba700 1 --2- 172.21.15.40:0/1953237973 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efdd00a1150 0x7efdd00a3600 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efde80683e0 tx=0x7efdd8009260 comp rx=0 tx=0).stop 2023-07-18T03:37:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.100+0000 7efdee7ba700 1 -- 172.21.15.40:0/1953237973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efde80ffc90 msgr2=0x7efde819aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.100+0000 7efdee7ba700 1 --2- 172.21.15.40:0/1953237973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efde80ffc90 0x7efde819aaa0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efddc00e9f0 tx=0x7efddc00ed00 comp rx=0 tx=0).stop 2023-07-18T03:37:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.100+0000 7efdee7ba700 1 -- 172.21.15.40:0/1953237973 shutdown_connections 2023-07-18T03:37:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.100+0000 7efdee7ba700 1 --2- 172.21.15.40:0/1953237973 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7efdd00a1150 0x7efdd00a3600 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.100+0000 7efdee7ba700 1 --2- 172.21.15.40:0/1953237973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde8100640 0x7efde819cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.100+0000 7efdee7ba700 1 --2- 172.21.15.40:0/1953237973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efde80fea00 0x7efde8198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.100+0000 7efdee7ba700 1 --2- 172.21.15.40:0/1953237973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efde80ffc90 0x7efde819aaa0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.100+0000 7efdee7ba700 1 -- 172.21.15.40:0/1953237973 >> 172.21.15.40:0/1953237973 conn(0x7efde80fa000 msgr2=0x7efde8103150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.100+0000 7efdee7ba700 1 -- 172.21.15.40:0/1953237973 shutdown_connections 2023-07-18T03:37:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:19.100+0000 7efdee7ba700 1 -- 172.21.15.40:0/1953237973 wait complete. 2023-07-18T03:37:19.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:19 smithi134 ceph-mon[98216]: pgmap v487: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 4.3 MiB/s wr, 309 op/s 2023-07-18T03:37:19.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:19 smithi040 ceph-mon[96439]: pgmap v487: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 4.3 MiB/s wr, 309 op/s 2023-07-18T03:37:19.849 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:19 smithi064 ceph-mon[98059]: pgmap v487: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 4.3 MiB/s wr, 309 op/s 2023-07-18T03:37:21.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:21 smithi134 ceph-mon[98216]: pgmap v488: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 2.9 MiB/s wr, 187 op/s 2023-07-18T03:37:21.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:21 smithi040 ceph-mon[96439]: pgmap v488: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 2.9 MiB/s wr, 187 op/s 2023-07-18T03:37:21.849 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:21 smithi064 ceph-mon[98059]: pgmap v488: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 2.9 MiB/s wr, 187 op/s 2023-07-18T03:37:22.331 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:37:22.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.478+0000 7f2d7e1b1700 1 -- 172.21.15.40:0/1521508664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d78100050 msgr2=0x7f2d78100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:22.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.478+0000 7f2d7e1b1700 1 --2- 172.21.15.40:0/1521508664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d78100050 0x7f2d78100460 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009990 tx=0x7f2d68009ca0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.478+0000 7f2d7e1b1700 1 -- 172.21.15.40:0/1521508664 shutdown_connections 2023-07-18T03:37:22.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.478+0000 7f2d7e1b1700 1 --2- 172.21.15.40:0/1521508664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d780ff5c0 0x7f2d78106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.478+0000 7f2d7e1b1700 1 --2- 172.21.15.40:0/1521508664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d78100050 0x7f2d78100460 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.478+0000 7f2d7e1b1700 1 --2- 172.21.15.40:0/1521508664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d780fec10 0x7f2d780ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.478+0000 7f2d7e1b1700 1 -- 172.21.15.40:0/1521508664 >> 172.21.15.40:0/1521508664 conn(0x7f2d780f9770 msgr2=0x7f2d780fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:22.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.478+0000 7f2d7e1b1700 1 -- 172.21.15.40:0/1521508664 shutdown_connections 2023-07-18T03:37:22.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.478+0000 7f2d7e1b1700 1 -- 172.21.15.40:0/1521508664 wait complete. 2023-07-18T03:37:22.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.479+0000 7f2d7e1b1700 1 Processor -- start 2023-07-18T03:37:22.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.479+0000 7f2d7e1b1700 1 -- start start 2023-07-18T03:37:22.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.479+0000 7f2d7e1b1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d780fec10 0x7f2d780fe670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.479+0000 7f2d7e1b1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d780ff5c0 0x7f2d7819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.479+0000 7f2d7e1b1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d78100050 0x7f2d7819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.479+0000 7f2d7e1b1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7819d360 con 0x7f2d780ff5c0 2023-07-18T03:37:22.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.479+0000 7f2d7e1b1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7819d4d0 con 0x7f2d780fec10 2023-07-18T03:37:22.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.479+0000 7f2d7e1b1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7819d690 con 0x7f2d78100050 2023-07-18T03:37:22.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.479+0000 7f2d76ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d780ff5c0 0x7f2d7819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.479+0000 7f2d76ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d780ff5c0 0x7f2d7819a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60488/0 (socket says 172.21.15.40:60488) 2023-07-18T03:37:22.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.479+0000 7f2d76ffd700 1 -- 172.21.15.40:0/3663134757 learned_addr learned my addr 172.21.15.40:0/3663134757 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:22.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.479+0000 7f2d777fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d780fec10 0x7f2d780fe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.479+0000 7f2d77fff700 1 --2- 172.21.15.40:0/3663134757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d78100050 0x7f2d7819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.480+0000 7f2d76ffd700 1 -- 172.21.15.40:0/3663134757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d78100050 msgr2=0x7f2d7819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:22.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.480+0000 7f2d76ffd700 1 --2- 172.21.15.40:0/3663134757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d78100050 0x7f2d7819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.480+0000 7f2d76ffd700 1 -- 172.21.15.40:0/3663134757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d780fec10 msgr2=0x7f2d780fe670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:22.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.480+0000 7f2d76ffd700 1 --2- 172.21.15.40:0/3663134757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d780fec10 0x7f2d780fe670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.480+0000 7f2d76ffd700 1 -- 172.21.15.40:0/3663134757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d680095f0 con 0x7f2d780ff5c0 2023-07-18T03:37:22.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.480+0000 7f2d76ffd700 1 --2- 172.21.15.40:0/3663134757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d780ff5c0 0x7f2d7819a580 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00e9a0 tx=0x7f2d6c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:22.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.480+0000 7f2d74ff9700 1 -- 172.21.15.40:0/3663134757 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c009790 con 0x7f2d780ff5c0 2023-07-18T03:37:22.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.480+0000 7f2d7e1b1700 1 -- 172.21.15.40:0/3663134757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7819d970 con 0x7f2d780ff5c0 2023-07-18T03:37:22.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.480+0000 7f2d74ff9700 1 -- 172.21.15.40:0/3663134757 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d6c010430 con 0x7f2d780ff5c0 2023-07-18T03:37:22.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.480+0000 7f2d74ff9700 1 -- 172.21.15.40:0/3663134757 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c018680 con 0x7f2d780ff5c0 2023-07-18T03:37:22.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.480+0000 7f2d7e1b1700 1 -- 172.21.15.40:0/3663134757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7819de80 con 0x7f2d780ff5c0 2023-07-18T03:37:22.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.482+0000 7f2d74ff9700 1 -- 172.21.15.40:0/3663134757 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2d6c0105a0 con 0x7f2d780ff5c0 2023-07-18T03:37:22.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.483+0000 7f2d74ff9700 1 --2- 172.21.15.40:0/3663134757 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2d600a1150 0x7f2d600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.483+0000 7f2d74ff9700 1 -- 172.21.15.40:0/3663134757 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2d6c014070 con 0x7f2d780ff5c0 2023-07-18T03:37:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.483+0000 7f2d777fe700 1 --2- 172.21.15.40:0/3663134757 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2d600a1150 0x7f2d600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.484+0000 7f2d777fe700 1 --2- 172.21.15.40:0/3663134757 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2d600a1150 0x7f2d600a3600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800b2e0 tx=0x7f2d68005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.484+0000 7f2d8a337b80 1 Processor -- start 2023-07-18T03:37:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.484+0000 7f2d8a337b80 1 -- start start 2023-07-18T03:37:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.484+0000 7f2d8a337b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b34d0 0x560f101b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.484+0000 7f2d8a337b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b3e20 0x560f101b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.484+0000 7f2d8a337b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.484+0000 7f2d8a337b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f100b8cb0 con 0x560f101b4c70 2023-07-18T03:37:22.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.484+0000 7f2d8a337b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f1010ba40 con 0x560f101b3e20 2023-07-18T03:37:22.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.484+0000 7f2d8a337b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f10104ed0 con 0x560f101b34d0 2023-07-18T03:37:22.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.484+0000 7f2d777fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b34d0 0x560f101b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.484+0000 7f2d77fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.484+0000 7f2d76ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b3e20 0x560f101b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.484+0000 7f2d777fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b34d0 0x560f101b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49522/0 (socket says 172.21.15.40:49522) 2023-07-18T03:37:22.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.484+0000 7f2d777fe700 1 -- 172.21.15.40:0/53072350 learned_addr learned my addr 172.21.15.40:0/53072350 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:22.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.485+0000 7f2d77fff700 1 -- 172.21.15.40:0/53072350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b34d0 msgr2=0x560f101b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:22.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.485+0000 7f2d77fff700 1 --2- 172.21.15.40:0/53072350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b34d0 0x560f101b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.485+0000 7f2d77fff700 1 -- 172.21.15.40:0/53072350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b3e20 msgr2=0x560f101b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:22.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.485+0000 7f2d77fff700 1 --2- 172.21.15.40:0/53072350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b3e20 0x560f101b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.485+0000 7f2d77fff700 1 -- 172.21.15.40:0/53072350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f0ff32de0 con 0x560f101b4c70 2023-07-18T03:37:22.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.485+0000 7f2d777fe700 1 --2- 172.21.15.40:0/53072350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b34d0 0x560f101b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:22.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.485+0000 7f2d77fff700 1 --2- 172.21.15.40:0/53072350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101b50e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2d70011580 tx=0x7f2d70009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e222346cdeba66f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.485+0000 7f2d5dffb700 1 -- 172.21.15.40:0/53072350 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000e040 con 0x560f101b4c70 2023-07-18T03:37:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.485+0000 7f2d5dffb700 1 -- 172.21.15.40:0/53072350 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d7000f040 con 0x560f101b4c70 2023-07-18T03:37:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.485+0000 7f2d5dffb700 1 -- 172.21.15.40:0/53072350 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70004430 con 0x560f101b4c70 2023-07-18T03:37:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.486+0000 7f2d8a337b80 1 -- 172.21.15.40:0/53072350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 msgr2=0x560f101b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.486+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/53072350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101b50e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2d70011580 tx=0x7f2d70009920 comp rx=0 tx=0).stop 2023-07-18T03:37:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.486+0000 7f2d8a337b80 1 -- 172.21.15.40:0/53072350 shutdown_connections 2023-07-18T03:37:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.486+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/53072350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101b50e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.486+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/53072350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b34d0 0x560f101b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.486+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/53072350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b3e20 0x560f101b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.486+0000 7f2d8a337b80 1 -- 172.21.15.40:0/53072350 >> 172.21.15.40:0/53072350 conn(0x560f101a8da0 msgr2=0x560f101ae1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.486+0000 7f2d8a337b80 1 -- 172.21.15.40:0/53072350 shutdown_connections 2023-07-18T03:37:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.486+0000 7f2d8a337b80 1 -- 172.21.15.40:0/53072350 wait complete. 2023-07-18T03:37:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.486+0000 7f2d8a337b80 1 -- start start 2023-07-18T03:37:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.486+0000 7f2d8a337b80 1 Processor -- start 2023-07-18T03:37:22.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d8a337b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b34d0 0x560f101bd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d8a337b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b3e20 0x560f101bd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d8a337b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d8a337b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f100a4ac0 con 0x560f101b4c70 2023-07-18T03:37:22.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d8a337b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f100a47c0 con 0x560f101b34d0 2023-07-18T03:37:22.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d8a337b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0ff88920 con 0x560f101b3e20 2023-07-18T03:37:22.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d8a337b80 10 client.0 _ll_register_callbacks cb 0x560f100a2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:22.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d77fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d777fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b34d0 0x560f101bd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d77fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60504/0 (socket says 172.21.15.40:60504) 2023-07-18T03:37:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d77fff700 1 -- 172.21.15.40:0/3982683681 learned_addr learned my addr 172.21.15.40:0/3982683681 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d76ffd700 1 --2- 172.21.15.40:0/3982683681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b3e20 0x560f101bd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d77fff700 1 -- 172.21.15.40:0/3982683681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b3e20 msgr2=0x560f101bd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d77fff700 1 --2- 172.21.15.40:0/3982683681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b3e20 0x560f101bd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d77fff700 1 -- 172.21.15.40:0/3982683681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b34d0 msgr2=0x560f101bd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d77fff700 1 --2- 172.21.15.40:0/3982683681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b34d0 0x560f101bd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d77fff700 1 -- 172.21.15.40:0/3982683681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d78107820 con 0x560f101b4c70 2023-07-18T03:37:22.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d77fff700 1 --2- 172.21.15.40:0/3982683681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c46a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2d70000f80 tx=0x7f2d70004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2145a931c3b7e23c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:22.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:37:22.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.487+0000 7f2d56ffd700 1 -- 172.21.15.40:0/3982683681 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000e040 con 0x560f101b4c70 2023-07-18T03:37:22.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.488+0000 7f2d56ffd700 1 -- 172.21.15.40:0/3982683681 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2d70004ad0 con 0x560f101b4c70 2023-07-18T03:37:22.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.488+0000 7f2d8a337b80 1 -- 172.21.15.40:0/3982683681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f1005e4f0 con 0x560f101b4c70 2023-07-18T03:37:22.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.488+0000 7f2d56ffd700 1 -- 172.21.15.40:0/3982683681 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d700041b0 con 0x560f101b4c70 2023-07-18T03:37:22.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.488+0000 7f2d8a337b80 10 client.16440 fetch_fsmap learned FSMap version 47 2023-07-18T03:37:22.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.488+0000 7f2d8a337b80 1 -- 172.21.15.40:0/3982683681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f2d70011230 con 0x560f101b4c70 2023-07-18T03:37:22.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.488+0000 7f2d56ffd700 1 -- 172.21.15.40:0/3982683681 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560f1005e4f0 con 0x560f101b4c70 2023-07-18T03:37:22.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.488+0000 7f2d56ffd700 1 -- 172.21.15.40:0/3982683681 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 47) v1 ==== 7522+0+0 (secure 0 0 0) 0x7f2d7002f020 con 0x560f101b4c70 2023-07-18T03:37:22.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.488+0000 7f2d8a337b80 10 client.16440 fetch_fsmap finished waiting for FSMap version 47 2023-07-18T03:37:22.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.488+0000 7f2d8a337b80 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:37:22.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.488+0000 7f2d8a337b80 20 client.16440 populate_metadata read hostname 'smithi040' 2023-07-18T03:37:22.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.488+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/3982683681 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560f101c8e70 0x560f101cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.488+0000 7f2d8a337b80 4 client.16440 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:37:22.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.488+0000 7f2d8a337b80 1 -- 172.21.15.40:0/3982683681 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f100fdeb0 con 0x560f101c8e70 2023-07-18T03:37:22.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.488+0000 7f2d777fe700 1 --2- 172.21.15.40:0/3982683681 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560f101c8e70 0x560f101cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.489+0000 7f2d56ffd700 10 client.16440 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:22.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.489+0000 7f2d777fe700 1 --2- 172.21.15.40:0/3982683681 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560f101c8e70 0x560f101cb280 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a605e69502b51a5 server_cookie=47f9b51615d9eac9 in_seq=0 out_seq=0 2023-07-18T03:37:22.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.490+0000 7f2d56ffd700 1 -- 172.21.15.40:0/3982683681 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560f100fdeb0 con 0x560f101c8e70 2023-07-18T03:37:22.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.490+0000 7f2d56ffd700 10 client.16440 handle_command_reply: tid=0 2023-07-18T03:37:22.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.490+0000 7f2d8a337b80 1 client.16440 shutdown 2023-07-18T03:37:22.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.490+0000 7f2d8a337b80 1 -- 172.21.15.40:0/3982683681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 msgr2=0x560f101c46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:22.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.490+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/3982683681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c46a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2d70000f80 tx=0x7f2d70004e40 comp rx=0 tx=0).stop 2023-07-18T03:37:22.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.490+0000 7f2d8a337b80 1 -- 172.21.15.40:0/3982683681 shutdown_connections 2023-07-18T03:37:22.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.490+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/3982683681 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560f101c8e70 0x560f101cb280 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.490+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/3982683681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c46a0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.490+0000 7f2d56ffd700 0 client.16440 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:22.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.490+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/3982683681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b34d0 0x560f101bd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.490+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/3982683681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b3e20 0x560f101bd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.490+0000 7f2d8a337b80 1 -- 172.21.15.40:0/3982683681 >> 172.21.15.40:0/3982683681 conn(0x560f101a8da0 msgr2=0x560f101ad8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:22.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.490+0000 7f2d8a337b80 1 -- 172.21.15.40:0/3982683681 shutdown_connections 2023-07-18T03:37:22.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.490+0000 7f2d8a337b80 1 -- 172.21.15.40:0/3982683681 wait complete. 2023-07-18T03:37:22.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.490+0000 7f2d8a337b80 20 client.16440 trim_cache size 0 max 16384 2023-07-18T03:37:22.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.508+0000 7f2d8a337b80 1 Processor -- start 2023-07-18T03:37:22.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.508+0000 7f2d8a337b80 1 -- start start 2023-07-18T03:37:22.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.508+0000 7f2d8a337b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b34d0 0x560f101bbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.508+0000 7f2d8a337b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b3e20 0x560f101c4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.508+0000 7f2d8a337b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.508+0000 7f2d8a337b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f101bb8d0 con 0x560f101b4c70 2023-07-18T03:37:22.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.508+0000 7f2d8a337b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f101badd0 con 0x560f101b34d0 2023-07-18T03:37:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.508+0000 7f2d8a337b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f101c8bc0 con 0x560f101b3e20 2023-07-18T03:37:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.508+0000 7f2d77fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.508+0000 7f2d77fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60512/0 (socket says 172.21.15.40:60512) 2023-07-18T03:37:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.508+0000 7f2d77fff700 1 -- 172.21.15.40:0/4113810024 learned_addr learned my addr 172.21.15.40:0/4113810024 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.508+0000 7f2d76ffd700 1 --2- 172.21.15.40:0/4113810024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b3e20 0x560f101c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.508+0000 7f2d777fe700 1 --2- 172.21.15.40:0/4113810024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b34d0 0x560f101bbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.509+0000 7f2d77fff700 1 -- 172.21.15.40:0/4113810024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b3e20 msgr2=0x560f101c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:22.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.509+0000 7f2d77fff700 1 --2- 172.21.15.40:0/4113810024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b3e20 0x560f101c4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.509+0000 7f2d77fff700 1 -- 172.21.15.40:0/4113810024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b34d0 msgr2=0x560f101bbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.509+0000 7f2d77fff700 1 --2- 172.21.15.40:0/4113810024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b34d0 0x560f101bbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.509+0000 7f2d77fff700 1 -- 172.21.15.40:0/4113810024 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f0ff1bc60 con 0x560f101b4c70 2023-07-18T03:37:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.509+0000 7f2d77fff700 1 --2- 172.21.15.40:0/4113810024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c4f90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009b30 tx=0x7f2d70011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58c13d5e8a7a965 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.509+0000 7f2d5e7fc700 1 -- 172.21.15.40:0/4113810024 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d700075e0 con 0x560f101b4c70 2023-07-18T03:37:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.509+0000 7f2d5e7fc700 1 -- 172.21.15.40:0/4113810024 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d70022070 con 0x560f101b4c70 2023-07-18T03:37:22.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.509+0000 7f2d5e7fc700 1 -- 172.21.15.40:0/4113810024 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70027430 con 0x560f101b4c70 2023-07-18T03:37:22.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.510+0000 7f2d8a337b80 1 -- 172.21.15.40:0/4113810024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 msgr2=0x560f101c4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:22.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.510+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/4113810024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c4f90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009b30 tx=0x7f2d70011e80 comp rx=0 tx=0).stop 2023-07-18T03:37:22.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.510+0000 7f2d8a337b80 1 -- 172.21.15.40:0/4113810024 shutdown_connections 2023-07-18T03:37:22.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.510+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/4113810024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c4f90 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.510+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/4113810024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b34d0 0x560f101bbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.510+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/4113810024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b3e20 0x560f101c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.510+0000 7f2d8a337b80 1 -- 172.21.15.40:0/4113810024 >> 172.21.15.40:0/4113810024 conn(0x560f101a8da0 msgr2=0x560f101cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:22.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.510+0000 7f2d8a337b80 1 -- 172.21.15.40:0/4113810024 shutdown_connections 2023-07-18T03:37:22.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.510+0000 7f2d8a337b80 1 -- 172.21.15.40:0/4113810024 wait complete. 2023-07-18T03:37:22.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.510+0000 7f2d8a337b80 1 -- start start 2023-07-18T03:37:22.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.510+0000 7f2d8a337b80 1 Processor -- start 2023-07-18T03:37:22.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.510+0000 7f2d8a337b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b34d0 0x560f101c04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d8a337b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b3e20 0x560f101c09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d8a337b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d8a337b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f101c3f50 con 0x560f101b4c70 2023-07-18T03:37:22.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d8a337b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f101c40c0 con 0x560f101b3e20 2023-07-18T03:37:22.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d8a337b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f101c4280 con 0x560f101b34d0 2023-07-18T03:37:22.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d8a337b80 10 client.0 _ll_register_callbacks cb 0x7f2d6c005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:22.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d77fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d777fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b34d0 0x560f101c04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d77fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60518/0 (socket says 172.21.15.40:60518) 2023-07-18T03:37:22.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d777fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b34d0 0x560f101c04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49542/0 (socket says 172.21.15.40:49542) 2023-07-18T03:37:22.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d77fff700 1 -- 172.21.15.40:0/3363847405 learned_addr learned my addr 172.21.15.40:0/3363847405 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:22.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d76ffd700 1 --2- 172.21.15.40:0/3363847405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b3e20 0x560f101c09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d77fff700 1 -- 172.21.15.40:0/3363847405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b34d0 msgr2=0x560f101c04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:22.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d77fff700 1 --2- 172.21.15.40:0/3363847405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b34d0 0x560f101c04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d77fff700 1 -- 172.21.15.40:0/3363847405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b3e20 msgr2=0x560f101c09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:22.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d77fff700 1 --2- 172.21.15.40:0/3363847405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b3e20 0x560f101c09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d77fff700 1 -- 172.21.15.40:0/3363847405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d70011230 con 0x560f101b4c70 2023-07-18T03:37:22.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d76ffd700 1 --2- 172.21.15.40:0/3363847405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b3e20 0x560f101c09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:22.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d77fff700 1 --2- 172.21.15.40:0/3363847405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c3a10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2d70007b60 tx=0x7f2d7001dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38da0e1e047f34a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:22.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:37:22.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d56ffd700 1 -- 172.21.15.40:0/3363847405 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000e070 con 0x560f101b4c70 2023-07-18T03:37:22.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d56ffd700 1 -- 172.21.15.40:0/3363847405 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2d700073e0 con 0x560f101b4c70 2023-07-18T03:37:22.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.511+0000 7f2d8a337b80 1 -- 172.21.15.40:0/3363847405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d7000f040 con 0x560f101b4c70 2023-07-18T03:37:22.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.512+0000 7f2d56ffd700 1 -- 172.21.15.40:0/3363847405 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d7001d920 con 0x560f101b4c70 2023-07-18T03:37:22.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.512+0000 7f2d8a337b80 10 client.16446 fetch_fsmap learned FSMap version 47 2023-07-18T03:37:22.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.512+0000 7f2d8a337b80 1 -- 172.21.15.40:0/3363847405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x560f0ff59250 con 0x560f101b4c70 2023-07-18T03:37:22.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.512+0000 7f2d56ffd700 1 -- 172.21.15.40:0/3363847405 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d70049020 con 0x560f101b4c70 2023-07-18T03:37:22.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.512+0000 7f2d56ffd700 1 -- 172.21.15.40:0/3363847405 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 47) v1 ==== 7522+0+0 (secure 0 0 0) 0x7f2d70011890 con 0x560f101b4c70 2023-07-18T03:37:22.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.512+0000 7f2d8a337b80 10 client.16446 fetch_fsmap finished waiting for FSMap version 47 2023-07-18T03:37:22.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.512+0000 7f2d8a337b80 10 client.16446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:37:22.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.512+0000 7f2d8a337b80 20 client.16446 populate_metadata read hostname 'smithi040' 2023-07-18T03:37:22.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.512+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/3363847405 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560f1021da10 0x560f1021fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:22.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.512+0000 7f2d8a337b80 4 client.16446 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:37:22.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.512+0000 7f2d8a337b80 1 -- 172.21.15.40:0/3363847405 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560f100c0050 con 0x560f1021da10 2023-07-18T03:37:22.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.512+0000 7f2d777fe700 1 --2- 172.21.15.40:0/3363847405 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560f1021da10 0x560f1021fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:22.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.513+0000 7f2d56ffd700 10 client.16446 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:22.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.513+0000 7f2d777fe700 1 --2- 172.21.15.40:0/3363847405 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560f1021da10 0x560f1021fe20 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bebf0a4de581b565 server_cookie=32e1315b715ace70 in_seq=0 out_seq=0 2023-07-18T03:37:22.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.513+0000 7f2d56ffd700 1 -- 172.21.15.40:0/3363847405 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x560f100c0050 con 0x560f1021da10 2023-07-18T03:37:22.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.513+0000 7f2d56ffd700 10 client.16446 handle_command_reply: tid=0 2023-07-18T03:37:22.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.513+0000 7f2d8a337b80 1 client.16446 shutdown 2023-07-18T03:37:22.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.513+0000 7f2d8a337b80 1 -- 172.21.15.40:0/3363847405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 msgr2=0x560f101c3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:22.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.513+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/3363847405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c3a10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2d70007b60 tx=0x7f2d7001dfc0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.513+0000 7f2d8a337b80 1 -- 172.21.15.40:0/3363847405 shutdown_connections 2023-07-18T03:37:22.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.513+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/3363847405 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560f1021da10 0x560f1021fe20 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.513+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/3363847405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f101b4c70 0x560f101c3a10 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.513+0000 7f2d56ffd700 0 client.16446 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:22.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.513+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/3363847405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f101b34d0 0x560f101c04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.513+0000 7f2d8a337b80 1 --2- 172.21.15.40:0/3363847405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f101b3e20 0x560f101c09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.513+0000 7f2d8a337b80 1 -- 172.21.15.40:0/3363847405 >> 172.21.15.40:0/3363847405 conn(0x560f101a8da0 msgr2=0x560f101c4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:22.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.513+0000 7f2d8a337b80 1 -- 172.21.15.40:0/3363847405 shutdown_connections 2023-07-18T03:37:22.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.513+0000 7f2d8a337b80 1 -- 172.21.15.40:0/3363847405 wait complete. 2023-07-18T03:37:22.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.514+0000 7f2d8a337b80 20 client.16446 trim_cache size 0 max 16384 2023-07-18T03:37:22.519 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:37:22.519 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-18T03:37:22.520 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:37:22.520 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:37:22.520 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:37:22.520 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:37:22.520 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:37:22.520 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:37:22.520 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:37:22.520 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:37:22.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.514+0000 7f2d7e1b1700 1 -- 172.21.15.40:0/3663134757 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2d600a1150 msgr2=0x7f2d600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:22.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.514+0000 7f2d7e1b1700 1 --2- 172.21.15.40:0/3663134757 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2d600a1150 0x7f2d600a3600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2d6800b2e0 tx=0x7f2d68005fb0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.514+0000 7f2d7e1b1700 1 -- 172.21.15.40:0/3663134757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d780ff5c0 msgr2=0x7f2d7819a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:22.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.514+0000 7f2d7e1b1700 1 --2- 172.21.15.40:0/3663134757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d780ff5c0 0x7f2d7819a580 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00e9a0 tx=0x7f2d6c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.514+0000 7f2d7e1b1700 1 -- 172.21.15.40:0/3663134757 shutdown_connections 2023-07-18T03:37:22.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.514+0000 7f2d7e1b1700 1 --2- 172.21.15.40:0/3663134757 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2d600a1150 0x7f2d600a3600 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.514+0000 7f2d7e1b1700 1 --2- 172.21.15.40:0/3663134757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d78100050 0x7f2d7819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.514+0000 7f2d7e1b1700 1 --2- 172.21.15.40:0/3663134757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d780fec10 0x7f2d780fe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.514+0000 7f2d7e1b1700 1 --2- 172.21.15.40:0/3663134757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d780ff5c0 0x7f2d7819a580 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:22.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.514+0000 7f2d7e1b1700 1 -- 172.21.15.40:0/3663134757 >> 172.21.15.40:0/3663134757 conn(0x7f2d780f9770 msgr2=0x7f2d780fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:22.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.514+0000 7f2d7e1b1700 1 -- 172.21.15.40:0/3663134757 shutdown_connections 2023-07-18T03:37:22.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:22.514+0000 7f2d7e1b1700 1 -- 172.21.15.40:0/3663134757 wait complete. 2023-07-18T03:37:22.523 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:37:23.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:22 smithi134 ceph-mon[98216]: pgmap v489: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 4.1 MiB/s wr, 292 op/s 2023-07-18T03:37:23.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:22 smithi040 ceph-mon[96439]: pgmap v489: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 4.1 MiB/s wr, 292 op/s 2023-07-18T03:37:23.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:22 smithi064 ceph-mon[98059]: pgmap v489: 97 pgs: 97 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 4.1 MiB/s wr, 292 op/s 2023-07-18T03:37:25.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:25 smithi134 ceph-mon[98216]: pgmap v490: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 3.3 MiB/s wr, 242 op/s 2023-07-18T03:37:25.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:25 smithi040 ceph-mon[96439]: pgmap v490: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 3.3 MiB/s wr, 242 op/s 2023-07-18T03:37:25.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:25 smithi064 ceph-mon[98059]: pgmap v490: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 3.3 MiB/s wr, 242 op/s 2023-07-18T03:37:27.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:27 smithi134 ceph-mon[98216]: pgmap v491: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 3.4 MiB/s wr, 247 op/s 2023-07-18T03:37:27.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:27 smithi040 ceph-mon[96439]: pgmap v491: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 3.4 MiB/s wr, 247 op/s 2023-07-18T03:37:27.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:27 smithi064 ceph-mon[98059]: pgmap v491: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 3.4 MiB/s wr, 247 op/s 2023-07-18T03:37:29.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:29 smithi134 ceph-mon[98216]: pgmap v492: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 4.5 MiB/s wr, 319 op/s 2023-07-18T03:37:29.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:29 smithi040 ceph-mon[96439]: pgmap v492: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 4.5 MiB/s wr, 319 op/s 2023-07-18T03:37:29.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:29 smithi064 ceph-mon[98059]: pgmap v492: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 4.5 MiB/s wr, 319 op/s 2023-07-18T03:37:31.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:31 smithi134 ceph-mon[98216]: pgmap v493: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 3.5 MiB/s wr, 258 op/s 2023-07-18T03:37:31.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:31 smithi040 ceph-mon[96439]: pgmap v493: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 3.5 MiB/s wr, 258 op/s 2023-07-18T03:37:31.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:31 smithi064 ceph-mon[98059]: pgmap v493: 97 pgs: 97 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 3.5 MiB/s wr, 258 op/s 2023-07-18T03:37:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.220+0000 7fdfc73aa700 1 -- 172.21.15.40:0/1334687943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfc00ff5f0 msgr2=0x7fdfc0106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.220+0000 7fdfc73aa700 1 --2- 172.21.15.40:0/1334687943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfc00ff5f0 0x7fdfc0106ea0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800b0a0 tx=0x7fdfb800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.220+0000 7fdfc73aa700 1 -- 172.21.15.40:0/1334687943 shutdown_connections 2023-07-18T03:37:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.220+0000 7fdfc73aa700 1 --2- 172.21.15.40:0/1334687943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfc00ff5f0 0x7fdfc0106ea0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.220+0000 7fdfc73aa700 1 --2- 172.21.15.40:0/1334687943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfc0100080 0x7fdfc0100490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.220+0000 7fdfc73aa700 1 --2- 172.21.15.40:0/1334687943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfc00fec40 0x7fdfc00ff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.220+0000 7fdfc73aa700 1 -- 172.21.15.40:0/1334687943 >> 172.21.15.40:0/1334687943 conn(0x7fdfc00f9780 msgr2=0x7fdfc00fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.220+0000 7fdfc73aa700 1 -- 172.21.15.40:0/1334687943 shutdown_connections 2023-07-18T03:37:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.220+0000 7fdfc73aa700 1 -- 172.21.15.40:0/1334687943 wait complete. 2023-07-18T03:37:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc73aa700 1 Processor -- start 2023-07-18T03:37:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc73aa700 1 -- start start 2023-07-18T03:37:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc73aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfc00fec40 0x7fdfc0195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc73aa700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfc00ff5f0 0x7fdfc0195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc73aa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfc0100080 0x7fdfc0196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc73aa700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc0197480 con 0x7fdfc0100080 2023-07-18T03:37:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc73aa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc01975f0 con 0x7fdfc00fec40 2023-07-18T03:37:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc73aa700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc01977b0 con 0x7fdfc00ff5f0 2023-07-18T03:37:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc5947700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfc0100080 0x7fdfc0196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc5146700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfc00fec40 0x7fdfc0195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc4945700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfc00ff5f0 0x7fdfc0195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc5146700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfc00fec40 0x7fdfc0195a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43854/0 (socket says 172.21.15.40:43854) 2023-07-18T03:37:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc5947700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfc0100080 0x7fdfc0196e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60532/0 (socket says 172.21.15.40:60532) 2023-07-18T03:37:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc4945700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfc00ff5f0 0x7fdfc0195fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49558/0 (socket says 172.21.15.40:49558) 2023-07-18T03:37:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc5146700 1 -- 172.21.15.40:0/3156951554 learned_addr learned my addr 172.21.15.40:0/3156951554 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc5947700 1 -- 172.21.15.40:0/3156951554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfc00ff5f0 msgr2=0x7fdfc0195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc5947700 1 --2- 172.21.15.40:0/3156951554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfc00ff5f0 0x7fdfc0195fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc5947700 1 -- 172.21.15.40:0/3156951554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfc00fec40 msgr2=0x7fdfc0195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc5947700 1 --2- 172.21.15.40:0/3156951554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfc00fec40 0x7fdfc0195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.221+0000 7fdfc5947700 1 -- 172.21.15.40:0/3156951554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb8009d00 con 0x7fdfc0100080 2023-07-18T03:37:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.222+0000 7fdfc5947700 1 --2- 172.21.15.40:0/3156951554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfc0100080 0x7fdfc0196e20 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009fd0 tx=0x7fdfb8007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.222+0000 7fdfb67fc700 1 -- 172.21.15.40:0/3156951554 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb800e040 con 0x7fdfc0100080 2023-07-18T03:37:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.222+0000 7fdfb67fc700 1 -- 172.21.15.40:0/3156951554 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfb8004c50 con 0x7fdfc0100080 2023-07-18T03:37:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.222+0000 7fdfc73aa700 1 -- 172.21.15.40:0/3156951554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc0197a30 con 0x7fdfc0100080 2023-07-18T03:37:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.222+0000 7fdfb67fc700 1 -- 172.21.15.40:0/3156951554 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb801b6c0 con 0x7fdfc0100080 2023-07-18T03:37:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.222+0000 7fdfc73aa700 1 -- 172.21.15.40:0/3156951554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc019e250 con 0x7fdfc0100080 2023-07-18T03:37:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.224+0000 7fdfb67fc700 1 -- 172.21.15.40:0/3156951554 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdfb801d470 con 0x7fdfc0100080 2023-07-18T03:37:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.224+0000 7fdfb67fc700 1 --2- 172.21.15.40:0/3156951554 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdfac0a1150 0x7fdfac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.225+0000 7fdfc5146700 1 --2- 172.21.15.40:0/3156951554 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdfac0a1150 0x7fdfac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.225+0000 7fdfb67fc700 1 -- 172.21.15.40:0/3156951554 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fdfb80cdb90 con 0x7fdfc0100080 2023-07-18T03:37:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.225+0000 7fdfc5146700 1 --2- 172.21.15.40:0/3156951554 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdfac0a1150 0x7fdfac0a3600 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009d20 tx=0x7fdfb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfd3530b80 1 Processor -- start 2023-07-18T03:37:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfd3530b80 1 -- start start 2023-07-18T03:37:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfd3530b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a6ed6b0 0x55711a895710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfd3530b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a895c50 0x55711a8965b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfd3530b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a896af0 0x55711a896f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfd3530b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711a796c50 con 0x55711a6ed6b0 2023-07-18T03:37:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfd3530b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711a7e99e0 con 0x55711a895c50 2023-07-18T03:37:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfd3530b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711a7e2e70 con 0x55711a896af0 2023-07-18T03:37:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfc5146700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a6ed6b0 0x55711a895710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfc5146700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a6ed6b0 0x55711a895710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60536/0 (socket says 172.21.15.40:60536) 2023-07-18T03:37:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfc5146700 1 -- 172.21.15.40:0/4083606047 learned_addr learned my addr 172.21.15.40:0/4083606047 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfc4945700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a895c50 0x55711a8965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfc5947700 1 --2- 172.21.15.40:0/4083606047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a896af0 0x55711a896f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfc5146700 1 -- 172.21.15.40:0/4083606047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a896af0 msgr2=0x55711a896f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfc5146700 1 --2- 172.21.15.40:0/4083606047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a896af0 0x55711a896f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfc5146700 1 -- 172.21.15.40:0/4083606047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a895c50 msgr2=0x55711a8965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfc5146700 1 --2- 172.21.15.40:0/4083606047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a895c50 0x55711a8965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfc5146700 1 -- 172.21.15.40:0/4083606047 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55711a610d80 con 0x55711a6ed6b0 2023-07-18T03:37:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfc4945700 1 --2- 172.21.15.40:0/4083606047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a895c50 0x55711a8965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.226+0000 7fdfc5146700 1 --2- 172.21.15.40:0/4083606047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a6ed6b0 0x55711a895710 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000e540 tx=0x7fdfb0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a118647b8c7af0a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.227+0000 7fdfab7fe700 1 -- 172.21.15.40:0/4083606047 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0026430 con 0x55711a6ed6b0 2023-07-18T03:37:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.227+0000 7fdfab7fe700 1 -- 172.21.15.40:0/4083606047 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfb0021070 con 0x55711a6ed6b0 2023-07-18T03:37:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.227+0000 7fdfab7fe700 1 -- 172.21.15.40:0/4083606047 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb001d6e0 con 0x55711a6ed6b0 2023-07-18T03:37:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.227+0000 7fdfd3530b80 1 -- 172.21.15.40:0/4083606047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a6ed6b0 msgr2=0x55711a895710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.227+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/4083606047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a6ed6b0 0x55711a895710 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000e540 tx=0x7fdfb0014000 comp rx=0 tx=0).stop 2023-07-18T03:37:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.227+0000 7fdfd3530b80 1 -- 172.21.15.40:0/4083606047 shutdown_connections 2023-07-18T03:37:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.227+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/4083606047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a896af0 0x55711a896f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.227+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/4083606047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a6ed6b0 0x55711a895710 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.227+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/4083606047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a895c50 0x55711a8965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.227+0000 7fdfd3530b80 1 -- 172.21.15.40:0/4083606047 >> 172.21.15.40:0/4083606047 conn(0x55711a886be0 msgr2=0x55711a88bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.228+0000 7fdfd3530b80 1 -- 172.21.15.40:0/4083606047 shutdown_connections 2023-07-18T03:37:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.228+0000 7fdfd3530b80 1 -- 172.21.15.40:0/4083606047 wait complete. 2023-07-18T03:37:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.228+0000 7fdfd3530b80 1 -- start start 2023-07-18T03:37:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.228+0000 7fdfd3530b80 1 Processor -- start 2023-07-18T03:37:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.228+0000 7fdfd3530b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a6ed6b0 0x55711a89ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.228+0000 7fdfd3530b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a895c50 0x55711a89f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.228+0000 7fdfd3530b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a896af0 0x55711a8a6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.228+0000 7fdfd3530b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711a782a60 con 0x55711a895c50 2023-07-18T03:37:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.228+0000 7fdfd3530b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711a782760 con 0x55711a6ed6b0 2023-07-18T03:37:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.228+0000 7fdfd3530b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711a6668c0 con 0x55711a896af0 2023-07-18T03:37:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.228+0000 7fdfd3530b80 10 client.0 _ll_register_callbacks cb 0x55711a780700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.228+0000 7fdfc4945700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a895c50 0x55711a89f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.229+0000 7fdfc4945700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a895c50 0x55711a89f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60544/0 (socket says 172.21.15.40:60544) 2023-07-18T03:37:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.229+0000 7fdfc4945700 1 -- 172.21.15.40:0/3803914772 learned_addr learned my addr 172.21.15.40:0/3803914772 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.229+0000 7fdfc5146700 1 --2- 172.21.15.40:0/3803914772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a6ed6b0 0x55711a89ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.229+0000 7fdfc5947700 1 --2- 172.21.15.40:0/3803914772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a896af0 0x55711a8a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.229+0000 7fdfc4945700 1 -- 172.21.15.40:0/3803914772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a896af0 msgr2=0x55711a8a6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.229+0000 7fdfc4945700 1 --2- 172.21.15.40:0/3803914772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a896af0 0x55711a8a6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.229+0000 7fdfc4945700 1 -- 172.21.15.40:0/3803914772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a6ed6b0 msgr2=0x55711a89ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.229+0000 7fdfc4945700 1 --2- 172.21.15.40:0/3803914772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a6ed6b0 0x55711a89ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.229+0000 7fdfc4945700 1 -- 172.21.15.40:0/3803914772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfbc0098e0 con 0x55711a895c50 2023-07-18T03:37:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.229+0000 7fdfc4945700 1 --2- 172.21.15.40:0/3803914772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a895c50 0x55711a89f1d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc012cf0 tx=0x7fdfbc009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1cbb5b082e6be7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.229+0000 7fdfa8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:37:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.229+0000 7fdfa8ff9700 1 -- 172.21.15.40:0/3803914772 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc017070 con 0x55711a895c50 2023-07-18T03:37:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.229+0000 7fdfa8ff9700 1 -- 172.21.15.40:0/3803914772 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fdfbc004020 con 0x55711a895c50 2023-07-18T03:37:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.229+0000 7fdfd3530b80 1 -- 172.21.15.40:0/3803914772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55711a73c490 con 0x55711a895c50 2023-07-18T03:37:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.229+0000 7fdfa8ff9700 1 -- 172.21.15.40:0/3803914772 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfbc009e90 con 0x55711a895c50 2023-07-18T03:37:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.229+0000 7fdfd3530b80 10 client.16458 fetch_fsmap learned FSMap version 47 2023-07-18T03:37:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.229+0000 7fdfd3530b80 1 -- 172.21.15.40:0/3803914772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7fdfb0017070 con 0x55711a895c50 2023-07-18T03:37:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.230+0000 7fdfa8ff9700 1 -- 172.21.15.40:0/3803914772 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55711a73c490 con 0x55711a895c50 2023-07-18T03:37:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.230+0000 7fdfa8ff9700 1 -- 172.21.15.40:0/3803914772 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 47) v1 ==== 7522+0+0 (secure 0 0 0) 0x7fdfbc025050 con 0x55711a895c50 2023-07-18T03:37:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.230+0000 7fdfd3530b80 10 client.16458 fetch_fsmap finished waiting for FSMap version 47 2023-07-18T03:37:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.230+0000 7fdfd3530b80 10 client.16458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:37:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.230+0000 7fdfd3530b80 20 client.16458 populate_metadata read hostname 'smithi040' 2023-07-18T03:37:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.230+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/3803914772 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55711a8aab90 0x55711a8acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.230+0000 7fdfd3530b80 4 client.16458 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:37:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.230+0000 7fdfd3530b80 1 -- 172.21.15.40:0/3803914772 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55711a7dbe50 con 0x55711a8aab90 2023-07-18T03:37:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.230+0000 7fdfc5146700 1 --2- 172.21.15.40:0/3803914772 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55711a8aab90 0x55711a8acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.230+0000 7fdfa8ff9700 10 client.16458 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.230+0000 7fdfc5146700 1 --2- 172.21.15.40:0/3803914772 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55711a8aab90 0x55711a8acfa0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da2657b7adfbf372 server_cookie=b07415d368bc4bd3 in_seq=0 out_seq=0 2023-07-18T03:37:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.231+0000 7fdfa8ff9700 1 -- 172.21.15.40:0/3803914772 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55711a7dbe50 con 0x55711a8aab90 2023-07-18T03:37:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.231+0000 7fdfa8ff9700 10 client.16458 handle_command_reply: tid=0 2023-07-18T03:37:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.231+0000 7fdfd3530b80 1 client.16458 shutdown 2023-07-18T03:37:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.231+0000 7fdfd3530b80 1 -- 172.21.15.40:0/3803914772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a895c50 msgr2=0x55711a89f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.231+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/3803914772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a895c50 0x55711a89f1d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc012cf0 tx=0x7fdfbc009d00 comp rx=0 tx=0).stop 2023-07-18T03:37:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.231+0000 7fdfd3530b80 1 -- 172.21.15.40:0/3803914772 shutdown_connections 2023-07-18T03:37:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.231+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/3803914772 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55711a8aab90 0x55711a8acfa0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.231+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/3803914772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a896af0 0x55711a8a6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.232+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/3803914772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a6ed6b0 0x55711a89ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.232+0000 7fdfa8ff9700 0 client.16458 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.232+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/3803914772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a895c50 0x55711a89f1d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.232+0000 7fdfd3530b80 1 -- 172.21.15.40:0/3803914772 >> 172.21.15.40:0/3803914772 conn(0x55711a886be0 msgr2=0x55711a88b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.232+0000 7fdfd3530b80 1 -- 172.21.15.40:0/3803914772 shutdown_connections 2023-07-18T03:37:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.232+0000 7fdfd3530b80 1 -- 172.21.15.40:0/3803914772 wait complete. 2023-07-18T03:37:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.232+0000 7fdfd3530b80 20 client.16458 trim_cache size 0 max 16384 2023-07-18T03:37:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.246+0000 7fdfd3530b80 1 Processor -- start 2023-07-18T03:37:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.246+0000 7fdfd3530b80 1 -- start start 2023-07-18T03:37:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfd3530b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a6ed6b0 0x55711a8956d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfd3530b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a895c50 0x55711a8a6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfd3530b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a896af0 0x55711a89d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfd3530b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711a888d00 con 0x55711a6ed6b0 2023-07-18T03:37:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfd3530b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711a89d770 con 0x55711a895c50 2023-07-18T03:37:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfd3530b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711a88d780 con 0x55711a896af0 2023-07-18T03:37:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfc5146700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a6ed6b0 0x55711a8956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfc5146700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a6ed6b0 0x55711a8956d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60550/0 (socket says 172.21.15.40:60550) 2023-07-18T03:37:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfc5146700 1 -- 172.21.15.40:0/78543583 learned_addr learned my addr 172.21.15.40:0/78543583 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfc5947700 1 --2- 172.21.15.40:0/78543583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a896af0 0x55711a89d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfc4945700 1 --2- 172.21.15.40:0/78543583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a895c50 0x55711a8a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfc5146700 1 -- 172.21.15.40:0/78543583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a896af0 msgr2=0x55711a89d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfc5146700 1 --2- 172.21.15.40:0/78543583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a896af0 0x55711a89d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfc5146700 1 -- 172.21.15.40:0/78543583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a895c50 msgr2=0x55711a8a6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfc5146700 1 --2- 172.21.15.40:0/78543583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a895c50 0x55711a8a6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfc5146700 1 -- 172.21.15.40:0/78543583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55711a5f9c00 con 0x55711a6ed6b0 2023-07-18T03:37:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfc5146700 1 --2- 172.21.15.40:0/78543583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a6ed6b0 0x55711a8956d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0026000 tx=0x7fdfb00092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bedd8f7e0c7c070a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfabfff700 1 -- 172.21.15.40:0/78543583 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0033ae0 con 0x55711a6ed6b0 2023-07-18T03:37:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfabfff700 1 -- 172.21.15.40:0/78543583 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfb0033c40 con 0x55711a6ed6b0 2023-07-18T03:37:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.247+0000 7fdfabfff700 1 -- 172.21.15.40:0/78543583 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0005680 con 0x55711a6ed6b0 2023-07-18T03:37:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.248+0000 7fdfd3530b80 1 -- 172.21.15.40:0/78543583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a6ed6b0 msgr2=0x55711a8956d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.248+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/78543583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a6ed6b0 0x55711a8956d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0026000 tx=0x7fdfb00092d0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.248+0000 7fdfd3530b80 1 -- 172.21.15.40:0/78543583 shutdown_connections 2023-07-18T03:37:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.248+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/78543583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a896af0 0x55711a89d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.248+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/78543583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a6ed6b0 0x55711a8956d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.248+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/78543583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a895c50 0x55711a8a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.248+0000 7fdfd3530b80 1 -- 172.21.15.40:0/78543583 >> 172.21.15.40:0/78543583 conn(0x55711a886be0 msgr2=0x55711a8b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.248+0000 7fdfd3530b80 1 -- 172.21.15.40:0/78543583 shutdown_connections 2023-07-18T03:37:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.248+0000 7fdfd3530b80 1 -- 172.21.15.40:0/78543583 wait complete. 2023-07-18T03:37:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.248+0000 7fdfd3530b80 1 -- start start 2023-07-18T03:37:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.248+0000 7fdfd3530b80 1 Processor -- start 2023-07-18T03:37:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfd3530b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a6ed6b0 0x55711a8a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfd3530b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a895c50 0x55711a8a2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfd3530b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a896af0 0x55711a8a5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfd3530b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711a8a5670 con 0x55711a896af0 2023-07-18T03:37:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfd3530b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711a8a57e0 con 0x55711a6ed6b0 2023-07-18T03:37:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfd3530b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711a8a59a0 con 0x55711a895c50 2023-07-18T03:37:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfd3530b80 10 client.0 _ll_register_callbacks cb 0x7fdfbc0127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfc5947700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a896af0 0x55711a8a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfc5146700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a6ed6b0 0x55711a8a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfc5947700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a896af0 0x55711a8a5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60560/0 (socket says 172.21.15.40:60560) 2023-07-18T03:37:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfc4945700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a895c50 0x55711a8a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfc5947700 1 -- 172.21.15.40:0/1156676430 learned_addr learned my addr 172.21.15.40:0/1156676430 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfc5947700 1 -- 172.21.15.40:0/1156676430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a895c50 msgr2=0x55711a8a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfc5947700 1 --2- 172.21.15.40:0/1156676430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a895c50 0x55711a8a2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfc5947700 1 -- 172.21.15.40:0/1156676430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a6ed6b0 msgr2=0x55711a8a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfc5947700 1 --2- 172.21.15.40:0/1156676430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a6ed6b0 0x55711a8a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfc5947700 1 -- 172.21.15.40:0/1156676430 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb002a040 con 0x55711a896af0 2023-07-18T03:37:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfc5146700 1 --2- 172.21.15.40:0/1156676430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a6ed6b0 0x55711a8a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfc5947700 1 --2- 172.21.15.40:0/1156676430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a896af0 0x55711a8a5130 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdfb809afa0 tx=0x7fdfb800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d276f211b44455ee server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.249+0000 7fdfa8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:37:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.250+0000 7fdfa8ff9700 1 -- 172.21.15.40:0/1156676430 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb809b470 con 0x55711a896af0 2023-07-18T03:37:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.250+0000 7fdfa8ff9700 1 -- 172.21.15.40:0/1156676430 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fdfb809b010 con 0x55711a896af0 2023-07-18T03:37:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.250+0000 7fdfd3530b80 1 -- 172.21.15.40:0/1156676430 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdfb0021070 con 0x55711a896af0 2023-07-18T03:37:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.250+0000 7fdfa8ff9700 1 -- 172.21.15.40:0/1156676430 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfb809f040 con 0x55711a896af0 2023-07-18T03:37:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.250+0000 7fdfd3530b80 10 client.16464 fetch_fsmap learned FSMap version 47 2023-07-18T03:37:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.250+0000 7fdfd3530b80 1 -- 172.21.15.40:0/1156676430 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55711a6371f0 con 0x55711a896af0 2023-07-18T03:37:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.250+0000 7fdfa8ff9700 1 -- 172.21.15.40:0/1156676430 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfb0021070 con 0x55711a896af0 2023-07-18T03:37:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.250+0000 7fdfa8ff9700 1 -- 172.21.15.40:0/1156676430 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 47) v1 ==== 7522+0+0 (secure 0 0 0) 0x7fdfb80a2f40 con 0x55711a896af0 2023-07-18T03:37:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.250+0000 7fdfd3530b80 10 client.16464 fetch_fsmap finished waiting for FSMap version 47 2023-07-18T03:37:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.250+0000 7fdfd3530b80 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:37:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.250+0000 7fdfd3530b80 20 client.16464 populate_metadata read hostname 'smithi040' 2023-07-18T03:37:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.250+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/1156676430 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55711a8ff530 0x55711a901940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.250+0000 7fdfd3530b80 4 client.16464 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:37:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.250+0000 7fdfd3530b80 1 -- 172.21.15.40:0/1156676430 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55711a79dff0 con 0x55711a8ff530 2023-07-18T03:37:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.250+0000 7fdfc5146700 1 --2- 172.21.15.40:0/1156676430 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55711a8ff530 0x55711a901940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.251+0000 7fdfa8ff9700 10 client.16464 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.251+0000 7fdfc5146700 1 --2- 172.21.15.40:0/1156676430 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55711a8ff530 0x55711a901940 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5626eef453898f25 server_cookie=c8929146fab8b1c0 in_seq=0 out_seq=0 2023-07-18T03:37:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.315+0000 7fdfa8ff9700 1 -- 172.21.15.40:0/1156676430 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55711a79dff0 con 0x55711a8ff530 2023-07-18T03:37:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.315+0000 7fdfa8ff9700 10 client.16464 handle_command_reply: tid=0 2023-07-18T03:37:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.315+0000 7fdfd3530b80 1 client.16464 shutdown 2023-07-18T03:37:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.315+0000 7fdfd3530b80 1 -- 172.21.15.40:0/1156676430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a896af0 msgr2=0x55711a8a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.315+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/1156676430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a896af0 0x55711a8a5130 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdfb809afa0 tx=0x7fdfb800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.315+0000 7fdfd3530b80 1 -- 172.21.15.40:0/1156676430 shutdown_connections 2023-07-18T03:37:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.315+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/1156676430 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55711a8ff530 0x55711a901940 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.315+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/1156676430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55711a896af0 0x55711a8a5130 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.315+0000 7fdfa8ff9700 0 client.16464 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.315+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/1156676430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55711a6ed6b0 0x55711a8a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.315+0000 7fdfd3530b80 1 --2- 172.21.15.40:0/1156676430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55711a895c50 0x55711a8a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.315+0000 7fdfd3530b80 1 -- 172.21.15.40:0/1156676430 >> 172.21.15.40:0/1156676430 conn(0x55711a886be0 msgr2=0x55711a89d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.315+0000 7fdfd3530b80 1 -- 172.21.15.40:0/1156676430 shutdown_connections 2023-07-18T03:37:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.315+0000 7fdfd3530b80 1 -- 172.21.15.40:0/1156676430 wait complete. 2023-07-18T03:37:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.315+0000 7fdfd3530b80 20 client.16464 trim_cache size 0 max 16384 2023-07-18T03:37:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:37:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:37:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:37:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:37:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.316+0000 7fdfc73aa700 1 -- 172.21.15.40:0/3156951554 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdfac0a1150 msgr2=0x7fdfac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.316+0000 7fdfc73aa700 1 --2- 172.21.15.40:0/3156951554 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdfac0a1150 0x7fdfac0a3600 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009d20 tx=0x7fdfb0009260 comp rx=0 tx=0).stop 2023-07-18T03:37:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.316+0000 7fdfc73aa700 1 -- 172.21.15.40:0/3156951554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfc0100080 msgr2=0x7fdfc0196e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.316+0000 7fdfc73aa700 1 --2- 172.21.15.40:0/3156951554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfc0100080 0x7fdfc0196e20 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009fd0 tx=0x7fdfb8007930 comp rx=0 tx=0).stop 2023-07-18T03:37:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.316+0000 7fdfc73aa700 1 -- 172.21.15.40:0/3156951554 shutdown_connections 2023-07-18T03:37:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.316+0000 7fdfc73aa700 1 --2- 172.21.15.40:0/3156951554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfc0100080 0x7fdfc0196e20 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.316+0000 7fdfc73aa700 1 --2- 172.21.15.40:0/3156951554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfc00fec40 0x7fdfc0195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.316+0000 7fdfc73aa700 1 --2- 172.21.15.40:0/3156951554 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fdfac0a1150 0x7fdfac0a3600 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.316+0000 7fdfc73aa700 1 --2- 172.21.15.40:0/3156951554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfc00ff5f0 0x7fdfc0195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.316+0000 7fdfc73aa700 1 -- 172.21.15.40:0/3156951554 >> 172.21.15.40:0/3156951554 conn(0x7fdfc00f9780 msgr2=0x7fdfc00fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.316+0000 7fdfc73aa700 1 -- 172.21.15.40:0/3156951554 shutdown_connections 2023-07-18T03:37:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:32.316+0000 7fdfc73aa700 1 -- 172.21.15.40:0/3156951554 wait complete. 2023-07-18T03:37:33.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:32 smithi134 ceph-mon[98216]: pgmap v494: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 4.6 MiB/s wr, 327 op/s 2023-07-18T03:37:33.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:32 smithi040 ceph-mon[96439]: pgmap v494: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 4.6 MiB/s wr, 327 op/s 2023-07-18T03:37:33.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:32 smithi064 ceph-mon[98059]: pgmap v494: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 4.6 MiB/s wr, 327 op/s 2023-07-18T03:37:35.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:35 smithi134 ceph-mon[98216]: pgmap v495: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2023-07-18T03:37:35.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:35 smithi040 ceph-mon[96439]: pgmap v495: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2023-07-18T03:37:35.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:35 smithi064 ceph-mon[98059]: pgmap v495: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2023-07-18T03:37:37.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:37 smithi134 ceph-mon[98216]: pgmap v496: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 3.7 MiB/s wr, 249 op/s 2023-07-18T03:37:37.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:37 smithi040 ceph-mon[96439]: pgmap v496: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 3.7 MiB/s wr, 249 op/s 2023-07-18T03:37:37.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:37 smithi064 ceph-mon[98059]: pgmap v496: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 3.7 MiB/s wr, 249 op/s 2023-07-18T03:37:38.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:38 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:37:38.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:38 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:37:38.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:38 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:37:39.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:39 smithi134 ceph-mon[98216]: pgmap v497: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 4.2 MiB/s wr, 274 op/s 2023-07-18T03:37:39.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:39 smithi040 ceph-mon[96439]: pgmap v497: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 4.2 MiB/s wr, 274 op/s 2023-07-18T03:37:39.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:39 smithi064 ceph-mon[98059]: pgmap v497: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 4.2 MiB/s wr, 274 op/s 2023-07-18T03:37:41.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:41 smithi134 ceph-mon[98216]: pgmap v498: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 2.7 MiB/s wr, 180 op/s 2023-07-18T03:37:41.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:41 smithi040 ceph-mon[96439]: pgmap v498: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 2.7 MiB/s wr, 180 op/s 2023-07-18T03:37:41.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:41 smithi064 ceph-mon[98059]: pgmap v498: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 2.7 MiB/s wr, 180 op/s 2023-07-18T03:37:43.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:42 smithi134 ceph-mon[98216]: pgmap v499: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 197 op/s 2023-07-18T03:37:43.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:42 smithi040 ceph-mon[96439]: pgmap v499: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 197 op/s 2023-07-18T03:37:43.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:42 smithi064 ceph-mon[98059]: pgmap v499: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 197 op/s 2023-07-18T03:37:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.438+0000 7f5abae4e700 1 -- 172.21.15.40:0/2927976317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab40ffc30 msgr2=0x7f5ab4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.438+0000 7f5abae4e700 1 --2- 172.21.15.40:0/2927976317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab40ffc30 0x7f5ab4100080 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009990 tx=0x7f5aa0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.438+0000 7f5abae4e700 1 -- 172.21.15.40:0/2927976317 shutdown_connections 2023-07-18T03:37:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.438+0000 7f5abae4e700 1 --2- 172.21.15.40:0/2927976317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab41005c0 0x7f5ab4106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.438+0000 7f5abae4e700 1 --2- 172.21.15.40:0/2927976317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab40fea40 0x7f5ab40fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.438+0000 7f5abae4e700 1 --2- 172.21.15.40:0/2927976317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab40ffc30 0x7f5ab4100080 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.438+0000 7f5abae4e700 1 -- 172.21.15.40:0/2927976317 >> 172.21.15.40:0/2927976317 conn(0x7f5ab40fa060 msgr2=0x7f5ab40fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.438+0000 7f5abae4e700 1 -- 172.21.15.40:0/2927976317 shutdown_connections 2023-07-18T03:37:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.438+0000 7f5abae4e700 1 -- 172.21.15.40:0/2927976317 wait complete. 2023-07-18T03:37:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.439+0000 7f5abae4e700 1 Processor -- start 2023-07-18T03:37:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.439+0000 7f5abae4e700 1 -- start start 2023-07-18T03:37:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.439+0000 7f5abae4e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab40fea40 0x7f5ab419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.439+0000 7f5abae4e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab40ffc30 0x7f5ab419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.439+0000 7f5abae4e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab41005c0 0x7f5ab419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.439+0000 7f5abae4e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab4069370 con 0x7f5ab40ffc30 2023-07-18T03:37:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.439+0000 7f5abae4e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab419f9b0 con 0x7f5ab40fea40 2023-07-18T03:37:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.439+0000 7f5abae4e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab419fb20 con 0x7f5ab41005c0 2023-07-18T03:37:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.439+0000 7f5ab3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab40ffc30 0x7f5ab419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.439+0000 7f5ab8bea700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab40fea40 0x7f5ab419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.439+0000 7f5ab93eb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab41005c0 0x7f5ab419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.439+0000 7f5ab8bea700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab40fea40 0x7f5ab419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43896/0 (socket says 172.21.15.40:43896) 2023-07-18T03:37:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.439+0000 7f5ab93eb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab41005c0 0x7f5ab419f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49602/0 (socket says 172.21.15.40:49602) 2023-07-18T03:37:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.439+0000 7f5ab3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab40ffc30 0x7f5ab419cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60572/0 (socket says 172.21.15.40:60572) 2023-07-18T03:37:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.439+0000 7f5ab8bea700 1 -- 172.21.15.40:0/1879050563 learned_addr learned my addr 172.21.15.40:0/1879050563 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.440+0000 7f5ab3fff700 1 -- 172.21.15.40:0/1879050563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab41005c0 msgr2=0x7f5ab419f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.440+0000 7f5ab3fff700 1 --2- 172.21.15.40:0/1879050563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab41005c0 0x7f5ab419f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.440+0000 7f5ab3fff700 1 -- 172.21.15.40:0/1879050563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab40fea40 msgr2=0x7f5ab419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.440+0000 7f5ab3fff700 1 --2- 172.21.15.40:0/1879050563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab40fea40 0x7f5ab419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.440+0000 7f5ab3fff700 1 -- 172.21.15.40:0/1879050563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa00095f0 con 0x7f5ab40ffc30 2023-07-18T03:37:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.440+0000 7f5ab8bea700 1 --2- 172.21.15.40:0/1879050563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab40fea40 0x7f5ab419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.440+0000 7f5ab3fff700 1 --2- 172.21.15.40:0/1879050563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab40ffc30 0x7f5ab419cf20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000b620 tx=0x7f5aa0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.440+0000 7f5ab1ffb700 1 -- 172.21.15.40:0/1879050563 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa001d070 con 0x7f5ab40ffc30 2023-07-18T03:37:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.440+0000 7f5ab1ffb700 1 -- 172.21.15.40:0/1879050563 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5aa0021410 con 0x7f5ab40ffc30 2023-07-18T03:37:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.440+0000 7f5abae4e700 1 -- 172.21.15.40:0/1879050563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab419fda0 con 0x7f5ab40ffc30 2023-07-18T03:37:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.440+0000 7f5ab1ffb700 1 -- 172.21.15.40:0/1879050563 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0017650 con 0x7f5ab40ffc30 2023-07-18T03:37:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.440+0000 7f5abae4e700 1 -- 172.21.15.40:0/1879050563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab41a01c0 con 0x7f5ab40ffc30 2023-07-18T03:37:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.442+0000 7f5ab1ffb700 1 -- 172.21.15.40:0/1879050563 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5aa00217f0 con 0x7f5ab40ffc30 2023-07-18T03:37:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.443+0000 7f5ab1ffb700 1 --2- 172.21.15.40:0/1879050563 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5aa40a1150 0x7f5aa40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.443+0000 7f5ab1ffb700 1 -- 172.21.15.40:0/1879050563 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5aa00cdca0 con 0x7f5ab40ffc30 2023-07-18T03:37:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.443+0000 7f5ab8bea700 1 --2- 172.21.15.40:0/1879050563 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5aa40a1150 0x7f5aa40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.444+0000 7f5ab8bea700 1 --2- 172.21.15.40:0/1879050563 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5aa40a1150 0x7f5aa40a3600 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4068ee0 tx=0x7f5aa8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.444+0000 7f5ac6fd4b80 1 Processor -- start 2023-07-18T03:37:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.444+0000 7f5ac6fd4b80 1 -- start start 2023-07-18T03:37:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.444+0000 7f5ac6fd4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea82606b0 0x557ea8404300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.444+0000 7f5ac6fd4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea8404840 0x557ea84051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.444+0000 7f5ac6fd4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea84056e0 0x557ea8405b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.444+0000 7f5ac6fd4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea8309c50 con 0x557ea84056e0 2023-07-18T03:37:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.444+0000 7f5ac6fd4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea835c9e0 con 0x557ea8404840 2023-07-18T03:37:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.444+0000 7f5ac6fd4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea8355e70 con 0x557ea82606b0 2023-07-18T03:37:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.445+0000 7f5ab8bea700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea82606b0 0x557ea8404300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.445+0000 7f5ab93eb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea84056e0 0x557ea8405b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.445+0000 7f5ab3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea8404840 0x557ea84051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.445+0000 7f5ab8bea700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea82606b0 0x557ea8404300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49606/0 (socket says 172.21.15.40:49606) 2023-07-18T03:37:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.445+0000 7f5ab3fff700 1 --2- 172.21.15.40:0/1094791653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea8404840 0x557ea84051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43906/0 (socket says 172.21.15.40:43906) 2023-07-18T03:37:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.445+0000 7f5ab8bea700 1 -- 172.21.15.40:0/1094791653 learned_addr learned my addr 172.21.15.40:0/1094791653 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.445+0000 7f5ab93eb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea84056e0 0x557ea8405b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60582/0 (socket says 172.21.15.40:60582) 2023-07-18T03:37:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.445+0000 7f5ab93eb700 1 -- 172.21.15.40:0/1094791653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea82606b0 msgr2=0x557ea8404300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.445+0000 7f5ab93eb700 1 --2- 172.21.15.40:0/1094791653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea82606b0 0x557ea8404300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.445+0000 7f5ab93eb700 1 -- 172.21.15.40:0/1094791653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea8404840 msgr2=0x557ea84051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.445+0000 7f5ab93eb700 1 --2- 172.21.15.40:0/1094791653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea8404840 0x557ea84051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.445+0000 7f5ab93eb700 1 -- 172.21.15.40:0/1094791653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ea8183d80 con 0x557ea84056e0 2023-07-18T03:37:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.445+0000 7f5ab93eb700 1 --2- 172.21.15.40:0/1094791653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea84056e0 0x557ea8405b80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5aac011580 tx=0x7f5aac009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c792d471d32b9f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.445+0000 7f5a9effd700 1 -- 172.21.15.40:0/1094791653 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00e040 con 0x557ea84056e0 2023-07-18T03:37:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.445+0000 7f5a9effd700 1 -- 172.21.15.40:0/1094791653 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5aac00f040 con 0x557ea84056e0 2023-07-18T03:37:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.445+0000 7f5a9effd700 1 -- 172.21.15.40:0/1094791653 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac0043f0 con 0x557ea84056e0 2023-07-18T03:37:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.446+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/1094791653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea84056e0 msgr2=0x557ea8405b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.446+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/1094791653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea84056e0 0x557ea8405b80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5aac011580 tx=0x7f5aac009920 comp rx=0 tx=0).stop 2023-07-18T03:37:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.446+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/1094791653 shutdown_connections 2023-07-18T03:37:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.446+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/1094791653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea84056e0 0x557ea8405b80 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.446+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/1094791653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea82606b0 0x557ea8404300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.446+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/1094791653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea8404840 0x557ea84051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.446+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/1094791653 >> 172.21.15.40:0/1094791653 conn(0x557ea83f9be0 msgr2=0x557ea83fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.446+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/1094791653 shutdown_connections 2023-07-18T03:37:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.446+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/1094791653 wait complete. 2023-07-18T03:37:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.446+0000 7f5ac6fd4b80 1 -- start start 2023-07-18T03:37:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.446+0000 7f5ac6fd4b80 1 Processor -- start 2023-07-18T03:37:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.447+0000 7f5ac6fd4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea82606b0 0x557ea840d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.447+0000 7f5ac6fd4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea8404840 0x557ea840dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.447+0000 7f5ac6fd4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea84056e0 0x557ea8414d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.447+0000 7f5ac6fd4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea82f5a60 con 0x557ea8404840 2023-07-18T03:37:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.447+0000 7f5ac6fd4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea82f5760 con 0x557ea84056e0 2023-07-18T03:37:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.447+0000 7f5ac6fd4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea81d98c0 con 0x557ea82606b0 2023-07-18T03:37:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.447+0000 7f5ac6fd4b80 10 client.0 _ll_register_callbacks cb 0x557ea82f3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.447+0000 7f5ab3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea8404840 0x557ea840dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.447+0000 7f5ab8bea700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea82606b0 0x557ea840d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.447+0000 7f5ab3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea8404840 0x557ea840dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60586/0 (socket says 172.21.15.40:60586) 2023-07-18T03:37:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.447+0000 7f5ab3fff700 1 -- 172.21.15.40:0/3080956253 learned_addr learned my addr 172.21.15.40:0/3080956253 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.447+0000 7f5ab93eb700 1 --2- 172.21.15.40:0/3080956253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea84056e0 0x557ea8414d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.447+0000 7f5ab3fff700 1 -- 172.21.15.40:0/3080956253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea82606b0 msgr2=0x557ea840d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.447+0000 7f5ab3fff700 1 --2- 172.21.15.40:0/3080956253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea82606b0 0x557ea840d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.447+0000 7f5ab3fff700 1 -- 172.21.15.40:0/3080956253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea84056e0 msgr2=0x557ea8414d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.447+0000 7f5ab3fff700 1 --2- 172.21.15.40:0/3080956253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea84056e0 0x557ea8414d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.447+0000 7f5ab3fff700 1 -- 172.21.15.40:0/3080956253 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aac011230 con 0x557ea8404840 2023-07-18T03:37:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.448+0000 7f5ab3fff700 1 --2- 172.21.15.40:0/3080956253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea8404840 0x557ea840dd50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5aa009b0b0 tx=0x7f5aa009b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5607d294e1aafa server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.448+0000 7f5a93fff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:37:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.448+0000 7f5a93fff700 1 -- 172.21.15.40:0/3080956253 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa009b120 con 0x557ea8404840 2023-07-18T03:37:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.448+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/3080956253 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ea82af490 con 0x557ea8404840 2023-07-18T03:37:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.448+0000 7f5a93fff700 1 -- 172.21.15.40:0/3080956253 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5aa009f150 con 0x557ea8404840 2023-07-18T03:37:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.448+0000 7f5a93fff700 1 -- 172.21.15.40:0/3080956253 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5aa00a34e0 con 0x557ea8404840 2023-07-18T03:37:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.448+0000 7f5ac6fd4b80 10 client.16476 fetch_fsmap learned FSMap version 48 2023-07-18T03:37:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.448+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/3080956253 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x557ea816cc00 con 0x557ea8404840 2023-07-18T03:37:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.448+0000 7f5a93fff700 1 -- 172.21.15.40:0/3080956253 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5aa00c9080 con 0x557ea8404840 2023-07-18T03:37:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.448+0000 7f5a93fff700 1 -- 172.21.15.40:0/3080956253 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 48) v1 ==== 7518+0+0 (secure 0 0 0) 0x7f5aa000fad0 con 0x557ea8404840 2023-07-18T03:37:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.448+0000 7f5ac6fd4b80 10 client.16476 fetch_fsmap finished waiting for FSMap version 48 2023-07-18T03:37:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.448+0000 7f5ac6fd4b80 10 client.16476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:37:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.448+0000 7f5ac6fd4b80 20 client.16476 populate_metadata read hostname 'smithi040' 2023-07-18T03:37:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.448+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/3080956253 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557ea8419420 0x557ea841b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.448+0000 7f5ac6fd4b80 4 client.16476 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:37:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.448+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/3080956253 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ea834ee50 con 0x557ea8419420 2023-07-18T03:37:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.449+0000 7f5ab8bea700 1 --2- 172.21.15.40:0/3080956253 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557ea8419420 0x557ea841b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.449+0000 7f5ab8bea700 1 --2- 172.21.15.40:0/3080956253 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557ea8419420 0x557ea841b830 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66cb5a61460974b9 server_cookie=72bdf6c760332a1c in_seq=0 out_seq=0 2023-07-18T03:37:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.449+0000 7f5a93fff700 10 client.16476 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.450+0000 7f5a93fff700 1 -- 172.21.15.40:0/3080956253 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557ea834ee50 con 0x557ea8419420 2023-07-18T03:37:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.450+0000 7f5a93fff700 10 client.16476 handle_command_reply: tid=0 2023-07-18T03:37:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.450+0000 7f5ac6fd4b80 1 client.16476 shutdown 2023-07-18T03:37:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.450+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/3080956253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea8404840 msgr2=0x557ea840dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.450+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/3080956253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea8404840 0x557ea840dd50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5aa009b0b0 tx=0x7f5aa009b660 comp rx=0 tx=0).stop 2023-07-18T03:37:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.450+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/3080956253 shutdown_connections 2023-07-18T03:37:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.450+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/3080956253 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557ea8419420 0x557ea841b830 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.450+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/3080956253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea84056e0 0x557ea8414d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.450+0000 7f5a93fff700 0 client.16476 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.450+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/3080956253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea82606b0 0x557ea840d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.450+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/3080956253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea8404840 0x557ea840dd50 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.450+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/3080956253 >> 172.21.15.40:0/3080956253 conn(0x557ea83f9be0 msgr2=0x557ea83fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.450+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/3080956253 shutdown_connections 2023-07-18T03:37:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.450+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/3080956253 wait complete. 2023-07-18T03:37:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.450+0000 7f5ac6fd4b80 20 client.16476 trim_cache size 0 max 16384 2023-07-18T03:37:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.465+0000 7f5ac6fd4b80 1 Processor -- start 2023-07-18T03:37:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.465+0000 7f5ac6fd4b80 1 -- start start 2023-07-18T03:37:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.465+0000 7f5ac6fd4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea82606b0 0x557ea840b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.465+0000 7f5ac6fd4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea8404840 0x557ea8414d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.465+0000 7f5ac6fd4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea84056e0 0x557ea840bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.465+0000 7f5ac6fd4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea834ee50 con 0x557ea82606b0 2023-07-18T03:37:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.465+0000 7f5ac6fd4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea83ffa50 con 0x557ea84056e0 2023-07-18T03:37:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.465+0000 7f5ac6fd4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea83ca060 con 0x557ea8404840 2023-07-18T03:37:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.465+0000 7f5ab8bea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea82606b0 0x557ea840b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.465+0000 7f5ab8bea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea82606b0 0x557ea840b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60592/0 (socket says 172.21.15.40:60592) 2023-07-18T03:37:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.465+0000 7f5ab8bea700 1 -- 172.21.15.40:0/4043255318 learned_addr learned my addr 172.21.15.40:0/4043255318 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.465+0000 7f5ab3fff700 1 --2- 172.21.15.40:0/4043255318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea8404840 0x557ea8414d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.465+0000 7f5ab93eb700 1 --2- 172.21.15.40:0/4043255318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea84056e0 0x557ea840bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.466+0000 7f5ab8bea700 1 -- 172.21.15.40:0/4043255318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea8404840 msgr2=0x557ea8414d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.466+0000 7f5ab8bea700 1 --2- 172.21.15.40:0/4043255318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea8404840 0x557ea8414d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.466+0000 7f5ab8bea700 1 -- 172.21.15.40:0/4043255318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea84056e0 msgr2=0x557ea840bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.466+0000 7f5ab8bea700 1 --2- 172.21.15.40:0/4043255318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea84056e0 0x557ea840bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.466+0000 7f5ab8bea700 1 -- 172.21.15.40:0/4043255318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ea81aa1f0 con 0x557ea82606b0 2023-07-18T03:37:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.466+0000 7f5ab8bea700 1 --2- 172.21.15.40:0/4043255318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea82606b0 0x557ea840b9c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x557ea841c0d0 tx=0x7f5aa8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ae55bc884490c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.466+0000 7f5a9f7fe700 1 -- 172.21.15.40:0/4043255318 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8024040 con 0x557ea82606b0 2023-07-18T03:37:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.466+0000 7f5a9f7fe700 1 -- 172.21.15.40:0/4043255318 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5aa8029e10 con 0x557ea82606b0 2023-07-18T03:37:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.466+0000 7f5a9f7fe700 1 -- 172.21.15.40:0/4043255318 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8004a50 con 0x557ea82606b0 2023-07-18T03:37:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.467+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/4043255318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea82606b0 msgr2=0x557ea840b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.467+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/4043255318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea82606b0 0x557ea840b9c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x557ea841c0d0 tx=0x7f5aa8014000 comp rx=0 tx=0).stop 2023-07-18T03:37:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.467+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/4043255318 shutdown_connections 2023-07-18T03:37:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.467+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/4043255318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea84056e0 0x557ea840bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.467+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/4043255318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea82606b0 0x557ea840b9c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.467+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/4043255318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea8404840 0x557ea8414d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.467+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/4043255318 >> 172.21.15.40:0/4043255318 conn(0x557ea83f9be0 msgr2=0x557ea84084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.467+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/4043255318 shutdown_connections 2023-07-18T03:37:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.467+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/4043255318 wait complete. 2023-07-18T03:37:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.467+0000 7f5ac6fd4b80 1 -- start start 2023-07-18T03:37:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.467+0000 7f5ac6fd4b80 1 Processor -- start 2023-07-18T03:37:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.467+0000 7f5ac6fd4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea82606b0 0x557ea8410bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.467+0000 7f5ac6fd4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea8404840 0x557ea84110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.467+0000 7f5ac6fd4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea84056e0 0x557ea8414110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.467+0000 7f5ac6fd4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea8414650 con 0x557ea84056e0 2023-07-18T03:37:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.467+0000 7f5ac6fd4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea84147c0 con 0x557ea82606b0 2023-07-18T03:37:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.467+0000 7f5ac6fd4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ea8414980 con 0x557ea8404840 2023-07-18T03:37:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.468+0000 7f5ac6fd4b80 10 client.0 _ll_register_callbacks cb 0x7f5aa009aef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.468+0000 7f5ab8bea700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea82606b0 0x557ea8410bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.468+0000 7f5ab3fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea8404840 0x557ea84110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.468+0000 7f5ab8bea700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea82606b0 0x557ea8410bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43924/0 (socket says 172.21.15.40:43924) 2023-07-18T03:37:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.468+0000 7f5ab8bea700 1 -- 172.21.15.40:0/4262613909 learned_addr learned my addr 172.21.15.40:0/4262613909 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.468+0000 7f5ab93eb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea84056e0 0x557ea8414110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.468+0000 7f5ab93eb700 1 -- 172.21.15.40:0/4262613909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea8404840 msgr2=0x557ea84110f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.468+0000 7f5ab93eb700 1 --2- 172.21.15.40:0/4262613909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea8404840 0x557ea84110f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.468+0000 7f5ab93eb700 1 -- 172.21.15.40:0/4262613909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea82606b0 msgr2=0x557ea8410bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.468+0000 7f5ab93eb700 1 --2- 172.21.15.40:0/4262613909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea82606b0 0x557ea8410bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.468+0000 7f5ab93eb700 1 -- 172.21.15.40:0/4262613909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa8017070 con 0x557ea84056e0 2023-07-18T03:37:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.468+0000 7f5ab93eb700 1 --2- 172.21.15.40:0/4262613909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea84056e0 0x557ea8414110 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5aac011550 tx=0x7f5aac009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54e88ad78a80800 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.468+0000 7f5a93fff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:37:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.468+0000 7f5a93fff700 1 -- 172.21.15.40:0/4262613909 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00e040 con 0x557ea84056e0 2023-07-18T03:37:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.468+0000 7f5a93fff700 1 -- 172.21.15.40:0/4262613909 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f5aac0055a0 con 0x557ea84056e0 2023-07-18T03:37:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.468+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/4262613909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ea840cb20 con 0x557ea84056e0 2023-07-18T03:37:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.468+0000 7f5a93fff700 1 -- 172.21.15.40:0/4262613909 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5aac0057a0 con 0x557ea84056e0 2023-07-18T03:37:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.469+0000 7f5ac6fd4b80 10 client.16482 fetch_fsmap learned FSMap version 48 2023-07-18T03:37:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.469+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/4262613909 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x557ea8182250 con 0x557ea84056e0 2023-07-18T03:37:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.469+0000 7f5a93fff700 1 -- 172.21.15.40:0/4262613909 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5aac035080 con 0x557ea84056e0 2023-07-18T03:37:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.469+0000 7f5a93fff700 1 -- 172.21.15.40:0/4262613909 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 48) v1 ==== 7518+0+0 (secure 0 0 0) 0x7f5aac02f030 con 0x557ea84056e0 2023-07-18T03:37:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.469+0000 7f5ac6fd4b80 10 client.16482 fetch_fsmap finished waiting for FSMap version 48 2023-07-18T03:37:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.469+0000 7f5ac6fd4b80 10 client.16482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:37:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.469+0000 7f5ac6fd4b80 20 client.16482 populate_metadata read hostname 'smithi040' 2023-07-18T03:37:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.469+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/4262613909 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557ea846e300 0x557ea8470710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.469+0000 7f5ac6fd4b80 4 client.16482 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:37:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.469+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/4262613909 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f5aac00f040 con 0x557ea846e300 2023-07-18T03:37:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.469+0000 7f5ab8bea700 1 --2- 172.21.15.40:0/4262613909 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557ea846e300 0x557ea8470710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.469+0000 7f5a93fff700 10 client.16482 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.469+0000 7f5ab8bea700 1 --2- 172.21.15.40:0/4262613909 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557ea846e300 0x557ea8470710 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a960b14e70b1271f server_cookie=25ef08d9c8a08594 in_seq=0 out_seq=0 2023-07-18T03:37:45.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:45 smithi134 ceph-mon[98216]: pgmap v500: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 132 op/s 2023-07-18T03:37:45.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:45 smithi040 ceph-mon[96439]: pgmap v500: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 132 op/s 2023-07-18T03:37:45.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:45 smithi064 ceph-mon[98059]: pgmap v500: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 132 op/s 2023-07-18T03:37:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.627+0000 7f5a93fff700 1 -- 172.21.15.40:0/4262613909 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f5aac00f040 con 0x557ea846e300 2023-07-18T03:37:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.627+0000 7f5a93fff700 10 client.16482 handle_command_reply: tid=0 2023-07-18T03:37:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.627+0000 7f5ac6fd4b80 1 client.16482 shutdown 2023-07-18T03:37:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.628+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/4262613909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea84056e0 msgr2=0x557ea8414110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.628+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/4262613909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea84056e0 0x557ea8414110 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5aac011550 tx=0x7f5aac009510 comp rx=0 tx=0).stop 2023-07-18T03:37:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.628+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/4262613909 shutdown_connections 2023-07-18T03:37:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.628+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/4262613909 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557ea846e300 0x557ea8470710 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.628+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/4262613909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557ea84056e0 0x557ea8414110 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.628+0000 7f5a93fff700 0 client.16482 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.628+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/4262613909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ea82606b0 0x557ea8410bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.628+0000 7f5ac6fd4b80 1 --2- 172.21.15.40:0/4262613909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557ea8404840 0x557ea84110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.628+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/4262613909 >> 172.21.15.40:0/4262613909 conn(0x557ea83f9be0 msgr2=0x557ea840d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.628+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/4262613909 shutdown_connections 2023-07-18T03:37:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.628+0000 7f5ac6fd4b80 1 -- 172.21.15.40:0/4262613909 wait complete. 2023-07-18T03:37:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.628+0000 7f5ac6fd4b80 20 client.16482 trim_cache size 0 max 16384 2023-07-18T03:37:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:37:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:37:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:37:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:37:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.629+0000 7f5abae4e700 1 -- 172.21.15.40:0/1879050563 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5aa40a1150 msgr2=0x7f5aa40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.629+0000 7f5abae4e700 1 --2- 172.21.15.40:0/1879050563 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5aa40a1150 0x7f5aa40a3600 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4068ee0 tx=0x7f5aa8009260 comp rx=0 tx=0).stop 2023-07-18T03:37:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.629+0000 7f5abae4e700 1 -- 172.21.15.40:0/1879050563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab40ffc30 msgr2=0x7f5ab419cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.629+0000 7f5abae4e700 1 --2- 172.21.15.40:0/1879050563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab40ffc30 0x7f5ab419cf20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000b620 tx=0x7f5aa0004730 comp rx=0 tx=0).stop 2023-07-18T03:37:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.629+0000 7f5abae4e700 1 -- 172.21.15.40:0/1879050563 shutdown_connections 2023-07-18T03:37:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.629+0000 7f5abae4e700 1 --2- 172.21.15.40:0/1879050563 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f5aa40a1150 0x7f5aa40a3600 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.629+0000 7f5abae4e700 1 --2- 172.21.15.40:0/1879050563 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ab41005c0 0x7f5ab419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.629+0000 7f5abae4e700 1 --2- 172.21.15.40:0/1879050563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ab40fea40 0x7f5ab419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.629+0000 7f5abae4e700 1 --2- 172.21.15.40:0/1879050563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab40ffc30 0x7f5ab419cf20 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.629+0000 7f5abae4e700 1 -- 172.21.15.40:0/1879050563 >> 172.21.15.40:0/1879050563 conn(0x7f5ab40fa060 msgr2=0x7f5ab40fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.629+0000 7f5abae4e700 1 -- 172.21.15.40:0/1879050563 shutdown_connections 2023-07-18T03:37:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:45.629+0000 7f5abae4e700 1 -- 172.21.15.40:0/1879050563 wait complete. 2023-07-18T03:37:47.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:47 smithi134 ceph-mon[98216]: pgmap v501: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 122 op/s 2023-07-18T03:37:47.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:47 smithi040 ceph-mon[96439]: pgmap v501: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 122 op/s 2023-07-18T03:37:47.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:47 smithi064 ceph-mon[98059]: pgmap v501: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 122 op/s 2023-07-18T03:37:49.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:49 smithi134 ceph-mon[98216]: pgmap v502: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 127 op/s 2023-07-18T03:37:49.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:49 smithi040 ceph-mon[96439]: pgmap v502: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 127 op/s 2023-07-18T03:37:49.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:49 smithi064 ceph-mon[98059]: pgmap v502: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 127 op/s 2023-07-18T03:37:51.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:51 smithi134 ceph-mon[98216]: pgmap v503: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 72 op/s 2023-07-18T03:37:51.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:51 smithi040 ceph-mon[96439]: pgmap v503: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 72 op/s 2023-07-18T03:37:51.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:51 smithi064 ceph-mon[98059]: pgmap v503: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 72 op/s 2023-07-18T03:37:52.525 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:37:52.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.680+0000 7f37aaf31700 1 -- 172.21.15.40:0/3461687687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a40ffaf0 msgr2=0x7f37a40fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:52.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.680+0000 7f37aaf31700 1 --2- 172.21.15.40:0/3461687687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a40ffaf0 0x7f37a40fff00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3798009990 tx=0x7f3798009ca0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.680+0000 7f37aaf31700 1 -- 172.21.15.40:0/3461687687 shutdown_connections 2023-07-18T03:37:52.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.680+0000 7f37aaf31700 1 --2- 172.21.15.40:0/3461687687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37a4101670 0x7f37a4107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.680+0000 7f37aaf31700 1 --2- 172.21.15.40:0/3461687687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a40ffaf0 0x7f37a40fff00 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.680+0000 7f37aaf31700 1 --2- 172.21.15.40:0/3461687687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37a4100ce0 0x7f37a4101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.680+0000 7f37aaf31700 1 -- 172.21.15.40:0/3461687687 >> 172.21.15.40:0/3461687687 conn(0x7f37a40fb130 msgr2=0x7f37a40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:52.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.680+0000 7f37aaf31700 1 -- 172.21.15.40:0/3461687687 shutdown_connections 2023-07-18T03:37:52.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.680+0000 7f37aaf31700 1 -- 172.21.15.40:0/3461687687 wait complete. 2023-07-18T03:37:52.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37aaf31700 1 Processor -- start 2023-07-18T03:37:52.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37aaf31700 1 -- start start 2023-07-18T03:37:52.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37aaf31700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37a40ffaf0 0x7f37a4195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37aaf31700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a4100ce0 0x7f37a4195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37aaf31700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37a4101670 0x7f37a419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37aaf31700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a419d3a0 con 0x7f37a4100ce0 2023-07-18T03:37:52.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37aaf31700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a419d4e0 con 0x7f37a40ffaf0 2023-07-18T03:37:52.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37aaf31700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a419d670 con 0x7f37a4101670 2023-07-18T03:37:52.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a4100ce0 0x7f37a4195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37a8ccd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37a40ffaf0 0x7f37a4195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a4100ce0 0x7f37a4195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60614/0 (socket says 172.21.15.40:60614) 2023-07-18T03:37:52.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37a3fff700 1 -- 172.21.15.40:0/1637576093 learned_addr learned my addr 172.21.15.40:0/1637576093 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:52.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37a94ce700 1 --2- 172.21.15.40:0/1637576093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37a4101670 0x7f37a419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37a3fff700 1 -- 172.21.15.40:0/1637576093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37a4101670 msgr2=0x7f37a419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:52.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37a3fff700 1 --2- 172.21.15.40:0/1637576093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37a4101670 0x7f37a419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37a3fff700 1 -- 172.21.15.40:0/1637576093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37a40ffaf0 msgr2=0x7f37a4195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:52.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37a3fff700 1 --2- 172.21.15.40:0/1637576093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37a40ffaf0 0x7f37a4195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.681+0000 7f37a3fff700 1 -- 172.21.15.40:0/1637576093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37980095f0 con 0x7f37a4100ce0 2023-07-18T03:37:52.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.682+0000 7f37a3fff700 1 --2- 172.21.15.40:0/1637576093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a4100ce0 0x7f37a4195d60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f379000e9a0 tx=0x7f379000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:52.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.682+0000 7f37a1ffb700 1 -- 172.21.15.40:0/1637576093 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3790009790 con 0x7f37a4100ce0 2023-07-18T03:37:52.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.682+0000 7f37a1ffb700 1 -- 172.21.15.40:0/1637576093 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3790010430 con 0x7f37a4100ce0 2023-07-18T03:37:52.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.682+0000 7f37aaf31700 1 -- 172.21.15.40:0/1637576093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a419d950 con 0x7f37a4100ce0 2023-07-18T03:37:52.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.682+0000 7f37a1ffb700 1 -- 172.21.15.40:0/1637576093 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3790018680 con 0x7f37a4100ce0 2023-07-18T03:37:52.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.682+0000 7f37aaf31700 1 -- 172.21.15.40:0/1637576093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a419de60 con 0x7f37a4100ce0 2023-07-18T03:37:52.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.684+0000 7f37a1ffb700 1 -- 172.21.15.40:0/1637576093 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f37900105a0 con 0x7f37a4100ce0 2023-07-18T03:37:52.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.685+0000 7f37a1ffb700 1 --2- 172.21.15.40:0/1637576093 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f37940a1150 0x7f37940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.685+0000 7f37a1ffb700 1 -- 172.21.15.40:0/1637576093 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3790014070 con 0x7f37a4100ce0 2023-07-18T03:37:52.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.685+0000 7f37a8ccd700 1 --2- 172.21.15.40:0/1637576093 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f37940a1150 0x7f37940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.685+0000 7f37a8ccd700 1 --2- 172.21.15.40:0/1637576093 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f37940a1150 0x7f37940a3600 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3798006010 tx=0x7f3798005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:52.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37b70b7b80 1 Processor -- start 2023-07-18T03:37:52.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37b70b7b80 1 -- start start 2023-07-18T03:37:52.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37b70b7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 0x55c63897f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37b70b7b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897fe20 0x55c6389806e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37b70b7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c638980c70 0x55c6389810e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37b70b7b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c638884cb0 con 0x55c63897fe20 2023-07-18T03:37:52.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37b70b7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6388d7a40 con 0x55c63897f4d0 2023-07-18T03:37:52.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37b70b7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6388d0ed0 con 0x55c638980c70 2023-07-18T03:37:52.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897fe20 0x55c6389806e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37a8ccd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 0x55c63897f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897fe20 0x55c6389806e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60622/0 (socket says 172.21.15.40:60622) 2023-07-18T03:37:52.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37a3fff700 1 -- 172.21.15.40:0/2729582392 learned_addr learned my addr 172.21.15.40:0/2729582392 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:52.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37a94ce700 1 --2- 172.21.15.40:0/2729582392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c638980c70 0x55c6389810e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37a3fff700 1 -- 172.21.15.40:0/2729582392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c638980c70 msgr2=0x55c6389810e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:52.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37a3fff700 1 --2- 172.21.15.40:0/2729582392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c638980c70 0x55c6389810e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37a3fff700 1 -- 172.21.15.40:0/2729582392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 msgr2=0x55c63897f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:52.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37a3fff700 1 --2- 172.21.15.40:0/2729582392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 0x55c63897f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37a3fff700 1 -- 172.21.15.40:0/2729582392 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6386fede0 con 0x55c63897fe20 2023-07-18T03:37:52.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.686+0000 7f37a8ccd700 1 --2- 172.21.15.40:0/2729582392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 0x55c63897f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:52.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.687+0000 7f37a3fff700 1 --2- 172.21.15.40:0/2729582392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897fe20 0x55c6389806e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3790099e70 tx=0x7f3790099940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4fb4b742869073 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:52.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.687+0000 7f378effd700 1 -- 172.21.15.40:0/2729582392 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37900a2330 con 0x55c63897fe20 2023-07-18T03:37:52.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.687+0000 7f378effd700 1 -- 172.21.15.40:0/2729582392 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3790098280 con 0x55c63897fe20 2023-07-18T03:37:52.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.687+0000 7f378effd700 1 -- 172.21.15.40:0/2729582392 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37900a2100 con 0x55c63897fe20 2023-07-18T03:37:52.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.687+0000 7f37b70b7b80 1 -- 172.21.15.40:0/2729582392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897fe20 msgr2=0x55c6389806e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:52.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.687+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/2729582392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897fe20 0x55c6389806e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3790099e70 tx=0x7f3790099940 comp rx=0 tx=0).stop 2023-07-18T03:37:52.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.688+0000 7f37b70b7b80 1 -- 172.21.15.40:0/2729582392 shutdown_connections 2023-07-18T03:37:52.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.688+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/2729582392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c638980c70 0x55c6389810e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.688+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/2729582392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 0x55c63897f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.688+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/2729582392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897fe20 0x55c6389806e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.688+0000 7f37b70b7b80 1 -- 172.21.15.40:0/2729582392 >> 172.21.15.40:0/2729582392 conn(0x55c638974da0 msgr2=0x55c63897a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:52.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.688+0000 7f37b70b7b80 1 -- 172.21.15.40:0/2729582392 shutdown_connections 2023-07-18T03:37:52.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.688+0000 7f37b70b7b80 1 -- 172.21.15.40:0/2729582392 wait complete. 2023-07-18T03:37:52.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.688+0000 7f37b70b7b80 1 -- start start 2023-07-18T03:37:52.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.688+0000 7f37b70b7b80 1 Processor -- start 2023-07-18T03:37:52.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.688+0000 7f37b70b7b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897f4d0 0x55c6389891d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.688+0000 7f37b70b7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63897fe20 0x55c638989710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.688+0000 7f37b70b7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c638980c70 0x55c638990750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.688+0000 7f37b70b7b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c638870ac0 con 0x55c63897f4d0 2023-07-18T03:37:52.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f37b70b7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6388707c0 con 0x55c638980c70 2023-07-18T03:37:52.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f37b70b7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c638754920 con 0x55c63897fe20 2023-07-18T03:37:52.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f37b70b7b80 10 client.0 _ll_register_callbacks cb 0x55c63886e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:52.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f37a3fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63897fe20 0x55c638989710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f37a8ccd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897f4d0 0x55c6389891d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f37a3fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63897fe20 0x55c638989710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49656/0 (socket says 172.21.15.40:49656) 2023-07-18T03:37:52.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f37a3fff700 1 -- 172.21.15.40:0/1021798991 learned_addr learned my addr 172.21.15.40:0/1021798991 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:52.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f37a94ce700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c638980c70 0x55c638990750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f37a8ccd700 1 -- 172.21.15.40:0/1021798991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63897fe20 msgr2=0x55c638989710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:52.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f37a8ccd700 1 --2- 172.21.15.40:0/1021798991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63897fe20 0x55c638989710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f37a8ccd700 1 -- 172.21.15.40:0/1021798991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c638980c70 msgr2=0x55c638990750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:52.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f37a8ccd700 1 --2- 172.21.15.40:0/1021798991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c638980c70 0x55c638990750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f37a8ccd700 1 -- 172.21.15.40:0/1021798991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a4108860 con 0x55c63897f4d0 2023-07-18T03:37:52.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f37a8ccd700 1 --2- 172.21.15.40:0/1021798991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897f4d0 0x55c6389891d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f379800b620 tx=0x7f3798009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f65143e4ccb59e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:52.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f3783fff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:37:52.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f3783fff700 1 -- 172.21.15.40:0/1021798991 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798033070 con 0x55c63897f4d0 2023-07-18T03:37:52.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f37b70b7b80 1 -- 172.21.15.40:0/1021798991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c63882a4f0 con 0x55c63897f4d0 2023-07-18T03:37:52.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.689+0000 7f3783fff700 1 -- 172.21.15.40:0/1021798991 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f37980047d0 con 0x55c63897f4d0 2023-07-18T03:37:52.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.690+0000 7f3783fff700 1 -- 172.21.15.40:0/1021798991 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3798004990 con 0x55c63897f4d0 2023-07-18T03:37:52.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.690+0000 7f37b70b7b80 10 client.16494 fetch_fsmap learned FSMap version 48 2023-07-18T03:37:52.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.690+0000 7f37b70b7b80 1 -- 172.21.15.40:0/1021798991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x7f37a419de60 con 0x55c63897f4d0 2023-07-18T03:37:52.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.690+0000 7f3783fff700 1 -- 172.21.15.40:0/1021798991 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c63882a4f0 con 0x55c63897f4d0 2023-07-18T03:37:52.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.690+0000 7f3783fff700 1 -- 172.21.15.40:0/1021798991 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 48) v1 ==== 7518+0+0 (secure 0 0 0) 0x7f3798040020 con 0x55c63897f4d0 2023-07-18T03:37:52.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.690+0000 7f37b70b7b80 10 client.16494 fetch_fsmap finished waiting for FSMap version 48 2023-07-18T03:37:52.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.690+0000 7f37b70b7b80 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:37:52.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.690+0000 7f37b70b7b80 20 client.16494 populate_metadata read hostname 'smithi040' 2023-07-18T03:37:52.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.690+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/1021798991 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c638994f80 0x55c638997390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.690+0000 7f37b70b7b80 4 client.16494 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:37:52.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.690+0000 7f37b70b7b80 1 -- 172.21.15.40:0/1021798991 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6388c9eb0 con 0x55c638994f80 2023-07-18T03:37:52.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.690+0000 7f37a3fff700 1 --2- 172.21.15.40:0/1021798991 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c638994f80 0x55c638997390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.691+0000 7f37a3fff700 1 --2- 172.21.15.40:0/1021798991 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c638994f80 0x55c638997390 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57a1f5abb5896835 server_cookie=e26c84cb7b563fb7 in_seq=0 out_seq=0 2023-07-18T03:37:52.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.691+0000 7f3783fff700 10 client.16494 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:52.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.691+0000 7f3783fff700 1 -- 172.21.15.40:0/1021798991 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c6388c9eb0 con 0x55c638994f80 2023-07-18T03:37:52.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.691+0000 7f3783fff700 10 client.16494 handle_command_reply: tid=0 2023-07-18T03:37:52.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.691+0000 7f37b70b7b80 1 client.16494 shutdown 2023-07-18T03:37:52.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.692+0000 7f37b70b7b80 1 -- 172.21.15.40:0/1021798991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897f4d0 msgr2=0x55c6389891d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:52.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.692+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/1021798991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897f4d0 0x55c6389891d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f379800b620 tx=0x7f3798009ca0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.692+0000 7f37b70b7b80 1 -- 172.21.15.40:0/1021798991 shutdown_connections 2023-07-18T03:37:52.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.692+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/1021798991 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c638994f80 0x55c638997390 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.692+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/1021798991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c638980c70 0x55c638990750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.692+0000 7f3783fff700 0 client.16494 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:52.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.692+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/1021798991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897f4d0 0x55c6389891d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.692+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/1021798991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63897fe20 0x55c638989710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.692+0000 7f37b70b7b80 1 -- 172.21.15.40:0/1021798991 >> 172.21.15.40:0/1021798991 conn(0x55c638974da0 msgr2=0x55c6389798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:52.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.692+0000 7f37b70b7b80 1 -- 172.21.15.40:0/1021798991 shutdown_connections 2023-07-18T03:37:52.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.692+0000 7f37b70b7b80 1 -- 172.21.15.40:0/1021798991 wait complete. 2023-07-18T03:37:52.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.692+0000 7f37b70b7b80 20 client.16494 trim_cache size 0 max 16384 2023-07-18T03:37:52.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.709+0000 7f37b70b7b80 1 Processor -- start 2023-07-18T03:37:52.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.709+0000 7f37b70b7b80 1 -- start start 2023-07-18T03:37:52.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.709+0000 7f37b70b7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 0x55c6389810c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.709+0000 7f37b70b7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63897fe20 0x55c638990710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.709+0000 7f37b70b7b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c638980c70 0x55c638983c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.709+0000 7f37b70b7b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c638988510 con 0x55c638980c70 2023-07-18T03:37:52.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.709+0000 7f37b70b7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c638994b40 con 0x55c63897f4d0 2023-07-18T03:37:52.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.709+0000 7f37b70b7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6389923c0 con 0x55c63897fe20 2023-07-18T03:37:52.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.709+0000 7f37a94ce700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c638980c70 0x55c638983c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.709+0000 7f37a3fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63897fe20 0x55c638990710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.709+0000 7f37a8ccd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 0x55c6389810c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.709+0000 7f37a94ce700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c638980c70 0x55c638983c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60636/0 (socket says 172.21.15.40:60636) 2023-07-18T03:37:52.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.709+0000 7f37a94ce700 1 -- 172.21.15.40:0/2222835092 learned_addr learned my addr 172.21.15.40:0/2222835092 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:52.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.710+0000 7f37a8ccd700 1 -- 172.21.15.40:0/2222835092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63897fe20 msgr2=0x55c638990710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:52.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.710+0000 7f37a8ccd700 1 --2- 172.21.15.40:0/2222835092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63897fe20 0x55c638990710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.710+0000 7f37a8ccd700 1 -- 172.21.15.40:0/2222835092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c638980c70 msgr2=0x55c638983c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:52.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.710+0000 7f37a8ccd700 1 --2- 172.21.15.40:0/2222835092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c638980c70 0x55c638983c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.710+0000 7f37a8ccd700 1 -- 172.21.15.40:0/2222835092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6386e7c60 con 0x55c63897f4d0 2023-07-18T03:37:52.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.710+0000 7f37a94ce700 1 --2- 172.21.15.40:0/2222835092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c638980c70 0x55c638983c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:52.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.710+0000 7f37a3fff700 1 --2- 172.21.15.40:0/2222835092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63897fe20 0x55c638990710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:52.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.710+0000 7f37a8ccd700 1 --2- 172.21.15.40:0/2222835092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 0x55c6389810c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f379800b620 tx=0x7f3798009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc37020c8df5cb5d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:52.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.710+0000 7f378f7fe700 1 -- 172.21.15.40:0/2222835092 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798033070 con 0x55c63897f4d0 2023-07-18T03:37:52.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.710+0000 7f378f7fe700 1 -- 172.21.15.40:0/2222835092 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f379801e040 con 0x55c63897f4d0 2023-07-18T03:37:52.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.710+0000 7f378f7fe700 1 -- 172.21.15.40:0/2222835092 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798004990 con 0x55c63897f4d0 2023-07-18T03:37:52.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.711+0000 7f37b70b7b80 1 -- 172.21.15.40:0/2222835092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 msgr2=0x55c6389810c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:52.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.711+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/2222835092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 0x55c6389810c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f379800b620 tx=0x7f3798009290 comp rx=0 tx=0).stop 2023-07-18T03:37:52.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.711+0000 7f37b70b7b80 1 -- 172.21.15.40:0/2222835092 shutdown_connections 2023-07-18T03:37:52.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.711+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/2222835092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c638980c70 0x55c638983c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.711+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/2222835092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 0x55c6389810c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.711+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/2222835092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c63897fe20 0x55c638990710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.711+0000 7f37b70b7b80 1 -- 172.21.15.40:0/2222835092 >> 172.21.15.40:0/2222835092 conn(0x55c638974da0 msgr2=0x55c638991040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:52.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.711+0000 7f37b70b7b80 1 -- 172.21.15.40:0/2222835092 shutdown_connections 2023-07-18T03:37:52.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.711+0000 7f37b70b7b80 1 -- 172.21.15.40:0/2222835092 wait complete. 2023-07-18T03:37:52.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.711+0000 7f37b70b7b80 1 -- start start 2023-07-18T03:37:52.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.711+0000 7f37b70b7b80 1 Processor -- start 2023-07-18T03:37:52.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37b70b7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 0x55c638990520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37b70b7b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897fe20 0x55c63898eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37b70b7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c638980c70 0x55c63898f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37b70b7b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63898f650 con 0x55c63897fe20 2023-07-18T03:37:52.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37b70b7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63898f7c0 con 0x55c63897f4d0 2023-07-18T03:37:52.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37b70b7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c63898f980 con 0x55c638980c70 2023-07-18T03:37:52.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37b70b7b80 10 client.0 _ll_register_callbacks cb 0x7f379800bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:52.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897fe20 0x55c63898eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37a8ccd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 0x55c638990520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37a94ce700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c638980c70 0x55c63898f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897fe20 0x55c63898eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60640/0 (socket says 172.21.15.40:60640) 2023-07-18T03:37:52.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37a8ccd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 0x55c638990520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43964/0 (socket says 172.21.15.40:43964) 2023-07-18T03:37:52.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37a3fff700 1 -- 172.21.15.40:0/1214246928 learned_addr learned my addr 172.21.15.40:0/1214246928 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:52.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37a94ce700 1 --2- 172.21.15.40:0/1214246928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c638980c70 0x55c63898f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49670/0 (socket says 172.21.15.40:49670) 2023-07-18T03:37:52.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37a3fff700 1 -- 172.21.15.40:0/1214246928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c638980c70 msgr2=0x55c63898f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:52.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37a3fff700 1 --2- 172.21.15.40:0/1214246928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c638980c70 0x55c63898f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37a3fff700 1 -- 172.21.15.40:0/1214246928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 msgr2=0x55c638990520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:52.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37a3fff700 1 --2- 172.21.15.40:0/1214246928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 0x55c638990520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37a3fff700 1 -- 172.21.15.40:0/1214246928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6386fede0 con 0x55c63897fe20 2023-07-18T03:37:52.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.712+0000 7f37a8ccd700 1 --2- 172.21.15.40:0/1214246928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 0x55c638990520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:52.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.713+0000 7f37a3fff700 1 --2- 172.21.15.40:0/1214246928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897fe20 0x55c63898eb30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3790099e70 tx=0x7f3790099b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e06ef881aaed0d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:52.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.713+0000 7f3783fff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:37:52.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.713+0000 7f3783fff700 1 -- 172.21.15.40:0/1214246928 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3790099cb0 con 0x55c63897fe20 2023-07-18T03:37:52.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.713+0000 7f3783fff700 1 -- 172.21.15.40:0/1214246928 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f37900a2330 con 0x55c63897fe20 2023-07-18T03:37:52.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.713+0000 7f37b70b7b80 1 -- 172.21.15.40:0/1214246928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3790098280 con 0x55c63897fe20 2023-07-18T03:37:52.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.713+0000 7f3783fff700 1 -- 172.21.15.40:0/1214246928 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3790098020 con 0x55c63897fe20 2023-07-18T03:37:52.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.713+0000 7f37b70b7b80 10 client.16500 fetch_fsmap learned FSMap version 48 2023-07-18T03:37:52.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.713+0000 7f37b70b7b80 1 -- 172.21.15.40:0/1214246928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x7f379c014570 con 0x55c63897fe20 2023-07-18T03:37:52.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.713+0000 7f3783fff700 1 -- 172.21.15.40:0/1214246928 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3790063070 con 0x55c63897fe20 2023-07-18T03:37:52.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.713+0000 7f3783fff700 1 -- 172.21.15.40:0/1214246928 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 48) v1 ==== 7518+0+0 (secure 0 0 0) 0x7f37900adbe0 con 0x55c63897fe20 2023-07-18T03:37:52.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.713+0000 7f37b70b7b80 10 client.16500 fetch_fsmap finished waiting for FSMap version 48 2023-07-18T03:37:52.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.713+0000 7f37b70b7b80 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:37:52.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.713+0000 7f37b70b7b80 20 client.16500 populate_metadata read hostname 'smithi040' 2023-07-18T03:37:52.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.713+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/1214246928 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c6389ea450 0x55c6389ec860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:52.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.713+0000 7f37b70b7b80 4 client.16500 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:37:52.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.713+0000 7f37b70b7b80 1 -- 172.21.15.40:0/1214246928 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c63888c050 con 0x55c6389ea450 2023-07-18T03:37:52.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.714+0000 7f37a8ccd700 1 --2- 172.21.15.40:0/1214246928 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c6389ea450 0x55c6389ec860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:52.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.714+0000 7f37a8ccd700 1 --2- 172.21.15.40:0/1214246928 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c6389ea450 0x55c6389ec860 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6654c69c76252ff3 server_cookie=d3b64da7fc72d437 in_seq=0 out_seq=0 2023-07-18T03:37:52.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.714+0000 7f3783fff700 10 client.16500 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:52.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.714+0000 7f3783fff700 1 -- 172.21.15.40:0/1214246928 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c63888c050 con 0x55c6389ea450 2023-07-18T03:37:52.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.714+0000 7f3783fff700 10 client.16500 handle_command_reply: tid=0 2023-07-18T03:37:52.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.714+0000 7f37b70b7b80 1 client.16500 shutdown 2023-07-18T03:37:52.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.715+0000 7f37b70b7b80 1 -- 172.21.15.40:0/1214246928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897fe20 msgr2=0x55c63898eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:52.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.715+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/1214246928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897fe20 0x55c63898eb30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3790099e70 tx=0x7f3790099b30 comp rx=0 tx=0).stop 2023-07-18T03:37:52.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.715+0000 7f37b70b7b80 1 -- 172.21.15.40:0/1214246928 shutdown_connections 2023-07-18T03:37:52.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.715+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/1214246928 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c6389ea450 0x55c6389ec860 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.715+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/1214246928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c638980c70 0x55c63898f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.715+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/1214246928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c63897f4d0 0x55c638990520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.715+0000 7f3783fff700 0 client.16500 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:52.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.715+0000 7f37b70b7b80 1 --2- 172.21.15.40:0/1214246928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c63897fe20 0x55c63898eb30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.715+0000 7f37b70b7b80 1 -- 172.21.15.40:0/1214246928 >> 172.21.15.40:0/1214246928 conn(0x55c638974da0 msgr2=0x55c638983c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:52.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.715+0000 7f37b70b7b80 1 -- 172.21.15.40:0/1214246928 shutdown_connections 2023-07-18T03:37:52.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.715+0000 7f37b70b7b80 1 -- 172.21.15.40:0/1214246928 wait complete. 2023-07-18T03:37:52.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.715+0000 7f37b70b7b80 20 client.16500 trim_cache size 0 max 16384 2023-07-18T03:37:52.740 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:37:52.740 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-18T03:37:52.741 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:37:52.741 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:37:52.741 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:37:52.741 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:37:52.741 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:37:52.741 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:37:52.741 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:37:52.741 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:37:52.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.716+0000 7f37aaf31700 1 -- 172.21.15.40:0/1637576093 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f37940a1150 msgr2=0x7f37940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:52.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.716+0000 7f37aaf31700 1 --2- 172.21.15.40:0/1637576093 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f37940a1150 0x7f37940a3600 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3798006010 tx=0x7f3798005960 comp rx=0 tx=0).stop 2023-07-18T03:37:52.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.716+0000 7f37aaf31700 1 -- 172.21.15.40:0/1637576093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a4100ce0 msgr2=0x7f37a4195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:52.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.716+0000 7f37aaf31700 1 --2- 172.21.15.40:0/1637576093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a4100ce0 0x7f37a4195d60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f379000e9a0 tx=0x7f379000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.716+0000 7f37aaf31700 1 -- 172.21.15.40:0/1637576093 shutdown_connections 2023-07-18T03:37:52.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.716+0000 7f37aaf31700 1 --2- 172.21.15.40:0/1637576093 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f37940a1150 0x7f37940a3600 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.716+0000 7f37aaf31700 1 --2- 172.21.15.40:0/1637576093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f37a4101670 0x7f37a419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.716+0000 7f37aaf31700 1 --2- 172.21.15.40:0/1637576093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37a40ffaf0 0x7f37a4195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.716+0000 7f37aaf31700 1 --2- 172.21.15.40:0/1637576093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a4100ce0 0x7f37a4195d60 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:52.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.716+0000 7f37aaf31700 1 -- 172.21.15.40:0/1637576093 >> 172.21.15.40:0/1637576093 conn(0x7f37a40fb130 msgr2=0x7f37a40fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:52.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.716+0000 7f37aaf31700 1 -- 172.21.15.40:0/1637576093 shutdown_connections 2023-07-18T03:37:52.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:37:52.716+0000 7f37aaf31700 1 -- 172.21.15.40:0/1637576093 wait complete. 2023-07-18T03:37:52.745 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:37:53.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:52 smithi134 ceph-mon[98216]: pgmap v504: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 79 op/s 2023-07-18T03:37:53.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:52 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:37:53.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:52 smithi040 ceph-mon[96439]: pgmap v504: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 79 op/s 2023-07-18T03:37:53.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:52 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:37:53.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:52 smithi064 ceph-mon[98059]: pgmap v504: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 79 op/s 2023-07-18T03:37:53.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:52 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:37:54.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:53 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:37:54.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:53 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:37:54.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:53 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:37:54.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:53 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:37:54.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:53 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:37:54.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:53 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:37:54.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:53 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:37:54.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:53 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:37:54.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:53 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:37:55.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:54 smithi134 ceph-mon[98216]: pgmap v505: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 546 KiB/s wr, 66 op/s 2023-07-18T03:37:55.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:54 smithi040 ceph-mon[96439]: pgmap v505: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 546 KiB/s wr, 66 op/s 2023-07-18T03:37:55.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:54 smithi064 ceph-mon[98059]: pgmap v505: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 546 KiB/s wr, 66 op/s 2023-07-18T03:37:56.072 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.687+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001240 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-18.jpg auth v1098 ap=1 s=49884 n(v0 rc2023-07-18T03:32:32.257004+0000 b49884 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeb320000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001240:[<0x20000001254/picture-18.jpg v800>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001240:[<0x20000001254/picture-18.jpg v1098>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.072 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.687+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000010f6 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-18.xml auth v1100 ap=1 s=5804 n(v0 rc2023-07-18T03:32:30.860023+0000 b5804 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeb6edb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000010f6:[<0x20000001254/article-18.xml v84>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000010f6:[<0x20000001254/article-18.xml v1100>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.072 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.688+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001259 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-13.jpg auth v1094 ap=1 s=16256 n(v0 rc2023-07-18T03:32:32.314004+0000 b16256 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeac7c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001259:[<0x20000001254/picture-13.jpg v840>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001259:[<0x20000001254/picture-13.jpg v1094>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.072 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.688+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000038a [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-5.xml auth v1108 ap=1 s=2599 n(v0 rc2023-07-18T03:33:18.553379+0000 b2599 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeaf16c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000038a:[<0x20000001254/comment-5.xml v1056>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000038a:[<0x20000001254/comment-5.xml v1108>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.072 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.688+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000123c [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-19.jpg auth v1102 ap=1 s=64509 n(v0 rc2023-07-18T03:32:32.243004+0000 b64509 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeb6ac100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000123c:[<0x20000001254/picture-19.jpg v792>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000123c:[<0x20000001254/picture-19.jpg v1102>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.073 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.688+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000372 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-29.xml auth v1114 ap=1 s=9408 n(v0 rc2023-07-18T03:33:18.440381+0000 b9408 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeac7cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000372:[<0x20000001254/comment-29.xml v1019>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000372:[<0x20000001254/comment-29.xml v1114>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.073 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.688+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000117a [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-82.jpg auth v1066 ap=1 s=58156 n(v0 rc2023-07-18T03:32:31.073020+0000 b58156 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdec4bf180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000117a:[<0x20000001254/picture-82.jpg v300>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000117a:[<0x20000001254/picture-82.jpg v1066>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.073 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.689+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000010e5 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-27.xml auth v1106 ap=1 s=18314 n(v0 rc2023-07-18T03:32:30.801024+0000 b18314 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde9afc000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000010e5:[<0x20000001254/article-27.xml v12>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000010e5:[<0x20000001254/article-27.xml v1106>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.073 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.689+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000127b [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-4.jpg auth v1088 ap=1 s=41376 n(v0 rc2023-07-18T03:32:32.410002+0000 b41376 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec512c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000127b:[<0x20000001254/picture-4.jpg v912>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000127b:[<0x20000001254/picture-4.jpg v1088>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.073 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.689+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011f0 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-35.jpg auth v1072 ap=1 s=44781 n(v0 rc2023-07-18T03:32:31.758011+0000 b44781 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde9828c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011f0:[<0x20000001254/picture-35.jpg v676>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011f0:[<0x20000001254/picture-35.jpg v1072>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.073 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.689+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001283 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-2.jpg auth v1086 ap=1 s=42508 n(v0 rc2023-07-18T03:32:32.431002+0000 b42508 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec0dac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001283:[<0x20000001254/picture-2.jpg v928>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001283:[<0x20000001254/picture-2.jpg v1086>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.074 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.689+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001181 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-78.jpg auth v1068 ap=1 s=53800 n(v0 rc2023-07-18T03:32:31.100020+0000 b53800 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde9828100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001181:[<0x20000001254/picture-78.jpg v332>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001181:[<0x20000001254/picture-78.jpg v1068>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.074 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.690+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000010f3 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-20.xml auth v1104 ap=1 s=9753 n(v0 rc2023-07-18T03:32:30.849023+0000 b9753 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeba09080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000010f3:[<0x20000001254/article-20.xml v68>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000010f3:[<0x20000001254/article-20.xml v1104>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.074 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.690+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001250 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-14.jpg auth v1096 ap=1 s=19580 n(v0 rc2023-07-18T03:32:32.304004+0000 b19580 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec0d9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001250:[<0x20000001254/picture-14.jpg v832>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001250:[<0x20000001254/picture-14.jpg v1096>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.074 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.690+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000116f [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-89.jpg auth v1064 ap=1 s=16320 n(v0 rc2023-07-18T03:32:31.020021+0000 b16320 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdec529b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000116f:[<0x20000001254/picture-89.jpg v244>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000116f:[<0x20000001254/picture-89.jpg v1064>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.074 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.690+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000119d [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-61.jpg auth v1070 ap=1 s=52937 n(v0 rc2023-07-18T03:32:31.215018+0000 b52937 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde8f0f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000119d:[<0x20000001254/picture-61.jpg v468>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000119d:[<0x20000001254/picture-61.jpg v1070>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.074 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.690+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000126f [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-7.jpg auth v1090 ap=1 s=24734 n(v0 rc2023-07-18T03:32:32.377003+0000 b24734 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea588c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000126f:[<0x20000001254/picture-7.jpg v888>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000126f:[<0x20000001254/picture-7.jpg v1090>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.075 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.691+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001267 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-9.jpg auth v1092 ap=1 s=64435 n(v0 rc2023-07-18T03:32:32.357003+0000 b64435 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec550000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001267:[<0x20000001254/picture-9.jpg v872>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001267:[<0x20000001254/picture-9.jpg v1092>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.075 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.691+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000384 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-11.xml auth v1110 ap=1 s=5535 n(v0 rc2023-07-18T03:33:18.522380+0000 b5535 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea289180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000384:[<0x20000001254/comment-11.xml v1037>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000384:[<0x20000001254/comment-11.xml v1110>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.075 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.691+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000037b [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-20.xml auth v1112 ap=1 s=3294 n(v0 rc2023-07-18T03:33:18.480380+0000 b3294 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde9784100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000037b:[<0x20000001254/comment-20.xml v1035>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000037b:[<0x20000001254/comment-20.xml v1112>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.075 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.763+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001198 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-64.jpg auth v1067 ap=1 s=10064 n(v0 rc2023-07-18T03:32:31.196019+0000 b10064 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdeba09b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001198:[<0x20000001254/picture-64.jpg v444>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001198:[<0x20000001254/picture-64.jpg v1067>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.075 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.763+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036a [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-37.xml auth v1127 ap=1 s=5876 n(v0 rc2023-07-18T03:33:18.405381+0000 b5876 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea2fb600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000036a:[<0x20000001254/comment-37.xml v1001>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000036a:[<0x20000001254/comment-37.xml v1127>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.075 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.763+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001264 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-10.jpg auth v1105 ap=1 s=57956 n(v0 rc2023-07-18T03:32:32.346003+0000 b57956 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea05c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001264:[<0x20000001254/picture-10.jpg v864>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001264:[<0x20000001254/picture-10.jpg v1105>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.075 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.764+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001287 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-1.jpg auth v954 ap=1 s=15238 n(v0 rc2023-07-18T03:32:32.443002+0000 b15238 1=1+0) (iversion lock) | ptrwaiter=0 request=0 lock=0 caps=0 exportingcaps=0 dirtyparent=0 replicated=0 dirty=0 waiter=0 authpin=1 tempexporting=0 scrubqueue=0 0x55bde7d40b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001287:[<0x20000001254/picture-1.jpg v936>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001287:[<0x20000001254/picture-1.jpg v954>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.076 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.764+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000037f [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-16.xml auth v1117 ap=1 s=8054 n(v0 rc2023-07-18T03:33:18.503380+0000 b8054 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde7a52c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000037f:[<0x20000001254/comment-16.xml v1049>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000037f:[<0x20000001254/comment-16.xml v1117>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.076 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.764+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000115c [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-9.xml auth v1103 ap=1 s=15751 n(v0 rc2023-07-18T03:32:30.942022+0000 b15751 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeab19080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000115c:[<0x20000001254/article-9.xml v156>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000115c:[<0x20000001254/article-9.xml v1103>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.076 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.764+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001179 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-83.jpg auth v1063 ap=1 s=30182 n(v0 rc2023-07-18T03:32:31.063020+0000 b30182 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde87c9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001179:[<0x20000001254/picture-83.jpg v292>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001179:[<0x20000001254/picture-83.jpg v1063>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.076 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.765+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001273 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-6.jpg auth v1101 ap=1 s=45218 n(v0 rc2023-07-18T03:32:32.387003+0000 b45218 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea9dc000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001273:[<0x20000001254/picture-6.jpg v896>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001273:[<0x20000001254/picture-6.jpg v1101>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.076 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.765+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000358 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-55.xml auth v1131 ap=1 s=4528 n(v0 rc2023-07-18T03:33:18.236384+0000 b4528 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde6dba680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000358:[<0x20000001254/comment-55.xml v981>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000358:[<0x20000001254/comment-55.xml v1131>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.076 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.766+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000010eb [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-24.xml auth v1111 ap=1 s=26519 n(v0 rc2023-07-18T03:32:30.823024+0000 b26519 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde84e0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000010eb:[<0x20000001254/article-24.xml v36>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000010eb:[<0x20000001254/article-24.xml v1111>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.077 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.766+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036d [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-34.xml auth v1125 ap=1 s=6682 n(v0 rc2023-07-18T03:33:18.423381+0000 b6682 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde86e6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000036d:[<0x20000001254/comment-34.xml v1009>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000036d:[<0x20000001254/comment-34.xml v1125>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.077 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.766+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000376 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-25.xml auth v1123 ap=1 s=2284 n(v0 rc2023-07-18T03:33:18.460381+0000 b2284 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde7660100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000376:[<0x20000001254/comment-25.xml v1023>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000376:[<0x20000001254/comment-25.xml v1123>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.077 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.766+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001208 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-28.jpg auth v1113 ap=1 s=13398 n(v0 rc2023-07-18T03:32:31.945009+0000 b13398 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea5d1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001208:[<0x20000001254/picture-28.jpg v950>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001208:[<0x20000001254/picture-28.jpg v1113>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.077 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.766+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000366 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-41.xml auth v1129 ap=1 s=7081 n(v0 rc2023-07-18T03:33:18.378382+0000 b7081 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde765e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000366:[<0x20000001254/comment-41.xml v995>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000366:[<0x20000001254/comment-41.xml v1129>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.077 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.767+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011ea [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-37.jpg auth v1075 ap=1 s=22670 n(v0 rc2023-07-18T03:32:31.734011+0000 b22670 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdeb379700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011ea:[<0x20000001254/picture-37.jpg v660>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011ea:[<0x20000001254/picture-37.jpg v1075>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.077 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.767+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001244 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-17.jpg auth v1107 ap=1 s=24222 n(v0 rc2023-07-18T03:32:32.274004+0000 b24222 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea5d2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001244:[<0x20000001254/picture-17.jpg v808>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001244:[<0x20000001254/picture-17.jpg v1107>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.077 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.767+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001175 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-85.jpg auth v1061 ap=1 s=24403 n(v0 rc2023-07-18T03:32:31.043021+0000 b24403 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde9c3c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001175:[<0x20000001254/picture-85.jpg v276>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001175:[<0x20000001254/picture-85.jpg v1061>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.078 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.767+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001167 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-3.xml auth v1099 ap=1 s=23482 n(v0 rc2023-07-18T03:32:30.991021+0000 b23482 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea5d1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001167:[<0x20000001254/article-3.xml v204>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001167:[<0x20000001254/article-3.xml v1099>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.078 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.768+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001225 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-24.jpg auth v1109 ap=1 s=64712 n(v0 rc2023-07-18T03:32:32.072007+0000 b64712 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde7660680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001225:[<0x20000001254/picture-24.jpg v942>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001225:[<0x20000001254/picture-24.jpg v1109>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.078 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.768+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011f7 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-33.jpg auth v1077 ap=1 s=11244 n(v0 rc2023-07-18T03:32:31.792011+0000 b11244 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdeb22a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011f7:[<0x20000001254/picture-33.jpg v692>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011f7:[<0x20000001254/picture-33.jpg v1077>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.078 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.768+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011a3 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-57.jpg auth v1069 ap=1 s=54437 n(v0 rc2023-07-18T03:32:31.239018+0000 b54437 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdea9c0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011a3:[<0x20000001254/picture-57.jpg v500>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011a3:[<0x20000001254/picture-57.jpg v1069>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.078 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.768+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000354 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-7.xml auth v1115 ap=1 s=15728 n(v0 rc2023-07-18T03:33:02.622594+0000 b15728 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde905ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000354:[<0x20000001254/article-7.xml v973>,<0x10000000143/blog-34 v71539>,<0x100000007de/blogtest_in v676>,<0x100000007d9/src v1678>,<0x100000005e3/blogbench-1.0 v1458>,<0x100000005e2/tmp v58>,<0x100000001fb/client.0 v48>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v66>,<0x10000000001/sv_1 v79>,<0x10000000000/_nogroup v71>,<0x1/volumes v63>]//[]","memoryvalue":"(3)0x10000000354:[<0x20000001254/article-7.xml v1115>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.078 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.768+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011d9 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-42.jpg auth v1071 ap=1 s=15115 n(v0 rc2023-07-18T03:32:31.609013+0000 b15115 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde6f32000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011d9:[<0x20000001254/picture-42.jpg v620>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011d9:[<0x20000001254/picture-42.jpg v1071>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.078 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.769+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001192 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-68.jpg auth v1065 ap=1 s=57379 n(v0 rc2023-07-18T03:32:31.165019+0000 b57379 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdeaff0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001192:[<0x20000001254/picture-68.jpg v412>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001192:[<0x20000001254/picture-68.jpg v1065>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.079 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.769+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011e5 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-39.jpg auth v1073 ap=1 s=46429 n(v0 rc2023-07-18T03:32:31.685012+0000 b46429 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde94d8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011e5:[<0x20000001254/picture-39.jpg v644>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011e5:[<0x20000001254/picture-39.jpg v1073>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.079 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.769+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000378 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-23.xml auth v1119 ap=1 s=7304 n(v0 rc2023-07-18T03:33:18.469380+0000 b7304 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdebae0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000378:[<0x20000001254/comment-23.xml v1029>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000378:[<0x20000001254/comment-23.xml v1119>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.079 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.770+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000377 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-24.xml auth v1121 ap=1 s=6460 n(v0 rc2023-07-18T03:33:18.463380+0000 b6460 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea9e4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000377:[<0x20000001254/comment-24.xml v1027>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000377:[<0x20000001254/comment-24.xml v1121>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.079 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.780+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001633 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-25.jpg auth v1093 ap=1 s=49682 n(v0 rc2023-07-18T03:35:57.019237+0000 b49682 1=1+0) (ifile excl) (iversion lock) cr={15309=0-4194304@1} caps={15309=pAsLsXsFsxcrwb/pFscr@1},l=15309 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55bde70ee680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001633:[<0x20000001254/picture-25.jpg v1084>,<0x10000000143/blog-34 v111967>,<0x100000007de/blogtest_in v2096>,<0x100000007d9/src v1716>,<0x100000005e3/blogbench-1.0 v1496>,<0x100000005e2/tmp v96>,<0x100000001fb/client.0 v86>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v104>,<0x10000000001/sv_1 v117>,<0x10000000000/_nogroup v109>,<0x1/volumes v101>]//[]","memoryvalue":"(3)0x10000001633:[<0x20000001254/picture-25.jpg v1093>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.079 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.783+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000115e [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-8.xml auth v1102 ap=1 s=26316 n(v0 rc2023-07-18T03:32:30.949022+0000 b26316 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde8d62580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000115e:[<0x20000001254/article-8.xml v164>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000115e:[<0x20000001254/article-8.xml v1102>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.079 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.783+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000360 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-47.xml auth v1122 ap=1 s=9728 n(v0 rc2023-07-18T03:33:18.342382+0000 b9728 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde70ed600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000360:[<0x20000001254/comment-47.xml v984>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000360:[<0x20000001254/comment-47.xml v1122>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.079 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.784+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000361 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-46.xml auth v1120 ap=1 s=7928 n(v0 rc2023-07-18T03:33:18.354382+0000 b7928 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde70edb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000361:[<0x20000001254/comment-46.xml v986>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000361:[<0x20000001254/comment-46.xml v1120>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.080 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.784+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011fc [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-31.jpg auth v1065 ap=1 s=58190 n(v0 rc2023-07-18T03:32:31.822010+0000 b58190 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde8940100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011fc:[<0x20000001254/picture-31.jpg v708>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011fc:[<0x20000001254/picture-31.jpg v1065>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.080 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.784+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011bd [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-49.jpg auth v1063 ap=1 s=29607 n(v0 rc2023-07-18T03:32:31.412016+0000 b29607 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde7a86680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011bd:[<0x20000001254/picture-49.jpg v564>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011bd:[<0x20000001254/picture-49.jpg v1063>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.080 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.784+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000126b [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-8.jpg auth v1100 ap=1 s=62742 n(v0 rc2023-07-18T03:32:32.367003+0000 b62742 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea571600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000126b:[<0x20000001254/picture-8.jpg v880>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000126b:[<0x20000001254/picture-8.jpg v1100>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.080 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.784+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001171 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-88.jpg auth v1059 ap=1 s=41230 n(v0 rc2023-07-18T03:32:31.025021+0000 b41230 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdea2ba000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001171:[<0x20000001254/picture-88.jpg v252>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001171:[<0x20000001254/picture-88.jpg v1059>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.080 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.784+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000121a [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-26.jpg auth v1108 ap=1 s=36279 n(v0 rc2023-07-18T03:32:32.036007+0000 b36279 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde70ec000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000121a:[<0x20000001254/picture-26.jpg v946>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000121a:[<0x20000001254/picture-26.jpg v1108>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.080 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.785+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000010f1 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-21.xml auth v1106 ap=1 s=4372 n(v0 rc2023-07-18T03:32:30.842023+0000 b4372 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde70ed080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000010f1:[<0x20000001254/article-21.xml v60>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000010f1:[<0x20000001254/article-21.xml v1106>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.081 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.785+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036b [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-36.xml auth v1118 ap=1 s=6767 n(v0 rc2023-07-18T03:33:18.412381+0000 b6767 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea570000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000036b:[<0x20000001254/comment-36.xml v1004>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000036b:[<0x20000001254/comment-36.xml v1118>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.081 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.785+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001102 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-13.xml auth v1104 ap=1 s=18556 n(v0 rc2023-07-18T03:32:30.912023+0000 b18556 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdebc7cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001102:[<0x20000001254/article-13.xml v124>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001102:[<0x20000001254/article-13.xml v1104>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.081 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.788+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000038b [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-4.xml auth v1112 ap=1 s=5201 n(v0 rc2023-07-18T03:33:18.556379+0000 b5201 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde990ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000038b:[<0x20000001254/comment-4.xml v1055>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000038b:[<0x20000001254/comment-4.xml v1112>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.081 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.788+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001162 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-6.xml auth v1098 ap=1 s=32061 n(v0 rc2023-07-18T03:32:30.969022+0000 b32061 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde99cdb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001162:[<0x20000001254/article-6.xml v180>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001162:[<0x20000001254/article-6.xml v1098>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.081 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.788+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001169 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-2.xml auth v1096 ap=1 s=20569 n(v0 rc2023-07-18T03:32:30.996021+0000 b20569 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea61a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001169:[<0x20000001254/article-2.xml v212>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001169:[<0x20000001254/article-2.xml v1096>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.081 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.788+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000381 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-14.xml auth v1114 ap=1 s=9930 n(v0 rc2023-07-18T03:33:18.509380+0000 b9930 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea87ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000381:[<0x20000001254/comment-14.xml v1041>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000381:[<0x20000001254/comment-14.xml v1114>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.081 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.788+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011aa [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-54.jpg auth v1061 ap=1 s=51392 n(v0 rc2023-07-18T03:32:31.263018+0000 b51392 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde94dac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011aa:[<0x20000001254/picture-54.jpg v524>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011aa:[<0x20000001254/picture-54.jpg v1061>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.082 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.788+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001202 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-29.jpg auth v1110 ap=1 s=34021 n(v0 rc2023-07-18T03:32:31.855010+0000 b34021 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde765ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001202:[<0x20000001254/picture-29.jpg v724>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001202:[<0x20000001254/picture-29.jpg v1110>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.082 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.788+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000037c [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-19.xml auth v1116 ap=1 s=9189 n(v0 rc2023-07-18T03:33:18.486380+0000 b9189 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde8dc1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000037c:[<0x20000001254/comment-19.xml v1039>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000037c:[<0x20000001254/comment-19.xml v1116>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.082 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.789+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000385 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-10.xml auth v1102 ap=1 s=6280 n(v0 rc2023-07-18T03:33:18.525380+0000 b6280 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde7b75b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000385:[<0x20000001254/comment-10.xml v1038>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000385:[<0x20000001254/comment-10.xml v1102>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.082 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.789+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000127f [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-3.jpg auth v1090 ap=1 s=56223 n(v0 rc2023-07-18T03:32:32.421002+0000 b56223 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea287080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000127f:[<0x20000001254/picture-3.jpg v920>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000127f:[<0x20000001254/picture-3.jpg v1090>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.082 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.789+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000389 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-6.xml auth v1098 ap=1 s=10174 n(v0 rc2023-07-18T03:33:18.550379+0000 b10174 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde6fe0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000389:[<0x20000001254/comment-6.xml v1050>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000389:[<0x20000001254/comment-6.xml v1098>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.082 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.789+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000035a [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-53.xml auth v1110 ap=1 s=5349 n(v0 rc2023-07-18T03:33:18.288383+0000 b5349 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde8993180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000035a:[<0x20000001254/comment-53.xml v984>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000035a:[<0x20000001254/comment-53.xml v1110>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.082 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.789+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011a7 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-55.jpg auth v1064 ap=1 s=61583 n(v0 rc2023-07-18T03:32:31.252018+0000 b61583 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde8b74c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011a7:[<0x20000001254/picture-55.jpg v516>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011a7:[<0x20000001254/picture-55.jpg v1064>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.083 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.790+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001261 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-11.jpg auth v1094 ap=1 s=31335 n(v0 rc2023-07-18T03:32:32.336003+0000 b31335 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec55b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001261:[<0x20000001254/picture-11.jpg v856>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001261:[<0x20000001254/picture-11.jpg v1094>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.083 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.790+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000118e [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-70.jpg auth v1060 ap=1 s=40469 n(v0 rc2023-07-18T03:32:31.153019+0000 b40469 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde9afec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000118e:[<0x20000001254/picture-70.jpg v396>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000118e:[<0x20000001254/picture-70.jpg v1060>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.083 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.790+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001164 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-5.xml auth v1092 ap=1 s=5612 n(v0 rc2023-07-18T03:32:30.978022+0000 b5612 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde73ec680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001164:[<0x20000001254/article-5.xml v188>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001164:[<0x20000001254/article-5.xml v1092>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.083 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.790+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036c [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-35.xml auth v1106 ap=1 s=2737 n(v0 rc2023-07-18T03:33:18.415381+0000 b2737 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde70ef700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000036c:[<0x20000001254/comment-35.xml v1004>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000036c:[<0x20000001254/comment-35.xml v1106>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.083 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.790+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000373 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-28.xml auth v1104 ap=1 s=4621 n(v0 rc2023-07-18T03:33:18.448381+0000 b4621 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeace9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000373:[<0x20000001254/comment-28.xml v1012>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000373:[<0x20000001254/comment-28.xml v1104>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.083 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.790+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000118d [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-71.jpg auth v1058 ap=1 s=27238 n(v0 rc2023-07-18T03:32:31.144019+0000 b27238 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdea0d1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000118d:[<0x20000001254/picture-71.jpg v388>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000118d:[<0x20000001254/picture-71.jpg v1058>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.083 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.791+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000119a [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-63.jpg auth v1062 ap=1 s=46120 n(v0 rc2023-07-18T03:32:31.203019+0000 b46120 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde94d9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000119a:[<0x20000001254/picture-63.jpg v452>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000119a:[<0x20000001254/picture-63.jpg v1062>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.084 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.791+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011e1 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-40.jpg auth v1066 ap=1 s=62169 n(v0 rc2023-07-18T03:32:31.666012+0000 b62169 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde744e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011e1:[<0x20000001254/picture-40.jpg v636>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011e1:[<0x20000001254/picture-40.jpg v1066>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.084 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.791+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000010ef [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-22.xml auth v1096 ap=1 s=22190 n(v0 rc2023-07-18T03:32:30.836024+0000 b22190 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec0db700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000010ef:[<0x20000001254/article-22.xml v52>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000010ef:[<0x20000001254/article-22.xml v1096>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.084 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.791+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000369 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-38.xml auth v1108 ap=1 s=8330 n(v0 rc2023-07-18T03:33:18.397381+0000 b8330 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde6e6f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000369:[<0x20000001254/comment-38.xml v998>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000369:[<0x20000001254/comment-38.xml v1108>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.084 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.791+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001172 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-87.jpg auth v1054 ap=1 s=31536 n(v0 rc2023-07-18T03:32:31.031021+0000 b31536 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde8b62680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001172:[<0x20000001254/picture-87.jpg v260>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001172:[<0x20000001254/picture-87.jpg v1054>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.084 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.792+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001177 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-84.jpg auth v1056 ap=1 s=44029 n(v0 rc2023-07-18T03:32:31.058021+0000 b44029 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdeaba0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001177:[<0x20000001254/picture-84.jpg v284>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001177:[<0x20000001254/picture-84.jpg v1056>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.084 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.793+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000387 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-8.xml auth v1100 ap=1 s=10128 n(v0 rc2023-07-18T03:33:18.539379+0000 b10128 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde833e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000387:[<0x20000001254/comment-8.xml v1046>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000387:[<0x20000001254/comment-8.xml v1100>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.085 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.793+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000382 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-13.xml auth v1126 ap=1 s=7270 n(v0 rc2023-07-18T03:33:18.512380+0000 b7270 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeacb7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000382:[<0x20000001254/comment-13.xml v1054>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000382:[<0x20000001254/comment-13.xml v1126>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.085 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.793+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000037a [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-21.xml auth v1130 ap=1 s=3416 n(v0 rc2023-07-18T03:33:18.475380+0000 b3416 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec8aa000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000037a:[<0x20000001254/comment-21.xml v1032>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000037a:[<0x20000001254/comment-21.xml v1130>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.085 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.793+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000375 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-26.xml auth v1132 ap=1 s=2455 n(v0 rc2023-07-18T03:33:18.457381+0000 b2455 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec8aa580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000375:[<0x20000001254/comment-26.xml v1022>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000375:[<0x20000001254/comment-26.xml v1132>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.085 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.793+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001211 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-27.jpg auth v1120 ap=1 s=23985 n(v0 rc2023-07-18T03:32:31.980008+0000 b23985 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec8aab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001211:[<0x20000001254/picture-27.jpg v948>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001211:[<0x20000001254/picture-27.jpg v1120>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.085 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.793+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001156 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-12.xml auth v1112 ap=1 s=18218 n(v0 rc2023-07-18T03:32:30.919022+0000 b18218 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec8ab080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001156:[<0x20000001254/article-12.xml v132>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001156:[<0x20000001254/article-12.xml v1112>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.085 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.794+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011b1 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-52.jpg auth v1078 ap=1 s=49222 n(v0 rc2023-07-18T03:32:31.334017+0000 b49222 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec8ab600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011b1:[<0x20000001254/picture-52.jpg v540>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011b1:[<0x20000001254/picture-52.jpg v1078>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.085 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.794+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000116a [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-1.xml auth v1110 ap=1 s=24724 n(v0 rc2023-07-18T03:32:31.003021+0000 b24724 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec8abb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000116a:[<0x20000001254/article-1.xml v220>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000116a:[<0x20000001254/article-1.xml v1110>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.086 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.794+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000371 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-30.xml auth v1134 ap=1 s=2797 n(v0 rc2023-07-18T03:33:18.438381+0000 b2797 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec8ac100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000371:[<0x20000001254/comment-30.xml v1016>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000371:[<0x20000001254/comment-30.xml v1134>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.086 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.794+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011f3 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-34.jpg auth v1082 ap=1 s=20506 n(v0 rc2023-07-18T03:32:31.777011+0000 b20506 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec8ac680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011f3:[<0x20000001254/picture-34.jpg v684>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011f3:[<0x20000001254/picture-34.jpg v1082>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.086 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.794+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011ad [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-53.jpg auth v1076 ap=1 s=22817 n(v0 rc2023-07-18T03:32:31.287017+0000 b22817 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec8acc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011ad:[<0x20000001254/picture-53.jpg v532>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011ad:[<0x20000001254/picture-53.jpg v1076>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.086 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.795+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001193 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-67.jpg auth v1074 ap=1 s=25045 n(v0 rc2023-07-18T03:32:31.174019+0000 b25045 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec8ad180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001193:[<0x20000001254/picture-67.jpg v420>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001193:[<0x20000001254/picture-67.jpg v1074>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.086 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.795+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000116e [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-90.jpg auth v1068 ap=1 s=12691 n(v0 rc2023-07-18T03:32:31.014021+0000 b12691 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec8ad700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000116e:[<0x20000001254/picture-90.jpg v236>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000116e:[<0x20000001254/picture-90.jpg v1068>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.086 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.795+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000038c [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-3.xml auth v1122 ap=1 s=3994 n(v0 rc2023-07-18T03:33:18.559379+0000 b3994 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdebcc2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000038c:[<0x20000001254/comment-3.xml v1060>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000038c:[<0x20000001254/comment-3.xml v1122>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.086 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.795+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000037d [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-18.xml auth v1128 ap=1 s=4514 n(v0 rc2023-07-18T03:33:18.491380+0000 b4514 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdebcc2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000037d:[<0x20000001254/comment-18.xml v1044>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000037d:[<0x20000001254/comment-18.xml v1128>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.087 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.795+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001248 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-16.jpg auth v1114 ap=1 s=33009 n(v0 rc2023-07-18T03:32:32.284004+0000 b33009 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdebcc2b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001248:[<0x20000001254/picture-16.jpg v816>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001248:[<0x20000001254/picture-16.jpg v1114>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.087 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.795+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000383 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-12.xml auth v1124 ap=1 s=6357 n(v0 rc2023-07-18T03:33:18.515380+0000 b6357 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdebcc3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000383:[<0x20000001254/comment-12.xml v1058>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000383:[<0x20000001254/comment-12.xml v1124>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.087 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.796+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001188 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-74.jpg auth v1072 ap=1 s=29204 n(v0 rc2023-07-18T03:32:31.126020+0000 b29204 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdebcc3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001188:[<0x20000001254/picture-74.jpg v364>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001188:[<0x20000001254/picture-74.jpg v1072>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.087 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.796+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000035e [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-49.xml auth v1138 ap=1 s=3513 n(v0 rc2023-07-18T03:33:18.314383+0000 b3513 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdebcc3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000035e:[<0x20000001254/comment-49.xml v981>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000035e:[<0x20000001254/comment-49.xml v1138>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.087 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.798+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000365 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-42.xml auth v1136 ap=1 s=5765 n(v0 rc2023-07-18T03:33:18.373382+0000 b5765 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdebcc4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000365:[<0x20000001254/comment-42.xml v992>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000365:[<0x20000001254/comment-42.xml v1136>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.087 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.798+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000010ed [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-23.xml auth v1118 ap=1 s=28646 n(v0 rc2023-07-18T03:32:30.829024+0000 b28646 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdebcc4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000010ed:[<0x20000001254/article-23.xml v44>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000010ed:[<0x20000001254/article-23.xml v1118>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.087 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.799+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000010f8 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-17.xml auth v1116 ap=1 s=29132 n(v0 rc2023-07-18T03:32:30.866023+0000 b29132 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdebcc4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000010f8:[<0x20000001254/article-17.xml v92>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000010f8:[<0x20000001254/article-17.xml v1116>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.088 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.799+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001184 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-76.jpg auth v1070 ap=1 s=51113 n(v0 rc2023-07-18T03:32:31.113020+0000 b51113 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdebcc5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001184:[<0x20000001254/picture-76.jpg v348>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001184:[<0x20000001254/picture-76.jpg v1070>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.088 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.799+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011fa [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-32.jpg auth v1084 ap=1 s=33427 n(v0 rc2023-07-18T03:32:31.807010+0000 b33427 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdebcc5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011fa:[<0x20000001254/picture-32.jpg v700>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011fa:[<0x20000001254/picture-32.jpg v1084>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.088 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.799+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000035c [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-51.xml auth v1140 ap=1 s=2936 n(v0 rc2023-07-18T03:33:18.309383+0000 b2936 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea6ce580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000035c:[<0x20000001254/comment-51.xml v977>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000035c:[<0x20000001254/comment-51.xml v1140>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.088 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.799+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011c1 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-48.jpg auth v1080 ap=1 s=11189 n(v0 rc2023-07-18T03:32:31.440015+0000 b11189 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea6ce000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011c1:[<0x20000001254/picture-48.jpg v572>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011c1:[<0x20000001254/picture-48.jpg v1080>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.088 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.799+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000379 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-22.xml auth v1123 ap=1 s=9712 n(v0 rc2023-07-18T03:33:18.472380+0000 b9712 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde8479600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000379:[<0x20000001254/comment-22.xml v1031>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000379:[<0x20000001254/comment-22.xml v1123>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.089 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.800+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000038d [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-2.xml auth v1113 ap=1 s=6373 n(v0 rc2023-07-18T03:33:18.562379+0000 b6373 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde7e5d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000038d:[<0x20000001254/comment-2.xml v1059>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000038d:[<0x20000001254/comment-2.xml v1113>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.089 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.800+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001174 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-86.jpg auth v1063 ap=1 s=11426 n(v0 rc2023-07-18T03:32:31.037021+0000 b11426 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdec510000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001174:[<0x20000001254/picture-86.jpg v268>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001174:[<0x20000001254/picture-86.jpg v1063>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.089 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.800+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000363 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-44.xml auth v1129 ap=1 s=8222 n(v0 rc2023-07-18T03:33:18.365382+0000 b8222 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde6c7d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000363:[<0x20000001254/comment-44.xml v991>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000363:[<0x20000001254/comment-44.xml v1129>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.089 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.800+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000117c [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-81.jpg auth v1065 ap=1 s=62062 n(v0 rc2023-07-18T03:32:31.080020+0000 b62062 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdea2f3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000117c:[<0x20000001254/picture-81.jpg v308>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000117c:[<0x20000001254/picture-81.jpg v1065>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.089 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.801+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001277 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-5.jpg auth v1109 ap=1 s=38488 n(v0 rc2023-07-18T03:32:32.399002+0000 b38488 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea572680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001277:[<0x20000001254/picture-5.jpg v904>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001277:[<0x20000001254/picture-5.jpg v1109>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.089 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.801+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000374 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-27.xml auth v1125 ap=1 s=5517 n(v0 rc2023-07-18T03:33:18.451381+0000 b5517 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde78ca580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000374:[<0x20000001254/comment-27.xml v1015>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000374:[<0x20000001254/comment-27.xml v1125>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.089 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.801+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011b9 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-50.jpg auth v1079 ap=1 s=15811 n(v0 rc2023-07-18T03:32:31.386016+0000 b15811 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdea288100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011b9:[<0x20000001254/picture-50.jpg v556>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011b9:[<0x20000001254/picture-50.jpg v1079>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.090 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.802+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011a2 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-58.jpg auth v1075 ap=1 s=49487 n(v0 rc2023-07-18T03:32:31.233018+0000 b49487 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde94ba100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011a2:[<0x20000001254/picture-58.jpg v492>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011a2:[<0x20000001254/picture-58.jpg v1075>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.090 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.802+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000035b [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-52.xml auth v1133 ap=1 s=6231 n(v0 rc2023-07-18T03:33:18.301383+0000 b6231 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde8eebb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000035b:[<0x20000001254/comment-52.xml v987>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000035b:[<0x20000001254/comment-52.xml v1133>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.090 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.805+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001197 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-65.jpg auth v1071 ap=1 s=26941 n(v0 rc2023-07-18T03:32:31.189019+0000 b26941 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde9ccdb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001197:[<0x20000001254/picture-65.jpg v436>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001197:[<0x20000001254/picture-65.jpg v1071>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.090 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.805+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000388 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-7.xml auth v1115 ap=1 s=9538 n(v0 rc2023-07-18T03:33:18.544379+0000 b9538 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde754f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000388:[<0x20000001254/comment-7.xml v1052>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000388:[<0x20000001254/comment-7.xml v1115>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.090 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.806+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000380 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-15.xml auth v1119 ap=1 s=7789 n(v0 rc2023-07-18T03:33:18.506380+0000 b7789 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde7107700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000380:[<0x20000001254/comment-15.xml v1038>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000380:[<0x20000001254/comment-15.xml v1119>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.090 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.806+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000364 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-43.xml auth v1127 ap=1 s=6898 n(v0 rc2023-07-18T03:33:18.368382+0000 b6898 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea673080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000364:[<0x20000001254/comment-43.xml v993>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000364:[<0x20000001254/comment-43.xml v1127>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.091 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.806+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000037e [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-17.xml auth v1121 ap=1 s=7686 n(v0 rc2023-07-18T03:33:18.497380+0000 b7686 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdead36580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000037e:[<0x20000001254/comment-17.xml v1036>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000037e:[<0x20000001254/comment-17.xml v1121>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.091 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.806+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001255 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-28.xml auth v1111 ap=1 s=1555 n(v0 rc2023-07-18T03:32:30.797024+0000 b1555 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde9acac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001255:[<0x20000001254/article-28.xml v952>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001255:[<0x20000001254/article-28.xml v1111>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.091 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.806+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000386 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-9.xml auth v1117 ap=1 s=7227 n(v0 rc2023-07-18T03:33:18.532380+0000 b7227 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeab54680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000386:[<0x20000001254/comment-9.xml v1044>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000386:[<0x20000001254/comment-9.xml v1117>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.091 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.807+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001183 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-77.jpg auth v1067 ap=1 s=24422 n(v0 rc2023-07-18T03:32:31.106020+0000 b24422 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde93e1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001183:[<0x20000001254/picture-77.jpg v340>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001183:[<0x20000001254/picture-77.jpg v1067>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.091 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.808+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001190 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-69.jpg auth v1069 ap=1 s=51653 n(v0 rc2023-07-18T03:32:31.159019+0000 b51653 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde98fdb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001190:[<0x20000001254/picture-69.jpg v404>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001190:[<0x20000001254/picture-69.jpg v1069>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.091 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.808+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011b5 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-51.jpg auth v1077 ap=1 s=60423 n(v0 rc2023-07-18T03:32:31.362016+0000 b60423 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde70fa580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011b5:[<0x20000001254/picture-51.jpg v548>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011b5:[<0x20000001254/picture-51.jpg v1077>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.091 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.808+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000362 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-45.xml auth v1131 ap=1 s=6964 n(v0 rc2023-07-18T03:33:18.362382+0000 b6964 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde7105600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000362:[<0x20000001254/comment-45.xml v989>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000362:[<0x20000001254/comment-45.xml v1131>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.092 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.808+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000119c [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-62.jpg auth v1073 ap=1 s=52942 n(v0 rc2023-07-18T03:32:31.208019+0000 b52942 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bde9861180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000119c:[<0x20000001254/picture-62.jpg v460>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000119c:[<0x20000001254/picture-62.jpg v1073>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.092 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.809+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011c9 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-46.jpg auth v1081 ap=1 s=44028 n(v0 rc2023-07-18T03:32:31.489015+0000 b44028 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdea286000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011c9:[<0x20000001254/picture-46.jpg v588>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011c9:[<0x20000001254/picture-46.jpg v1081>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.092 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.809+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001189 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-73.jpg auth v1069 ap=1 s=54786 n(v0 rc2023-07-18T03:32:31.133020+0000 b54786 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdec248000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001189:[<0x20000001254/picture-73.jpg v372>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001189:[<0x20000001254/picture-73.jpg v1069>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.092 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.811+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001236 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-21.jpg auth v1101 ap=1 s=26930 n(v0 rc2023-07-18T03:32:32.202005+0000 b26930 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeb090000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001236:[<0x20000001254/picture-21.jpg v776>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001236:[<0x20000001254/picture-21.jpg v1101>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.092 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.811+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001186 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-75.jpg auth v1067 ap=1 s=60634 n(v0 rc2023-07-18T03:32:31.121020+0000 b60634 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdea287600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001186:[<0x20000001254/picture-75.jpg v356>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001186:[<0x20000001254/picture-75.jpg v1067>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.092 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.811+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000038e [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-1.xml auth v1103 ap=1 s=10172 n(v0 rc2023-07-18T03:33:18.565379+0000 b10172 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeacb4000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000038e:[<0x20000001254/comment-1.xml v1063>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000038e:[<0x20000001254/comment-1.xml v1103>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.092 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.811+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011ff [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-30.jpg auth v1077 ap=1 s=11310 n(v0 rc2023-07-18T03:32:31.836010+0000 b11310 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdec249080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011ff:[<0x20000001254/picture-30.jpg v716>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011ff:[<0x20000001254/picture-30.jpg v1077>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.093 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.812+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000010fc [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-15.xml auth v1095 ap=1 s=29068 n(v0 rc2023-07-18T03:32:30.883023+0000 b29068 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeacb4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000010fc:[<0x20000001254/article-15.xml v108>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000010fc:[<0x20000001254/article-15.xml v1095>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.093 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.812+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000367 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-40.xml auth v1111 ap=1 s=9006 n(v0 rc2023-07-18T03:33:18.391381+0000 b9006 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeacb4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000367:[<0x20000001254/comment-40.xml v996>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000367:[<0x20000001254/comment-40.xml v1111>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.093 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.812+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036e [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-33.xml auth v1107 ap=1 s=6969 n(v0 rc2023-07-18T03:33:18.426381+0000 b6969 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeacb5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000036e:[<0x20000001254/comment-33.xml v1008>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000036e:[<0x20000001254/comment-33.xml v1107>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.093 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.813+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000368 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-39.xml auth v1109 ap=1 s=4616 n(v0 rc2023-07-18T03:33:18.394381+0000 b4616 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeacb5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000368:[<0x20000001254/comment-39.xml v1000>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000368:[<0x20000001254/comment-39.xml v1109>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.093 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.813+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011ed [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-36.jpg auth v1075 ap=1 s=34753 n(v0 rc2023-07-18T03:32:31.750011+0000 b34753 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdec248b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011ed:[<0x20000001254/picture-36.jpg v668>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011ed:[<0x20000001254/picture-36.jpg v1075>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.093 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.813+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011d5 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-43.jpg auth v1071 ap=1 s=42477 n(v0 rc2023-07-18T03:32:31.577014+0000 b42477 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdec248580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011d5:[<0x20000001254/picture-43.jpg v612>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011d5:[<0x20000001254/picture-43.jpg v1071>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.093 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.813+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000010f5 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-19.xml auth v1097 ap=1 s=10311 n(v0 rc2023-07-18T03:32:30.854023+0000 b10311 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeacb5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000010f5:[<0x20000001254/article-19.xml v76>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000010f5:[<0x20000001254/article-19.xml v1097>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.094 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.813+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001239 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-20.jpg auth v1099 ap=1 s=46258 n(v0 rc2023-07-18T03:32:32.224005+0000 b46258 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeacb6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001239:[<0x20000001254/picture-20.jpg v784>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001239:[<0x20000001254/picture-20.jpg v1099>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.094 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.813+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036f [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-32.xml auth v1105 ap=1 s=2830 n(v0 rc2023-07-18T03:33:18.432381+0000 b2830 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeacb6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000036f:[<0x20000001254/comment-32.xml v1010>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000036f:[<0x20000001254/comment-32.xml v1105>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.094 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.814+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000359 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-54.xml auth v1113 ap=1 s=9892 n(v0 rc2023-07-18T03:33:18.267383+0000 b9892 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeacb6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000359:[<0x20000001254/comment-54.xml v973>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000359:[<0x20000001254/comment-54.xml v1113>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.094 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.814+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011e8 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-38.jpg auth v1073 ap=1 s=49917 n(v0 rc2023-07-18T03:32:31.718012+0000 b49917 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeacb7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011e8:[<0x20000001254/picture-38.jpg v652>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011e8:[<0x20000001254/picture-38.jpg v1073>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.094 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.814+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000122b [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-23.jpg auth v1111 ap=1 s=55058 n(v0 rc2023-07-18T03:32:32.097006+0000 b55058 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdec511b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000122b:[<0x20000001254/picture-23.jpg v940>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000122b:[<0x20000001254/picture-23.jpg v1111>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.094 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.814+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000117f [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-79.jpg auth v1033 ap=1 s=58321 n(v0 rc2023-07-18T03:32:31.093020+0000 b58321 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdec24a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000117f:[<0x20000001254/picture-79.jpg v324>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000117f:[<0x20000001254/picture-79.jpg v1033>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.094 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.814+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001100 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-14.xml auth v1103 ap=1 s=10488 n(v0 rc2023-07-18T03:32:30.904023+0000 b10488 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeba0b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001100:[<0x20000001254/article-14.xml v116>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001100:[<0x20000001254/article-14.xml v1103>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.095 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.814+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011cd [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-45.jpg auth v1047 ap=1 s=13859 n(v0 rc2023-07-18T03:32:31.512014+0000 b13859 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdea5c8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011cd:[<0x20000001254/picture-45.jpg v596>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011cd:[<0x20000001254/picture-45.jpg v1047>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.095 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.815+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001634 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-84.xml auth v1092 ap=1 s=12709 n(v0 rc2023-07-18T03:35:57.062236+0000 b12709 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeba08580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001634:[<0x20000001254/article-84.xml v1088>,<0x10000000143/blog-34 v111967>,<0x100000007de/blogtest_in v2096>,<0x100000007d9/src v1716>,<0x100000005e3/blogbench-1.0 v1496>,<0x100000005e2/tmp v96>,<0x100000001fb/client.0 v86>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v104>,<0x10000000001/sv_1 v117>,<0x10000000000/_nogroup v109>,<0x1/volumes v101>]//[]","memoryvalue":"(3)0x10000001634:[<0x20000001254/article-84.xml v1092>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.095 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.815+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011dd [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-41.jpg auth v1051 ap=1 s=52411 n(v0 rc2023-07-18T03:32:31.636013+0000 b52411 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdea5c9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011dd:[<0x20000001254/picture-41.jpg v628>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011dd:[<0x20000001254/picture-41.jpg v1051>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.095 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.815+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000119f [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-60.jpg auth v1039 ap=1 s=53673 n(v0 rc2023-07-18T03:32:31.221018+0000 b53673 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdec24b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000119f:[<0x20000001254/picture-60.jpg v476>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000119f:[<0x20000001254/picture-60.jpg v1039>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.095 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.818+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001195 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-66.jpg auth v1037 ap=1 s=37978 n(v0 rc2023-07-18T03:32:31.182019+0000 b37978 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdec24ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001195:[<0x20000001254/picture-66.jpg v428>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001195:[<0x20000001254/picture-66.jpg v1037>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.095 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.818+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000117e [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-80.jpg auth v1031 ap=1 s=14013 n(v0 rc2023-07-18T03:32:31.085020+0000 b14013 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdec249b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000117e:[<0x20000001254/picture-80.jpg v316>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000117e:[<0x20000001254/picture-80.jpg v1031>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.096 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.818+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000116c [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-91.jpg auth v1029 ap=1 s=25237 n(v0 rc2023-07-18T03:32:31.009021+0000 b25237 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdec249600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000116c:[<0x20000001254/picture-91.jpg v228>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000116c:[<0x20000001254/picture-91.jpg v1029>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.096 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.820+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001230 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-22.jpg auth v1109 ap=1 s=25479 n(v0 rc2023-07-18T03:32:32.120006+0000 b25479 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdea2c7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001230:[<0x20000001254/picture-22.jpg v768>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001230:[<0x20000001254/picture-22.jpg v1109>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.097 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.820+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001158 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-11.xml auth v1099 ap=1 s=17843 n(v0 rc2023-07-18T03:32:30.927022+0000 b17843 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeb55db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001158:[<0x20000001254/article-11.xml v140>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001158:[<0x20000001254/article-11.xml v1099>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.097 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.820+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000010fa [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-16.xml auth v1107 ap=1 s=24080 n(v0 rc2023-07-18T03:32:30.873023+0000 b24080 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeb55d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000010fa:[<0x20000001254/article-16.xml v100>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000010fa:[<0x20000001254/article-16.xml v1107>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.097 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.820+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000010e7 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-26.xml auth v1115 ap=1 s=23102 n(v0 rc2023-07-18T03:32:30.808024+0000 b23102 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde9c2eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000010e7:[<0x20000001254/article-26.xml v20>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000010e7:[<0x20000001254/article-26.xml v1115>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.097 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.820+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000124c [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-15.jpg auth v1105 ap=1 s=34085 n(v0 rc2023-07-18T03:32:32.294004+0000 b34085 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde9afd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000124c:[<0x20000001254/picture-15.jpg v824>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000124c:[<0x20000001254/picture-15.jpg v1105>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.097 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.820+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011a5 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-56.jpg auth v1043 ap=1 s=26359 n(v0 rc2023-07-18T03:32:31.244018+0000 b26359 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdea5c8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011a5:[<0x20000001254/picture-56.jpg v508>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011a5:[<0x20000001254/picture-56.jpg v1043>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.097 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.821+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011c5 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-47.jpg auth v1045 ap=1 s=45788 n(v0 rc2023-07-18T03:32:31.467015+0000 b45788 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdea5c8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011c5:[<0x20000001254/picture-47.jpg v580>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011c5:[<0x20000001254/picture-47.jpg v1045>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.097 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.821+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000115a [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-10.xml auth v1097 ap=1 s=9802 n(v0 rc2023-07-18T03:32:30.934022+0000 b9802 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde8cb9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000115a:[<0x20000001254/article-10.xml v148>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000115a:[<0x20000001254/article-10.xml v1097>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.098 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.821+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000125d [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-12.jpg auth v1101 ap=1 s=60528 n(v0 rc2023-07-18T03:32:32.325003+0000 b60528 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde81dc000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000125d:[<0x20000001254/picture-12.jpg v848>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000125d:[<0x20000001254/picture-12.jpg v1101>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.098 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.822+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011d1 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-44.jpg auth v1049 ap=1 s=17108 n(v0 rc2023-07-18T03:32:31.544014+0000 b17108 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdea5c9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011d1:[<0x20000001254/picture-44.jpg v604>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011d1:[<0x20000001254/picture-44.jpg v1049>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.098 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.822+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000035d [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-50.xml auth v1121 ap=1 s=6274 n(v0 rc2023-07-18T03:33:18.312383+0000 b6274 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeaf16100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000035d:[<0x20000001254/comment-50.xml v983>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000035d:[<0x20000001254/comment-50.xml v1121>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.098 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.822+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000035f [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-48.xml auth v1119 ap=1 s=3937 n(v0 rc2023-07-18T03:33:18.326382+0000 b3937 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde9833600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000035f:[<0x20000001254/comment-48.xml v985>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x1000000035f:[<0x20000001254/comment-48.xml v1119>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.098 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.822+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x20000001165 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-4.xml auth v1095 ap=1 s=20402 n(v0 rc2023-07-18T03:32:30.985022+0000 b20402 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeb6ee100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x20000001165:[<0x20000001254/article-4.xml v196>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x20000001165:[<0x20000001254/article-4.xml v1095>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.098 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.823+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000010e9 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-25.xml auth v1113 ap=1 s=13869 n(v0 rc2023-07-18T03:32:30.816024+0000 b13869 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bdeb574c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000010e9:[<0x20000001254/article-25.xml v28>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000010e9:[<0x20000001254/article-25.xml v1113>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.098 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.823+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x2000000118b [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-72.jpg auth v1035 ap=1 s=49054 n(v0 rc2023-07-18T03:32:31.138019+0000 b49054 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdec24a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x2000000118b:[<0x20000001254/picture-72.jpg v380>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x2000000118b:[<0x20000001254/picture-72.jpg v1035>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.099 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.823+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x200000011a0 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-59.jpg auth v1041 ap=1 s=56762 n(v0 rc2023-07-18T03:32:31.226018+0000 b56762 1=1+0) (iversion lock) | authpin=1 scrubqueue=0 0x55bdec24b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x200000011a0:[<0x20000001254/picture-59.jpg v484>,<0x10000000143/blog-34 v45759>,<0x100000007de/blogtest_in v430>,<0x100000007d9/src v1672>,<0x100000005e3/blogbench-1.0 v1451>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v42>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v60>,<0x10000000001/sv_1 v73>,<0x10000000000/_nogroup v64>,<0x1/volumes v55>]//[]","memoryvalue":"(3)0x200000011a0:[<0x20000001254/picture-59.jpg v1041>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.099 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:37:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:37:55.823+0000 7f2cc052b700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000370 [2,head] /volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-31.xml auth v1117 ap=1 s=4271 n(v0 rc2023-07-18T03:33:18.435381+0000 b4271 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55bde9b4eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000370:[<0x20000001254/comment-31.xml v1013>,<0x10000000143/blog-34 v110070>,<0x100000007de/blogtest_in v1152>,<0x100000007d9/src v1686>,<0x100000005e3/blogbench-1.0 v1466>,<0x100000005e2/tmp v66>,<0x100000001fb/client.0 v56>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v74>,<0x10000000001/sv_1 v87>,<0x10000000000/_nogroup v79>,<0x1/volumes v71>]//[]","memoryvalue":"(3)0x10000000370:[<0x20000001254/comment-31.xml v1117>,<0x10000000143/blog-34 v160491>,<0x100000007de/blogtest_in v2639>,<0x100000007d9/src v1738>,<0x100000005e3/blogbench-1.0 v1442>,<0x100000005e2/tmp v52>,<0x100000001fb/client.0 v33>,<0x100000001fa/9b5b0ec5-07bb-486b-9501-2d6d36216c9c v53>,<0x10000000001/sv_1 v69>,<0x10000000000/_nogroup v27>,<0x1/volumes v57>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-18T03:37:56.568 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001240 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-18.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.569 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000010f6 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-18.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.569 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001259 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-13.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.569 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000038a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-5.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.569 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000123c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-19.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.570 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000372 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-29.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.570 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000117a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-82.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.570 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000010e5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-27.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.570 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000127b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-4.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.570 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011f0 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-35.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.571 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001283 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-2.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.571 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001181 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-78.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.571 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000010f3 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-20.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.571 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001250 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-14.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.571 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000116f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-89.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000119d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-61.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000126f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-7.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001267 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-9.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000384 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-11.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000037b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-20.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001198 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-64.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.572 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000036a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-37.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.573 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001264 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-10.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.573 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001287 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-1.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.573 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000037f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-16.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.573 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000115c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-9.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.573 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001179 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-83.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.573 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001273 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-6.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.573 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000358 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-55.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.573 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000010eb (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-24.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.574 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000036d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-34.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.574 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000376 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-25.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.574 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001208 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-28.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.574 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000366 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-41.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.574 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011ea (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-37.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.574 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001244 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-17.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.574 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001175 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-85.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.575 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001167 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-3.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.575 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001225 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-24.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.575 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011f7 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-33.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.575 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011a3 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-57.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.575 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000354 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-7.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.575 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011d9 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-42.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.575 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001192 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-68.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.575 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011e5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-39.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.576 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000378 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-23.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.576 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000377 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-24.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.576 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000001633 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-25.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.576 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000115e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-8.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.577 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000360 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-47.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.577 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000361 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-46.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.577 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011fc (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-31.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.578 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011bd (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-49.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.578 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000126b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-8.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.578 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001171 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-88.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.578 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000121a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-26.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.578 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000010f1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-21.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.578 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000036b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-36.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.578 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001102 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-13.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.579 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000038b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-4.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.579 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001162 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-6.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.579 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001169 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-2.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.579 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000381 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-14.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.579 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011aa (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-54.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.579 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001202 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-29.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.579 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000037c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-19.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.580 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000385 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-10.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.580 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000127f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-3.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.580 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000389 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-6.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.580 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000035a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-53.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.580 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011a7 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-55.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.580 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001261 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-11.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.580 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000118e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-70.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.581 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001164 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-5.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.581 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000036c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-35.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.581 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000373 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-28.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.581 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000118d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-71.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.581 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000119a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-63.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.581 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011e1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-40.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.581 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000010ef (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-22.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.581 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000369 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-38.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.582 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001172 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-87.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.582 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001177 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-84.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.582 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000387 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-8.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.582 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000382 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-13.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.582 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000037a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-21.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.582 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000375 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-26.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.583 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001211 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-27.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.583 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001156 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-12.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.583 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011b1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-52.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.583 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000116a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-1.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.583 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000371 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-30.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.584 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011f3 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-34.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.584 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011ad (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-53.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.584 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001193 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-67.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.584 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000116e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-90.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.584 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000038c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-3.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.584 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000037d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-18.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.585 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001248 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-16.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.585 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000383 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-12.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.585 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001188 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-74.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.585 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000035e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-49.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.585 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000365 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-42.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.585 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000010ed (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-23.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.585 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000010f8 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-17.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.586 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001184 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-76.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.586 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011fa (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-32.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.586 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000035c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-51.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.586 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011c1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-48.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.586 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000379 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-22.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.586 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000038d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-2.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.586 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001174 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-86.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.587 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000363 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-44.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.587 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000117c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-81.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.587 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001277 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-5.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.587 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000374 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-27.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.587 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011b9 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-50.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.587 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011a2 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-58.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.587 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000035b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-52.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.587 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001197 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-65.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.588 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000388 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-7.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.588 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000380 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-15.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.588 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000364 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-43.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.588 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000037e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-17.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.588 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001255 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-28.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.588 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000386 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-9.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.588 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001183 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-77.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.589 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001190 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-69.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.589 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011b5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-51.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.589 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000362 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-45.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.589 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000119c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-62.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.589 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011c9 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-46.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.589 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001189 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-73.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.589 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001236 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-21.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.590 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001186 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-75.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.590 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000038e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-1.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.590 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011ff (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-30.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.590 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000010fc (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-15.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.590 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000367 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-40.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.590 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000036e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-33.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.590 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000368 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-39.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.590 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011ed (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-36.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.591 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011d5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-43.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.591 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000010f5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-19.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.591 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001239 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-20.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.591 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000036f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-32.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.591 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000359 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-54.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.591 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011e8 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-38.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.591 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000122b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-23.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.591 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000117f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-79.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.592 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001100 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-14.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.592 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011cd (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-45.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.592 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000001634 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-84.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.592 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011dd (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-41.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.592 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000119f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-60.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.592 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001195 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-66.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.592 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000117e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-80.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.593 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000116c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-91.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.593 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001230 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-22.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.593 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001158 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-11.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.593 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000010fa (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-16.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.593 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000010e7 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-26.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.593 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000124c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-15.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.593 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011a5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-56.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.593 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011c5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-47.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.594 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000115a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-10.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.594 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000125d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-12.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.594 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011d1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-44.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.594 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000035d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-50.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.594 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x1000000035f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-48.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.594 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x20000001165 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-4.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.594 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000010e9 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-25.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.595 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x2000000118b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-72.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.595 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x200000011a0 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-59.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.595 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:56 smithi134 ceph-mon[98216]: Scrub error on inode 0x10000000370 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-31.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.596 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001240 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-18.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.597 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000010f6 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-18.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.597 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001259 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-13.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.597 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000038a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-5.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.597 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000123c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-19.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.597 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000372 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-29.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.597 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000117a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-82.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.597 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000010e5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-27.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.598 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000127b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-4.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.598 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011f0 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-35.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.598 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001283 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-2.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.598 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001181 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-78.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.598 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000010f3 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-20.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.598 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001250 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-14.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.598 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000116f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-89.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.598 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000119d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-61.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.599 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000126f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-7.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.599 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001267 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-9.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.599 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000384 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-11.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.599 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000037b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-20.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.599 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001198 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-64.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.599 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000036a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-37.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.600 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001264 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-10.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.600 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001287 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-1.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.600 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000037f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-16.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.600 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000115c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-9.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.600 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001179 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-83.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.600 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001273 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-6.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.601 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000358 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-55.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.601 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000010eb (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-24.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.601 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000036d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-34.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.601 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000376 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-25.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.601 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001208 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-28.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.601 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000366 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-41.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.601 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011ea (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-37.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.602 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001244 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-17.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.602 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001175 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-85.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.602 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001167 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-3.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.602 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001225 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-24.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.602 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011f7 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-33.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.602 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011a3 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-57.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.602 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000354 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-7.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.602 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011d9 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-42.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.603 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001192 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-68.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.603 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011e5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-39.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.603 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000378 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-23.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.603 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000377 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-24.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.603 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000001633 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-25.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.603 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000115e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-8.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.603 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000360 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-47.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.603 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000361 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-46.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.604 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011fc (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-31.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.604 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011bd (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-49.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.604 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000126b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-8.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.604 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001171 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-88.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.604 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000121a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-26.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.604 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000010f1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-21.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.604 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000036b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-36.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.605 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001102 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-13.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.605 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000038b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-4.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.605 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001162 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-6.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.605 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001169 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-2.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.605 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000381 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-14.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.605 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011aa (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-54.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.605 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001202 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-29.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.606 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000037c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-19.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.606 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000385 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-10.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.606 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000127f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-3.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.606 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000389 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-6.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.606 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000035a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-53.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.606 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011a7 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-55.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.606 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001261 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-11.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.606 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000118e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-70.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.607 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001164 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-5.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.607 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000036c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-35.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.607 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000373 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-28.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.607 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000118d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-71.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.607 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000119a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-63.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.607 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011e1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-40.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.607 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000010ef (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-22.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.608 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000369 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-38.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.608 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001172 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-87.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.608 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001177 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-84.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.608 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000387 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-8.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.608 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000382 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-13.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.608 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000037a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-21.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.608 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000375 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-26.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.609 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001211 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-27.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.609 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001156 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-12.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.609 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011b1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-52.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.609 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000116a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-1.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.609 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000371 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-30.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.609 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011f3 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-34.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.609 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011ad (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-53.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.610 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001193 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-67.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.610 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000116e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-90.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.610 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000038c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-3.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.610 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000037d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-18.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.610 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001248 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-16.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.610 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000383 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-12.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.610 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001188 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-74.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.611 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000035e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-49.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.611 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000365 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-42.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.611 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000010ed (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-23.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.611 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000010f8 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-17.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.611 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001184 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-76.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.611 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011fa (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-32.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.611 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000035c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-51.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.612 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011c1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-48.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.612 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000379 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-22.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.612 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000038d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-2.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.613 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001174 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-86.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.613 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000363 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-44.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.613 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000117c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-81.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.613 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001277 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-5.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.614 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000374 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-27.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.614 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011b9 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-50.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.614 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011a2 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-58.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.614 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000035b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-52.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.615 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001197 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-65.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.615 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000388 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-7.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.615 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000380 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-15.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.615 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000364 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-43.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.616 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000037e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-17.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.616 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001255 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-28.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.616 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000386 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-9.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.617 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001183 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-77.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.617 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001190 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-69.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.617 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011b5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-51.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.618 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000362 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-45.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.618 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000119c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-62.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.618 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011c9 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-46.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.618 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001189 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-73.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.618 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001236 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-21.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.618 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001186 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-75.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.618 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000038e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-1.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.619 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011ff (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-30.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.619 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000010fc (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-15.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.619 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000367 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-40.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.619 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000036e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-33.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.619 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000368 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-39.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.619 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011ed (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-36.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.619 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011d5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-43.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.619 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000010f5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-19.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.620 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001239 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-20.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.620 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000036f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-32.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.620 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000359 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-54.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.620 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011e8 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-38.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.621 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000122b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-23.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.621 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000117f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-79.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.621 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001100 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-14.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.621 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011cd (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-45.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.621 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000001634 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-84.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.621 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011dd (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-41.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.622 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000119f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-60.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.622 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001195 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-66.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.622 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000117e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-80.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.622 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000116c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-91.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.622 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001230 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-22.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.622 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001158 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-11.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.622 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000010fa (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-16.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.623 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000010e7 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-26.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.623 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000124c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-15.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.623 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011a5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-56.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.623 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011c5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-47.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.623 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000115a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-10.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.623 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000125d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-12.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.623 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011d1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-44.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.623 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000035d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-50.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.624 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x1000000035f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-48.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.624 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x20000001165 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-4.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.624 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000010e9 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-25.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.624 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x2000000118b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-72.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.624 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x200000011a0 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-59.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.624 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:56 smithi040 ceph-mon[96439]: Scrub error on inode 0x10000000370 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-31.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.626 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001240 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-18.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.626 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000010f6 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-18.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.626 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001259 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-13.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.626 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000038a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-5.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.626 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000123c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-19.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.626 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000372 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-29.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.627 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000117a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-82.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.627 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000010e5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-27.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.627 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000127b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-4.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.627 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011f0 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-35.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.627 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001283 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-2.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.627 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001181 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-78.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.627 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000010f3 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-20.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.627 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001250 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-14.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.628 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000116f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-89.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.628 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000119d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-61.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.628 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000126f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-7.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.628 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001267 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-9.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.628 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000384 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-11.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.628 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000037b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-20.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.628 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001198 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-64.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000036a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-37.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001264 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-10.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001287 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-1.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000037f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-16.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000115c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-9.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001179 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-83.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.629 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001273 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-6.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.630 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000358 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-55.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.630 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000010eb (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-24.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.630 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000036d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-34.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.630 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000376 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-25.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.630 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001208 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-28.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.630 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000366 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-41.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.630 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011ea (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-37.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.630 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001244 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-17.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.631 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001175 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-85.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.631 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001167 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-3.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.631 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001225 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-24.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.631 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011f7 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-33.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.631 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011a3 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-57.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.631 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000354 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-7.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.631 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011d9 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-42.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.632 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001192 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-68.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.632 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011e5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-39.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.632 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000378 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-23.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.632 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000377 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-24.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.632 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000001633 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-25.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.632 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000115e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-8.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.633 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000360 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-47.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.633 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000361 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-46.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.633 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011fc (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-31.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.633 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011bd (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-49.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.633 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000126b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-8.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.633 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001171 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-88.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.633 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000121a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-26.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.633 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000010f1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-21.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.634 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000036b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-36.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.634 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001102 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-13.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.634 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000038b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-4.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.634 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001162 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-6.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.634 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001169 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-2.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.634 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000381 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-14.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.634 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011aa (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-54.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.635 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001202 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-29.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.635 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000037c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-19.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.635 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000385 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-10.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.635 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000127f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-3.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.635 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000389 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-6.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.636 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000035a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-53.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.636 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011a7 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-55.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.636 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001261 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-11.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.636 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000118e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-70.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.636 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001164 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-5.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.636 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000036c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-35.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.637 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000373 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-28.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.637 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000118d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-71.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.637 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000119a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-63.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.637 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011e1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-40.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.637 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000010ef (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-22.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.637 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000369 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-38.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.637 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001172 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-87.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.637 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001177 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-84.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.638 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000387 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-8.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.638 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000382 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-13.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.638 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000037a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-21.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.638 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000375 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-26.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.638 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001211 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-27.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.638 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001156 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-12.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.638 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011b1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-52.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.639 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000116a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-1.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.639 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000371 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-30.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.639 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011f3 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-34.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.639 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011ad (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-53.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.639 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001193 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-67.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.639 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000116e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-90.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.639 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000038c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-3.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.640 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000037d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-18.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.640 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001248 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-16.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.640 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000383 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-12.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.640 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001188 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-74.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.640 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000035e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-49.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.640 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000365 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-42.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.641 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000010ed (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-23.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.641 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000010f8 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-17.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.641 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001184 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-76.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.641 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011fa (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-32.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.641 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000035c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-51.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.641 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011c1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-48.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.641 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000379 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-22.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.641 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000038d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-2.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.642 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001174 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-86.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.642 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000363 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-44.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.642 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000117c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-81.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.642 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001277 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-5.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.642 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000374 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-27.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.642 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011b9 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-50.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.643 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011a2 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-58.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.643 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000035b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-52.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.643 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001197 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-65.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.643 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000388 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-7.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.644 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000380 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-15.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.644 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000364 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-43.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.644 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000037e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-17.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.645 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001255 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-28.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.645 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000386 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-9.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.645 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001183 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-77.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.645 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001190 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-69.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.646 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011b5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-51.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.646 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000362 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-45.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.646 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000119c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-62.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.647 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011c9 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-46.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.647 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001189 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-73.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.647 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001236 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-21.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.647 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001186 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-75.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.648 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000038e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-1.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.648 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011ff (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-30.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.648 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000010fc (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-15.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.648 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000367 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-40.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.649 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000036e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-33.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.649 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000368 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-39.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.649 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011ed (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-36.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.650 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011d5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-43.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.650 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000010f5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-19.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.650 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001239 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-20.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.650 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000036f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-32.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.651 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000359 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-54.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.651 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011e8 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-38.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.651 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000122b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-23.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.652 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000117f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-79.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.652 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001100 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-14.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.652 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011cd (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-45.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.652 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000001634 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-84.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.653 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011dd (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-41.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.653 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000119f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-60.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.653 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001195 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-66.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.654 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000117e (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-80.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.654 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000116c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-91.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.654 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001230 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-22.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.654 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001158 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-11.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.655 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000010fa (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-16.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.655 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000010e7 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-26.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.655 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000124c (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-15.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.656 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011a5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-56.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.656 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011c5 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-47.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.656 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000115a (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-10.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.656 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000125d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-12.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.657 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011d1 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-44.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.657 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000035d (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-50.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.657 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x1000000035f (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-48.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.657 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x20000001165 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-4.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.658 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000010e9 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-25.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.658 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x2000000118b (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-72.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.658 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x200000011a0 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-59.jpg) see mds.i log and `damage ls` output for details 2023-07-18T03:37:56.659 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:56 smithi064 ceph-mon[98059]: Scrub error on inode 0x10000000370 (/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-31.xml) see mds.i log and `damage ls` output for details 2023-07-18T03:37:57.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:57 smithi134 ceph-mon[98216]: pgmap v506: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 210 KiB/s wr, 72 op/s 2023-07-18T03:37:57.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:57 smithi040 ceph-mon[96439]: pgmap v506: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 210 KiB/s wr, 72 op/s 2023-07-18T03:37:57.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:57 smithi064 ceph-mon[98059]: pgmap v506: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 210 KiB/s wr, 72 op/s 2023-07-18T03:37:58.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:58 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:37:58.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:58 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:37:58.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:58 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:37:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.748+0000 7ff2bcd27700 1 -- 172.21.15.40:0/917597941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b80ffb90 msgr2=0x7ff2b80fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.748+0000 7ff2bcd27700 1 --2- 172.21.15.40:0/917597941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b80ffb90 0x7ff2b80fffe0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009990 tx=0x7ff2ac009ca0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.748+0000 7ff2bcd27700 1 -- 172.21.15.40:0/917597941 shutdown_connections 2023-07-18T03:37:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.748+0000 7ff2bcd27700 1 --2- 172.21.15.40:0/917597941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2b8100520 0x7ff2b8106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.748+0000 7ff2bcd27700 1 --2- 172.21.15.40:0/917597941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2b80fe9a0 0x7ff2b80fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.748+0000 7ff2bcd27700 1 --2- 172.21.15.40:0/917597941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b80ffb90 0x7ff2b80fffe0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.748+0000 7ff2bcd27700 1 -- 172.21.15.40:0/917597941 >> 172.21.15.40:0/917597941 conn(0x7ff2b80f9fa0 msgr2=0x7ff2b80fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.748+0000 7ff2bcd27700 1 -- 172.21.15.40:0/917597941 shutdown_connections 2023-07-18T03:37:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.748+0000 7ff2bcd27700 1 -- 172.21.15.40:0/917597941 wait complete. 2023-07-18T03:37:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.748+0000 7ff2bcd27700 1 Processor -- start 2023-07-18T03:37:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2bcd27700 1 -- start start 2023-07-18T03:37:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2bcd27700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2b80fe9a0 0x7ff2b8102c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2bcd27700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2b80ffb90 0x7ff2b8101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2bcd27700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8100520 0x7ff2b81017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2bcd27700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b8101f60 con 0x7ff2b8100520 2023-07-18T03:37:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2bcd27700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b81020d0 con 0x7ff2b80fe9a0 2023-07-18T03:37:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2bcd27700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b819f840 con 0x7ff2b80ffb90 2023-07-18T03:37:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2b6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8100520 0x7ff2b81017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2b659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2b80fe9a0 0x7ff2b8102c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2b6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8100520 0x7ff2b81017c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60656/0 (socket says 172.21.15.40:60656) 2023-07-18T03:37:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2b659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2b80fe9a0 0x7ff2b8102c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:43978/0 (socket says 172.21.15.40:43978) 2023-07-18T03:37:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2b6d9d700 1 -- 172.21.15.40:0/116729035 learned_addr learned my addr 172.21.15.40:0/116729035 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2b5d9b700 1 --2- 172.21.15.40:0/116729035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2b80ffb90 0x7ff2b8101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2b6d9d700 1 -- 172.21.15.40:0/116729035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2b80ffb90 msgr2=0x7ff2b8101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2b6d9d700 1 --2- 172.21.15.40:0/116729035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2b80ffb90 0x7ff2b8101280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2b6d9d700 1 -- 172.21.15.40:0/116729035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2b80fe9a0 msgr2=0x7ff2b8102c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2b6d9d700 1 --2- 172.21.15.40:0/116729035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2b80fe9a0 0x7ff2b8102c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2b6d9d700 1 -- 172.21.15.40:0/116729035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac0095f0 con 0x7ff2b8100520 2023-07-18T03:37:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.749+0000 7ff2b6d9d700 1 --2- 172.21.15.40:0/116729035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8100520 0x7ff2b81017c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000ef30 tx=0x7ff2b000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.750+0000 7ff2a77fe700 1 -- 172.21.15.40:0/116729035 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0018070 con 0x7ff2b8100520 2023-07-18T03:37:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.750+0000 7ff2bcd27700 1 -- 172.21.15.40:0/116729035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b819fac0 con 0x7ff2b8100520 2023-07-18T03:37:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.750+0000 7ff2a77fe700 1 -- 172.21.15.40:0/116729035 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2b000f040 con 0x7ff2b8100520 2023-07-18T03:37:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.750+0000 7ff2a77fe700 1 -- 172.21.15.40:0/116729035 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b000cd00 con 0x7ff2b8100520 2023-07-18T03:37:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.750+0000 7ff2bcd27700 1 -- 172.21.15.40:0/116729035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b81a0020 con 0x7ff2b8100520 2023-07-18T03:37:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.752+0000 7ff2a77fe700 1 -- 172.21.15.40:0/116729035 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff2b0003a00 con 0x7ff2b8100520 2023-07-18T03:37:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.752+0000 7ff2a77fe700 1 --2- 172.21.15.40:0/116729035 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff2a00a1150 0x7ff2a00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.753+0000 7ff2a77fe700 1 -- 172.21.15.40:0/116729035 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff2b00cc470 con 0x7ff2b8100520 2023-07-18T03:37:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.753+0000 7ff2b659c700 1 --2- 172.21.15.40:0/116729035 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff2a00a1150 0x7ff2a00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.753+0000 7ff2b659c700 1 --2- 172.21.15.40:0/116729035 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff2a00a1150 0x7ff2a00a3600 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80ff9f0 tx=0x7ff2a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.753+0000 7ff2c8eadb80 1 Processor -- start 2023-07-18T03:37:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.753+0000 7ff2c8eadb80 1 -- start start 2023-07-18T03:37:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2c8eadb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909eec86b0 0x55909f06c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2c8eadb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909f06c840 0x55909f06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2c8eadb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909f06d6e0 0x55909f06db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2c8eadb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55909ef71c50 con 0x55909eec86b0 2023-07-18T03:37:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2c8eadb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55909efc49e0 con 0x55909f06c840 2023-07-18T03:37:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2c8eadb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55909efbde70 con 0x55909f06d6e0 2023-07-18T03:37:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2b659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909eec86b0 0x55909f06c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2b5d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909f06c840 0x55909f06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2b659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909eec86b0 0x55909f06c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60660/0 (socket says 172.21.15.40:60660) 2023-07-18T03:37:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2b659c700 1 -- 172.21.15.40:0/76951944 learned_addr learned my addr 172.21.15.40:0/76951944 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2b6d9d700 1 --2- 172.21.15.40:0/76951944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909f06d6e0 0x55909f06db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2b659c700 1 -- 172.21.15.40:0/76951944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909f06d6e0 msgr2=0x55909f06db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2b659c700 1 --2- 172.21.15.40:0/76951944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909f06d6e0 0x55909f06db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2b659c700 1 -- 172.21.15.40:0/76951944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909f06c840 msgr2=0x55909f06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2b659c700 1 --2- 172.21.15.40:0/76951944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909f06c840 0x55909f06d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2b659c700 1 -- 172.21.15.40:0/76951944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55909edebd80 con 0x55909eec86b0 2023-07-18T03:37:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2b5d9b700 1 --2- 172.21.15.40:0/76951944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909f06c840 0x55909f06d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:37:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2b659c700 1 --2- 172.21.15.40:0/76951944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909eec86b0 0x55909f06c300 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8011d50 tx=0x7ff2a8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b83d18d08577049 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2a4ff9700 1 -- 172.21.15.40:0/76951944 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a800ed20 con 0x55909eec86b0 2023-07-18T03:37:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2a4ff9700 1 -- 172.21.15.40:0/76951944 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2a8021070 con 0x55909eec86b0 2023-07-18T03:37:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.754+0000 7ff2a4ff9700 1 -- 172.21.15.40:0/76951944 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a801d670 con 0x55909eec86b0 2023-07-18T03:37:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.755+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/76951944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909eec86b0 msgr2=0x55909f06c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.755+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/76951944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909eec86b0 0x55909f06c300 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8011d50 tx=0x7ff2a8014000 comp rx=0 tx=0).stop 2023-07-18T03:37:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.755+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/76951944 shutdown_connections 2023-07-18T03:37:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.755+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/76951944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909f06d6e0 0x55909f06db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.755+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/76951944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909eec86b0 0x55909f06c300 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.755+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/76951944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909f06c840 0x55909f06d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.755+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/76951944 >> 172.21.15.40:0/76951944 conn(0x55909f061be0 msgr2=0x55909f066cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.755+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/76951944 shutdown_connections 2023-07-18T03:37:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.755+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/76951944 wait complete. 2023-07-18T03:37:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2c8eadb80 1 -- start start 2023-07-18T03:37:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2c8eadb80 1 Processor -- start 2023-07-18T03:37:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2c8eadb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909eec86b0 0x55909f0758d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2c8eadb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909f06c840 0x55909f075e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2c8eadb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909f06d6e0 0x55909f07ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2c8eadb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55909ef5da60 con 0x55909eec86b0 2023-07-18T03:37:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2c8eadb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55909ef5d760 con 0x55909f06d6e0 2023-07-18T03:37:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2c8eadb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55909ee418c0 con 0x55909f06c840 2023-07-18T03:37:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2c8eadb80 10 client.0 _ll_register_callbacks cb 0x55909ef5b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2b659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909eec86b0 0x55909f0758d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2b5d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909f06c840 0x55909f075e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2b6d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909f06d6e0 0x55909f07ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2b5d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909f06c840 0x55909f075e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49694/0 (socket says 172.21.15.40:49694) 2023-07-18T03:37:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2b5d9b700 1 -- 172.21.15.40:0/1372684980 learned_addr learned my addr 172.21.15.40:0/1372684980 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2b659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909eec86b0 0x55909f0758d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60666/0 (socket says 172.21.15.40:60666) 2023-07-18T03:37:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2b659c700 1 -- 172.21.15.40:0/1372684980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909f06c840 msgr2=0x55909f075e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2b659c700 1 --2- 172.21.15.40:0/1372684980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909f06c840 0x55909f075e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2b659c700 1 -- 172.21.15.40:0/1372684980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909f06d6e0 msgr2=0x55909f07ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2b659c700 1 --2- 172.21.15.40:0/1372684980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909f06d6e0 0x55909f07ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.756+0000 7ff2b659c700 1 -- 172.21.15.40:0/1372684980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b8107710 con 0x55909eec86b0 2023-07-18T03:37:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.757+0000 7ff2b659c700 1 --2- 172.21.15.40:0/1372684980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909eec86b0 0x55909f0758d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800e970 tx=0x7ff2a801cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325782e455477a7d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.757+0000 7ff29dffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:37:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.757+0000 7ff29dffb700 1 -- 172.21.15.40:0/1372684980 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a801c410 con 0x55909eec86b0 2023-07-18T03:37:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.757+0000 7ff29dffb700 1 -- 172.21.15.40:0/1372684980 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff2a801c570 con 0x55909eec86b0 2023-07-18T03:37:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.757+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/1372684980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55909ef17490 con 0x55909eec86b0 2023-07-18T03:37:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.757+0000 7ff29dffb700 1 -- 172.21.15.40:0/1372684980 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2a801c730 con 0x55909eec86b0 2023-07-18T03:37:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.757+0000 7ff2c8eadb80 10 client.16512 fetch_fsmap learned FSMap version 49 2023-07-18T03:37:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.757+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/1372684980 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7ff2a8017070 con 0x55909eec86b0 2023-07-18T03:37:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.757+0000 7ff29dffb700 1 -- 172.21.15.40:0/1372684980 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55909ef17490 con 0x55909eec86b0 2023-07-18T03:37:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.757+0000 7ff29dffb700 1 -- 172.21.15.40:0/1372684980 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 49) v1 ==== 7522+0+0 (secure 0 0 0) 0x7ff2a8004950 con 0x55909eec86b0 2023-07-18T03:37:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.757+0000 7ff2c8eadb80 10 client.16512 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:37:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.757+0000 7ff2c8eadb80 10 client.16512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:37:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.757+0000 7ff2c8eadb80 20 client.16512 populate_metadata read hostname 'smithi040' 2023-07-18T03:37:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.757+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/1372684980 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55909f0816f0 0x55909f083b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.757+0000 7ff2c8eadb80 4 client.16512 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:37:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.757+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/1372684980 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55909efb6e50 con 0x55909f0816f0 2023-07-18T03:37:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.758+0000 7ff2b5d9b700 1 --2- 172.21.15.40:0/1372684980 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55909f0816f0 0x55909f083b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.758+0000 7ff2b5d9b700 1 --2- 172.21.15.40:0/1372684980 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55909f0816f0 0x55909f083b00 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98bef823e0308ea1 server_cookie=1702316d45edab68 in_seq=0 out_seq=0 2023-07-18T03:37:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.758+0000 7ff29dffb700 10 client.16512 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.759+0000 7ff29dffb700 1 -- 172.21.15.40:0/1372684980 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55909efb6e50 con 0x55909f0816f0 2023-07-18T03:37:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.759+0000 7ff29dffb700 10 client.16512 handle_command_reply: tid=0 2023-07-18T03:37:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.759+0000 7ff2c8eadb80 1 client.16512 shutdown 2023-07-18T03:37:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.759+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/1372684980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909eec86b0 msgr2=0x55909f0758d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.759+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/1372684980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909eec86b0 0x55909f0758d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800e970 tx=0x7ff2a801cc30 comp rx=0 tx=0).stop 2023-07-18T03:37:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.759+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/1372684980 shutdown_connections 2023-07-18T03:37:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.759+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/1372684980 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55909f0816f0 0x55909f083b00 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.759+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/1372684980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909f06d6e0 0x55909f07ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.759+0000 7ff29dffb700 0 client.16512 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.759+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/1372684980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909eec86b0 0x55909f0758d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.759+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/1372684980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909f06c840 0x55909f075e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.759+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/1372684980 >> 172.21.15.40:0/1372684980 conn(0x55909f061be0 msgr2=0x55909f0664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.759+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/1372684980 shutdown_connections 2023-07-18T03:37:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.759+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/1372684980 wait complete. 2023-07-18T03:37:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.759+0000 7ff2c8eadb80 20 client.16512 trim_cache size 0 max 16384 2023-07-18T03:37:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.774+0000 7ff2c8eadb80 1 Processor -- start 2023-07-18T03:37:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.774+0000 7ff2c8eadb80 1 -- start start 2023-07-18T03:37:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2c8eadb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909eec86b0 0x55909f073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2c8eadb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909f06c840 0x55909f073fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2c8eadb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909f06d6e0 0x55909f07ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2c8eadb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55909f081500 con 0x55909f06c840 2023-07-18T03:37:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2c8eadb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55909f032060 con 0x55909f06d6e0 2023-07-18T03:37:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2c8eadb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55909f063d00 con 0x55909eec86b0 2023-07-18T03:37:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2b5d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909f06c840 0x55909f073fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2b659c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909eec86b0 0x55909f073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2b5d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909f06c840 0x55909f073fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60676/0 (socket says 172.21.15.40:60676) 2023-07-18T03:37:58.788 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2b659c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909eec86b0 0x55909f073aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49702/0 (socket says 172.21.15.40:49702) 2023-07-18T03:37:58.788 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2b5d9b700 1 -- 172.21.15.40:0/3363483994 learned_addr learned my addr 172.21.15.40:0/3363483994 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2b6d9d700 1 --2- 172.21.15.40:0/3363483994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909f06d6e0 0x55909f07ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2b5d9b700 1 -- 172.21.15.40:0/3363483994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909eec86b0 msgr2=0x55909f073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2b5d9b700 1 --2- 172.21.15.40:0/3363483994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909eec86b0 0x55909f073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2b5d9b700 1 -- 172.21.15.40:0/3363483994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909f06d6e0 msgr2=0x55909f07ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2b5d9b700 1 --2- 172.21.15.40:0/3363483994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909f06d6e0 0x55909f07ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2b5d9b700 1 -- 172.21.15.40:0/3363483994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55909edd4c00 con 0x55909f06c840 2023-07-18T03:37:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2b5d9b700 1 --2- 172.21.15.40:0/3363483994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909f06c840 0x55909f073fe0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00b620 tx=0x7ff2ac005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e27626d3eee71c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2a57fa700 1 -- 172.21.15.40:0/3363483994 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac02a070 con 0x55909f06c840 2023-07-18T03:37:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2a57fa700 1 -- 172.21.15.40:0/3363483994 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2ac02e020 con 0x55909f06c840 2023-07-18T03:37:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.775+0000 7ff2a57fa700 1 -- 172.21.15.40:0/3363483994 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac01d040 con 0x55909f06c840 2023-07-18T03:37:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.776+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/3363483994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909f06c840 msgr2=0x55909f073fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.776+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/3363483994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909f06c840 0x55909f073fe0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00b620 tx=0x7ff2ac005960 comp rx=0 tx=0).stop 2023-07-18T03:37:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.776+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/3363483994 shutdown_connections 2023-07-18T03:37:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.776+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/3363483994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909f06d6e0 0x55909f07ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.776+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/3363483994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909eec86b0 0x55909f073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.776+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/3363483994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909f06c840 0x55909f073fe0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.776+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/3363483994 >> 172.21.15.40:0/3363483994 conn(0x55909f061be0 msgr2=0x55909f088460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.776+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/3363483994 shutdown_connections 2023-07-18T03:37:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.776+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/3363483994 wait complete. 2023-07-18T03:37:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.776+0000 7ff2c8eadb80 1 -- start start 2023-07-18T03:37:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.776+0000 7ff2c8eadb80 1 Processor -- start 2023-07-18T03:37:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2c8eadb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909eec86b0 0x55909f078ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2c8eadb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909f06c840 0x55909f0791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2c8eadb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909f06d6e0 0x55909f07c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2c8eadb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55909f07c740 con 0x55909f06d6e0 2023-07-18T03:37:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2c8eadb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55909f07c8b0 con 0x55909eec86b0 2023-07-18T03:37:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2c8eadb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55909f07ca70 con 0x55909f06c840 2023-07-18T03:37:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2c8eadb80 10 client.0 _ll_register_callbacks cb 0x7ff2a8005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:37:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2b6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909f06d6e0 0x55909f07c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2b6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909f06d6e0 0x55909f07c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60684/0 (socket says 172.21.15.40:60684) 2023-07-18T03:37:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2b5d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909f06c840 0x55909f0791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2b659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909eec86b0 0x55909f078ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2b6d9d700 1 -- 172.21.15.40:0/647018053 learned_addr learned my addr 172.21.15.40:0/647018053 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:37:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2b6d9d700 1 -- 172.21.15.40:0/647018053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909f06c840 msgr2=0x55909f0791e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2b6d9d700 1 --2- 172.21.15.40:0/647018053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909f06c840 0x55909f0791e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2b6d9d700 1 -- 172.21.15.40:0/647018053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909eec86b0 msgr2=0x55909f078ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:37:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2b6d9d700 1 --2- 172.21.15.40:0/647018053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909eec86b0 0x55909f078ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2b6d9d700 1 -- 172.21.15.40:0/647018053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ac0053e0 con 0x55909f06d6e0 2023-07-18T03:37:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2b6d9d700 1 --2- 172.21.15.40:0/647018053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909f06d6e0 0x55909f07c200 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0099820 tx=0x7ff2b0004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96223b80ea8eaa8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:37:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff29dffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:37:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff29dffb700 1 -- 172.21.15.40:0/647018053 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b00b92b0 con 0x55909f06d6e0 2023-07-18T03:37:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff29dffb700 1 -- 172.21.15.40:0/647018053 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff2b0099890 con 0x55909f06d6e0 2023-07-18T03:37:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.777+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/647018053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2a8021070 con 0x55909f06d6e0 2023-07-18T03:37:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.778+0000 7ff29dffb700 1 -- 172.21.15.40:0/647018053 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2b009d8c0 con 0x55909f06d6e0 2023-07-18T03:37:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.778+0000 7ff2c8eadb80 10 client.16518 fetch_fsmap learned FSMap version 49 2023-07-18T03:37:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.778+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/647018053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55909ee121f0 con 0x55909f06d6e0 2023-07-18T03:37:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.778+0000 7ff29dffb700 1 -- 172.21.15.40:0/647018053 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2a8021070 con 0x55909f06d6e0 2023-07-18T03:37:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.778+0000 7ff29dffb700 1 -- 172.21.15.40:0/647018053 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 49) v1 ==== 7522+0+0 (secure 0 0 0) 0x7ff2b0098d80 con 0x55909f06d6e0 2023-07-18T03:37:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.778+0000 7ff2c8eadb80 10 client.16518 fetch_fsmap finished waiting for FSMap version 49 2023-07-18T03:37:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.778+0000 7ff2c8eadb80 10 client.16518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:37:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.778+0000 7ff2c8eadb80 20 client.16518 populate_metadata read hostname 'smithi040' 2023-07-18T03:37:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.778+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/647018053 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55909f0d6780 0x55909f0d8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:37:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.778+0000 7ff2c8eadb80 4 client.16518 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:37:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.778+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/647018053 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55909ef78ff0 con 0x55909f0d6780 2023-07-18T03:37:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.778+0000 7ff2b659c700 1 --2- 172.21.15.40:0/647018053 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55909f0d6780 0x55909f0d8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:37:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.778+0000 7ff29dffb700 10 client.16518 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.778+0000 7ff2b659c700 1 --2- 172.21.15.40:0/647018053 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55909f0d6780 0x55909f0d8b90 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be06113a8894fc2a server_cookie=a11677800fa7bd6d in_seq=0 out_seq=0 2023-07-18T03:37:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.799+0000 7ff29dffb700 1 -- 172.21.15.40:0/647018053 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55909ef78ff0 con 0x55909f0d6780 2023-07-18T03:37:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.799+0000 7ff29dffb700 10 client.16518 handle_command_reply: tid=0 2023-07-18T03:37:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.799+0000 7ff2c8eadb80 1 client.16518 shutdown 2023-07-18T03:37:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.799+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/647018053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909f06d6e0 msgr2=0x55909f07c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.799+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/647018053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909f06d6e0 0x55909f07c200 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0099820 tx=0x7ff2b0004dc0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.799+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/647018053 shutdown_connections 2023-07-18T03:37:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.799+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/647018053 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55909f0d6780 0x55909f0d8b90 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.799+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/647018053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55909f06d6e0 0x55909f07c200 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.799+0000 7ff29dffb700 0 client.16518 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:37:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/647018053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909eec86b0 0x55909f078ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2c8eadb80 1 --2- 172.21.15.40:0/647018053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55909f06c840 0x55909f0791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/647018053 >> 172.21.15.40:0/647018053 conn(0x55909f061be0 msgr2=0x55909f073aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/647018053 shutdown_connections 2023-07-18T03:37:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2c8eadb80 1 -- 172.21.15.40:0/647018053 wait complete. 2023-07-18T03:37:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2c8eadb80 20 client.16518 trim_cache size 0 max 16384 2023-07-18T03:37:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:37:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:37:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:37:58.802 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:37:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2bcd27700 1 -- 172.21.15.40:0/116729035 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff2a00a1150 msgr2=0x7ff2a00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2bcd27700 1 --2- 172.21.15.40:0/116729035 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff2a00a1150 0x7ff2a00a3600 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80ff9f0 tx=0x7ff2a8009260 comp rx=0 tx=0).stop 2023-07-18T03:37:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2bcd27700 1 -- 172.21.15.40:0/116729035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8100520 msgr2=0x7ff2b81017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:37:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2bcd27700 1 --2- 172.21.15.40:0/116729035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8100520 0x7ff2b81017c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000ef30 tx=0x7ff2b000c3a0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2bcd27700 1 -- 172.21.15.40:0/116729035 shutdown_connections 2023-07-18T03:37:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2bcd27700 1 --2- 172.21.15.40:0/116729035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8100520 0x7ff2b81017c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2bcd27700 1 --2- 172.21.15.40:0/116729035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2b80fe9a0 0x7ff2b8102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2bcd27700 1 --2- 172.21.15.40:0/116729035 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff2a00a1150 0x7ff2a00a3600 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2bcd27700 1 --2- 172.21.15.40:0/116729035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff2b80ffb90 0x7ff2b8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:37:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2bcd27700 1 -- 172.21.15.40:0/116729035 >> 172.21.15.40:0/116729035 conn(0x7ff2b80f9fa0 msgr2=0x7ff2b80fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:37:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2bcd27700 1 -- 172.21.15.40:0/116729035 shutdown_connections 2023-07-18T03:37:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:37:58.800+0000 7ff2bcd27700 1 -- 172.21.15.40:0/116729035 wait complete. 2023-07-18T03:37:59.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:37:59 smithi134 ceph-mon[98216]: pgmap v507: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 267 KiB/s wr, 80 op/s 2023-07-18T03:37:59.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:37:59 smithi040 ceph-mon[96439]: pgmap v507: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 267 KiB/s wr, 80 op/s 2023-07-18T03:37:59.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:37:59 smithi064 ceph-mon[98059]: pgmap v507: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 267 KiB/s wr, 80 op/s 2023-07-18T03:38:01.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:01 smithi134 ceph-mon[98216]: pgmap v508: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 155 KiB/s wr, 47 op/s 2023-07-18T03:38:01.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:01 smithi040 ceph-mon[96439]: pgmap v508: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 155 KiB/s wr, 47 op/s 2023-07-18T03:38:01.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:01 smithi064 ceph-mon[98059]: pgmap v508: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 155 KiB/s wr, 47 op/s 2023-07-18T03:38:03.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:03 smithi134 ceph-mon[98216]: pgmap v509: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 263 KiB/s wr, 63 op/s 2023-07-18T03:38:03.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:03 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:03.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:03 smithi040 ceph-mon[96439]: pgmap v509: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 263 KiB/s wr, 63 op/s 2023-07-18T03:38:03.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:03 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:03.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:03 smithi064 ceph-mon[98059]: pgmap v509: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 263 KiB/s wr, 63 op/s 2023-07-18T03:38:03.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:03 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:05.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:05 smithi134 ceph-mon[98216]: pgmap v510: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 275 KiB/s wr, 62 op/s 2023-07-18T03:38:05.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:05 smithi040 ceph-mon[96439]: pgmap v510: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 275 KiB/s wr, 62 op/s 2023-07-18T03:38:05.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:05 smithi064 ceph-mon[98059]: pgmap v510: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 275 KiB/s wr, 62 op/s 2023-07-18T03:38:07.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:07 smithi134 ceph-mon[98216]: pgmap v511: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 276 KiB/s wr, 62 op/s 2023-07-18T03:38:07.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:07 smithi040 ceph-mon[96439]: pgmap v511: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 276 KiB/s wr, 62 op/s 2023-07-18T03:38:07.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:07 smithi064 ceph-mon[98059]: pgmap v511: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 276 KiB/s wr, 62 op/s 2023-07-18T03:38:09.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:09 smithi134 ceph-mon[98216]: pgmap v512: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 250 KiB/s wr, 56 op/s 2023-07-18T03:38:09.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:09 smithi040 ceph-mon[96439]: pgmap v512: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 250 KiB/s wr, 56 op/s 2023-07-18T03:38:09.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:09 smithi064 ceph-mon[98059]: pgmap v512: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 250 KiB/s wr, 56 op/s 2023-07-18T03:38:11.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:11 smithi134 ceph-mon[98216]: pgmap v513: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 143 KiB/s wr, 28 op/s 2023-07-18T03:38:11.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:11 smithi040 ceph-mon[96439]: pgmap v513: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 143 KiB/s wr, 28 op/s 2023-07-18T03:38:11.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:11 smithi064 ceph-mon[98059]: pgmap v513: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 143 KiB/s wr, 28 op/s 2023-07-18T03:38:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.922+0000 7ff986682700 1 -- 172.21.15.40:0/4259506733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9800ffd00 msgr2=0x7ff9801065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.922+0000 7ff986682700 1 --2- 172.21.15.40:0/4259506733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9800ffd00 0x7ff9801065c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff97800b0a0 tx=0x7ff97800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.922+0000 7ff986682700 1 -- 172.21.15.40:0/4259506733 shutdown_connections 2023-07-18T03:38:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.922+0000 7ff986682700 1 --2- 172.21.15.40:0/4259506733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9800ffd00 0x7ff9801065c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.922+0000 7ff986682700 1 --2- 172.21.15.40:0/4259506733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9800fe180 0x7ff9800fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.922+0000 7ff986682700 1 --2- 172.21.15.40:0/4259506733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9800ff370 0x7ff9800ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.922+0000 7ff986682700 1 -- 172.21.15.40:0/4259506733 >> 172.21.15.40:0/4259506733 conn(0x7ff9800f9780 msgr2=0x7ff9800fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.922+0000 7ff986682700 1 -- 172.21.15.40:0/4259506733 shutdown_connections 2023-07-18T03:38:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.922+0000 7ff986682700 1 -- 172.21.15.40:0/4259506733 wait complete. 2023-07-18T03:38:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.923+0000 7ff986682700 1 Processor -- start 2023-07-18T03:38:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.923+0000 7ff986682700 1 -- start start 2023-07-18T03:38:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.923+0000 7ff986682700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9800fe180 0x7ff980195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.923+0000 7ff986682700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9800ff370 0x7ff980195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.923+0000 7ff986682700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9800ffd00 0x7ff980196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.923+0000 7ff986682700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9801974a0 con 0x7ff9800ff370 2023-07-18T03:38:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.923+0000 7ff986682700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff980197610 con 0x7ff9800ffd00 2023-07-18T03:38:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.923+0000 7ff986682700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9801977d0 con 0x7ff9800fe180 2023-07-18T03:38:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.923+0000 7ff97f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9800ff370 0x7ff980195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.923+0000 7ff97ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9800fe180 0x7ff980195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.923+0000 7ff97f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9800ff370 0x7ff980195fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60694/0 (socket says 172.21.15.40:60694) 2023-07-18T03:38:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.923+0000 7ff97f7fe700 1 -- 172.21.15.40:0/99425357 learned_addr learned my addr 172.21.15.40:0/99425357 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.923+0000 7ff984c1f700 1 --2- 172.21.15.40:0/99425357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9800ffd00 0x7ff980196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.924+0000 7ff97f7fe700 1 -- 172.21.15.40:0/99425357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9800fe180 msgr2=0x7ff980195a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.924+0000 7ff97f7fe700 1 --2- 172.21.15.40:0/99425357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9800fe180 0x7ff980195a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.924+0000 7ff97f7fe700 1 -- 172.21.15.40:0/99425357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9800ffd00 msgr2=0x7ff980196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.924+0000 7ff97f7fe700 1 --2- 172.21.15.40:0/99425357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9800ffd00 0x7ff980196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.924+0000 7ff97f7fe700 1 -- 172.21.15.40:0/99425357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff978009d00 con 0x7ff9800ff370 2023-07-18T03:38:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.924+0000 7ff97f7fe700 1 --2- 172.21.15.40:0/99425357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9800ff370 0x7ff980195fc0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff97400e9a0 tx=0x7ff97400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.924+0000 7ff97d7fa700 1 -- 172.21.15.40:0/99425357 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff974009790 con 0x7ff9800ff370 2023-07-18T03:38:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.924+0000 7ff97d7fa700 1 -- 172.21.15.40:0/99425357 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff974010430 con 0x7ff9800ff370 2023-07-18T03:38:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.924+0000 7ff986682700 1 -- 172.21.15.40:0/99425357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff980197ab0 con 0x7ff9800ff370 2023-07-18T03:38:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.924+0000 7ff97d7fa700 1 -- 172.21.15.40:0/99425357 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff974018680 con 0x7ff9800ff370 2023-07-18T03:38:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.924+0000 7ff986682700 1 -- 172.21.15.40:0/99425357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98019e270 con 0x7ff9800ff370 2023-07-18T03:38:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.927+0000 7ff97d7fa700 1 -- 172.21.15.40:0/99425357 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff9740105a0 con 0x7ff9800ff370 2023-07-18T03:38:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.928+0000 7ff97d7fa700 1 --2- 172.21.15.40:0/99425357 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff9680a1150 0x7ff9680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.931 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.928+0000 7ff97ffff700 1 --2- 172.21.15.40:0/99425357 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff9680a1150 0x7ff9680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.928+0000 7ff97d7fa700 1 -- 172.21.15.40:0/99425357 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff974014070 con 0x7ff9800ff370 2023-07-18T03:38:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.928+0000 7ff992808b80 1 Processor -- start 2023-07-18T03:38:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.928+0000 7ff992808b80 1 -- start start 2023-07-18T03:38:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.928+0000 7ff97ffff700 1 --2- 172.21.15.40:0/99425357 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff9680a1150 0x7ff9680a3600 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff9800ff1d0 tx=0x7ff970009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.928+0000 7ff992808b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df949da6b0 0x55df94b826d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.928+0000 7ff992808b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df94b82c10 0x55df94b835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.928+0000 7ff992808b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df94b83b10 0x55df94b83fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.928+0000 7ff992808b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94a83c50 con 0x55df949da6b0 2023-07-18T03:38:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.928+0000 7ff992808b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94ad69e0 con 0x55df94b82c10 2023-07-18T03:38:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.928+0000 7ff992808b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94acfe70 con 0x55df94b83b10 2023-07-18T03:38:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.929+0000 7ff97ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df949da6b0 0x55df94b826d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.929+0000 7ff97ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df949da6b0 0x55df94b826d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60700/0 (socket says 172.21.15.40:60700) 2023-07-18T03:38:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.929+0000 7ff97ffff700 1 -- 172.21.15.40:0/1490864158 learned_addr learned my addr 172.21.15.40:0/1490864158 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.929+0000 7ff97f7fe700 1 --2- 172.21.15.40:0/1490864158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df94b82c10 0x55df94b835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.929+0000 7ff984c1f700 1 --2- 172.21.15.40:0/1490864158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df94b83b10 0x55df94b83fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.929+0000 7ff97ffff700 1 -- 172.21.15.40:0/1490864158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df94b83b10 msgr2=0x55df94b83fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.929+0000 7ff97ffff700 1 --2- 172.21.15.40:0/1490864158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df94b83b10 0x55df94b83fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.929+0000 7ff97ffff700 1 -- 172.21.15.40:0/1490864158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df94b82c10 msgr2=0x55df94b835d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.929+0000 7ff97ffff700 1 --2- 172.21.15.40:0/1490864158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df94b82c10 0x55df94b835d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.929+0000 7ff97ffff700 1 -- 172.21.15.40:0/1490864158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df948fdd80 con 0x55df949da6b0 2023-07-18T03:38:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.929+0000 7ff97f7fe700 1 --2- 172.21.15.40:0/1490864158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df94b82c10 0x55df94b835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.929+0000 7ff97ffff700 1 --2- 172.21.15.40:0/1490864158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df949da6b0 0x55df94b826d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff970011d50 tx=0x7ff970014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba6d8d5bf314269 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.929+0000 7ff9667fc700 1 -- 172.21.15.40:0/1490864158 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97000ed20 con 0x55df949da6b0 2023-07-18T03:38:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.929+0000 7ff9667fc700 1 -- 172.21.15.40:0/1490864158 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff970021070 con 0x55df949da6b0 2023-07-18T03:38:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.929+0000 7ff9667fc700 1 -- 172.21.15.40:0/1490864158 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97001d670 con 0x55df949da6b0 2023-07-18T03:38:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.930+0000 7ff992808b80 1 -- 172.21.15.40:0/1490864158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df949da6b0 msgr2=0x55df94b826d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.930+0000 7ff992808b80 1 --2- 172.21.15.40:0/1490864158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df949da6b0 0x55df94b826d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff970011d50 tx=0x7ff970014000 comp rx=0 tx=0).stop 2023-07-18T03:38:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.930+0000 7ff992808b80 1 -- 172.21.15.40:0/1490864158 shutdown_connections 2023-07-18T03:38:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.930+0000 7ff992808b80 1 --2- 172.21.15.40:0/1490864158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df94b83b10 0x55df94b83fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.930+0000 7ff992808b80 1 --2- 172.21.15.40:0/1490864158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df949da6b0 0x55df94b826d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.930+0000 7ff992808b80 1 --2- 172.21.15.40:0/1490864158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df94b82c10 0x55df94b835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.930+0000 7ff992808b80 1 -- 172.21.15.40:0/1490864158 >> 172.21.15.40:0/1490864158 conn(0x55df94b73be0 msgr2=0x55df94b78cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.930+0000 7ff992808b80 1 -- 172.21.15.40:0/1490864158 shutdown_connections 2023-07-18T03:38:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.930+0000 7ff992808b80 1 -- 172.21.15.40:0/1490864158 wait complete. 2023-07-18T03:38:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.930+0000 7ff992808b80 1 -- start start 2023-07-18T03:38:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff992808b80 1 Processor -- start 2023-07-18T03:38:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff992808b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df949da6b0 0x55df94b8bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff992808b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b82c10 0x55df94b8c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff992808b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df94b83b10 0x55df94b931d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff992808b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94a6fa60 con 0x55df94b82c10 2023-07-18T03:38:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff992808b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94a6f760 con 0x55df94b83b10 2023-07-18T03:38:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff992808b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df949538c0 con 0x55df949da6b0 2023-07-18T03:38:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff992808b80 10 client.0 _ll_register_callbacks cb 0x55df94a6d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff97f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b82c10 0x55df94b8c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff97ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df949da6b0 0x55df94b8bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff97ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df949da6b0 0x55df94b8bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49734/0 (socket says 172.21.15.40:49734) 2023-07-18T03:38:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff97ffff700 1 -- 172.21.15.40:0/1844205934 learned_addr learned my addr 172.21.15.40:0/1844205934 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff97f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b82c10 0x55df94b8c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60708/0 (socket says 172.21.15.40:60708) 2023-07-18T03:38:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff984c1f700 1 --2- 172.21.15.40:0/1844205934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df94b83b10 0x55df94b931d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff97ffff700 1 -- 172.21.15.40:0/1844205934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df94b83b10 msgr2=0x55df94b931d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff97ffff700 1 --2- 172.21.15.40:0/1844205934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df94b83b10 0x55df94b931d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff97ffff700 1 -- 172.21.15.40:0/1844205934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b82c10 msgr2=0x55df94b8c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff97ffff700 1 --2- 172.21.15.40:0/1844205934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b82c10 0x55df94b8c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.931+0000 7ff97ffff700 1 -- 172.21.15.40:0/1844205934 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98019e270 con 0x55df949da6b0 2023-07-18T03:38:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.932+0000 7ff97ffff700 1 --2- 172.21.15.40:0/1844205934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df949da6b0 0x55df94b8bc50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff97001cc30 tx=0x7ff97001cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=669e77e8c12c5436 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.932+0000 7ff95f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:38:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.932+0000 7ff95f7fe700 1 -- 172.21.15.40:0/1844205934 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97000ed80 con 0x55df949da6b0 2023-07-18T03:38:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.932+0000 7ff992808b80 1 -- 172.21.15.40:0/1844205934 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df94a29490 con 0x55df949da6b0 2023-07-18T03:38:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.932+0000 7ff95f7fe700 1 -- 172.21.15.40:0/1844205934 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff97001c620 con 0x55df949da6b0 2023-07-18T03:38:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.932+0000 7ff95f7fe700 1 -- 172.21.15.40:0/1844205934 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff97001c7e0 con 0x55df949da6b0 2023-07-18T03:38:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.932+0000 7ff992808b80 10 client.26200 fetch_fsmap learned FSMap version 50 2023-07-18T03:38:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.932+0000 7ff992808b80 1 -- 172.21.15.40:0/1844205934 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7ff970017070 con 0x55df949da6b0 2023-07-18T03:38:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.932+0000 7ff95f7fe700 1 -- 172.21.15.40:0/1844205934 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df94a29490 con 0x55df949da6b0 2023-07-18T03:38:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.933+0000 7ff95f7fe700 1 -- 172.21.15.40:0/1844205934 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 50) v1 ==== 7530+0+0 (secure 0 0 0) 0x7ff9700049f0 con 0x55df949da6b0 2023-07-18T03:38:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.933+0000 7ff992808b80 10 client.26200 fetch_fsmap finished waiting for FSMap version 50 2023-07-18T03:38:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.933+0000 7ff992808b80 10 client.26200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:38:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.933+0000 7ff992808b80 20 client.26200 populate_metadata read hostname 'smithi040' 2023-07-18T03:38:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.933+0000 7ff992808b80 1 --2- 172.21.15.40:0/1844205934 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55df94b97a70 0x55df94b99e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.933+0000 7ff992808b80 4 client.26200 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:38:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.933+0000 7ff992808b80 1 -- 172.21.15.40:0/1844205934 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df94ac8e50 con 0x55df94b97a70 2023-07-18T03:38:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.933+0000 7ff97f7fe700 1 --2- 172.21.15.40:0/1844205934 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55df94b97a70 0x55df94b99e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.933+0000 7ff97f7fe700 1 --2- 172.21.15.40:0/1844205934 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55df94b97a70 0x55df94b99e80 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62e2801dfd0a1059 server_cookie=e2ee8e0b2ba86b17 in_seq=0 out_seq=0 2023-07-18T03:38:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.933+0000 7ff95f7fe700 10 client.26200 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.935+0000 7ff95f7fe700 1 -- 172.21.15.40:0/1844205934 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55df94ac8e50 con 0x55df94b97a70 2023-07-18T03:38:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.935+0000 7ff95f7fe700 10 client.26200 handle_command_reply: tid=0 2023-07-18T03:38:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.935+0000 7ff992808b80 1 client.26200 shutdown 2023-07-18T03:38:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.936+0000 7ff992808b80 1 -- 172.21.15.40:0/1844205934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df949da6b0 msgr2=0x55df94b8bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.936+0000 7ff992808b80 1 --2- 172.21.15.40:0/1844205934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df949da6b0 0x55df94b8bc50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff97001cc30 tx=0x7ff97001cc60 comp rx=0 tx=0).stop 2023-07-18T03:38:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.936+0000 7ff992808b80 1 -- 172.21.15.40:0/1844205934 shutdown_connections 2023-07-18T03:38:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.936+0000 7ff992808b80 1 --2- 172.21.15.40:0/1844205934 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55df94b97a70 0x55df94b99e80 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.936+0000 7ff992808b80 1 --2- 172.21.15.40:0/1844205934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df94b83b10 0x55df94b931d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.936+0000 7ff95f7fe700 0 client.26200 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.936+0000 7ff992808b80 1 --2- 172.21.15.40:0/1844205934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df949da6b0 0x55df94b8bc50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.936+0000 7ff992808b80 1 --2- 172.21.15.40:0/1844205934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b82c10 0x55df94b8c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.936+0000 7ff992808b80 1 -- 172.21.15.40:0/1844205934 >> 172.21.15.40:0/1844205934 conn(0x55df94b73be0 msgr2=0x55df94b784e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.936+0000 7ff992808b80 1 -- 172.21.15.40:0/1844205934 shutdown_connections 2023-07-18T03:38:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.936+0000 7ff992808b80 1 -- 172.21.15.40:0/1844205934 wait complete. 2023-07-18T03:38:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.936+0000 7ff992808b80 20 client.26200 trim_cache size 0 max 16384 2023-07-18T03:38:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.950+0000 7ff992808b80 1 Processor -- start 2023-07-18T03:38:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff992808b80 1 -- start start 2023-07-18T03:38:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff992808b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df949da6b0 0x55df94b8b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff992808b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df94b82c10 0x55df94b93160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff992808b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b83b10 0x55df94b75150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff992808b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94b8b140 con 0x55df94b83b10 2023-07-18T03:38:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff992808b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94b79bc0 con 0x55df94b82c10 2023-07-18T03:38:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff992808b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94b75690 con 0x55df949da6b0 2023-07-18T03:38:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff984c1f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b83b10 0x55df94b75150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff984c1f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b83b10 0x55df94b75150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60718/0 (socket says 172.21.15.40:60718) 2023-07-18T03:38:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff984c1f700 1 -- 172.21.15.40:0/2315003630 learned_addr learned my addr 172.21.15.40:0/2315003630 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff97ffff700 1 --2- 172.21.15.40:0/2315003630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df949da6b0 0x55df94b8b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff97f7fe700 1 --2- 172.21.15.40:0/2315003630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df94b82c10 0x55df94b93160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff984c1f700 1 -- 172.21.15.40:0/2315003630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df949da6b0 msgr2=0x55df94b8b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff984c1f700 1 --2- 172.21.15.40:0/2315003630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df949da6b0 0x55df94b8b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff984c1f700 1 -- 172.21.15.40:0/2315003630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df94b82c10 msgr2=0x55df94b93160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff984c1f700 1 --2- 172.21.15.40:0/2315003630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df94b82c10 0x55df94b93160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff984c1f700 1 -- 172.21.15.40:0/2315003630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df948e6c00 con 0x55df94b83b10 2023-07-18T03:38:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.951+0000 7ff984c1f700 1 --2- 172.21.15.40:0/2315003630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b83b10 0x55df94b75150 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff97800b0a0 tx=0x7ff978009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ed15ae5ca40e54 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.952+0000 7ff966ffd700 1 -- 172.21.15.40:0/2315003630 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97800e040 con 0x55df94b83b10 2023-07-18T03:38:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.952+0000 7ff966ffd700 1 -- 172.21.15.40:0/2315003630 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9780077e0 con 0x55df94b83b10 2023-07-18T03:38:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.952+0000 7ff966ffd700 1 -- 172.21.15.40:0/2315003630 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9780053c0 con 0x55df94b83b10 2023-07-18T03:38:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.952+0000 7ff992808b80 1 -- 172.21.15.40:0/2315003630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b83b10 msgr2=0x55df94b75150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.952+0000 7ff992808b80 1 --2- 172.21.15.40:0/2315003630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b83b10 0x55df94b75150 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff97800b0a0 tx=0x7ff978009560 comp rx=0 tx=0).stop 2023-07-18T03:38:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.952+0000 7ff992808b80 1 -- 172.21.15.40:0/2315003630 shutdown_connections 2023-07-18T03:38:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.952+0000 7ff992808b80 1 --2- 172.21.15.40:0/2315003630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b83b10 0x55df94b75150 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.952+0000 7ff992808b80 1 --2- 172.21.15.40:0/2315003630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df949da6b0 0x55df94b8b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.952+0000 7ff992808b80 1 --2- 172.21.15.40:0/2315003630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df94b82c10 0x55df94b93160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.952+0000 7ff992808b80 1 -- 172.21.15.40:0/2315003630 >> 172.21.15.40:0/2315003630 conn(0x55df94b73be0 msgr2=0x55df94b78c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.952+0000 7ff992808b80 1 -- 172.21.15.40:0/2315003630 shutdown_connections 2023-07-18T03:38:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.952+0000 7ff992808b80 1 -- 172.21.15.40:0/2315003630 wait complete. 2023-07-18T03:38:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.953+0000 7ff992808b80 1 -- start start 2023-07-18T03:38:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.953+0000 7ff992808b80 1 Processor -- start 2023-07-18T03:38:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.953+0000 7ff992808b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df949da6b0 0x55df94b8f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.953+0000 7ff992808b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df94b82c10 0x55df94b8f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.953+0000 7ff992808b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b83b10 0x55df94b921f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.953+0000 7ff992808b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94b92730 con 0x55df94b83b10 2023-07-18T03:38:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.953+0000 7ff992808b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94b928a0 con 0x55df949da6b0 2023-07-18T03:38:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.953+0000 7ff992808b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94b92a60 con 0x55df94b82c10 2023-07-18T03:38:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.953+0000 7ff992808b80 10 client.0 _ll_register_callbacks cb 0x7ff974005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.953+0000 7ff984c1f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b83b10 0x55df94b921f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.953+0000 7ff97f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df94b82c10 0x55df94b8f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.953+0000 7ff984c1f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b83b10 0x55df94b921f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60724/0 (socket says 172.21.15.40:60724) 2023-07-18T03:38:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.953+0000 7ff984c1f700 1 -- 172.21.15.40:0/768026335 learned_addr learned my addr 172.21.15.40:0/768026335 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.953+0000 7ff97f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df94b82c10 0x55df94b8f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49750/0 (socket says 172.21.15.40:49750) 2023-07-18T03:38:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.953+0000 7ff97ffff700 1 --2- 172.21.15.40:0/768026335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df949da6b0 0x55df94b8f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff984c1f700 1 -- 172.21.15.40:0/768026335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df94b82c10 msgr2=0x55df94b8f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff984c1f700 1 --2- 172.21.15.40:0/768026335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df94b82c10 0x55df94b8f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff984c1f700 1 -- 172.21.15.40:0/768026335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df949da6b0 msgr2=0x55df94b8f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff984c1f700 1 --2- 172.21.15.40:0/768026335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df949da6b0 0x55df94b8f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff984c1f700 1 -- 172.21.15.40:0/768026335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9800673f0 con 0x55df94b83b10 2023-07-18T03:38:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff97f7fe700 1 --2- 172.21.15.40:0/768026335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df94b82c10 0x55df94b8f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff984c1f700 1 --2- 172.21.15.40:0/768026335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b83b10 0x55df94b921f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff978003fd0 tx=0x7ff978003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb1177f8a38c117 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff95f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:38:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff95f7fe700 1 -- 172.21.15.40:0/768026335 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97800e040 con 0x55df94b83b10 2023-07-18T03:38:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff95f7fe700 1 -- 172.21.15.40:0/768026335 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff978005350 con 0x55df94b83b10 2023-07-18T03:38:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff992808b80 1 -- 172.21.15.40:0/768026335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff970021070 con 0x55df94b83b10 2023-07-18T03:38:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff95f7fe700 1 -- 172.21.15.40:0/768026335 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff978005510 con 0x55df94b83b10 2023-07-18T03:38:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff992808b80 10 client.16536 fetch_fsmap learned FSMap version 50 2023-07-18T03:38:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff992808b80 1 -- 172.21.15.40:0/768026335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55df949241f0 con 0x55df94b83b10 2023-07-18T03:38:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff95f7fe700 1 -- 172.21.15.40:0/768026335 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff970021070 con 0x55df94b83b10 2023-07-18T03:38:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff95f7fe700 1 -- 172.21.15.40:0/768026335 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 50) v1 ==== 7530+0+0 (secure 0 0 0) 0x7ff97800b0d0 con 0x55df94b83b10 2023-07-18T03:38:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff992808b80 10 client.16536 fetch_fsmap finished waiting for FSMap version 50 2023-07-18T03:38:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff992808b80 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:38:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff992808b80 20 client.16536 populate_metadata read hostname 'smithi040' 2023-07-18T03:38:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff992808b80 1 --2- 172.21.15.40:0/768026335 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55df94bec500 0x55df94bee910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff992808b80 4 client.16536 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:38:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.954+0000 7ff992808b80 1 -- 172.21.15.40:0/768026335 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55df94a8aff0 con 0x55df94bec500 2023-07-18T03:38:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.955+0000 7ff97ffff700 1 --2- 172.21.15.40:0/768026335 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55df94bec500 0x55df94bee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.955+0000 7ff97ffff700 1 --2- 172.21.15.40:0/768026335 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55df94bec500 0x55df94bee910 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f63f521a1554d83d server_cookie=4ced9007cd610fd2 in_seq=0 out_seq=0 2023-07-18T03:38:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:11.955+0000 7ff95f7fe700 10 client.16536 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:13.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:12 smithi134 ceph-mon[98216]: pgmap v514: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 206 KiB/s wr, 34 op/s 2023-07-18T03:38:13.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:12 smithi040 ceph-mon[96439]: pgmap v514: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 206 KiB/s wr, 34 op/s 2023-07-18T03:38:13.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:12 smithi064 ceph-mon[98059]: pgmap v514: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 206 KiB/s wr, 34 op/s 2023-07-18T03:38:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.240+0000 7ff95f7fe700 1 -- 172.21.15.40:0/768026335 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55df94a8aff0 con 0x55df94bec500 2023-07-18T03:38:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.240+0000 7ff95f7fe700 10 client.16536 handle_command_reply: tid=0 2023-07-18T03:38:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.240+0000 7ff992808b80 1 client.16536 shutdown 2023-07-18T03:38:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.241+0000 7ff992808b80 1 -- 172.21.15.40:0/768026335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b83b10 msgr2=0x55df94b921f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.241+0000 7ff992808b80 1 --2- 172.21.15.40:0/768026335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b83b10 0x55df94b921f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff978003fd0 tx=0x7ff978003c30 comp rx=0 tx=0).stop 2023-07-18T03:38:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.241+0000 7ff992808b80 1 -- 172.21.15.40:0/768026335 shutdown_connections 2023-07-18T03:38:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.241+0000 7ff992808b80 1 --2- 172.21.15.40:0/768026335 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55df94bec500 0x55df94bee910 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.241+0000 7ff992808b80 1 --2- 172.21.15.40:0/768026335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df94b83b10 0x55df94b921f0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.241+0000 7ff992808b80 1 --2- 172.21.15.40:0/768026335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df949da6b0 0x55df94b8f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.241+0000 7ff992808b80 1 --2- 172.21.15.40:0/768026335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df94b82c10 0x55df94b8f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.241+0000 7ff95f7fe700 0 client.16536 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.241+0000 7ff992808b80 1 -- 172.21.15.40:0/768026335 >> 172.21.15.40:0/768026335 conn(0x55df94b73be0 msgr2=0x55df94b869c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.241+0000 7ff992808b80 1 -- 172.21.15.40:0/768026335 shutdown_connections 2023-07-18T03:38:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.241+0000 7ff992808b80 1 -- 172.21.15.40:0/768026335 wait complete. 2023-07-18T03:38:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.241+0000 7ff992808b80 20 client.16536 trim_cache size 0 max 16384 2023-07-18T03:38:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:38:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:38:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:38:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:38:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.241+0000 7ff986682700 1 -- 172.21.15.40:0/99425357 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff9680a1150 msgr2=0x7ff9680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.241+0000 7ff986682700 1 --2- 172.21.15.40:0/99425357 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff9680a1150 0x7ff9680a3600 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff9800ff1d0 tx=0x7ff970009260 comp rx=0 tx=0).stop 2023-07-18T03:38:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.241+0000 7ff986682700 1 -- 172.21.15.40:0/99425357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9800ff370 msgr2=0x7ff980195fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.241+0000 7ff986682700 1 --2- 172.21.15.40:0/99425357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9800ff370 0x7ff980195fc0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff97400e9a0 tx=0x7ff97400ecb0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.242+0000 7ff986682700 1 -- 172.21.15.40:0/99425357 shutdown_connections 2023-07-18T03:38:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.242+0000 7ff986682700 1 --2- 172.21.15.40:0/99425357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9800ffd00 0x7ff980196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.242+0000 7ff986682700 1 --2- 172.21.15.40:0/99425357 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff9680a1150 0x7ff9680a3600 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.242+0000 7ff986682700 1 --2- 172.21.15.40:0/99425357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9800fe180 0x7ff980195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.242+0000 7ff986682700 1 --2- 172.21.15.40:0/99425357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9800ff370 0x7ff980195fc0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.242+0000 7ff986682700 1 -- 172.21.15.40:0/99425357 >> 172.21.15.40:0/99425357 conn(0x7ff9800f9780 msgr2=0x7ff9800fb950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.242+0000 7ff986682700 1 -- 172.21.15.40:0/99425357 shutdown_connections 2023-07-18T03:38:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:15.242+0000 7ff986682700 1 -- 172.21.15.40:0/99425357 wait complete. 2023-07-18T03:38:15.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:15 smithi134 ceph-mon[98216]: pgmap v515: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 419 KiB/s wr, 21 op/s 2023-07-18T03:38:15.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:15 smithi040 ceph-mon[96439]: pgmap v515: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 419 KiB/s wr, 21 op/s 2023-07-18T03:38:15.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:15 smithi064 ceph-mon[98059]: pgmap v515: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 419 KiB/s wr, 21 op/s 2023-07-18T03:38:17.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:17 smithi134 ceph-mon[98216]: pgmap v516: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.1 MiB/s wr, 38 op/s 2023-07-18T03:38:17.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:17 smithi040 ceph-mon[96439]: pgmap v516: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.1 MiB/s wr, 38 op/s 2023-07-18T03:38:17.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:17 smithi064 ceph-mon[98059]: pgmap v516: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.1 MiB/s wr, 38 op/s 2023-07-18T03:38:19.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:19 smithi134 ceph-mon[98216]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 73 op/s 2023-07-18T03:38:19.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:19 smithi040 ceph-mon[96439]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 73 op/s 2023-07-18T03:38:19.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:19 smithi064 ceph-mon[98059]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 73 op/s 2023-07-18T03:38:21.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:21 smithi134 ceph-mon[98216]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 69 op/s 2023-07-18T03:38:21.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:21 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:21.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:21 smithi040 ceph-mon[96439]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 69 op/s 2023-07-18T03:38:21.579 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:21 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:21.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:21 smithi064 ceph-mon[98059]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 69 op/s 2023-07-18T03:38:21.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:21 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:22.745 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:38:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.895+0000 7fcb7076b700 1 -- 172.21.15.40:0/1319143437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb680fe9e0 msgr2=0x7fcb680fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.895+0000 7fcb7076b700 1 --2- 172.21.15.40:0/1319143437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb680fe9e0 0x7fcb680fedf0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009990 tx=0x7fcb64009ca0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.895+0000 7fcb7076b700 1 -- 172.21.15.40:0/1319143437 shutdown_connections 2023-07-18T03:38:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.895+0000 7fcb7076b700 1 --2- 172.21.15.40:0/1319143437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb68100560 0x7fcb68106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.895+0000 7fcb7076b700 1 --2- 172.21.15.40:0/1319143437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb680fe9e0 0x7fcb680fedf0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.895+0000 7fcb7076b700 1 --2- 172.21.15.40:0/1319143437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb680ffbd0 0x7fcb68100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.895+0000 7fcb7076b700 1 -- 172.21.15.40:0/1319143437 >> 172.21.15.40:0/1319143437 conn(0x7fcb680fa000 msgr2=0x7fcb680fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.895+0000 7fcb7076b700 1 -- 172.21.15.40:0/1319143437 shutdown_connections 2023-07-18T03:38:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.895+0000 7fcb7076b700 1 -- 172.21.15.40:0/1319143437 wait complete. 2023-07-18T03:38:22.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.895+0000 7fcb7076b700 1 Processor -- start 2023-07-18T03:38:22.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.895+0000 7fcb7076b700 1 -- start start 2023-07-18T03:38:22.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb7076b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb680fe9e0 0x7fcb68068200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb7076b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb680ffbd0 0x7fcb6819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb7076b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb68100560 0x7fcb6819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb7076b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6819d440 con 0x7fcb68100560 2023-07-18T03:38:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb7076b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6819d5b0 con 0x7fcb680ffbd0 2023-07-18T03:38:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb7076b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6819d770 con 0x7fcb680fe9e0 2023-07-18T03:38:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb6ed08700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb68100560 0x7fcb6819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb6ed08700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb68100560 0x7fcb6819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60740/0 (socket says 172.21.15.40:60740) 2023-07-18T03:38:22.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb6dd06700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb680ffbd0 0x7fcb6819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb6ed08700 1 -- 172.21.15.40:0/3154723382 learned_addr learned my addr 172.21.15.40:0/3154723382 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb6e507700 1 --2- 172.21.15.40:0/3154723382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb680fe9e0 0x7fcb68068200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb6ed08700 1 -- 172.21.15.40:0/3154723382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb680fe9e0 msgr2=0x7fcb68068200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb6ed08700 1 --2- 172.21.15.40:0/3154723382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb680fe9e0 0x7fcb68068200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb6ed08700 1 -- 172.21.15.40:0/3154723382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb680ffbd0 msgr2=0x7fcb6819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb6ed08700 1 --2- 172.21.15.40:0/3154723382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb680ffbd0 0x7fcb6819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb6ed08700 1 -- 172.21.15.40:0/3154723382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb640095f0 con 0x7fcb68100560 2023-07-18T03:38:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb6dd06700 1 --2- 172.21.15.40:0/3154723382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb680ffbd0 0x7fcb6819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb6ed08700 1 --2- 172.21.15.40:0/3154723382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb68100560 0x7fcb6819cc70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000ef30 tx=0x7fcb6000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.896+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3154723382 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb60018070 con 0x7fcb68100560 2023-07-18T03:38:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.897+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3154723382 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb6000f040 con 0x7fcb68100560 2023-07-18T03:38:22.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.897+0000 7fcb7076b700 1 -- 172.21.15.40:0/3154723382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6819d9f0 con 0x7fcb68100560 2023-07-18T03:38:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.897+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3154723382 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6000cd00 con 0x7fcb68100560 2023-07-18T03:38:22.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.897+0000 7fcb7076b700 1 -- 172.21.15.40:0/3154723382 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6819deb0 con 0x7fcb68100560 2023-07-18T03:38:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.899+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3154723382 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcb60003a00 con 0x7fcb68100560 2023-07-18T03:38:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.899+0000 7fcb5f7fe700 1 --2- 172.21.15.40:0/3154723382 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcb540a1150 0x7fcb540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.899+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3154723382 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcb600cc480 con 0x7fcb68100560 2023-07-18T03:38:22.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.899+0000 7fcb6e507700 1 --2- 172.21.15.40:0/3154723382 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcb540a1150 0x7fcb540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.900+0000 7fcb6e507700 1 --2- 172.21.15.40:0/3154723382 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcb540a1150 0x7fcb540a3600 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400b2e0 tx=0x7fcb64005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.900+0000 7fcb7c8f1b80 1 Processor -- start 2023-07-18T03:38:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.900+0000 7fcb7c8f1b80 1 -- start start 2023-07-18T03:38:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.900+0000 7fcb7c8f1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821d4d0 0x55df1821d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.900+0000 7fcb7c8f1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821de20 0x55df1821e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb7c8f1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821ec70 0x55df1821f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb7c8f1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df18122cb0 con 0x55df1821de20 2023-07-18T03:38:22.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb7c8f1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df18175a40 con 0x55df1821ec70 2023-07-18T03:38:22.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb7c8f1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1816eed0 con 0x55df1821d4d0 2023-07-18T03:38:22.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb6e507700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821d4d0 0x55df1821d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb6dd06700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821de20 0x55df1821e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb6e507700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821d4d0 0x55df1821d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49772/0 (socket says 172.21.15.40:49772) 2023-07-18T03:38:22.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb6dd06700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821de20 0x55df1821e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60746/0 (socket says 172.21.15.40:60746) 2023-07-18T03:38:22.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb6e507700 1 -- 172.21.15.40:0/2803753790 learned_addr learned my addr 172.21.15.40:0/2803753790 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:22.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb6ed08700 1 --2- 172.21.15.40:0/2803753790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821ec70 0x55df1821f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb6e507700 1 -- 172.21.15.40:0/2803753790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821ec70 msgr2=0x55df1821f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:22.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb6e507700 1 --2- 172.21.15.40:0/2803753790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821ec70 0x55df1821f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb6e507700 1 -- 172.21.15.40:0/2803753790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821de20 msgr2=0x55df1821e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:22.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb6e507700 1 --2- 172.21.15.40:0/2803753790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821de20 0x55df1821e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb6e507700 1 -- 172.21.15.40:0/2803753790 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df17f9cde0 con 0x55df1821d4d0 2023-07-18T03:38:22.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb6dd06700 1 --2- 172.21.15.40:0/2803753790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821de20 0x55df1821e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:22.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb6e507700 1 --2- 172.21.15.40:0/2803753790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821d4d0 0x55df1821d8e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcb680ffa30 tx=0x7fcb64020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9380001ca3110094 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:22.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb5cff9700 1 -- 172.21.15.40:0/2803753790 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6400f660 con 0x55df1821d4d0 2023-07-18T03:38:22.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb5cff9700 1 -- 172.21.15.40:0/2803753790 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb6401e040 con 0x55df1821d4d0 2023-07-18T03:38:22.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.901+0000 7fcb5cff9700 1 -- 172.21.15.40:0/2803753790 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6402fb30 con 0x55df1821d4d0 2023-07-18T03:38:22.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.902+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/2803753790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821d4d0 msgr2=0x55df1821d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:22.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.902+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/2803753790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821d4d0 0x55df1821d8e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcb680ffa30 tx=0x7fcb64020000 comp rx=0 tx=0).stop 2023-07-18T03:38:22.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.902+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/2803753790 shutdown_connections 2023-07-18T03:38:22.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.902+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/2803753790 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821ec70 0x55df1821f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.902+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/2803753790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821d4d0 0x55df1821d8e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.902+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/2803753790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821de20 0x55df1821e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.902+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/2803753790 >> 172.21.15.40:0/2803753790 conn(0x55df18212da0 msgr2=0x55df182181d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:22.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.902+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/2803753790 shutdown_connections 2023-07-18T03:38:22.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.902+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/2803753790 wait complete. 2023-07-18T03:38:22.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.903+0000 7fcb7c8f1b80 1 -- start start 2023-07-18T03:38:22.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.903+0000 7fcb7c8f1b80 1 Processor -- start 2023-07-18T03:38:22.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.903+0000 7fcb7c8f1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821d4d0 0x55df18227140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.903+0000 7fcb7c8f1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821de20 0x55df18227680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.903+0000 7fcb7c8f1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821ec70 0x55df1822e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.903+0000 7fcb7c8f1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1810eac0 con 0x55df1821de20 2023-07-18T03:38:22.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.903+0000 7fcb7c8f1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1810e7c0 con 0x55df1821d4d0 2023-07-18T03:38:22.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.903+0000 7fcb7c8f1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df17ff2920 con 0x55df1821ec70 2023-07-18T03:38:22.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.903+0000 7fcb7c8f1b80 10 client.0 _ll_register_callbacks cb 0x55df1810c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:22.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.903+0000 7fcb6dd06700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821de20 0x55df18227680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.903+0000 7fcb6e507700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821d4d0 0x55df18227140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.903+0000 7fcb6dd06700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821de20 0x55df18227680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60752/0 (socket says 172.21.15.40:60752) 2023-07-18T03:38:22.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.903+0000 7fcb6dd06700 1 -- 172.21.15.40:0/1677682237 learned_addr learned my addr 172.21.15.40:0/1677682237 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:22.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.903+0000 7fcb6ed08700 1 --2- 172.21.15.40:0/1677682237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821ec70 0x55df1822e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.903+0000 7fcb6dd06700 1 -- 172.21.15.40:0/1677682237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821ec70 msgr2=0x55df1822e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:22.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.903+0000 7fcb6dd06700 1 --2- 172.21.15.40:0/1677682237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821ec70 0x55df1822e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.904+0000 7fcb6dd06700 1 -- 172.21.15.40:0/1677682237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821d4d0 msgr2=0x55df18227140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:22.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.904+0000 7fcb6dd06700 1 --2- 172.21.15.40:0/1677682237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821d4d0 0x55df18227140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.904+0000 7fcb6dd06700 1 -- 172.21.15.40:0/1677682237 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb58009c90 con 0x55df1821de20 2023-07-18T03:38:22.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.904+0000 7fcb6e507700 1 --2- 172.21.15.40:0/1677682237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821d4d0 0x55df18227140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:22.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.904+0000 7fcb6ed08700 1 --2- 172.21.15.40:0/1677682237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821ec70 0x55df1822e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:22.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.904+0000 7fcb6dd06700 1 --2- 172.21.15.40:0/1677682237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821de20 0x55df18227680 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcb58012c90 tx=0x7fcb5800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fd23186624c56f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:22.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.904+0000 7fcb51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:38:22.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.904+0000 7fcb51ffb700 1 -- 172.21.15.40:0/1677682237 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58017070 con 0x55df1821de20 2023-07-18T03:38:22.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.904+0000 7fcb51ffb700 1 -- 172.21.15.40:0/1677682237 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcb5800f960 con 0x55df1821de20 2023-07-18T03:38:22.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.904+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/1677682237 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df180c84f0 con 0x55df1821de20 2023-07-18T03:38:22.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.904+0000 7fcb51ffb700 1 -- 172.21.15.40:0/1677682237 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb5800fb20 con 0x55df1821de20 2023-07-18T03:38:22.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.904+0000 7fcb7c8f1b80 10 client.16548 fetch_fsmap learned FSMap version 51 2023-07-18T03:38:22.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.904+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/1677682237 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7fcb68107750 con 0x55df1821de20 2023-07-18T03:38:22.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.904+0000 7fcb51ffb700 1 -- 172.21.15.40:0/1677682237 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df180c84f0 con 0x55df1821de20 2023-07-18T03:38:22.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.905+0000 7fcb51ffb700 1 -- 172.21.15.40:0/1677682237 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 51) v1 ==== 7526+0+0 (secure 0 0 0) 0x7fcb580038b0 con 0x55df1821de20 2023-07-18T03:38:22.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.905+0000 7fcb7c8f1b80 10 client.16548 fetch_fsmap finished waiting for FSMap version 51 2023-07-18T03:38:22.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.905+0000 7fcb7c8f1b80 10 client.16548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:38:22.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.905+0000 7fcb7c8f1b80 20 client.16548 populate_metadata read hostname 'smithi040' 2023-07-18T03:38:22.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.905+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/1677682237 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55df18233020 0x55df18235430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.905+0000 7fcb7c8f1b80 4 client.16548 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:38:22.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.905+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/1677682237 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df18167eb0 con 0x55df18233020 2023-07-18T03:38:22.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.905+0000 7fcb6e507700 1 --2- 172.21.15.40:0/1677682237 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55df18233020 0x55df18235430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.905+0000 7fcb51ffb700 10 client.16548 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:22.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.905+0000 7fcb6e507700 1 --2- 172.21.15.40:0/1677682237 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55df18233020 0x55df18235430 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bde8aaeef4e0f03 server_cookie=436c6004f2779c34 in_seq=0 out_seq=0 2023-07-18T03:38:22.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.906+0000 7fcb51ffb700 1 -- 172.21.15.40:0/1677682237 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55df18167eb0 con 0x55df18233020 2023-07-18T03:38:22.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.906+0000 7fcb51ffb700 10 client.16548 handle_command_reply: tid=0 2023-07-18T03:38:22.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.906+0000 7fcb7c8f1b80 1 client.16548 shutdown 2023-07-18T03:38:22.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.906+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/1677682237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821de20 msgr2=0x55df18227680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:22.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.906+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/1677682237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821de20 0x55df18227680 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcb58012c90 tx=0x7fcb5800f3c0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.906+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/1677682237 shutdown_connections 2023-07-18T03:38:22.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.906+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/1677682237 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55df18233020 0x55df18235430 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.906+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/1677682237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821ec70 0x55df1822e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.906+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/1677682237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821d4d0 0x55df18227140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.906+0000 7fcb51ffb700 0 client.16548 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:22.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.906+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/1677682237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821de20 0x55df18227680 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.906+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/1677682237 >> 172.21.15.40:0/1677682237 conn(0x55df18212da0 msgr2=0x55df182178b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:22.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.906+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/1677682237 shutdown_connections 2023-07-18T03:38:22.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.906+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/1677682237 wait complete. 2023-07-18T03:38:22.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.907+0000 7fcb7c8f1b80 20 client.16548 trim_cache size 0 max 16384 2023-07-18T03:38:22.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.924+0000 7fcb7c8f1b80 1 Processor -- start 2023-07-18T03:38:22.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.924+0000 7fcb7c8f1b80 1 -- start start 2023-07-18T03:38:22.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.924+0000 7fcb7c8f1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821d4d0 0x55df1822efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.924+0000 7fcb7c8f1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821de20 0x55df1822e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.924+0000 7fcb7c8f1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821ec70 0x55df18221c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.924+0000 7fcb7c8f1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df18212970 con 0x55df1821d4d0 2023-07-18T03:38:22.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.924+0000 7fcb7c8f1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df18230330 con 0x55df1821de20 2023-07-18T03:38:22.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.924+0000 7fcb7c8f1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df18219d00 con 0x55df1821ec70 2023-07-18T03:38:22.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.924+0000 7fcb6e507700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821d4d0 0x55df1822efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.924+0000 7fcb6ed08700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821ec70 0x55df18221c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.924+0000 7fcb6e507700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821d4d0 0x55df1822efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60758/0 (socket says 172.21.15.40:60758) 2023-07-18T03:38:22.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.925+0000 7fcb6e507700 1 -- 172.21.15.40:0/4069887962 learned_addr learned my addr 172.21.15.40:0/4069887962 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:22.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.924+0000 7fcb6dd06700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821de20 0x55df1822e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.925+0000 7fcb6e507700 1 -- 172.21.15.40:0/4069887962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821ec70 msgr2=0x55df18221c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:22.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.925+0000 7fcb6e507700 1 --2- 172.21.15.40:0/4069887962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821ec70 0x55df18221c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.925+0000 7fcb6e507700 1 -- 172.21.15.40:0/4069887962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821de20 msgr2=0x55df1822e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:22.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.925+0000 7fcb6e507700 1 --2- 172.21.15.40:0/4069887962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821de20 0x55df1822e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.925+0000 7fcb6e507700 1 -- 172.21.15.40:0/4069887962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df17f85c60 con 0x55df1821d4d0 2023-07-18T03:38:22.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.925+0000 7fcb6e507700 1 --2- 172.21.15.40:0/4069887962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821d4d0 0x55df1822efb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fcb64005a60 tx=0x7fcb64027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304863a2ffe9eaf1 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:22.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.925+0000 7fcb5d7fa700 1 -- 172.21.15.40:0/4069887962 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6403eb90 con 0x55df1821d4d0 2023-07-18T03:38:22.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.925+0000 7fcb5d7fa700 1 -- 172.21.15.40:0/4069887962 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb6403ecf0 con 0x55df1821d4d0 2023-07-18T03:38:22.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.925+0000 7fcb5d7fa700 1 -- 172.21.15.40:0/4069887962 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6403a9b0 con 0x55df1821d4d0 2023-07-18T03:38:22.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.926+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/4069887962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821d4d0 msgr2=0x55df1822efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:22.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.926+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/4069887962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821d4d0 0x55df1822efb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fcb64005a60 tx=0x7fcb64027040 comp rx=0 tx=0).stop 2023-07-18T03:38:22.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.926+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/4069887962 shutdown_connections 2023-07-18T03:38:22.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.926+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/4069887962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821ec70 0x55df18221c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.926+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/4069887962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821d4d0 0x55df1822efb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.926+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/4069887962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821de20 0x55df1822e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.926+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/4069887962 >> 172.21.15.40:0/4069887962 conn(0x55df18212da0 msgr2=0x55df18225df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:22.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.926+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/4069887962 shutdown_connections 2023-07-18T03:38:22.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.926+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/4069887962 wait complete. 2023-07-18T03:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.926+0000 7fcb7c8f1b80 1 -- start start 2023-07-18T03:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.926+0000 7fcb7c8f1b80 1 Processor -- start 2023-07-18T03:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb7c8f1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821d4d0 0x55df1822c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb7c8f1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821de20 0x55df1822c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb7c8f1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821ec70 0x55df1822d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb7c8f1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1822de80 con 0x55df1821d4d0 2023-07-18T03:38:22.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb7c8f1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1822dff0 con 0x55df1821de20 2023-07-18T03:38:22.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb7c8f1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df1822e1b0 con 0x55df1821ec70 2023-07-18T03:38:22.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb7c8f1b80 10 client.0 _ll_register_callbacks cb 0x7fcb58012760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:22.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb6e507700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821d4d0 0x55df1822c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb6dd06700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821de20 0x55df1822c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb6e507700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821d4d0 0x55df1822c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60764/0 (socket says 172.21.15.40:60764) 2023-07-18T03:38:22.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb6e507700 1 -- 172.21.15.40:0/528759091 learned_addr learned my addr 172.21.15.40:0/528759091 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:22.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb6ed08700 1 --2- 172.21.15.40:0/528759091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821ec70 0x55df1822d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb6e507700 1 -- 172.21.15.40:0/528759091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821ec70 msgr2=0x55df1822d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:22.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb6e507700 1 --2- 172.21.15.40:0/528759091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821ec70 0x55df1822d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb6e507700 1 -- 172.21.15.40:0/528759091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821de20 msgr2=0x55df1822c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:22.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb6e507700 1 --2- 172.21.15.40:0/528759091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821de20 0x55df1822c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb6e507700 1 -- 172.21.15.40:0/528759091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df17f9cde0 con 0x55df1821d4d0 2023-07-18T03:38:22.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb6e507700 1 --2- 172.21.15.40:0/528759091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821d4d0 0x55df1822c3f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fcb64000c00 tx=0x7fcb64005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd1ebd62a5d6028 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:22.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:38:22.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb51ffb700 1 -- 172.21.15.40:0/528759091 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6403e3d0 con 0x55df1821d4d0 2023-07-18T03:38:22.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb51ffb700 1 -- 172.21.15.40:0/528759091 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcb6403e530 con 0x55df1821d4d0 2023-07-18T03:38:22.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.927+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/528759091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb6401e040 con 0x55df1821d4d0 2023-07-18T03:38:22.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.928+0000 7fcb51ffb700 1 -- 172.21.15.40:0/528759091 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb6403a730 con 0x55df1821d4d0 2023-07-18T03:38:22.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.928+0000 7fcb7c8f1b80 10 client.16554 fetch_fsmap learned FSMap version 51 2023-07-18T03:38:22.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.928+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/528759091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55df17fc3250 con 0x55df1821d4d0 2023-07-18T03:38:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.928+0000 7fcb51ffb700 1 -- 172.21.15.40:0/528759091 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb64057080 con 0x55df1821d4d0 2023-07-18T03:38:22.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.928+0000 7fcb51ffb700 1 -- 172.21.15.40:0/528759091 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 51) v1 ==== 7526+0+0 (secure 0 0 0) 0x7fcb64051030 con 0x55df1821d4d0 2023-07-18T03:38:22.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.928+0000 7fcb7c8f1b80 10 client.16554 fetch_fsmap finished waiting for FSMap version 51 2023-07-18T03:38:22.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.928+0000 7fcb7c8f1b80 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:38:22.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.928+0000 7fcb7c8f1b80 20 client.16554 populate_metadata read hostname 'smithi040' 2023-07-18T03:38:22.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.928+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/528759091 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55df18287b20 0x55df18289f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:22.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.928+0000 7fcb7c8f1b80 4 client.16554 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:38:22.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.928+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/528759091 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55df1812a050 con 0x55df18287b20 2023-07-18T03:38:22.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.928+0000 7fcb6dd06700 1 --2- 172.21.15.40:0/528759091 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55df18287b20 0x55df18289f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:22.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.928+0000 7fcb51ffb700 10 client.16554 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:22.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.928+0000 7fcb6dd06700 1 --2- 172.21.15.40:0/528759091 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55df18287b20 0x55df18289f30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18ae0bbe32523b89 server_cookie=2f5521f0befa94de in_seq=0 out_seq=0 2023-07-18T03:38:22.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.929+0000 7fcb51ffb700 1 -- 172.21.15.40:0/528759091 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55df1812a050 con 0x55df18287b20 2023-07-18T03:38:22.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.929+0000 7fcb51ffb700 10 client.16554 handle_command_reply: tid=0 2023-07-18T03:38:22.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.929+0000 7fcb7c8f1b80 1 client.16554 shutdown 2023-07-18T03:38:22.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.929+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/528759091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821d4d0 msgr2=0x55df1822c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:22.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.929+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/528759091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821d4d0 0x55df1822c3f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fcb64000c00 tx=0x7fcb64005690 comp rx=0 tx=0).stop 2023-07-18T03:38:22.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.929+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/528759091 shutdown_connections 2023-07-18T03:38:22.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.929+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/528759091 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55df18287b20 0x55df18289f30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.929+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/528759091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55df1821ec70 0x55df1822d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.929+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/528759091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55df1821d4d0 0x55df1822c3f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.929+0000 7fcb7c8f1b80 1 --2- 172.21.15.40:0/528759091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df1821de20 0x55df1822c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.929+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/528759091 >> 172.21.15.40:0/528759091 conn(0x55df18212da0 msgr2=0x55df1822efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.929+0000 7fcb51ffb700 0 client.16554 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.929+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/528759091 shutdown_connections 2023-07-18T03:38:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.929+0000 7fcb7c8f1b80 1 -- 172.21.15.40:0/528759091 wait complete. 2023-07-18T03:38:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.929+0000 7fcb7c8f1b80 20 client.16554 trim_cache size 0 max 16384 2023-07-18T03:38:22.938 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:38:22.938 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-18T03:38:22.938 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:38:22.938 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:38:22.939 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:38:22.939 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:38:22.939 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:38:22.939 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:38:22.939 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:38:22.939 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:38:22.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.930+0000 7fcb7076b700 1 -- 172.21.15.40:0/3154723382 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcb540a1150 msgr2=0x7fcb540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:22.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.930+0000 7fcb7076b700 1 --2- 172.21.15.40:0/3154723382 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcb540a1150 0x7fcb540a3600 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400b2e0 tx=0x7fcb64005fb0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.930+0000 7fcb7076b700 1 -- 172.21.15.40:0/3154723382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb68100560 msgr2=0x7fcb6819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:22.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.930+0000 7fcb7076b700 1 --2- 172.21.15.40:0/3154723382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb68100560 0x7fcb6819cc70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000ef30 tx=0x7fcb6000c3a0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.930+0000 7fcb7076b700 1 -- 172.21.15.40:0/3154723382 shutdown_connections 2023-07-18T03:38:22.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.930+0000 7fcb7076b700 1 --2- 172.21.15.40:0/3154723382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb68100560 0x7fcb6819cc70 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.930+0000 7fcb7076b700 1 --2- 172.21.15.40:0/3154723382 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcb540a1150 0x7fcb540a3600 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.930+0000 7fcb7076b700 1 --2- 172.21.15.40:0/3154723382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb680fe9e0 0x7fcb68068200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.930+0000 7fcb7076b700 1 --2- 172.21.15.40:0/3154723382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb680ffbd0 0x7fcb6819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:22.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.930+0000 7fcb7076b700 1 -- 172.21.15.40:0/3154723382 >> 172.21.15.40:0/3154723382 conn(0x7fcb680fa000 msgr2=0x7fcb680fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:22.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.930+0000 7fcb7076b700 1 -- 172.21.15.40:0/3154723382 shutdown_connections 2023-07-18T03:38:22.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:22.930+0000 7fcb7076b700 1 -- 172.21.15.40:0/3154723382 wait complete. 2023-07-18T03:38:22.943 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:38:23.078 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:22 smithi040 ceph-mon[96439]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 82 op/s 2023-07-18T03:38:23.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:22 smithi134 ceph-mon[98216]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 82 op/s 2023-07-18T03:38:23.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:22 smithi064 ceph-mon[98059]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 82 op/s 2023-07-18T03:38:25.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:25 smithi134 ceph-mon[98216]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 84 op/s 2023-07-18T03:38:25.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:25 smithi040 ceph-mon[96439]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 84 op/s 2023-07-18T03:38:25.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:25 smithi064 ceph-mon[98059]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 84 op/s 2023-07-18T03:38:27.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:27 smithi134 ceph-mon[98216]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 85 op/s 2023-07-18T03:38:27.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:27 smithi040 ceph-mon[96439]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 85 op/s 2023-07-18T03:38:27.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:27 smithi064 ceph-mon[98059]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.1 MiB/s wr, 85 op/s 2023-07-18T03:38:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.367+0000 7fb5219ca700 1 -- 172.21.15.40:0/2916405825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb51c0ffb90 msgr2=0x7fb51c0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.367+0000 7fb5219ca700 1 --2- 172.21.15.40:0/2916405825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb51c0ffb90 0x7fb51c0fffa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009990 tx=0x7fb50c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.368+0000 7fb5219ca700 1 -- 172.21.15.40:0/2916405825 shutdown_connections 2023-07-18T03:38:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.368+0000 7fb5219ca700 1 --2- 172.21.15.40:0/2916405825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb51c101710 0x7fb51c107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.368+0000 7fb5219ca700 1 --2- 172.21.15.40:0/2916405825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb51c0ffb90 0x7fb51c0fffa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.368+0000 7fb5219ca700 1 --2- 172.21.15.40:0/2916405825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51c100d80 0x7fb51c1011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.368+0000 7fb5219ca700 1 -- 172.21.15.40:0/2916405825 >> 172.21.15.40:0/2916405825 conn(0x7fb51c0fb190 msgr2=0x7fb51c0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.368+0000 7fb5219ca700 1 -- 172.21.15.40:0/2916405825 shutdown_connections 2023-07-18T03:38:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.368+0000 7fb5219ca700 1 -- 172.21.15.40:0/2916405825 wait complete. 2023-07-18T03:38:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.368+0000 7fb5219ca700 1 Processor -- start 2023-07-18T03:38:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.368+0000 7fb5219ca700 1 -- start start 2023-07-18T03:38:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb5219ca700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb51c0ffb90 0x7fb51c195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb5219ca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51c100d80 0x7fb51c195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb5219ca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb51c101710 0x7fb51c196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb5219ca700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c197320 con 0x7fb51c100d80 2023-07-18T03:38:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb5219ca700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c197490 con 0x7fb51c101710 2023-07-18T03:38:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb5219ca700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c197650 con 0x7fb51c0ffb90 2023-07-18T03:38:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb51a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51c100d80 0x7fb51c195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb51affd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb51c0ffb90 0x7fb51c195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb51a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51c100d80 0x7fb51c195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60780/0 (socket says 172.21.15.40:60780) 2023-07-18T03:38:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb51a7fc700 1 -- 172.21.15.40:0/3926969307 learned_addr learned my addr 172.21.15.40:0/3926969307 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb51affd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb51c0ffb90 0x7fb51c195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49806/0 (socket says 172.21.15.40:49806) 2023-07-18T03:38:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb51b7fe700 1 --2- 172.21.15.40:0/3926969307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb51c101710 0x7fb51c196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb51a7fc700 1 -- 172.21.15.40:0/3926969307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb51c0ffb90 msgr2=0x7fb51c195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb51a7fc700 1 --2- 172.21.15.40:0/3926969307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb51c0ffb90 0x7fb51c195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb51a7fc700 1 -- 172.21.15.40:0/3926969307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb51c101710 msgr2=0x7fb51c196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb51a7fc700 1 --2- 172.21.15.40:0/3926969307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb51c101710 0x7fb51c196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb51a7fc700 1 -- 172.21.15.40:0/3926969307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c0095f0 con 0x7fb51c100d80 2023-07-18T03:38:28.369 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb51affd700 1 --2- 172.21.15.40:0/3926969307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb51c0ffb90 0x7fb51c195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.369+0000 7fb51a7fc700 1 --2- 172.21.15.40:0/3926969307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51c100d80 0x7fb51c195e40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb51000e9a0 tx=0x7fb51000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.370+0000 7fb5209c8700 1 -- 172.21.15.40:0/3926969307 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510009790 con 0x7fb51c100d80 2023-07-18T03:38:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.370+0000 7fb5209c8700 1 -- 172.21.15.40:0/3926969307 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb510010430 con 0x7fb51c100d80 2023-07-18T03:38:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.370+0000 7fb5209c8700 1 -- 172.21.15.40:0/3926969307 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510018680 con 0x7fb51c100d80 2023-07-18T03:38:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.370+0000 7fb5219ca700 1 -- 172.21.15.40:0/3926969307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c197930 con 0x7fb51c100d80 2023-07-18T03:38:28.370 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.370+0000 7fb5219ca700 1 -- 172.21.15.40:0/3926969307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51c19e0f0 con 0x7fb51c100d80 2023-07-18T03:38:28.374 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.373+0000 7fb5209c8700 1 -- 172.21.15.40:0/3926969307 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb5100105a0 con 0x7fb51c100d80 2023-07-18T03:38:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.373+0000 7fb5209c8700 1 --2- 172.21.15.40:0/3926969307 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb5040a11d0 0x7fb5040a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.373+0000 7fb5209c8700 1 -- 172.21.15.40:0/3926969307 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb510014070 con 0x7fb51c100d80 2023-07-18T03:38:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.373+0000 7fb51affd700 1 --2- 172.21.15.40:0/3926969307 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb5040a11d0 0x7fb5040a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.374+0000 7fb51affd700 1 --2- 172.21.15.40:0/3926969307 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb5040a11d0 0x7fb5040a3680 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb50c006010 tx=0x7fb50c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.374+0000 7fb52db50b80 1 Processor -- start 2023-07-18T03:38:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.374+0000 7fb52db50b80 1 -- start start 2023-07-18T03:38:28.376 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.374+0000 7fb52db50b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e3239e6b0 0x559e32542300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.374+0000 7fb52db50b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 0x559e325431a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.374+0000 7fb52db50b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e325436e0 0x559e32543b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.374+0000 7fb52db50b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e32447c50 con 0x559e325436e0 2023-07-18T03:38:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.374+0000 7fb52db50b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3249a9e0 con 0x559e3239e6b0 2023-07-18T03:38:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.374+0000 7fb52db50b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e32493e70 con 0x559e32542840 2023-07-18T03:38:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.374+0000 7fb51affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e3239e6b0 0x559e32542300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.374+0000 7fb51b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e325436e0 0x559e32543b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.374+0000 7fb51a7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 0x559e325431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.374+0000 7fb51affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e3239e6b0 0x559e32542300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:44112/0 (socket says 172.21.15.40:44112) 2023-07-18T03:38:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.374+0000 7fb51b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e325436e0 0x559e32543b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60790/0 (socket says 172.21.15.40:60790) 2023-07-18T03:38:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.374+0000 7fb51a7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 0x559e325431a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49816/0 (socket says 172.21.15.40:49816) 2023-07-18T03:38:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.374+0000 7fb51affd700 1 -- 172.21.15.40:0/4177472059 learned_addr learned my addr 172.21.15.40:0/4177472059 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.375+0000 7fb51affd700 1 -- 172.21.15.40:0/4177472059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 msgr2=0x559e325431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.375+0000 7fb51affd700 1 --2- 172.21.15.40:0/4177472059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 0x559e325431a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.375+0000 7fb51affd700 1 -- 172.21.15.40:0/4177472059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e325436e0 msgr2=0x559e32543b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.375+0000 7fb51affd700 1 --2- 172.21.15.40:0/4177472059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e325436e0 0x559e32543b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.375+0000 7fb51affd700 1 -- 172.21.15.40:0/4177472059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e322c1d80 con 0x559e3239e6b0 2023-07-18T03:38:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.375+0000 7fb51b7fe700 1 --2- 172.21.15.40:0/4177472059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e325436e0 0x559e32543b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.375+0000 7fb51affd700 1 --2- 172.21.15.40:0/4177472059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e3239e6b0 0x559e32542300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb50c015000 tx=0x7fb50c020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ff04dea3357f4e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.375+0000 7fb501ffb700 1 -- 172.21.15.40:0/4177472059 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c02b070 con 0x559e3239e6b0 2023-07-18T03:38:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.375+0000 7fb501ffb700 1 -- 172.21.15.40:0/4177472059 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb50c01e040 con 0x559e3239e6b0 2023-07-18T03:38:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.375+0000 7fb501ffb700 1 -- 172.21.15.40:0/4177472059 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c004af0 con 0x559e3239e6b0 2023-07-18T03:38:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.376+0000 7fb52db50b80 1 -- 172.21.15.40:0/4177472059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e3239e6b0 msgr2=0x559e32542300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.376+0000 7fb52db50b80 1 --2- 172.21.15.40:0/4177472059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e3239e6b0 0x559e32542300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb50c015000 tx=0x7fb50c020000 comp rx=0 tx=0).stop 2023-07-18T03:38:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.376+0000 7fb52db50b80 1 -- 172.21.15.40:0/4177472059 shutdown_connections 2023-07-18T03:38:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.376+0000 7fb52db50b80 1 --2- 172.21.15.40:0/4177472059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e325436e0 0x559e32543b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.376+0000 7fb52db50b80 1 --2- 172.21.15.40:0/4177472059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e3239e6b0 0x559e32542300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.382 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.376+0000 7fb52db50b80 1 --2- 172.21.15.40:0/4177472059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 0x559e325431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.376+0000 7fb52db50b80 1 -- 172.21.15.40:0/4177472059 >> 172.21.15.40:0/4177472059 conn(0x559e32537be0 msgr2=0x559e3253ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.376+0000 7fb52db50b80 1 -- 172.21.15.40:0/4177472059 shutdown_connections 2023-07-18T03:38:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.376+0000 7fb52db50b80 1 -- 172.21.15.40:0/4177472059 wait complete. 2023-07-18T03:38:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.376+0000 7fb52db50b80 1 -- start start 2023-07-18T03:38:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb52db50b80 1 Processor -- start 2023-07-18T03:38:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb52db50b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e3239e6b0 0x559e3254b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb52db50b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 0x559e3254bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb52db50b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e325436e0 0x559e32552df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.384 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb52db50b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e32433a60 con 0x559e3239e6b0 2023-07-18T03:38:28.384 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb52db50b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e32433760 con 0x559e325436e0 2023-07-18T03:38:28.384 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb52db50b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e323178c0 con 0x559e32542840 2023-07-18T03:38:28.384 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb52db50b80 10 client.0 _ll_register_callbacks cb 0x559e32431700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:28.384 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb51a7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 0x559e3254bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.384 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb51affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e3239e6b0 0x559e3254b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.384 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb51a7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 0x559e3254bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49822/0 (socket says 172.21.15.40:49822) 2023-07-18T03:38:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb51b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e325436e0 0x559e32552df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb51a7fc700 1 -- 172.21.15.40:0/3414999821 learned_addr learned my addr 172.21.15.40:0/3414999821 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb51a7fc700 1 -- 172.21.15.40:0/3414999821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e325436e0 msgr2=0x559e32552df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb51a7fc700 1 --2- 172.21.15.40:0/3414999821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e325436e0 0x559e32552df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb51a7fc700 1 -- 172.21.15.40:0/3414999821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e3239e6b0 msgr2=0x559e3254b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb51a7fc700 1 --2- 172.21.15.40:0/3414999821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e3239e6b0 0x559e3254b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb51a7fc700 1 -- 172.21.15.40:0/3414999821 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb514011230 con 0x559e32542840 2023-07-18T03:38:28.386 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb51affd700 1 --2- 172.21.15.40:0/3414999821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e3239e6b0 0x559e3254b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:28.386 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.377+0000 7fb51b7fe700 1 --2- 172.21.15.40:0/3414999821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e325436e0 0x559e32552df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:28.386 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.378+0000 7fb51a7fc700 1 --2- 172.21.15.40:0/3414999821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 0x559e3254bdb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb510095bf0 tx=0x7fb51009a3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38ad524a9336a678 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:28.386 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.378+0000 7fb4faffd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:38:28.386 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.378+0000 7fb4faffd700 1 -- 172.21.15.40:0/3414999821 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51009dea0 con 0x559e32542840 2023-07-18T03:38:28.386 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.378+0000 7fb4faffd700 1 -- 172.21.15.40:0/3414999821 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb5100a22c0 con 0x559e32542840 2023-07-18T03:38:28.386 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.378+0000 7fb52db50b80 1 -- 172.21.15.40:0/3414999821 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e323ed490 con 0x559e32542840 2023-07-18T03:38:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.378+0000 7fb4faffd700 1 -- 172.21.15.40:0/3414999821 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5100b8200 con 0x559e32542840 2023-07-18T03:38:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.378+0000 7fb52db50b80 10 client.26236 fetch_fsmap learned FSMap version 51 2023-07-18T03:38:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.378+0000 7fb52db50b80 1 -- 172.21.15.40:0/3414999821 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7fb51c108900 con 0x559e32542840 2023-07-18T03:38:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.378+0000 7fb4faffd700 1 -- 172.21.15.40:0/3414999821 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5100c8080 con 0x559e32542840 2023-07-18T03:38:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.378+0000 7fb4faffd700 1 -- 172.21.15.40:0/3414999821 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 51) v1 ==== 7526+0+0 (secure 0 0 0) 0x7fb5100c9fc0 con 0x559e32542840 2023-07-18T03:38:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.378+0000 7fb52db50b80 10 client.26236 fetch_fsmap finished waiting for FSMap version 51 2023-07-18T03:38:28.387 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.378+0000 7fb52db50b80 10 client.26236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:38:28.388 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.378+0000 7fb52db50b80 20 client.26236 populate_metadata read hostname 'smithi040' 2023-07-18T03:38:28.388 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.378+0000 7fb52db50b80 1 --2- 172.21.15.40:0/3414999821 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559e325577d0 0x559e32559be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.388 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.378+0000 7fb52db50b80 4 client.26236 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:38:28.388 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.378+0000 7fb52db50b80 1 -- 172.21.15.40:0/3414999821 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e3248ce50 con 0x559e325577d0 2023-07-18T03:38:28.388 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.379+0000 7fb51affd700 1 --2- 172.21.15.40:0/3414999821 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559e325577d0 0x559e32559be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.379+0000 7fb4faffd700 10 client.26236 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.379+0000 7fb51affd700 1 --2- 172.21.15.40:0/3414999821 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559e325577d0 0x559e32559be0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9a454e2c0f37e3b server_cookie=495e4cc9768ea09a in_seq=0 out_seq=0 2023-07-18T03:38:28.390 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.380+0000 7fb4faffd700 1 -- 172.21.15.40:0/3414999821 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559e3248ce50 con 0x559e325577d0 2023-07-18T03:38:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.380+0000 7fb4faffd700 10 client.26236 handle_command_reply: tid=0 2023-07-18T03:38:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.380+0000 7fb52db50b80 1 client.26236 shutdown 2023-07-18T03:38:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.380+0000 7fb52db50b80 1 -- 172.21.15.40:0/3414999821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 msgr2=0x559e3254bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.380+0000 7fb52db50b80 1 --2- 172.21.15.40:0/3414999821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 0x559e3254bdb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb510095bf0 tx=0x7fb51009a3b0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.380+0000 7fb52db50b80 1 -- 172.21.15.40:0/3414999821 shutdown_connections 2023-07-18T03:38:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.380+0000 7fb52db50b80 1 --2- 172.21.15.40:0/3414999821 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559e325577d0 0x559e32559be0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.380+0000 7fb52db50b80 1 --2- 172.21.15.40:0/3414999821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e325436e0 0x559e32552df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.380+0000 7fb4faffd700 0 client.26236 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.380+0000 7fb52db50b80 1 --2- 172.21.15.40:0/3414999821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e3239e6b0 0x559e3254b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.380+0000 7fb52db50b80 1 --2- 172.21.15.40:0/3414999821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 0x559e3254bdb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.380+0000 7fb52db50b80 1 -- 172.21.15.40:0/3414999821 >> 172.21.15.40:0/3414999821 conn(0x559e32537be0 msgr2=0x559e3253c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.380+0000 7fb52db50b80 1 -- 172.21.15.40:0/3414999821 shutdown_connections 2023-07-18T03:38:28.392 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.380+0000 7fb52db50b80 1 -- 172.21.15.40:0/3414999821 wait complete. 2023-07-18T03:38:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.381+0000 7fb52db50b80 20 client.26236 trim_cache size 0 max 16384 2023-07-18T03:38:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb52db50b80 1 Processor -- start 2023-07-18T03:38:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb52db50b80 1 -- start start 2023-07-18T03:38:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb52db50b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e3239e6b0 0x559e325465d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb52db50b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 0x559e32546b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb52db50b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e325436e0 0x559e3254b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb52db50b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3254ad20 con 0x559e325436e0 2023-07-18T03:38:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb52db50b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3254b3e0 con 0x559e3239e6b0 2023-07-18T03:38:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb52db50b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3253d900 con 0x559e32542840 2023-07-18T03:38:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb51b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e325436e0 0x559e3254b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb51affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e3239e6b0 0x559e325465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb51a7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 0x559e32546b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb51affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e3239e6b0 0x559e325465d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:44126/0 (socket says 172.21.15.40:44126) 2023-07-18T03:38:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb51affd700 1 -- 172.21.15.40:0/2578309604 learned_addr learned my addr 172.21.15.40:0/2578309604 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb51affd700 1 -- 172.21.15.40:0/2578309604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 msgr2=0x559e32546b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb51affd700 1 --2- 172.21.15.40:0/2578309604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 0x559e32546b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb51affd700 1 -- 172.21.15.40:0/2578309604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e325436e0 msgr2=0x559e3254b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb51affd700 1 --2- 172.21.15.40:0/2578309604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e325436e0 0x559e3254b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb51affd700 1 -- 172.21.15.40:0/2578309604 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e322aac00 con 0x559e3239e6b0 2023-07-18T03:38:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb51b7fe700 1 --2- 172.21.15.40:0/2578309604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e325436e0 0x559e3254b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.396+0000 7fb51affd700 1 --2- 172.21.15.40:0/2578309604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e3239e6b0 0x559e325465d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009990 tx=0x7fb50c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82b1f3b7d729ab25 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.397+0000 7fb5027fc700 1 -- 172.21.15.40:0/2578309604 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c035040 con 0x559e3239e6b0 2023-07-18T03:38:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.397+0000 7fb5027fc700 1 -- 172.21.15.40:0/2578309604 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb50c02c040 con 0x559e3239e6b0 2023-07-18T03:38:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.397+0000 7fb5027fc700 1 -- 172.21.15.40:0/2578309604 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c02b040 con 0x559e3239e6b0 2023-07-18T03:38:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.397+0000 7fb52db50b80 1 -- 172.21.15.40:0/2578309604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e3239e6b0 msgr2=0x559e325465d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.397+0000 7fb52db50b80 1 --2- 172.21.15.40:0/2578309604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e3239e6b0 0x559e325465d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009990 tx=0x7fb50c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.397+0000 7fb52db50b80 1 -- 172.21.15.40:0/2578309604 shutdown_connections 2023-07-18T03:38:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.397+0000 7fb52db50b80 1 --2- 172.21.15.40:0/2578309604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e325436e0 0x559e3254b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.397+0000 7fb52db50b80 1 --2- 172.21.15.40:0/2578309604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e3239e6b0 0x559e325465d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.397+0000 7fb52db50b80 1 --2- 172.21.15.40:0/2578309604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e32542840 0x559e32546b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.397+0000 7fb52db50b80 1 -- 172.21.15.40:0/2578309604 >> 172.21.15.40:0/2578309604 conn(0x559e32537be0 msgr2=0x559e3255e500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.397+0000 7fb52db50b80 1 -- 172.21.15.40:0/2578309604 shutdown_connections 2023-07-18T03:38:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.397+0000 7fb52db50b80 1 -- 172.21.15.40:0/2578309604 wait complete. 2023-07-18T03:38:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.398+0000 7fb52db50b80 1 -- start start 2023-07-18T03:38:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.398+0000 7fb52db50b80 1 Processor -- start 2023-07-18T03:38:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.398+0000 7fb52db50b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e3239e6b0 0x559e325a9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.398+0000 7fb52db50b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e32542840 0x559e325abba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.398+0000 7fb52db50b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e325436e0 0x559e325ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.398+0000 7fb52db50b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e325ae990 con 0x559e3239e6b0 2023-07-18T03:38:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.398+0000 7fb52db50b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e325aeb00 con 0x559e32542840 2023-07-18T03:38:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.398+0000 7fb52db50b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e325aecc0 con 0x559e325436e0 2023-07-18T03:38:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.398+0000 7fb52db50b80 10 client.0 _ll_register_callbacks cb 0x559e32431700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.398+0000 7fb51affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e3239e6b0 0x559e325a9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.398+0000 7fb51a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e32542840 0x559e325abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.398+0000 7fb51affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e3239e6b0 0x559e325a9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60806/0 (socket says 172.21.15.40:60806) 2023-07-18T03:38:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.398+0000 7fb51affd700 1 -- 172.21.15.40:0/3254160569 learned_addr learned my addr 172.21.15.40:0/3254160569 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.398+0000 7fb51b7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e325436e0 0x559e325ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.399+0000 7fb51affd700 1 -- 172.21.15.40:0/3254160569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e325436e0 msgr2=0x559e325ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.399+0000 7fb51affd700 1 --2- 172.21.15.40:0/3254160569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e325436e0 0x559e325ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.399+0000 7fb51affd700 1 -- 172.21.15.40:0/3254160569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e32542840 msgr2=0x559e325abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.399+0000 7fb51affd700 1 --2- 172.21.15.40:0/3254160569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e32542840 0x559e325abba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.399+0000 7fb51affd700 1 -- 172.21.15.40:0/3254160569 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb514007960 con 0x559e3239e6b0 2023-07-18T03:38:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.399+0000 7fb51affd700 1 --2- 172.21.15.40:0/3254160569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e3239e6b0 0x559e325a9630 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb50c03c040 tx=0x7fb50c004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb93951c9dfa7384 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.399+0000 7fb4faffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:38:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.399+0000 7fb4faffd700 1 -- 172.21.15.40:0/3254160569 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c035070 con 0x559e3239e6b0 2023-07-18T03:38:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.399+0000 7fb4faffd700 1 -- 172.21.15.40:0/3254160569 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb50c039ae0 con 0x559e3239e6b0 2023-07-18T03:38:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.399+0000 7fb52db50b80 1 -- 172.21.15.40:0/3254160569 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb50c01e040 con 0x559e3239e6b0 2023-07-18T03:38:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.399+0000 7fb4faffd700 1 -- 172.21.15.40:0/3254160569 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb50c039780 con 0x559e3239e6b0 2023-07-18T03:38:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.399+0000 7fb52db50b80 10 client.16572 fetch_fsmap learned FSMap version 51 2023-07-18T03:38:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.399+0000 7fb52db50b80 1 -- 172.21.15.40:0/3254160569 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x559e322c1d80 con 0x559e3239e6b0 2023-07-18T03:38:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.399+0000 7fb4faffd700 1 -- 172.21.15.40:0/3254160569 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb50c01e040 con 0x559e3239e6b0 2023-07-18T03:38:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.400+0000 7fb4faffd700 1 -- 172.21.15.40:0/3254160569 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 51) v1 ==== 7526+0+0 (secure 0 0 0) 0x7fb50c040070 con 0x559e3239e6b0 2023-07-18T03:38:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.400+0000 7fb52db50b80 10 client.16572 fetch_fsmap finished waiting for FSMap version 51 2023-07-18T03:38:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.400+0000 7fb52db50b80 10 client.16572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:38:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.400+0000 7fb52db50b80 20 client.16572 populate_metadata read hostname 'smithi040' 2023-07-18T03:38:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.400+0000 7fb52db50b80 1 --2- 172.21.15.40:0/3254160569 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559e325b0340 0x559e325b2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.400+0000 7fb52db50b80 4 client.16572 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:38:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.400+0000 7fb52db50b80 1 -- 172.21.15.40:0/3254160569 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559e3244eff0 con 0x559e325b0340 2023-07-18T03:38:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.400+0000 7fb51a7fc700 1 --2- 172.21.15.40:0/3254160569 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559e325b0340 0x559e325b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.400+0000 7fb51a7fc700 1 --2- 172.21.15.40:0/3254160569 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559e325b0340 0x559e325b2750 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b7f0baf15b49d80 server_cookie=b97053f81572f917 in_seq=0 out_seq=0 2023-07-18T03:38:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.400+0000 7fb4faffd700 10 client.16572 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.833+0000 7fb4faffd700 1 -- 172.21.15.40:0/3254160569 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559e3244eff0 con 0x559e325b0340 2023-07-18T03:38:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.833+0000 7fb4faffd700 10 client.16572 handle_command_reply: tid=0 2023-07-18T03:38:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.833+0000 7fb52db50b80 1 client.16572 shutdown 2023-07-18T03:38:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.833+0000 7fb52db50b80 1 -- 172.21.15.40:0/3254160569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e3239e6b0 msgr2=0x559e325a9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.833+0000 7fb52db50b80 1 --2- 172.21.15.40:0/3254160569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e3239e6b0 0x559e325a9630 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb50c03c040 tx=0x7fb50c004500 comp rx=0 tx=0).stop 2023-07-18T03:38:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.833+0000 7fb52db50b80 1 -- 172.21.15.40:0/3254160569 shutdown_connections 2023-07-18T03:38:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.833+0000 7fb52db50b80 1 --2- 172.21.15.40:0/3254160569 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x559e325b0340 0x559e325b2750 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.833+0000 7fb52db50b80 1 --2- 172.21.15.40:0/3254160569 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e325436e0 0x559e325ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.833+0000 7fb52db50b80 1 --2- 172.21.15.40:0/3254160569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x559e3239e6b0 0x559e325a9630 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.833+0000 7fb52db50b80 1 --2- 172.21.15.40:0/3254160569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559e32542840 0x559e325abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.833+0000 7fb52db50b80 1 -- 172.21.15.40:0/3254160569 >> 172.21.15.40:0/3254160569 conn(0x559e32537be0 msgr2=0x559e325465d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.833+0000 7fb4faffd700 0 client.16572 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.833+0000 7fb52db50b80 1 -- 172.21.15.40:0/3254160569 shutdown_connections 2023-07-18T03:38:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.834+0000 7fb52db50b80 1 -- 172.21.15.40:0/3254160569 wait complete. 2023-07-18T03:38:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.834+0000 7fb52db50b80 20 client.16572 trim_cache size 0 max 16384 2023-07-18T03:38:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:38:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:38:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:38:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:38:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.834+0000 7fb5219ca700 1 -- 172.21.15.40:0/3926969307 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb5040a11d0 msgr2=0x7fb5040a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.834+0000 7fb5219ca700 1 --2- 172.21.15.40:0/3926969307 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb5040a11d0 0x7fb5040a3680 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb50c006010 tx=0x7fb50c005960 comp rx=0 tx=0).stop 2023-07-18T03:38:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.834+0000 7fb5219ca700 1 -- 172.21.15.40:0/3926969307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51c100d80 msgr2=0x7fb51c195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.834+0000 7fb5219ca700 1 --2- 172.21.15.40:0/3926969307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51c100d80 0x7fb51c195e40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb51000e9a0 tx=0x7fb51000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.834+0000 7fb5219ca700 1 -- 172.21.15.40:0/3926969307 shutdown_connections 2023-07-18T03:38:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.834+0000 7fb5219ca700 1 --2- 172.21.15.40:0/3926969307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb51c101710 0x7fb51c196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.835+0000 7fb5219ca700 1 --2- 172.21.15.40:0/3926969307 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb5040a11d0 0x7fb5040a3680 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.835+0000 7fb5219ca700 1 --2- 172.21.15.40:0/3926969307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb51c0ffb90 0x7fb51c195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.835+0000 7fb5219ca700 1 --2- 172.21.15.40:0/3926969307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb51c100d80 0x7fb51c195e40 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.835+0000 7fb5219ca700 1 -- 172.21.15.40:0/3926969307 >> 172.21.15.40:0/3926969307 conn(0x7fb51c0fb190 msgr2=0x7fb51c104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.835+0000 7fb5219ca700 1 -- 172.21.15.40:0/3926969307 shutdown_connections 2023-07-18T03:38:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:28.835+0000 7fb5219ca700 1 -- 172.21.15.40:0/3926969307 wait complete. 2023-07-18T03:38:29.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:29 smithi134 ceph-mon[98216]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 71 op/s 2023-07-18T03:38:29.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:29 smithi040 ceph-mon[96439]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 71 op/s 2023-07-18T03:38:29.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:29 smithi064 ceph-mon[98059]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 71 op/s 2023-07-18T03:38:31.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:31 smithi134 ceph-mon[98216]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 32 op/s 2023-07-18T03:38:31.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:31 smithi040 ceph-mon[96439]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 32 op/s 2023-07-18T03:38:31.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:31 smithi064 ceph-mon[98059]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 32 op/s 2023-07-18T03:38:33.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:33 smithi134 ceph-mon[98216]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 100 op/s 2023-07-18T03:38:33.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:33 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:33.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:33 smithi040 ceph-mon[96439]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 100 op/s 2023-07-18T03:38:33.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:33 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:33.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:33 smithi064 ceph-mon[98059]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 100 op/s 2023-07-18T03:38:33.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:33 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:35.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:35 smithi134 ceph-mon[98216]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2023-07-18T03:38:35.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:35 smithi040 ceph-mon[96439]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2023-07-18T03:38:35.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:35 smithi064 ceph-mon[98059]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2023-07-18T03:38:37.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:37 smithi134 ceph-mon[98216]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 379 KiB/s wr, 113 op/s 2023-07-18T03:38:37.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:37 smithi040 ceph-mon[96439]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 379 KiB/s wr, 113 op/s 2023-07-18T03:38:37.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:37 smithi064 ceph-mon[98059]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 379 KiB/s wr, 113 op/s 2023-07-18T03:38:38.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:38 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:38.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:38 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:38.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:38 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:39.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:39 smithi134 ceph-mon[98216]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 378 KiB/s wr, 114 op/s 2023-07-18T03:38:39.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:39 smithi040 ceph-mon[96439]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 378 KiB/s wr, 114 op/s 2023-07-18T03:38:39.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:39 smithi064 ceph-mon[98059]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 378 KiB/s wr, 114 op/s 2023-07-18T03:38:41.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:41 smithi134 ceph-mon[98216]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 294 KiB/s wr, 106 op/s 2023-07-18T03:38:41.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:41 smithi040 ceph-mon[96439]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 294 KiB/s wr, 106 op/s 2023-07-18T03:38:41.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:41 smithi064 ceph-mon[98059]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 294 KiB/s wr, 106 op/s 2023-07-18T03:38:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.960+0000 7f8864afe700 1 -- 172.21.15.40:0/1028484886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88601056e0 msgr2=0x7f8860107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.960+0000 7f8864afe700 1 --2- 172.21.15.40:0/1028484886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88601056e0 0x7f8860107b90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8858009ec0 tx=0x7f885800b330 comp rx=0 tx=0).stop 2023-07-18T03:38:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.960+0000 7f8864afe700 1 -- 172.21.15.40:0/1028484886 shutdown_connections 2023-07-18T03:38:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.960+0000 7f8864afe700 1 --2- 172.21.15.40:0/1028484886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88601056e0 0x7f8860107b90 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.960+0000 7f8864afe700 1 --2- 172.21.15.40:0/1028484886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8860068600 0x7f88601026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.960+0000 7f8864afe700 1 --2- 172.21.15.40:0/1028484886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8860102c10 0x7f8860105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.960+0000 7f8864afe700 1 -- 172.21.15.40:0/1028484886 >> 172.21.15.40:0/1028484886 conn(0x7f88600fa000 msgr2=0x7f88600fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.960+0000 7f8864afe700 1 -- 172.21.15.40:0/1028484886 shutdown_connections 2023-07-18T03:38:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.960+0000 7f8864afe700 1 -- 172.21.15.40:0/1028484886 wait complete. 2023-07-18T03:38:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f8864afe700 1 Processor -- start 2023-07-18T03:38:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f8864afe700 1 -- start start 2023-07-18T03:38:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f8864afe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8860068600 0x7f88601958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f8864afe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8860102c10 0x7f8860195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f8864afe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88601056e0 0x7f8860196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f8864afe700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88601972f0 con 0x7f8860068600 2023-07-18T03:38:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f8864afe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8860197460 con 0x7f8860102c10 2023-07-18T03:38:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f8864afe700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8860197620 con 0x7f88601056e0 2023-07-18T03:38:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f885e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8860068600 0x7f88601958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f885e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8860068600 0x7f88601958d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60828/0 (socket says 172.21.15.40:60828) 2023-07-18T03:38:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f885e59c700 1 -- 172.21.15.40:0/2368027639 learned_addr learned my addr 172.21.15.40:0/2368027639 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f885ed9d700 1 --2- 172.21.15.40:0/2368027639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88601056e0 0x7f8860196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f885dd9b700 1 --2- 172.21.15.40:0/2368027639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8860102c10 0x7f8860195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f885e59c700 1 -- 172.21.15.40:0/2368027639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88601056e0 msgr2=0x7f8860196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f885e59c700 1 --2- 172.21.15.40:0/2368027639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88601056e0 0x7f8860196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f885e59c700 1 -- 172.21.15.40:0/2368027639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8860102c10 msgr2=0x7f8860195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f885e59c700 1 --2- 172.21.15.40:0/2368027639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8860102c10 0x7f8860195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.961+0000 7f885e59c700 1 -- 172.21.15.40:0/2368027639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8858005e70 con 0x7f8860068600 2023-07-18T03:38:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.962+0000 7f885ed9d700 1 --2- 172.21.15.40:0/2368027639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88601056e0 0x7f8860196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.962+0000 7f885e59c700 1 --2- 172.21.15.40:0/2368027639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8860068600 0x7f88601958d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f885000e970 tx=0x7f885000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.962+0000 7f884f7fe700 1 -- 172.21.15.40:0/2368027639 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850009790 con 0x7f8860068600 2023-07-18T03:38:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.962+0000 7f884f7fe700 1 -- 172.21.15.40:0/2368027639 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f885000c7e0 con 0x7f8860068600 2023-07-18T03:38:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.962+0000 7f8864afe700 1 -- 172.21.15.40:0/2368027639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8860197900 con 0x7f8860068600 2023-07-18T03:38:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.962+0000 7f884f7fe700 1 -- 172.21.15.40:0/2368027639 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88500054e0 con 0x7f8860068600 2023-07-18T03:38:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.962+0000 7f8864afe700 1 -- 172.21.15.40:0/2368027639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f886019e0c0 con 0x7f8860068600 2023-07-18T03:38:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.964+0000 7f884f7fe700 1 -- 172.21.15.40:0/2368027639 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f885000c950 con 0x7f8860068600 2023-07-18T03:38:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.965+0000 7f884f7fe700 1 --2- 172.21.15.40:0/2368027639 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f88480a1150 0x7f88480a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.965+0000 7f885dd9b700 1 --2- 172.21.15.40:0/2368027639 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f88480a1150 0x7f88480a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.965+0000 7f884f7fe700 1 -- 172.21.15.40:0/2368027639 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8850014070 con 0x7f8860068600 2023-07-18T03:38:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.965+0000 7f885dd9b700 1 --2- 172.21.15.40:0/2368027639 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f88480a1150 0x7f88480a3600 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8854009c60 tx=0x7f8854009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.966+0000 7f8870c84b80 1 Processor -- start 2023-07-18T03:38:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.966+0000 7f8870c84b80 1 -- start start 2023-07-18T03:38:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.966+0000 7f8870c84b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569897086b0 0x5569898ac300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.966+0000 7f8870c84b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 0x5569898ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.966+0000 7f8870c84b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569898ad6e0 0x5569898adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.966+0000 7f8870c84b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569897b1c50 con 0x5569898ac840 2023-07-18T03:38:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.966+0000 7f8870c84b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569898049e0 con 0x5569898ad6e0 2023-07-18T03:38:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.966+0000 7f8870c84b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569897fde70 con 0x5569897086b0 2023-07-18T03:38:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.966+0000 7f885e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569897086b0 0x5569898ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.966+0000 7f885dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 0x5569898ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.966+0000 7f885e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569897086b0 0x5569898ac300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49864/0 (socket says 172.21.15.40:49864) 2023-07-18T03:38:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.966+0000 7f885e59c700 1 -- 172.21.15.40:0/3682028668 learned_addr learned my addr 172.21.15.40:0/3682028668 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.966+0000 7f885ed9d700 1 --2- 172.21.15.40:0/3682028668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569898ad6e0 0x5569898adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.967+0000 7f885dd9b700 1 -- 172.21.15.40:0/3682028668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569897086b0 msgr2=0x5569898ac300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.967+0000 7f885dd9b700 1 --2- 172.21.15.40:0/3682028668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569897086b0 0x5569898ac300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.967+0000 7f885dd9b700 1 -- 172.21.15.40:0/3682028668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569898ad6e0 msgr2=0x5569898adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.967+0000 7f885dd9b700 1 --2- 172.21.15.40:0/3682028668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569898ad6e0 0x5569898adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.967+0000 7f885dd9b700 1 -- 172.21.15.40:0/3682028668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55698962bd80 con 0x5569898ac840 2023-07-18T03:38:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.967+0000 7f885ed9d700 1 --2- 172.21.15.40:0/3682028668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569898ad6e0 0x5569898adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.967+0000 7f885dd9b700 1 --2- 172.21.15.40:0/3682028668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 0x5569898ad1a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f885400e3b0 tx=0x7f8854014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a266b60180d614b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.967+0000 7f884cff9700 1 -- 172.21.15.40:0/3682028668 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885400ed50 con 0x5569898ac840 2023-07-18T03:38:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.967+0000 7f884cff9700 1 -- 172.21.15.40:0/3682028668 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8854021070 con 0x5569898ac840 2023-07-18T03:38:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.967+0000 7f884cff9700 1 -- 172.21.15.40:0/3682028668 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8854003ea0 con 0x5569898ac840 2023-07-18T03:38:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.968+0000 7f8870c84b80 1 -- 172.21.15.40:0/3682028668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 msgr2=0x5569898ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.968+0000 7f8870c84b80 1 --2- 172.21.15.40:0/3682028668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 0x5569898ad1a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f885400e3b0 tx=0x7f8854014000 comp rx=0 tx=0).stop 2023-07-18T03:38:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.968+0000 7f8870c84b80 1 -- 172.21.15.40:0/3682028668 shutdown_connections 2023-07-18T03:38:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.968+0000 7f8870c84b80 1 --2- 172.21.15.40:0/3682028668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569898ad6e0 0x5569898adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.968+0000 7f8870c84b80 1 --2- 172.21.15.40:0/3682028668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569897086b0 0x5569898ac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.968+0000 7f8870c84b80 1 --2- 172.21.15.40:0/3682028668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 0x5569898ad1a0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.968+0000 7f8870c84b80 1 -- 172.21.15.40:0/3682028668 >> 172.21.15.40:0/3682028668 conn(0x5569898a1be0 msgr2=0x5569898a6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.968+0000 7f8870c84b80 1 -- 172.21.15.40:0/3682028668 shutdown_connections 2023-07-18T03:38:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.968+0000 7f8870c84b80 1 -- 172.21.15.40:0/3682028668 wait complete. 2023-07-18T03:38:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.968+0000 7f8870c84b80 1 -- start start 2023-07-18T03:38:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.968+0000 7f8870c84b80 1 Processor -- start 2023-07-18T03:38:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f8870c84b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569897086b0 0x5569898b5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f8870c84b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 0x5569898b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f8870c84b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569898ad6e0 0x5569898bcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f8870c84b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698979da60 con 0x5569898ac840 2023-07-18T03:38:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f8870c84b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698979d760 con 0x5569897086b0 2023-07-18T03:38:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f8870c84b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569896818c0 con 0x5569898ad6e0 2023-07-18T03:38:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f8870c84b80 10 client.0 _ll_register_callbacks cb 0x55698979b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f885dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 0x5569898b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f885e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569897086b0 0x5569898b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f885ed9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569898ad6e0 0x5569898bcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f885dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 0x5569898b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60844/0 (socket says 172.21.15.40:60844) 2023-07-18T03:38:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f885e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569897086b0 0x5569898b5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:44168/0 (socket says 172.21.15.40:44168) 2023-07-18T03:38:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f885dd9b700 1 -- 172.21.15.40:0/1422341975 learned_addr learned my addr 172.21.15.40:0/1422341975 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f885dd9b700 1 -- 172.21.15.40:0/1422341975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569898ad6e0 msgr2=0x5569898bcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f885dd9b700 1 --2- 172.21.15.40:0/1422341975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569898ad6e0 0x5569898bcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f885dd9b700 1 -- 172.21.15.40:0/1422341975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569897086b0 msgr2=0x5569898b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f885dd9b700 1 --2- 172.21.15.40:0/1422341975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569897086b0 0x5569898b5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f885dd9b700 1 -- 172.21.15.40:0/1422341975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8860108850 con 0x5569898ac840 2023-07-18T03:38:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f885e59c700 1 --2- 172.21.15.40:0/1422341975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569897086b0 0x5569898b5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f885dd9b700 1 --2- 172.21.15.40:0/1422341975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 0x5569898b5d80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8854009b20 tx=0x7f8854003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58e6be85c66f8b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f8845ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:38:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.969+0000 7f8845ffb700 1 -- 172.21.15.40:0/1422341975 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885402a040 con 0x5569898ac840 2023-07-18T03:38:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.970+0000 7f8845ffb700 1 -- 172.21.15.40:0/1422341975 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f885400ee30 con 0x5569898ac840 2023-07-18T03:38:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.970+0000 7f8870c84b80 1 -- 172.21.15.40:0/1422341975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556989757490 con 0x5569898ac840 2023-07-18T03:38:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.970+0000 7f8845ffb700 1 -- 172.21.15.40:0/1422341975 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88540054d0 con 0x5569898ac840 2023-07-18T03:38:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.970+0000 7f8870c84b80 10 client.16584 fetch_fsmap learned FSMap version 53 2023-07-18T03:38:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.970+0000 7f8870c84b80 1 -- 172.21.15.40:0/1422341975 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7f8854017070 con 0x5569898ac840 2023-07-18T03:38:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.970+0000 7f8845ffb700 1 -- 172.21.15.40:0/1422341975 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556989757490 con 0x5569898ac840 2023-07-18T03:38:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.970+0000 7f8845ffb700 1 -- 172.21.15.40:0/1422341975 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 53) v1 ==== 7522+0+0 (secure 0 0 0) 0x7f8854037020 con 0x5569898ac840 2023-07-18T03:38:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.970+0000 7f8870c84b80 10 client.16584 fetch_fsmap finished waiting for FSMap version 53 2023-07-18T03:38:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.970+0000 7f8870c84b80 10 client.16584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:38:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.970+0000 7f8870c84b80 20 client.16584 populate_metadata read hostname 'smithi040' 2023-07-18T03:38:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.970+0000 7f8870c84b80 1 --2- 172.21.15.40:0/1422341975 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5569898c1560 0x5569898c3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.970+0000 7f8870c84b80 4 client.16584 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:38:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.970+0000 7f8870c84b80 1 -- 172.21.15.40:0/1422341975 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569897f6e50 con 0x5569898c1560 2023-07-18T03:38:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.970+0000 7f885e59c700 1 --2- 172.21.15.40:0/1422341975 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5569898c1560 0x5569898c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.971+0000 7f8845ffb700 10 client.16584 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.971+0000 7f885e59c700 1 --2- 172.21.15.40:0/1422341975 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5569898c1560 0x5569898c3970 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d70202a7adf42999 server_cookie=7de6d89805922d53 in_seq=0 out_seq=0 2023-07-18T03:38:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.972+0000 7f8845ffb700 1 -- 172.21.15.40:0/1422341975 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5569897f6e50 con 0x5569898c1560 2023-07-18T03:38:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.972+0000 7f8845ffb700 10 client.16584 handle_command_reply: tid=0 2023-07-18T03:38:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.972+0000 7f8870c84b80 1 client.16584 shutdown 2023-07-18T03:38:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.972+0000 7f8870c84b80 1 -- 172.21.15.40:0/1422341975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 msgr2=0x5569898b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.972+0000 7f8870c84b80 1 --2- 172.21.15.40:0/1422341975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 0x5569898b5d80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8854009b20 tx=0x7f8854003920 comp rx=0 tx=0).stop 2023-07-18T03:38:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.972+0000 7f8870c84b80 1 -- 172.21.15.40:0/1422341975 shutdown_connections 2023-07-18T03:38:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.972+0000 7f8870c84b80 1 --2- 172.21.15.40:0/1422341975 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5569898c1560 0x5569898c3970 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.972+0000 7f8870c84b80 1 --2- 172.21.15.40:0/1422341975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569898ad6e0 0x5569898bcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.972+0000 7f8845ffb700 0 client.16584 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.972+0000 7f8870c84b80 1 --2- 172.21.15.40:0/1422341975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569897086b0 0x5569898b5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.972+0000 7f8870c84b80 1 --2- 172.21.15.40:0/1422341975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 0x5569898b5d80 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.972+0000 7f8870c84b80 1 -- 172.21.15.40:0/1422341975 >> 172.21.15.40:0/1422341975 conn(0x5569898a1be0 msgr2=0x5569898a64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.972+0000 7f8870c84b80 1 -- 172.21.15.40:0/1422341975 shutdown_connections 2023-07-18T03:38:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.972+0000 7f8870c84b80 1 -- 172.21.15.40:0/1422341975 wait complete. 2023-07-18T03:38:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.972+0000 7f8870c84b80 20 client.16584 trim_cache size 0 max 16384 2023-07-18T03:38:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.987+0000 7f8870c84b80 1 Processor -- start 2023-07-18T03:38:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.987+0000 7f8870c84b80 1 -- start start 2023-07-18T03:38:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.987+0000 7f8870c84b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569897086b0 0x5569898adab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.987+0000 7f8870c84b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 0x5569898bcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.987+0000 7f8870c84b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569898ad6e0 0x5569898b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.987+0000 7f8870c84b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569898b6740 con 0x5569898ac840 2023-07-18T03:38:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.987+0000 7f8870c84b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569898b53b0 con 0x5569898ad6e0 2023-07-18T03:38:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.987+0000 7f8870c84b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569898a3d00 con 0x5569897086b0 2023-07-18T03:38:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.987+0000 7f885dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 0x5569898bcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.987+0000 7f885e59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569897086b0 0x5569898adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.987+0000 7f885ed9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569898ad6e0 0x5569898b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.987+0000 7f885dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 0x5569898bcd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60852/0 (socket says 172.21.15.40:60852) 2023-07-18T03:38:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.987+0000 7f885dd9b700 1 -- 172.21.15.40:0/1579134229 learned_addr learned my addr 172.21.15.40:0/1579134229 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.988+0000 7f885e59c700 1 -- 172.21.15.40:0/1579134229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569898ad6e0 msgr2=0x5569898b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.988+0000 7f885e59c700 1 --2- 172.21.15.40:0/1579134229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569898ad6e0 0x5569898b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.988+0000 7f885e59c700 1 -- 172.21.15.40:0/1579134229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 msgr2=0x5569898bcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.988+0000 7f885e59c700 1 --2- 172.21.15.40:0/1579134229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 0x5569898bcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.988+0000 7f885e59c700 1 -- 172.21.15.40:0/1579134229 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556989614c00 con 0x5569897086b0 2023-07-18T03:38:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.988+0000 7f885dd9b700 1 --2- 172.21.15.40:0/1579134229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 0x5569898bcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.988+0000 7f885e59c700 1 --2- 172.21.15.40:0/1579134229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569897086b0 0x5569898adab0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f88500a1e50 tx=0x7f88500cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1afb5d54ed6bd6e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.988+0000 7f884d7fa700 1 -- 172.21.15.40:0/1579134229 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885009df60 con 0x5569897086b0 2023-07-18T03:38:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.988+0000 7f884d7fa700 1 -- 172.21.15.40:0/1579134229 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88500a2380 con 0x5569897086b0 2023-07-18T03:38:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.988+0000 7f884d7fa700 1 -- 172.21.15.40:0/1579134229 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88500056f0 con 0x5569897086b0 2023-07-18T03:38:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 1 -- 172.21.15.40:0/1579134229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569897086b0 msgr2=0x5569898adab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 1 --2- 172.21.15.40:0/1579134229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569897086b0 0x5569898adab0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f88500a1e50 tx=0x7f88500cc040 comp rx=0 tx=0).stop 2023-07-18T03:38:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 1 -- 172.21.15.40:0/1579134229 shutdown_connections 2023-07-18T03:38:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 1 --2- 172.21.15.40:0/1579134229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569898ad6e0 0x5569898b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 1 --2- 172.21.15.40:0/1579134229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569897086b0 0x5569898adab0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 1 --2- 172.21.15.40:0/1579134229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569898ac840 0x5569898bcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 1 -- 172.21.15.40:0/1579134229 >> 172.21.15.40:0/1579134229 conn(0x5569898a1be0 msgr2=0x5569898b4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 1 -- 172.21.15.40:0/1579134229 shutdown_connections 2023-07-18T03:38:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 1 -- 172.21.15.40:0/1579134229 wait complete. 2023-07-18T03:38:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 1 -- start start 2023-07-18T03:38:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 1 Processor -- start 2023-07-18T03:38:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569897086b0 0x5569898b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569898ac840 0x5569898b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569898ad6e0 0x5569898bbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569898bc1d0 con 0x5569897086b0 2023-07-18T03:38:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569898bc340 con 0x5569898ad6e0 2023-07-18T03:38:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569898bc500 con 0x5569898ac840 2023-07-18T03:38:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f8870c84b80 10 client.0 _ll_register_callbacks cb 0x7f8854005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f885e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569897086b0 0x5569898b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f885e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569897086b0 0x5569898b8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60856/0 (socket says 172.21.15.40:60856) 2023-07-18T03:38:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.989+0000 7f885dd9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569898ac840 0x5569898b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.990+0000 7f885e59c700 1 -- 172.21.15.40:0/609922485 learned_addr learned my addr 172.21.15.40:0/609922485 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.990+0000 7f885ed9d700 1 --2- 172.21.15.40:0/609922485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569898ad6e0 0x5569898bbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.990+0000 7f885e59c700 1 -- 172.21.15.40:0/609922485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569898ac840 msgr2=0x5569898b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.990+0000 7f885e59c700 1 --2- 172.21.15.40:0/609922485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569898ac840 0x5569898b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.990+0000 7f885e59c700 1 -- 172.21.15.40:0/609922485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569898ad6e0 msgr2=0x5569898bbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.990+0000 7f885e59c700 1 --2- 172.21.15.40:0/609922485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569898ad6e0 0x5569898bbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.990+0000 7f885e59c700 1 -- 172.21.15.40:0/609922485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8854017070 con 0x5569897086b0 2023-07-18T03:38:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.990+0000 7f885e59c700 1 --2- 172.21.15.40:0/609922485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569897086b0 0x5569898b8730 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8850096950 tx=0x7f885000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247b46d9a0e47017 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.990+0000 7f8845ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:38:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.990+0000 7f8845ffb700 1 -- 172.21.15.40:0/609922485 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850098070 con 0x5569897086b0 2023-07-18T03:38:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.990+0000 7f8845ffb700 1 -- 172.21.15.40:0/609922485 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f885009df60 con 0x5569897086b0 2023-07-18T03:38:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.990+0000 7f8870c84b80 1 -- 172.21.15.40:0/609922485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8854021070 con 0x5569897086b0 2023-07-18T03:38:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.990+0000 7f8845ffb700 1 -- 172.21.15.40:0/609922485 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8850099420 con 0x5569897086b0 2023-07-18T03:38:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.990+0000 7f8870c84b80 10 client.16590 fetch_fsmap learned FSMap version 53 2023-07-18T03:38:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.990+0000 7f8870c84b80 1 -- 172.21.15.40:0/609922485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7f886019e0c0 con 0x5569897086b0 2023-07-18T03:38:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.990+0000 7f8845ffb700 1 -- 172.21.15.40:0/609922485 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8854021070 con 0x5569897086b0 2023-07-18T03:38:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.991+0000 7f8845ffb700 1 -- 172.21.15.40:0/609922485 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 53) v1 ==== 7522+0+0 (secure 0 0 0) 0x7f88500caae0 con 0x5569897086b0 2023-07-18T03:38:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.991+0000 7f8870c84b80 10 client.16590 fetch_fsmap finished waiting for FSMap version 53 2023-07-18T03:38:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.991+0000 7f8870c84b80 10 client.16590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:38:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.991+0000 7f8870c84b80 20 client.16590 populate_metadata read hostname 'smithi040' 2023-07-18T03:38:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.991+0000 7f8870c84b80 1 --2- 172.21.15.40:0/609922485 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556989915c70 0x556989918080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.991+0000 7f8870c84b80 4 client.16590 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:38:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.991+0000 7f8870c84b80 1 -- 172.21.15.40:0/609922485 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5569897b8ff0 con 0x556989915c70 2023-07-18T03:38:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.991+0000 7f885dd9b700 1 --2- 172.21.15.40:0/609922485 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556989915c70 0x556989918080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.991+0000 7f8845ffb700 10 client.16590 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:41.991+0000 7f885dd9b700 1 --2- 172.21.15.40:0/609922485 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556989915c70 0x556989918080 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37cf28d923c9d7eb server_cookie=b7c8b572d4e115f3 in_seq=0 out_seq=0 2023-07-18T03:38:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.001+0000 7f8845ffb700 1 -- 172.21.15.40:0/609922485 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5569897b8ff0 con 0x556989915c70 2023-07-18T03:38:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.001+0000 7f8845ffb700 10 client.16590 handle_command_reply: tid=0 2023-07-18T03:38:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.001+0000 7f8870c84b80 1 client.16590 shutdown 2023-07-18T03:38:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.001+0000 7f8870c84b80 1 -- 172.21.15.40:0/609922485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569897086b0 msgr2=0x5569898b8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.001+0000 7f8870c84b80 1 --2- 172.21.15.40:0/609922485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569897086b0 0x5569898b8730 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8850096950 tx=0x7f885000cc50 comp rx=0 tx=0).stop 2023-07-18T03:38:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.001+0000 7f8870c84b80 1 -- 172.21.15.40:0/609922485 shutdown_connections 2023-07-18T03:38:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.001+0000 7f8870c84b80 1 --2- 172.21.15.40:0/609922485 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556989915c70 0x556989918080 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.001+0000 7f8870c84b80 1 --2- 172.21.15.40:0/609922485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569898ad6e0 0x5569898bbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.001+0000 7f8870c84b80 1 --2- 172.21.15.40:0/609922485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569897086b0 0x5569898b8730 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.001+0000 7f8870c84b80 1 --2- 172.21.15.40:0/609922485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569898ac840 0x5569898b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.001+0000 7f8845ffb700 0 client.16590 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.001+0000 7f8870c84b80 1 -- 172.21.15.40:0/609922485 >> 172.21.15.40:0/609922485 conn(0x5569898a1be0 msgr2=0x5569898b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.002+0000 7f8870c84b80 1 -- 172.21.15.40:0/609922485 shutdown_connections 2023-07-18T03:38:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.002+0000 7f8870c84b80 1 -- 172.21.15.40:0/609922485 wait complete. 2023-07-18T03:38:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.002+0000 7f8870c84b80 20 client.16590 trim_cache size 0 max 16384 2023-07-18T03:38:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:38:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:38:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:38:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:38:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.002+0000 7f8864afe700 1 -- 172.21.15.40:0/2368027639 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f88480a1150 msgr2=0x7f88480a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.002+0000 7f8864afe700 1 --2- 172.21.15.40:0/2368027639 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f88480a1150 0x7f88480a3600 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8854009c60 tx=0x7f8854009260 comp rx=0 tx=0).stop 2023-07-18T03:38:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.002+0000 7f8864afe700 1 -- 172.21.15.40:0/2368027639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8860068600 msgr2=0x7f88601958d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.002+0000 7f8864afe700 1 --2- 172.21.15.40:0/2368027639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8860068600 0x7f88601958d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f885000e970 tx=0x7f885000ec80 comp rx=0 tx=0).stop 2023-07-18T03:38:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.002+0000 7f8864afe700 1 -- 172.21.15.40:0/2368027639 shutdown_connections 2023-07-18T03:38:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.002+0000 7f8864afe700 1 --2- 172.21.15.40:0/2368027639 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f88480a1150 0x7f88480a3600 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.002+0000 7f8864afe700 1 --2- 172.21.15.40:0/2368027639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88601056e0 0x7f8860196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.002+0000 7f8864afe700 1 --2- 172.21.15.40:0/2368027639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8860068600 0x7f88601958d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.002+0000 7f8864afe700 1 --2- 172.21.15.40:0/2368027639 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8860102c10 0x7f8860195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.002+0000 7f8864afe700 1 -- 172.21.15.40:0/2368027639 >> 172.21.15.40:0/2368027639 conn(0x7f88600fa000 msgr2=0x7f8860100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.002+0000 7f8864afe700 1 -- 172.21.15.40:0/2368027639 shutdown_connections 2023-07-18T03:38:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:42.002+0000 7f8864afe700 1 -- 172.21.15.40:0/2368027639 wait complete. 2023-07-18T03:38:43.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:43 smithi134 ceph-mon[98216]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 346 KiB/s wr, 113 op/s 2023-07-18T03:38:43.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:43 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:43.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:43 smithi040 ceph-mon[96439]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 346 KiB/s wr, 113 op/s 2023-07-18T03:38:43.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:43 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:43.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:43 smithi064 ceph-mon[98059]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 346 KiB/s wr, 113 op/s 2023-07-18T03:38:43.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:43 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:45.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:45 smithi134 ceph-mon[98216]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 198 KiB/s wr, 49 op/s 2023-07-18T03:38:45.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:45 smithi040 ceph-mon[96439]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 198 KiB/s wr, 49 op/s 2023-07-18T03:38:45.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:45 smithi064 ceph-mon[98059]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 198 KiB/s wr, 49 op/s 2023-07-18T03:38:46.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:46 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:46.592 INFO:tasks.workunit.client.0.smithi040.stdout: 2023-07-18T03:38:46.593 INFO:tasks.workunit.client.0.smithi040.stdout:Final score for writes: 369 2023-07-18T03:38:46.593 INFO:tasks.workunit.client.0.smithi040.stdout:Final score for reads : 933836 2023-07-18T03:38:46.593 INFO:tasks.workunit.client.0.smithi040.stdout: 2023-07-18T03:38:46.594 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-07-18T03:38:46.594 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-07-18T03:38:46.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:46 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:46.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:46 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:47.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:47 smithi040 ceph-mon[96439]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 237 KiB/s wr, 28 op/s 2023-07-18T03:38:47.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:47 smithi064 ceph-mon[98059]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 237 KiB/s wr, 28 op/s 2023-07-18T03:38:47.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:47 smithi134 ceph-mon[98216]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 237 KiB/s wr, 28 op/s 2023-07-18T03:38:48.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:48 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:48.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:48 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:48.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:48 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:49.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:49 smithi064 ceph-mon[98059]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 383 KiB/s wr, 39 op/s 2023-07-18T03:38:49.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:49 smithi134 ceph-mon[98216]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 383 KiB/s wr, 39 op/s 2023-07-18T03:38:49.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:49 smithi040 ceph-mon[96439]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 383 KiB/s wr, 39 op/s 2023-07-18T03:38:51.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:51 smithi064 ceph-mon[98059]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 374 KiB/s wr, 34 op/s 2023-07-18T03:38:51.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:51 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:51.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:51 smithi134 ceph-mon[98216]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 374 KiB/s wr, 34 op/s 2023-07-18T03:38:51.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:51 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:51.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:51 smithi040 ceph-mon[96439]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 374 KiB/s wr, 34 op/s 2023-07-18T03:38:51.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:51 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:38:52.944 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:38:53.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.077+0000 7fc5ebc42700 1 -- 172.21.15.40:0/2700042536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5e4100590 msgr2=0x7fc5e4106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.077+0000 7fc5ebc42700 1 --2- 172.21.15.40:0/2700042536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5e4100590 0x7fc5e4106e50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00b0a0 tx=0x7fc5dc00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.077+0000 7fc5ebc42700 1 -- 172.21.15.40:0/2700042536 shutdown_connections 2023-07-18T03:38:53.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.077+0000 7fc5ebc42700 1 --2- 172.21.15.40:0/2700042536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5e4100590 0x7fc5e4106e50 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.077+0000 7fc5ebc42700 1 --2- 172.21.15.40:0/2700042536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e40fea10 0x7fc5e40fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.077+0000 7fc5ebc42700 1 --2- 172.21.15.40:0/2700042536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5e40ffc00 0x7fc5e4100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.077+0000 7fc5ebc42700 1 -- 172.21.15.40:0/2700042536 >> 172.21.15.40:0/2700042536 conn(0x7fc5e40fa030 msgr2=0x7fc5e40fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:53.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.077+0000 7fc5ebc42700 1 -- 172.21.15.40:0/2700042536 shutdown_connections 2023-07-18T03:38:53.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.077+0000 7fc5ebc42700 1 -- 172.21.15.40:0/2700042536 wait complete. 2023-07-18T03:38:53.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.078+0000 7fc5ebc42700 1 Processor -- start 2023-07-18T03:38:53.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.078+0000 7fc5ebc42700 1 -- start start 2023-07-18T03:38:53.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.078+0000 7fc5ebc42700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5e40fea10 0x7fc5e4195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.078+0000 7fc5ebc42700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5e40ffc00 0x7fc5e4195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.078+0000 7fc5ebc42700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e4100590 0x7fc5e419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.078+0000 7fc5ebc42700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e419d4f0 con 0x7fc5e40fea10 2023-07-18T03:38:53.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.078+0000 7fc5ebc42700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e419d630 con 0x7fc5e40ffc00 2023-07-18T03:38:53.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.078+0000 7fc5ebc42700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e419d7c0 con 0x7fc5e4100590 2023-07-18T03:38:53.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.078+0000 7fc5e99de700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5e40fea10 0x7fc5e4195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.078+0000 7fc5e99de700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5e40fea10 0x7fc5e4195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60870/0 (socket says 172.21.15.40:60870) 2023-07-18T03:38:53.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.078+0000 7fc5e91dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5e40ffc00 0x7fc5e4195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.078+0000 7fc5e99de700 1 -- 172.21.15.40:0/2191495829 learned_addr learned my addr 172.21.15.40:0/2191495829 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:53.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.078+0000 7fc5ea1df700 1 --2- 172.21.15.40:0/2191495829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e4100590 0x7fc5e419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.079+0000 7fc5e91dd700 1 -- 172.21.15.40:0/2191495829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e4100590 msgr2=0x7fc5e419cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.079+0000 7fc5e91dd700 1 --2- 172.21.15.40:0/2191495829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e4100590 0x7fc5e419cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.079+0000 7fc5e91dd700 1 -- 172.21.15.40:0/2191495829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5e40fea10 msgr2=0x7fc5e4195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.079+0000 7fc5e91dd700 1 --2- 172.21.15.40:0/2191495829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5e40fea10 0x7fc5e4195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.079+0000 7fc5e91dd700 1 -- 172.21.15.40:0/2191495829 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc009d00 con 0x7fc5e40ffc00 2023-07-18T03:38:53.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.079+0000 7fc5e99de700 1 --2- 172.21.15.40:0/2191495829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5e40fea10 0x7fc5e4195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:53.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.079+0000 7fc5e91dd700 1 --2- 172.21.15.40:0/2191495829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5e40ffc00 0x7fc5e4195eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009c50 tx=0x7fc5d4009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:53.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.079+0000 7fc5daffd700 1 -- 172.21.15.40:0/2191495829 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4015070 con 0x7fc5e40ffc00 2023-07-18T03:38:53.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.079+0000 7fc5ebc42700 1 -- 172.21.15.40:0/2191495829 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e4108bf0 con 0x7fc5e40ffc00 2023-07-18T03:38:53.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.079+0000 7fc5daffd700 1 -- 172.21.15.40:0/2191495829 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5d4019410 con 0x7fc5e40ffc00 2023-07-18T03:38:53.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.079+0000 7fc5daffd700 1 -- 172.21.15.40:0/2191495829 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4018610 con 0x7fc5e40ffc00 2023-07-18T03:38:53.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.079+0000 7fc5ebc42700 1 -- 172.21.15.40:0/2191495829 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e4109150 con 0x7fc5e40ffc00 2023-07-18T03:38:53.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.081+0000 7fc5daffd700 1 -- 172.21.15.40:0/2191495829 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc5d4019880 con 0x7fc5e40ffc00 2023-07-18T03:38:53.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.082+0000 7fc5daffd700 1 --2- 172.21.15.40:0/2191495829 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5d00a1150 0x7fc5d00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.082+0000 7fc5e99de700 1 --2- 172.21.15.40:0/2191495829 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5d00a1150 0x7fc5d00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.082+0000 7fc5daffd700 1 -- 172.21.15.40:0/2191495829 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc5d40cffa0 con 0x7fc5e40ffc00 2023-07-18T03:38:53.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.082+0000 7fc5e99de700 1 --2- 172.21.15.40:0/2191495829 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5d00a1150 0x7fc5d00a3600 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0005950 tx=0x7fc5e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:53.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.083+0000 7fc5f7dc8b80 1 Processor -- start 2023-07-18T03:38:53.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.083+0000 7fc5f7dc8b80 1 -- start start 2023-07-18T03:38:53.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.083+0000 7fc5f7dc8b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e34d0 0x55d24f9e38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.083+0000 7fc5f7dc8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e3e20 0x55d24f9e46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.083+0000 7fc5f7dc8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e4c70 0x55d24f9e50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.083+0000 7fc5f7dc8b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24f8e8cb0 con 0x55d24f9e34d0 2023-07-18T03:38:53.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.083+0000 7fc5f7dc8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24f93ba40 con 0x55d24f9e3e20 2023-07-18T03:38:53.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.083+0000 7fc5f7dc8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24f934ed0 con 0x55d24f9e4c70 2023-07-18T03:38:53.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.083+0000 7fc5e99de700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e34d0 0x55d24f9e38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.083+0000 7fc5e99de700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e34d0 0x55d24f9e38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60878/0 (socket says 172.21.15.40:60878) 2023-07-18T03:38:53.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.083+0000 7fc5e99de700 1 -- 172.21.15.40:0/2121596619 learned_addr learned my addr 172.21.15.40:0/2121596619 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:53.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.083+0000 7fc5ea1df700 1 --2- 172.21.15.40:0/2121596619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e4c70 0x55d24f9e50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.083+0000 7fc5e91dd700 1 --2- 172.21.15.40:0/2121596619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e3e20 0x55d24f9e46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.084+0000 7fc5e99de700 1 -- 172.21.15.40:0/2121596619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e4c70 msgr2=0x55d24f9e50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:53.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.084+0000 7fc5e99de700 1 --2- 172.21.15.40:0/2121596619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e4c70 0x55d24f9e50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.084+0000 7fc5e99de700 1 -- 172.21.15.40:0/2121596619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e3e20 msgr2=0x55d24f9e46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:53.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.084+0000 7fc5e99de700 1 --2- 172.21.15.40:0/2121596619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e3e20 0x55d24f9e46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.084+0000 7fc5e99de700 1 -- 172.21.15.40:0/2121596619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d24f762de0 con 0x55d24f9e34d0 2023-07-18T03:38:53.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.084+0000 7fc5e99de700 1 --2- 172.21.15.40:0/2121596619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e34d0 0x55d24f9e38e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009880 tx=0x7fc5e0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438d10c1365ae6ad server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:53.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.084+0000 7fc5cffff700 1 -- 172.21.15.40:0/2121596619 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0004a30 con 0x55d24f9e34d0 2023-07-18T03:38:53.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.084+0000 7fc5cffff700 1 -- 172.21.15.40:0/2121596619 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5e0021070 con 0x55d24f9e34d0 2023-07-18T03:38:53.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.084+0000 7fc5cffff700 1 -- 172.21.15.40:0/2121596619 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e001c670 con 0x55d24f9e34d0 2023-07-18T03:38:53.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.085+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/2121596619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e34d0 msgr2=0x55d24f9e38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.085+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/2121596619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e34d0 0x55d24f9e38e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009880 tx=0x7fc5e0014000 comp rx=0 tx=0).stop 2023-07-18T03:38:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.085+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/2121596619 shutdown_connections 2023-07-18T03:38:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.085+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/2121596619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e4c70 0x55d24f9e50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.085+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/2121596619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e34d0 0x55d24f9e38e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.085+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/2121596619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e3e20 0x55d24f9e46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.085+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/2121596619 >> 172.21.15.40:0/2121596619 conn(0x55d24f9d8da0 msgr2=0x55d24f9de1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:53.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.085+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/2121596619 shutdown_connections 2023-07-18T03:38:53.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.085+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/2121596619 wait complete. 2023-07-18T03:38:53.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.085+0000 7fc5f7dc8b80 1 -- start start 2023-07-18T03:38:53.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.085+0000 7fc5f7dc8b80 1 Processor -- start 2023-07-18T03:38:53.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5f7dc8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e34d0 0x55d24f9ed1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5f7dc8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e3e20 0x55d24f9ed6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5f7dc8b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e4c70 0x55d24f9f4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5f7dc8b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24f8d4ac0 con 0x55d24f9e4c70 2023-07-18T03:38:53.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5f7dc8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24f8d47c0 con 0x55d24f9e3e20 2023-07-18T03:38:53.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5f7dc8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24f7b8920 con 0x55d24f9e34d0 2023-07-18T03:38:53.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5ea1df700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e4c70 0x55d24f9f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5f7dc8b80 10 client.0 _ll_register_callbacks cb 0x55d24f8d2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5e91dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e3e20 0x55d24f9ed6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5ea1df700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e4c70 0x55d24f9f4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60888/0 (socket says 172.21.15.40:60888) 2023-07-18T03:38:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5ea1df700 1 -- 172.21.15.40:0/3311621776 learned_addr learned my addr 172.21.15.40:0/3311621776 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:53.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5e99de700 1 --2- 172.21.15.40:0/3311621776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e34d0 0x55d24f9ed1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5ea1df700 1 -- 172.21.15.40:0/3311621776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e34d0 msgr2=0x55d24f9ed1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:53.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5ea1df700 1 --2- 172.21.15.40:0/3311621776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e34d0 0x55d24f9ed1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5ea1df700 1 -- 172.21.15.40:0/3311621776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e3e20 msgr2=0x55d24f9ed6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:53.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5ea1df700 1 --2- 172.21.15.40:0/3311621776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e3e20 0x55d24f9ed6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5ea1df700 1 -- 172.21.15.40:0/3311621776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e0017070 con 0x55d24f9e4c70 2023-07-18T03:38:53.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5e91dd700 1 --2- 172.21.15.40:0/3311621776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e3e20 0x55d24f9ed6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:53.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5ea1df700 1 --2- 172.21.15.40:0/3311621776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e4c70 0x55d24f9f4720 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc015000 tx=0x7fc5dc007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16020964e765be6c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.086+0000 7fc5cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:38:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.087+0000 7fc5cd7fa700 1 -- 172.21.15.40:0/3311621776 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc00e040 con 0x55d24f9e4c70 2023-07-18T03:38:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.087+0000 7fc5cd7fa700 1 -- 172.21.15.40:0/3311621776 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc5dc00ba50 con 0x55d24f9e4c70 2023-07-18T03:38:53.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.087+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/3311621776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d24f88e4f0 con 0x55d24f9e4c70 2023-07-18T03:38:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.087+0000 7fc5cd7fa700 1 -- 172.21.15.40:0/3311621776 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5dc00bc10 con 0x55d24f9e4c70 2023-07-18T03:38:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.087+0000 7fc5f7dc8b80 10 client.16602 fetch_fsmap learned FSMap version 57 2023-07-18T03:38:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.087+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/3311621776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55d24f74bc60 con 0x55d24f9e4c70 2023-07-18T03:38:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.087+0000 7fc5cd7fa700 1 -- 172.21.15.40:0/3311621776 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d24f88e4f0 con 0x55d24f9e4c70 2023-07-18T03:38:53.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.087+0000 7fc5cd7fa700 1 -- 172.21.15.40:0/3311621776 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 57) v1 ==== 7526+0+0 (secure 0 0 0) 0x7fc5dc02e020 con 0x55d24f9e4c70 2023-07-18T03:38:53.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.087+0000 7fc5f7dc8b80 10 client.16602 fetch_fsmap finished waiting for FSMap version 57 2023-07-18T03:38:53.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.087+0000 7fc5f7dc8b80 10 client.16602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:38:53.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.087+0000 7fc5f7dc8b80 20 client.16602 populate_metadata read hostname 'smithi040' 2023-07-18T03:38:53.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.087+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/3311621776 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d24f9f90a0 0x55d24f9fb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.087+0000 7fc5f7dc8b80 4 client.16602 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:38:53.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.087+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/3311621776 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d24f92deb0 con 0x55d24f9f90a0 2023-07-18T03:38:53.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.087+0000 7fc5e99de700 1 --2- 172.21.15.40:0/3311621776 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d24f9f90a0 0x55d24f9fb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.088+0000 7fc5cd7fa700 10 client.16602 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:53.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.088+0000 7fc5e99de700 1 --2- 172.21.15.40:0/3311621776 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d24f9f90a0 0x55d24f9fb4b0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dd7a014173f341a server_cookie=c7553a20bb64cbd9 in_seq=0 out_seq=0 2023-07-18T03:38:53.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.089+0000 7fc5cd7fa700 1 -- 172.21.15.40:0/3311621776 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d24f92deb0 con 0x55d24f9f90a0 2023-07-18T03:38:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.089+0000 7fc5cd7fa700 10 client.16602 handle_command_reply: tid=0 2023-07-18T03:38:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.089+0000 7fc5f7dc8b80 1 client.16602 shutdown 2023-07-18T03:38:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.089+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/3311621776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e4c70 msgr2=0x55d24f9f4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.089+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/3311621776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e4c70 0x55d24f9f4720 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc015000 tx=0x7fc5dc007f30 comp rx=0 tx=0).stop 2023-07-18T03:38:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.089+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/3311621776 shutdown_connections 2023-07-18T03:38:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.089+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/3311621776 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d24f9f90a0 0x55d24f9fb4b0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.089+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/3311621776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e4c70 0x55d24f9f4720 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.089+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/3311621776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e34d0 0x55d24f9ed1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.089+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/3311621776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e3e20 0x55d24f9ed6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.089+0000 7fc5cd7fa700 0 client.16602 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.089+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/3311621776 >> 172.21.15.40:0/3311621776 conn(0x55d24f9d8da0 msgr2=0x55d24f9dd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.089+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/3311621776 shutdown_connections 2023-07-18T03:38:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.089+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/3311621776 wait complete. 2023-07-18T03:38:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.089+0000 7fc5f7dc8b80 20 client.16602 trim_cache size 0 max 16384 2023-07-18T03:38:53.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5f7dc8b80 1 Processor -- start 2023-07-18T03:38:53.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5f7dc8b80 1 -- start start 2023-07-18T03:38:53.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5f7dc8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e34d0 0x55d24f9ebeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5f7dc8b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e3e20 0x55d24f9f4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5f7dc8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e4c70 0x55d24f9eb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5f7dc8b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24f9d8970 con 0x55d24f9e3e20 2023-07-18T03:38:53.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5f7dc8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24f9eae30 con 0x55d24f9e4c70 2023-07-18T03:38:53.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5f7dc8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24f9f8d70 con 0x55d24f9e34d0 2023-07-18T03:38:53.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5e91dd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e3e20 0x55d24f9f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5e91dd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e3e20 0x55d24f9f4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60894/0 (socket says 172.21.15.40:60894) 2023-07-18T03:38:53.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5e91dd700 1 -- 172.21.15.40:0/3668950742 learned_addr learned my addr 172.21.15.40:0/3668950742 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:53.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5ea1df700 1 --2- 172.21.15.40:0/3668950742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e4c70 0x55d24f9eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5e99de700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e34d0 0x55d24f9ebeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5e91dd700 1 -- 172.21.15.40:0/3668950742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e34d0 msgr2=0x55d24f9ebeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:53.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5e91dd700 1 --2- 172.21.15.40:0/3668950742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e34d0 0x55d24f9ebeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5e91dd700 1 -- 172.21.15.40:0/3668950742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e4c70 msgr2=0x55d24f9eb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:53.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5e91dd700 1 --2- 172.21.15.40:0/3668950742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e4c70 0x55d24f9eb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5e91dd700 1 -- 172.21.15.40:0/3668950742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d24f789250 con 0x55d24f9e3e20 2023-07-18T03:38:53.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.106+0000 7fc5ea1df700 1 --2- 172.21.15.40:0/3668950742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e4c70 0x55d24f9eb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:53.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.107+0000 7fc5e91dd700 1 --2- 172.21.15.40:0/3668950742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e3e20 0x55d24f9f4640 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4099db0 tx=0x7fc5d4088040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720e31e73ba5a0a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:53.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.107+0000 7fc5d8dac700 1 -- 172.21.15.40:0/3668950742 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d409ced0 con 0x55d24f9e3e20 2023-07-18T03:38:53.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.107+0000 7fc5d8dac700 1 -- 172.21.15.40:0/3668950742 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5d409ccf0 con 0x55d24f9e3e20 2023-07-18T03:38:53.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.107+0000 7fc5d8dac700 1 -- 172.21.15.40:0/3668950742 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d409d850 con 0x55d24f9e3e20 2023-07-18T03:38:53.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.107+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/3668950742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e3e20 msgr2=0x55d24f9f4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.107+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/3668950742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e3e20 0x55d24f9f4640 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4099db0 tx=0x7fc5d4088040 comp rx=0 tx=0).stop 2023-07-18T03:38:53.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.107+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/3668950742 shutdown_connections 2023-07-18T03:38:53.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.107+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/3668950742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e4c70 0x55d24f9eb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.107+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/3668950742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e34d0 0x55d24f9ebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.107+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/3668950742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e3e20 0x55d24f9f4640 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.107+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/3668950742 >> 172.21.15.40:0/3668950742 conn(0x55d24f9d8da0 msgr2=0x55d24f9ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:53.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.107+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/3668950742 shutdown_connections 2023-07-18T03:38:53.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.107+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/3668950742 wait complete. 2023-07-18T03:38:53.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.108+0000 7fc5f7dc8b80 1 -- start start 2023-07-18T03:38:53.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.108+0000 7fc5f7dc8b80 1 Processor -- start 2023-07-18T03:38:53.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.108+0000 7fc5f7dc8b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e34d0 0x55d24f9f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.108+0000 7fc5f7dc8b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e3e20 0x55d24f9f4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.108+0000 7fc5f7dc8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e4c70 0x55d24f9f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.108+0000 7fc5f7dc8b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24f9f10b0 con 0x55d24f9e3e20 2023-07-18T03:38:53.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.108+0000 7fc5f7dc8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24f9f1220 con 0x55d24f9e4c70 2023-07-18T03:38:53.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.108+0000 7fc5f7dc8b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24f9f13e0 con 0x55d24f9e34d0 2023-07-18T03:38:53.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.108+0000 7fc5f7dc8b80 10 client.0 _ll_register_callbacks cb 0x7fc5dc009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:53.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.108+0000 7fc5e91dd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e3e20 0x55d24f9f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.108+0000 7fc5e91dd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e3e20 0x55d24f9f4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60900/0 (socket says 172.21.15.40:60900) 2023-07-18T03:38:53.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.108+0000 7fc5ea1df700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e4c70 0x55d24f9f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.108+0000 7fc5e91dd700 1 -- 172.21.15.40:0/335498817 learned_addr learned my addr 172.21.15.40:0/335498817 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:53.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.108+0000 7fc5e99de700 1 --2- 172.21.15.40:0/335498817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e34d0 0x55d24f9f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5e91dd700 1 -- 172.21.15.40:0/335498817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e34d0 msgr2=0x55d24f9f3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:53.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5e91dd700 1 --2- 172.21.15.40:0/335498817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e34d0 0x55d24f9f3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5e91dd700 1 -- 172.21.15.40:0/335498817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e4c70 msgr2=0x55d24f9f09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:53.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5e91dd700 1 --2- 172.21.15.40:0/335498817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e4c70 0x55d24f9f09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5e91dd700 1 -- 172.21.15.40:0/335498817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d24f74bc60 con 0x55d24f9e3e20 2023-07-18T03:38:53.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5ea1df700 1 --2- 172.21.15.40:0/335498817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e4c70 0x55d24f9f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:53.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5e91dd700 1 --2- 172.21.15.40:0/335498817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e3e20 0x55d24f9f4470 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009c20 tx=0x7fc5d409cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79965fff189901ac server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:53.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:38:53.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5cd7fa700 1 -- 172.21.15.40:0/335498817 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d409d7f0 con 0x55d24f9e3e20 2023-07-18T03:38:53.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5cd7fa700 1 -- 172.21.15.40:0/335498817 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fc5d40a57e0 con 0x55d24f9e3e20 2023-07-18T03:38:53.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/335498817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5e0021070 con 0x55d24f9e3e20 2023-07-18T03:38:53.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5cd7fa700 1 -- 172.21.15.40:0/335498817 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5d409d390 con 0x55d24f9e3e20 2023-07-18T03:38:53.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5f7dc8b80 10 client.16608 fetch_fsmap learned FSMap version 57 2023-07-18T03:38:53.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/335498817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x7fc5e4109150 con 0x55d24f9e3e20 2023-07-18T03:38:53.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5cd7fa700 1 -- 172.21.15.40:0/335498817 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5e0021070 con 0x55d24f9e3e20 2023-07-18T03:38:53.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5cd7fa700 1 -- 172.21.15.40:0/335498817 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 57) v1 ==== 7526+0+0 (secure 0 0 0) 0x7fc5d407d030 con 0x55d24f9e3e20 2023-07-18T03:38:53.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5f7dc8b80 10 client.16608 fetch_fsmap finished waiting for FSMap version 57 2023-07-18T03:38:53.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5f7dc8b80 10 client.16608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:38:53.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5f7dc8b80 20 client.16608 populate_metadata read hostname 'smithi040' 2023-07-18T03:38:53.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/335498817 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d24fa4e140 0x55d24fa50550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:53.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5f7dc8b80 4 client.16608 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:38:53.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.109+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/335498817 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d24f8f0050 con 0x55d24fa4e140 2023-07-18T03:38:53.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.110+0000 7fc5e99de700 1 --2- 172.21.15.40:0/335498817 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d24fa4e140 0x55d24fa50550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:53.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.110+0000 7fc5cd7fa700 10 client.16608 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:53.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.110+0000 7fc5e99de700 1 --2- 172.21.15.40:0/335498817 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d24fa4e140 0x55d24fa50550 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=977627713e58e237 server_cookie=31bbf06b0c5af63e in_seq=0 out_seq=0 2023-07-18T03:38:53.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.110+0000 7fc5cd7fa700 1 -- 172.21.15.40:0/335498817 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d24f8f0050 con 0x55d24fa4e140 2023-07-18T03:38:53.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.110+0000 7fc5cd7fa700 10 client.16608 handle_command_reply: tid=0 2023-07-18T03:38:53.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.110+0000 7fc5f7dc8b80 1 client.16608 shutdown 2023-07-18T03:38:53.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.110+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/335498817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e3e20 msgr2=0x55d24f9f4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.110+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/335498817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e3e20 0x55d24f9f4470 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009c20 tx=0x7fc5d409cc10 comp rx=0 tx=0).stop 2023-07-18T03:38:53.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.110+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/335498817 shutdown_connections 2023-07-18T03:38:53.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.110+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/335498817 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55d24fa4e140 0x55d24fa50550 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.110+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/335498817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d24f9e4c70 0x55d24f9f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.110+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/335498817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d24f9e34d0 0x55d24f9f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.110+0000 7fc5cd7fa700 0 client.16608 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:53.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.110+0000 7fc5f7dc8b80 1 --2- 172.21.15.40:0/335498817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d24f9e3e20 0x55d24f9f4470 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.110+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/335498817 >> 172.21.15.40:0/335498817 conn(0x55d24f9d8da0 msgr2=0x55d24f9eb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:53.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.110+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/335498817 shutdown_connections 2023-07-18T03:38:53.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.110+0000 7fc5f7dc8b80 1 -- 172.21.15.40:0/335498817 wait complete. 2023-07-18T03:38:53.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.111+0000 7fc5f7dc8b80 20 client.16608 trim_cache size 0 max 16384 2023-07-18T03:38:53.128 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:38:53.128 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-18T03:38:53.129 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:38:53.129 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:38:53.129 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:38:53.129 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:38:53.129 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:38:53.129 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:38:53.129 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:38:53.130 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:38:53.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.111+0000 7fc5ebc42700 1 -- 172.21.15.40:0/2191495829 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5d00a1150 msgr2=0x7fc5d00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.111+0000 7fc5ebc42700 1 --2- 172.21.15.40:0/2191495829 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5d00a1150 0x7fc5d00a3600 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0005950 tx=0x7fc5e0009260 comp rx=0 tx=0).stop 2023-07-18T03:38:53.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.111+0000 7fc5ebc42700 1 -- 172.21.15.40:0/2191495829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5e40ffc00 msgr2=0x7fc5e4195eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:53.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.111+0000 7fc5ebc42700 1 --2- 172.21.15.40:0/2191495829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5e40ffc00 0x7fc5e4195eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009c50 tx=0x7fc5d4009f60 comp rx=0 tx=0).stop 2023-07-18T03:38:53.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.111+0000 7fc5ebc42700 1 -- 172.21.15.40:0/2191495829 shutdown_connections 2023-07-18T03:38:53.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.111+0000 7fc5ebc42700 1 --2- 172.21.15.40:0/2191495829 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fc5d00a1150 0x7fc5d00a3600 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.111+0000 7fc5ebc42700 1 --2- 172.21.15.40:0/2191495829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5e4100590 0x7fc5e419cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.111+0000 7fc5ebc42700 1 --2- 172.21.15.40:0/2191495829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5e40fea10 0x7fc5e4195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.111+0000 7fc5ebc42700 1 --2- 172.21.15.40:0/2191495829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5e40ffc00 0x7fc5e4195eb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:53.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.111+0000 7fc5ebc42700 1 -- 172.21.15.40:0/2191495829 >> 172.21.15.40:0/2191495829 conn(0x7fc5e40fa030 msgr2=0x7fc5e40fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:53.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.111+0000 7fc5ebc42700 1 -- 172.21.15.40:0/2191495829 shutdown_connections 2023-07-18T03:38:53.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:38:53.111+0000 7fc5ebc42700 1 -- 172.21.15.40:0/2191495829 wait complete. 2023-07-18T03:38:53.133 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:38:53.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:52 smithi134 ceph-mon[98216]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 471 KiB/s wr, 91 op/s 2023-07-18T03:38:53.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:52 smithi040 ceph-mon[96439]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 471 KiB/s wr, 91 op/s 2023-07-18T03:38:53.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:52 smithi064 ceph-mon[98059]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 471 KiB/s wr, 91 op/s 2023-07-18T03:38:54.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:53 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:38:54.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:53 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:54.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:53 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:54.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:53 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:54.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:53 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:54.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:53 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:38:54.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:53 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:54.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:53 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:54.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:53 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:54.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:53 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:54.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:53 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:38:54.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:53 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:54.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:53 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:54.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:53 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:54.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:53 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.120+0000 7f77c0989700 1 -- 172.21.15.40:0/1302639681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77bc1005a0 msgr2=0x7f77bc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.120+0000 7f77c0989700 1 --2- 172.21.15.40:0/1302639681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77bc1005a0 0x7f77bc106e60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f77b400b0a0 tx=0x7f77b400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.120+0000 7f77c0989700 1 -- 172.21.15.40:0/1302639681 shutdown_connections 2023-07-18T03:38:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.120+0000 7f77c0989700 1 --2- 172.21.15.40:0/1302639681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77bc1005a0 0x7f77bc106e60 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.120+0000 7f77c0989700 1 --2- 172.21.15.40:0/1302639681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77bc0fea20 0x7f77bc0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.120+0000 7f77c0989700 1 --2- 172.21.15.40:0/1302639681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77bc0ffc10 0x7f77bc100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.120+0000 7f77c0989700 1 -- 172.21.15.40:0/1302639681 >> 172.21.15.40:0/1302639681 conn(0x7f77bc0fa020 msgr2=0x7f77bc0fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.120+0000 7f77c0989700 1 -- 172.21.15.40:0/1302639681 shutdown_connections 2023-07-18T03:38:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.120+0000 7f77c0989700 1 -- 172.21.15.40:0/1302639681 wait complete. 2023-07-18T03:38:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.121+0000 7f77c0989700 1 Processor -- start 2023-07-18T03:38:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.121+0000 7f77c0989700 1 -- start start 2023-07-18T03:38:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.121+0000 7f77c0989700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77bc0fea20 0x7f77bc195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.121+0000 7f77c0989700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77bc0ffc10 0x7f77bc196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.121+0000 7f77c0989700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77bc1005a0 0x7f77bc196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.121+0000 7f77c0989700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc197510 con 0x7f77bc0ffc10 2023-07-18T03:38:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.121+0000 7f77c0989700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc197680 con 0x7f77bc1005a0 2023-07-18T03:38:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.121+0000 7f77c0989700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc197840 con 0x7f77bc0fea20 2023-07-18T03:38:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.121+0000 7f77b9d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77bc0ffc10 0x7f77bc196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.121+0000 7f77ba59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77bc0fea20 0x7f77bc195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.121+0000 7f77b9d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77bc0ffc10 0x7f77bc196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60912/0 (socket says 172.21.15.40:60912) 2023-07-18T03:38:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.121+0000 7f77b9d9b700 1 -- 172.21.15.40:0/566646414 learned_addr learned my addr 172.21.15.40:0/566646414 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.121+0000 7f77bad9d700 1 --2- 172.21.15.40:0/566646414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77bc1005a0 0x7f77bc196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.122+0000 7f77ba59c700 1 -- 172.21.15.40:0/566646414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77bc1005a0 msgr2=0x7f77bc196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.122+0000 7f77ba59c700 1 --2- 172.21.15.40:0/566646414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77bc1005a0 0x7f77bc196eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.122+0000 7f77ba59c700 1 -- 172.21.15.40:0/566646414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77bc0ffc10 msgr2=0x7f77bc196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.122+0000 7f77ba59c700 1 --2- 172.21.15.40:0/566646414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77bc0ffc10 0x7f77bc196030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.122+0000 7f77ba59c700 1 -- 172.21.15.40:0/566646414 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b4009d00 con 0x7f77bc0fea20 2023-07-18T03:38:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.122+0000 7f77b9d9b700 1 --2- 172.21.15.40:0/566646414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77bc0ffc10 0x7f77bc196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.122+0000 7f77ba59c700 1 --2- 172.21.15.40:0/566646414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77bc0fea20 0x7f77bc195af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00e9c0 tx=0x7f77ac00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.123+0000 7f77ab7fe700 1 -- 172.21.15.40:0/566646414 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac004230 con 0x7f77bc0fea20 2023-07-18T03:38:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.123+0000 7f77ab7fe700 1 -- 172.21.15.40:0/566646414 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77ac004390 con 0x7f77bc0fea20 2023-07-18T03:38:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.123+0000 7f77ab7fe700 1 -- 172.21.15.40:0/566646414 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac003d30 con 0x7f77bc0fea20 2023-07-18T03:38:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.123+0000 7f77c0989700 1 -- 172.21.15.40:0/566646414 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77bc197b20 con 0x7f77bc0fea20 2023-07-18T03:38:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.123+0000 7f77c0989700 1 -- 172.21.15.40:0/566646414 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77bc19e2e0 con 0x7f77bc0fea20 2023-07-18T03:38:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.126+0000 7f77ab7fe700 1 -- 172.21.15.40:0/566646414 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f77ac00c7c0 con 0x7f77bc0fea20 2023-07-18T03:38:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.127+0000 7f77ab7fe700 1 --2- 172.21.15.40:0/566646414 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f77a40a11d0 0x7f77a40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.127+0000 7f77b9d9b700 1 --2- 172.21.15.40:0/566646414 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f77a40a11d0 0x7f77a40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.127+0000 7f77ab7fe700 1 -- 172.21.15.40:0/566646414 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f77ac014070 con 0x7f77bc0fea20 2023-07-18T03:38:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.127+0000 7f77b9d9b700 1 --2- 172.21.15.40:0/566646414 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f77a40a11d0 0x7f77a40a3680 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f77b00103d0 tx=0x7f77b0007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.127+0000 7f77ccb0fb80 1 Processor -- start 2023-07-18T03:38:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77ccb0fb80 1 -- start start 2023-07-18T03:38:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77ccb0fb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69aa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77ccb0fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd69aa840 0x55dfd69ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77ccb0fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 0x55dfd69abb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77ccb0fb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd68afc50 con 0x55dfd68066b0 2023-07-18T03:38:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77ccb0fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd69029e0 con 0x55dfd69aa840 2023-07-18T03:38:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77ccb0fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd68fbe70 con 0x55dfd69ab6e0 2023-07-18T03:38:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77ba59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77ba59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69aa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60918/0 (socket says 172.21.15.40:60918) 2023-07-18T03:38:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77ba59c700 1 -- 172.21.15.40:0/2966387015 learned_addr learned my addr 172.21.15.40:0/2966387015 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77b9d9b700 1 --2- 172.21.15.40:0/2966387015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd69aa840 0x55dfd69ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77bad9d700 1 --2- 172.21.15.40:0/2966387015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 0x55dfd69abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77ba59c700 1 -- 172.21.15.40:0/2966387015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 msgr2=0x55dfd69abb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77ba59c700 1 --2- 172.21.15.40:0/2966387015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 0x55dfd69abb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77ba59c700 1 -- 172.21.15.40:0/2966387015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd69aa840 msgr2=0x55dfd69ab1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77ba59c700 1 --2- 172.21.15.40:0/2966387015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd69aa840 0x55dfd69ab1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77ba59c700 1 -- 172.21.15.40:0/2966387015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfd6729d80 con 0x55dfd68066b0 2023-07-18T03:38:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.128+0000 7f77ba59c700 1 --2- 172.21.15.40:0/2966387015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69aa300 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0a1eb0 tx=0x7f77ac099ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af2d3ad0c85217d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.129+0000 7f77a8ff9700 1 -- 172.21.15.40:0/2966387015 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac09dfc0 con 0x55dfd68066b0 2023-07-18T03:38:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.129+0000 7f77a8ff9700 1 -- 172.21.15.40:0/2966387015 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77ac0a23e0 con 0x55dfd68066b0 2023-07-18T03:38:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.129+0000 7f77a8ff9700 1 -- 172.21.15.40:0/2966387015 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0a1f50 con 0x55dfd68066b0 2023-07-18T03:38:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.129+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/2966387015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 msgr2=0x55dfd69aa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.129+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/2966387015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69aa300 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0a1eb0 tx=0x7f77ac099ad0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.129+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/2966387015 shutdown_connections 2023-07-18T03:38:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.129+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/2966387015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 0x55dfd69abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.129+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/2966387015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69aa300 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.129+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/2966387015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd69aa840 0x55dfd69ab1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.129+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/2966387015 >> 172.21.15.40:0/2966387015 conn(0x55dfd699fbe0 msgr2=0x55dfd69a4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.130+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/2966387015 shutdown_connections 2023-07-18T03:38:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.130+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/2966387015 wait complete. 2023-07-18T03:38:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.130+0000 7f77ccb0fb80 1 -- start start 2023-07-18T03:38:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.130+0000 7f77ccb0fb80 1 Processor -- start 2023-07-18T03:38:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.130+0000 7f77ccb0fb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.130+0000 7f77ccb0fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd69aa840 0x55dfd69b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.130+0000 7f77ccb0fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 0x55dfd69bae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.130+0000 7f77ccb0fb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd689ba60 con 0x55dfd68066b0 2023-07-18T03:38:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.130+0000 7f77ccb0fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd689b760 con 0x55dfd69aa840 2023-07-18T03:38:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.130+0000 7f77ccb0fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd677f8c0 con 0x55dfd69ab6e0 2023-07-18T03:38:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.130+0000 7f77ccb0fb80 10 client.0 _ll_register_callbacks cb 0x55dfd6899700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.130+0000 7f77ba59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.130+0000 7f77ba59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60924/0 (socket says 172.21.15.40:60924) 2023-07-18T03:38:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.130+0000 7f77ba59c700 1 -- 172.21.15.40:0/183335876 learned_addr learned my addr 172.21.15.40:0/183335876 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.130+0000 7f77bad9d700 1 --2- 172.21.15.40:0/183335876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 0x55dfd69bae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.130+0000 7f77b9d9b700 1 --2- 172.21.15.40:0/183335876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd69aa840 0x55dfd69b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77ba59c700 1 -- 172.21.15.40:0/183335876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 msgr2=0x55dfd69bae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77ba59c700 1 --2- 172.21.15.40:0/183335876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 0x55dfd69bae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77ba59c700 1 -- 172.21.15.40:0/183335876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd69aa840 msgr2=0x55dfd69b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77ba59c700 1 --2- 172.21.15.40:0/183335876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd69aa840 0x55dfd69b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77ba59c700 1 -- 172.21.15.40:0/183335876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77bc19e2e0 con 0x55dfd68066b0 2023-07-18T03:38:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77ba59c700 1 --2- 172.21.15.40:0/183335876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69b3880 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f77ac096490 tx=0x7f77ac09a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73741023c3fdb59 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:38:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77a1ffb700 1 -- 172.21.15.40:0/183335876 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0b81f0 con 0x55dfd68066b0 2023-07-18T03:38:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77a1ffb700 1 -- 172.21.15.40:0/183335876 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f77ac0b8350 con 0x55dfd68066b0 2023-07-18T03:38:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/183335876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfd6855490 con 0x55dfd68066b0 2023-07-18T03:38:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77a1ffb700 1 -- 172.21.15.40:0/183335876 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77ac0ca2a0 con 0x55dfd68066b0 2023-07-18T03:38:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77ccb0fb80 10 client.16620 fetch_fsmap learned FSMap version 57 2023-07-18T03:38:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/183335876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55dfd6712c00 con 0x55dfd68066b0 2023-07-18T03:38:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77a1ffb700 1 -- 172.21.15.40:0/183335876 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dfd6855490 con 0x55dfd68066b0 2023-07-18T03:38:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77a1ffb700 1 -- 172.21.15.40:0/183335876 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 57) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f77ac095ef0 con 0x55dfd68066b0 2023-07-18T03:38:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77ccb0fb80 10 client.16620 fetch_fsmap finished waiting for FSMap version 57 2023-07-18T03:38:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77ccb0fb80 10 client.16620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:38:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77ccb0fb80 20 client.16620 populate_metadata read hostname 'smithi040' 2023-07-18T03:38:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/183335876 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55dfd69bf460 0x55dfd69c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77ccb0fb80 4 client.16620 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:38:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.131+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/183335876 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfd68f4e50 con 0x55dfd69bf460 2023-07-18T03:38:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.132+0000 7f77b9d9b700 1 --2- 172.21.15.40:0/183335876 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55dfd69bf460 0x55dfd69c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.132+0000 7f77a1ffb700 10 client.16620 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.132+0000 7f77b9d9b700 1 --2- 172.21.15.40:0/183335876 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55dfd69bf460 0x55dfd69c1870 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c65cba90ca493b3a server_cookie=f09fadb0440d4f7a in_seq=0 out_seq=0 2023-07-18T03:38:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.133+0000 7f77a1ffb700 1 -- 172.21.15.40:0/183335876 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dfd68f4e50 con 0x55dfd69bf460 2023-07-18T03:38:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.133+0000 7f77a1ffb700 10 client.16620 handle_command_reply: tid=0 2023-07-18T03:38:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.133+0000 7f77ccb0fb80 1 client.16620 shutdown 2023-07-18T03:38:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.133+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/183335876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 msgr2=0x55dfd69b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.133+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/183335876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69b3880 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f77ac096490 tx=0x7f77ac09a100 comp rx=0 tx=0).stop 2023-07-18T03:38:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.133+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/183335876 shutdown_connections 2023-07-18T03:38:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.133+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/183335876 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55dfd69bf460 0x55dfd69c1870 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.133+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/183335876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 0x55dfd69bae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.133+0000 7f77a1ffb700 0 client.16620 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.133+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/183335876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69b3880 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.133+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/183335876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd69aa840 0x55dfd69b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.133+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/183335876 >> 172.21.15.40:0/183335876 conn(0x55dfd699fbe0 msgr2=0x55dfd69a44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.133+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/183335876 shutdown_connections 2023-07-18T03:38:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.133+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/183335876 wait complete. 2023-07-18T03:38:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.134+0000 7f77ccb0fb80 20 client.16620 trim_cache size 0 max 16384 2023-07-18T03:38:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.148+0000 7f77ccb0fb80 1 Processor -- start 2023-07-18T03:38:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.148+0000 7f77ccb0fb80 1 -- start start 2023-07-18T03:38:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77ccb0fb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69ae630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77ccb0fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd69aa840 0x55dfd69aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77ccb0fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 0x55dfd69badc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77ccb0fb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd68f4e50 con 0x55dfd68066b0 2023-07-18T03:38:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77ccb0fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd6855490 con 0x55dfd69aa840 2023-07-18T03:38:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77ccb0fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd69b2d30 con 0x55dfd69ab6e0 2023-07-18T03:38:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77ba59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77ba59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69ae630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60932/0 (socket says 172.21.15.40:60932) 2023-07-18T03:38:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77ba59c700 1 -- 172.21.15.40:0/4030530565 learned_addr learned my addr 172.21.15.40:0/4030530565 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77b9d9b700 1 --2- 172.21.15.40:0/4030530565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd69aa840 0x55dfd69aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77bad9d700 1 --2- 172.21.15.40:0/4030530565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 0x55dfd69badc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77ba59c700 1 -- 172.21.15.40:0/4030530565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 msgr2=0x55dfd69badc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77ba59c700 1 --2- 172.21.15.40:0/4030530565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 0x55dfd69badc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77ba59c700 1 -- 172.21.15.40:0/4030530565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd69aa840 msgr2=0x55dfd69aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77ba59c700 1 --2- 172.21.15.40:0/4030530565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd69aa840 0x55dfd69aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77ba59c700 1 -- 172.21.15.40:0/4030530565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfd67501f0 con 0x55dfd68066b0 2023-07-18T03:38:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77b9d9b700 1 --2- 172.21.15.40:0/4030530565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd69aa840 0x55dfd69aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:38:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77ba59c700 1 --2- 172.21.15.40:0/4030530565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69ae630 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0982e0 tx=0x7f77ac099480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cf3c0a8fa21820 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77a97fa700 1 -- 172.21.15.40:0/4030530565 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0990e0 con 0x55dfd68066b0 2023-07-18T03:38:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77a97fa700 1 -- 172.21.15.40:0/4030530565 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77ac098330 con 0x55dfd68066b0 2023-07-18T03:38:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.149+0000 7f77a97fa700 1 -- 172.21.15.40:0/4030530565 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0a1f50 con 0x55dfd68066b0 2023-07-18T03:38:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.150+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/4030530565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 msgr2=0x55dfd69ae630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.150+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/4030530565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69ae630 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0982e0 tx=0x7f77ac099480 comp rx=0 tx=0).stop 2023-07-18T03:38:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.150+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/4030530565 shutdown_connections 2023-07-18T03:38:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.150+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/4030530565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 0x55dfd69badc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.150+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/4030530565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd68066b0 0x55dfd69ae630 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.150+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/4030530565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd69aa840 0x55dfd69aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.150+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/4030530565 >> 172.21.15.40:0/4030530565 conn(0x55dfd699fbe0 msgr2=0x55dfd69b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.150+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/4030530565 shutdown_connections 2023-07-18T03:38:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.150+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/4030530565 wait complete. 2023-07-18T03:38:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.150+0000 7f77ccb0fb80 1 -- start start 2023-07-18T03:38:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77ccb0fb80 1 Processor -- start 2023-07-18T03:38:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77ccb0fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd68066b0 0x55dfd69b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77ccb0fb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd69aa840 0x55dfd69b7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77ccb0fb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 0x55dfd69b9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77ccb0fb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd69ba3c0 con 0x55dfd69aa840 2023-07-18T03:38:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77ccb0fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd69ba530 con 0x55dfd68066b0 2023-07-18T03:38:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77ccb0fb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfd69ba6f0 con 0x55dfd69ab6e0 2023-07-18T03:38:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77ccb0fb80 10 client.0 _ll_register_callbacks cb 0x7f77b0005d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:38:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77b9d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd69aa840 0x55dfd69b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77b9d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd69aa840 0x55dfd69b7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60940/0 (socket says 172.21.15.40:60940) 2023-07-18T03:38:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77b9d9b700 1 -- 172.21.15.40:0/298310691 learned_addr learned my addr 172.21.15.40:0/298310691 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:38:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77bad9d700 1 --2- 172.21.15.40:0/298310691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 0x55dfd69b9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77ba59c700 1 --2- 172.21.15.40:0/298310691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd68066b0 0x55dfd69b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77b9d9b700 1 -- 172.21.15.40:0/298310691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 msgr2=0x55dfd69b9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77b9d9b700 1 --2- 172.21.15.40:0/298310691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 0x55dfd69b9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77b9d9b700 1 -- 172.21.15.40:0/298310691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd68066b0 msgr2=0x55dfd69b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:38:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77b9d9b700 1 --2- 172.21.15.40:0/298310691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd68066b0 0x55dfd69b6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77b9d9b700 1 -- 172.21.15.40:0/298310691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b00117a0 con 0x55dfd69aa840 2023-07-18T03:38:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77b9d9b700 1 --2- 172.21.15.40:0/298310691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd69aa840 0x55dfd69b7570 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x55dfd69c2140 tx=0x7f77b00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8843109734e267f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:38:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.151+0000 7f77a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:38:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.152+0000 7f77a1ffb700 1 -- 172.21.15.40:0/298310691 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0024040 con 0x55dfd69aa840 2023-07-18T03:38:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.152+0000 7f77a1ffb700 1 -- 172.21.15.40:0/298310691 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f77b0017070 con 0x55dfd69aa840 2023-07-18T03:38:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.152+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/298310691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77ac0a23e0 con 0x55dfd69aa840 2023-07-18T03:38:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.152+0000 7f77a1ffb700 1 -- 172.21.15.40:0/298310691 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77b0029e20 con 0x55dfd69aa840 2023-07-18T03:38:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.152+0000 7f77ccb0fb80 10 client.16626 fetch_fsmap learned FSMap version 57 2023-07-18T03:38:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.152+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/298310691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55dfd6712c00 con 0x55dfd69aa840 2023-07-18T03:38:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.152+0000 7f77a1ffb700 1 -- 172.21.15.40:0/298310691 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77ac0a23e0 con 0x55dfd69aa840 2023-07-18T03:38:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.152+0000 7f77a1ffb700 1 -- 172.21.15.40:0/298310691 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 57) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f77b0032a70 con 0x55dfd69aa840 2023-07-18T03:38:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.152+0000 7f77ccb0fb80 10 client.16626 fetch_fsmap finished waiting for FSMap version 57 2023-07-18T03:38:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.152+0000 7f77ccb0fb80 10 client.16626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:38:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.152+0000 7f77ccb0fb80 20 client.16626 populate_metadata read hostname 'smithi040' 2023-07-18T03:38:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.152+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/298310691 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55dfd69bf460 0x55dfd6a16080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:38:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.152+0000 7f77ccb0fb80 4 client.16626 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:38:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.152+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/298310691 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dfd68b6ff0 con 0x55dfd69bf460 2023-07-18T03:38:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.152+0000 7f77ba59c700 1 --2- 172.21.15.40:0/298310691 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55dfd69bf460 0x55dfd6a16080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:38:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.153+0000 7f77a1ffb700 10 client.16626 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.153+0000 7f77ba59c700 1 --2- 172.21.15.40:0/298310691 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55dfd69bf460 0x55dfd6a16080 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3e66e8cab3143f5 server_cookie=e2fb92c1d8d52b65 in_seq=0 out_seq=0 2023-07-18T03:38:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.164+0000 7f77a1ffb700 1 -- 172.21.15.40:0/298310691 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dfd68b6ff0 con 0x55dfd69bf460 2023-07-18T03:38:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.164+0000 7f77a1ffb700 10 client.16626 handle_command_reply: tid=0 2023-07-18T03:38:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.164+0000 7f77ccb0fb80 1 client.16626 shutdown 2023-07-18T03:38:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/298310691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd69aa840 msgr2=0x55dfd69b7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/298310691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd69aa840 0x55dfd69b7570 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x55dfd69c2140 tx=0x7f77b00047c0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/298310691 shutdown_connections 2023-07-18T03:38:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/298310691 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55dfd69bf460 0x55dfd6a16080 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/298310691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dfd69ab6e0 0x55dfd69b9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77a1ffb700 0 client.16626 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:38:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/298310691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfd68066b0 0x55dfd69b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77ccb0fb80 1 --2- 172.21.15.40:0/298310691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55dfd69aa840 0x55dfd69b7570 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/298310691 >> 172.21.15.40:0/298310691 conn(0x55dfd699fbe0 msgr2=0x55dfd69ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/298310691 shutdown_connections 2023-07-18T03:38:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77ccb0fb80 1 -- 172.21.15.40:0/298310691 wait complete. 2023-07-18T03:38:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77ccb0fb80 20 client.16626 trim_cache size 0 max 16384 2023-07-18T03:38:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:38:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:38:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:38:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:38:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77c0989700 1 -- 172.21.15.40:0/566646414 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f77a40a11d0 msgr2=0x7f77a40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77c0989700 1 --2- 172.21.15.40:0/566646414 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f77a40a11d0 0x7f77a40a3680 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f77b00103d0 tx=0x7f77b0007480 comp rx=0 tx=0).stop 2023-07-18T03:38:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77c0989700 1 -- 172.21.15.40:0/566646414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77bc0fea20 msgr2=0x7f77bc195af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:38:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77c0989700 1 --2- 172.21.15.40:0/566646414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77bc0fea20 0x7f77bc195af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00e9c0 tx=0x7f77ac00ecd0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77c0989700 1 -- 172.21.15.40:0/566646414 shutdown_connections 2023-07-18T03:38:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77c0989700 1 --2- 172.21.15.40:0/566646414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77bc1005a0 0x7f77bc196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77c0989700 1 --2- 172.21.15.40:0/566646414 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f77a40a11d0 0x7f77a40a3680 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77c0989700 1 --2- 172.21.15.40:0/566646414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77bc0fea20 0x7f77bc195af0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77c0989700 1 --2- 172.21.15.40:0/566646414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77bc0ffc10 0x7f77bc196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:38:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.165+0000 7f77c0989700 1 -- 172.21.15.40:0/566646414 >> 172.21.15.40:0/566646414 conn(0x7f77bc0fa020 msgr2=0x7f77bc068460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:38:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.166+0000 7f77c0989700 1 -- 172.21.15.40:0/566646414 shutdown_connections 2023-07-18T03:38:55.166 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:38:55.166+0000 7f77c0989700 1 -- 172.21.15.40:0/566646414 wait complete. 2023-07-18T03:38:55.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: Adjusting osd_memory_target on smithi040 to 197.2M 2023-07-18T03:38:55.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: Unable to set osd_memory_target on smithi040 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-07-18T03:38:55.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.321 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.321 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-07-18T03:38:55.321 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-18T03:38:55.322 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.322 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.322 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.323 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.323 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: Adjusting osd_memory_target on smithi064 to 196.2M 2023-07-18T03:38:55.323 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: Unable to set osd_memory_target on smithi064 to 205776076: error parsing value: Value '205776076' is below minimum 939524096 2023-07-18T03:38:55.324 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:38:55.324 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:38:55.325 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:38:55.325 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:54 smithi134 ceph-mon[98216]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 900 KiB/s wr, 107 op/s 2023-07-18T03:38:55.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: Adjusting osd_memory_target on smithi040 to 197.2M 2023-07-18T03:38:55.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: Unable to set osd_memory_target on smithi040 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-07-18T03:38:55.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-07-18T03:38:55.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-18T03:38:55.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: Adjusting osd_memory_target on smithi064 to 196.2M 2023-07-18T03:38:55.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: Unable to set osd_memory_target on smithi064 to 205776076: error parsing value: Value '205776076' is below minimum 939524096 2023-07-18T03:38:55.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:38:55.334 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:38:55.334 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:38:55.334 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:54 smithi040 ceph-mon[96439]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 900 KiB/s wr, 107 op/s 2023-07-18T03:38:55.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: Adjusting osd_memory_target on smithi040 to 197.2M 2023-07-18T03:38:55.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: Unable to set osd_memory_target on smithi040 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-07-18T03:38:55.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: Adjusting osd_memory_target on smithi134 to 196.2M 2023-07-18T03:38:55.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: Unable to set osd_memory_target on smithi134 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-07-18T03:38:55.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-18T03:38:55.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: Adjusting osd_memory_target on smithi064 to 196.2M 2023-07-18T03:38:55.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: Unable to set osd_memory_target on smithi064 to 205776076: error parsing value: Value '205776076' is below minimum 939524096 2023-07-18T03:38:55.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:38:55.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:38:55.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:38:55.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:54 smithi064 ceph-mon[98059]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 900 KiB/s wr, 107 op/s 2023-07-18T03:38:57.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:57 smithi134 ceph-mon[98216]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 867 KiB/s wr, 129 op/s 2023-07-18T03:38:57.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:57 smithi040 ceph-mon[96439]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 867 KiB/s wr, 129 op/s 2023-07-18T03:38:57.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:57 smithi064 ceph-mon[98059]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 867 KiB/s wr, 129 op/s 2023-07-18T03:38:59.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:38:59 smithi134 ceph-mon[98216]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 195 op/s 2023-07-18T03:38:59.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:38:59 smithi040 ceph-mon[96439]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 195 op/s 2023-07-18T03:38:59.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:38:59 smithi064 ceph-mon[98059]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 195 op/s 2023-07-18T03:39:01.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:01 smithi134 ceph-mon[98216]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 923 KiB/s wr, 181 op/s 2023-07-18T03:39:01.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:01 smithi040 ceph-mon[96439]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 923 KiB/s wr, 181 op/s 2023-07-18T03:39:01.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:01 smithi064 ceph-mon[98059]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 923 KiB/s wr, 181 op/s 2023-07-18T03:39:03.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:03 smithi134 ceph-mon[98216]: pgmap v539: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 934 KiB/s wr, 249 op/s 2023-07-18T03:39:03.566 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:03 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:03.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:03 smithi040 ceph-mon[96439]: pgmap v539: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 934 KiB/s wr, 249 op/s 2023-07-18T03:39:03.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:03 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:03.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:03 smithi064 ceph-mon[98059]: pgmap v539: 97 pgs: 97 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 934 KiB/s wr, 249 op/s 2023-07-18T03:39:03.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:03 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:05.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:05 smithi134 ceph-mon[98216]: pgmap v540: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 222 op/s 2023-07-18T03:39:05.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:05 smithi040 ceph-mon[96439]: pgmap v540: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 222 op/s 2023-07-18T03:39:05.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:05 smithi064 ceph-mon[98059]: pgmap v540: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 222 op/s 2023-07-18T03:39:07.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:07 smithi134 ceph-mon[98216]: pgmap v541: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 840 KiB/s wr, 223 op/s 2023-07-18T03:39:07.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:07 smithi040 ceph-mon[96439]: pgmap v541: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 840 KiB/s wr, 223 op/s 2023-07-18T03:39:07.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:07 smithi064 ceph-mon[98059]: pgmap v541: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 840 KiB/s wr, 223 op/s 2023-07-18T03:39:08.278 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.280+0000 7f66c292f700 1 -- 172.21.15.40:0/297393206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc102f40 msgr2=0x7f66bc105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.280+0000 7f66c292f700 1 --2- 172.21.15.40:0/297393206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc102f40 0x7f66bc105320 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009990 tx=0x7f66b0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.280+0000 7f66c292f700 1 -- 172.21.15.40:0/297393206 shutdown_connections 2023-07-18T03:39:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.280+0000 7f66c292f700 1 --2- 172.21.15.40:0/297393206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc105860 0x7f66bc107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.280+0000 7f66c292f700 1 --2- 172.21.15.40:0/297393206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66bc100620 0x7f66bc102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.280+0000 7f66c292f700 1 --2- 172.21.15.40:0/297393206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc102f40 0x7f66bc105320 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.280+0000 7f66c292f700 1 -- 172.21.15.40:0/297393206 >> 172.21.15.40:0/297393206 conn(0x7f66bc0fa000 msgr2=0x7f66bc0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.280+0000 7f66c292f700 1 -- 172.21.15.40:0/297393206 shutdown_connections 2023-07-18T03:39:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.280+0000 7f66c292f700 1 -- 172.21.15.40:0/297393206 wait complete. 2023-07-18T03:39:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66c292f700 1 Processor -- start 2023-07-18T03:39:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66c292f700 1 -- start start 2023-07-18T03:39:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66c292f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66bc100620 0x7f66bc195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66c292f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc102f40 0x7f66bc195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66c292f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc105860 0x7f66bc196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66c292f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc197360 con 0x7f66bc105860 2023-07-18T03:39:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66c292f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc1974d0 con 0x7f66bc102f40 2023-07-18T03:39:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66c292f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc197690 con 0x7f66bc100620 2023-07-18T03:39:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66bbfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66bc100620 0x7f66bc195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66bb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc102f40 0x7f66bc195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66bbfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66bc100620 0x7f66bc195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:49980/0 (socket says 172.21.15.40:49980) 2023-07-18T03:39:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66c0ecc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc105860 0x7f66bc196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66bbfff700 1 -- 172.21.15.40:0/2127862120 learned_addr learned my addr 172.21.15.40:0/2127862120 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66bb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc102f40 0x7f66bc195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:44280/0 (socket says 172.21.15.40:44280) 2023-07-18T03:39:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66bbfff700 1 -- 172.21.15.40:0/2127862120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc102f40 msgr2=0x7f66bc195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66bbfff700 1 --2- 172.21.15.40:0/2127862120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc102f40 0x7f66bc195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66bbfff700 1 -- 172.21.15.40:0/2127862120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc105860 msgr2=0x7f66bc196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66bbfff700 1 --2- 172.21.15.40:0/2127862120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc105860 0x7f66bc196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66bbfff700 1 -- 172.21.15.40:0/2127862120 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac009520 con 0x7f66bc100620 2023-07-18T03:39:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.281+0000 7f66bb7fe700 1 --2- 172.21.15.40:0/2127862120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc102f40 0x7f66bc195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.282+0000 7f66bbfff700 1 --2- 172.21.15.40:0/2127862120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66bc100620 0x7f66bc195940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00ea30 tx=0x7f66ac00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.282+0000 7f66b97fa700 1 -- 172.21.15.40:0/2127862120 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac00cbd0 con 0x7f66bc100620 2023-07-18T03:39:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.282+0000 7f66c292f700 1 -- 172.21.15.40:0/2127862120 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b00095f0 con 0x7f66bc100620 2023-07-18T03:39:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.282+0000 7f66b97fa700 1 -- 172.21.15.40:0/2127862120 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66ac00eed0 con 0x7f66bc100620 2023-07-18T03:39:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.282+0000 7f66b97fa700 1 -- 172.21.15.40:0/2127862120 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac003d20 con 0x7f66bc100620 2023-07-18T03:39:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.282+0000 7f66c292f700 1 -- 172.21.15.40:0/2127862120 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66bc19df50 con 0x7f66bc100620 2023-07-18T03:39:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.284+0000 7f66b97fa700 1 -- 172.21.15.40:0/2127862120 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f66ac003ee0 con 0x7f66bc100620 2023-07-18T03:39:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.285+0000 7f66b97fa700 1 --2- 172.21.15.40:0/2127862120 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f66a40a1150 0x7f66a40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.285+0000 7f66bb7fe700 1 --2- 172.21.15.40:0/2127862120 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f66a40a1150 0x7f66a40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.285+0000 7f66b97fa700 1 -- 172.21.15.40:0/2127862120 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f66ac014070 con 0x7f66bc100620 2023-07-18T03:39:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.285+0000 7f66bb7fe700 1 --2- 172.21.15.40:0/2127862120 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f66a40a1150 0x7f66a40a3600 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f66b000b680 tx=0x7f66b0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.286+0000 7f66ceab5b80 1 Processor -- start 2023-07-18T03:39:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66ceab5b80 1 -- start start 2023-07-18T03:39:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66ceab5b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bfd766b0 0x55c1bff1e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66ceab5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bff1ec50 0x55c1bff1f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66ceab5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bff1faf0 0x55c1bff1ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66ceab5b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1bfe1fc50 con 0x55c1bfd766b0 2023-07-18T03:39:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66ceab5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1bfe729e0 con 0x55c1bff1ec50 2023-07-18T03:39:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66ceab5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1bfe6be70 con 0x55c1bff1faf0 2023-07-18T03:39:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66bbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bfd766b0 0x55c1bff1e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66bbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bfd766b0 0x55c1bff1e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60960/0 (socket says 172.21.15.40:60960) 2023-07-18T03:39:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66bbfff700 1 -- 172.21.15.40:0/2360033347 learned_addr learned my addr 172.21.15.40:0/2360033347 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66bb7fe700 1 --2- 172.21.15.40:0/2360033347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bff1ec50 0x55c1bff1f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66c0ecc700 1 --2- 172.21.15.40:0/2360033347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bff1faf0 0x55c1bff1ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66bbfff700 1 -- 172.21.15.40:0/2360033347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bff1faf0 msgr2=0x55c1bff1ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66bbfff700 1 --2- 172.21.15.40:0/2360033347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bff1faf0 0x55c1bff1ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66bbfff700 1 -- 172.21.15.40:0/2360033347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bff1ec50 msgr2=0x55c1bff1f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66bbfff700 1 --2- 172.21.15.40:0/2360033347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bff1ec50 0x55c1bff1f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66bbfff700 1 -- 172.21.15.40:0/2360033347 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1bfc99d80 con 0x55c1bfd766b0 2023-07-18T03:39:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66bbfff700 1 --2- 172.21.15.40:0/2360033347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bfd766b0 0x55c1bff1e710 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0a1f70 tx=0x7f66ac0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c511fa752a79445 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:08.302 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.287+0000 7f66a27fc700 1 -- 172.21.15.40:0/2360033347 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0a24a0 con 0x55c1bfd766b0 2023-07-18T03:39:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.288+0000 7f66a27fc700 1 -- 172.21.15.40:0/2360033347 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66ac0983f0 con 0x55c1bfd766b0 2023-07-18T03:39:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.288+0000 7f66a27fc700 1 -- 172.21.15.40:0/2360033347 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac09a510 con 0x55c1bfd766b0 2023-07-18T03:39:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.288+0000 7f66ceab5b80 1 -- 172.21.15.40:0/2360033347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bfd766b0 msgr2=0x55c1bff1e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.288+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/2360033347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bfd766b0 0x55c1bff1e710 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0a1f70 tx=0x7f66ac0045e0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.288+0000 7f66ceab5b80 1 -- 172.21.15.40:0/2360033347 shutdown_connections 2023-07-18T03:39:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.288+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/2360033347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bff1faf0 0x55c1bff1ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.288+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/2360033347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bfd766b0 0x55c1bff1e710 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.288+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/2360033347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bff1ec50 0x55c1bff1f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.288+0000 7f66ceab5b80 1 -- 172.21.15.40:0/2360033347 >> 172.21.15.40:0/2360033347 conn(0x55c1bff0fbe0 msgr2=0x55c1bff14cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.288+0000 7f66ceab5b80 1 -- 172.21.15.40:0/2360033347 shutdown_connections 2023-07-18T03:39:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.288+0000 7f66ceab5b80 1 -- 172.21.15.40:0/2360033347 wait complete. 2023-07-18T03:39:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.289+0000 7f66ceab5b80 1 -- start start 2023-07-18T03:39:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.289+0000 7f66ceab5b80 1 Processor -- start 2023-07-18T03:39:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.289+0000 7f66ceab5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bfd766b0 0x55c1bff27c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.289+0000 7f66ceab5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bff1ec50 0x55c1bff281a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.289+0000 7f66ceab5b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bff1faf0 0x55c1bff2f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.289+0000 7f66ceab5b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1bfe0ba60 con 0x55c1bff1faf0 2023-07-18T03:39:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.289+0000 7f66ceab5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1bfe0b760 con 0x55c1bff1ec50 2023-07-18T03:39:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.289+0000 7f66ceab5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1bfcef8c0 con 0x55c1bfd766b0 2023-07-18T03:39:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.289+0000 7f66ceab5b80 10 client.0 _ll_register_callbacks cb 0x55c1bfe09700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:39:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.289+0000 7f66c0ecc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bff1faf0 0x55c1bff2f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.289+0000 7f66bbfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bfd766b0 0x55c1bff27c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.289+0000 7f66c0ecc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bff1faf0 0x55c1bff2f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60970/0 (socket says 172.21.15.40:60970) 2023-07-18T03:39:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.289+0000 7f66c0ecc700 1 -- 172.21.15.40:0/612423523 learned_addr learned my addr 172.21.15.40:0/612423523 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.289+0000 7f66bb7fe700 1 --2- 172.21.15.40:0/612423523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bff1ec50 0x55c1bff281a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f66bbfff700 1 -- 172.21.15.40:0/612423523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bff1ec50 msgr2=0x55c1bff281a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f66bbfff700 1 --2- 172.21.15.40:0/612423523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bff1ec50 0x55c1bff281a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f66bbfff700 1 -- 172.21.15.40:0/612423523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bff1faf0 msgr2=0x55c1bff2f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f66bbfff700 1 --2- 172.21.15.40:0/612423523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bff1faf0 0x55c1bff2f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f66bbfff700 1 -- 172.21.15.40:0/612423523 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66bc19df50 con 0x55c1bfd766b0 2023-07-18T03:39:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f66c0ecc700 1 --2- 172.21.15.40:0/612423523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bff1faf0 0x55c1bff2f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f66bbfff700 1 --2- 172.21.15.40:0/612423523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bfd766b0 0x55c1bff27c60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00c5b0 tx=0x7f66ac0964c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b34b789b728b764 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f669b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:39:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f669b7fe700 1 -- 172.21.15.40:0/612423523 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac09a4b0 con 0x55c1bfd766b0 2023-07-18T03:39:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f669b7fe700 1 -- 172.21.15.40:0/612423523 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f66ac0a24a0 con 0x55c1bfd766b0 2023-07-18T03:39:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f66ceab5b80 1 -- 172.21.15.40:0/612423523 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1bfdc5490 con 0x55c1bfd766b0 2023-07-18T03:39:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f669b7fe700 1 -- 172.21.15.40:0/612423523 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66ac0ca250 con 0x55c1bfd766b0 2023-07-18T03:39:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f66ceab5b80 10 client.26308 fetch_fsmap learned FSMap version 58 2023-07-18T03:39:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f66ceab5b80 1 -- 172.21.15.40:0/612423523 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55c1bfc82c00 con 0x55c1bfd766b0 2023-07-18T03:39:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f669b7fe700 1 -- 172.21.15.40:0/612423523 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66ac07e080 con 0x55c1bfd766b0 2023-07-18T03:39:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f669b7fe700 1 -- 172.21.15.40:0/612423523 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 58) v1 ==== 7530+0+0 (secure 0 0 0) 0x7f66ac0b80d0 con 0x55c1bfd766b0 2023-07-18T03:39:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f66ceab5b80 10 client.26308 fetch_fsmap finished waiting for FSMap version 58 2023-07-18T03:39:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f66ceab5b80 10 client.26308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:39:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f66ceab5b80 20 client.26308 populate_metadata read hostname 'smithi040' 2023-07-18T03:39:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/612423523 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c1bff33680 0x55c1bff35a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f66ceab5b80 4 client.26308 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:39:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.290+0000 7f66ceab5b80 1 -- 172.21.15.40:0/612423523 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1bfe64e50 con 0x55c1bff33680 2023-07-18T03:39:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.291+0000 7f66bb7fe700 1 --2- 172.21.15.40:0/612423523 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c1bff33680 0x55c1bff35a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.291+0000 7f669b7fe700 10 client.26308 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.291+0000 7f66bb7fe700 1 --2- 172.21.15.40:0/612423523 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c1bff33680 0x55c1bff35a90 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=590f5ee21c00b6ed server_cookie=e5a082ed0368dd2f in_seq=0 out_seq=0 2023-07-18T03:39:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.292+0000 7f669b7fe700 1 -- 172.21.15.40:0/612423523 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c1bfe64e50 con 0x55c1bff33680 2023-07-18T03:39:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.292+0000 7f669b7fe700 10 client.26308 handle_command_reply: tid=0 2023-07-18T03:39:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.292+0000 7f66ceab5b80 1 client.26308 shutdown 2023-07-18T03:39:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.292+0000 7f66ceab5b80 1 -- 172.21.15.40:0/612423523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bfd766b0 msgr2=0x55c1bff27c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.292+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/612423523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bfd766b0 0x55c1bff27c60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00c5b0 tx=0x7f66ac0964c0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.292+0000 7f66ceab5b80 1 -- 172.21.15.40:0/612423523 shutdown_connections 2023-07-18T03:39:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.292+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/612423523 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c1bff33680 0x55c1bff35a90 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.292+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/612423523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bff1faf0 0x55c1bff2f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.292+0000 7f669b7fe700 0 client.26308 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.292+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/612423523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bfd766b0 0x55c1bff27c60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.292+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/612423523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bff1ec50 0x55c1bff281a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.292+0000 7f66ceab5b80 1 -- 172.21.15.40:0/612423523 >> 172.21.15.40:0/612423523 conn(0x55c1bff0fbe0 msgr2=0x55c1bff144e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.292+0000 7f66ceab5b80 1 -- 172.21.15.40:0/612423523 shutdown_connections 2023-07-18T03:39:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.292+0000 7f66ceab5b80 1 -- 172.21.15.40:0/612423523 wait complete. 2023-07-18T03:39:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.292+0000 7f66ceab5b80 20 client.26308 trim_cache size 0 max 16384 2023-07-18T03:39:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.307+0000 7f66ceab5b80 1 Processor -- start 2023-07-18T03:39:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66ceab5b80 1 -- start start 2023-07-18T03:39:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66ceab5b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bfd766b0 0x55c1bff22a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66ceab5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bff1ec50 0x55c1bff2f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66ceab5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bff1faf0 0x55c1bff22f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66ceab5b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1bff27830 con 0x55c1bfd766b0 2023-07-18T03:39:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66ceab5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1bfee0060 con 0x55c1bff1ec50 2023-07-18T03:39:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66ceab5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1bff159d0 con 0x55c1bff1faf0 2023-07-18T03:39:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66bbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bfd766b0 0x55c1bff22a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66bb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bff1ec50 0x55c1bff2f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66bbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bfd766b0 0x55c1bff22a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60974/0 (socket says 172.21.15.40:60974) 2023-07-18T03:39:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66bbfff700 1 -- 172.21.15.40:0/34782264 learned_addr learned my addr 172.21.15.40:0/34782264 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66c0ecc700 1 --2- 172.21.15.40:0/34782264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bff1faf0 0x55c1bff22f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66bbfff700 1 -- 172.21.15.40:0/34782264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bff1faf0 msgr2=0x55c1bff22f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66bbfff700 1 --2- 172.21.15.40:0/34782264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bff1faf0 0x55c1bff22f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66bbfff700 1 -- 172.21.15.40:0/34782264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bff1ec50 msgr2=0x55c1bff2f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66bbfff700 1 --2- 172.21.15.40:0/34782264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bff1ec50 0x55c1bff2f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66bbfff700 1 -- 172.21.15.40:0/34782264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1bfcc01f0 con 0x55c1bfd766b0 2023-07-18T03:39:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66bb7fe700 1 --2- 172.21.15.40:0/34782264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bff1ec50 0x55c1bff2f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66c0ecc700 1 --2- 172.21.15.40:0/34782264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bff1faf0 0x55c1bff22f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.308+0000 7f66bbfff700 1 --2- 172.21.15.40:0/34782264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bfd766b0 0x55c1bff22a00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0983a0 tx=0x7f66ac09a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2b098f545894ec server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.309+0000 7f66a2ffd700 1 -- 172.21.15.40:0/34782264 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0a24a0 con 0x55c1bfd766b0 2023-07-18T03:39:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.309+0000 7f66a2ffd700 1 -- 172.21.15.40:0/34782264 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c1bfdc5490 con 0x55c1bfd766b0 2023-07-18T03:39:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.309+0000 7f66a2ffd700 1 -- 172.21.15.40:0/34782264 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac095e40 con 0x55c1bfd766b0 2023-07-18T03:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.309+0000 7f66ceab5b80 1 -- 172.21.15.40:0/34782264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bfd766b0 msgr2=0x55c1bff22a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.309+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/34782264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bfd766b0 0x55c1bff22a00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0983a0 tx=0x7f66ac09a1d0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.309+0000 7f66ceab5b80 1 -- 172.21.15.40:0/34782264 shutdown_connections 2023-07-18T03:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.309+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/34782264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bff1faf0 0x55c1bff22f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.309+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/34782264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bfd766b0 0x55c1bff22a00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.309+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/34782264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bff1ec50 0x55c1bff2f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.309+0000 7f66ceab5b80 1 -- 172.21.15.40:0/34782264 >> 172.21.15.40:0/34782264 conn(0x55c1bff0fbe0 msgr2=0x55c1bff25ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.309+0000 7f66ceab5b80 1 -- 172.21.15.40:0/34782264 shutdown_connections 2023-07-18T03:39:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.309+0000 7f66ceab5b80 1 -- 172.21.15.40:0/34782264 wait complete. 2023-07-18T03:39:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.310+0000 7f66ceab5b80 1 -- start start 2023-07-18T03:39:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.310+0000 7f66ceab5b80 1 Processor -- start 2023-07-18T03:39:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.310+0000 7f66ceab5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bfd766b0 0x55c1bff2ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.310+0000 7f66ceab5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bff1ec50 0x55c1bff2b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.310+0000 7f66ceab5b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bff1faf0 0x55c1bff2e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.310+0000 7f66ceab5b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1bff2e600 con 0x55c1bff1faf0 2023-07-18T03:39:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.310+0000 7f66ceab5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1bff2e770 con 0x55c1bfd766b0 2023-07-18T03:39:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.310+0000 7f66ceab5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1bff2e930 con 0x55c1bff1ec50 2023-07-18T03:39:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.310+0000 7f66ceab5b80 10 client.0 _ll_register_callbacks cb 0x7f66b000bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:39:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.310+0000 7f66c0ecc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bff1faf0 0x55c1bff2e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.310+0000 7f66bbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bfd766b0 0x55c1bff2ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.310+0000 7f66c0ecc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bff1faf0 0x55c1bff2e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60984/0 (socket says 172.21.15.40:60984) 2023-07-18T03:39:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.310+0000 7f66c0ecc700 1 -- 172.21.15.40:0/2235073899 learned_addr learned my addr 172.21.15.40:0/2235073899 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.310+0000 7f66bb7fe700 1 --2- 172.21.15.40:0/2235073899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bff1ec50 0x55c1bff2b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.311+0000 7f66bbfff700 1 -- 172.21.15.40:0/2235073899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bff1ec50 msgr2=0x55c1bff2b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.311+0000 7f66bbfff700 1 --2- 172.21.15.40:0/2235073899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bff1ec50 0x55c1bff2b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.311+0000 7f66bbfff700 1 -- 172.21.15.40:0/2235073899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bff1faf0 msgr2=0x55c1bff2e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.311+0000 7f66bbfff700 1 --2- 172.21.15.40:0/2235073899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bff1faf0 0x55c1bff2e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.311+0000 7f66bbfff700 1 -- 172.21.15.40:0/2235073899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66bc108860 con 0x55c1bfd766b0 2023-07-18T03:39:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.311+0000 7f66c0ecc700 1 --2- 172.21.15.40:0/2235073899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bff1faf0 0x55c1bff2e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.311+0000 7f66bbfff700 1 --2- 172.21.15.40:0/2235073899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bfd766b0 0x55c1bff2ab60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f66ac099fe0 tx=0x7f66ac0964c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d42f57c88f9e6398 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.311+0000 7f669b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:39:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.311+0000 7f669b7fe700 1 -- 172.21.15.40:0/2235073899 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0a24a0 con 0x55c1bfd766b0 2023-07-18T03:39:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.311+0000 7f66ceab5b80 1 -- 172.21.15.40:0/2235073899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66ac0983f0 con 0x55c1bfd766b0 2023-07-18T03:39:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.311+0000 7f669b7fe700 1 -- 172.21.15.40:0/2235073899 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f66ac07f060 con 0x55c1bfd766b0 2023-07-18T03:39:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.311+0000 7f669b7fe700 1 -- 172.21.15.40:0/2235073899 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66ac0ca250 con 0x55c1bfd766b0 2023-07-18T03:39:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.311+0000 7f66ceab5b80 10 client.26300 fetch_fsmap learned FSMap version 58 2023-07-18T03:39:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.311+0000 7f66ceab5b80 1 -- 172.21.15.40:0/2235073899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x7f66b4005e70 con 0x55c1bfd766b0 2023-07-18T03:39:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.312+0000 7f669b7fe700 1 -- 172.21.15.40:0/2235073899 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66ac0983f0 con 0x55c1bfd766b0 2023-07-18T03:39:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.312+0000 7f669b7fe700 1 -- 172.21.15.40:0/2235073899 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 58) v1 ==== 7530+0+0 (secure 0 0 0) 0x7f66ac083460 con 0x55c1bfd766b0 2023-07-18T03:39:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.312+0000 7f66ceab5b80 10 client.26300 fetch_fsmap finished waiting for FSMap version 58 2023-07-18T03:39:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.312+0000 7f66ceab5b80 10 client.26300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:39:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.312+0000 7f66ceab5b80 20 client.26300 populate_metadata read hostname 'smithi040' 2023-07-18T03:39:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.312+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/2235073899 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c1bff87cf0 0x55c1bff8a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.312+0000 7f66ceab5b80 4 client.26300 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:39:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.312+0000 7f66ceab5b80 1 -- 172.21.15.40:0/2235073899 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c1bfe26ff0 con 0x55c1bff87cf0 2023-07-18T03:39:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.312+0000 7f66bb7fe700 1 --2- 172.21.15.40:0/2235073899 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c1bff87cf0 0x55c1bff8a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.312+0000 7f66bb7fe700 1 --2- 172.21.15.40:0/2235073899 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c1bff87cf0 0x55c1bff8a100 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87c29ecbd586601f server_cookie=35e551546ce87a82 in_seq=0 out_seq=0 2023-07-18T03:39:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.312+0000 7f669b7fe700 10 client.26300 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.770+0000 7f669b7fe700 1 -- 172.21.15.40:0/2235073899 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c1bfe26ff0 con 0x55c1bff87cf0 2023-07-18T03:39:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.770+0000 7f669b7fe700 10 client.26300 handle_command_reply: tid=0 2023-07-18T03:39:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.770+0000 7f66ceab5b80 1 client.26300 shutdown 2023-07-18T03:39:08.767 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.771+0000 7f66ceab5b80 1 -- 172.21.15.40:0/2235073899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bfd766b0 msgr2=0x55c1bff2ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.771+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/2235073899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bfd766b0 0x55c1bff2ab60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f66ac099fe0 tx=0x7f66ac0964c0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.771+0000 7f66ceab5b80 1 -- 172.21.15.40:0/2235073899 shutdown_connections 2023-07-18T03:39:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.771+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/2235073899 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c1bff87cf0 0x55c1bff8a100 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.771+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/2235073899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1bff1faf0 0x55c1bff2e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.771+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/2235073899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1bfd766b0 0x55c1bff2ab60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.771+0000 7f66ceab5b80 1 --2- 172.21.15.40:0/2235073899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1bff1ec50 0x55c1bff2b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.771+0000 7f66ceab5b80 1 -- 172.21.15.40:0/2235073899 >> 172.21.15.40:0/2235073899 conn(0x55c1bff0fbe0 msgr2=0x55c1bff25ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.771+0000 7f669b7fe700 0 client.26300 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.771+0000 7f66ceab5b80 1 -- 172.21.15.40:0/2235073899 shutdown_connections 2023-07-18T03:39:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.771+0000 7f66ceab5b80 1 -- 172.21.15.40:0/2235073899 wait complete. 2023-07-18T03:39:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.771+0000 7f66ceab5b80 20 client.26300 trim_cache size 0 max 16384 2023-07-18T03:39:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:39:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:39:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:39:08.770 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:39:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.773+0000 7f66c292f700 1 -- 172.21.15.40:0/2127862120 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f66a40a1150 msgr2=0x7f66a40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.773+0000 7f66c292f700 1 --2- 172.21.15.40:0/2127862120 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f66a40a1150 0x7f66a40a3600 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f66b000b680 tx=0x7f66b0005fd0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.773+0000 7f66c292f700 1 -- 172.21.15.40:0/2127862120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66bc100620 msgr2=0x7f66bc195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.773+0000 7f66c292f700 1 --2- 172.21.15.40:0/2127862120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66bc100620 0x7f66bc195940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00ea30 tx=0x7f66ac00ed40 comp rx=0 tx=0).stop 2023-07-18T03:39:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.773+0000 7f66c292f700 1 -- 172.21.15.40:0/2127862120 shutdown_connections 2023-07-18T03:39:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.773+0000 7f66c292f700 1 --2- 172.21.15.40:0/2127862120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc105860 0x7f66bc196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.773+0000 7f66c292f700 1 --2- 172.21.15.40:0/2127862120 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f66a40a1150 0x7f66a40a3600 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.773+0000 7f66c292f700 1 --2- 172.21.15.40:0/2127862120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66bc100620 0x7f66bc195940 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.773+0000 7f66c292f700 1 --2- 172.21.15.40:0/2127862120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66bc102f40 0x7f66bc195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.773+0000 7f66c292f700 1 -- 172.21.15.40:0/2127862120 >> 172.21.15.40:0/2127862120 conn(0x7f66bc0fa000 msgr2=0x7f66bc101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.773+0000 7f66c292f700 1 -- 172.21.15.40:0/2127862120 shutdown_connections 2023-07-18T03:39:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:08.773+0000 7f66c292f700 1 -- 172.21.15.40:0/2127862120 wait complete. 2023-07-18T03:39:09.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:09 smithi134 ceph-mon[98216]: pgmap v542: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 293 op/s 2023-07-18T03:39:09.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:09 smithi040 ceph-mon[96439]: pgmap v542: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 293 op/s 2023-07-18T03:39:09.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:09 smithi064 ceph-mon[98059]: pgmap v542: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 293 op/s 2023-07-18T03:39:11.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:11 smithi134 ceph-mon[98216]: pgmap v543: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 218 op/s 2023-07-18T03:39:11.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:11 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:11.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:11 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:39:11.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:11 smithi040 ceph-mon[96439]: pgmap v543: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 218 op/s 2023-07-18T03:39:11.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:11 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:11.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:11 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:39:11.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:11 smithi064 ceph-mon[98059]: pgmap v543: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 218 op/s 2023-07-18T03:39:11.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:11 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:11.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:11 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:39:13.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:12 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:39:13.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:12 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:39:13.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:12 smithi134 ceph-mon[98216]: pgmap v544: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 493 op/s 2023-07-18T03:39:13.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:12 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:39:13.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:12 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:39:13.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:12 smithi040 ceph-mon[96439]: pgmap v544: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 493 op/s 2023-07-18T03:39:13.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:12 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:39:13.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:12 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:39:13.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:12 smithi064 ceph-mon[98059]: pgmap v544: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 493 op/s 2023-07-18T03:39:15.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:15 smithi134 ceph-mon[98216]: pgmap v545: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 528 op/s 2023-07-18T03:39:15.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:15 smithi040 ceph-mon[96439]: pgmap v545: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 528 op/s 2023-07-18T03:39:15.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:15 smithi064 ceph-mon[98059]: pgmap v545: 97 pgs: 97 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 528 op/s 2023-07-18T03:39:16.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:16 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:16.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:16 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:16.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:16 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:17.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:17 smithi134 ceph-mon[98216]: pgmap v546: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 529 op/s 2023-07-18T03:39:17.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:17 smithi040 ceph-mon[96439]: pgmap v546: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 529 op/s 2023-07-18T03:39:17.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:17 smithi064 ceph-mon[98059]: pgmap v546: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 529 op/s 2023-07-18T03:39:19.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:19 smithi134 ceph-mon[98216]: pgmap v547: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 605 op/s 2023-07-18T03:39:19.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:19 smithi040 ceph-mon[96439]: pgmap v547: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 605 op/s 2023-07-18T03:39:19.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:19 smithi064 ceph-mon[98059]: pgmap v547: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 605 op/s 2023-07-18T03:39:21.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:21 smithi134 ceph-mon[98216]: pgmap v548: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 1.1 MiB/s wr, 509 op/s 2023-07-18T03:39:21.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:21 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:21.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:21 smithi040 ceph-mon[96439]: pgmap v548: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 1.1 MiB/s wr, 509 op/s 2023-07-18T03:39:21.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:21 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:21.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:21 smithi064 ceph-mon[98059]: pgmap v548: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 1.1 MiB/s wr, 509 op/s 2023-07-18T03:39:21.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:21 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.888+0000 7f3fb9fe8700 1 -- 172.21.15.40:0/229643507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fb40ffc90 msgr2=0x7f3fb4100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.888+0000 7f3fb9fe8700 1 --2- 172.21.15.40:0/229643507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fb40ffc90 0x7f3fb4100100 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009990 tx=0x7f3f9c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.888+0000 7f3fb9fe8700 1 -- 172.21.15.40:0/229643507 shutdown_connections 2023-07-18T03:39:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.888+0000 7f3fb9fe8700 1 --2- 172.21.15.40:0/229643507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fb4100640 0x7f3fb4106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.888+0000 7f3fb9fe8700 1 --2- 172.21.15.40:0/229643507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fb40fea00 0x7f3fb40fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.888+0000 7f3fb9fe8700 1 --2- 172.21.15.40:0/229643507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fb40ffc90 0x7f3fb4100100 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.888+0000 7f3fb9fe8700 1 -- 172.21.15.40:0/229643507 >> 172.21.15.40:0/229643507 conn(0x7f3fb40fa000 msgr2=0x7f3fb40fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.888+0000 7f3fb9fe8700 1 -- 172.21.15.40:0/229643507 shutdown_connections 2023-07-18T03:39:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.888+0000 7f3fb9fe8700 1 -- 172.21.15.40:0/229643507 wait complete. 2023-07-18T03:39:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.889+0000 7f3fb9fe8700 1 Processor -- start 2023-07-18T03:39:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.889+0000 7f3fb9fe8700 1 -- start start 2023-07-18T03:39:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.889+0000 7f3fb9fe8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fb40fea00 0x7f3fb4194870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.889+0000 7f3fb9fe8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fb40ffc90 0x7f3fb4194db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.889+0000 7f3fb9fe8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fb4100640 0x7f3fb4195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.889+0000 7f3fb9fe8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb4196290 con 0x7f3fb4100640 2023-07-18T03:39:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.889+0000 7f3fb9fe8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb4196400 con 0x7f3fb40fea00 2023-07-18T03:39:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.889+0000 7f3fb9fe8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb41965c0 con 0x7f3fb40ffc90 2023-07-18T03:39:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.889+0000 7f3fb3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fb4100640 0x7f3fb4195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.889+0000 7f3fb37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fb40fea00 0x7f3fb4194870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.889+0000 7f3fb2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fb40ffc90 0x7f3fb4194db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.889+0000 7f3fb2ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fb40ffc90 0x7f3fb4194db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50024/0 (socket says 172.21.15.40:50024) 2023-07-18T03:39:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.889+0000 7f3fb37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fb40fea00 0x7f3fb4194870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:44320/0 (socket says 172.21.15.40:44320) 2023-07-18T03:39:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.889+0000 7f3fb3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fb4100640 0x7f3fb4195c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60998/0 (socket says 172.21.15.40:60998) 2023-07-18T03:39:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.889+0000 7f3fb2ffd700 1 -- 172.21.15.40:0/684415408 learned_addr learned my addr 172.21.15.40:0/684415408 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.890+0000 7f3fb2ffd700 1 -- 172.21.15.40:0/684415408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fb40fea00 msgr2=0x7f3fb4194870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.890+0000 7f3fb2ffd700 1 --2- 172.21.15.40:0/684415408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fb40fea00 0x7f3fb4194870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.890+0000 7f3fb2ffd700 1 -- 172.21.15.40:0/684415408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fb4100640 msgr2=0x7f3fb4195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.890+0000 7f3fb2ffd700 1 --2- 172.21.15.40:0/684415408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fb4100640 0x7f3fb4195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.890+0000 7f3fb2ffd700 1 -- 172.21.15.40:0/684415408 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9c0095f0 con 0x7f3fb40ffc90 2023-07-18T03:39:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.890+0000 7f3fb37fe700 1 --2- 172.21.15.40:0/684415408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fb40fea00 0x7f3fb4194870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.890+0000 7f3fb3fff700 1 --2- 172.21.15.40:0/684415408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fb4100640 0x7f3fb4195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.890+0000 7f3fb2ffd700 1 --2- 172.21.15.40:0/684415408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fb40ffc90 0x7f3fb4194db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0059d0 tx=0x7f3f9c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.890+0000 7f3fb0ff9700 1 -- 172.21.15.40:0/684415408 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c01d070 con 0x7f3fb40ffc90 2023-07-18T03:39:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.890+0000 7f3fb9fe8700 1 -- 172.21.15.40:0/684415408 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb4196840 con 0x7f3fb40ffc90 2023-07-18T03:39:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.890+0000 7f3fb0ff9700 1 -- 172.21.15.40:0/684415408 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f9c00fb90 con 0x7f3fb40ffc90 2023-07-18T03:39:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.890+0000 7f3fb0ff9700 1 -- 172.21.15.40:0/684415408 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c0173f0 con 0x7f3fb40ffc90 2023-07-18T03:39:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.890+0000 7f3fb9fe8700 1 -- 172.21.15.40:0/684415408 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb419d060 con 0x7f3fb40ffc90 2023-07-18T03:39:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.892+0000 7f3fb0ff9700 1 -- 172.21.15.40:0/684415408 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3f9c00fd00 con 0x7f3fb40ffc90 2023-07-18T03:39:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.893+0000 7f3fb0ff9700 1 --2- 172.21.15.40:0/684415408 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3fa00a1150 0x7f3fa00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.893+0000 7f3fb0ff9700 1 -- 172.21.15.40:0/684415408 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3f9c0cdd00 con 0x7f3fb40ffc90 2023-07-18T03:39:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.893+0000 7f3fb37fe700 1 --2- 172.21.15.40:0/684415408 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3fa00a1150 0x7f3fa00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.893+0000 7f3fb37fe700 1 --2- 172.21.15.40:0/684415408 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3fa00a1150 0x7f3fa00a3600 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4011490 tx=0x7f3fa4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fc616eb80 1 Processor -- start 2023-07-18T03:39:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fc616eb80 1 -- start start 2023-07-18T03:39:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fc616eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7ce956b0 0x55fd7d03b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fc616eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7d03ba50 0x55fd7d03c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fc616eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7d03c8f0 0x55fd7d03cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fc616eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7cf3ec50 con 0x55fd7ce956b0 2023-07-18T03:39:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fc616eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7cf919e0 con 0x55fd7d03c8f0 2023-07-18T03:39:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fc616eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7cf8ae70 con 0x55fd7d03ba50 2023-07-18T03:39:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fb37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7ce956b0 0x55fd7d03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fb37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7ce956b0 0x55fd7d03b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32770/0 (socket says 172.21.15.40:32770) 2023-07-18T03:39:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fb37fe700 1 -- 172.21.15.40:0/3425963312 learned_addr learned my addr 172.21.15.40:0/3425963312 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fb2ffd700 1 --2- 172.21.15.40:0/3425963312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7d03ba50 0x55fd7d03c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fb3fff700 1 --2- 172.21.15.40:0/3425963312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7d03c8f0 0x55fd7d03cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fb37fe700 1 -- 172.21.15.40:0/3425963312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7d03ba50 msgr2=0x55fd7d03c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fb37fe700 1 --2- 172.21.15.40:0/3425963312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7d03ba50 0x55fd7d03c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fb37fe700 1 -- 172.21.15.40:0/3425963312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7d03c8f0 msgr2=0x55fd7d03cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fb37fe700 1 --2- 172.21.15.40:0/3425963312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7d03c8f0 0x55fd7d03cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fb37fe700 1 -- 172.21.15.40:0/3425963312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd7cdb8d80 con 0x55fd7ce956b0 2023-07-18T03:39:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.894+0000 7f3fb37fe700 1 --2- 172.21.15.40:0/3425963312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7ce956b0 0x55fd7d03b510 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40043c0 tx=0x7f3fa4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c65dcee9581f1ef server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.895+0000 7f3fa9ffb700 1 -- 172.21.15.40:0/3425963312 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4003d10 con 0x55fd7ce956b0 2023-07-18T03:39:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.895+0000 7f3fa9ffb700 1 -- 172.21.15.40:0/3425963312 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fa4003e70 con 0x55fd7ce956b0 2023-07-18T03:39:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.895+0000 7f3fa9ffb700 1 -- 172.21.15.40:0/3425963312 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa400e7a0 con 0x55fd7ce956b0 2023-07-18T03:39:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.895+0000 7f3fc616eb80 1 -- 172.21.15.40:0/3425963312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7ce956b0 msgr2=0x55fd7d03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.895+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/3425963312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7ce956b0 0x55fd7d03b510 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40043c0 tx=0x7f3fa4014000 comp rx=0 tx=0).stop 2023-07-18T03:39:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.895+0000 7f3fc616eb80 1 -- 172.21.15.40:0/3425963312 shutdown_connections 2023-07-18T03:39:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.895+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/3425963312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7d03c8f0 0x55fd7d03cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.895+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/3425963312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7ce956b0 0x55fd7d03b510 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.895+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/3425963312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7d03ba50 0x55fd7d03c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.895+0000 7f3fc616eb80 1 -- 172.21.15.40:0/3425963312 >> 172.21.15.40:0/3425963312 conn(0x55fd7d02ebe0 msgr2=0x55fd7d033cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.895+0000 7f3fc616eb80 1 -- 172.21.15.40:0/3425963312 shutdown_connections 2023-07-18T03:39:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.895+0000 7f3fc616eb80 1 -- 172.21.15.40:0/3425963312 wait complete. 2023-07-18T03:39:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.896+0000 7f3fc616eb80 1 -- start start 2023-07-18T03:39:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.896+0000 7f3fc616eb80 1 Processor -- start 2023-07-18T03:39:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.896+0000 7f3fc616eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7ce956b0 0x55fd7d044a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.896+0000 7f3fc616eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7d03ba50 0x55fd7d044fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.896+0000 7f3fc616eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7d03c8f0 0x55fd7d04bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.896+0000 7f3fc616eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7cf2aa60 con 0x55fd7d03ba50 2023-07-18T03:39:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.896+0000 7f3fc616eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7cf2a760 con 0x55fd7d03c8f0 2023-07-18T03:39:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.896+0000 7f3fc616eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7ce0e8c0 con 0x55fd7ce956b0 2023-07-18T03:39:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.896+0000 7f3fc616eb80 10 client.0 _ll_register_callbacks cb 0x55fd7cf28700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:39:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.896+0000 7f3fb2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7d03ba50 0x55fd7d044fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.896+0000 7f3fb37fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7ce956b0 0x55fd7d044a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.896+0000 7f3fb2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7d03ba50 0x55fd7d044fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32778/0 (socket says 172.21.15.40:32778) 2023-07-18T03:39:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.896+0000 7f3fb2ffd700 1 -- 172.21.15.40:0/4086416341 learned_addr learned my addr 172.21.15.40:0/4086416341 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.896+0000 7f3fb3fff700 1 --2- 172.21.15.40:0/4086416341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7d03c8f0 0x55fd7d04bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3fb2ffd700 1 -- 172.21.15.40:0/4086416341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7ce956b0 msgr2=0x55fd7d044a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3fb2ffd700 1 --2- 172.21.15.40:0/4086416341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7ce956b0 0x55fd7d044a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3fb2ffd700 1 -- 172.21.15.40:0/4086416341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7d03c8f0 msgr2=0x55fd7d04bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3fb2ffd700 1 --2- 172.21.15.40:0/4086416341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7d03c8f0 0x55fd7d04bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3fb2ffd700 1 -- 172.21.15.40:0/4086416341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa4017070 con 0x55fd7d03ba50 2023-07-18T03:39:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3fb37fe700 1 --2- 172.21.15.40:0/4086416341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7ce956b0 0x55fd7d044a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3fb2ffd700 1 --2- 172.21.15.40:0/4086416341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7d03ba50 0x55fd7d044fb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0cb7e0 tx=0x7f3f9c09b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33832c8e83b884d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3f92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:39:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3f92ffd700 1 -- 172.21.15.40:0/4086416341 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c0a32b0 con 0x55fd7d03ba50 2023-07-18T03:39:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3f92ffd700 1 -- 172.21.15.40:0/4086416341 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3f9c09f1b0 con 0x55fd7d03ba50 2023-07-18T03:39:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3fc616eb80 1 -- 172.21.15.40:0/4086416341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd7cee4490 con 0x55fd7d03ba50 2023-07-18T03:39:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3f92ffd700 1 -- 172.21.15.40:0/4086416341 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f9c0a3540 con 0x55fd7d03ba50 2023-07-18T03:39:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3fc616eb80 10 client.16656 fetch_fsmap learned FSMap version 61 2023-07-18T03:39:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3fc616eb80 1 -- 172.21.15.40:0/4086416341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55fd7cda1c00 con 0x55fd7d03ba50 2023-07-18T03:39:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3f92ffd700 1 -- 172.21.15.40:0/4086416341 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f9c0c9080 con 0x55fd7d03ba50 2023-07-18T03:39:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3f92ffd700 1 -- 172.21.15.40:0/4086416341 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 61) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f3f9c09a170 con 0x55fd7d03ba50 2023-07-18T03:39:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3fc616eb80 10 client.16656 fetch_fsmap finished waiting for FSMap version 61 2023-07-18T03:39:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.897+0000 7f3fc616eb80 10 client.16656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:39:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.898+0000 7f3fc616eb80 20 client.16656 populate_metadata read hostname 'smithi040' 2023-07-18T03:39:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.898+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/4086416341 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fd7d050560 0x55fd7d052970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.898+0000 7f3fc616eb80 4 client.16656 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:39:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.898+0000 7f3fc616eb80 1 -- 172.21.15.40:0/4086416341 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd7cf83e50 con 0x55fd7d050560 2023-07-18T03:39:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.898+0000 7f3fb37fe700 1 --2- 172.21.15.40:0/4086416341 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fd7d050560 0x55fd7d052970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.898+0000 7f3f92ffd700 10 client.16656 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.898+0000 7f3fb37fe700 1 --2- 172.21.15.40:0/4086416341 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fd7d050560 0x55fd7d052970 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0a0356069a3464b server_cookie=5edf3718a5b92024 in_seq=0 out_seq=0 2023-07-18T03:39:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.899+0000 7f3f92ffd700 1 -- 172.21.15.40:0/4086416341 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fd7cf83e50 con 0x55fd7d050560 2023-07-18T03:39:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.899+0000 7f3f92ffd700 10 client.16656 handle_command_reply: tid=0 2023-07-18T03:39:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.899+0000 7f3fc616eb80 1 client.16656 shutdown 2023-07-18T03:39:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.899+0000 7f3fc616eb80 1 -- 172.21.15.40:0/4086416341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7d03ba50 msgr2=0x55fd7d044fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.899+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/4086416341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7d03ba50 0x55fd7d044fb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0cb7e0 tx=0x7f3f9c09b6c0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.899+0000 7f3fc616eb80 1 -- 172.21.15.40:0/4086416341 shutdown_connections 2023-07-18T03:39:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.899+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/4086416341 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fd7d050560 0x55fd7d052970 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.899+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/4086416341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7d03c8f0 0x55fd7d04bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.899+0000 7f3f92ffd700 0 client.16656 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.899+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/4086416341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7ce956b0 0x55fd7d044a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.899+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/4086416341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7d03ba50 0x55fd7d044fb0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.899+0000 7f3fc616eb80 1 -- 172.21.15.40:0/4086416341 >> 172.21.15.40:0/4086416341 conn(0x55fd7d02ebe0 msgr2=0x55fd7d0334e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.899+0000 7f3fc616eb80 1 -- 172.21.15.40:0/4086416341 shutdown_connections 2023-07-18T03:39:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.899+0000 7f3fc616eb80 1 -- 172.21.15.40:0/4086416341 wait complete. 2023-07-18T03:39:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.900+0000 7f3fc616eb80 20 client.16656 trim_cache size 0 max 16384 2023-07-18T03:39:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.914+0000 7f3fc616eb80 1 Processor -- start 2023-07-18T03:39:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fc616eb80 1 -- start start 2023-07-18T03:39:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fc616eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7ce956b0 0x55fd7d03b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fc616eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7d03ba50 0x55fd7d04bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fc616eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7d03c8f0 0x55fd7d0572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fc616eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d045c60 con 0x55fd7ce956b0 2023-07-18T03:39:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fc616eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d050370 con 0x55fd7d03ba50 2023-07-18T03:39:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fc616eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d02bc00 con 0x55fd7d03c8f0 2023-07-18T03:39:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fb37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7ce956b0 0x55fd7d03b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fb37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7ce956b0 0x55fd7d03b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32784/0 (socket says 172.21.15.40:32784) 2023-07-18T03:39:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fb37fe700 1 -- 172.21.15.40:0/3549135984 learned_addr learned my addr 172.21.15.40:0/3549135984 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fb2ffd700 1 --2- 172.21.15.40:0/3549135984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7d03ba50 0x55fd7d04bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fb3fff700 1 --2- 172.21.15.40:0/3549135984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7d03c8f0 0x55fd7d0572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fb37fe700 1 -- 172.21.15.40:0/3549135984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7d03c8f0 msgr2=0x55fd7d0572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fb37fe700 1 --2- 172.21.15.40:0/3549135984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7d03c8f0 0x55fd7d0572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fb37fe700 1 -- 172.21.15.40:0/3549135984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7d03ba50 msgr2=0x55fd7d04bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fb37fe700 1 --2- 172.21.15.40:0/3549135984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7d03ba50 0x55fd7d04bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fb37fe700 1 -- 172.21.15.40:0/3549135984 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd7cddf1f0 con 0x55fd7ce956b0 2023-07-18T03:39:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fb3fff700 1 --2- 172.21.15.40:0/3549135984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7d03c8f0 0x55fd7d0572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3fb37fe700 1 --2- 172.21.15.40:0/3549135984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7ce956b0 0x55fd7d03b4d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40043c0 tx=0x7f3fa40058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61ea03950da851f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3faa7fc700 1 -- 172.21.15.40:0/3549135984 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa402c030 con 0x55fd7ce956b0 2023-07-18T03:39:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3faa7fc700 1 -- 172.21.15.40:0/3549135984 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fa402d040 con 0x55fd7ce956b0 2023-07-18T03:39:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.915+0000 7f3faa7fc700 1 -- 172.21.15.40:0/3549135984 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4004eb0 con 0x55fd7ce956b0 2023-07-18T03:39:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.916+0000 7f3fc616eb80 1 -- 172.21.15.40:0/3549135984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7ce956b0 msgr2=0x55fd7d03b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.916+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/3549135984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7ce956b0 0x55fd7d03b4d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40043c0 tx=0x7f3fa40058e0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.916+0000 7f3fc616eb80 1 -- 172.21.15.40:0/3549135984 shutdown_connections 2023-07-18T03:39:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.916+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/3549135984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7d03c8f0 0x55fd7d0572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.916+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/3549135984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7ce956b0 0x55fd7d03b4d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.916+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/3549135984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7d03ba50 0x55fd7d04bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.916+0000 7f3fc616eb80 1 -- 172.21.15.40:0/3549135984 >> 172.21.15.40:0/3549135984 conn(0x55fd7d02ebe0 msgr2=0x55fd7d030150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.916+0000 7f3fc616eb80 1 -- 172.21.15.40:0/3549135984 shutdown_connections 2023-07-18T03:39:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.916+0000 7f3fc616eb80 1 -- 172.21.15.40:0/3549135984 wait complete. 2023-07-18T03:39:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.916+0000 7f3fc616eb80 1 -- start start 2023-07-18T03:39:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fc616eb80 1 Processor -- start 2023-07-18T03:39:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fc616eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7ce956b0 0x55fd7d0477a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fc616eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7d03ba50 0x55fd7d047ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fc616eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7d03c8f0 0x55fd7d04ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fc616eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d04b240 con 0x55fd7d03ba50 2023-07-18T03:39:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fc616eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d04b3b0 con 0x55fd7ce956b0 2023-07-18T03:39:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fc616eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7d04b570 con 0x55fd7d03c8f0 2023-07-18T03:39:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fc616eb80 10 client.0 _ll_register_callbacks cb 0x7f3f9c09aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:39:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fb2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7d03ba50 0x55fd7d047ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fb3fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7d03c8f0 0x55fd7d04ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fb37fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7ce956b0 0x55fd7d0477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fb2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7d03ba50 0x55fd7d047ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32792/0 (socket says 172.21.15.40:32792) 2023-07-18T03:39:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fb2ffd700 1 -- 172.21.15.40:0/1871585738 learned_addr learned my addr 172.21.15.40:0/1871585738 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fb2ffd700 1 -- 172.21.15.40:0/1871585738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7d03c8f0 msgr2=0x55fd7d04ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fb2ffd700 1 --2- 172.21.15.40:0/1871585738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7d03c8f0 0x55fd7d04ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fb2ffd700 1 -- 172.21.15.40:0/1871585738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7ce956b0 msgr2=0x55fd7d0477a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fb2ffd700 1 --2- 172.21.15.40:0/1871585738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7ce956b0 0x55fd7d0477a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fb2ffd700 1 -- 172.21.15.40:0/1871585738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd7cdb8d80 con 0x55fd7d03ba50 2023-07-18T03:39:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3fb2ffd700 1 --2- 172.21.15.40:0/1871585738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7d03ba50 0x55fd7d047ce0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c09ae30 tx=0x7f3f9c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd384b2b00dc01b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.917+0000 7f3f92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:39:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.918+0000 7f3f92ffd700 1 -- 172.21.15.40:0/1871585738 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c09f1b0 con 0x55fd7d03ba50 2023-07-18T03:39:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.918+0000 7f3f92ffd700 1 -- 172.21.15.40:0/1871585738 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3f9c07d060 con 0x55fd7d03ba50 2023-07-18T03:39:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.918+0000 7f3fc616eb80 1 -- 172.21.15.40:0/1871585738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fa4003e70 con 0x55fd7d03ba50 2023-07-18T03:39:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.918+0000 7f3f92ffd700 1 -- 172.21.15.40:0/1871585738 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f9c0a3540 con 0x55fd7d03ba50 2023-07-18T03:39:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.918+0000 7f3fc616eb80 10 client.16662 fetch_fsmap learned FSMap version 61 2023-07-18T03:39:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.918+0000 7f3fc616eb80 1 -- 172.21.15.40:0/1871585738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55fd7cdb7250 con 0x55fd7d03ba50 2023-07-18T03:39:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.918+0000 7f3f92ffd700 1 -- 172.21.15.40:0/1871585738 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fa4003e70 con 0x55fd7d03ba50 2023-07-18T03:39:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.918+0000 7f3f92ffd700 1 -- 172.21.15.40:0/1871585738 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 61) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f3f9c0b9270 con 0x55fd7d03ba50 2023-07-18T03:39:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.918+0000 7f3fc616eb80 10 client.16662 fetch_fsmap finished waiting for FSMap version 61 2023-07-18T03:39:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.918+0000 7f3fc616eb80 10 client.16662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:39:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.918+0000 7f3fc616eb80 20 client.16662 populate_metadata read hostname 'smithi040' 2023-07-18T03:39:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.918+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/1871585738 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fd7d050560 0x55fd7d0a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.918+0000 7f3fc616eb80 4 client.16662 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:39:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.918+0000 7f3fc616eb80 1 -- 172.21.15.40:0/1871585738 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fd7cf45ff0 con 0x55fd7d050560 2023-07-18T03:39:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.918+0000 7f3fb37fe700 1 --2- 172.21.15.40:0/1871585738 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fd7d050560 0x55fd7d0a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.919+0000 7f3f92ffd700 10 client.16662 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.919+0000 7f3fb37fe700 1 --2- 172.21.15.40:0/1871585738 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fd7d050560 0x55fd7d0a6b30 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fb302d004304669 server_cookie=f038666dccbbae57 in_seq=0 out_seq=0 2023-07-18T03:39:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.931+0000 7f3f92ffd700 1 -- 172.21.15.40:0/1871585738 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fd7cf45ff0 con 0x55fd7d050560 2023-07-18T03:39:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.931+0000 7f3f92ffd700 10 client.16662 handle_command_reply: tid=0 2023-07-18T03:39:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.931+0000 7f3fc616eb80 1 client.16662 shutdown 2023-07-18T03:39:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.931+0000 7f3fc616eb80 1 -- 172.21.15.40:0/1871585738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7d03ba50 msgr2=0x55fd7d047ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.931+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/1871585738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7d03ba50 0x55fd7d047ce0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c09ae30 tx=0x7f3f9c005ef0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.931+0000 7f3fc616eb80 1 -- 172.21.15.40:0/1871585738 shutdown_connections 2023-07-18T03:39:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.931+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/1871585738 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fd7d050560 0x55fd7d0a6b30 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.931+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/1871585738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd7d03c8f0 0x55fd7d04ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.931+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/1871585738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd7ce956b0 0x55fd7d0477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.931+0000 7f3f92ffd700 0 client.16662 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.931+0000 7f3fc616eb80 1 --2- 172.21.15.40:0/1871585738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd7d03ba50 0x55fd7d047ce0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.931+0000 7f3fc616eb80 1 -- 172.21.15.40:0/1871585738 >> 172.21.15.40:0/1871585738 conn(0x55fd7d02ebe0 msgr2=0x55fd7d0572d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.931+0000 7f3fc616eb80 1 -- 172.21.15.40:0/1871585738 shutdown_connections 2023-07-18T03:39:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.931+0000 7f3fc616eb80 1 -- 172.21.15.40:0/1871585738 wait complete. 2023-07-18T03:39:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.931+0000 7f3fc616eb80 20 client.16662 trim_cache size 0 max 16384 2023-07-18T03:39:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:39:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:39:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:39:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:39:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.932+0000 7f3fb9fe8700 1 -- 172.21.15.40:0/684415408 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3fa00a1150 msgr2=0x7f3fa00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.932+0000 7f3fb9fe8700 1 --2- 172.21.15.40:0/684415408 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3fa00a1150 0x7f3fa00a3600 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4011490 tx=0x7f3fa4009260 comp rx=0 tx=0).stop 2023-07-18T03:39:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.932+0000 7f3fb9fe8700 1 -- 172.21.15.40:0/684415408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fb40ffc90 msgr2=0x7f3fb4194db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.932+0000 7f3fb9fe8700 1 --2- 172.21.15.40:0/684415408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fb40ffc90 0x7f3fb4194db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0059d0 tx=0x7f3f9c004730 comp rx=0 tx=0).stop 2023-07-18T03:39:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.932+0000 7f3fb9fe8700 1 -- 172.21.15.40:0/684415408 shutdown_connections 2023-07-18T03:39:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.932+0000 7f3fb9fe8700 1 --2- 172.21.15.40:0/684415408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fb4100640 0x7f3fb4195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.932+0000 7f3fb9fe8700 1 --2- 172.21.15.40:0/684415408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3fb40fea00 0x7f3fb4194870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.932+0000 7f3fb9fe8700 1 --2- 172.21.15.40:0/684415408 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3fa00a1150 0x7f3fa00a3600 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.932+0000 7f3fb9fe8700 1 --2- 172.21.15.40:0/684415408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fb40ffc90 0x7f3fb4194db0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.932+0000 7f3fb9fe8700 1 -- 172.21.15.40:0/684415408 >> 172.21.15.40:0/684415408 conn(0x7f3fb40fa000 msgr2=0x7f3fb4103170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.932+0000 7f3fb9fe8700 1 -- 172.21.15.40:0/684415408 shutdown_connections 2023-07-18T03:39:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:21.932+0000 7f3fb9fe8700 1 -- 172.21.15.40:0/684415408 wait complete. 2023-07-18T03:39:23.133 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:39:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.270+0000 7f36489b3700 1 -- 172.21.15.40:0/4245420879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440fe9e0 msgr2=0x7f36440fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.270+0000 7f36489b3700 1 --2- 172.21.15.40:0/4245420879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440fe9e0 0x7f36440fedf0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3634009990 tx=0x7f3634009ca0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.270+0000 7f36489b3700 1 -- 172.21.15.40:0/4245420879 shutdown_connections 2023-07-18T03:39:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.270+0000 7f36489b3700 1 --2- 172.21.15.40:0/4245420879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3644100560 0x7f3644106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.270+0000 7f36489b3700 1 --2- 172.21.15.40:0/4245420879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440fe9e0 0x7f36440fedf0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.270+0000 7f36489b3700 1 --2- 172.21.15.40:0/4245420879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36440ffbd0 0x7f3644100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.270+0000 7f36489b3700 1 -- 172.21.15.40:0/4245420879 >> 172.21.15.40:0/4245420879 conn(0x7f36440fa000 msgr2=0x7f36440fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.270+0000 7f36489b3700 1 -- 172.21.15.40:0/4245420879 shutdown_connections 2023-07-18T03:39:23.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.270+0000 7f36489b3700 1 -- 172.21.15.40:0/4245420879 wait complete. 2023-07-18T03:39:23.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.270+0000 7f36489b3700 1 Processor -- start 2023-07-18T03:39:23.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f36489b3700 1 -- start start 2023-07-18T03:39:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f36489b3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36440fe9e0 0x7f3644197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f36489b3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440ffbd0 0x7f3644197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f36489b3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3644100560 0x7f364419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f36489b3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364419f5e0 con 0x7f36440ffbd0 2023-07-18T03:39:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f36489b3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364419f720 con 0x7f3644100560 2023-07-18T03:39:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f36489b3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364419f8b0 con 0x7f36440fe9e0 2023-07-18T03:39:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f3641d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440ffbd0 0x7f3644197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f3642d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3644100560 0x7f364419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f364259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36440fe9e0 0x7f3644197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f364259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36440fe9e0 0x7f3644197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50064/0 (socket says 172.21.15.40:50064) 2023-07-18T03:39:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f3642d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3644100560 0x7f364419f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:44366/0 (socket says 172.21.15.40:44366) 2023-07-18T03:39:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f364259c700 1 -- 172.21.15.40:0/3722325493 learned_addr learned my addr 172.21.15.40:0/3722325493 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f364259c700 1 -- 172.21.15.40:0/3722325493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3644100560 msgr2=0x7f364419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f364259c700 1 --2- 172.21.15.40:0/3722325493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3644100560 0x7f364419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f364259c700 1 -- 172.21.15.40:0/3722325493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440ffbd0 msgr2=0x7f3644197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f364259c700 1 --2- 172.21.15.40:0/3722325493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440ffbd0 0x7f3644197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f364259c700 1 -- 172.21.15.40:0/3722325493 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36340095f0 con 0x7f36440fe9e0 2023-07-18T03:39:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f3641d9b700 1 --2- 172.21.15.40:0/3722325493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440ffbd0 0x7f3644197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:23.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.271+0000 7f364259c700 1 --2- 172.21.15.40:0/3722325493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36440fe9e0 0x7f3644197a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f363400b620 tx=0x7f3634004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:23.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.272+0000 7f36337fe700 1 -- 172.21.15.40:0/3722325493 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363401d070 con 0x7f36440fe9e0 2023-07-18T03:39:23.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.272+0000 7f36489b3700 1 -- 172.21.15.40:0/3722325493 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364419fb30 con 0x7f36440fe9e0 2023-07-18T03:39:23.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.272+0000 7f36337fe700 1 -- 172.21.15.40:0/3722325493 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3634005030 con 0x7f36440fe9e0 2023-07-18T03:39:23.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.272+0000 7f36337fe700 1 -- 172.21.15.40:0/3722325493 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634004b80 con 0x7f36440fe9e0 2023-07-18T03:39:23.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.272+0000 7f36489b3700 1 -- 172.21.15.40:0/3722325493 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364419ff50 con 0x7f36440fe9e0 2023-07-18T03:39:23.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.274+0000 7f36337fe700 1 -- 172.21.15.40:0/3722325493 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f36340051a0 con 0x7f36440fe9e0 2023-07-18T03:39:23.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.274+0000 7f36337fe700 1 --2- 172.21.15.40:0/3722325493 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f362c0a1150 0x7f362c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.274+0000 7f3641d9b700 1 --2- 172.21.15.40:0/3722325493 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f362c0a1150 0x7f362c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.274+0000 7f36337fe700 1 -- 172.21.15.40:0/3722325493 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f36340cd910 con 0x7f36440fe9e0 2023-07-18T03:39:23.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.275+0000 7f3641d9b700 1 --2- 172.21.15.40:0/3722325493 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f362c0a1150 0x7f362c0a3600 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3638011490 tx=0x7f3638009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:23.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.275+0000 7f3654b39b80 1 Processor -- start 2023-07-18T03:39:23.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.275+0000 7f3654b39b80 1 -- start start 2023-07-18T03:39:23.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.275+0000 7f3654b39b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82b4d0 0x55f1fc82b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.275+0000 7f3654b39b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82be20 0x55f1fc82c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.275+0000 7f3654b39b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82cc70 0x55f1fc82d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.275+0000 7f3654b39b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc730cb0 con 0x55f1fc82b4d0 2023-07-18T03:39:23.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.275+0000 7f3654b39b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc783a40 con 0x55f1fc82cc70 2023-07-18T03:39:23.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.275+0000 7f3654b39b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc77ced0 con 0x55f1fc82be20 2023-07-18T03:39:23.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.276+0000 7f364259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82b4d0 0x55f1fc82b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.276+0000 7f364259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82b4d0 0x55f1fc82b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32812/0 (socket says 172.21.15.40:32812) 2023-07-18T03:39:23.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.276+0000 7f364259c700 1 -- 172.21.15.40:0/943921776 learned_addr learned my addr 172.21.15.40:0/943921776 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:23.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.276+0000 7f3641d9b700 1 --2- 172.21.15.40:0/943921776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82be20 0x55f1fc82c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.276+0000 7f3642d9d700 1 --2- 172.21.15.40:0/943921776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82cc70 0x55f1fc82d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.276+0000 7f364259c700 1 -- 172.21.15.40:0/943921776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82be20 msgr2=0x55f1fc82c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:23.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.276+0000 7f364259c700 1 --2- 172.21.15.40:0/943921776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82be20 0x55f1fc82c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.276+0000 7f364259c700 1 -- 172.21.15.40:0/943921776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82cc70 msgr2=0x55f1fc82d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:23.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.276+0000 7f364259c700 1 --2- 172.21.15.40:0/943921776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82cc70 0x55f1fc82d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.276+0000 7f364259c700 1 -- 172.21.15.40:0/943921776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1fc5aade0 con 0x55f1fc82b4d0 2023-07-18T03:39:23.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.276+0000 7f364259c700 1 --2- 172.21.15.40:0/943921776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82b4d0 0x55f1fc82b8e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f363409aa90 tx=0x7f3634005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1906b119ab8fe4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:23.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.276+0000 7f3630ff9700 1 -- 172.21.15.40:0/943921776 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363409ad90 con 0x55f1fc82b4d0 2023-07-18T03:39:23.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.276+0000 7f3630ff9700 1 -- 172.21.15.40:0/943921776 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f363409edc0 con 0x55f1fc82b4d0 2023-07-18T03:39:23.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.276+0000 7f3630ff9700 1 -- 172.21.15.40:0/943921776 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363409a4e0 con 0x55f1fc82b4d0 2023-07-18T03:39:23.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.277+0000 7f3654b39b80 1 -- 172.21.15.40:0/943921776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82b4d0 msgr2=0x55f1fc82b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.277+0000 7f3654b39b80 1 --2- 172.21.15.40:0/943921776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82b4d0 0x55f1fc82b8e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f363409aa90 tx=0x7f3634005ef0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.277+0000 7f3654b39b80 1 -- 172.21.15.40:0/943921776 shutdown_connections 2023-07-18T03:39:23.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.277+0000 7f3654b39b80 1 --2- 172.21.15.40:0/943921776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82cc70 0x55f1fc82d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.277+0000 7f3654b39b80 1 --2- 172.21.15.40:0/943921776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82b4d0 0x55f1fc82b8e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.277+0000 7f3654b39b80 1 --2- 172.21.15.40:0/943921776 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82be20 0x55f1fc82c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.277+0000 7f3654b39b80 1 -- 172.21.15.40:0/943921776 >> 172.21.15.40:0/943921776 conn(0x55f1fc820da0 msgr2=0x55f1fc8261d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:23.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.277+0000 7f3654b39b80 1 -- 172.21.15.40:0/943921776 shutdown_connections 2023-07-18T03:39:23.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.277+0000 7f3654b39b80 1 -- 172.21.15.40:0/943921776 wait complete. 2023-07-18T03:39:23.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.277+0000 7f3654b39b80 1 -- start start 2023-07-18T03:39:23.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f3654b39b80 1 Processor -- start 2023-07-18T03:39:23.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f3654b39b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82b4d0 0x55f1fc834e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f3654b39b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82be20 0x55f1fc8353a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f3654b39b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82cc70 0x55f1fc83c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f3654b39b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc71cac0 con 0x55f1fc82b4d0 2023-07-18T03:39:23.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f3654b39b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc71c7c0 con 0x55f1fc82be20 2023-07-18T03:39:23.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f3654b39b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc600920 con 0x55f1fc82cc70 2023-07-18T03:39:23.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f3654b39b80 10 client.0 _ll_register_callbacks cb 0x55f1fc71a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:39:23.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f364259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82b4d0 0x55f1fc834e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f364259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82b4d0 0x55f1fc834e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32818/0 (socket says 172.21.15.40:32818) 2023-07-18T03:39:23.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f364259c700 1 -- 172.21.15.40:0/269154349 learned_addr learned my addr 172.21.15.40:0/269154349 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:23.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f3641d9b700 1 --2- 172.21.15.40:0/269154349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82be20 0x55f1fc8353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f3642d9d700 1 --2- 172.21.15.40:0/269154349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82cc70 0x55f1fc83c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f364259c700 1 -- 172.21.15.40:0/269154349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82cc70 msgr2=0x55f1fc83c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:23.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f364259c700 1 --2- 172.21.15.40:0/269154349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82cc70 0x55f1fc83c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f364259c700 1 -- 172.21.15.40:0/269154349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82be20 msgr2=0x55f1fc8353a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f364259c700 1 --2- 172.21.15.40:0/269154349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82be20 0x55f1fc8353a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.278+0000 7f364259c700 1 -- 172.21.15.40:0/269154349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364419ff50 con 0x55f1fc82b4d0 2023-07-18T03:39:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.279+0000 7f364259c700 1 --2- 172.21.15.40:0/269154349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82b4d0 0x55f1fc834e60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f36340cdbf0 tx=0x7f3634005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e443d84ab353825 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.279+0000 7f3629ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:39:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.279+0000 7f3629ffb700 1 -- 172.21.15.40:0/269154349 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36340a3150 con 0x55f1fc82b4d0 2023-07-18T03:39:23.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.279+0000 7f3629ffb700 1 -- 172.21.15.40:0/269154349 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f36340b90d0 con 0x55f1fc82b4d0 2023-07-18T03:39:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.279+0000 7f3654b39b80 1 -- 172.21.15.40:0/269154349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1fc6d64f0 con 0x55f1fc82b4d0 2023-07-18T03:39:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.279+0000 7f3629ffb700 1 -- 172.21.15.40:0/269154349 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f363409ad90 con 0x55f1fc82b4d0 2023-07-18T03:39:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.279+0000 7f3654b39b80 10 client.16674 fetch_fsmap learned FSMap version 61 2023-07-18T03:39:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.279+0000 7f3654b39b80 1 -- 172.21.15.40:0/269154349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55f1fc593c60 con 0x55f1fc82b4d0 2023-07-18T03:39:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.279+0000 7f3629ffb700 1 -- 172.21.15.40:0/269154349 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f1fc6d64f0 con 0x55f1fc82b4d0 2023-07-18T03:39:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.279+0000 7f3629ffb700 1 -- 172.21.15.40:0/269154349 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 61) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f36340a2cc0 con 0x55f1fc82b4d0 2023-07-18T03:39:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.279+0000 7f3654b39b80 10 client.16674 fetch_fsmap finished waiting for FSMap version 61 2023-07-18T03:39:23.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.279+0000 7f3654b39b80 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:39:23.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.279+0000 7f3654b39b80 20 client.16674 populate_metadata read hostname 'smithi040' 2023-07-18T03:39:23.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.279+0000 7f3654b39b80 1 --2- 172.21.15.40:0/269154349 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55f1fc840860 0x55f1fc842c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.279+0000 7f3654b39b80 4 client.16674 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:39:23.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.279+0000 7f3654b39b80 1 -- 172.21.15.40:0/269154349 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1fc775eb0 con 0x55f1fc840860 2023-07-18T03:39:23.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.279+0000 7f3641d9b700 1 --2- 172.21.15.40:0/269154349 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55f1fc840860 0x55f1fc842c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.280+0000 7f3629ffb700 10 client.16674 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:23.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.280+0000 7f3641d9b700 1 --2- 172.21.15.40:0/269154349 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55f1fc840860 0x55f1fc842c70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bae28dfff429d8e3 server_cookie=c6b7f3be92ea4ce4 in_seq=0 out_seq=0 2023-07-18T03:39:23.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.281+0000 7f3629ffb700 1 -- 172.21.15.40:0/269154349 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f1fc775eb0 con 0x55f1fc840860 2023-07-18T03:39:23.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.281+0000 7f3629ffb700 10 client.16674 handle_command_reply: tid=0 2023-07-18T03:39:23.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.281+0000 7f3654b39b80 1 client.16674 shutdown 2023-07-18T03:39:23.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.281+0000 7f3654b39b80 1 -- 172.21.15.40:0/269154349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82b4d0 msgr2=0x55f1fc834e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.281+0000 7f3654b39b80 1 --2- 172.21.15.40:0/269154349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82b4d0 0x55f1fc834e60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f36340cdbf0 tx=0x7f3634005a90 comp rx=0 tx=0).stop 2023-07-18T03:39:23.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.281+0000 7f3654b39b80 1 -- 172.21.15.40:0/269154349 shutdown_connections 2023-07-18T03:39:23.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.281+0000 7f3654b39b80 1 --2- 172.21.15.40:0/269154349 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55f1fc840860 0x55f1fc842c70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.281+0000 7f3654b39b80 1 --2- 172.21.15.40:0/269154349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82cc70 0x55f1fc83c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.281+0000 7f3629ffb700 0 client.16674 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.281+0000 7f3654b39b80 1 --2- 172.21.15.40:0/269154349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82b4d0 0x55f1fc834e60 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.281+0000 7f3654b39b80 1 --2- 172.21.15.40:0/269154349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82be20 0x55f1fc8353a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.281+0000 7f3654b39b80 1 -- 172.21.15.40:0/269154349 >> 172.21.15.40:0/269154349 conn(0x55f1fc820da0 msgr2=0x55f1fc8258b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.281+0000 7f3654b39b80 1 -- 172.21.15.40:0/269154349 shutdown_connections 2023-07-18T03:39:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.281+0000 7f3654b39b80 1 -- 172.21.15.40:0/269154349 wait complete. 2023-07-18T03:39:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.281+0000 7f3654b39b80 20 client.16674 trim_cache size 0 max 16384 2023-07-18T03:39:23.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.298+0000 7f3654b39b80 1 Processor -- start 2023-07-18T03:39:23.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.298+0000 7f3654b39b80 1 -- start start 2023-07-18T03:39:23.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.298+0000 7f3654b39b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82b4d0 0x55f1fc8258b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.298+0000 7f3654b39b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82be20 0x55f1fc83c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.298+0000 7f3654b39b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82cc70 0x55f1fc833020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.298+0000 7f3654b39b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc8270f0 con 0x55f1fc82be20 2023-07-18T03:39:23.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.298+0000 7f3654b39b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc775eb0 con 0x55f1fc82cc70 2023-07-18T03:39:23.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.298+0000 7f3654b39b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc834310 con 0x55f1fc82b4d0 2023-07-18T03:39:23.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.298+0000 7f3641d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82be20 0x55f1fc83c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.298+0000 7f3641d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82be20 0x55f1fc83c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32828/0 (socket says 172.21.15.40:32828) 2023-07-18T03:39:23.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.298+0000 7f3641d9b700 1 -- 172.21.15.40:0/1805462728 learned_addr learned my addr 172.21.15.40:0/1805462728 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:23.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.298+0000 7f364259c700 1 --2- 172.21.15.40:0/1805462728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82b4d0 0x55f1fc8258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.299+0000 7f3642d9d700 1 --2- 172.21.15.40:0/1805462728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82cc70 0x55f1fc833020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.299+0000 7f3641d9b700 1 -- 172.21.15.40:0/1805462728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82b4d0 msgr2=0x55f1fc8258b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:23.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.299+0000 7f3641d9b700 1 --2- 172.21.15.40:0/1805462728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82b4d0 0x55f1fc8258b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.299+0000 7f3641d9b700 1 -- 172.21.15.40:0/1805462728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82cc70 msgr2=0x55f1fc833020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.299+0000 7f3641d9b700 1 --2- 172.21.15.40:0/1805462728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82cc70 0x55f1fc833020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.299+0000 7f3641d9b700 1 -- 172.21.15.40:0/1805462728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1fc5d1250 con 0x55f1fc82be20 2023-07-18T03:39:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.299+0000 7f364259c700 1 --2- 172.21.15.40:0/1805462728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82b4d0 0x55f1fc8258b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.299+0000 7f3641d9b700 1 --2- 172.21.15.40:0/1805462728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82be20 0x55f1fc83c370 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55f1fc843510 tx=0x7f3638014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439ab4c3b53adf4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.299+0000 7f36317fa700 1 -- 172.21.15.40:0/1805462728 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638024040 con 0x55f1fc82be20 2023-07-18T03:39:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.299+0000 7f36317fa700 1 -- 172.21.15.40:0/1805462728 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3638003e40 con 0x55f1fc82be20 2023-07-18T03:39:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.299+0000 7f36317fa700 1 -- 172.21.15.40:0/1805462728 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363801e7f0 con 0x55f1fc82be20 2023-07-18T03:39:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 1 -- 172.21.15.40:0/1805462728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82be20 msgr2=0x55f1fc83c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 1 --2- 172.21.15.40:0/1805462728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82be20 0x55f1fc83c370 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55f1fc843510 tx=0x7f3638014000 comp rx=0 tx=0).stop 2023-07-18T03:39:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 1 -- 172.21.15.40:0/1805462728 shutdown_connections 2023-07-18T03:39:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 1 --2- 172.21.15.40:0/1805462728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82cc70 0x55f1fc833020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 1 --2- 172.21.15.40:0/1805462728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82b4d0 0x55f1fc8258b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 1 --2- 172.21.15.40:0/1805462728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82be20 0x55f1fc83c370 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 1 -- 172.21.15.40:0/1805462728 >> 172.21.15.40:0/1805462728 conn(0x55f1fc820da0 msgr2=0x55f1fc833b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:23.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 1 -- 172.21.15.40:0/1805462728 shutdown_connections 2023-07-18T03:39:23.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 1 -- 172.21.15.40:0/1805462728 wait complete. 2023-07-18T03:39:23.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 1 -- start start 2023-07-18T03:39:23.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 1 Processor -- start 2023-07-18T03:39:23.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82b4d0 0x55f1fc83c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82be20 0x55f1fc838d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82cc70 0x55f1fc839280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc8398a0 con 0x55f1fc82cc70 2023-07-18T03:39:23.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc8946d0 con 0x55f1fc82be20 2023-07-18T03:39:23.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc894810 con 0x55f1fc82b4d0 2023-07-18T03:39:23.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3654b39b80 10 client.0 _ll_register_callbacks cb 0x7f363c009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:39:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.300+0000 7f3642d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82cc70 0x55f1fc839280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f364259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82b4d0 0x55f1fc83c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f3642d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82cc70 0x55f1fc839280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32836/0 (socket says 172.21.15.40:32836) 2023-07-18T03:39:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f3642d9d700 1 -- 172.21.15.40:0/2222103715 learned_addr learned my addr 172.21.15.40:0/2222103715 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f3641d9b700 1 --2- 172.21.15.40:0/2222103715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82be20 0x55f1fc838d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f3642d9d700 1 -- 172.21.15.40:0/2222103715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82b4d0 msgr2=0x55f1fc83c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f3642d9d700 1 --2- 172.21.15.40:0/2222103715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82b4d0 0x55f1fc83c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f3642d9d700 1 -- 172.21.15.40:0/2222103715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82be20 msgr2=0x55f1fc838d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:23.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f3642d9d700 1 --2- 172.21.15.40:0/2222103715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82be20 0x55f1fc838d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f3642d9d700 1 -- 172.21.15.40:0/2222103715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1fc593c60 con 0x55f1fc82cc70 2023-07-18T03:39:23.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f364259c700 1 --2- 172.21.15.40:0/2222103715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82b4d0 0x55f1fc83c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:23.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f3641d9b700 1 --2- 172.21.15.40:0/2222103715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82be20 0x55f1fc838d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:23.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f3642d9d700 1 --2- 172.21.15.40:0/2222103715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82cc70 0x55f1fc839280 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f363c007c60 tx=0x7f363c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1dc67afb302860 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:23.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f3629ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:39:23.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f3629ffb700 1 -- 172.21.15.40:0/2222103715 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c01c070 con 0x55f1fc82cc70 2023-07-18T03:39:23.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f3629ffb700 1 -- 172.21.15.40:0/2222103715 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f363c011bf0 con 0x55f1fc82cc70 2023-07-18T03:39:23.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f3654b39b80 1 -- 172.21.15.40:0/2222103715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f363409edc0 con 0x55f1fc82cc70 2023-07-18T03:39:23.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f3629ffb700 1 -- 172.21.15.40:0/2222103715 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f363c00e040 con 0x55f1fc82cc70 2023-07-18T03:39:23.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f3654b39b80 10 client.16680 fetch_fsmap learned FSMap version 61 2023-07-18T03:39:23.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.301+0000 7f3654b39b80 1 -- 172.21.15.40:0/2222103715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f3638017070 con 0x55f1fc82cc70 2023-07-18T03:39:23.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.302+0000 7f3629ffb700 1 -- 172.21.15.40:0/2222103715 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f363c02f080 con 0x55f1fc82cc70 2023-07-18T03:39:23.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.302+0000 7f3629ffb700 1 -- 172.21.15.40:0/2222103715 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 61) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f363c029030 con 0x55f1fc82cc70 2023-07-18T03:39:23.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.302+0000 7f3654b39b80 10 client.16680 fetch_fsmap finished waiting for FSMap version 61 2023-07-18T03:39:23.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.302+0000 7f3654b39b80 10 client.16680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:39:23.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.302+0000 7f3654b39b80 20 client.16680 populate_metadata read hostname 'smithi040' 2023-07-18T03:39:23.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.302+0000 7f3654b39b80 1 --2- 172.21.15.40:0/2222103715 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55f1fc896350 0x55f1fc898760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:23.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.302+0000 7f3654b39b80 4 client.16680 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:39:23.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.302+0000 7f3654b39b80 1 -- 172.21.15.40:0/2222103715 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f1fc738050 con 0x55f1fc896350 2023-07-18T03:39:23.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.302+0000 7f364259c700 1 --2- 172.21.15.40:0/2222103715 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55f1fc896350 0x55f1fc898760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:23.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.302+0000 7f3629ffb700 10 client.16680 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:23.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.302+0000 7f364259c700 1 --2- 172.21.15.40:0/2222103715 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55f1fc896350 0x55f1fc898760 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=534b0ce97b4e9408 server_cookie=709c98deaf7fc792 in_seq=0 out_seq=0 2023-07-18T03:39:23.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f3629ffb700 1 -- 172.21.15.40:0/2222103715 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f1fc738050 con 0x55f1fc896350 2023-07-18T03:39:23.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f3629ffb700 10 client.16680 handle_command_reply: tid=0 2023-07-18T03:39:23.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f3654b39b80 1 client.16680 shutdown 2023-07-18T03:39:23.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f3654b39b80 1 -- 172.21.15.40:0/2222103715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82cc70 msgr2=0x55f1fc839280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f3654b39b80 1 --2- 172.21.15.40:0/2222103715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82cc70 0x55f1fc839280 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f363c007c60 tx=0x7f363c007f70 comp rx=0 tx=0).stop 2023-07-18T03:39:23.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f3654b39b80 1 -- 172.21.15.40:0/2222103715 shutdown_connections 2023-07-18T03:39:23.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f3654b39b80 1 --2- 172.21.15.40:0/2222103715 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55f1fc896350 0x55f1fc898760 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f3654b39b80 1 --2- 172.21.15.40:0/2222103715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f1fc82cc70 0x55f1fc839280 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f3629ffb700 0 client.16680 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:23.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f3654b39b80 1 --2- 172.21.15.40:0/2222103715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1fc82b4d0 0x55f1fc83c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f3654b39b80 1 --2- 172.21.15.40:0/2222103715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1fc82be20 0x55f1fc838d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f3654b39b80 1 -- 172.21.15.40:0/2222103715 >> 172.21.15.40:0/2222103715 conn(0x55f1fc820da0 msgr2=0x55f1fc8258b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:23.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f3654b39b80 1 -- 172.21.15.40:0/2222103715 shutdown_connections 2023-07-18T03:39:23.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f3654b39b80 1 -- 172.21.15.40:0/2222103715 wait complete. 2023-07-18T03:39:23.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f3654b39b80 20 client.16680 trim_cache size 0 max 16384 2023-07-18T03:39:23.309 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:39:23.309 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-18T03:39:23.309 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:39:23.309 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:39:23.310 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:39:23.310 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:39:23.310 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:39:23.310 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:39:23.310 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:39:23.310 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:39:23.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f36489b3700 1 -- 172.21.15.40:0/3722325493 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f362c0a1150 msgr2=0x7f362c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f36489b3700 1 --2- 172.21.15.40:0/3722325493 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f362c0a1150 0x7f362c0a3600 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3638011490 tx=0x7f3638009260 comp rx=0 tx=0).stop 2023-07-18T03:39:23.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f36489b3700 1 -- 172.21.15.40:0/3722325493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36440fe9e0 msgr2=0x7f3644197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:23.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.303+0000 7f36489b3700 1 --2- 172.21.15.40:0/3722325493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36440fe9e0 0x7f3644197a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f363400b620 tx=0x7f3634004730 comp rx=0 tx=0).stop 2023-07-18T03:39:23.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.304+0000 7f36489b3700 1 -- 172.21.15.40:0/3722325493 shutdown_connections 2023-07-18T03:39:23.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.304+0000 7f36489b3700 1 --2- 172.21.15.40:0/3722325493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3644100560 0x7f364419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.304+0000 7f36489b3700 1 --2- 172.21.15.40:0/3722325493 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f362c0a1150 0x7f362c0a3600 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.304+0000 7f36489b3700 1 --2- 172.21.15.40:0/3722325493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36440fe9e0 0x7f3644197a60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.304+0000 7f36489b3700 1 --2- 172.21.15.40:0/3722325493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440ffbd0 0x7f3644197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:23.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.304+0000 7f36489b3700 1 -- 172.21.15.40:0/3722325493 >> 172.21.15.40:0/3722325493 conn(0x7f36440fa000 msgr2=0x7f3644067690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:23.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.304+0000 7f36489b3700 1 -- 172.21.15.40:0/3722325493 shutdown_connections 2023-07-18T03:39:23.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:23.304+0000 7f36489b3700 1 -- 172.21.15.40:0/3722325493 wait complete. 2023-07-18T03:39:23.313 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:22 smithi040 ceph-mon[96439]: pgmap v549: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 577 op/s 2023-07-18T03:39:23.314 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:39:23.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:22 smithi134 ceph-mon[98216]: pgmap v549: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 577 op/s 2023-07-18T03:39:23.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:22 smithi064 ceph-mon[98059]: pgmap v549: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 577 op/s 2023-07-18T03:39:25.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:25 smithi134 ceph-mon[98216]: pgmap v550: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 332 op/s 2023-07-18T03:39:25.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:25 smithi040 ceph-mon[96439]: pgmap v550: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 332 op/s 2023-07-18T03:39:25.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:25 smithi064 ceph-mon[98059]: pgmap v550: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 332 op/s 2023-07-18T03:39:27.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:27 smithi134 ceph-mon[98216]: pgmap v551: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 257 op/s 2023-07-18T03:39:27.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:27 smithi040 ceph-mon[96439]: pgmap v551: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 257 op/s 2023-07-18T03:39:27.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:27 smithi064 ceph-mon[98059]: pgmap v551: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 257 op/s 2023-07-18T03:39:29.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:29 smithi134 ceph-mon[98216]: pgmap v552: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 291 op/s 2023-07-18T03:39:29.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:29 smithi040 ceph-mon[96439]: pgmap v552: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 291 op/s 2023-07-18T03:39:29.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:29 smithi064 ceph-mon[98059]: pgmap v552: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 291 op/s 2023-07-18T03:39:31.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:31 smithi134 ceph-mon[98216]: pgmap v553: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 747 KiB/s wr, 191 op/s 2023-07-18T03:39:31.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:31 smithi040 ceph-mon[96439]: pgmap v553: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 747 KiB/s wr, 191 op/s 2023-07-18T03:39:31.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:31 smithi064 ceph-mon[98059]: pgmap v553: 97 pgs: 97 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 747 KiB/s wr, 191 op/s 2023-07-18T03:39:33.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:32 smithi134 ceph-mon[98216]: pgmap v554: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 277 op/s 2023-07-18T03:39:33.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:32 smithi040 ceph-mon[96439]: pgmap v554: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 277 op/s 2023-07-18T03:39:33.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:32 smithi064 ceph-mon[98059]: pgmap v554: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 277 op/s 2023-07-18T03:39:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.044+0000 7f80f796e700 1 -- 172.21.15.40:0/754993995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80f00feab0 msgr2=0x7f80f00feec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.044+0000 7f80f796e700 1 --2- 172.21.15.40:0/754993995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80f00feab0 0x7f80f00feec0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f80e000b0a0 tx=0x7f80e000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.044+0000 7f80f796e700 1 -- 172.21.15.40:0/754993995 shutdown_connections 2023-07-18T03:39:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.044+0000 7f80f796e700 1 --2- 172.21.15.40:0/754993995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f0100630 0x7f80f0106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.044+0000 7f80f796e700 1 --2- 172.21.15.40:0/754993995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80f00feab0 0x7f80f00feec0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.044+0000 7f80f796e700 1 --2- 172.21.15.40:0/754993995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f00ffca0 0x7f80f01000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.044+0000 7f80f796e700 1 -- 172.21.15.40:0/754993995 >> 172.21.15.40:0/754993995 conn(0x7f80f00fa0d0 msgr2=0x7f80f00fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.044+0000 7f80f796e700 1 -- 172.21.15.40:0/754993995 shutdown_connections 2023-07-18T03:39:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.044+0000 7f80f796e700 1 -- 172.21.15.40:0/754993995 wait complete. 2023-07-18T03:39:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.045+0000 7f80f796e700 1 Processor -- start 2023-07-18T03:39:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.045+0000 7f80f796e700 1 -- start start 2023-07-18T03:39:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.045+0000 7f80f796e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f00feab0 0x7f80f0195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.045+0000 7f80f796e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f00ffca0 0x7f80f0196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.045+0000 7f80f796e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80f0100630 0x7f80f0196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.045+0000 7f80f796e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f01974e0 con 0x7f80f0100630 2023-07-18T03:39:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.045+0000 7f80f796e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f0197650 con 0x7f80f00ffca0 2023-07-18T03:39:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.045+0000 7f80f796e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f0197810 con 0x7f80f00feab0 2023-07-18T03:39:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.045+0000 7f80f5f0b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80f0100630 0x7f80f0196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.043 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.045+0000 7f80f570a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f00feab0 0x7f80f0195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.045+0000 7f80f570a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f00feab0 0x7f80f0195ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50106/0 (socket says 172.21.15.40:50106) 2023-07-18T03:39:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.045+0000 7f80f570a700 1 -- 172.21.15.40:0/52443958 learned_addr learned my addr 172.21.15.40:0/52443958 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.045+0000 7f80f4f09700 1 --2- 172.21.15.40:0/52443958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f00ffca0 0x7f80f0196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.045+0000 7f80f5f0b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80f0100630 0x7f80f0196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32850/0 (socket says 172.21.15.40:32850) 2023-07-18T03:39:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.046+0000 7f80f570a700 1 -- 172.21.15.40:0/52443958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f00ffca0 msgr2=0x7f80f0196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.044 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.046+0000 7f80f570a700 1 --2- 172.21.15.40:0/52443958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f00ffca0 0x7f80f0196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.046+0000 7f80f570a700 1 -- 172.21.15.40:0/52443958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80f0100630 msgr2=0x7f80f0196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.046+0000 7f80f570a700 1 --2- 172.21.15.40:0/52443958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80f0100630 0x7f80f0196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.046+0000 7f80f570a700 1 -- 172.21.15.40:0/52443958 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e0009d00 con 0x7f80f00feab0 2023-07-18T03:39:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.046+0000 7f80f570a700 1 --2- 172.21.15.40:0/52443958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f00feab0 0x7f80f0195ac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f80e000b070 tx=0x7f80e000ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.046+0000 7f80ee7fc700 1 -- 172.21.15.40:0/52443958 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e000e040 con 0x7f80f00feab0 2023-07-18T03:39:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.046+0000 7f80f796e700 1 -- 172.21.15.40:0/52443958 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f0197a90 con 0x7f80f00feab0 2023-07-18T03:39:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.046+0000 7f80ee7fc700 1 -- 172.21.15.40:0/52443958 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80e0003c00 con 0x7f80f00feab0 2023-07-18T03:39:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.046+0000 7f80ee7fc700 1 -- 172.21.15.40:0/52443958 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e00052a0 con 0x7f80f00feab0 2023-07-18T03:39:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.046+0000 7f80f796e700 1 -- 172.21.15.40:0/52443958 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f019e2b0 con 0x7f80f00feab0 2023-07-18T03:39:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.048+0000 7f80ee7fc700 1 -- 172.21.15.40:0/52443958 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f80e001d480 con 0x7f80f00feab0 2023-07-18T03:39:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.049+0000 7f80ee7fc700 1 --2- 172.21.15.40:0/52443958 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f80d80a1150 0x7f80d80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.049+0000 7f80ee7fc700 1 -- 172.21.15.40:0/52443958 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f80e00cd7e0 con 0x7f80f00feab0 2023-07-18T03:39:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.049+0000 7f80f4f09700 1 --2- 172.21.15.40:0/52443958 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f80d80a1150 0x7f80d80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f80f4f09700 1 --2- 172.21.15.40:0/52443958 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f80d80a1150 0x7f80d80a3600 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009c60 tx=0x7f80e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f8103af4b80 1 Processor -- start 2023-07-18T03:39:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f8103af4b80 1 -- start start 2023-07-18T03:39:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f8103af4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601eab06b0 0x55601ec54300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f8103af4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601ec54840 0x55601ec551a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f8103af4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec556e0 0x55601ec55b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f8103af4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601eb59c50 con 0x55601ec54840 2023-07-18T03:39:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f8103af4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601ebac9e0 con 0x55601eab06b0 2023-07-18T03:39:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f8103af4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601eba5e70 con 0x55601ec556e0 2023-07-18T03:39:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f80f4f09700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601ec54840 0x55601ec551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f80f570a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601eab06b0 0x55601ec54300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f80f4f09700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601ec54840 0x55601ec551a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32856/0 (socket says 172.21.15.40:32856) 2023-07-18T03:39:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f80f4f09700 1 -- 172.21.15.40:0/1034576446 learned_addr learned my addr 172.21.15.40:0/1034576446 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f80f570a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601eab06b0 0x55601ec54300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:44412/0 (socket says 172.21.15.40:44412) 2023-07-18T03:39:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f80f5f0b700 1 --2- 172.21.15.40:0/1034576446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec556e0 0x55601ec55b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f80f570a700 1 -- 172.21.15.40:0/1034576446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec556e0 msgr2=0x55601ec55b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f80f570a700 1 --2- 172.21.15.40:0/1034576446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec556e0 0x55601ec55b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f80f570a700 1 -- 172.21.15.40:0/1034576446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601ec54840 msgr2=0x55601ec551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f80f570a700 1 --2- 172.21.15.40:0/1034576446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601ec54840 0x55601ec551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.050+0000 7f80f570a700 1 -- 172.21.15.40:0/1034576446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55601e9d3d80 con 0x55601eab06b0 2023-07-18T03:39:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.051+0000 7f80f570a700 1 --2- 172.21.15.40:0/1034576446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601eab06b0 0x55601ec54300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f80e00079f0 tx=0x7f80e000be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=889a36bae697dab8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.051+0000 7f80d77fe700 1 -- 172.21.15.40:0/1034576446 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e009ec90 con 0x55601eab06b0 2023-07-18T03:39:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.051+0000 7f80d77fe700 1 -- 172.21.15.40:0/1034576446 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80e00a3020 con 0x55601eab06b0 2023-07-18T03:39:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.051+0000 7f80d77fe700 1 -- 172.21.15.40:0/1034576446 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e009a3b0 con 0x55601eab06b0 2023-07-18T03:39:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.052+0000 7f8103af4b80 1 -- 172.21.15.40:0/1034576446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601eab06b0 msgr2=0x55601ec54300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.052+0000 7f8103af4b80 1 --2- 172.21.15.40:0/1034576446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601eab06b0 0x55601ec54300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f80e00079f0 tx=0x7f80e000be90 comp rx=0 tx=0).stop 2023-07-18T03:39:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.052+0000 7f8103af4b80 1 -- 172.21.15.40:0/1034576446 shutdown_connections 2023-07-18T03:39:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.052+0000 7f8103af4b80 1 --2- 172.21.15.40:0/1034576446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec556e0 0x55601ec55b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.052+0000 7f8103af4b80 1 --2- 172.21.15.40:0/1034576446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601eab06b0 0x55601ec54300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.052+0000 7f8103af4b80 1 --2- 172.21.15.40:0/1034576446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601ec54840 0x55601ec551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.052+0000 7f8103af4b80 1 -- 172.21.15.40:0/1034576446 >> 172.21.15.40:0/1034576446 conn(0x55601ec49be0 msgr2=0x55601ec4ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.052+0000 7f8103af4b80 1 -- 172.21.15.40:0/1034576446 shutdown_connections 2023-07-18T03:39:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.052+0000 7f8103af4b80 1 -- 172.21.15.40:0/1034576446 wait complete. 2023-07-18T03:39:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.052+0000 7f8103af4b80 1 -- start start 2023-07-18T03:39:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.052+0000 7f8103af4b80 1 Processor -- start 2023-07-18T03:39:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.052+0000 7f8103af4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601eab06b0 0x55601ec5d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.052+0000 7f8103af4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec54840 0x55601ec5dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.052+0000 7f8103af4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601ec556e0 0x55601ec64c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.057 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f8103af4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601eb45a60 con 0x55601eab06b0 2023-07-18T03:39:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f8103af4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601eb45760 con 0x55601ec556e0 2023-07-18T03:39:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f8103af4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601ea298c0 con 0x55601ec54840 2023-07-18T03:39:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f8103af4b80 10 client.0 _ll_register_callbacks cb 0x55601eb43700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:39:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f80f570a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601eab06b0 0x55601ec5d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f80f4f09700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec54840 0x55601ec5dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f80f570a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601eab06b0 0x55601ec5d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32860/0 (socket says 172.21.15.40:32860) 2023-07-18T03:39:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f80f5f0b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601ec556e0 0x55601ec64c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f80f4f09700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec54840 0x55601ec5dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50122/0 (socket says 172.21.15.40:50122) 2023-07-18T03:39:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f80f570a700 1 -- 172.21.15.40:0/1033245786 learned_addr learned my addr 172.21.15.40:0/1033245786 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f80f570a700 1 -- 172.21.15.40:0/1033245786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec54840 msgr2=0x55601ec5dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f80f570a700 1 --2- 172.21.15.40:0/1033245786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec54840 0x55601ec5dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f80f570a700 1 -- 172.21.15.40:0/1033245786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601ec556e0 msgr2=0x55601ec64c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f80f570a700 1 --2- 172.21.15.40:0/1033245786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601ec556e0 0x55601ec64c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f80f570a700 1 -- 172.21.15.40:0/1033245786 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f019e2b0 con 0x55601eab06b0 2023-07-18T03:39:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f80f5f0b700 1 --2- 172.21.15.40:0/1033245786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601ec556e0 0x55601ec64c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f80f570a700 1 --2- 172.21.15.40:0/1033245786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601eab06b0 0x55601ec5d6a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f80e00a2af0 tx=0x7f80e00099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89138ed721c1a74c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f80d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:39:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f80d4ff9700 1 -- 172.21.15.40:0/1033245786 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e009ec90 con 0x55601eab06b0 2023-07-18T03:39:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f80d4ff9700 1 -- 172.21.15.40:0/1033245786 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f80e00b3120 con 0x55601eab06b0 2023-07-18T03:39:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f8103af4b80 1 -- 172.21.15.40:0/1033245786 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55601eaff490 con 0x55601eab06b0 2023-07-18T03:39:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f80d4ff9700 1 -- 172.21.15.40:0/1033245786 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80e00b32e0 con 0x55601eab06b0 2023-07-18T03:39:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.053+0000 7f8103af4b80 10 client.16692 fetch_fsmap learned FSMap version 62 2023-07-18T03:39:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.054+0000 7f8103af4b80 1 -- 172.21.15.40:0/1033245786 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x55601e9bcc00 con 0x55601eab06b0 2023-07-18T03:39:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.054+0000 7f80d4ff9700 1 -- 172.21.15.40:0/1033245786 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55601eaff490 con 0x55601eab06b0 2023-07-18T03:39:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.054+0000 7f80d4ff9700 1 -- 172.21.15.40:0/1033245786 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 62) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f80e001d7a0 con 0x55601eab06b0 2023-07-18T03:39:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.054+0000 7f8103af4b80 10 client.16692 fetch_fsmap finished waiting for FSMap version 62 2023-07-18T03:39:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.054+0000 7f8103af4b80 10 client.16692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:39:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.054+0000 7f8103af4b80 20 client.16692 populate_metadata read hostname 'smithi040' 2023-07-18T03:39:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.054+0000 7f8103af4b80 1 --2- 172.21.15.40:0/1033245786 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55601ec68f80 0x55601ec6b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.054+0000 7f8103af4b80 4 client.16692 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:39:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.054+0000 7f8103af4b80 1 -- 172.21.15.40:0/1033245786 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55601eb9ee50 con 0x55601ec68f80 2023-07-18T03:39:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.054+0000 7f80f4f09700 1 --2- 172.21.15.40:0/1033245786 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55601ec68f80 0x55601ec6b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.054+0000 7f80d4ff9700 10 client.16692 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.054+0000 7f80f4f09700 1 --2- 172.21.15.40:0/1033245786 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55601ec68f80 0x55601ec6b390 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c21bd8185ba41688 server_cookie=e962cd4f05a2875e in_seq=0 out_seq=0 2023-07-18T03:39:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.055+0000 7f80d4ff9700 1 -- 172.21.15.40:0/1033245786 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55601eb9ee50 con 0x55601ec68f80 2023-07-18T03:39:35.062 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.055+0000 7f80d4ff9700 10 client.16692 handle_command_reply: tid=0 2023-07-18T03:39:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.055+0000 7f8103af4b80 1 client.16692 shutdown 2023-07-18T03:39:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.055+0000 7f8103af4b80 1 -- 172.21.15.40:0/1033245786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601eab06b0 msgr2=0x55601ec5d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.055+0000 7f8103af4b80 1 --2- 172.21.15.40:0/1033245786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601eab06b0 0x55601ec5d6a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f80e00a2af0 tx=0x7f80e00099e0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.055+0000 7f8103af4b80 1 -- 172.21.15.40:0/1033245786 shutdown_connections 2023-07-18T03:39:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.055+0000 7f8103af4b80 1 --2- 172.21.15.40:0/1033245786 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55601ec68f80 0x55601ec6b390 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.055+0000 7f8103af4b80 1 --2- 172.21.15.40:0/1033245786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601ec556e0 0x55601ec64c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.055+0000 7f80d4ff9700 0 client.16692 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.055+0000 7f8103af4b80 1 --2- 172.21.15.40:0/1033245786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601eab06b0 0x55601ec5d6a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.055+0000 7f8103af4b80 1 --2- 172.21.15.40:0/1033245786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec54840 0x55601ec5dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.055+0000 7f8103af4b80 1 -- 172.21.15.40:0/1033245786 >> 172.21.15.40:0/1033245786 conn(0x55601ec49be0 msgr2=0x55601ec4e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.055+0000 7f8103af4b80 1 -- 172.21.15.40:0/1033245786 shutdown_connections 2023-07-18T03:39:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.055+0000 7f8103af4b80 1 -- 172.21.15.40:0/1033245786 wait complete. 2023-07-18T03:39:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.056+0000 7f8103af4b80 20 client.16692 trim_cache size 0 max 16384 2023-07-18T03:39:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.070+0000 7f8103af4b80 1 Processor -- start 2023-07-18T03:39:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.070+0000 7f8103af4b80 1 -- start start 2023-07-18T03:39:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f8103af4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601eab06b0 0x55601ec5db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f8103af4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec54840 0x55601ec64bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f8103af4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601ec556e0 0x55601ec5ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f8103af4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601eaff490 con 0x55601eab06b0 2023-07-18T03:39:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f8103af4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601ec1a060 con 0x55601ec556e0 2023-07-18T03:39:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f8103af4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601ec5cb50 con 0x55601ec54840 2023-07-18T03:39:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f80f570a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601eab06b0 0x55601ec5db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f80f570a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601eab06b0 0x55601ec5db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32868/0 (socket says 172.21.15.40:32868) 2023-07-18T03:39:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f80f570a700 1 -- 172.21.15.40:0/3277016478 learned_addr learned my addr 172.21.15.40:0/3277016478 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f80f4f09700 1 --2- 172.21.15.40:0/3277016478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec54840 0x55601ec64bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f80f5f0b700 1 --2- 172.21.15.40:0/3277016478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601ec556e0 0x55601ec5ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f80f570a700 1 -- 172.21.15.40:0/3277016478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec54840 msgr2=0x55601ec64bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f80f570a700 1 --2- 172.21.15.40:0/3277016478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec54840 0x55601ec64bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f80f570a700 1 -- 172.21.15.40:0/3277016478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601ec556e0 msgr2=0x55601ec5ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f80f570a700 1 --2- 172.21.15.40:0/3277016478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601ec556e0 0x55601ec5ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f80f570a700 1 -- 172.21.15.40:0/3277016478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55601e9fa1f0 con 0x55601eab06b0 2023-07-18T03:39:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f80f5f0b700 1 --2- 172.21.15.40:0/3277016478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601ec556e0 0x55601ec5ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f80f570a700 1 --2- 172.21.15.40:0/3277016478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601eab06b0 0x55601ec5db60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f80e00079f0 tx=0x7f80e000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c104a2972a90aef server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f80d7fff700 1 -- 172.21.15.40:0/3277016478 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e009b010 con 0x55601eab06b0 2023-07-18T03:39:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f80d7fff700 1 -- 172.21.15.40:0/3277016478 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80e009ec90 con 0x55601eab06b0 2023-07-18T03:39:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.071+0000 7f80d7fff700 1 -- 172.21.15.40:0/3277016478 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e00a2b90 con 0x55601eab06b0 2023-07-18T03:39:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.072+0000 7f8103af4b80 1 -- 172.21.15.40:0/3277016478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601eab06b0 msgr2=0x55601ec5db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.072+0000 7f8103af4b80 1 --2- 172.21.15.40:0/3277016478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601eab06b0 0x55601ec5db60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f80e00079f0 tx=0x7f80e000be90 comp rx=0 tx=0).stop 2023-07-18T03:39:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.072+0000 7f8103af4b80 1 -- 172.21.15.40:0/3277016478 shutdown_connections 2023-07-18T03:39:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.072+0000 7f8103af4b80 1 --2- 172.21.15.40:0/3277016478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601ec556e0 0x55601ec5ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.072+0000 7f8103af4b80 1 --2- 172.21.15.40:0/3277016478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601eab06b0 0x55601ec5db60 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.072+0000 7f8103af4b80 1 --2- 172.21.15.40:0/3277016478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec54840 0x55601ec64bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.072+0000 7f8103af4b80 1 -- 172.21.15.40:0/3277016478 >> 172.21.15.40:0/3277016478 conn(0x55601ec49be0 msgr2=0x55601ec5d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.072+0000 7f8103af4b80 1 -- 172.21.15.40:0/3277016478 shutdown_connections 2023-07-18T03:39:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.072+0000 7f8103af4b80 1 -- 172.21.15.40:0/3277016478 wait complete. 2023-07-18T03:39:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.072+0000 7f8103af4b80 1 -- start start 2023-07-18T03:39:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.072+0000 7f8103af4b80 1 Processor -- start 2023-07-18T03:39:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f8103af4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601eab06b0 0x55601ec60520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f8103af4b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601ec54840 0x55601ec60a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f8103af4b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec556e0 0x55601ec63a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f8103af4b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601ec63fc0 con 0x55601ec54840 2023-07-18T03:39:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f8103af4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601ec64130 con 0x55601eab06b0 2023-07-18T03:39:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f8103af4b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601ec642f0 con 0x55601ec556e0 2023-07-18T03:39:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f8103af4b80 10 client.0 _ll_register_callbacks cb 0x55601eb43700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:39:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f80f4f09700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601ec54840 0x55601ec60a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f80f4f09700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601ec54840 0x55601ec60a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32876/0 (socket says 172.21.15.40:32876) 2023-07-18T03:39:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f80f4f09700 1 -- 172.21.15.40:0/304018826 learned_addr learned my addr 172.21.15.40:0/304018826 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f80f5f0b700 1 --2- 172.21.15.40:0/304018826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec556e0 0x55601ec63a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f80f570a700 1 --2- 172.21.15.40:0/304018826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601eab06b0 0x55601ec60520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f80f4f09700 1 -- 172.21.15.40:0/304018826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec556e0 msgr2=0x55601ec63a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:35.076 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f80f4f09700 1 --2- 172.21.15.40:0/304018826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec556e0 0x55601ec63a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f80f4f09700 1 -- 172.21.15.40:0/304018826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601eab06b0 msgr2=0x55601ec60520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f80f4f09700 1 --2- 172.21.15.40:0/304018826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601eab06b0 0x55601ec60520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f80f4f09700 1 -- 172.21.15.40:0/304018826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f019e2b0 con 0x55601ec54840 2023-07-18T03:39:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f80f4f09700 1 --2- 172.21.15.40:0/304018826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601ec54840 0x55601ec60a60 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f80e4013010 tx=0x7f80e40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f59604f2c20a26b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.073+0000 7f80d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:39:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.074+0000 7f80d4ff9700 1 -- 172.21.15.40:0/304018826 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4022040 con 0x55601ec54840 2023-07-18T03:39:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.074+0000 7f80d4ff9700 1 -- 172.21.15.40:0/304018826 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f80e402b250 con 0x55601ec54840 2023-07-18T03:39:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.074+0000 7f8103af4b80 1 -- 172.21.15.40:0/304018826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80e00a3020 con 0x55601ec54840 2023-07-18T03:39:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.074+0000 7f80d4ff9700 1 -- 172.21.15.40:0/304018826 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80e402b410 con 0x55601ec54840 2023-07-18T03:39:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.074+0000 7f8103af4b80 10 client.16698 fetch_fsmap learned FSMap version 62 2023-07-18T03:39:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.074+0000 7f8103af4b80 1 -- 172.21.15.40:0/304018826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x55601e9d2250 con 0x55601ec54840 2023-07-18T03:39:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.074+0000 7f80d4ff9700 1 -- 172.21.15.40:0/304018826 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80e00a3020 con 0x55601ec54840 2023-07-18T03:39:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.075+0000 7f80d4ff9700 1 -- 172.21.15.40:0/304018826 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 62) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f80e403d050 con 0x55601ec54840 2023-07-18T03:39:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.075+0000 7f8103af4b80 10 client.16698 fetch_fsmap finished waiting for FSMap version 62 2023-07-18T03:39:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.075+0000 7f8103af4b80 10 client.16698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:39:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.075+0000 7f8103af4b80 20 client.16698 populate_metadata read hostname 'smithi040' 2023-07-18T03:39:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.075+0000 7f8103af4b80 1 --2- 172.21.15.40:0/304018826 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55601ecbd9e0 0x55601ecbfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.075+0000 7f8103af4b80 4 client.16698 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:39:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.075+0000 7f8103af4b80 1 -- 172.21.15.40:0/304018826 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55601eb60ff0 con 0x55601ecbd9e0 2023-07-18T03:39:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.075+0000 7f80f570a700 1 --2- 172.21.15.40:0/304018826 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55601ecbd9e0 0x55601ecbfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.075+0000 7f80d4ff9700 10 client.16698 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.075+0000 7f80f570a700 1 --2- 172.21.15.40:0/304018826 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55601ecbd9e0 0x55601ecbfdf0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f16438ce900de0ef server_cookie=9e7553065768c176 in_seq=0 out_seq=0 2023-07-18T03:39:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.085+0000 7f80d4ff9700 1 -- 172.21.15.40:0/304018826 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55601eb60ff0 con 0x55601ecbd9e0 2023-07-18T03:39:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.085+0000 7f80d4ff9700 10 client.16698 handle_command_reply: tid=0 2023-07-18T03:39:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.085+0000 7f8103af4b80 1 client.16698 shutdown 2023-07-18T03:39:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.085+0000 7f8103af4b80 1 -- 172.21.15.40:0/304018826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601ec54840 msgr2=0x55601ec60a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.085+0000 7f8103af4b80 1 --2- 172.21.15.40:0/304018826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601ec54840 0x55601ec60a60 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f80e4013010 tx=0x7f80e40047c0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.085+0000 7f8103af4b80 1 -- 172.21.15.40:0/304018826 shutdown_connections 2023-07-18T03:39:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.085+0000 7f8103af4b80 1 --2- 172.21.15.40:0/304018826 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55601ecbd9e0 0x55601ecbfdf0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.085+0000 7f8103af4b80 1 --2- 172.21.15.40:0/304018826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55601ec556e0 0x55601ec63a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.085+0000 7f80d4ff9700 0 client.16698 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.085+0000 7f8103af4b80 1 --2- 172.21.15.40:0/304018826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55601eab06b0 0x55601ec60520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.085+0000 7f8103af4b80 1 --2- 172.21.15.40:0/304018826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55601ec54840 0x55601ec60a60 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.085+0000 7f8103af4b80 1 -- 172.21.15.40:0/304018826 >> 172.21.15.40:0/304018826 conn(0x55601ec49be0 msgr2=0x55601ec53030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.086+0000 7f8103af4b80 1 -- 172.21.15.40:0/304018826 shutdown_connections 2023-07-18T03:39:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.086+0000 7f8103af4b80 1 -- 172.21.15.40:0/304018826 wait complete. 2023-07-18T03:39:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.086+0000 7f8103af4b80 20 client.16698 trim_cache size 0 max 16384 2023-07-18T03:39:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:39:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:39:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:39:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:39:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.086+0000 7f80f796e700 1 -- 172.21.15.40:0/52443958 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f80d80a1150 msgr2=0x7f80d80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.086+0000 7f80f796e700 1 --2- 172.21.15.40:0/52443958 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f80d80a1150 0x7f80d80a3600 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009c60 tx=0x7f80e4009260 comp rx=0 tx=0).stop 2023-07-18T03:39:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.086+0000 7f80f796e700 1 -- 172.21.15.40:0/52443958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f00feab0 msgr2=0x7f80f0195ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.086+0000 7f80f796e700 1 --2- 172.21.15.40:0/52443958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f00feab0 0x7f80f0195ac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f80e000b070 tx=0x7f80e000ba50 comp rx=0 tx=0).stop 2023-07-18T03:39:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.086+0000 7f80f796e700 1 -- 172.21.15.40:0/52443958 shutdown_connections 2023-07-18T03:39:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.086+0000 7f80f796e700 1 --2- 172.21.15.40:0/52443958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80f0100630 0x7f80f0196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.086+0000 7f80f796e700 1 --2- 172.21.15.40:0/52443958 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f80d80a1150 0x7f80d80a3600 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.086+0000 7f80f796e700 1 --2- 172.21.15.40:0/52443958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80f00feab0 0x7f80f0195ac0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.086+0000 7f80f796e700 1 --2- 172.21.15.40:0/52443958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f00ffca0 0x7f80f0196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.086+0000 7f80f796e700 1 -- 172.21.15.40:0/52443958 >> 172.21.15.40:0/52443958 conn(0x7f80f00fa0d0 msgr2=0x7f80f00fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.086+0000 7f80f796e700 1 -- 172.21.15.40:0/52443958 shutdown_connections 2023-07-18T03:39:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:35.086+0000 7f80f796e700 1 -- 172.21.15.40:0/52443958 wait complete. 2023-07-18T03:39:35.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:35 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:35.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:35 smithi134 ceph-mon[98216]: pgmap v555: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 975 KiB/s wr, 232 op/s 2023-07-18T03:39:35.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:35 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:35.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:35 smithi040 ceph-mon[96439]: pgmap v555: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 975 KiB/s wr, 232 op/s 2023-07-18T03:39:35.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:35 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:35.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:35 smithi064 ceph-mon[98059]: pgmap v555: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 975 KiB/s wr, 232 op/s 2023-07-18T03:39:37.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:37 smithi134 ceph-mon[98216]: pgmap v556: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1013 KiB/s wr, 239 op/s 2023-07-18T03:39:37.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:37 smithi040 ceph-mon[96439]: pgmap v556: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1013 KiB/s wr, 239 op/s 2023-07-18T03:39:37.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:37 smithi064 ceph-mon[98059]: pgmap v556: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1013 KiB/s wr, 239 op/s 2023-07-18T03:39:39.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:39 smithi134 ceph-mon[98216]: pgmap v557: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1010 KiB/s wr, 293 op/s 2023-07-18T03:39:39.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:39 smithi040 ceph-mon[96439]: pgmap v557: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1010 KiB/s wr, 293 op/s 2023-07-18T03:39:39.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:39 smithi064 ceph-mon[98059]: pgmap v557: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1010 KiB/s wr, 293 op/s 2023-07-18T03:39:41.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:41 smithi134 ceph-mon[98216]: pgmap v558: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 705 KiB/s wr, 227 op/s 2023-07-18T03:39:41.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:41 smithi040 ceph-mon[96439]: pgmap v558: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 705 KiB/s wr, 227 op/s 2023-07-18T03:39:41.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:41 smithi064 ceph-mon[98059]: pgmap v558: 97 pgs: 97 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 705 KiB/s wr, 227 op/s 2023-07-18T03:39:43.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:42 smithi134 ceph-mon[98216]: pgmap v559: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 302 op/s 2023-07-18T03:39:43.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:42 smithi040 ceph-mon[96439]: pgmap v559: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 302 op/s 2023-07-18T03:39:43.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:42 smithi064 ceph-mon[98059]: pgmap v559: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 302 op/s 2023-07-18T03:39:45.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:45 smithi134 ceph-mon[98216]: pgmap v560: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 239 op/s 2023-07-18T03:39:45.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:45 smithi040 ceph-mon[96439]: pgmap v560: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 239 op/s 2023-07-18T03:39:45.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:45 smithi064 ceph-mon[98059]: pgmap v560: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 239 op/s 2023-07-18T03:39:47.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:47 smithi134 ceph-mon[98216]: pgmap v561: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 241 op/s 2023-07-18T03:39:47.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:47 smithi040 ceph-mon[96439]: pgmap v561: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 241 op/s 2023-07-18T03:39:47.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:47 smithi064 ceph-mon[98059]: pgmap v561: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 241 op/s 2023-07-18T03:39:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.198+0000 7f9b23251700 1 -- 172.21.15.40:0/1255056655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c0fea00 msgr2=0x7f9b1c0fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.198+0000 7f9b23251700 1 --2- 172.21.15.40:0/1255056655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c0fea00 0x7f9b1c0fee10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009990 tx=0x7f9b10009ca0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.198+0000 7f9b23251700 1 -- 172.21.15.40:0/1255056655 shutdown_connections 2023-07-18T03:39:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.198+0000 7f9b23251700 1 --2- 172.21.15.40:0/1255056655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b1c100640 0x7f9b1c106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.198+0000 7f9b23251700 1 --2- 172.21.15.40:0/1255056655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c0fea00 0x7f9b1c0fee10 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.198+0000 7f9b23251700 1 --2- 172.21.15.40:0/1255056655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b1c0ffc90 0x7f9b1c100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.198+0000 7f9b23251700 1 -- 172.21.15.40:0/1255056655 >> 172.21.15.40:0/1255056655 conn(0x7f9b1c0fa000 msgr2=0x7f9b1c0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.198+0000 7f9b23251700 1 -- 172.21.15.40:0/1255056655 shutdown_connections 2023-07-18T03:39:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.198+0000 7f9b23251700 1 -- 172.21.15.40:0/1255056655 wait complete. 2023-07-18T03:39:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b23251700 1 Processor -- start 2023-07-18T03:39:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b23251700 1 -- start start 2023-07-18T03:39:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b23251700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b1c0fea00 0x7f9b1c197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b23251700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c0ffc90 0x7f9b1c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b23251700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b1c100640 0x7f9b1c198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b23251700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c199590 con 0x7f9b1c0ffc90 2023-07-18T03:39:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b23251700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c199700 con 0x7f9b1c0fea00 2023-07-18T03:39:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b23251700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c1998c0 con 0x7f9b1c100640 2023-07-18T03:39:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b1bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c0ffc90 0x7f9b1c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b20fed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b1c0fea00 0x7f9b1c197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b217ee700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b1c100640 0x7f9b1c198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b1bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c0ffc90 0x7f9b1c1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32886/0 (socket says 172.21.15.40:32886) 2023-07-18T03:39:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b1bfff700 1 -- 172.21.15.40:0/1852499113 learned_addr learned my addr 172.21.15.40:0/1852499113 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b1bfff700 1 -- 172.21.15.40:0/1852499113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b1c100640 msgr2=0x7f9b1c198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b1bfff700 1 --2- 172.21.15.40:0/1852499113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b1c100640 0x7f9b1c198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b1bfff700 1 -- 172.21.15.40:0/1852499113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b1c0fea00 msgr2=0x7f9b1c197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b1bfff700 1 --2- 172.21.15.40:0/1852499113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b1c0fea00 0x7f9b1c197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b1bfff700 1 -- 172.21.15.40:0/1852499113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b100095f0 con 0x7f9b1c0ffc90 2023-07-18T03:39:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.199+0000 7f9b1bfff700 1 --2- 172.21.15.40:0/1852499113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c0ffc90 0x7f9b1c1980b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800e9a0 tx=0x7f9b0800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.200+0000 7f9b19ffb700 1 -- 172.21.15.40:0/1852499113 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08009790 con 0x7f9b1c0ffc90 2023-07-18T03:39:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.200+0000 7f9b19ffb700 1 -- 172.21.15.40:0/1852499113 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b08010430 con 0x7f9b1c0ffc90 2023-07-18T03:39:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.200+0000 7f9b23251700 1 -- 172.21.15.40:0/1852499113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1c199ba0 con 0x7f9b1c0ffc90 2023-07-18T03:39:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.200+0000 7f9b19ffb700 1 -- 172.21.15.40:0/1852499113 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08018680 con 0x7f9b1c0ffc90 2023-07-18T03:39:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.200+0000 7f9b23251700 1 -- 172.21.15.40:0/1852499113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1c1a0360 con 0x7f9b1c0ffc90 2023-07-18T03:39:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.203+0000 7f9b19ffb700 1 -- 172.21.15.40:0/1852499113 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9b080105a0 con 0x7f9b1c0ffc90 2023-07-18T03:39:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.203+0000 7f9b19ffb700 1 --2- 172.21.15.40:0/1852499113 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9b0c0a11d0 0x7f9b0c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.203+0000 7f9b19ffb700 1 -- 172.21.15.40:0/1852499113 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9b08014070 con 0x7f9b1c0ffc90 2023-07-18T03:39:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.203+0000 7f9b20fed700 1 --2- 172.21.15.40:0/1852499113 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9b0c0a11d0 0x7f9b0c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.204+0000 7f9b20fed700 1 --2- 172.21.15.40:0/1852499113 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9b0c0a11d0 0x7f9b0c0a3680 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000b2e0 tx=0x7f9b10005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.204+0000 7f9b2f3d7b80 1 Processor -- start 2023-07-18T03:39:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.204+0000 7f9b2f3d7b80 1 -- start start 2023-07-18T03:39:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.204+0000 7f9b2f3d7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d67f6b0 0x560f5d823300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.204+0000 7f9b2f3d7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d823840 0x560f5d8241a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.204+0000 7f9b2f3d7b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d8246e0 0x560f5d824b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.204+0000 7f9b2f3d7b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5d728c50 con 0x560f5d8246e0 2023-07-18T03:39:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.204+0000 7f9b2f3d7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5d77b9e0 con 0x560f5d67f6b0 2023-07-18T03:39:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.204+0000 7f9b2f3d7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5d774e70 con 0x560f5d823840 2023-07-18T03:39:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.204+0000 7f9b20fed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d67f6b0 0x560f5d823300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.204+0000 7f9b217ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d8246e0 0x560f5d824b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.204+0000 7f9b20fed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d67f6b0 0x560f5d823300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:44452/0 (socket says 172.21.15.40:44452) 2023-07-18T03:39:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.204+0000 7f9b20fed700 1 -- 172.21.15.40:0/3466419060 learned_addr learned my addr 172.21.15.40:0/3466419060 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.204+0000 7f9b1bfff700 1 --2- 172.21.15.40:0/3466419060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d823840 0x560f5d8241a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.205+0000 7f9b217ee700 1 -- 172.21.15.40:0/3466419060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d823840 msgr2=0x560f5d8241a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.205+0000 7f9b217ee700 1 --2- 172.21.15.40:0/3466419060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d823840 0x560f5d8241a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.205+0000 7f9b217ee700 1 -- 172.21.15.40:0/3466419060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d67f6b0 msgr2=0x560f5d823300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.205+0000 7f9b217ee700 1 --2- 172.21.15.40:0/3466419060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d67f6b0 0x560f5d823300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.205+0000 7f9b217ee700 1 -- 172.21.15.40:0/3466419060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f5d5a2d80 con 0x560f5d8246e0 2023-07-18T03:39:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.205+0000 7f9b20fed700 1 --2- 172.21.15.40:0/3466419060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d67f6b0 0x560f5d823300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.205+0000 7f9b217ee700 1 --2- 172.21.15.40:0/3466419060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d8246e0 0x560f5d824b80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9b140113f0 tx=0x7f9b14009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea015756d41844ab server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.205+0000 7f9b06ffd700 1 -- 172.21.15.40:0/3466419060 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1400e040 con 0x560f5d8246e0 2023-07-18T03:39:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.205+0000 7f9b06ffd700 1 -- 172.21.15.40:0/3466419060 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b1400f040 con 0x560f5d8246e0 2023-07-18T03:39:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.205+0000 7f9b06ffd700 1 -- 172.21.15.40:0/3466419060 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b14015630 con 0x560f5d8246e0 2023-07-18T03:39:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.206+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3466419060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d8246e0 msgr2=0x560f5d824b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.206+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/3466419060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d8246e0 0x560f5d824b80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9b140113f0 tx=0x7f9b14009920 comp rx=0 tx=0).stop 2023-07-18T03:39:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.206+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3466419060 shutdown_connections 2023-07-18T03:39:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.206+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/3466419060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d8246e0 0x560f5d824b80 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.206+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/3466419060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d67f6b0 0x560f5d823300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.206+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/3466419060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d823840 0x560f5d8241a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.206+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3466419060 >> 172.21.15.40:0/3466419060 conn(0x560f5d818be0 msgr2=0x560f5d81dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.206+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3466419060 shutdown_connections 2023-07-18T03:39:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.206+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3466419060 wait complete. 2023-07-18T03:39:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.206+0000 7f9b2f3d7b80 1 -- start start 2023-07-18T03:39:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.206+0000 7f9b2f3d7b80 1 Processor -- start 2023-07-18T03:39:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b2f3d7b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d67f6b0 0x560f5d82c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b2f3d7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d823840 0x560f5d82cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b2f3d7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d8246e0 0x560f5d833da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b2f3d7b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5d714a60 con 0x560f5d67f6b0 2023-07-18T03:39:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b2f3d7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5d714760 con 0x560f5d823840 2023-07-18T03:39:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b2f3d7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5d5f88c0 con 0x560f5d8246e0 2023-07-18T03:39:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b2f3d7b80 10 client.0 _ll_register_callbacks cb 0x560f5d712700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:39:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b20fed700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d67f6b0 0x560f5d82c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b20fed700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d67f6b0 0x560f5d82c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32900/0 (socket says 172.21.15.40:32900) 2023-07-18T03:39:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b20fed700 1 -- 172.21.15.40:0/3110949732 learned_addr learned my addr 172.21.15.40:0/3110949732 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b1bfff700 1 --2- 172.21.15.40:0/3110949732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d823840 0x560f5d82cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b217ee700 1 --2- 172.21.15.40:0/3110949732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d8246e0 0x560f5d833da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b20fed700 1 -- 172.21.15.40:0/3110949732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d8246e0 msgr2=0x560f5d833da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b20fed700 1 --2- 172.21.15.40:0/3110949732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d8246e0 0x560f5d833da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b20fed700 1 -- 172.21.15.40:0/3110949732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d823840 msgr2=0x560f5d82cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b20fed700 1 --2- 172.21.15.40:0/3110949732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d823840 0x560f5d82cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b20fed700 1 -- 172.21.15.40:0/3110949732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1c107750 con 0x560f5d67f6b0 2023-07-18T03:39:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b217ee700 1 --2- 172.21.15.40:0/3110949732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d8246e0 0x560f5d833da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:39:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b20fed700 1 --2- 172.21.15.40:0/3110949732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d67f6b0 0x560f5d82c820 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009990 tx=0x7f9b1001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f289d88aa6ef67 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9afbfff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:39:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9afbfff700 1 -- 172.21.15.40:0/3110949732 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1002b050 con 0x560f5d67f6b0 2023-07-18T03:39:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9afbfff700 1 -- 172.21.15.40:0/3110949732 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9b1000f660 con 0x560f5d67f6b0 2023-07-18T03:39:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.207+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3110949732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f5d6ce490 con 0x560f5d67f6b0 2023-07-18T03:39:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.208+0000 7f9afbfff700 1 -- 172.21.15.40:0/3110949732 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b1000f820 con 0x560f5d67f6b0 2023-07-18T03:39:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.208+0000 7f9b2f3d7b80 10 client.16710 fetch_fsmap learned FSMap version 62 2023-07-18T03:39:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.208+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3110949732 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x7f9b14005e70 con 0x560f5d67f6b0 2023-07-18T03:39:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.208+0000 7f9afbfff700 1 -- 172.21.15.40:0/3110949732 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b1003f080 con 0x560f5d67f6b0 2023-07-18T03:39:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.208+0000 7f9afbfff700 1 -- 172.21.15.40:0/3110949732 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 62) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f9b10039030 con 0x560f5d67f6b0 2023-07-18T03:39:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.208+0000 7f9b2f3d7b80 10 client.16710 fetch_fsmap finished waiting for FSMap version 62 2023-07-18T03:39:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.208+0000 7f9b2f3d7b80 10 client.16710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:39:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.208+0000 7f9b2f3d7b80 20 client.16710 populate_metadata read hostname 'smithi040' 2023-07-18T03:39:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.208+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/3110949732 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560f5d838680 0x560f5d83aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.208+0000 7f9b2f3d7b80 4 client.16710 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:39:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.208+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3110949732 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f5d76de50 con 0x560f5d838680 2023-07-18T03:39:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.208+0000 7f9b1bfff700 1 --2- 172.21.15.40:0/3110949732 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560f5d838680 0x560f5d83aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.209+0000 7f9afbfff700 10 client.16710 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.209+0000 7f9b1bfff700 1 --2- 172.21.15.40:0/3110949732 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560f5d838680 0x560f5d83aa90 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d9c12d28cbfda6e server_cookie=b91f7b739f48d69d in_seq=0 out_seq=0 2023-07-18T03:39:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.209+0000 7f9afbfff700 1 -- 172.21.15.40:0/3110949732 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560f5d76de50 con 0x560f5d838680 2023-07-18T03:39:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.209+0000 7f9afbfff700 10 client.16710 handle_command_reply: tid=0 2023-07-18T03:39:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.209+0000 7f9b2f3d7b80 1 client.16710 shutdown 2023-07-18T03:39:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.210+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3110949732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d67f6b0 msgr2=0x560f5d82c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.210+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/3110949732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d67f6b0 0x560f5d82c820 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009990 tx=0x7f9b1001a050 comp rx=0 tx=0).stop 2023-07-18T03:39:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.210+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3110949732 shutdown_connections 2023-07-18T03:39:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.210+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/3110949732 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560f5d838680 0x560f5d83aa90 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.210+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/3110949732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d8246e0 0x560f5d833da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.210+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/3110949732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d67f6b0 0x560f5d82c820 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.210+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/3110949732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d823840 0x560f5d82cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.210+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3110949732 >> 172.21.15.40:0/3110949732 conn(0x560f5d818be0 msgr2=0x560f5d81d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.210+0000 7f9afbfff700 0 client.16710 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.210+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3110949732 shutdown_connections 2023-07-18T03:39:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.210+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3110949732 wait complete. 2023-07-18T03:39:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.210+0000 7f9b2f3d7b80 20 client.16710 trim_cache size 0 max 16384 2023-07-18T03:39:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b2f3d7b80 1 Processor -- start 2023-07-18T03:39:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b2f3d7b80 1 -- start start 2023-07-18T03:39:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b2f3d7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d67f6b0 0x560f5d827550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b2f3d7b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d823840 0x560f5d827a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b2f3d7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d8246e0 0x560f5d833d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b2f3d7b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5d82c3f0 con 0x560f5d823840 2023-07-18T03:39:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b2f3d7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5d815bb0 con 0x560f5d67f6b0 2023-07-18T03:39:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b2f3d7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5d823c10 con 0x560f5d8246e0 2023-07-18T03:39:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b1bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d823840 0x560f5d827a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b1bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d823840 0x560f5d827a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32910/0 (socket says 172.21.15.40:32910) 2023-07-18T03:39:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b20fed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d67f6b0 0x560f5d827550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b217ee700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d8246e0 0x560f5d833d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b1bfff700 1 -- 172.21.15.40:0/528994893 learned_addr learned my addr 172.21.15.40:0/528994893 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b1bfff700 1 -- 172.21.15.40:0/528994893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d8246e0 msgr2=0x560f5d833d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b1bfff700 1 --2- 172.21.15.40:0/528994893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d8246e0 0x560f5d833d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b1bfff700 1 -- 172.21.15.40:0/528994893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d67f6b0 msgr2=0x560f5d827550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b1bfff700 1 --2- 172.21.15.40:0/528994893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d67f6b0 0x560f5d827550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b1bfff700 1 -- 172.21.15.40:0/528994893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f5d58bc00 con 0x560f5d823840 2023-07-18T03:39:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b20fed700 1 --2- 172.21.15.40:0/528994893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d67f6b0 0x560f5d827550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b217ee700 1 --2- 172.21.15.40:0/528994893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d8246e0 0x560f5d833d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.225+0000 7f9b1bfff700 1 --2- 172.21.15.40:0/528994893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d823840 0x560f5d827a90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9b080ca5d0 tx=0x7f9b080cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e8659b2b2e6833 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.226+0000 7f9b077fe700 1 -- 172.21.15.40:0/528994893 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b080b6070 con 0x560f5d823840 2023-07-18T03:39:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.226+0000 7f9b077fe700 1 -- 172.21.15.40:0/528994893 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b080a22e0 con 0x560f5d823840 2023-07-18T03:39:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.226+0000 7f9b077fe700 1 -- 172.21.15.40:0/528994893 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b080995e0 con 0x560f5d823840 2023-07-18T03:39:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.226+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/528994893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d823840 msgr2=0x560f5d827a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.226+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/528994893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d823840 0x560f5d827a90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9b080ca5d0 tx=0x7f9b080cc040 comp rx=0 tx=0).stop 2023-07-18T03:39:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.226+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/528994893 shutdown_connections 2023-07-18T03:39:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.226+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/528994893 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d8246e0 0x560f5d833d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.226+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/528994893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d67f6b0 0x560f5d827550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.226+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/528994893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d823840 0x560f5d827a90 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.226+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/528994893 >> 172.21.15.40:0/528994893 conn(0x560f5d818be0 msgr2=0x560f5d83a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.226+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/528994893 shutdown_connections 2023-07-18T03:39:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.226+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/528994893 wait complete. 2023-07-18T03:39:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b2f3d7b80 1 -- start start 2023-07-18T03:39:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b2f3d7b80 1 Processor -- start 2023-07-18T03:39:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b2f3d7b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d67f6b0 0x560f5d82fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b2f3d7b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d823840 0x560f5d82ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b2f3d7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d8246e0 0x560f5d832fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b2f3d7b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5d8334f0 con 0x560f5d823840 2023-07-18T03:39:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b2f3d7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5d833660 con 0x560f5d8246e0 2023-07-18T03:39:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b2f3d7b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f5d833820 con 0x560f5d67f6b0 2023-07-18T03:39:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b2f3d7b80 10 client.0 _ll_register_callbacks cb 0x560f5d712700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:39:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b1bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d823840 0x560f5d82ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b1bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d823840 0x560f5d82ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32916/0 (socket says 172.21.15.40:32916) 2023-07-18T03:39:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b1bfff700 1 -- 172.21.15.40:0/3740751036 learned_addr learned my addr 172.21.15.40:0/3740751036 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b217ee700 1 --2- 172.21.15.40:0/3740751036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d8246e0 0x560f5d832fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b20fed700 1 --2- 172.21.15.40:0/3740751036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d67f6b0 0x560f5d82fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b1bfff700 1 -- 172.21.15.40:0/3740751036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d67f6b0 msgr2=0x560f5d82fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b1bfff700 1 --2- 172.21.15.40:0/3740751036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d67f6b0 0x560f5d82fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b1bfff700 1 -- 172.21.15.40:0/3740751036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d8246e0 msgr2=0x560f5d832fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b1bfff700 1 --2- 172.21.15.40:0/3740751036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d8246e0 0x560f5d832fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.227+0000 7f9b1bfff700 1 -- 172.21.15.40:0/3740751036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b14005e70 con 0x560f5d823840 2023-07-18T03:39:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.228+0000 7f9b1bfff700 1 --2- 172.21.15.40:0/3740751036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d823840 0x560f5d82ff90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800c2d0 tx=0x7f9b08010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65e90991b57e613 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.228+0000 7f9afbfff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:39:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.228+0000 7f9afbfff700 1 -- 172.21.15.40:0/3740751036 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b080b6030 con 0x560f5d823840 2023-07-18T03:39:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.228+0000 7f9afbfff700 1 -- 172.21.15.40:0/3740751036 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9b080b7070 con 0x560f5d823840 2023-07-18T03:39:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.228+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3740751036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b1400f040 con 0x560f5d823840 2023-07-18T03:39:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.228+0000 7f9afbfff700 1 -- 172.21.15.40:0/3740751036 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b080ca2c0 con 0x560f5d823840 2023-07-18T03:39:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.228+0000 7f9b2f3d7b80 10 client.16716 fetch_fsmap learned FSMap version 62 2023-07-18T03:39:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.228+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3740751036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x560f5d5a2d80 con 0x560f5d823840 2023-07-18T03:39:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.228+0000 7f9afbfff700 1 -- 172.21.15.40:0/3740751036 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b1400f040 con 0x560f5d823840 2023-07-18T03:39:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.228+0000 7f9afbfff700 1 -- 172.21.15.40:0/3740751036 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 62) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f9b080a1e50 con 0x560f5d823840 2023-07-18T03:39:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.228+0000 7f9b2f3d7b80 10 client.16716 fetch_fsmap finished waiting for FSMap version 62 2023-07-18T03:39:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.228+0000 7f9b2f3d7b80 10 client.16716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:39:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.228+0000 7f9b2f3d7b80 20 client.16716 populate_metadata read hostname 'smithi040' 2023-07-18T03:39:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.228+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/3740751036 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560f5d88d240 0x560f5d88f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.228+0000 7f9b2f3d7b80 4 client.16716 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:39:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.228+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3740751036 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560f5d72fff0 con 0x560f5d88d240 2023-07-18T03:39:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.229+0000 7f9b20fed700 1 --2- 172.21.15.40:0/3740751036 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560f5d88d240 0x560f5d88f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.229+0000 7f9afbfff700 10 client.16716 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.229+0000 7f9b20fed700 1 --2- 172.21.15.40:0/3740751036 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560f5d88d240 0x560f5d88f650 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7939a5c18ab82ae server_cookie=e7d81bf59aca756a in_seq=0 out_seq=0 2023-07-18T03:39:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.241+0000 7f9afbfff700 1 -- 172.21.15.40:0/3740751036 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560f5d72fff0 con 0x560f5d88d240 2023-07-18T03:39:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.241+0000 7f9afbfff700 10 client.16716 handle_command_reply: tid=0 2023-07-18T03:39:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.241+0000 7f9b2f3d7b80 1 client.16716 shutdown 2023-07-18T03:39:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.241+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3740751036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d823840 msgr2=0x560f5d82ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.241+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/3740751036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d823840 0x560f5d82ff90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800c2d0 tx=0x7f9b08010ba0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.241+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3740751036 shutdown_connections 2023-07-18T03:39:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.241+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/3740751036 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x560f5d88d240 0x560f5d88f650 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.241+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/3740751036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f5d8246e0 0x560f5d832fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.241+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/3740751036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f5d67f6b0 0x560f5d82fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.241+0000 7f9b2f3d7b80 1 --2- 172.21.15.40:0/3740751036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x560f5d823840 0x560f5d82ff90 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.241+0000 7f9afbfff700 0 client.16716 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.241+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3740751036 >> 172.21.15.40:0/3740751036 conn(0x560f5d818be0 msgr2=0x560f5d827550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.241+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3740751036 shutdown_connections 2023-07-18T03:39:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.241+0000 7f9b2f3d7b80 1 -- 172.21.15.40:0/3740751036 wait complete. 2023-07-18T03:39:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.241+0000 7f9b2f3d7b80 20 client.16716 trim_cache size 0 max 16384 2023-07-18T03:39:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:39:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:39:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:39:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:39:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.242+0000 7f9b23251700 1 -- 172.21.15.40:0/1852499113 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9b0c0a11d0 msgr2=0x7f9b0c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.242+0000 7f9b23251700 1 --2- 172.21.15.40:0/1852499113 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9b0c0a11d0 0x7f9b0c0a3680 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000b2e0 tx=0x7f9b10005fb0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.242+0000 7f9b23251700 1 -- 172.21.15.40:0/1852499113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c0ffc90 msgr2=0x7f9b1c1980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.242+0000 7f9b23251700 1 --2- 172.21.15.40:0/1852499113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c0ffc90 0x7f9b1c1980b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800e9a0 tx=0x7f9b0800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.242+0000 7f9b23251700 1 -- 172.21.15.40:0/1852499113 shutdown_connections 2023-07-18T03:39:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.242+0000 7f9b23251700 1 --2- 172.21.15.40:0/1852499113 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f9b0c0a11d0 0x7f9b0c0a3680 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.242+0000 7f9b23251700 1 --2- 172.21.15.40:0/1852499113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b1c100640 0x7f9b1c198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.242+0000 7f9b23251700 1 --2- 172.21.15.40:0/1852499113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b1c0fea00 0x7f9b1c197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.242+0000 7f9b23251700 1 --2- 172.21.15.40:0/1852499113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b1c0ffc90 0x7f9b1c1980b0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.242+0000 7f9b23251700 1 -- 172.21.15.40:0/1852499113 >> 172.21.15.40:0/1852499113 conn(0x7f9b1c0fa000 msgr2=0x7f9b1c067920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.242+0000 7f9b23251700 1 -- 172.21.15.40:0/1852499113 shutdown_connections 2023-07-18T03:39:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:39:48.242+0000 7f9b23251700 1 -- 172.21.15.40:0/1852499113 wait complete. 2023-07-18T03:39:49.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:49 smithi134 ceph-mon[98216]: pgmap v562: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 268 op/s 2023-07-18T03:39:49.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:49 smithi040 ceph-mon[96439]: pgmap v562: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 268 op/s 2023-07-18T03:39:49.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:49 smithi064 ceph-mon[98059]: pgmap v562: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 268 op/s 2023-07-18T03:39:51.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:51 smithi134 ceph-mon[98216]: pgmap v563: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2023-07-18T03:39:51.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:51 smithi040 ceph-mon[96439]: pgmap v563: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2023-07-18T03:39:51.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:51 smithi064 ceph-mon[98059]: pgmap v563: 97 pgs: 97 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2023-07-18T03:39:53.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:52 smithi134 ceph-mon[98216]: pgmap v564: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 268 op/s 2023-07-18T03:39:53.315 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:39:53.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:52 smithi040 ceph-mon[96439]: pgmap v564: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 268 op/s 2023-07-18T03:39:53.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:52 smithi064 ceph-mon[98059]: pgmap v564: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 268 op/s 2023-07-18T03:39:53.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.451+0000 7f7d7475f700 1 -- 172.21.15.40:0/19515513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d6c100560 msgr2=0x7f7d6c106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.451+0000 7f7d7475f700 1 --2- 172.21.15.40:0/19515513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d6c100560 0x7f7d6c106e20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400b0a0 tx=0x7f7d6400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.451+0000 7f7d7475f700 1 -- 172.21.15.40:0/19515513 shutdown_connections 2023-07-18T03:39:53.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.451+0000 7f7d7475f700 1 --2- 172.21.15.40:0/19515513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d6c100560 0x7f7d6c106e20 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.451+0000 7f7d7475f700 1 --2- 172.21.15.40:0/19515513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d6c0fe9e0 0x7f7d6c0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.451+0000 7f7d7475f700 1 --2- 172.21.15.40:0/19515513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0ffbd0 0x7f7d6c100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.451+0000 7f7d7475f700 1 -- 172.21.15.40:0/19515513 >> 172.21.15.40:0/19515513 conn(0x7f7d6c0fa000 msgr2=0x7f7d6c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:53.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.451+0000 7f7d7475f700 1 -- 172.21.15.40:0/19515513 shutdown_connections 2023-07-18T03:39:53.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.451+0000 7f7d7475f700 1 -- 172.21.15.40:0/19515513 wait complete. 2023-07-18T03:39:53.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.452+0000 7f7d7475f700 1 Processor -- start 2023-07-18T03:39:53.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.452+0000 7f7d7475f700 1 -- start start 2023-07-18T03:39:53.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.452+0000 7f7d7475f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d6c0fe9e0 0x7f7d6c194830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.452+0000 7f7d7475f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0ffbd0 0x7f7d6c194d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.452+0000 7f7d7475f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d6c100560 0x7f7d6c19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.452+0000 7f7d7475f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c19c3b0 con 0x7f7d6c0fe9e0 2023-07-18T03:39:53.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.452+0000 7f7d7475f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c19c4f0 con 0x7f7d6c0ffbd0 2023-07-18T03:39:53.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.452+0000 7f7d7475f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c19c680 con 0x7f7d6c100560 2023-07-18T03:39:53.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.453+0000 7f7d724fb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d6c0fe9e0 0x7f7d6c194830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.453+0000 7f7d724fb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d6c0fe9e0 0x7f7d6c194830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32928/0 (socket says 172.21.15.40:32928) 2023-07-18T03:39:53.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.453+0000 7f7d724fb700 1 -- 172.21.15.40:0/1476864155 learned_addr learned my addr 172.21.15.40:0/1476864155 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:53.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.453+0000 7f7d71cfa700 1 --2- 172.21.15.40:0/1476864155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0ffbd0 0x7f7d6c194d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.453+0000 7f7d72cfc700 1 --2- 172.21.15.40:0/1476864155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d6c100560 0x7f7d6c19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.453+0000 7f7d724fb700 1 -- 172.21.15.40:0/1476864155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d6c100560 msgr2=0x7f7d6c19bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.453+0000 7f7d724fb700 1 --2- 172.21.15.40:0/1476864155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d6c100560 0x7f7d6c19bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.453+0000 7f7d724fb700 1 -- 172.21.15.40:0/1476864155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0ffbd0 msgr2=0x7f7d6c194d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.453+0000 7f7d724fb700 1 --2- 172.21.15.40:0/1476864155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0ffbd0 0x7f7d6c194d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.453+0000 7f7d724fb700 1 -- 172.21.15.40:0/1476864155 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d64009d00 con 0x7f7d6c0fe9e0 2023-07-18T03:39:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.453+0000 7f7d724fb700 1 --2- 172.21.15.40:0/1476864155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d6c0fe9e0 0x7f7d6c194830 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00e970 tx=0x7f7d5c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.453+0000 7f7d637fe700 1 -- 172.21.15.40:0/1476864155 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c009790 con 0x7f7d6c0fe9e0 2023-07-18T03:39:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.453+0000 7f7d637fe700 1 -- 172.21.15.40:0/1476864155 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d5c00c7e0 con 0x7f7d6c0fe9e0 2023-07-18T03:39:53.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.453+0000 7f7d637fe700 1 -- 172.21.15.40:0/1476864155 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c003c70 con 0x7f7d6c0fe9e0 2023-07-18T03:39:53.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.453+0000 7f7d7475f700 1 -- 172.21.15.40:0/1476864155 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6c19c960 con 0x7f7d6c0fe9e0 2023-07-18T03:39:53.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.453+0000 7f7d7475f700 1 -- 172.21.15.40:0/1476864155 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6c19ce70 con 0x7f7d6c0fe9e0 2023-07-18T03:39:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.455+0000 7f7d637fe700 1 -- 172.21.15.40:0/1476864155 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7d5c00cc50 con 0x7f7d6c0fe9e0 2023-07-18T03:39:53.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.456+0000 7f7d637fe700 1 --2- 172.21.15.40:0/1476864155 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7d580a11d0 0x7f7d580a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.456+0000 7f7d71cfa700 1 --2- 172.21.15.40:0/1476864155 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7d580a11d0 0x7f7d580a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.456+0000 7f7d637fe700 1 -- 172.21.15.40:0/1476864155 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7d5c014070 con 0x7f7d6c0fe9e0 2023-07-18T03:39:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.457+0000 7f7d71cfa700 1 --2- 172.21.15.40:0/1476864155 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7d580a11d0 0x7f7d580a3680 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009c60 tx=0x7f7d68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.457+0000 7f7d808e5b80 1 Processor -- start 2023-07-18T03:39:53.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d808e5b80 1 -- start start 2023-07-18T03:39:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d808e5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e74d0 0x55f41b9e78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d808e5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e7e20 0x55f41b9e86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d808e5b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e8c70 0x55f41b9e90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d808e5b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f41b8eccb0 con 0x55f41b9e8c70 2023-07-18T03:39:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d808e5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f41b93fa40 con 0x55f41b9e7e20 2023-07-18T03:39:53.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d808e5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f41b938ed0 con 0x55f41b9e74d0 2023-07-18T03:39:53.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d72cfc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e8c70 0x55f41b9e90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d71cfa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e7e20 0x55f41b9e86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d724fb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e74d0 0x55f41b9e78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d72cfc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e8c70 0x55f41b9e90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32940/0 (socket says 172.21.15.40:32940) 2023-07-18T03:39:53.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d72cfc700 1 -- 172.21.15.40:0/3711794910 learned_addr learned my addr 172.21.15.40:0/3711794910 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:53.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d72cfc700 1 -- 172.21.15.40:0/3711794910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e74d0 msgr2=0x55f41b9e78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:53.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d72cfc700 1 --2- 172.21.15.40:0/3711794910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e74d0 0x55f41b9e78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d72cfc700 1 -- 172.21.15.40:0/3711794910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e7e20 msgr2=0x55f41b9e86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:53.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d72cfc700 1 --2- 172.21.15.40:0/3711794910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e7e20 0x55f41b9e86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d72cfc700 1 -- 172.21.15.40:0/3711794910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f41b766de0 con 0x55f41b9e8c70 2023-07-18T03:39:53.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d71cfa700 1 --2- 172.21.15.40:0/3711794910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e7e20 0x55f41b9e86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:53.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d72cfc700 1 --2- 172.21.15.40:0/3711794910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e8c70 0x55f41b9e90e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009c60 tx=0x7f7d64009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eecad510d1c3607 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:53.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.458+0000 7f7d60ff9700 1 -- 172.21.15.40:0/3711794910 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6400e040 con 0x55f41b9e8c70 2023-07-18T03:39:53.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.459+0000 7f7d60ff9700 1 -- 172.21.15.40:0/3711794910 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d64007bc0 con 0x55f41b9e8c70 2023-07-18T03:39:53.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.459+0000 7f7d60ff9700 1 -- 172.21.15.40:0/3711794910 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64003d30 con 0x55f41b9e8c70 2023-07-18T03:39:53.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.459+0000 7f7d808e5b80 1 -- 172.21.15.40:0/3711794910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e8c70 msgr2=0x55f41b9e90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.459+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/3711794910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e8c70 0x55f41b9e90e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009c60 tx=0x7f7d64009c90 comp rx=0 tx=0).stop 2023-07-18T03:39:53.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.459+0000 7f7d808e5b80 1 -- 172.21.15.40:0/3711794910 shutdown_connections 2023-07-18T03:39:53.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.459+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/3711794910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e8c70 0x55f41b9e90e0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.459+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/3711794910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e74d0 0x55f41b9e78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.459+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/3711794910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e7e20 0x55f41b9e86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.459+0000 7f7d808e5b80 1 -- 172.21.15.40:0/3711794910 >> 172.21.15.40:0/3711794910 conn(0x55f41b9dcda0 msgr2=0x55f41b9e21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:53.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.459+0000 7f7d808e5b80 1 -- 172.21.15.40:0/3711794910 shutdown_connections 2023-07-18T03:39:53.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.459+0000 7f7d808e5b80 1 -- 172.21.15.40:0/3711794910 wait complete. 2023-07-18T03:39:53.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.460+0000 7f7d808e5b80 1 -- start start 2023-07-18T03:39:53.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.460+0000 7f7d808e5b80 1 Processor -- start 2023-07-18T03:39:53.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.460+0000 7f7d808e5b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e74d0 0x55f41b9f1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.460+0000 7f7d808e5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e7e20 0x55f41b9f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.460+0000 7f7d808e5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e8c70 0x55f41b9f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.460+0000 7f7d808e5b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f41b8d8ac0 con 0x55f41b9e74d0 2023-07-18T03:39:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.460+0000 7f7d808e5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f41b8d87c0 con 0x55f41b9e8c70 2023-07-18T03:39:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.460+0000 7f7d808e5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f41b7bc920 con 0x55f41b9e7e20 2023-07-18T03:39:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.460+0000 7f7d808e5b80 10 client.0 _ll_register_callbacks cb 0x55f41b8d6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:39:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.460+0000 7f7d724fb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e74d0 0x55f41b9f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.460+0000 7f7d724fb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e74d0 0x55f41b9f1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32942/0 (socket says 172.21.15.40:32942) 2023-07-18T03:39:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.460+0000 7f7d724fb700 1 -- 172.21.15.40:0/770522357 learned_addr learned my addr 172.21.15.40:0/770522357 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.460+0000 7f7d72cfc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e8c70 0x55f41b9f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.460+0000 7f7d71cfa700 1 --2- 172.21.15.40:0/770522357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e7e20 0x55f41b9f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d724fb700 1 -- 172.21.15.40:0/770522357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e7e20 msgr2=0x55f41b9f15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d724fb700 1 --2- 172.21.15.40:0/770522357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e7e20 0x55f41b9f15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d724fb700 1 -- 172.21.15.40:0/770522357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e8c70 msgr2=0x55f41b9f85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:53.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d724fb700 1 --2- 172.21.15.40:0/770522357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e8c70 0x55f41b9f85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d724fb700 1 -- 172.21.15.40:0/770522357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d68017070 con 0x55f41b9e74d0 2023-07-18T03:39:53.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d72cfc700 1 --2- 172.21.15.40:0/770522357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e8c70 0x55f41b9f85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:53.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d71cfa700 1 --2- 172.21.15.40:0/770522357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e7e20 0x55f41b9f15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:53.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d724fb700 1 --2- 172.21.15.40:0/770522357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e74d0 0x55f41b9f1070 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c099dc0 tx=0x7f7d5c003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad54bc49f6db0a60 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:53.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:39:53.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d55ffb700 1 -- 172.21.15.40:0/770522357 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c09a290 con 0x55f41b9e74d0 2023-07-18T03:39:53.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d55ffb700 1 -- 172.21.15.40:0/770522357 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7d5c0b1070 con 0x55f41b9e74d0 2023-07-18T03:39:53.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d808e5b80 1 -- 172.21.15.40:0/770522357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f41b8924f0 con 0x55f41b9e74d0 2023-07-18T03:39:53.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d55ffb700 1 -- 172.21.15.40:0/770522357 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d5c0b8250 con 0x55f41b9e74d0 2023-07-18T03:39:53.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d808e5b80 10 client.16728 fetch_fsmap learned FSMap version 62 2023-07-18T03:39:53.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d808e5b80 1 -- 172.21.15.40:0/770522357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x7f7d6c107750 con 0x55f41b9e74d0 2023-07-18T03:39:53.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d55ffb700 1 -- 172.21.15.40:0/770522357 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f41b8924f0 con 0x55f41b9e74d0 2023-07-18T03:39:53.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d55ffb700 1 -- 172.21.15.40:0/770522357 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 62) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f7d5c0a1df0 con 0x55f41b9e74d0 2023-07-18T03:39:53.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d808e5b80 10 client.16728 fetch_fsmap finished waiting for FSMap version 62 2023-07-18T03:39:53.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d808e5b80 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:39:53.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.461+0000 7f7d808e5b80 20 client.16728 populate_metadata read hostname 'smithi040' 2023-07-18T03:39:53.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.462+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/770522357 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55f41b9fce80 0x55f41b9ff290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.462+0000 7f7d808e5b80 4 client.16728 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:39:53.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.462+0000 7f7d808e5b80 1 -- 172.21.15.40:0/770522357 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f41b931eb0 con 0x55f41b9fce80 2023-07-18T03:39:53.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.462+0000 7f7d71cfa700 1 --2- 172.21.15.40:0/770522357 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55f41b9fce80 0x55f41b9ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.462+0000 7f7d55ffb700 10 client.16728 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:53.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.462+0000 7f7d71cfa700 1 --2- 172.21.15.40:0/770522357 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55f41b9fce80 0x55f41b9ff290 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfbd8e1cac876160 server_cookie=c89820c5d4c1e7d6 in_seq=0 out_seq=0 2023-07-18T03:39:53.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.463+0000 7f7d55ffb700 1 -- 172.21.15.40:0/770522357 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f41b931eb0 con 0x55f41b9fce80 2023-07-18T03:39:53.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.463+0000 7f7d55ffb700 10 client.16728 handle_command_reply: tid=0 2023-07-18T03:39:53.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.463+0000 7f7d808e5b80 1 client.16728 shutdown 2023-07-18T03:39:53.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.463+0000 7f7d808e5b80 1 -- 172.21.15.40:0/770522357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e74d0 msgr2=0x55f41b9f1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.463+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/770522357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e74d0 0x55f41b9f1070 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c099dc0 tx=0x7f7d5c003dd0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.463+0000 7f7d808e5b80 1 -- 172.21.15.40:0/770522357 shutdown_connections 2023-07-18T03:39:53.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.463+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/770522357 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55f41b9fce80 0x55f41b9ff290 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.463+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/770522357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e8c70 0x55f41b9f85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.463+0000 7f7d55ffb700 0 client.16728 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:53.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.463+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/770522357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e74d0 0x55f41b9f1070 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.463+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/770522357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e7e20 0x55f41b9f15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.463+0000 7f7d808e5b80 1 -- 172.21.15.40:0/770522357 >> 172.21.15.40:0/770522357 conn(0x55f41b9dcda0 msgr2=0x55f41b9e18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:53.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.463+0000 7f7d808e5b80 1 -- 172.21.15.40:0/770522357 shutdown_connections 2023-07-18T03:39:53.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.463+0000 7f7d808e5b80 1 -- 172.21.15.40:0/770522357 wait complete. 2023-07-18T03:39:53.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.463+0000 7f7d808e5b80 20 client.16728 trim_cache size 0 max 16384 2023-07-18T03:39:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.480+0000 7f7d808e5b80 1 Processor -- start 2023-07-18T03:39:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.480+0000 7f7d808e5b80 1 -- start start 2023-07-18T03:39:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.480+0000 7f7d808e5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e74d0 0x55f41b9de310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.480+0000 7f7d808e5b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e7e20 0x55f41b9f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.480+0000 7f7d808e5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e8c70 0x55f41ba03bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.480+0000 7f7d808e5b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f41b9fa260 con 0x55f41b9e7e20 2023-07-18T03:39:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.480+0000 7f7d808e5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f41ba04130 con 0x55f41b9e74d0 2023-07-18T03:39:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.480+0000 7f7d808e5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f41b9fcb80 con 0x55f41b9e8c70 2023-07-18T03:39:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.481+0000 7f7d71cfa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e7e20 0x55f41b9f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.481+0000 7f7d724fb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e74d0 0x55f41b9de310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.481+0000 7f7d71cfa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e7e20 0x55f41b9f85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32952/0 (socket says 172.21.15.40:32952) 2023-07-18T03:39:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.481+0000 7f7d72cfc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e8c70 0x55f41ba03bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.481+0000 7f7d71cfa700 1 -- 172.21.15.40:0/4287649161 learned_addr learned my addr 172.21.15.40:0/4287649161 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.481+0000 7f7d71cfa700 1 -- 172.21.15.40:0/4287649161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e8c70 msgr2=0x55f41ba03bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.481+0000 7f7d71cfa700 1 --2- 172.21.15.40:0/4287649161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e8c70 0x55f41ba03bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.481+0000 7f7d71cfa700 1 -- 172.21.15.40:0/4287649161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e74d0 msgr2=0x55f41b9de310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.481+0000 7f7d71cfa700 1 --2- 172.21.15.40:0/4287649161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e74d0 0x55f41b9de310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.481+0000 7f7d71cfa700 1 -- 172.21.15.40:0/4287649161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f41b74fc60 con 0x55f41b9e7e20 2023-07-18T03:39:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.481+0000 7f7d71cfa700 1 --2- 172.21.15.40:0/4287649161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e7e20 0x55f41b9f85b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7d68013010 tx=0x7f7d68014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e970f34049010e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.481+0000 7f7d617fa700 1 -- 172.21.15.40:0/4287649161 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d68022070 con 0x55f41b9e7e20 2023-07-18T03:39:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.481+0000 7f7d617fa700 1 -- 172.21.15.40:0/4287649161 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d68023040 con 0x55f41b9e7e20 2023-07-18T03:39:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.481+0000 7f7d617fa700 1 -- 172.21.15.40:0/4287649161 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6801d830 con 0x55f41b9e7e20 2023-07-18T03:39:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.482+0000 7f7d808e5b80 1 -- 172.21.15.40:0/4287649161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e7e20 msgr2=0x55f41b9f85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.482+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/4287649161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e7e20 0x55f41b9f85b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7d68013010 tx=0x7f7d68014000 comp rx=0 tx=0).stop 2023-07-18T03:39:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.482+0000 7f7d808e5b80 1 -- 172.21.15.40:0/4287649161 shutdown_connections 2023-07-18T03:39:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.482+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/4287649161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e8c70 0x55f41ba03bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.482+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/4287649161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e74d0 0x55f41b9de310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.482+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/4287649161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e7e20 0x55f41b9f85b0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.482+0000 7f7d808e5b80 1 -- 172.21.15.40:0/4287649161 >> 172.21.15.40:0/4287649161 conn(0x55f41b9dcda0 msgr2=0x55f41b9e21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.482+0000 7f7d808e5b80 1 -- 172.21.15.40:0/4287649161 shutdown_connections 2023-07-18T03:39:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.482+0000 7f7d808e5b80 1 -- 172.21.15.40:0/4287649161 wait complete. 2023-07-18T03:39:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.482+0000 7f7d808e5b80 1 -- start start 2023-07-18T03:39:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.482+0000 7f7d808e5b80 1 Processor -- start 2023-07-18T03:39:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.482+0000 7f7d808e5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e74d0 0x55f41b9f8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d808e5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e7e20 0x55f41b9f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d808e5b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e8c70 0x55f41b9f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d808e5b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f41b9f57c0 con 0x55f41b9e8c70 2023-07-18T03:39:53.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d808e5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f41b9f8780 con 0x55f41b9e7e20 2023-07-18T03:39:53.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d808e5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f41b9f8910 con 0x55f41b9e74d0 2023-07-18T03:39:53.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d808e5b80 10 client.0 _ll_register_callbacks cb 0x55f41b8d6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:39:53.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d724fb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e74d0 0x55f41b9f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d72cfc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e8c70 0x55f41b9f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d71cfa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e7e20 0x55f41b9f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d724fb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e74d0 0x55f41b9f8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50216/0 (socket says 172.21.15.40:50216) 2023-07-18T03:39:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d724fb700 1 -- 172.21.15.40:0/3199434807 learned_addr learned my addr 172.21.15.40:0/3199434807 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:39:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d72cfc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e8c70 0x55f41b9f50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32960/0 (socket says 172.21.15.40:32960) 2023-07-18T03:39:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d71cfa700 1 -- 172.21.15.40:0/3199434807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e74d0 msgr2=0x55f41b9f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d71cfa700 1 --2- 172.21.15.40:0/3199434807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e74d0 0x55f41b9f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d71cfa700 1 -- 172.21.15.40:0/3199434807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e8c70 msgr2=0x55f41b9f50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:39:53.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d71cfa700 1 --2- 172.21.15.40:0/3199434807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e8c70 0x55f41b9f50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d71cfa700 1 -- 172.21.15.40:0/3199434807 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6802bab0 con 0x55f41b9e7e20 2023-07-18T03:39:53.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d724fb700 1 --2- 172.21.15.40:0/3199434807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e74d0 0x55f41b9f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:53.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d72cfc700 1 --2- 172.21.15.40:0/3199434807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e8c70 0x55f41b9f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:39:53.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d71cfa700 1 --2- 172.21.15.40:0/3199434807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e7e20 0x55f41b9f4ba0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7d6802e040 tx=0x7f7d680044c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b402c4b8f37a4e50 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:39:53.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:39:53.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d55ffb700 1 -- 172.21.15.40:0/3199434807 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d68022050 con 0x55f41b9e7e20 2023-07-18T03:39:53.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d55ffb700 1 -- 172.21.15.40:0/3199434807 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7d6802b2c0 con 0x55f41b9e7e20 2023-07-18T03:39:53.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.483+0000 7f7d808e5b80 1 -- 172.21.15.40:0/3199434807 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d64007bc0 con 0x55f41b9e7e20 2023-07-18T03:39:53.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.484+0000 7f7d55ffb700 1 -- 172.21.15.40:0/3199434807 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d6802b480 con 0x55f41b9e7e20 2023-07-18T03:39:53.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.484+0000 7f7d808e5b80 10 client.26390 fetch_fsmap learned FSMap version 62 2023-07-18T03:39:53.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.484+0000 7f7d808e5b80 1 -- 172.21.15.40:0/3199434807 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x55f41b78d250 con 0x55f41b9e7e20 2023-07-18T03:39:53.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.484+0000 7f7d55ffb700 1 -- 172.21.15.40:0/3199434807 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d64007bc0 con 0x55f41b9e7e20 2023-07-18T03:39:53.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.484+0000 7f7d55ffb700 1 -- 172.21.15.40:0/3199434807 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 62) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f7d68004ac0 con 0x55f41b9e7e20 2023-07-18T03:39:53.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.484+0000 7f7d808e5b80 10 client.26390 fetch_fsmap finished waiting for FSMap version 62 2023-07-18T03:39:53.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.484+0000 7f7d808e5b80 10 client.26390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:39:53.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.484+0000 7f7d808e5b80 20 client.26390 populate_metadata read hostname 'smithi040' 2023-07-18T03:39:53.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.484+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/3199434807 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55f41ba52540 0x55f41ba54950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:39:53.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.484+0000 7f7d808e5b80 4 client.26390 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:39:53.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.484+0000 7f7d808e5b80 1 -- 172.21.15.40:0/3199434807 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f41b8f4050 con 0x55f41ba52540 2023-07-18T03:39:53.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.485+0000 7f7d724fb700 1 --2- 172.21.15.40:0/3199434807 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55f41ba52540 0x55f41ba54950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:39:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.485+0000 7f7d724fb700 1 --2- 172.21.15.40:0/3199434807 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55f41ba52540 0x55f41ba54950 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43f2ab45d1cb5ee2 server_cookie=4ef0552f4d1145cd in_seq=0 out_seq=0 2023-07-18T03:39:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.485+0000 7f7d55ffb700 10 client.26390 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.485+0000 7f7d55ffb700 1 -- 172.21.15.40:0/3199434807 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f41b8f4050 con 0x55f41ba52540 2023-07-18T03:39:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.485+0000 7f7d55ffb700 10 client.26390 handle_command_reply: tid=0 2023-07-18T03:39:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.485+0000 7f7d808e5b80 1 client.26390 shutdown 2023-07-18T03:39:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.485+0000 7f7d808e5b80 1 -- 172.21.15.40:0/3199434807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e7e20 msgr2=0x55f41b9f4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.485+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/3199434807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e7e20 0x55f41b9f4ba0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7d6802e040 tx=0x7f7d680044c0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.485+0000 7f7d808e5b80 1 -- 172.21.15.40:0/3199434807 shutdown_connections 2023-07-18T03:39:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.485+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/3199434807 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55f41ba52540 0x55f41ba54950 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.485+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/3199434807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55f41b9e8c70 0x55f41b9f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.485+0000 7f7d55ffb700 0 client.26390 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:39:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.485+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/3199434807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f41b9e74d0 0x55f41b9f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.485+0000 7f7d808e5b80 1 --2- 172.21.15.40:0/3199434807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f41b9e7e20 0x55f41b9f4ba0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.485+0000 7f7d808e5b80 1 -- 172.21.15.40:0/3199434807 >> 172.21.15.40:0/3199434807 conn(0x55f41b9dcda0 msgr2=0x55f41b9eba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:53.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.486+0000 7f7d808e5b80 1 -- 172.21.15.40:0/3199434807 shutdown_connections 2023-07-18T03:39:53.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.486+0000 7f7d808e5b80 1 -- 172.21.15.40:0/3199434807 wait complete. 2023-07-18T03:39:53.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.486+0000 7f7d808e5b80 20 client.26390 trim_cache size 0 max 16384 2023-07-18T03:39:53.495 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:39:53.495 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-18T03:39:53.495 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:39:53.495 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:39:53.496 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:39:53.496 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:39:53.496 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:39:53.496 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:39:53.496 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:39:53.496 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:39:53.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.486+0000 7f7d7475f700 1 -- 172.21.15.40:0/1476864155 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7d580a11d0 msgr2=0x7f7d580a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.486+0000 7f7d7475f700 1 --2- 172.21.15.40:0/1476864155 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7d580a11d0 0x7f7d580a3680 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009c60 tx=0x7f7d68009260 comp rx=0 tx=0).stop 2023-07-18T03:39:53.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.486+0000 7f7d7475f700 1 -- 172.21.15.40:0/1476864155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d6c0fe9e0 msgr2=0x7f7d6c194830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:39:53.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.486+0000 7f7d7475f700 1 --2- 172.21.15.40:0/1476864155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d6c0fe9e0 0x7f7d6c194830 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00e970 tx=0x7f7d5c00ec80 comp rx=0 tx=0).stop 2023-07-18T03:39:53.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.486+0000 7f7d7475f700 1 -- 172.21.15.40:0/1476864155 shutdown_connections 2023-07-18T03:39:53.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.486+0000 7f7d7475f700 1 --2- 172.21.15.40:0/1476864155 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7d580a11d0 0x7f7d580a3680 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.486+0000 7f7d7475f700 1 --2- 172.21.15.40:0/1476864155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d6c100560 0x7f7d6c19bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.486+0000 7f7d7475f700 1 --2- 172.21.15.40:0/1476864155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d6c0fe9e0 0x7f7d6c194830 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.486+0000 7f7d7475f700 1 --2- 172.21.15.40:0/1476864155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d6c0ffbd0 0x7f7d6c194d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:39:53.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.486+0000 7f7d7475f700 1 -- 172.21.15.40:0/1476864155 >> 172.21.15.40:0/1476864155 conn(0x7f7d6c0fa000 msgr2=0x7f7d6c103010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:39:53.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.486+0000 7f7d7475f700 1 -- 172.21.15.40:0/1476864155 shutdown_connections 2023-07-18T03:39:53.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:39:53.486+0000 7f7d7475f700 1 -- 172.21.15.40:0/1476864155 wait complete. 2023-07-18T03:39:53.500 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:39:55.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:55 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:55.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:55 smithi134 ceph-mon[98216]: pgmap v565: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 217 op/s 2023-07-18T03:39:55.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:55 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:39:55.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:55 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:55.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:55 smithi040 ceph-mon[96439]: pgmap v565: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 217 op/s 2023-07-18T03:39:55.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:55 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:39:55.608 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:55 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:39:55.608 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:55 smithi064 ceph-mon[98059]: pgmap v565: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 217 op/s 2023-07-18T03:39:55.608 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:55 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:39:56.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:56 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:39:56.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:56 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:39:56.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:56 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:39:56.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:56 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:39:56.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:56 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:39:56.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:56 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:39:56.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:56 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:39:56.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:56 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:39:56.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:56 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:39:57.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:57 smithi134 ceph-mon[98216]: pgmap v566: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 759 KiB/s wr, 226 op/s 2023-07-18T03:39:57.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:57 smithi040 ceph-mon[96439]: pgmap v566: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 759 KiB/s wr, 226 op/s 2023-07-18T03:39:57.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:57 smithi064 ceph-mon[98059]: pgmap v566: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 759 KiB/s wr, 226 op/s 2023-07-18T03:39:59.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:39:59 smithi134 ceph-mon[98216]: pgmap v567: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1014 KiB/s wr, 269 op/s 2023-07-18T03:39:59.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:39:59 smithi040 ceph-mon[96439]: pgmap v567: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1014 KiB/s wr, 269 op/s 2023-07-18T03:39:59.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:39:59 smithi064 ceph-mon[98059]: pgmap v567: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 1014 KiB/s wr, 269 op/s 2023-07-18T03:40:00.305 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:00 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mon-a[96415]: 2023-07-18T03:39:59.999+0000 7f858afe7700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-07-18T03:40:00.305 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:00 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mon-a[96415]: 2023-07-18T03:39:59.999+0000 7f858afe7700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-07-18T03:40:00.305 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:00 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mon-a[96415]: 2023-07-18T03:39:59.999+0000 7f858afe7700 -1 log_channel(cluster) log [ERR] : mds.i(mds.1): Metadata damage detected 2023-07-18T03:40:00.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:00 smithi134 ceph-mon[98216]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-07-18T03:40:00.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:00 smithi134 ceph-mon[98216]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-07-18T03:40:00.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:00 smithi134 ceph-mon[98216]: mds.i(mds.1): Metadata damage detected 2023-07-18T03:40:00.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:00 smithi040 ceph-mon[96439]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-07-18T03:40:00.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:00 smithi040 ceph-mon[96439]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-07-18T03:40:00.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:00 smithi040 ceph-mon[96439]: mds.i(mds.1): Metadata damage detected 2023-07-18T03:40:00.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:00 smithi064 ceph-mon[98059]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-07-18T03:40:00.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:00 smithi064 ceph-mon[98059]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-07-18T03:40:00.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:00 smithi064 ceph-mon[98059]: mds.i(mds.1): Metadata damage detected 2023-07-18T03:40:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.396+0000 7f44500f8700 1 -- 172.21.15.40:0/198368979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4448104660 msgr2=0x7f4448106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.396+0000 7f44500f8700 1 --2- 172.21.15.40:0/198368979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4448104660 0x7f4448106b10 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4440009990 tx=0x7f4440009ca0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.396+0000 7f44500f8700 1 -- 172.21.15.40:0/198368979 shutdown_connections 2023-07-18T03:40:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.396+0000 7f44500f8700 1 --2- 172.21.15.40:0/198368979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4448104660 0x7f4448106b10 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.396+0000 7f44500f8700 1 --2- 172.21.15.40:0/198368979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44480ff200 0x7f44481015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.396+0000 7f44500f8700 1 --2- 172.21.15.40:0/198368979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4448101bb0 0x7f4448103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.397+0000 7f44500f8700 1 -- 172.21.15.40:0/198368979 >> 172.21.15.40:0/198368979 conn(0x7f44480f9330 msgr2=0x7f44480fb780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.397+0000 7f44500f8700 1 -- 172.21.15.40:0/198368979 shutdown_connections 2023-07-18T03:40:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.397+0000 7f44500f8700 1 -- 172.21.15.40:0/198368979 wait complete. 2023-07-18T03:40:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.397+0000 7f44500f8700 1 Processor -- start 2023-07-18T03:40:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.397+0000 7f44500f8700 1 -- start start 2023-07-18T03:40:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.398+0000 7f44500f8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44480ff200 0x7f4448198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.398+0000 7f44500f8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4448101bb0 0x7f444819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.398+0000 7f44500f8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4448104660 0x7f444819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.398+0000 7f44500f8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4448194310 con 0x7f4448104660 2023-07-18T03:40:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.398+0000 7f44500f8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4448194480 con 0x7f4448101bb0 2023-07-18T03:40:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.398+0000 7f44500f8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44481945f0 con 0x7f44480ff200 2023-07-18T03:40:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.398+0000 7f444d693700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4448101bb0 0x7f444819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.398+0000 7f444d693700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4448101bb0 0x7f444819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:44530/0 (socket says 172.21.15.40:44530) 2023-07-18T03:40:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.398+0000 7f444d693700 1 -- 172.21.15.40:0/1149310614 learned_addr learned my addr 172.21.15.40:0/1149310614 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.398+0000 7f444e695700 1 --2- 172.21.15.40:0/1149310614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4448104660 0x7f444819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.398+0000 7f444d693700 1 -- 172.21.15.40:0/1149310614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44480ff200 msgr2=0x7f4448198530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:40:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.398+0000 7f444d693700 1 --2- 172.21.15.40:0/1149310614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44480ff200 0x7f4448198530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.398+0000 7f444d693700 1 -- 172.21.15.40:0/1149310614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4448104660 msgr2=0x7f444819cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.398+0000 7f444d693700 1 --2- 172.21.15.40:0/1149310614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4448104660 0x7f444819cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.398+0000 7f444d693700 1 -- 172.21.15.40:0/1149310614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44400095f0 con 0x7f4448101bb0 2023-07-18T03:40:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.399+0000 7f444d693700 1 --2- 172.21.15.40:0/1149310614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4448101bb0 0x7f444819aa80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f443800ef10 tx=0x7f443800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.399+0000 7f4446ffd700 1 -- 172.21.15.40:0/1149310614 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443800ca70 con 0x7f4448101bb0 2023-07-18T03:40:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.399+0000 7f44500f8700 1 -- 172.21.15.40:0/1149310614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4448194870 con 0x7f4448101bb0 2023-07-18T03:40:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.399+0000 7f4446ffd700 1 -- 172.21.15.40:0/1149310614 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f443800f040 con 0x7f4448101bb0 2023-07-18T03:40:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.399+0000 7f4446ffd700 1 -- 172.21.15.40:0/1149310614 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438014480 con 0x7f4448101bb0 2023-07-18T03:40:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.399+0000 7f44500f8700 1 -- 172.21.15.40:0/1149310614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4448194dd0 con 0x7f4448101bb0 2023-07-18T03:40:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.402+0000 7f4446ffd700 1 -- 172.21.15.40:0/1149310614 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4438007500 con 0x7f4448101bb0 2023-07-18T03:40:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.403+0000 7f4446ffd700 1 --2- 172.21.15.40:0/1149310614 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f44300a1150 0x7f44300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.403+0000 7f4446ffd700 1 -- 172.21.15.40:0/1149310614 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f44380ccd10 con 0x7f4448101bb0 2023-07-18T03:40:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.403+0000 7f444de94700 1 --2- 172.21.15.40:0/1149310614 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f44300a1150 0x7f44300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.403+0000 7f445c27eb80 1 Processor -- start 2023-07-18T03:40:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.403+0000 7f444de94700 1 --2- 172.21.15.40:0/1149310614 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f44300a1150 0x7f44300a3600 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f44480fe9d0 tx=0x7f443c00b3f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.404+0000 7f445c27eb80 1 -- start start 2023-07-18T03:40:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.404+0000 7f445c27eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20c8a6b0 0x564f20e2e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.404+0000 7f445c27eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20e2e840 0x564f20e2f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.404+0000 7f445c27eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2f6e0 0x564f20e2fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.404+0000 7f445c27eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f20d33c50 con 0x564f20e2f6e0 2023-07-18T03:40:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.404+0000 7f445c27eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f20d869e0 con 0x564f20e2e840 2023-07-18T03:40:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.404+0000 7f445c27eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f20d7fe70 con 0x564f20c8a6b0 2023-07-18T03:40:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.404+0000 7f444d693700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20e2e840 0x564f20e2f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.404+0000 7f444d693700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20e2e840 0x564f20e2f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:44538/0 (socket says 172.21.15.40:44538) 2023-07-18T03:40:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.404+0000 7f444d693700 1 -- 172.21.15.40:0/677427006 learned_addr learned my addr 172.21.15.40:0/677427006 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.404+0000 7f444e695700 1 --2- 172.21.15.40:0/677427006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2f6e0 0x564f20e2fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.404+0000 7f444de94700 1 --2- 172.21.15.40:0/677427006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20c8a6b0 0x564f20e2e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.404+0000 7f444d693700 1 -- 172.21.15.40:0/677427006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20c8a6b0 msgr2=0x564f20e2e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.404+0000 7f444d693700 1 --2- 172.21.15.40:0/677427006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20c8a6b0 0x564f20e2e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.404+0000 7f444d693700 1 -- 172.21.15.40:0/677427006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2f6e0 msgr2=0x564f20e2fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.404+0000 7f444d693700 1 --2- 172.21.15.40:0/677427006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2f6e0 0x564f20e2fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.404+0000 7f444d693700 1 -- 172.21.15.40:0/677427006 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f20badd80 con 0x564f20e2e840 2023-07-18T03:40:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.405+0000 7f444d693700 1 --2- 172.21.15.40:0/677427006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20e2e840 0x564f20e2f1a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4438096b10 tx=0x7f4438009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=636e9d23d86444ff server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.405+0000 7f442ffff700 1 -- 172.21.15.40:0/677427006 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443809a0a0 con 0x564f20e2e840 2023-07-18T03:40:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.405+0000 7f442ffff700 1 -- 172.21.15.40:0/677427006 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f443809e0d0 con 0x564f20e2e840 2023-07-18T03:40:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.405+0000 7f442ffff700 1 -- 172.21.15.40:0/677427006 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438099590 con 0x564f20e2e840 2023-07-18T03:40:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.406+0000 7f445c27eb80 1 -- 172.21.15.40:0/677427006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20e2e840 msgr2=0x564f20e2f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.406+0000 7f445c27eb80 1 --2- 172.21.15.40:0/677427006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20e2e840 0x564f20e2f1a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4438096b10 tx=0x7f4438009bd0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.406+0000 7f445c27eb80 1 -- 172.21.15.40:0/677427006 shutdown_connections 2023-07-18T03:40:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.406+0000 7f445c27eb80 1 --2- 172.21.15.40:0/677427006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2f6e0 0x564f20e2fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.406+0000 7f445c27eb80 1 --2- 172.21.15.40:0/677427006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20c8a6b0 0x564f20e2e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.406+0000 7f445c27eb80 1 --2- 172.21.15.40:0/677427006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20e2e840 0x564f20e2f1a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.406+0000 7f445c27eb80 1 -- 172.21.15.40:0/677427006 >> 172.21.15.40:0/677427006 conn(0x564f20e23be0 msgr2=0x564f20e28cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.407+0000 7f445c27eb80 1 -- 172.21.15.40:0/677427006 shutdown_connections 2023-07-18T03:40:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.407+0000 7f445c27eb80 1 -- 172.21.15.40:0/677427006 wait complete. 2023-07-18T03:40:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.408+0000 7f445c27eb80 1 -- start start 2023-07-18T03:40:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.408+0000 7f445c27eb80 1 Processor -- start 2023-07-18T03:40:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.408+0000 7f445c27eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20c8a6b0 0x564f20e37840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.408+0000 7f445c27eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20e2e840 0x564f20e37d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.408+0000 7f445c27eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2f6e0 0x564f20e3edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.408+0000 7f445c27eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f20d1fa60 con 0x564f20e2f6e0 2023-07-18T03:40:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.408+0000 7f445c27eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f20d1f760 con 0x564f20e2e840 2023-07-18T03:40:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.409+0000 7f445c27eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f20c038c0 con 0x564f20c8a6b0 2023-07-18T03:40:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.409+0000 7f445c27eb80 10 client.0 _ll_register_callbacks cb 0x564f20d1d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.409+0000 7f444de94700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20c8a6b0 0x564f20e37840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.409+0000 7f444de94700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20c8a6b0 0x564f20e37840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50244/0 (socket says 172.21.15.40:50244) 2023-07-18T03:40:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.409+0000 7f444de94700 1 -- 172.21.15.40:0/187253803 learned_addr learned my addr 172.21.15.40:0/187253803 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.409+0000 7f444e695700 1 --2- 172.21.15.40:0/187253803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2f6e0 0x564f20e3edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.409+0000 7f444d693700 1 --2- 172.21.15.40:0/187253803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20e2e840 0x564f20e37d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.409+0000 7f444de94700 1 -- 172.21.15.40:0/187253803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20e2e840 msgr2=0x564f20e37d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.409+0000 7f444de94700 1 --2- 172.21.15.40:0/187253803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20e2e840 0x564f20e37d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.409+0000 7f444de94700 1 -- 172.21.15.40:0/187253803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2f6e0 msgr2=0x564f20e3edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.409+0000 7f444de94700 1 --2- 172.21.15.40:0/187253803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2f6e0 0x564f20e3edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.409+0000 7f444de94700 1 -- 172.21.15.40:0/187253803 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4448194dd0 con 0x564f20c8a6b0 2023-07-18T03:40:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.409+0000 7f444de94700 1 --2- 172.21.15.40:0/187253803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20c8a6b0 0x564f20e37840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f443c00dd80 tx=0x7f443c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bebf1138690db4ae server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.410+0000 7f442d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:40:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.410+0000 7f442d7fa700 1 -- 172.21.15.40:0/187253803 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c010dd0 con 0x564f20c8a6b0 2023-07-18T03:40:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.410+0000 7f442d7fa700 1 -- 172.21.15.40:0/187253803 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f443c01e420 con 0x564f20c8a6b0 2023-07-18T03:40:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.410+0000 7f445c27eb80 1 -- 172.21.15.40:0/187253803 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f20cd9490 con 0x564f20c8a6b0 2023-07-18T03:40:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.410+0000 7f442d7fa700 1 -- 172.21.15.40:0/187253803 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f443c01e5e0 con 0x564f20c8a6b0 2023-07-18T03:40:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.410+0000 7f445c27eb80 10 client.26413 fetch_fsmap learned FSMap version 63 2023-07-18T03:40:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.410+0000 7f445c27eb80 1 -- 172.21.15.40:0/187253803 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x564f20b96c00 con 0x564f20c8a6b0 2023-07-18T03:40:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.411+0000 7f442d7fa700 1 -- 172.21.15.40:0/187253803 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f20cd9490 con 0x564f20c8a6b0 2023-07-18T03:40:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.411+0000 7f442d7fa700 1 -- 172.21.15.40:0/187253803 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 63) v1 ==== 7534+0+0 (secure 0 0 0) 0x7f443c00f7d0 con 0x564f20c8a6b0 2023-07-18T03:40:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.411+0000 7f445c27eb80 10 client.26413 fetch_fsmap finished waiting for FSMap version 63 2023-07-18T03:40:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.411+0000 7f445c27eb80 10 client.26413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:40:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.411+0000 7f445c27eb80 20 client.26413 populate_metadata read hostname 'smithi040' 2023-07-18T03:40:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.411+0000 7f445c27eb80 1 --2- 172.21.15.40:0/187253803 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564f20e43750 0x564f20e45b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.411+0000 7f445c27eb80 4 client.26413 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:40:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.411+0000 7f445c27eb80 1 -- 172.21.15.40:0/187253803 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f20d78e50 con 0x564f20e43750 2023-07-18T03:40:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.411+0000 7f444d693700 1 --2- 172.21.15.40:0/187253803 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564f20e43750 0x564f20e45b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.412+0000 7f442d7fa700 10 client.26413 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.412+0000 7f444d693700 1 --2- 172.21.15.40:0/187253803 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564f20e43750 0x564f20e45b60 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16021b97f0bb1a2a server_cookie=94fd7f08c94ab7f9 in_seq=0 out_seq=0 2023-07-18T03:40:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.413+0000 7f442d7fa700 1 -- 172.21.15.40:0/187253803 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564f20d78e50 con 0x564f20e43750 2023-07-18T03:40:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.413+0000 7f442d7fa700 10 client.26413 handle_command_reply: tid=0 2023-07-18T03:40:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.413+0000 7f445c27eb80 1 client.26413 shutdown 2023-07-18T03:40:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.414+0000 7f445c27eb80 1 -- 172.21.15.40:0/187253803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20c8a6b0 msgr2=0x564f20e37840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.414+0000 7f445c27eb80 1 --2- 172.21.15.40:0/187253803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20c8a6b0 0x564f20e37840 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f443c00dd80 tx=0x7f443c0047c0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.414+0000 7f445c27eb80 1 -- 172.21.15.40:0/187253803 shutdown_connections 2023-07-18T03:40:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.414+0000 7f445c27eb80 1 --2- 172.21.15.40:0/187253803 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564f20e43750 0x564f20e45b60 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.414+0000 7f445c27eb80 1 --2- 172.21.15.40:0/187253803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2f6e0 0x564f20e3edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.414+0000 7f442d7fa700 0 client.26413 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.414+0000 7f445c27eb80 1 --2- 172.21.15.40:0/187253803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20c8a6b0 0x564f20e37840 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.414+0000 7f445c27eb80 1 --2- 172.21.15.40:0/187253803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20e2e840 0x564f20e37d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.414+0000 7f445c27eb80 1 -- 172.21.15.40:0/187253803 >> 172.21.15.40:0/187253803 conn(0x564f20e23be0 msgr2=0x564f20e284e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.414+0000 7f445c27eb80 1 -- 172.21.15.40:0/187253803 shutdown_connections 2023-07-18T03:40:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.414+0000 7f445c27eb80 1 -- 172.21.15.40:0/187253803 wait complete. 2023-07-18T03:40:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.414+0000 7f445c27eb80 20 client.26413 trim_cache size 0 max 16384 2023-07-18T03:40:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.437+0000 7f445c27eb80 1 Processor -- start 2023-07-18T03:40:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.437+0000 7f445c27eb80 1 -- start start 2023-07-18T03:40:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.437+0000 7f445c27eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20c8a6b0 0x564f20e36ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.437+0000 7f445c27eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20e2e840 0x564f20e3ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.437+0000 7f445c27eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2f6e0 0x564f20e35a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.437+0000 7f445c27eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f20e38a30 con 0x564f20e2f6e0 2023-07-18T03:40:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.437+0000 7f445c27eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f20e43510 con 0x564f20c8a6b0 2023-07-18T03:40:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.437+0000 7f445c27eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f20e25d00 con 0x564f20e2e840 2023-07-18T03:40:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.438+0000 7f444e695700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2f6e0 0x564f20e35a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.438+0000 7f444e695700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2f6e0 0x564f20e35a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32992/0 (socket says 172.21.15.40:32992) 2023-07-18T03:40:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.438+0000 7f444e695700 1 -- 172.21.15.40:0/3466958489 learned_addr learned my addr 172.21.15.40:0/3466958489 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.438+0000 7f444d693700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20e2e840 0x564f20e3ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.438+0000 7f444de94700 1 --2- 172.21.15.40:0/3466958489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20c8a6b0 0x564f20e36ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.438+0000 7f444d693700 1 -- 172.21.15.40:0/3466958489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20c8a6b0 msgr2=0x564f20e36ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.438+0000 7f444d693700 1 --2- 172.21.15.40:0/3466958489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20c8a6b0 0x564f20e36ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.438+0000 7f444d693700 1 -- 172.21.15.40:0/3466958489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2f6e0 msgr2=0x564f20e35a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.438+0000 7f444d693700 1 --2- 172.21.15.40:0/3466958489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2f6e0 0x564f20e35a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.438+0000 7f444d693700 1 -- 172.21.15.40:0/3466958489 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f20bd41f0 con 0x564f20e2e840 2023-07-18T03:40:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.438+0000 7f444de94700 1 --2- 172.21.15.40:0/3466958489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20c8a6b0 0x564f20e36ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:01.440 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.439+0000 7f444d693700 1 --2- 172.21.15.40:0/3466958489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20e2e840 0x564f20e3ed50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f44380965a0 tx=0x7f443800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=158a0fe3e00a9ba7 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.440+0000 7f4444ff9700 1 -- 172.21.15.40:0/3466958489 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443809a0a0 con 0x564f20e2e840 2023-07-18T03:40:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.440+0000 7f4444ff9700 1 -- 172.21.15.40:0/3466958489 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4438098440 con 0x564f20e2e840 2023-07-18T03:40:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.440+0000 7f4444ff9700 1 -- 172.21.15.40:0/3466958489 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44380a2060 con 0x564f20e2e840 2023-07-18T03:40:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.440+0000 7f445c27eb80 1 -- 172.21.15.40:0/3466958489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20e2e840 msgr2=0x564f20e3ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.440+0000 7f445c27eb80 1 --2- 172.21.15.40:0/3466958489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20e2e840 0x564f20e3ed50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f44380965a0 tx=0x7f443800ef90 comp rx=0 tx=0).stop 2023-07-18T03:40:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.440+0000 7f445c27eb80 1 -- 172.21.15.40:0/3466958489 shutdown_connections 2023-07-18T03:40:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.440+0000 7f445c27eb80 1 --2- 172.21.15.40:0/3466958489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2f6e0 0x564f20e35a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.440+0000 7f445c27eb80 1 --2- 172.21.15.40:0/3466958489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20c8a6b0 0x564f20e36ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.440+0000 7f445c27eb80 1 --2- 172.21.15.40:0/3466958489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20e2e840 0x564f20e3ed50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.440+0000 7f445c27eb80 1 -- 172.21.15.40:0/3466958489 >> 172.21.15.40:0/3466958489 conn(0x564f20e23be0 msgr2=0x564f20e4a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.441+0000 7f445c27eb80 1 -- 172.21.15.40:0/3466958489 shutdown_connections 2023-07-18T03:40:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.441+0000 7f445c27eb80 1 -- 172.21.15.40:0/3466958489 wait complete. 2023-07-18T03:40:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.441+0000 7f445c27eb80 1 -- start start 2023-07-18T03:40:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.441+0000 7f445c27eb80 1 Processor -- start 2023-07-18T03:40:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f445c27eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20c8a6b0 0x564f20e3af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f445c27eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2e840 0x564f20e3d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f445c27eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20e2f6e0 0x564f20e3da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f445c27eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f20e3e100 con 0x564f20e2e840 2023-07-18T03:40:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f445c27eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f20e3e270 con 0x564f20c8a6b0 2023-07-18T03:40:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f445c27eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f20e3e430 con 0x564f20e2f6e0 2023-07-18T03:40:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f445c27eb80 10 client.0 _ll_register_callbacks cb 0x7f443c00d940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f444d693700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2e840 0x564f20e3d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f444de94700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20c8a6b0 0x564f20e3af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f444d693700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2e840 0x564f20e3d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32998/0 (socket says 172.21.15.40:32998) 2023-07-18T03:40:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f444d693700 1 -- 172.21.15.40:0/1896062011 learned_addr learned my addr 172.21.15.40:0/1896062011 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f444e695700 1 --2- 172.21.15.40:0/1896062011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20e2f6e0 0x564f20e3da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f444de94700 1 -- 172.21.15.40:0/1896062011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20e2f6e0 msgr2=0x564f20e3da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f444de94700 1 --2- 172.21.15.40:0/1896062011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20e2f6e0 0x564f20e3da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f444de94700 1 -- 172.21.15.40:0/1896062011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2e840 msgr2=0x564f20e3d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f444de94700 1 --2- 172.21.15.40:0/1896062011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2e840 0x564f20e3d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f444de94700 1 -- 172.21.15.40:0/1896062011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f20b96c00 con 0x564f20c8a6b0 2023-07-18T03:40:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f444d693700 1 --2- 172.21.15.40:0/1896062011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2e840 0x564f20e3d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f444de94700 1 --2- 172.21.15.40:0/1896062011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20c8a6b0 0x564f20e3af80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f443c00d5c0 tx=0x7f443c010cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee2f1acbf457e96c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.442+0000 7f442d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:40:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.443+0000 7f442d7fa700 1 -- 172.21.15.40:0/1896062011 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c00f7d0 con 0x564f20c8a6b0 2023-07-18T03:40:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.443+0000 7f442d7fa700 1 -- 172.21.15.40:0/1896062011 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f443c00fea0 con 0x564f20c8a6b0 2023-07-18T03:40:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.443+0000 7f445c27eb80 1 -- 172.21.15.40:0/1896062011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f443809e0d0 con 0x564f20c8a6b0 2023-07-18T03:40:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.443+0000 7f442d7fa700 1 -- 172.21.15.40:0/1896062011 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f443c005940 con 0x564f20c8a6b0 2023-07-18T03:40:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.443+0000 7f445c27eb80 10 client.26408 fetch_fsmap learned FSMap version 63 2023-07-18T03:40:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.443+0000 7f445c27eb80 1 -- 172.21.15.40:0/1896062011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x564f20badd80 con 0x564f20c8a6b0 2023-07-18T03:40:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.443+0000 7f442d7fa700 1 -- 172.21.15.40:0/1896062011 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f443809e0d0 con 0x564f20c8a6b0 2023-07-18T03:40:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.443+0000 7f442d7fa700 1 -- 172.21.15.40:0/1896062011 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 63) v1 ==== 7534+0+0 (secure 0 0 0) 0x7f443c03a030 con 0x564f20c8a6b0 2023-07-18T03:40:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.443+0000 7f445c27eb80 10 client.26408 fetch_fsmap finished waiting for FSMap version 63 2023-07-18T03:40:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.443+0000 7f445c27eb80 10 client.26408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:40:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.443+0000 7f445c27eb80 20 client.26408 populate_metadata read hostname 'smithi040' 2023-07-18T03:40:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.443+0000 7f445c27eb80 1 --2- 172.21.15.40:0/1896062011 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564f20e43750 0x564f20eaaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.443+0000 7f445c27eb80 4 client.26408 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:40:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.443+0000 7f445c27eb80 1 -- 172.21.15.40:0/1896062011 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564f20d3aff0 con 0x564f20e43750 2023-07-18T03:40:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.444+0000 7f444d693700 1 --2- 172.21.15.40:0/1896062011 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564f20e43750 0x564f20eaaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.444+0000 7f442d7fa700 10 client.26408 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.444+0000 7f444d693700 1 --2- 172.21.15.40:0/1896062011 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564f20e43750 0x564f20eaaf90 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7775a413bb2b6e1 server_cookie=ec0827ec66d067f in_seq=0 out_seq=0 2023-07-18T03:40:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.461+0000 7f442d7fa700 1 -- 172.21.15.40:0/1896062011 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564f20d3aff0 con 0x564f20e43750 2023-07-18T03:40:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.461+0000 7f442d7fa700 10 client.26408 handle_command_reply: tid=0 2023-07-18T03:40:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.461+0000 7f445c27eb80 1 client.26408 shutdown 2023-07-18T03:40:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.462+0000 7f445c27eb80 1 -- 172.21.15.40:0/1896062011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20c8a6b0 msgr2=0x564f20e3af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.462+0000 7f445c27eb80 1 --2- 172.21.15.40:0/1896062011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20c8a6b0 0x564f20e3af80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f443c00d5c0 tx=0x7f443c010cd0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.462+0000 7f445c27eb80 1 -- 172.21.15.40:0/1896062011 shutdown_connections 2023-07-18T03:40:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.462+0000 7f445c27eb80 1 --2- 172.21.15.40:0/1896062011 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564f20e43750 0x564f20eaaf90 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.462+0000 7f445c27eb80 1 --2- 172.21.15.40:0/1896062011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564f20e2f6e0 0x564f20e3da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.462+0000 7f445c27eb80 1 --2- 172.21.15.40:0/1896062011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f20c8a6b0 0x564f20e3af80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.462+0000 7f445c27eb80 1 --2- 172.21.15.40:0/1896062011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564f20e2e840 0x564f20e3d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.462+0000 7f445c27eb80 1 -- 172.21.15.40:0/1896062011 >> 172.21.15.40:0/1896062011 conn(0x564f20e23be0 msgr2=0x564f20e35a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.462+0000 7f442d7fa700 0 client.26408 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.462+0000 7f445c27eb80 1 -- 172.21.15.40:0/1896062011 shutdown_connections 2023-07-18T03:40:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.462+0000 7f445c27eb80 1 -- 172.21.15.40:0/1896062011 wait complete. 2023-07-18T03:40:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.462+0000 7f445c27eb80 20 client.26408 trim_cache size 0 max 16384 2023-07-18T03:40:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:40:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:40:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:40:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:40:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.463+0000 7f44500f8700 1 -- 172.21.15.40:0/1149310614 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f44300a1150 msgr2=0x7f44300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.463+0000 7f44500f8700 1 --2- 172.21.15.40:0/1149310614 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f44300a1150 0x7f44300a3600 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f44480fe9d0 tx=0x7f443c00b3f0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.463+0000 7f44500f8700 1 -- 172.21.15.40:0/1149310614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4448101bb0 msgr2=0x7f444819aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.463+0000 7f44500f8700 1 --2- 172.21.15.40:0/1149310614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4448101bb0 0x7f444819aa80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f443800ef10 tx=0x7f443800c3a0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.463+0000 7f44500f8700 1 -- 172.21.15.40:0/1149310614 shutdown_connections 2023-07-18T03:40:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.463+0000 7f44500f8700 1 --2- 172.21.15.40:0/1149310614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4448104660 0x7f444819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.463+0000 7f44500f8700 1 --2- 172.21.15.40:0/1149310614 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f44300a1150 0x7f44300a3600 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.463+0000 7f44500f8700 1 --2- 172.21.15.40:0/1149310614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44480ff200 0x7f4448198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.463+0000 7f44500f8700 1 --2- 172.21.15.40:0/1149310614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4448101bb0 0x7f444819aa80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.463+0000 7f44500f8700 1 -- 172.21.15.40:0/1149310614 >> 172.21.15.40:0/1149310614 conn(0x7f44480f9330 msgr2=0x7f44480ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.463+0000 7f44500f8700 1 -- 172.21.15.40:0/1149310614 shutdown_connections 2023-07-18T03:40:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:01.464+0000 7f44500f8700 1 -- 172.21.15.40:0/1149310614 wait complete. 2023-07-18T03:40:01.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:01 smithi134 ceph-mon[98216]: pgmap v568: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 649 KiB/s wr, 206 op/s 2023-07-18T03:40:01.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:01 smithi040 ceph-mon[96439]: pgmap v568: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 649 KiB/s wr, 206 op/s 2023-07-18T03:40:01.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:01 smithi064 ceph-mon[98059]: pgmap v568: 97 pgs: 97 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 649 KiB/s wr, 206 op/s 2023-07-18T03:40:03.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:02 smithi134 ceph-mon[98216]: pgmap v569: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 319 op/s 2023-07-18T03:40:03.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:02 smithi040 ceph-mon[96439]: pgmap v569: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 319 op/s 2023-07-18T03:40:03.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:02 smithi064 ceph-mon[98059]: pgmap v569: 97 pgs: 97 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 319 op/s 2023-07-18T03:40:05.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:05 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:40:05.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:05 smithi134 ceph-mon[98216]: pgmap v570: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.1 MiB/s wr, 280 op/s 2023-07-18T03:40:05.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:05 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:40:05.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:05 smithi040 ceph-mon[96439]: pgmap v570: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.1 MiB/s wr, 280 op/s 2023-07-18T03:40:05.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:05 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:40:05.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:05 smithi064 ceph-mon[98059]: pgmap v570: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 1.1 MiB/s wr, 280 op/s 2023-07-18T03:40:07.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:07 smithi134 ceph-mon[98216]: pgmap v571: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.2 MiB/s wr, 319 op/s 2023-07-18T03:40:07.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:07 smithi040 ceph-mon[96439]: pgmap v571: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.2 MiB/s wr, 319 op/s 2023-07-18T03:40:07.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:07 smithi064 ceph-mon[98059]: pgmap v571: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.2 MiB/s wr, 319 op/s 2023-07-18T03:40:09.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:09 smithi134 ceph-mon[98216]: pgmap v572: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 386 op/s 2023-07-18T03:40:09.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:09 smithi040 ceph-mon[96439]: pgmap v572: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 386 op/s 2023-07-18T03:40:09.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:09 smithi064 ceph-mon[98059]: pgmap v572: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 386 op/s 2023-07-18T03:40:11.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:11 smithi134 ceph-mon[98216]: pgmap v573: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 319 op/s 2023-07-18T03:40:11.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:11 smithi040 ceph-mon[96439]: pgmap v573: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 319 op/s 2023-07-18T03:40:11.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:11 smithi064 ceph-mon[98059]: pgmap v573: 97 pgs: 97 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 319 op/s 2023-07-18T03:40:13.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:12 smithi134 ceph-mon[98216]: pgmap v574: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 422 op/s 2023-07-18T03:40:13.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:12 smithi040 ceph-mon[96439]: pgmap v574: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 422 op/s 2023-07-18T03:40:13.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:12 smithi064 ceph-mon[98059]: pgmap v574: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 422 op/s 2023-07-18T03:40:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.599+0000 7f250bcc0700 1 -- 172.21.15.40:0/288658713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25041056e0 msgr2=0x7f2504107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.599+0000 7f250bcc0700 1 --2- 172.21.15.40:0/288658713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25041056e0 0x7f2504107b90 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009ec0 tx=0x7f24fc00b330 comp rx=0 tx=0).stop 2023-07-18T03:40:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.599+0000 7f250bcc0700 1 -- 172.21.15.40:0/288658713 shutdown_connections 2023-07-18T03:40:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.599+0000 7f250bcc0700 1 --2- 172.21.15.40:0/288658713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25041056e0 0x7f2504107b90 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.599+0000 7f250bcc0700 1 --2- 172.21.15.40:0/288658713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2504068640 0x7f2504102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.599+0000 7f250bcc0700 1 --2- 172.21.15.40:0/288658713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2504102bd0 0x7f2504105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.599+0000 7f250bcc0700 1 -- 172.21.15.40:0/288658713 >> 172.21.15.40:0/288658713 conn(0x7f25040fa000 msgr2=0x7f25040fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.599+0000 7f250bcc0700 1 -- 172.21.15.40:0/288658713 shutdown_connections 2023-07-18T03:40:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.599+0000 7f250bcc0700 1 -- 172.21.15.40:0/288658713 wait complete. 2023-07-18T03:40:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.600+0000 7f250bcc0700 1 Processor -- start 2023-07-18T03:40:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.600+0000 7f250bcc0700 1 -- start start 2023-07-18T03:40:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.600+0000 7f250bcc0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2504068640 0x7f25040fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.600+0000 7f250bcc0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2504102bd0 0x7f25040feb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.600+0000 7f250bcc0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25041056e0 0x7f25040ff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.600+0000 7f250bcc0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250419f890 con 0x7f25041056e0 2023-07-18T03:40:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.600+0000 7f250bcc0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250419f9d0 con 0x7f2504068640 2023-07-18T03:40:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.600+0000 7f250bcc0700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250419fb10 con 0x7f2504102bd0 2023-07-18T03:40:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f250a25d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25041056e0 0x7f25040ff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f2509a5c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2504068640 0x7f25040fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f250925b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2504102bd0 0x7f25040feb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f250925b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2504102bd0 0x7f25040feb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50274/0 (socket says 172.21.15.40:50274) 2023-07-18T03:40:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f2509a5c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2504068640 0x7f25040fe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:44570/0 (socket says 172.21.15.40:44570) 2023-07-18T03:40:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f250a25d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25041056e0 0x7f25040ff0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33016/0 (socket says 172.21.15.40:33016) 2023-07-18T03:40:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f250925b700 1 -- 172.21.15.40:0/707720391 learned_addr learned my addr 172.21.15.40:0/707720391 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f250a25d700 1 -- 172.21.15.40:0/707720391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2504102bd0 msgr2=0x7f25040feb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f250a25d700 1 --2- 172.21.15.40:0/707720391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2504102bd0 0x7f25040feb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f250a25d700 1 -- 172.21.15.40:0/707720391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2504068640 msgr2=0x7f25040fe630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f250a25d700 1 --2- 172.21.15.40:0/707720391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2504068640 0x7f25040fe630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f250a25d700 1 -- 172.21.15.40:0/707720391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc005e70 con 0x7f25041056e0 2023-07-18T03:40:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f250a25d700 1 --2- 172.21.15.40:0/707720391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25041056e0 0x7f25040ff0b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009ec0 tx=0x7f24fc009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f24faffd700 1 -- 172.21.15.40:0/707720391 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc00e040 con 0x7f25041056e0 2023-07-18T03:40:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f24faffd700 1 -- 172.21.15.40:0/707720391 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24fc01d940 con 0x7f25041056e0 2023-07-18T03:40:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f250bcc0700 1 -- 172.21.15.40:0/707720391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f250419fd40 con 0x7f25041056e0 2023-07-18T03:40:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f24faffd700 1 -- 172.21.15.40:0/707720391 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc01b910 con 0x7f25041056e0 2023-07-18T03:40:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.601+0000 7f250bcc0700 1 -- 172.21.15.40:0/707720391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25041a01c0 con 0x7f25041056e0 2023-07-18T03:40:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.604+0000 7f24faffd700 1 -- 172.21.15.40:0/707720391 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f24fc01d480 con 0x7f25041056e0 2023-07-18T03:40:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.604+0000 7f24faffd700 1 --2- 172.21.15.40:0/707720391 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f24f00a1150 0x7f24f00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.605+0000 7f2509a5c700 1 --2- 172.21.15.40:0/707720391 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f24f00a1150 0x7f24f00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.605+0000 7f24faffd700 1 -- 172.21.15.40:0/707720391 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f24fc0ce430 con 0x7f25041056e0 2023-07-18T03:40:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.605+0000 7f2509a5c700 1 --2- 172.21.15.40:0/707720391 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f24f00a1150 0x7f24f00a3600 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f25040fff40 tx=0x7f2500009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.605+0000 7f2517e46b80 1 Processor -- start 2023-07-18T03:40:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f2517e46b80 1 -- start start 2023-07-18T03:40:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f2517e46b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfcc7c6b0 0x555dfce20300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f2517e46b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfce20840 0x555dfce211a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f2517e46b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfce216e0 0x555dfce21b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f2517e46b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfcd25c50 con 0x555dfcc7c6b0 2023-07-18T03:40:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f2517e46b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfcd789e0 con 0x555dfce216e0 2023-07-18T03:40:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f2517e46b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfcd71e70 con 0x555dfce20840 2023-07-18T03:40:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f2509a5c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfcc7c6b0 0x555dfce20300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f2509a5c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfcc7c6b0 0x555dfce20300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33020/0 (socket says 172.21.15.40:33020) 2023-07-18T03:40:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f2509a5c700 1 -- 172.21.15.40:0/2615293386 learned_addr learned my addr 172.21.15.40:0/2615293386 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f250925b700 1 --2- 172.21.15.40:0/2615293386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfce20840 0x555dfce211a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f250a25d700 1 --2- 172.21.15.40:0/2615293386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfce216e0 0x555dfce21b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f2509a5c700 1 -- 172.21.15.40:0/2615293386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfce20840 msgr2=0x555dfce211a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f2509a5c700 1 --2- 172.21.15.40:0/2615293386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfce20840 0x555dfce211a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f2509a5c700 1 -- 172.21.15.40:0/2615293386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfce216e0 msgr2=0x555dfce21b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f2509a5c700 1 --2- 172.21.15.40:0/2615293386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfce216e0 0x555dfce21b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f2509a5c700 1 -- 172.21.15.40:0/2615293386 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dfcb9fd80 con 0x555dfcc7c6b0 2023-07-18T03:40:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f250925b700 1 --2- 172.21.15.40:0/2615293386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfce20840 0x555dfce211a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f250a25d700 1 --2- 172.21.15.40:0/2615293386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfce216e0 0x555dfce21b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.606+0000 7f2509a5c700 1 --2- 172.21.15.40:0/2615293386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfcc7c6b0 0x555dfce20300 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2500011d20 tx=0x7f2500014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a55a0c92e095c7a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.607+0000 7f24effff700 1 -- 172.21.15.40:0/2615293386 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250000eb20 con 0x555dfcc7c6b0 2023-07-18T03:40:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.607+0000 7f24effff700 1 -- 172.21.15.40:0/2615293386 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2500021070 con 0x555dfcc7c6b0 2023-07-18T03:40:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.607+0000 7f24effff700 1 -- 172.21.15.40:0/2615293386 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500003c70 con 0x555dfcc7c6b0 2023-07-18T03:40:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.607+0000 7f2517e46b80 1 -- 172.21.15.40:0/2615293386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfcc7c6b0 msgr2=0x555dfce20300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.607+0000 7f2517e46b80 1 --2- 172.21.15.40:0/2615293386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfcc7c6b0 0x555dfce20300 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2500011d20 tx=0x7f2500014000 comp rx=0 tx=0).stop 2023-07-18T03:40:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.607+0000 7f2517e46b80 1 -- 172.21.15.40:0/2615293386 shutdown_connections 2023-07-18T03:40:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.607+0000 7f2517e46b80 1 --2- 172.21.15.40:0/2615293386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfce216e0 0x555dfce21b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.607+0000 7f2517e46b80 1 --2- 172.21.15.40:0/2615293386 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfcc7c6b0 0x555dfce20300 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.607+0000 7f2517e46b80 1 --2- 172.21.15.40:0/2615293386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfce20840 0x555dfce211a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.607+0000 7f2517e46b80 1 -- 172.21.15.40:0/2615293386 >> 172.21.15.40:0/2615293386 conn(0x555dfce15be0 msgr2=0x555dfce1acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.607+0000 7f2517e46b80 1 -- 172.21.15.40:0/2615293386 shutdown_connections 2023-07-18T03:40:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.607+0000 7f2517e46b80 1 -- 172.21.15.40:0/2615293386 wait complete. 2023-07-18T03:40:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.608+0000 7f2517e46b80 1 -- start start 2023-07-18T03:40:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.608+0000 7f2517e46b80 1 Processor -- start 2023-07-18T03:40:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.608+0000 7f2517e46b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfcc7c6b0 0x555dfce298e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.608+0000 7f2517e46b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfce20840 0x555dfce29e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.608+0000 7f2517e46b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfce216e0 0x555dfce30e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.608+0000 7f2517e46b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfcd11a60 con 0x555dfcc7c6b0 2023-07-18T03:40:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.608+0000 7f2517e46b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfcd11760 con 0x555dfce20840 2023-07-18T03:40:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.608+0000 7f2517e46b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfcbf58c0 con 0x555dfce216e0 2023-07-18T03:40:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.608+0000 7f2517e46b80 10 client.0 _ll_register_callbacks cb 0x555dfcd0f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.608+0000 7f2509a5c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfcc7c6b0 0x555dfce298e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.608+0000 7f250925b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfce20840 0x555dfce29e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.608+0000 7f2509a5c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfcc7c6b0 0x555dfce298e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33026/0 (socket says 172.21.15.40:33026) 2023-07-18T03:40:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.608+0000 7f2509a5c700 1 -- 172.21.15.40:0/1016584934 learned_addr learned my addr 172.21.15.40:0/1016584934 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.608+0000 7f250a25d700 1 --2- 172.21.15.40:0/1016584934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfce216e0 0x555dfce30e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.609+0000 7f250925b700 1 -- 172.21.15.40:0/1016584934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfce216e0 msgr2=0x555dfce30e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.609+0000 7f250925b700 1 --2- 172.21.15.40:0/1016584934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfce216e0 0x555dfce30e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.609+0000 7f250925b700 1 -- 172.21.15.40:0/1016584934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfcc7c6b0 msgr2=0x555dfce298e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.609+0000 7f250925b700 1 --2- 172.21.15.40:0/1016584934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfcc7c6b0 0x555dfce298e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.609+0000 7f250925b700 1 -- 172.21.15.40:0/1016584934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f4009970 con 0x555dfce20840 2023-07-18T03:40:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.609+0000 7f250925b700 1 --2- 172.21.15.40:0/1016584934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfce20840 0x555dfce29e20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f25040ff930 tx=0x7f24f4012cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2db8433181a67301 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.609+0000 7f24ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:40:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.609+0000 7f24ed7fa700 1 -- 172.21.15.40:0/1016584934 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f4017070 con 0x555dfce20840 2023-07-18T03:40:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.609+0000 7f24ed7fa700 1 -- 172.21.15.40:0/1016584934 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f24f4004050 con 0x555dfce20840 2023-07-18T03:40:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.609+0000 7f2517e46b80 1 -- 172.21.15.40:0/1016584934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555dfcccb490 con 0x555dfce20840 2023-07-18T03:40:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.609+0000 7f24ed7fa700 1 -- 172.21.15.40:0/1016584934 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24f4004210 con 0x555dfce20840 2023-07-18T03:40:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.609+0000 7f2517e46b80 10 client.26420 fetch_fsmap learned FSMap version 64 2023-07-18T03:40:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.609+0000 7f2517e46b80 1 -- 172.21.15.40:0/1016584934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x7f25041a01c0 con 0x555dfce20840 2023-07-18T03:40:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.609+0000 7f24ed7fa700 1 -- 172.21.15.40:0/1016584934 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555dfcccb490 con 0x555dfce20840 2023-07-18T03:40:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.610+0000 7f24ed7fa700 1 -- 172.21.15.40:0/1016584934 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 64) v1 ==== 7530+0+0 (secure 0 0 0) 0x7f24f4003820 con 0x555dfce20840 2023-07-18T03:40:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.610+0000 7f2517e46b80 10 client.26420 fetch_fsmap finished waiting for FSMap version 64 2023-07-18T03:40:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.610+0000 7f2517e46b80 10 client.26420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:40:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.610+0000 7f2517e46b80 20 client.26420 populate_metadata read hostname 'smithi040' 2023-07-18T03:40:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.610+0000 7f2517e46b80 1 --2- 172.21.15.40:0/1016584934 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555dfce357f0 0x555dfce37c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.610+0000 7f2517e46b80 4 client.26420 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:40:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.610+0000 7f2517e46b80 1 -- 172.21.15.40:0/1016584934 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555dfcd6ae50 con 0x555dfce357f0 2023-07-18T03:40:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.610+0000 7f2509a5c700 1 --2- 172.21.15.40:0/1016584934 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555dfce357f0 0x555dfce37c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.610+0000 7f2509a5c700 1 --2- 172.21.15.40:0/1016584934 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555dfce357f0 0x555dfce37c00 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d649fa61cc9d070 server_cookie=7d265515caa0a2fa in_seq=0 out_seq=0 2023-07-18T03:40:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.610+0000 7f24ed7fa700 10 client.26420 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.611+0000 7f24ed7fa700 1 -- 172.21.15.40:0/1016584934 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555dfcd6ae50 con 0x555dfce357f0 2023-07-18T03:40:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.611+0000 7f24ed7fa700 10 client.26420 handle_command_reply: tid=0 2023-07-18T03:40:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.611+0000 7f2517e46b80 1 client.26420 shutdown 2023-07-18T03:40:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.611+0000 7f2517e46b80 1 -- 172.21.15.40:0/1016584934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfce20840 msgr2=0x555dfce29e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.611+0000 7f2517e46b80 1 --2- 172.21.15.40:0/1016584934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfce20840 0x555dfce29e20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f25040ff930 tx=0x7f24f4012cc0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.611+0000 7f2517e46b80 1 -- 172.21.15.40:0/1016584934 shutdown_connections 2023-07-18T03:40:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.611+0000 7f2517e46b80 1 --2- 172.21.15.40:0/1016584934 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555dfce357f0 0x555dfce37c00 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.611+0000 7f2517e46b80 1 --2- 172.21.15.40:0/1016584934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfce216e0 0x555dfce30e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.611+0000 7f2517e46b80 1 --2- 172.21.15.40:0/1016584934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfcc7c6b0 0x555dfce298e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.611+0000 7f2517e46b80 1 --2- 172.21.15.40:0/1016584934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfce20840 0x555dfce29e20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.611+0000 7f24ed7fa700 0 client.26420 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.611+0000 7f2517e46b80 1 -- 172.21.15.40:0/1016584934 >> 172.21.15.40:0/1016584934 conn(0x555dfce15be0 msgr2=0x555dfce1a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.611+0000 7f2517e46b80 1 -- 172.21.15.40:0/1016584934 shutdown_connections 2023-07-18T03:40:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.611+0000 7f2517e46b80 1 -- 172.21.15.40:0/1016584934 wait complete. 2023-07-18T03:40:14.619 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.611+0000 7f2517e46b80 20 client.26420 trim_cache size 0 max 16384 2023-07-18T03:40:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.626+0000 7f2517e46b80 1 Processor -- start 2023-07-18T03:40:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.626+0000 7f2517e46b80 1 -- start start 2023-07-18T03:40:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.626+0000 7f2517e46b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfcc7c6b0 0x555dfce24590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.626+0000 7f2517e46b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfce20840 0x555dfce30e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.626+0000 7f2517e46b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfce216e0 0x555dfce24ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f2517e46b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfce29450 con 0x555dfce216e0 2023-07-18T03:40:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f2517e46b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfce17d00 con 0x555dfce20840 2023-07-18T03:40:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f2517e46b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfce2aa30 con 0x555dfcc7c6b0 2023-07-18T03:40:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f2509a5c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfcc7c6b0 0x555dfce24590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f250a25d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfce216e0 0x555dfce24ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f2509a5c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfcc7c6b0 0x555dfce24590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50294/0 (socket says 172.21.15.40:50294) 2023-07-18T03:40:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f2509a5c700 1 -- 172.21.15.40:0/2801283617 learned_addr learned my addr 172.21.15.40:0/2801283617 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f250925b700 1 --2- 172.21.15.40:0/2801283617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfce20840 0x555dfce30e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f250a25d700 1 -- 172.21.15.40:0/2801283617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfcc7c6b0 msgr2=0x555dfce24590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f250a25d700 1 --2- 172.21.15.40:0/2801283617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfcc7c6b0 0x555dfce24590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f250a25d700 1 -- 172.21.15.40:0/2801283617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfce20840 msgr2=0x555dfce30e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f250a25d700 1 --2- 172.21.15.40:0/2801283617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfce20840 0x555dfce30e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f250a25d700 1 -- 172.21.15.40:0/2801283617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2500017070 con 0x555dfce216e0 2023-07-18T03:40:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f2509a5c700 1 --2- 172.21.15.40:0/2801283617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfcc7c6b0 0x555dfce24590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f250a25d700 1 --2- 172.21.15.40:0/2801283617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfce216e0 0x555dfce24ad0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f24fc007870 tx=0x7f24fc0a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdedc6e893878a5d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f24f8ff9700 1 -- 172.21.15.40:0/2801283617 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc09f8e0 con 0x555dfce216e0 2023-07-18T03:40:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f24f8ff9700 1 -- 172.21.15.40:0/2801283617 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24fc0a3c70 con 0x555dfce216e0 2023-07-18T03:40:14.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.627+0000 7f24f8ff9700 1 -- 172.21.15.40:0/2801283617 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc09bb10 con 0x555dfce216e0 2023-07-18T03:40:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.628+0000 7f2517e46b80 1 -- 172.21.15.40:0/2801283617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfce216e0 msgr2=0x555dfce24ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.628+0000 7f2517e46b80 1 --2- 172.21.15.40:0/2801283617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfce216e0 0x555dfce24ad0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f24fc007870 tx=0x7f24fc0a1040 comp rx=0 tx=0).stop 2023-07-18T03:40:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.628+0000 7f2517e46b80 1 -- 172.21.15.40:0/2801283617 shutdown_connections 2023-07-18T03:40:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.628+0000 7f2517e46b80 1 --2- 172.21.15.40:0/2801283617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfce216e0 0x555dfce24ad0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.628+0000 7f2517e46b80 1 --2- 172.21.15.40:0/2801283617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfcc7c6b0 0x555dfce24590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.628+0000 7f2517e46b80 1 --2- 172.21.15.40:0/2801283617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfce20840 0x555dfce30e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.628+0000 7f2517e46b80 1 -- 172.21.15.40:0/2801283617 >> 172.21.15.40:0/2801283617 conn(0x555dfce15be0 msgr2=0x555dfce27a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.628+0000 7f2517e46b80 1 -- 172.21.15.40:0/2801283617 shutdown_connections 2023-07-18T03:40:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.628+0000 7f2517e46b80 1 -- 172.21.15.40:0/2801283617 wait complete. 2023-07-18T03:40:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.628+0000 7f2517e46b80 1 -- start start 2023-07-18T03:40:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.628+0000 7f2517e46b80 1 Processor -- start 2023-07-18T03:40:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f2517e46b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfcc7c6b0 0x555dfce2cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f2517e46b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfce20840 0x555dfce2d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f2517e46b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfce216e0 0x555dfce300d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f2517e46b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfce30610 con 0x555dfce20840 2023-07-18T03:40:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f2517e46b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfce30780 con 0x555dfcc7c6b0 2023-07-18T03:40:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f2517e46b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dfce30940 con 0x555dfce216e0 2023-07-18T03:40:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f2517e46b80 10 client.0 _ll_register_callbacks cb 0x7f24f40124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f250925b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfce20840 0x555dfce2d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f2509a5c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfcc7c6b0 0x555dfce2cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f250925b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfce20840 0x555dfce2d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33042/0 (socket says 172.21.15.40:33042) 2023-07-18T03:40:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f250925b700 1 -- 172.21.15.40:0/964909775 learned_addr learned my addr 172.21.15.40:0/964909775 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f250a25d700 1 --2- 172.21.15.40:0/964909775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfce216e0 0x555dfce300d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f250925b700 1 -- 172.21.15.40:0/964909775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfce216e0 msgr2=0x555dfce300d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f250925b700 1 --2- 172.21.15.40:0/964909775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfce216e0 0x555dfce300d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f250925b700 1 -- 172.21.15.40:0/964909775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfcc7c6b0 msgr2=0x555dfce2cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f250925b700 1 --2- 172.21.15.40:0/964909775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfcc7c6b0 0x555dfce2cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f250925b700 1 -- 172.21.15.40:0/964909775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555dfcb9fd80 con 0x555dfce20840 2023-07-18T03:40:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f2509a5c700 1 --2- 172.21.15.40:0/964909775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfcc7c6b0 0x555dfce2cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f250925b700 1 --2- 172.21.15.40:0/964909775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfce20840 0x555dfce2d0b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f24f40104d0 tx=0x7f24f4003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8e058d79e5b25a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f24ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:40:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f24ed7fa700 1 -- 172.21.15.40:0/964909775 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f4017040 con 0x555dfce20840 2023-07-18T03:40:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.629+0000 7f24ed7fa700 1 -- 172.21.15.40:0/964909775 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f24f40055e0 con 0x555dfce20840 2023-07-18T03:40:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.630+0000 7f2517e46b80 1 -- 172.21.15.40:0/964909775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2500021070 con 0x555dfce20840 2023-07-18T03:40:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.630+0000 7f24ed7fa700 1 -- 172.21.15.40:0/964909775 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24f400f6a0 con 0x555dfce20840 2023-07-18T03:40:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.630+0000 7f2517e46b80 10 client.16770 fetch_fsmap learned FSMap version 64 2023-07-18T03:40:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.630+0000 7f2517e46b80 1 -- 172.21.15.40:0/964909775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x7f250419fd40 con 0x555dfce20840 2023-07-18T03:40:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.630+0000 7f24ed7fa700 1 -- 172.21.15.40:0/964909775 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2500021070 con 0x555dfce20840 2023-07-18T03:40:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.630+0000 7f24ed7fa700 1 -- 172.21.15.40:0/964909775 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 64) v1 ==== 7530+0+0 (secure 0 0 0) 0x7f24f4010860 con 0x555dfce20840 2023-07-18T03:40:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.630+0000 7f2517e46b80 10 client.16770 fetch_fsmap finished waiting for FSMap version 64 2023-07-18T03:40:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.630+0000 7f2517e46b80 10 client.16770 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:40:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.630+0000 7f2517e46b80 20 client.16770 populate_metadata read hostname 'smithi040' 2023-07-18T03:40:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.630+0000 7f2517e46b80 1 --2- 172.21.15.40:0/964909775 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555dfce8a5a0 0x555dfce8c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.630+0000 7f2517e46b80 4 client.16770 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:40:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.630+0000 7f2517e46b80 1 -- 172.21.15.40:0/964909775 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555dfcd2cff0 con 0x555dfce8a5a0 2023-07-18T03:40:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.630+0000 7f2509a5c700 1 --2- 172.21.15.40:0/964909775 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555dfce8a5a0 0x555dfce8c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.631+0000 7f24ed7fa700 10 client.16770 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.631+0000 7f2509a5c700 1 --2- 172.21.15.40:0/964909775 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555dfce8a5a0 0x555dfce8c9b0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce6a94a434e1c46 server_cookie=2a9333eda220ff2 in_seq=0 out_seq=0 2023-07-18T03:40:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.638+0000 7f24ed7fa700 1 -- 172.21.15.40:0/964909775 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555dfcd2cff0 con 0x555dfce8a5a0 2023-07-18T03:40:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.638+0000 7f24ed7fa700 10 client.16770 handle_command_reply: tid=0 2023-07-18T03:40:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.638+0000 7f2517e46b80 1 client.16770 shutdown 2023-07-18T03:40:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f2517e46b80 1 -- 172.21.15.40:0/964909775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfce20840 msgr2=0x555dfce2d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f2517e46b80 1 --2- 172.21.15.40:0/964909775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfce20840 0x555dfce2d0b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f24f40104d0 tx=0x7f24f4003a70 comp rx=0 tx=0).stop 2023-07-18T03:40:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f2517e46b80 1 -- 172.21.15.40:0/964909775 shutdown_connections 2023-07-18T03:40:14.636 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f2517e46b80 1 --2- 172.21.15.40:0/964909775 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555dfce8a5a0 0x555dfce8c9b0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f2517e46b80 1 --2- 172.21.15.40:0/964909775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555dfce216e0 0x555dfce300d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f24ed7fa700 0 client.16770 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f2517e46b80 1 --2- 172.21.15.40:0/964909775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555dfcc7c6b0 0x555dfce2cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f2517e46b80 1 --2- 172.21.15.40:0/964909775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555dfce20840 0x555dfce2d0b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f2517e46b80 1 -- 172.21.15.40:0/964909775 >> 172.21.15.40:0/964909775 conn(0x555dfce15be0 msgr2=0x555dfce27a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f2517e46b80 1 -- 172.21.15.40:0/964909775 shutdown_connections 2023-07-18T03:40:14.637 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f2517e46b80 1 -- 172.21.15.40:0/964909775 wait complete. 2023-07-18T03:40:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f2517e46b80 20 client.16770 trim_cache size 0 max 16384 2023-07-18T03:40:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:40:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:40:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:40:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:40:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f250bcc0700 1 -- 172.21.15.40:0/707720391 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f24f00a1150 msgr2=0x7f24f00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f250bcc0700 1 --2- 172.21.15.40:0/707720391 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f24f00a1150 0x7f24f00a3600 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f25040fff40 tx=0x7f2500009260 comp rx=0 tx=0).stop 2023-07-18T03:40:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f250bcc0700 1 -- 172.21.15.40:0/707720391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25041056e0 msgr2=0x7f25040ff0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f250bcc0700 1 --2- 172.21.15.40:0/707720391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25041056e0 0x7f25040ff0b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009ec0 tx=0x7f24fc009590 comp rx=0 tx=0).stop 2023-07-18T03:40:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f250bcc0700 1 -- 172.21.15.40:0/707720391 shutdown_connections 2023-07-18T03:40:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f250bcc0700 1 --2- 172.21.15.40:0/707720391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25041056e0 0x7f25040ff0b0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f250bcc0700 1 --2- 172.21.15.40:0/707720391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2504068640 0x7f25040fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f250bcc0700 1 --2- 172.21.15.40:0/707720391 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f24f00a1150 0x7f24f00a3600 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f250bcc0700 1 --2- 172.21.15.40:0/707720391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2504102bd0 0x7f25040feb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.639+0000 7f250bcc0700 1 -- 172.21.15.40:0/707720391 >> 172.21.15.40:0/707720391 conn(0x7f25040fa000 msgr2=0x7f25041016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.640+0000 7f250bcc0700 1 -- 172.21.15.40:0/707720391 shutdown_connections 2023-07-18T03:40:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:14.640+0000 7f250bcc0700 1 -- 172.21.15.40:0/707720391 wait complete. 2023-07-18T03:40:15.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:15 smithi134 ceph-mon[98216]: pgmap v575: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 347 op/s 2023-07-18T03:40:15.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:15 smithi040 ceph-mon[96439]: pgmap v575: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 347 op/s 2023-07-18T03:40:15.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:15 smithi064 ceph-mon[98059]: pgmap v575: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 347 op/s 2023-07-18T03:40:17.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:17 smithi134 ceph-mon[98216]: pgmap v576: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 359 op/s 2023-07-18T03:40:17.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:17 smithi040 ceph-mon[96439]: pgmap v576: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 359 op/s 2023-07-18T03:40:17.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:17 smithi064 ceph-mon[98059]: pgmap v576: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 359 op/s 2023-07-18T03:40:19.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:19 smithi134 ceph-mon[98216]: pgmap v577: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 398 op/s 2023-07-18T03:40:19.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:19 smithi040 ceph-mon[96439]: pgmap v577: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 398 op/s 2023-07-18T03:40:19.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:19 smithi064 ceph-mon[98059]: pgmap v577: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 398 op/s 2023-07-18T03:40:21.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:21 smithi134 ceph-mon[98216]: pgmap v578: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 298 op/s 2023-07-18T03:40:21.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:21 smithi040 ceph-mon[96439]: pgmap v578: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 298 op/s 2023-07-18T03:40:21.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:21 smithi064 ceph-mon[98059]: pgmap v578: 97 pgs: 97 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 298 op/s 2023-07-18T03:40:22.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:22 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:40:22.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:22 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:40:22.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:22 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:40:23.500 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:40:23.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:23 smithi040 ceph-mon[96439]: pgmap v579: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 403 op/s 2023-07-18T03:40:23.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:23 smithi064 ceph-mon[98059]: pgmap v579: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 403 op/s 2023-07-18T03:40:23.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.636+0000 7f01dc10d700 1 -- 172.21.15.40:0/527257686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d41057a0 msgr2=0x7f01d4107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.636+0000 7f01dc10d700 1 --2- 172.21.15.40:0/527257686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d41057a0 0x7f01d4107c00 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00b0a0 tx=0x7f01cc00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.636+0000 7f01dc10d700 1 -- 172.21.15.40:0/527257686 shutdown_connections 2023-07-18T03:40:23.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.636+0000 7f01dc10d700 1 --2- 172.21.15.40:0/527257686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d41057a0 0x7f01d4107c00 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.636+0000 7f01dc10d700 1 --2- 172.21.15.40:0/527257686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d4100560 0x7f01d4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.636+0000 7f01dc10d700 1 --2- 172.21.15.40:0/527257686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d4102e80 0x7f01d4105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.636+0000 7f01dc10d700 1 -- 172.21.15.40:0/527257686 >> 172.21.15.40:0/527257686 conn(0x7f01d40fa000 msgr2=0x7f01d40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:23.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.636+0000 7f01dc10d700 1 -- 172.21.15.40:0/527257686 shutdown_connections 2023-07-18T03:40:23.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.636+0000 7f01dc10d700 1 -- 172.21.15.40:0/527257686 wait complete. 2023-07-18T03:40:23.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01dc10d700 1 Processor -- start 2023-07-18T03:40:23.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01dc10d700 1 -- start start 2023-07-18T03:40:23.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01dc10d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d4100560 0x7f01d40ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01dc10d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d4102e80 0x7f01d40fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01dc10d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d41057a0 0x7f01d40fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01dc10d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d40ff1f0 con 0x7f01d4102e80 2023-07-18T03:40:23.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01dc10d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d40ff360 con 0x7f01d41057a0 2023-07-18T03:40:23.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01dc10d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d419f9c0 con 0x7f01d4100560 2023-07-18T03:40:23.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01d96a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d4102e80 0x7f01d40fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01d9ea9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d4100560 0x7f01d40ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01d96a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d4102e80 0x7f01d40fe510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33058/0 (socket says 172.21.15.40:33058) 2023-07-18T03:40:23.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01d9ea9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d4100560 0x7f01d40ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50314/0 (socket says 172.21.15.40:50314) 2023-07-18T03:40:23.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01d96a8700 1 -- 172.21.15.40:0/2574031921 learned_addr learned my addr 172.21.15.40:0/2574031921 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:23.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01da6aa700 1 --2- 172.21.15.40:0/2574031921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d41057a0 0x7f01d40fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01d96a8700 1 -- 172.21.15.40:0/2574031921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d4100560 msgr2=0x7f01d40ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01d96a8700 1 --2- 172.21.15.40:0/2574031921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d4100560 0x7f01d40ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01d96a8700 1 -- 172.21.15.40:0/2574031921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d41057a0 msgr2=0x7f01d40fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01d96a8700 1 --2- 172.21.15.40:0/2574031921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d41057a0 0x7f01d40fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.637+0000 7f01d96a8700 1 -- 172.21.15.40:0/2574031921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc009d00 con 0x7f01d4102e80 2023-07-18T03:40:23.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.638+0000 7f01d96a8700 1 --2- 172.21.15.40:0/2574031921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d4102e80 0x7f01d40fe510 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f01d000e9a0 tx=0x7f01d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:23.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.638+0000 7f01c6ffd700 1 -- 172.21.15.40:0/2574031921 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0009790 con 0x7f01d4102e80 2023-07-18T03:40:23.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.638+0000 7f01dc10d700 1 -- 172.21.15.40:0/2574031921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d419fca0 con 0x7f01d4102e80 2023-07-18T03:40:23.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.638+0000 7f01c6ffd700 1 -- 172.21.15.40:0/2574031921 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01d0010430 con 0x7f01d4102e80 2023-07-18T03:40:23.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.638+0000 7f01c6ffd700 1 -- 172.21.15.40:0/2574031921 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0018680 con 0x7f01d4102e80 2023-07-18T03:40:23.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.638+0000 7f01dc10d700 1 -- 172.21.15.40:0/2574031921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d41a0180 con 0x7f01d4102e80 2023-07-18T03:40:23.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.640+0000 7f01c6ffd700 1 -- 172.21.15.40:0/2574031921 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f01d00105a0 con 0x7f01d4102e80 2023-07-18T03:40:23.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.641+0000 7f01c6ffd700 1 --2- 172.21.15.40:0/2574031921 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f01c00a1150 0x7f01c00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.641+0000 7f01d9ea9700 1 --2- 172.21.15.40:0/2574031921 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f01c00a1150 0x7f01c00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.641+0000 7f01c6ffd700 1 -- 172.21.15.40:0/2574031921 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f01d0014070 con 0x7f01d4102e80 2023-07-18T03:40:23.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.642+0000 7f01d9ea9700 1 --2- 172.21.15.40:0/2574031921 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f01c00a1150 0x7f01c00a3600 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009c60 tx=0x7f01c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:23.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.642+0000 7f01e8293b80 1 Processor -- start 2023-07-18T03:40:23.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.642+0000 7f01e8293b80 1 -- start start 2023-07-18T03:40:23.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.642+0000 7f01e8293b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e84d0 0x5626087e88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.642+0000 7f01e8293b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.642+0000 7f01e8293b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e9c70 0x5626087ea0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.642+0000 7f01e8293b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626086edcb0 con 0x5626087e8e20 2023-07-18T03:40:23.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.642+0000 7f01e8293b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562608740a40 con 0x5626087e9c70 2023-07-18T03:40:23.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.642+0000 7f01e8293b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562608739ed0 con 0x5626087e84d0 2023-07-18T03:40:23.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.642+0000 7f01d96a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.642+0000 7f01d9ea9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e84d0 0x5626087e88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.642+0000 7f01d96a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087e96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33064/0 (socket says 172.21.15.40:33064) 2023-07-18T03:40:23.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.642+0000 7f01da6aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e9c70 0x5626087ea0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.642+0000 7f01d96a8700 1 -- 172.21.15.40:0/1756103817 learned_addr learned my addr 172.21.15.40:0/1756103817 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:23.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.643+0000 7f01da6aa700 1 -- 172.21.15.40:0/1756103817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e84d0 msgr2=0x5626087e88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:23.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.643+0000 7f01da6aa700 1 --2- 172.21.15.40:0/1756103817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e84d0 0x5626087e88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.643+0000 7f01da6aa700 1 -- 172.21.15.40:0/1756103817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 msgr2=0x5626087e96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:23.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.643+0000 7f01da6aa700 1 --2- 172.21.15.40:0/1756103817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087e96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.643+0000 7f01da6aa700 1 -- 172.21.15.40:0/1756103817 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562608567de0 con 0x5626087e9c70 2023-07-18T03:40:23.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.643+0000 7f01d96a8700 1 --2- 172.21.15.40:0/1756103817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087e96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:23.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.643+0000 7f01da6aa700 1 --2- 172.21.15.40:0/1756103817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e9c70 0x5626087ea0e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009fd0 tx=0x7f01cc009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4a2e5df92bb9e2d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:23.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.643+0000 7f01bffff700 1 -- 172.21.15.40:0/1756103817 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc00e040 con 0x5626087e9c70 2023-07-18T03:40:23.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.643+0000 7f01bffff700 1 -- 172.21.15.40:0/1756103817 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01cc0076e0 con 0x5626087e9c70 2023-07-18T03:40:23.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.643+0000 7f01bffff700 1 -- 172.21.15.40:0/1756103817 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc01c6a0 con 0x5626087e9c70 2023-07-18T03:40:23.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.644+0000 7f01e8293b80 1 -- 172.21.15.40:0/1756103817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e9c70 msgr2=0x5626087ea0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.644+0000 7f01e8293b80 1 --2- 172.21.15.40:0/1756103817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e9c70 0x5626087ea0e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009fd0 tx=0x7f01cc009c20 comp rx=0 tx=0).stop 2023-07-18T03:40:23.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.644+0000 7f01e8293b80 1 -- 172.21.15.40:0/1756103817 shutdown_connections 2023-07-18T03:40:23.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.644+0000 7f01e8293b80 1 --2- 172.21.15.40:0/1756103817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e9c70 0x5626087ea0e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.644+0000 7f01e8293b80 1 --2- 172.21.15.40:0/1756103817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e84d0 0x5626087e88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.644+0000 7f01e8293b80 1 --2- 172.21.15.40:0/1756103817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087e96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.644+0000 7f01e8293b80 1 -- 172.21.15.40:0/1756103817 >> 172.21.15.40:0/1756103817 conn(0x5626087ddda0 msgr2=0x5626087e31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:23.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.644+0000 7f01e8293b80 1 -- 172.21.15.40:0/1756103817 shutdown_connections 2023-07-18T03:40:23.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.644+0000 7f01e8293b80 1 -- 172.21.15.40:0/1756103817 wait complete. 2023-07-18T03:40:23.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.644+0000 7f01e8293b80 1 -- start start 2023-07-18T03:40:23.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.644+0000 7f01e8293b80 1 Processor -- start 2023-07-18T03:40:23.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01e8293b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e84d0 0x5626087f2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01e8293b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e8e20 0x5626087f2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01e8293b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e9c70 0x5626087f9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01e8293b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626086d9ac0 con 0x5626087e9c70 2023-07-18T03:40:23.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01e8293b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626086d97c0 con 0x5626087e84d0 2023-07-18T03:40:23.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01e8293b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626085bd920 con 0x5626087e8e20 2023-07-18T03:40:23.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01e8293b80 10 client.0 _ll_register_callbacks cb 0x5626086d7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:23.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01d9ea9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e84d0 0x5626087f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01d9ea9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e84d0 0x5626087f2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:44626/0 (socket says 172.21.15.40:44626) 2023-07-18T03:40:23.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01d9ea9700 1 -- 172.21.15.40:0/4168270110 learned_addr learned my addr 172.21.15.40:0/4168270110 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:23.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01da6aa700 1 --2- 172.21.15.40:0/4168270110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e9c70 0x5626087f9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01d96a8700 1 --2- 172.21.15.40:0/4168270110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e8e20 0x5626087f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01d9ea9700 1 -- 172.21.15.40:0/4168270110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e8e20 msgr2=0x5626087f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:23.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01d9ea9700 1 --2- 172.21.15.40:0/4168270110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e8e20 0x5626087f2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01d9ea9700 1 -- 172.21.15.40:0/4168270110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e9c70 msgr2=0x5626087f9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:23.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01d9ea9700 1 --2- 172.21.15.40:0/4168270110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e9c70 0x5626087f9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01d9ea9700 1 -- 172.21.15.40:0/4168270110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d41a0180 con 0x5626087e84d0 2023-07-18T03:40:23.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01d9ea9700 1 --2- 172.21.15.40:0/4168270110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e84d0 0x5626087f2110 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f01c8003900 tx=0x7f01c80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c108c2ad4844cef server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:23.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.645+0000 7f01bd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:40:23.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.646+0000 7f01bd7fa700 1 -- 172.21.15.40:0/4168270110 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8022070 con 0x5626087e84d0 2023-07-18T03:40:23.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.646+0000 7f01e8293b80 1 -- 172.21.15.40:0/4168270110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626086934f0 con 0x5626087e84d0 2023-07-18T03:40:23.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.646+0000 7f01bd7fa700 1 -- 172.21.15.40:0/4168270110 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f01c801d410 con 0x5626087e84d0 2023-07-18T03:40:23.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.646+0000 7f01bd7fa700 1 -- 172.21.15.40:0/4168270110 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01c801d5d0 con 0x5626087e84d0 2023-07-18T03:40:23.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.646+0000 7f01e8293b80 10 client.26438 fetch_fsmap learned FSMap version 65 2023-07-18T03:40:23.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.646+0000 7f01e8293b80 1 -- 172.21.15.40:0/4168270110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x7f01d41087f0 con 0x5626087e84d0 2023-07-18T03:40:23.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.646+0000 7f01bd7fa700 1 -- 172.21.15.40:0/4168270110 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5626086934f0 con 0x5626087e84d0 2023-07-18T03:40:23.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.646+0000 7f01bd7fa700 1 -- 172.21.15.40:0/4168270110 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 65) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f01c8027880 con 0x5626087e84d0 2023-07-18T03:40:23.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.646+0000 7f01e8293b80 10 client.26438 fetch_fsmap finished waiting for FSMap version 65 2023-07-18T03:40:23.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.646+0000 7f01e8293b80 10 client.26438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:40:23.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.646+0000 7f01e8293b80 20 client.26438 populate_metadata read hostname 'smithi040' 2023-07-18T03:40:23.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.646+0000 7f01e8293b80 1 --2- 172.21.15.40:0/4168270110 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5626087fdf50 0x562608800360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.646+0000 7f01e8293b80 4 client.26438 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:40:23.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.646+0000 7f01e8293b80 1 -- 172.21.15.40:0/4168270110 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562608732eb0 con 0x5626087fdf50 2023-07-18T03:40:23.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.646+0000 7f01d96a8700 1 --2- 172.21.15.40:0/4168270110 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5626087fdf50 0x562608800360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.647+0000 7f01bd7fa700 10 client.26438 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:23.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.647+0000 7f01d96a8700 1 --2- 172.21.15.40:0/4168270110 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5626087fdf50 0x562608800360 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1194f8c25096be9 server_cookie=b3e67c4ff23fa4d9 in_seq=0 out_seq=0 2023-07-18T03:40:23.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.647+0000 7f01bd7fa700 1 -- 172.21.15.40:0/4168270110 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562608732eb0 con 0x5626087fdf50 2023-07-18T03:40:23.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.647+0000 7f01bd7fa700 10 client.26438 handle_command_reply: tid=0 2023-07-18T03:40:23.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.647+0000 7f01e8293b80 1 client.26438 shutdown 2023-07-18T03:40:23.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.648+0000 7f01e8293b80 1 -- 172.21.15.40:0/4168270110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e84d0 msgr2=0x5626087f2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.648+0000 7f01e8293b80 1 --2- 172.21.15.40:0/4168270110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e84d0 0x5626087f2110 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f01c8003900 tx=0x7f01c80047c0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.648+0000 7f01e8293b80 1 -- 172.21.15.40:0/4168270110 shutdown_connections 2023-07-18T03:40:23.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.648+0000 7f01e8293b80 1 --2- 172.21.15.40:0/4168270110 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5626087fdf50 0x562608800360 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.648+0000 7f01e8293b80 1 --2- 172.21.15.40:0/4168270110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e9c70 0x5626087f9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.648+0000 7f01e8293b80 1 --2- 172.21.15.40:0/4168270110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e84d0 0x5626087f2110 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.648+0000 7f01e8293b80 1 --2- 172.21.15.40:0/4168270110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e8e20 0x5626087f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.648+0000 7f01bd7fa700 0 client.26438 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:23.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.648+0000 7f01e8293b80 1 -- 172.21.15.40:0/4168270110 >> 172.21.15.40:0/4168270110 conn(0x5626087ddda0 msgr2=0x5626087e28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:23.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.648+0000 7f01e8293b80 1 -- 172.21.15.40:0/4168270110 shutdown_connections 2023-07-18T03:40:23.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.648+0000 7f01e8293b80 1 -- 172.21.15.40:0/4168270110 wait complete. 2023-07-18T03:40:23.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.648+0000 7f01e8293b80 20 client.26438 trim_cache size 0 max 16384 2023-07-18T03:40:23.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01e8293b80 1 Processor -- start 2023-07-18T03:40:23.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01e8293b80 1 -- start start 2023-07-18T03:40:23.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01e8293b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e84d0 0x5626087f0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01e8293b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087f9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01e8293b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e9c70 0x5626087ecb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01e8293b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626087f1ce0 con 0x5626087e8e20 2023-07-18T03:40:23.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01e8293b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626087e4d00 con 0x5626087e84d0 2023-07-18T03:40:23.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01e8293b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626087efe10 con 0x5626087e9c70 2023-07-18T03:40:23.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01d96a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087f9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01d96a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087f9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33078/0 (socket says 172.21.15.40:33078) 2023-07-18T03:40:23.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01d96a8700 1 -- 172.21.15.40:0/3092495225 learned_addr learned my addr 172.21.15.40:0/3092495225 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:23.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01da6aa700 1 --2- 172.21.15.40:0/3092495225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e9c70 0x5626087ecb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01d9ea9700 1 --2- 172.21.15.40:0/3092495225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e84d0 0x5626087f0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01d96a8700 1 -- 172.21.15.40:0/3092495225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e9c70 msgr2=0x5626087ecb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:23.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01d96a8700 1 --2- 172.21.15.40:0/3092495225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e9c70 0x5626087ecb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01d96a8700 1 -- 172.21.15.40:0/3092495225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e84d0 msgr2=0x5626087f0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:23.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01d96a8700 1 --2- 172.21.15.40:0/3092495225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e84d0 0x5626087f0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01d96a8700 1 -- 172.21.15.40:0/3092495225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562608550c60 con 0x5626087e8e20 2023-07-18T03:40:23.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.671+0000 7f01d96a8700 1 --2- 172.21.15.40:0/3092495225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087f9620 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f01d0099bd0 tx=0x7f01d001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5bb8354dc6bf1b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:23.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.672+0000 7f01c4ff9700 1 -- 172.21.15.40:0/3092495225 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00a2320 con 0x5626087e8e20 2023-07-18T03:40:23.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.672+0000 7f01c4ff9700 1 -- 172.21.15.40:0/3092495225 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01d00b1070 con 0x5626087e8e20 2023-07-18T03:40:23.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.672+0000 7f01c4ff9700 1 -- 172.21.15.40:0/3092495225 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0099600 con 0x5626087e8e20 2023-07-18T03:40:23.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.672+0000 7f01e8293b80 1 -- 172.21.15.40:0/3092495225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 msgr2=0x5626087f9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.672+0000 7f01e8293b80 1 --2- 172.21.15.40:0/3092495225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087f9620 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f01d0099bd0 tx=0x7f01d001a040 comp rx=0 tx=0).stop 2023-07-18T03:40:23.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.672+0000 7f01e8293b80 1 -- 172.21.15.40:0/3092495225 shutdown_connections 2023-07-18T03:40:23.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.672+0000 7f01e8293b80 1 --2- 172.21.15.40:0/3092495225 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e9c70 0x5626087ecb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.672+0000 7f01e8293b80 1 --2- 172.21.15.40:0/3092495225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e84d0 0x5626087f0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.672+0000 7f01e8293b80 1 --2- 172.21.15.40:0/3092495225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087f9620 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.672+0000 7f01e8293b80 1 -- 172.21.15.40:0/3092495225 >> 172.21.15.40:0/3092495225 conn(0x5626087ddda0 msgr2=0x5626087f0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:23.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.672+0000 7f01e8293b80 1 -- 172.21.15.40:0/3092495225 shutdown_connections 2023-07-18T03:40:23.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.672+0000 7f01e8293b80 1 -- 172.21.15.40:0/3092495225 wait complete. 2023-07-18T03:40:23.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.673+0000 7f01e8293b80 1 -- start start 2023-07-18T03:40:23.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.673+0000 7f01e8293b80 1 Processor -- start 2023-07-18T03:40:23.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.673+0000 7f01e8293b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e84d0 0x5626087f9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.673+0000 7f01e8293b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.673+0000 7f01e8293b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e9c70 0x5626087f6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.673+0000 7f01e8293b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626087f6ac0 con 0x5626087e8e20 2023-07-18T03:40:23.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.673+0000 7f01e8293b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562608851e00 con 0x5626087e9c70 2023-07-18T03:40:23.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.673+0000 7f01e8293b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562608851f40 con 0x5626087e84d0 2023-07-18T03:40:23.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.673+0000 7f01e8293b80 10 client.0 _ll_register_callbacks cb 0x5626086d7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:23.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.673+0000 7f01d96a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.673+0000 7f01da6aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e9c70 0x5626087f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.673+0000 7f01d96a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087f5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33082/0 (socket says 172.21.15.40:33082) 2023-07-18T03:40:23.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.673+0000 7f01d9ea9700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e84d0 0x5626087f9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.673+0000 7f01d96a8700 1 -- 172.21.15.40:0/2878417990 learned_addr learned my addr 172.21.15.40:0/2878417990 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:23.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.674+0000 7f01da6aa700 1 -- 172.21.15.40:0/2878417990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e84d0 msgr2=0x5626087f9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:23.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.674+0000 7f01da6aa700 1 --2- 172.21.15.40:0/2878417990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e84d0 0x5626087f9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.674+0000 7f01da6aa700 1 -- 172.21.15.40:0/2878417990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 msgr2=0x5626087f5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:23.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.674+0000 7f01da6aa700 1 --2- 172.21.15.40:0/2878417990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087f5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.674+0000 7f01da6aa700 1 -- 172.21.15.40:0/2878417990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d419fca0 con 0x5626087e9c70 2023-07-18T03:40:23.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.674+0000 7f01d96a8700 1 --2- 172.21.15.40:0/2878417990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087f5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:23.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.674+0000 7f01d9ea9700 1 --2- 172.21.15.40:0/2878417990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e84d0 0x5626087f9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:23.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.674+0000 7f01da6aa700 1 --2- 172.21.15.40:0/2878417990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e9c70 0x5626087f6470 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00b070 tx=0x7f01cc0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbf570ab0d9d7421 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:23.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.674+0000 7f01bd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:40:23.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.674+0000 7f01bd7fa700 1 -- 172.21.15.40:0/2878417990 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc029070 con 0x5626087e9c70 2023-07-18T03:40:23.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.674+0000 7f01e8293b80 1 -- 172.21.15.40:0/2878417990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01cc0076e0 con 0x5626087e9c70 2023-07-18T03:40:23.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.674+0000 7f01bd7fa700 1 -- 172.21.15.40:0/2878417990 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f01cc003c10 con 0x5626087e9c70 2023-07-18T03:40:23.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.674+0000 7f01bd7fa700 1 -- 172.21.15.40:0/2878417990 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01cc00e040 con 0x5626087e9c70 2023-07-18T03:40:23.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.674+0000 7f01e8293b80 10 client.26444 fetch_fsmap learned FSMap version 65 2023-07-18T03:40:23.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.674+0000 7f01e8293b80 1 -- 172.21.15.40:0/2878417990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x56260858e250 con 0x5626087e9c70 2023-07-18T03:40:23.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.675+0000 7f01bd7fa700 1 -- 172.21.15.40:0/2878417990 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01cc0076e0 con 0x5626087e9c70 2023-07-18T03:40:23.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.675+0000 7f01bd7fa700 1 -- 172.21.15.40:0/2878417990 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 65) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f01cc035020 con 0x5626087e9c70 2023-07-18T03:40:23.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.675+0000 7f01e8293b80 10 client.26444 fetch_fsmap finished waiting for FSMap version 65 2023-07-18T03:40:23.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.675+0000 7f01e8293b80 10 client.26444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:40:23.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.675+0000 7f01e8293b80 20 client.26444 populate_metadata read hostname 'smithi040' 2023-07-18T03:40:23.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.675+0000 7f01e8293b80 1 --2- 172.21.15.40:0/2878417990 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5626088534e0 0x5626088558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:23.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.675+0000 7f01e8293b80 4 client.26444 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:40:23.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.675+0000 7f01e8293b80 1 -- 172.21.15.40:0/2878417990 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5626086f5050 con 0x5626088534e0 2023-07-18T03:40:23.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.675+0000 7f01d9ea9700 1 --2- 172.21.15.40:0/2878417990 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5626088534e0 0x5626088558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:23.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.675+0000 7f01d9ea9700 1 --2- 172.21.15.40:0/2878417990 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5626088534e0 0x5626088558f0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee885dc9bb6cff8e server_cookie=81e2d6e41f8b1aae in_seq=0 out_seq=0 2023-07-18T03:40:23.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.675+0000 7f01bd7fa700 10 client.26444 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:23.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.675+0000 7f01bd7fa700 1 -- 172.21.15.40:0/2878417990 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5626086f5050 con 0x5626088534e0 2023-07-18T03:40:23.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.675+0000 7f01bd7fa700 10 client.26444 handle_command_reply: tid=0 2023-07-18T03:40:23.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.675+0000 7f01e8293b80 1 client.26444 shutdown 2023-07-18T03:40:23.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01e8293b80 1 -- 172.21.15.40:0/2878417990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e9c70 msgr2=0x5626087f6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01e8293b80 1 --2- 172.21.15.40:0/2878417990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e9c70 0x5626087f6470 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00b070 tx=0x7f01cc0095c0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01e8293b80 1 -- 172.21.15.40:0/2878417990 shutdown_connections 2023-07-18T03:40:23.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01e8293b80 1 --2- 172.21.15.40:0/2878417990 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5626088534e0 0x5626088558f0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01e8293b80 1 --2- 172.21.15.40:0/2878417990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626087e9c70 0x5626087f6470 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01bd7fa700 0 client.26444 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:23.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01e8293b80 1 --2- 172.21.15.40:0/2878417990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5626087e84d0 0x5626087f9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01e8293b80 1 --2- 172.21.15.40:0/2878417990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5626087e8e20 0x5626087f5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01e8293b80 1 -- 172.21.15.40:0/2878417990 >> 172.21.15.40:0/2878417990 conn(0x5626087ddda0 msgr2=0x5626087f0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:23.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01e8293b80 1 -- 172.21.15.40:0/2878417990 shutdown_connections 2023-07-18T03:40:23.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01e8293b80 1 -- 172.21.15.40:0/2878417990 wait complete. 2023-07-18T03:40:23.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01e8293b80 20 client.26444 trim_cache size 0 max 16384 2023-07-18T03:40:23.693 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:40:23.694 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-18T03:40:23.694 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:40:23.695 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:40:23.696 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:40:23.696 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:40:23.696 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:40:23.697 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:40:23.697 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:40:23.697 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:40:23.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01dc10d700 1 -- 172.21.15.40:0/2574031921 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f01c00a1150 msgr2=0x7f01c00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01dc10d700 1 --2- 172.21.15.40:0/2574031921 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f01c00a1150 0x7f01c00a3600 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009c60 tx=0x7f01c8009260 comp rx=0 tx=0).stop 2023-07-18T03:40:23.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01dc10d700 1 -- 172.21.15.40:0/2574031921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d4102e80 msgr2=0x7f01d40fe510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:23.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01dc10d700 1 --2- 172.21.15.40:0/2574031921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d4102e80 0x7f01d40fe510 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f01d000e9a0 tx=0x7f01d000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01dc10d700 1 -- 172.21.15.40:0/2574031921 shutdown_connections 2023-07-18T03:40:23.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01dc10d700 1 --2- 172.21.15.40:0/2574031921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01d41057a0 0x7f01d40fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01dc10d700 1 --2- 172.21.15.40:0/2574031921 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f01c00a1150 0x7f01c00a3600 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01dc10d700 1 --2- 172.21.15.40:0/2574031921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d4100560 0x7f01d40ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01dc10d700 1 --2- 172.21.15.40:0/2574031921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d4102e80 0x7f01d40fe510 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:23.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.676+0000 7f01dc10d700 1 -- 172.21.15.40:0/2574031921 >> 172.21.15.40:0/2574031921 conn(0x7f01d40fa000 msgr2=0x7f01d41011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:23.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.677+0000 7f01dc10d700 1 -- 172.21.15.40:0/2574031921 shutdown_connections 2023-07-18T03:40:23.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:23.677+0000 7f01dc10d700 1 -- 172.21.15.40:0/2574031921 wait complete. 2023-07-18T03:40:23.700 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:40:23.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:23 smithi134 ceph-mon[98216]: pgmap v579: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 403 op/s 2023-07-18T03:40:25.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:25 smithi040 ceph-mon[96439]: pgmap v580: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 328 op/s 2023-07-18T03:40:25.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:25 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:40:25.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:25 smithi064 ceph-mon[98059]: pgmap v580: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 328 op/s 2023-07-18T03:40:25.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:25 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:40:25.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:25 smithi134 ceph-mon[98216]: pgmap v580: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 328 op/s 2023-07-18T03:40:25.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:25 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:40:27.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:27 smithi040 ceph-mon[96439]: pgmap v581: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 345 op/s 2023-07-18T03:40:27.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:27 smithi064 ceph-mon[98059]: pgmap v581: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 345 op/s 2023-07-18T03:40:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.792+0000 7f68b2e68700 1 -- 172.21.15.40:0/1365561244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac1005c0 msgr2=0x7f68ac106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.792+0000 7f68b2e68700 1 --2- 172.21.15.40:0/1365561244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac1005c0 0x7f68ac106e80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f68a400b0a0 tx=0x7f68a400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.792+0000 7f68b2e68700 1 -- 172.21.15.40:0/1365561244 shutdown_connections 2023-07-18T03:40:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.792+0000 7f68b2e68700 1 --2- 172.21.15.40:0/1365561244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac1005c0 0x7f68ac106e80 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.792+0000 7f68b2e68700 1 --2- 172.21.15.40:0/1365561244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ac0fea40 0x7f68ac0fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.792+0000 7f68b2e68700 1 --2- 172.21.15.40:0/1365561244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac0ffc30 0x7f68ac100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.792+0000 7f68b2e68700 1 -- 172.21.15.40:0/1365561244 >> 172.21.15.40:0/1365561244 conn(0x7f68ac0fa060 msgr2=0x7f68ac0fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.792+0000 7f68b2e68700 1 -- 172.21.15.40:0/1365561244 shutdown_connections 2023-07-18T03:40:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.792+0000 7f68b2e68700 1 -- 172.21.15.40:0/1365561244 wait complete. 2023-07-18T03:40:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.793+0000 7f68b2e68700 1 Processor -- start 2023-07-18T03:40:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.793+0000 7f68b2e68700 1 -- start start 2023-07-18T03:40:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.793+0000 7f68b2e68700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac0fea40 0x7f68ac1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.793+0000 7f68b2e68700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac0ffc30 0x7f68ac19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.793+0000 7f68b2e68700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ac1005c0 0x7f68ac19d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.793+0000 7f68b2e68700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac19d640 con 0x7f68ac0ffc30 2023-07-18T03:40:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.793+0000 7f68b2e68700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac19d780 con 0x7f68ac0fea40 2023-07-18T03:40:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.793+0000 7f68b2e68700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac19d940 con 0x7f68ac1005c0 2023-07-18T03:40:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.794+0000 7f68abfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac0ffc30 0x7f68ac19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.794+0000 7f68abfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac0ffc30 0x7f68ac19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33096/0 (socket says 172.21.15.40:33096) 2023-07-18T03:40:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.794+0000 7f68abfff700 1 -- 172.21.15.40:0/367640349 learned_addr learned my addr 172.21.15.40:0/367640349 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.794+0000 7f68b0c04700 1 --2- 172.21.15.40:0/367640349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac0fea40 0x7f68ac1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.794+0000 7f68b1405700 1 --2- 172.21.15.40:0/367640349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ac1005c0 0x7f68ac19d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.794+0000 7f68abfff700 1 -- 172.21.15.40:0/367640349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ac1005c0 msgr2=0x7f68ac19d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:27.797 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.794+0000 7f68abfff700 1 --2- 172.21.15.40:0/367640349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ac1005c0 0x7f68ac19d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.794+0000 7f68abfff700 1 -- 172.21.15.40:0/367640349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac0fea40 msgr2=0x7f68ac1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.794+0000 7f68abfff700 1 --2- 172.21.15.40:0/367640349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac0fea40 0x7f68ac1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.794+0000 7f68abfff700 1 -- 172.21.15.40:0/367640349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a4009d00 con 0x7f68ac0ffc30 2023-07-18T03:40:27.798 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.794+0000 7f68b1405700 1 --2- 172.21.15.40:0/367640349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ac1005c0 0x7f68ac19d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.794+0000 7f68b0c04700 1 --2- 172.21.15.40:0/367640349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac0fea40 0x7f68ac1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.794+0000 7f68abfff700 1 --2- 172.21.15.40:0/367640349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac0ffc30 0x7f68ac19ab20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f689c00e9a0 tx=0x7f689c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.794+0000 7f68a9ffb700 1 -- 172.21.15.40:0/367640349 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c009790 con 0x7f68ac0ffc30 2023-07-18T03:40:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.794+0000 7f68a9ffb700 1 -- 172.21.15.40:0/367640349 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f689c00cbd0 con 0x7f68ac0ffc30 2023-07-18T03:40:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.794+0000 7f68b2e68700 1 -- 172.21.15.40:0/367640349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ac19dc20 con 0x7f68ac0ffc30 2023-07-18T03:40:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.794+0000 7f68a9ffb700 1 -- 172.21.15.40:0/367640349 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c018680 con 0x7f68ac0ffc30 2023-07-18T03:40:27.800 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.795+0000 7f68b2e68700 1 -- 172.21.15.40:0/367640349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac19e180 con 0x7f68ac0ffc30 2023-07-18T03:40:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.797+0000 7f68a9ffb700 1 -- 172.21.15.40:0/367640349 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f689c00cd40 con 0x7f68ac0ffc30 2023-07-18T03:40:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.798+0000 7f68a9ffb700 1 --2- 172.21.15.40:0/367640349 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f68940a11d0 0x7f68940a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.798+0000 7f68b0c04700 1 --2- 172.21.15.40:0/367640349 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f68940a11d0 0x7f68940a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.801 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.798+0000 7f68a9ffb700 1 -- 172.21.15.40:0/367640349 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f689c014070 con 0x7f68ac0ffc30 2023-07-18T03:40:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.798+0000 7f68b0c04700 1 --2- 172.21.15.40:0/367640349 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f68940a11d0 0x7f68940a3680 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f68ac068ee0 tx=0x7f68a0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.799+0000 7f68befeeb80 1 Processor -- start 2023-07-18T03:40:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.799+0000 7f68befeeb80 1 -- start start 2023-07-18T03:40:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.799+0000 7f68befeeb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce49136b0 0x555ce4ab7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.799+0000 7f68befeeb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab7840 0x555ce4ab81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.799+0000 7f68befeeb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce4ab86e0 0x555ce4ab8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.799+0000 7f68befeeb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce49bcc50 con 0x555ce49136b0 2023-07-18T03:40:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.799+0000 7f68befeeb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce4a0f9e0 con 0x555ce4ab86e0 2023-07-18T03:40:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.799+0000 7f68befeeb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce4a08e70 con 0x555ce4ab7840 2023-07-18T03:40:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.799+0000 7f68b0c04700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce49136b0 0x555ce4ab7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.799+0000 7f68b0c04700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce49136b0 0x555ce4ab7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33102/0 (socket says 172.21.15.40:33102) 2023-07-18T03:40:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.799+0000 7f68b0c04700 1 -- 172.21.15.40:0/2024742405 learned_addr learned my addr 172.21.15.40:0/2024742405 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.799+0000 7f68abfff700 1 --2- 172.21.15.40:0/2024742405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab7840 0x555ce4ab81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.799+0000 7f68b1405700 1 --2- 172.21.15.40:0/2024742405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce4ab86e0 0x555ce4ab8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.800+0000 7f68b0c04700 1 -- 172.21.15.40:0/2024742405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab7840 msgr2=0x555ce4ab81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.800+0000 7f68b0c04700 1 --2- 172.21.15.40:0/2024742405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab7840 0x555ce4ab81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.813 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.800+0000 7f68b0c04700 1 -- 172.21.15.40:0/2024742405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce4ab86e0 msgr2=0x555ce4ab8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.800+0000 7f68b0c04700 1 --2- 172.21.15.40:0/2024742405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce4ab86e0 0x555ce4ab8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.800+0000 7f68b0c04700 1 -- 172.21.15.40:0/2024742405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ce4836d80 con 0x555ce49136b0 2023-07-18T03:40:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.800+0000 7f68abfff700 1 --2- 172.21.15.40:0/2024742405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab7840 0x555ce4ab81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.800+0000 7f68b1405700 1 --2- 172.21.15.40:0/2024742405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce4ab86e0 0x555ce4ab8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.800+0000 7f68b0c04700 1 --2- 172.21.15.40:0/2024742405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce49136b0 0x555ce4ab7300 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f68a000ee00 tx=0x7f68a0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4c60e2d7a88b59 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.800+0000 7f6892ffd700 1 -- 172.21.15.40:0/2024742405 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a0015910 con 0x555ce49136b0 2023-07-18T03:40:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.800+0000 7f6892ffd700 1 -- 172.21.15.40:0/2024742405 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68a0013450 con 0x555ce49136b0 2023-07-18T03:40:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.800+0000 7f6892ffd700 1 -- 172.21.15.40:0/2024742405 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a001f650 con 0x555ce49136b0 2023-07-18T03:40:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.801+0000 7f68befeeb80 1 -- 172.21.15.40:0/2024742405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce49136b0 msgr2=0x555ce4ab7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.801+0000 7f68befeeb80 1 --2- 172.21.15.40:0/2024742405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce49136b0 0x555ce4ab7300 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f68a000ee00 tx=0x7f68a0011000 comp rx=0 tx=0).stop 2023-07-18T03:40:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.801+0000 7f68befeeb80 1 -- 172.21.15.40:0/2024742405 shutdown_connections 2023-07-18T03:40:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.801+0000 7f68befeeb80 1 --2- 172.21.15.40:0/2024742405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce4ab86e0 0x555ce4ab8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.801+0000 7f68befeeb80 1 --2- 172.21.15.40:0/2024742405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce49136b0 0x555ce4ab7300 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.801+0000 7f68befeeb80 1 --2- 172.21.15.40:0/2024742405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab7840 0x555ce4ab81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.801+0000 7f68befeeb80 1 -- 172.21.15.40:0/2024742405 >> 172.21.15.40:0/2024742405 conn(0x555ce4aacbe0 msgr2=0x555ce4ab1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.801+0000 7f68befeeb80 1 -- 172.21.15.40:0/2024742405 shutdown_connections 2023-07-18T03:40:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.801+0000 7f68befeeb80 1 -- 172.21.15.40:0/2024742405 wait complete. 2023-07-18T03:40:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.801+0000 7f68befeeb80 1 -- start start 2023-07-18T03:40:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.801+0000 7f68befeeb80 1 Processor -- start 2023-07-18T03:40:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68befeeb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce49136b0 0x555ce4ac4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68befeeb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce4ab7840 0x555ce4ac51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.817 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68befeeb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab86e0 0x555ce4acc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.817 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68befeeb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce49a8a60 con 0x555ce49136b0 2023-07-18T03:40:27.817 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68befeeb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce49a8760 con 0x555ce4ab7840 2023-07-18T03:40:27.817 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68befeeb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce488c8c0 con 0x555ce4ab86e0 2023-07-18T03:40:27.817 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68befeeb80 10 client.0 _ll_register_callbacks cb 0x555ce49a6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:27.817 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68b0c04700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce49136b0 0x555ce4ac4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68b0c04700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce49136b0 0x555ce4ac4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33108/0 (socket says 172.21.15.40:33108) 2023-07-18T03:40:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68b0c04700 1 -- 172.21.15.40:0/3679706672 learned_addr learned my addr 172.21.15.40:0/3679706672 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68abfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce4ab7840 0x555ce4ac51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68b1405700 1 --2- 172.21.15.40:0/3679706672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab86e0 0x555ce4acc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68b0c04700 1 -- 172.21.15.40:0/3679706672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab86e0 msgr2=0x555ce4acc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68b0c04700 1 --2- 172.21.15.40:0/3679706672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab86e0 0x555ce4acc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.818 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68b0c04700 1 -- 172.21.15.40:0/3679706672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce4ab7840 msgr2=0x555ce4ac51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68b0c04700 1 --2- 172.21.15.40:0/3679706672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce4ab7840 0x555ce4ac51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68b0c04700 1 -- 172.21.15.40:0/3679706672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac19e180 con 0x555ce49136b0 2023-07-18T03:40:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68abfff700 1 --2- 172.21.15.40:0/3679706672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce4ab7840 0x555ce4ac51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f68b0c04700 1 --2- 172.21.15.40:0/3679706672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce49136b0 0x555ce4ac4cb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f68a0016b50 tx=0x7f68a0016b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2424c14ac189a768 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f688bfff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:40:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.802+0000 7f688bfff700 1 -- 172.21.15.40:0/3679706672 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a00285f0 con 0x555ce49136b0 2023-07-18T03:40:27.819 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.803+0000 7f688bfff700 1 -- 172.21.15.40:0/3679706672 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f68a0013a80 con 0x555ce49136b0 2023-07-18T03:40:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.803+0000 7f68befeeb80 1 -- 172.21.15.40:0/3679706672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ce4962490 con 0x555ce49136b0 2023-07-18T03:40:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.803+0000 7f688bfff700 1 -- 172.21.15.40:0/3679706672 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68a0013ec0 con 0x555ce49136b0 2023-07-18T03:40:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.803+0000 7f68befeeb80 10 client.16800 fetch_fsmap learned FSMap version 66 2023-07-18T03:40:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.803+0000 7f68befeeb80 1 -- 172.21.15.40:0/3679706672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x7f68ac1077b0 con 0x555ce49136b0 2023-07-18T03:40:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.803+0000 7f688bfff700 1 -- 172.21.15.40:0/3679706672 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555ce4962490 con 0x555ce49136b0 2023-07-18T03:40:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.803+0000 7f688bfff700 1 -- 172.21.15.40:0/3679706672 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 66) v1 ==== 7530+0+0 (secure 0 0 0) 0x7f68a003a020 con 0x555ce49136b0 2023-07-18T03:40:27.820 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.803+0000 7f68befeeb80 10 client.16800 fetch_fsmap finished waiting for FSMap version 66 2023-07-18T03:40:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.803+0000 7f68befeeb80 10 client.16800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:40:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.803+0000 7f68befeeb80 20 client.16800 populate_metadata read hostname 'smithi040' 2023-07-18T03:40:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.803+0000 7f68befeeb80 1 --2- 172.21.15.40:0/3679706672 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555ce4ad0a90 0x555ce4ad2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.803+0000 7f68befeeb80 4 client.16800 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:40:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.803+0000 7f68befeeb80 1 -- 172.21.15.40:0/3679706672 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ce4a01e50 con 0x555ce4ad0a90 2023-07-18T03:40:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.803+0000 7f68abfff700 1 --2- 172.21.15.40:0/3679706672 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555ce4ad0a90 0x555ce4ad2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.821 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.804+0000 7f688bfff700 10 client.16800 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.804+0000 7f68abfff700 1 --2- 172.21.15.40:0/3679706672 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555ce4ad0a90 0x555ce4ad2ea0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86fd9c8ecc6e82b3 server_cookie=934cb71955e15c82 in_seq=0 out_seq=0 2023-07-18T03:40:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.806+0000 7f688bfff700 1 -- 172.21.15.40:0/3679706672 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555ce4a01e50 con 0x555ce4ad0a90 2023-07-18T03:40:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.806+0000 7f688bfff700 10 client.16800 handle_command_reply: tid=0 2023-07-18T03:40:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.806+0000 7f68befeeb80 1 client.16800 shutdown 2023-07-18T03:40:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.806+0000 7f68befeeb80 1 -- 172.21.15.40:0/3679706672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce49136b0 msgr2=0x555ce4ac4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.806+0000 7f68befeeb80 1 --2- 172.21.15.40:0/3679706672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce49136b0 0x555ce4ac4cb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f68a0016b50 tx=0x7f68a0016b80 comp rx=0 tx=0).stop 2023-07-18T03:40:27.822 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.806+0000 7f68befeeb80 1 -- 172.21.15.40:0/3679706672 shutdown_connections 2023-07-18T03:40:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.806+0000 7f68befeeb80 1 --2- 172.21.15.40:0/3679706672 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555ce4ad0a90 0x555ce4ad2ea0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.806+0000 7f68befeeb80 1 --2- 172.21.15.40:0/3679706672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab86e0 0x555ce4acc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.806+0000 7f688bfff700 0 client.16800 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.806+0000 7f68befeeb80 1 --2- 172.21.15.40:0/3679706672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce49136b0 0x555ce4ac4cb0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.806+0000 7f68befeeb80 1 --2- 172.21.15.40:0/3679706672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce4ab7840 0x555ce4ac51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.806+0000 7f68befeeb80 1 -- 172.21.15.40:0/3679706672 >> 172.21.15.40:0/3679706672 conn(0x555ce4aacbe0 msgr2=0x555ce4ab14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.806+0000 7f68befeeb80 1 -- 172.21.15.40:0/3679706672 shutdown_connections 2023-07-18T03:40:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.806+0000 7f68befeeb80 1 -- 172.21.15.40:0/3679706672 wait complete. 2023-07-18T03:40:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.806+0000 7f68befeeb80 20 client.16800 trim_cache size 0 max 16384 2023-07-18T03:40:27.824 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:27 smithi134 ceph-mon[98216]: pgmap v581: 97 pgs: 97 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 345 op/s 2023-07-18T03:40:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68befeeb80 1 Processor -- start 2023-07-18T03:40:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68befeeb80 1 -- start start 2023-07-18T03:40:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68befeeb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce49136b0 0x555ce4abf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68befeeb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce4ab7840 0x555ce4acc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68befeeb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab86e0 0x555ce4ad7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68befeeb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce4acdea0 con 0x555ce4ab7840 2023-07-18T03:40:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68befeeb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce4aaed00 con 0x555ce49136b0 2023-07-18T03:40:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68befeeb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce4abe830 con 0x555ce4ab86e0 2023-07-18T03:40:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68abfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce4ab7840 0x555ce4acc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68b0c04700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce49136b0 0x555ce4abf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68abfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce4ab7840 0x555ce4acc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33120/0 (socket says 172.21.15.40:33120) 2023-07-18T03:40:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68b0c04700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce49136b0 0x555ce4abf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:44674/0 (socket says 172.21.15.40:44674) 2023-07-18T03:40:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68abfff700 1 -- 172.21.15.40:0/4289594609 learned_addr learned my addr 172.21.15.40:0/4289594609 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68b1405700 1 --2- 172.21.15.40:0/4289594609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab86e0 0x555ce4ad7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68abfff700 1 -- 172.21.15.40:0/4289594609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab86e0 msgr2=0x555ce4ad7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68abfff700 1 --2- 172.21.15.40:0/4289594609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab86e0 0x555ce4ad7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68abfff700 1 -- 172.21.15.40:0/4289594609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce49136b0 msgr2=0x555ce4abf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:27.831 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68abfff700 1 --2- 172.21.15.40:0/4289594609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce49136b0 0x555ce4abf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.831 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68abfff700 1 -- 172.21.15.40:0/4289594609 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a000eae0 con 0x555ce4ab7840 2023-07-18T03:40:27.831 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68b1405700 1 --2- 172.21.15.40:0/4289594609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab86e0 0x555ce4ad7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:40:27.831 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.822+0000 7f68b0c04700 1 --2- 172.21.15.40:0/4289594609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce49136b0 0x555ce4abf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:27.831 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.823+0000 7f68abfff700 1 --2- 172.21.15.40:0/4289594609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce4ab7840 0x555ce4acc1f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f689c0ca9a0 tx=0x7f689c00efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae715ae8ee4c427a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:27.831 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.823+0000 7f68937fe700 1 -- 172.21.15.40:0/4289594609 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c0a2390 con 0x555ce4ab7840 2023-07-18T03:40:27.831 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.823+0000 7f68937fe700 1 -- 172.21.15.40:0/4289594609 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f689c0ca740 con 0x555ce4ab7840 2023-07-18T03:40:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.823+0000 7f68937fe700 1 -- 172.21.15.40:0/4289594609 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c094b50 con 0x555ce4ab7840 2023-07-18T03:40:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.823+0000 7f68befeeb80 1 -- 172.21.15.40:0/4289594609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce4ab7840 msgr2=0x555ce4acc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.823+0000 7f68befeeb80 1 --2- 172.21.15.40:0/4289594609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce4ab7840 0x555ce4acc1f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f689c0ca9a0 tx=0x7f689c00efd0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.823+0000 7f68befeeb80 1 -- 172.21.15.40:0/4289594609 shutdown_connections 2023-07-18T03:40:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.823+0000 7f68befeeb80 1 --2- 172.21.15.40:0/4289594609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab86e0 0x555ce4ad7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.823+0000 7f68befeeb80 1 --2- 172.21.15.40:0/4289594609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce49136b0 0x555ce4abf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.823+0000 7f68befeeb80 1 --2- 172.21.15.40:0/4289594609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce4ab7840 0x555ce4acc1f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.823+0000 7f68befeeb80 1 -- 172.21.15.40:0/4289594609 >> 172.21.15.40:0/4289594609 conn(0x555ce4aacbe0 msgr2=0x555ce4ab1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.823+0000 7f68befeeb80 1 -- 172.21.15.40:0/4289594609 shutdown_connections 2023-07-18T03:40:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.823+0000 7f68befeeb80 1 -- 172.21.15.40:0/4289594609 wait complete. 2023-07-18T03:40:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.824+0000 7f68befeeb80 1 -- start start 2023-07-18T03:40:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.824+0000 7f68befeeb80 1 Processor -- start 2023-07-18T03:40:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.824+0000 7f68befeeb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce49136b0 0x555ce4ac7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.824+0000 7f68befeeb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce4ab7840 0x555ce4ac8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.824+0000 7f68befeeb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab86e0 0x555ce4acb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.824+0000 7f68befeeb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce4acb7c0 con 0x555ce4ab7840 2023-07-18T03:40:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.824+0000 7f68befeeb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce4acb930 con 0x555ce49136b0 2023-07-18T03:40:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.824+0000 7f68befeeb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce4acbaf0 con 0x555ce4ab86e0 2023-07-18T03:40:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f68befeeb80 10 client.0 _ll_register_callbacks cb 0x7f68a4009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f68abfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce4ab7840 0x555ce4ac8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f68b0c04700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce49136b0 0x555ce4ac7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f68abfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce4ab7840 0x555ce4ac8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33124/0 (socket says 172.21.15.40:33124) 2023-07-18T03:40:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f68abfff700 1 -- 172.21.15.40:0/2856765991 learned_addr learned my addr 172.21.15.40:0/2856765991 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f68b1405700 1 --2- 172.21.15.40:0/2856765991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab86e0 0x555ce4acb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f68abfff700 1 -- 172.21.15.40:0/2856765991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab86e0 msgr2=0x555ce4acb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f68abfff700 1 --2- 172.21.15.40:0/2856765991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab86e0 0x555ce4acb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f68abfff700 1 -- 172.21.15.40:0/2856765991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce49136b0 msgr2=0x555ce4ac7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f68abfff700 1 --2- 172.21.15.40:0/2856765991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce49136b0 0x555ce4ac7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f68abfff700 1 -- 172.21.15.40:0/2856765991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac1077b0 con 0x555ce4ab7840 2023-07-18T03:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f68abfff700 1 --2- 172.21.15.40:0/2856765991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce4ab7840 0x555ce4ac8260 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f689c0ca680 tx=0x7f689c0963b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bcfe83cae02897 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f688bfff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f688bfff700 1 -- 172.21.15.40:0/2856765991 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c098080 con 0x555ce4ab7840 2023-07-18T03:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f688bfff700 1 -- 172.21.15.40:0/2856765991 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f689c0a2390 con 0x555ce4ab7840 2023-07-18T03:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f68befeeb80 1 -- 172.21.15.40:0/2856765991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68a0013450 con 0x555ce4ab7840 2023-07-18T03:40:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f688bfff700 1 -- 172.21.15.40:0/2856765991 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f689c094b50 con 0x555ce4ab7840 2023-07-18T03:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f68befeeb80 10 client.16806 fetch_fsmap learned FSMap version 66 2023-07-18T03:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.825+0000 7f68befeeb80 1 -- 172.21.15.40:0/2856765991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x7f68ac19dc20 con 0x555ce4ab7840 2023-07-18T03:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.826+0000 7f688bfff700 1 -- 172.21.15.40:0/2856765991 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f689c07c080 con 0x555ce4ab7840 2023-07-18T03:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.826+0000 7f688bfff700 1 -- 172.21.15.40:0/2856765991 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 66) v1 ==== 7530+0+0 (secure 0 0 0) 0x7f689c098ea0 con 0x555ce4ab7840 2023-07-18T03:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.826+0000 7f68befeeb80 10 client.16806 fetch_fsmap finished waiting for FSMap version 66 2023-07-18T03:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.826+0000 7f68befeeb80 10 client.16806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:40:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.826+0000 7f68befeeb80 20 client.16806 populate_metadata read hostname 'smithi040' 2023-07-18T03:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.826+0000 7f68befeeb80 1 --2- 172.21.15.40:0/2856765991 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555ce4b25510 0x555ce4b27920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.826+0000 7f68befeeb80 4 client.16806 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.826+0000 7f68befeeb80 1 -- 172.21.15.40:0/2856765991 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555ce49c3ff0 con 0x555ce4b25510 2023-07-18T03:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.826+0000 7f68b0c04700 1 --2- 172.21.15.40:0/2856765991 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555ce4b25510 0x555ce4b27920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.826+0000 7f688bfff700 10 client.16806 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:27.838 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:27.826+0000 7f68b0c04700 1 --2- 172.21.15.40:0/2856765991 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555ce4b25510 0x555ce4b27920 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2553503f6c265a71 server_cookie=1c169b81301c9666 in_seq=0 out_seq=0 2023-07-18T03:40:29.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:29 smithi040 ceph-mon[96439]: pgmap v582: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 372 op/s 2023-07-18T03:40:29.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:29 smithi064 ceph-mon[98059]: pgmap v582: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 372 op/s 2023-07-18T03:40:29.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:29 smithi134 ceph-mon[98216]: pgmap v582: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 372 op/s 2023-07-18T03:40:30.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:30 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:40:30.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:30 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:40:30.827 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:30 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:40:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.973+0000 7f688bfff700 1 -- 172.21.15.40:0/2856765991 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555ce49c3ff0 con 0x555ce4b25510 2023-07-18T03:40:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.973+0000 7f688bfff700 10 client.16806 handle_command_reply: tid=0 2023-07-18T03:40:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.973+0000 7f68befeeb80 1 client.16806 shutdown 2023-07-18T03:40:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.973+0000 7f68befeeb80 1 -- 172.21.15.40:0/2856765991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce4ab7840 msgr2=0x555ce4ac8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.973+0000 7f68befeeb80 1 --2- 172.21.15.40:0/2856765991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce4ab7840 0x555ce4ac8260 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f689c0ca680 tx=0x7f689c0963b0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.973+0000 7f68befeeb80 1 -- 172.21.15.40:0/2856765991 shutdown_connections 2023-07-18T03:40:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.973+0000 7f68befeeb80 1 --2- 172.21.15.40:0/2856765991 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x555ce4b25510 0x555ce4b27920 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.973+0000 7f68befeeb80 1 --2- 172.21.15.40:0/2856765991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ce4ab86e0 0x555ce4acb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.973+0000 7f68befeeb80 1 --2- 172.21.15.40:0/2856765991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ce49136b0 0x555ce4ac7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.973+0000 7f688bfff700 0 client.16806 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.973+0000 7f68befeeb80 1 --2- 172.21.15.40:0/2856765991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555ce4ab7840 0x555ce4ac8260 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.973+0000 7f68befeeb80 1 -- 172.21.15.40:0/2856765991 >> 172.21.15.40:0/2856765991 conn(0x555ce4aacbe0 msgr2=0x555ce4abb590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.973+0000 7f68befeeb80 1 -- 172.21.15.40:0/2856765991 shutdown_connections 2023-07-18T03:40:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.973+0000 7f68befeeb80 1 -- 172.21.15.40:0/2856765991 wait complete. 2023-07-18T03:40:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.974+0000 7f68befeeb80 20 client.16806 trim_cache size 0 max 16384 2023-07-18T03:40:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:40:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:40:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:40:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:40:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.974+0000 7f68b2e68700 1 -- 172.21.15.40:0/367640349 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f68940a11d0 msgr2=0x7f68940a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.974+0000 7f68b2e68700 1 --2- 172.21.15.40:0/367640349 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f68940a11d0 0x7f68940a3680 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f68ac068ee0 tx=0x7f68a0009190 comp rx=0 tx=0).stop 2023-07-18T03:40:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.974+0000 7f68b2e68700 1 -- 172.21.15.40:0/367640349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac0ffc30 msgr2=0x7f68ac19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.974+0000 7f68b2e68700 1 --2- 172.21.15.40:0/367640349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac0ffc30 0x7f68ac19ab20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f689c00e9a0 tx=0x7f689c00ecb0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.974+0000 7f68b2e68700 1 -- 172.21.15.40:0/367640349 shutdown_connections 2023-07-18T03:40:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.974+0000 7f68b2e68700 1 --2- 172.21.15.40:0/367640349 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f68940a11d0 0x7f68940a3680 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.974+0000 7f68b2e68700 1 --2- 172.21.15.40:0/367640349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68ac1005c0 0x7f68ac19d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.974+0000 7f68b2e68700 1 --2- 172.21.15.40:0/367640349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68ac0fea40 0x7f68ac1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.974+0000 7f68b2e68700 1 --2- 172.21.15.40:0/367640349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68ac0ffc30 0x7f68ac19ab20 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.974+0000 7f68b2e68700 1 -- 172.21.15.40:0/367640349 >> 172.21.15.40:0/367640349 conn(0x7f68ac0fa060 msgr2=0x7f68ac103080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.975+0000 7f68b2e68700 1 -- 172.21.15.40:0/367640349 shutdown_connections 2023-07-18T03:40:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:30.975+0000 7f68b2e68700 1 -- 172.21.15.40:0/367640349 wait complete. 2023-07-18T03:40:31.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:31 smithi064 ceph-mon[98059]: pgmap v583: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1011 KiB/s wr, 270 op/s 2023-07-18T03:40:31.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:31 smithi134 ceph-mon[98216]: pgmap v583: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1011 KiB/s wr, 270 op/s 2023-07-18T03:40:31.827 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:31 smithi040 ceph-mon[96439]: pgmap v583: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1011 KiB/s wr, 270 op/s 2023-07-18T03:40:33.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:32 smithi134 ceph-mon[98216]: pgmap v584: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 355 op/s 2023-07-18T03:40:33.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:32 smithi040 ceph-mon[96439]: pgmap v584: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 355 op/s 2023-07-18T03:40:33.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:32 smithi064 ceph-mon[98059]: pgmap v584: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 355 op/s 2023-07-18T03:40:34.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:33 smithi134 ceph-mon[98216]: scrub summary: active paths [/] 2023-07-18T03:40:34.366 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:33 smithi064 ceph-mon[98059]: scrub summary: active paths [/] 2023-07-18T03:40:34.368 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:33 smithi040 ceph-mon[96439]: scrub summary: active paths [/] 2023-07-18T03:40:35.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:34 smithi134 ceph-mon[98216]: pgmap v585: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 895 KiB/s wr, 270 op/s 2023-07-18T03:40:35.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:34 smithi040 ceph-mon[96439]: pgmap v585: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 895 KiB/s wr, 270 op/s 2023-07-18T03:40:35.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:34 smithi064 ceph-mon[98059]: pgmap v585: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 895 KiB/s wr, 270 op/s 2023-07-18T03:40:37.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:36 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:40:37.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:36 smithi134 ceph-mon[98216]: pgmap v586: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 277 op/s 2023-07-18T03:40:37.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:36 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:40:37.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:36 smithi040 ceph-mon[96439]: pgmap v586: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 277 op/s 2023-07-18T03:40:37.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:36 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:40:37.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:36 smithi064 ceph-mon[98059]: pgmap v586: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 277 op/s 2023-07-18T03:40:39.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:39 smithi134 ceph-mon[98216]: pgmap v587: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 284 op/s 2023-07-18T03:40:39.565 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:39 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:40:39.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:39 smithi040 ceph-mon[96439]: pgmap v587: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 284 op/s 2023-07-18T03:40:39.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:39 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:40:39.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:39 smithi064 ceph-mon[98059]: pgmap v587: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 284 op/s 2023-07-18T03:40:39.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:39 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:40:41.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:41 smithi134 ceph-mon[98216]: pgmap v588: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 767 KiB/s wr, 201 op/s 2023-07-18T03:40:41.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:41 smithi040 ceph-mon[96439]: pgmap v588: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 767 KiB/s wr, 201 op/s 2023-07-18T03:40:41.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:41 smithi064 ceph-mon[98059]: pgmap v588: 97 pgs: 97 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 767 KiB/s wr, 201 op/s 2023-07-18T03:40:43.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:42 smithi134 ceph-mon[98216]: pgmap v589: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 296 op/s 2023-07-18T03:40:43.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:42 smithi040 ceph-mon[96439]: pgmap v589: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 296 op/s 2023-07-18T03:40:43.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:42 smithi064 ceph-mon[98059]: pgmap v589: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 296 op/s 2023-07-18T03:40:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.093+0000 7ff4d39d1700 1 -- 172.21.15.40:0/938387713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4cc100640 msgr2=0x7ff4cc106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.093+0000 7ff4d39d1700 1 --2- 172.21.15.40:0/938387713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4cc100640 0x7ff4cc106ee0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009ec0 tx=0x7ff4c400b330 comp rx=0 tx=0).stop 2023-07-18T03:40:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.093+0000 7ff4d39d1700 1 -- 172.21.15.40:0/938387713 shutdown_connections 2023-07-18T03:40:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.093+0000 7ff4d39d1700 1 --2- 172.21.15.40:0/938387713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4cc100640 0x7ff4cc106ee0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.093+0000 7ff4d39d1700 1 --2- 172.21.15.40:0/938387713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4cc0fea00 0x7ff4cc0fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.093+0000 7ff4d39d1700 1 --2- 172.21.15.40:0/938387713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4cc0ffc90 0x7ff4cc100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.093+0000 7ff4d39d1700 1 -- 172.21.15.40:0/938387713 >> 172.21.15.40:0/938387713 conn(0x7ff4cc0fa000 msgr2=0x7ff4cc0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.093+0000 7ff4d39d1700 1 -- 172.21.15.40:0/938387713 shutdown_connections 2023-07-18T03:40:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.093+0000 7ff4d39d1700 1 -- 172.21.15.40:0/938387713 wait complete. 2023-07-18T03:40:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.094+0000 7ff4d39d1700 1 Processor -- start 2023-07-18T03:40:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.097+0000 7ff4d39d1700 1 -- start start 2023-07-18T03:40:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.097+0000 7ff4d39d1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4cc0fea00 0x7ff4cc06ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.097+0000 7ff4d39d1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4cc0ffc90 0x7ff4cc06f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.097+0000 7ff4d39d1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4cc100640 0x7ff4cc0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.097+0000 7ff4d39d1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc070a00 con 0x7ff4cc0ffc90 2023-07-18T03:40:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.097+0000 7ff4d39d1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc070b70 con 0x7ff4cc100640 2023-07-18T03:40:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.097+0000 7ff4d39d1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc070d30 con 0x7ff4cc0fea00 2023-07-18T03:40:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.099+0000 7ff4d1f6e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4cc100640 0x7ff4cc0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.099+0000 7ff4d176d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4cc0fea00 0x7ff4cc06ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.099+0000 7ff4d176d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4cc0fea00 0x7ff4cc06ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50408/0 (socket says 172.21.15.40:50408) 2023-07-18T03:40:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.099+0000 7ff4d176d700 1 -- 172.21.15.40:0/3632669251 learned_addr learned my addr 172.21.15.40:0/3632669251 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.100+0000 7ff4d0f6c700 1 --2- 172.21.15.40:0/3632669251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4cc0ffc90 0x7ff4cc06f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.100+0000 7ff4d1f6e700 1 -- 172.21.15.40:0/3632669251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4cc0fea00 msgr2=0x7ff4cc06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.100+0000 7ff4d1f6e700 1 --2- 172.21.15.40:0/3632669251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4cc0fea00 0x7ff4cc06ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.100+0000 7ff4d1f6e700 1 -- 172.21.15.40:0/3632669251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4cc0ffc90 msgr2=0x7ff4cc06f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.100+0000 7ff4d1f6e700 1 --2- 172.21.15.40:0/3632669251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4cc0ffc90 0x7ff4cc06f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.097 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.100+0000 7ff4d1f6e700 1 -- 172.21.15.40:0/3632669251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c4005e70 con 0x7ff4cc100640 2023-07-18T03:40:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.106+0000 7ff4d1f6e700 1 --2- 172.21.15.40:0/3632669251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4cc100640 0x7ff4cc0703a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009ec0 tx=0x7ff4c4007f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.108+0000 7ff4c27fc700 1 -- 172.21.15.40:0/3632669251 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400e040 con 0x7ff4cc100640 2023-07-18T03:40:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.108+0000 7ff4c27fc700 1 -- 172.21.15.40:0/3632669251 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4c401cd80 con 0x7ff4cc100640 2023-07-18T03:40:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.108+0000 7ff4c27fc700 1 -- 172.21.15.40:0/3632669251 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4012ce0 con 0x7ff4cc100640 2023-07-18T03:40:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.114+0000 7ff4d39d1700 1 -- 172.21.15.40:0/3632669251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc070fb0 con 0x7ff4cc100640 2023-07-18T03:40:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.114+0000 7ff4d39d1700 1 -- 172.21.15.40:0/3632669251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc067b60 con 0x7ff4cc100640 2023-07-18T03:40:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.117+0000 7ff4c27fc700 1 -- 172.21.15.40:0/3632669251 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff4c40123f0 con 0x7ff4cc100640 2023-07-18T03:40:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.117+0000 7ff4c27fc700 1 --2- 172.21.15.40:0/3632669251 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4b80a12a0 0x7ff4b80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.118+0000 7ff4c27fc700 1 -- 172.21.15.40:0/3632669251 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff4c40d1cc0 con 0x7ff4cc100640 2023-07-18T03:40:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.118+0000 7ff4d176d700 1 --2- 172.21.15.40:0/3632669251 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4b80a12a0 0x7ff4b80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.118+0000 7ff4d176d700 1 --2- 172.21.15.40:0/3632669251 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4b80a12a0 0x7ff4b80a3750 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800b240 tx=0x7ff4c800b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4dfb57b80 1 Processor -- start 2023-07-18T03:40:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4dfb57b80 1 -- start start 2023-07-18T03:40:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4dfb57b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec0fc16b0 0x564ec1165300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4dfb57b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec1165840 0x564ec11661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.120 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4dfb57b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec11666e0 0x564ec1166b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4dfb57b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec106ac50 con 0x564ec0fc16b0 2023-07-18T03:40:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4dfb57b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec10bd9e0 con 0x564ec11666e0 2023-07-18T03:40:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4dfb57b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec10b6e70 con 0x564ec1165840 2023-07-18T03:40:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4d176d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec0fc16b0 0x564ec1165300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4d176d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec0fc16b0 0x564ec1165300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33154/0 (socket says 172.21.15.40:33154) 2023-07-18T03:40:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4d176d700 1 -- 172.21.15.40:0/3530142447 learned_addr learned my addr 172.21.15.40:0/3530142447 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4d1f6e700 1 --2- 172.21.15.40:0/3530142447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec11666e0 0x564ec1166b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4d0f6c700 1 --2- 172.21.15.40:0/3530142447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec1165840 0x564ec11661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4d1f6e700 1 -- 172.21.15.40:0/3530142447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec1165840 msgr2=0x564ec11661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4d1f6e700 1 --2- 172.21.15.40:0/3530142447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec1165840 0x564ec11661a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4d1f6e700 1 -- 172.21.15.40:0/3530142447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec0fc16b0 msgr2=0x564ec1165300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4d1f6e700 1 --2- 172.21.15.40:0/3530142447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec0fc16b0 0x564ec1165300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4d1f6e700 1 -- 172.21.15.40:0/3530142447 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ec0ee4d80 con 0x564ec11666e0 2023-07-18T03:40:44.122 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.119+0000 7ff4d176d700 1 --2- 172.21.15.40:0/3530142447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec0fc16b0 0x564ec1165300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.120+0000 7ff4d1f6e700 1 --2- 172.21.15.40:0/3530142447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec11666e0 0x564ec1166b80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40cf7f0 tx=0x7ff4c40099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82e2b01e55a89478 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.120+0000 7ff4b77fe700 1 -- 172.21.15.40:0/3530142447 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4012e40 con 0x564ec11666e0 2023-07-18T03:40:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.120+0000 7ff4b77fe700 1 -- 172.21.15.40:0/3530142447 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4c409f140 con 0x564ec11666e0 2023-07-18T03:40:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.120+0000 7ff4b77fe700 1 -- 172.21.15.40:0/3530142447 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c40a7070 con 0x564ec11666e0 2023-07-18T03:40:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.120+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3530142447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec11666e0 msgr2=0x564ec1166b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.120+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/3530142447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec11666e0 0x564ec1166b80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40cf7f0 tx=0x7ff4c40099e0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.120+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3530142447 shutdown_connections 2023-07-18T03:40:44.123 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.120+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/3530142447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec11666e0 0x564ec1166b80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.121+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/3530142447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec0fc16b0 0x564ec1165300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.121+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/3530142447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec1165840 0x564ec11661a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.121+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3530142447 >> 172.21.15.40:0/3530142447 conn(0x564ec115abe0 msgr2=0x564ec115fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.121+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3530142447 shutdown_connections 2023-07-18T03:40:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.121+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3530142447 wait complete. 2023-07-18T03:40:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.121+0000 7ff4dfb57b80 1 -- start start 2023-07-18T03:40:44.124 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.121+0000 7ff4dfb57b80 1 Processor -- start 2023-07-18T03:40:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.121+0000 7ff4dfb57b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec0fc16b0 0x564ec116e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.121+0000 7ff4dfb57b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec1165840 0x564ec116eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.121+0000 7ff4dfb57b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec11666e0 0x564ec1175b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.121+0000 7ff4dfb57b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec1056a60 con 0x564ec11666e0 2023-07-18T03:40:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.121+0000 7ff4dfb57b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec1056760 con 0x564ec0fc16b0 2023-07-18T03:40:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.121+0000 7ff4dfb57b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec0f3a8c0 con 0x564ec1165840 2023-07-18T03:40:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.121+0000 7ff4dfb57b80 10 client.0 _ll_register_callbacks cb 0x564ec1054700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.121+0000 7ff4d1f6e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec11666e0 0x564ec1175b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4d176d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec0fc16b0 0x564ec116e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4d0f6c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec1165840 0x564ec116eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4d1f6e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec11666e0 0x564ec1175b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33164/0 (socket says 172.21.15.40:33164) 2023-07-18T03:40:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4d0f6c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec1165840 0x564ec116eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50422/0 (socket says 172.21.15.40:50422) 2023-07-18T03:40:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4d176d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec0fc16b0 0x564ec116e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:44718/0 (socket says 172.21.15.40:44718) 2023-07-18T03:40:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4d1f6e700 1 -- 172.21.15.40:0/3754100923 learned_addr learned my addr 172.21.15.40:0/3754100923 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4d1f6e700 1 -- 172.21.15.40:0/3754100923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec1165840 msgr2=0x564ec116eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4d1f6e700 1 --2- 172.21.15.40:0/3754100923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec1165840 0x564ec116eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4d1f6e700 1 -- 172.21.15.40:0/3754100923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec0fc16b0 msgr2=0x564ec116e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4d1f6e700 1 --2- 172.21.15.40:0/3754100923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec0fc16b0 0x564ec116e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4d1f6e700 1 -- 172.21.15.40:0/3754100923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c800e970 con 0x564ec11666e0 2023-07-18T03:40:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4d0f6c700 1 --2- 172.21.15.40:0/3754100923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec1165840 0x564ec116eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4d1f6e700 1 --2- 172.21.15.40:0/3754100923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec11666e0 0x564ec1175b40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff4c409b640 tx=0x7ff4c40126e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3bf6a46e8eda02 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:40:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4b4ff9700 1 -- 172.21.15.40:0/3754100923 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c40a7270 con 0x564ec11666e0 2023-07-18T03:40:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4b4ff9700 1 -- 172.21.15.40:0/3754100923 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff4c4012e40 con 0x564ec11666e0 2023-07-18T03:40:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3754100923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ec1010490 con 0x564ec11666e0 2023-07-18T03:40:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4b4ff9700 1 -- 172.21.15.40:0/3754100923 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4c40bd200 con 0x564ec11666e0 2023-07-18T03:40:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4dfb57b80 10 client.16818 fetch_fsmap learned FSMap version 68 2023-07-18T03:40:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.122+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3754100923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7ff4cc067b60 con 0x564ec11666e0 2023-07-18T03:40:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.123+0000 7ff4b4ff9700 1 -- 172.21.15.40:0/3754100923 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564ec1010490 con 0x564ec11666e0 2023-07-18T03:40:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.123+0000 7ff4b4ff9700 1 -- 172.21.15.40:0/3754100923 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 68) v1 ==== 7538+0+0 (secure 0 0 0) 0x7ff4c409afc0 con 0x564ec11666e0 2023-07-18T03:40:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.123+0000 7ff4dfb57b80 10 client.16818 fetch_fsmap finished waiting for FSMap version 68 2023-07-18T03:40:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.123+0000 7ff4dfb57b80 10 client.16818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:40:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.123+0000 7ff4dfb57b80 20 client.16818 populate_metadata read hostname 'smithi040' 2023-07-18T03:40:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.123+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/3754100923 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564ec117a010 0x564ec117c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.123+0000 7ff4dfb57b80 4 client.16818 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:40:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.123+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3754100923 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ec10afe50 con 0x564ec117a010 2023-07-18T03:40:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.123+0000 7ff4d176d700 1 --2- 172.21.15.40:0/3754100923 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564ec117a010 0x564ec117c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.123+0000 7ff4b4ff9700 10 client.16818 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.123+0000 7ff4d176d700 1 --2- 172.21.15.40:0/3754100923 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564ec117a010 0x564ec117c420 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23db6c09a5b6bf53 server_cookie=fecc6c0c4265789c in_seq=0 out_seq=0 2023-07-18T03:40:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.124+0000 7ff4b4ff9700 1 -- 172.21.15.40:0/3754100923 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564ec10afe50 con 0x564ec117a010 2023-07-18T03:40:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.124+0000 7ff4b4ff9700 10 client.16818 handle_command_reply: tid=0 2023-07-18T03:40:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.124+0000 7ff4dfb57b80 1 client.16818 shutdown 2023-07-18T03:40:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.124+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3754100923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec11666e0 msgr2=0x564ec1175b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.124+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/3754100923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec11666e0 0x564ec1175b40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff4c409b640 tx=0x7ff4c40126e0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.124+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3754100923 shutdown_connections 2023-07-18T03:40:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.124+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/3754100923 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564ec117a010 0x564ec117c420 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.124+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/3754100923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec11666e0 0x564ec1175b40 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.124+0000 7ff4b4ff9700 0 client.16818 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.124+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/3754100923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec0fc16b0 0x564ec116e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.124+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/3754100923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec1165840 0x564ec116eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.124+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3754100923 >> 172.21.15.40:0/3754100923 conn(0x564ec115abe0 msgr2=0x564ec115f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.124+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3754100923 shutdown_connections 2023-07-18T03:40:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.124+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3754100923 wait complete. 2023-07-18T03:40:44.132 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.125+0000 7ff4dfb57b80 20 client.16818 trim_cache size 0 max 16384 2023-07-18T03:40:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.140+0000 7ff4dfb57b80 1 Processor -- start 2023-07-18T03:40:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.140+0000 7ff4dfb57b80 1 -- start start 2023-07-18T03:40:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.140+0000 7ff4dfb57b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec0fc16b0 0x564ec116c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.140+0000 7ff4dfb57b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec1165840 0x564ec1175b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.140+0000 7ff4dfb57b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec11666e0 0x564ec116cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.140+0000 7ff4dfb57b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec10104c0 con 0x564ec1165840 2023-07-18T03:40:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.140+0000 7ff4dfb57b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec112b090 con 0x564ec11666e0 2023-07-18T03:40:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.140+0000 7ff4dfb57b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec116e130 con 0x564ec0fc16b0 2023-07-18T03:40:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.140+0000 7ff4d0f6c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec1165840 0x564ec1175b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.141+0000 7ff4d0f6c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec1165840 0x564ec1175b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33170/0 (socket says 172.21.15.40:33170) 2023-07-18T03:40:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.141+0000 7ff4d0f6c700 1 -- 172.21.15.40:0/3025423840 learned_addr learned my addr 172.21.15.40:0/3025423840 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.141+0000 7ff4d176d700 1 --2- 172.21.15.40:0/3025423840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec0fc16b0 0x564ec116c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.141+0000 7ff4d1f6e700 1 --2- 172.21.15.40:0/3025423840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec11666e0 0x564ec116cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.141+0000 7ff4d0f6c700 1 -- 172.21.15.40:0/3025423840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec0fc16b0 msgr2=0x564ec116c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.141+0000 7ff4d0f6c700 1 --2- 172.21.15.40:0/3025423840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec0fc16b0 0x564ec116c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.141+0000 7ff4d0f6c700 1 -- 172.21.15.40:0/3025423840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec11666e0 msgr2=0x564ec116cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.141+0000 7ff4d0f6c700 1 --2- 172.21.15.40:0/3025423840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec11666e0 0x564ec116cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.141+0000 7ff4d0f6c700 1 -- 172.21.15.40:0/3025423840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ec0ecdc00 con 0x564ec1165840 2023-07-18T03:40:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.141+0000 7ff4d1f6e700 1 --2- 172.21.15.40:0/3025423840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec11666e0 0x564ec116cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.141+0000 7ff4d0f6c700 1 --2- 172.21.15.40:0/3025423840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec1165840 0x564ec1175b00 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009f20 tx=0x7ff4bc009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4407a75414d7a6a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.141+0000 7ff4b7fff700 1 -- 172.21.15.40:0/3025423840 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc014ae0 con 0x564ec1165840 2023-07-18T03:40:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.141+0000 7ff4b7fff700 1 -- 172.21.15.40:0/3025423840 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4bc014c40 con 0x564ec1165840 2023-07-18T03:40:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.141+0000 7ff4b7fff700 1 -- 172.21.15.40:0/3025423840 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc015680 con 0x564ec1165840 2023-07-18T03:40:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.142+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3025423840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec1165840 msgr2=0x564ec1175b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.142+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/3025423840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec1165840 0x564ec1175b00 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009f20 tx=0x7ff4bc009f50 comp rx=0 tx=0).stop 2023-07-18T03:40:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.142+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3025423840 shutdown_connections 2023-07-18T03:40:44.141 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.142+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/3025423840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec11666e0 0x564ec116cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.142+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/3025423840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec0fc16b0 0x564ec116c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.142+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/3025423840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec1165840 0x564ec1175b00 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.142+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3025423840 >> 172.21.15.40:0/3025423840 conn(0x564ec115abe0 msgr2=0x564ec116e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.142+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3025423840 shutdown_connections 2023-07-18T03:40:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.142+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/3025423840 wait complete. 2023-07-18T03:40:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.142+0000 7ff4dfb57b80 1 -- start start 2023-07-18T03:40:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.142+0000 7ff4dfb57b80 1 Processor -- start 2023-07-18T03:40:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.142+0000 7ff4dfb57b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec0fc16b0 0x564ec1171530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.142+0000 7ff4dfb57b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec1165840 0x564ec1171a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.142+0000 7ff4dfb57b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec11666e0 0x564ec1174a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.142+0000 7ff4dfb57b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec1174fd0 con 0x564ec1165840 2023-07-18T03:40:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.142+0000 7ff4dfb57b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec1175140 con 0x564ec0fc16b0 2023-07-18T03:40:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.142+0000 7ff4dfb57b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ec1175300 con 0x564ec11666e0 2023-07-18T03:40:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4dfb57b80 10 client.0 _ll_register_callbacks cb 0x7ff4c8014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4d0f6c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec1165840 0x564ec1171a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4d0f6c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec1165840 0x564ec1171a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33176/0 (socket says 172.21.15.40:33176) 2023-07-18T03:40:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4d0f6c700 1 -- 172.21.15.40:0/88290019 learned_addr learned my addr 172.21.15.40:0/88290019 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4d1f6e700 1 --2- 172.21.15.40:0/88290019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec11666e0 0x564ec1174a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4d176d700 1 --2- 172.21.15.40:0/88290019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec0fc16b0 0x564ec1171530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4d0f6c700 1 -- 172.21.15.40:0/88290019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec11666e0 msgr2=0x564ec1174a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4d0f6c700 1 --2- 172.21.15.40:0/88290019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec11666e0 0x564ec1174a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4d0f6c700 1 -- 172.21.15.40:0/88290019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec0fc16b0 msgr2=0x564ec1171530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:44.144 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4d0f6c700 1 --2- 172.21.15.40:0/88290019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec0fc16b0 0x564ec1171530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4d0f6c700 1 -- 172.21.15.40:0/88290019 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4bc009bd0 con 0x564ec1165840 2023-07-18T03:40:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4d1f6e700 1 --2- 172.21.15.40:0/88290019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec11666e0 0x564ec1174a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4d0f6c700 1 --2- 172.21.15.40:0/88290019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec1165840 0x564ec1171a70 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009ef0 tx=0x7ff4bc01dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63486dacc34789 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:40:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4b4ff9700 1 -- 172.21.15.40:0/88290019 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc029070 con 0x564ec1165840 2023-07-18T03:40:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4b4ff9700 1 -- 172.21.15.40:0/88290019 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff4bc015420 con 0x564ec1165840 2023-07-18T03:40:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/88290019 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4c409f140 con 0x564ec1165840 2023-07-18T03:40:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4b4ff9700 1 -- 172.21.15.40:0/88290019 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4bc0155e0 con 0x564ec1165840 2023-07-18T03:40:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4dfb57b80 10 client.16824 fetch_fsmap learned FSMap version 68 2023-07-18T03:40:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.143+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/88290019 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x564ec0f0b1f0 con 0x564ec1165840 2023-07-18T03:40:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.144+0000 7ff4b4ff9700 1 -- 172.21.15.40:0/88290019 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4bc033020 con 0x564ec1165840 2023-07-18T03:40:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.144+0000 7ff4b4ff9700 1 -- 172.21.15.40:0/88290019 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 68) v1 ==== 7538+0+0 (secure 0 0 0) 0x7ff4bc02c030 con 0x564ec1165840 2023-07-18T03:40:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.144+0000 7ff4dfb57b80 10 client.16824 fetch_fsmap finished waiting for FSMap version 68 2023-07-18T03:40:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.144+0000 7ff4dfb57b80 10 client.16824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:40:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.144+0000 7ff4dfb57b80 20 client.16824 populate_metadata read hostname 'smithi040' 2023-07-18T03:40:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.144+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/88290019 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564ec11ce9f0 0x564ec11d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.144+0000 7ff4dfb57b80 4 client.16824 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:40:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.144+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/88290019 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564ec1071ff0 con 0x564ec11ce9f0 2023-07-18T03:40:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.144+0000 7ff4d176d700 1 --2- 172.21.15.40:0/88290019 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564ec11ce9f0 0x564ec11d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.144+0000 7ff4b4ff9700 10 client.16824 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.144+0000 7ff4d176d700 1 --2- 172.21.15.40:0/88290019 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564ec11ce9f0 0x564ec11d0e00 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=706d99ee1575045d server_cookie=1548a4a553409fd3 in_seq=0 out_seq=0 2023-07-18T03:40:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.158+0000 7ff4b4ff9700 1 -- 172.21.15.40:0/88290019 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564ec1071ff0 con 0x564ec11ce9f0 2023-07-18T03:40:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.158+0000 7ff4b4ff9700 10 client.16824 handle_command_reply: tid=0 2023-07-18T03:40:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.158+0000 7ff4dfb57b80 1 client.16824 shutdown 2023-07-18T03:40:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.158+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/88290019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec1165840 msgr2=0x564ec1171a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.158+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/88290019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec1165840 0x564ec1171a70 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009ef0 tx=0x7ff4bc01dad0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.158+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/88290019 shutdown_connections 2023-07-18T03:40:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.158+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/88290019 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x564ec11ce9f0 0x564ec11d0e00 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.158+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/88290019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ec11666e0 0x564ec1174a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.158+0000 7ff4b4ff9700 0 client.16824 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.158+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/88290019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ec0fc16b0 0x564ec1171530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.158+0000 7ff4dfb57b80 1 --2- 172.21.15.40:0/88290019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x564ec1165840 0x564ec1171a70 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.158+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/88290019 >> 172.21.15.40:0/88290019 conn(0x564ec115abe0 msgr2=0x564ec1164030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.158+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/88290019 shutdown_connections 2023-07-18T03:40:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.158+0000 7ff4dfb57b80 1 -- 172.21.15.40:0/88290019 wait complete. 2023-07-18T03:40:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.158+0000 7ff4dfb57b80 20 client.16824 trim_cache size 0 max 16384 2023-07-18T03:40:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:40:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:40:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:40:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:40:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.159+0000 7ff4d39d1700 1 -- 172.21.15.40:0/3632669251 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4b80a12a0 msgr2=0x7ff4b80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.159+0000 7ff4d39d1700 1 --2- 172.21.15.40:0/3632669251 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4b80a12a0 0x7ff4b80a3750 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800b240 tx=0x7ff4c800b4a0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.159+0000 7ff4d39d1700 1 -- 172.21.15.40:0/3632669251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4cc100640 msgr2=0x7ff4cc0703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.159+0000 7ff4d39d1700 1 --2- 172.21.15.40:0/3632669251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4cc100640 0x7ff4cc0703a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009ec0 tx=0x7ff4c4007f30 comp rx=0 tx=0).stop 2023-07-18T03:40:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.159+0000 7ff4d39d1700 1 -- 172.21.15.40:0/3632669251 shutdown_connections 2023-07-18T03:40:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.159+0000 7ff4d39d1700 1 --2- 172.21.15.40:0/3632669251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4cc100640 0x7ff4cc0703a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.159+0000 7ff4d39d1700 1 --2- 172.21.15.40:0/3632669251 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4b80a12a0 0x7ff4b80a3750 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.159+0000 7ff4d39d1700 1 --2- 172.21.15.40:0/3632669251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4cc0fea00 0x7ff4cc06ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.159+0000 7ff4d39d1700 1 --2- 172.21.15.40:0/3632669251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4cc0ffc90 0x7ff4cc06f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.159+0000 7ff4d39d1700 1 -- 172.21.15.40:0/3632669251 >> 172.21.15.40:0/3632669251 conn(0x7ff4cc0fa000 msgr2=0x7ff4cc0fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.159+0000 7ff4d39d1700 1 -- 172.21.15.40:0/3632669251 shutdown_connections 2023-07-18T03:40:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:44.159+0000 7ff4d39d1700 1 -- 172.21.15.40:0/3632669251 wait complete. 2023-07-18T03:40:45.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:45 smithi134 ceph-mon[98216]: pgmap v590: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 250 op/s 2023-07-18T03:40:45.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:45 smithi040 ceph-mon[96439]: pgmap v590: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 250 op/s 2023-07-18T03:40:45.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:45 smithi064 ceph-mon[98059]: pgmap v590: 97 pgs: 97 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 250 op/s 2023-07-18T03:40:47.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:47 smithi040 ceph-mon[96439]: pgmap v591: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 296 op/s 2023-07-18T03:40:47.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:47 smithi064 ceph-mon[98059]: pgmap v591: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 296 op/s 2023-07-18T03:40:47.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:47 smithi134 ceph-mon[98216]: pgmap v591: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 296 op/s 2023-07-18T03:40:49.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:49 smithi040 ceph-mon[96439]: pgmap v592: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 365 op/s 2023-07-18T03:40:49.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:49 smithi064 ceph-mon[98059]: pgmap v592: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 365 op/s 2023-07-18T03:40:49.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:49 smithi134 ceph-mon[98216]: pgmap v592: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 365 op/s 2023-07-18T03:40:51.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:51 smithi040 ceph-mon[96439]: pgmap v593: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.0 MiB/s wr, 304 op/s 2023-07-18T03:40:51.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:51 smithi064 ceph-mon[98059]: pgmap v593: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.0 MiB/s wr, 304 op/s 2023-07-18T03:40:51.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:51 smithi134 ceph-mon[98216]: pgmap v593: 97 pgs: 97 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.0 MiB/s wr, 304 op/s 2023-07-18T03:40:53.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:52 smithi134 ceph-mon[98216]: pgmap v594: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 458 op/s 2023-07-18T03:40:53.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:52 smithi040 ceph-mon[96439]: pgmap v594: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 458 op/s 2023-07-18T03:40:53.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:52 smithi064 ceph-mon[98059]: pgmap v594: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 458 op/s 2023-07-18T03:40:53.701 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:40:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.838+0000 7fa5cacd1700 1 -- 172.21.15.40:0/726029283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5c40fe9e0 msgr2=0x7fa5c40fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.838+0000 7fa5cacd1700 1 --2- 172.21.15.40:0/726029283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5c40fe9e0 0x7fa5c40fedf0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009990 tx=0x7fa5b8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.838+0000 7fa5cacd1700 1 -- 172.21.15.40:0/726029283 shutdown_connections 2023-07-18T03:40:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.838+0000 7fa5cacd1700 1 --2- 172.21.15.40:0/726029283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5c4100560 0x7fa5c4106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.839+0000 7fa5cacd1700 1 --2- 172.21.15.40:0/726029283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5c40fe9e0 0x7fa5c40fedf0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.839+0000 7fa5cacd1700 1 --2- 172.21.15.40:0/726029283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5c40ffbd0 0x7fa5c4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.839+0000 7fa5cacd1700 1 -- 172.21.15.40:0/726029283 >> 172.21.15.40:0/726029283 conn(0x7fa5c40fa000 msgr2=0x7fa5c40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:53.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.839+0000 7fa5cacd1700 1 -- 172.21.15.40:0/726029283 shutdown_connections 2023-07-18T03:40:53.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.839+0000 7fa5cacd1700 1 -- 172.21.15.40:0/726029283 wait complete. 2023-07-18T03:40:53.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.839+0000 7fa5cacd1700 1 Processor -- start 2023-07-18T03:40:53.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.839+0000 7fa5cacd1700 1 -- start start 2023-07-18T03:40:53.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.839+0000 7fa5cacd1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5c40fe9e0 0x7fa5c41958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.839+0000 7fa5cacd1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5c40ffbd0 0x7fa5c4195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.839+0000 7fa5cacd1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5c4100560 0x7fa5c419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.839+0000 7fa5cacd1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c419d470 con 0x7fa5c40ffbd0 2023-07-18T03:40:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.839+0000 7fa5cacd1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c419d5b0 con 0x7fa5c40fe9e0 2023-07-18T03:40:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.839+0000 7fa5cacd1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c419d740 con 0x7fa5c4100560 2023-07-18T03:40:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.839+0000 7fa5c3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5c40ffbd0 0x7fa5c4195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.839+0000 7fa5c3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5c40ffbd0 0x7fa5c4195e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33190/0 (socket says 172.21.15.40:33190) 2023-07-18T03:40:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.839+0000 7fa5c3fff700 1 -- 172.21.15.40:0/1703026091 learned_addr learned my addr 172.21.15.40:0/1703026091 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.840+0000 7fa5c926e700 1 --2- 172.21.15.40:0/1703026091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5c4100560 0x7fa5c419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.840+0000 7fa5c8a6d700 1 --2- 172.21.15.40:0/1703026091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5c40fe9e0 0x7fa5c41958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.840+0000 7fa5c3fff700 1 -- 172.21.15.40:0/1703026091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5c4100560 msgr2=0x7fa5c419cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.840+0000 7fa5c3fff700 1 --2- 172.21.15.40:0/1703026091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5c4100560 0x7fa5c419cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.840+0000 7fa5c3fff700 1 -- 172.21.15.40:0/1703026091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5c40fe9e0 msgr2=0x7fa5c41958f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.840+0000 7fa5c3fff700 1 --2- 172.21.15.40:0/1703026091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5c40fe9e0 0x7fa5c41958f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.840+0000 7fa5c3fff700 1 -- 172.21.15.40:0/1703026091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b80095f0 con 0x7fa5c40ffbd0 2023-07-18T03:40:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.840+0000 7fa5c3fff700 1 --2- 172.21.15.40:0/1703026091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5c40ffbd0 0x7fa5c4195e30 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000e9a0 tx=0x7fa5b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.840+0000 7fa5c1ffb700 1 -- 172.21.15.40:0/1703026091 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0009790 con 0x7fa5c40ffbd0 2023-07-18T03:40:53.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.840+0000 7fa5c1ffb700 1 -- 172.21.15.40:0/1703026091 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5b000cbd0 con 0x7fa5c40ffbd0 2023-07-18T03:40:53.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.840+0000 7fa5c1ffb700 1 -- 172.21.15.40:0/1703026091 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0018680 con 0x7fa5c40ffbd0 2023-07-18T03:40:53.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.840+0000 7fa5cacd1700 1 -- 172.21.15.40:0/1703026091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c419da20 con 0x7fa5c40ffbd0 2023-07-18T03:40:53.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.840+0000 7fa5cacd1700 1 -- 172.21.15.40:0/1703026091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c419dee0 con 0x7fa5c40ffbd0 2023-07-18T03:40:53.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.843+0000 7fa5c1ffb700 1 -- 172.21.15.40:0/1703026091 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa5b000cd40 con 0x7fa5c40ffbd0 2023-07-18T03:40:53.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.843+0000 7fa5c1ffb700 1 --2- 172.21.15.40:0/1703026091 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa5b40a1150 0x7fa5b40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.843+0000 7fa5c1ffb700 1 -- 172.21.15.40:0/1703026091 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa5b0014070 con 0x7fa5c40ffbd0 2023-07-18T03:40:53.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.843+0000 7fa5c8a6d700 1 --2- 172.21.15.40:0/1703026091 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa5b40a1150 0x7fa5b40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.844+0000 7fa5c8a6d700 1 --2- 172.21.15.40:0/1703026091 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa5b40a1150 0x7fa5b40a3600 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800b2e0 tx=0x7fa5b8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:53.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.844+0000 7fa5d6e57b80 1 Processor -- start 2023-07-18T03:40:53.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.844+0000 7fa5d6e57b80 1 -- start start 2023-07-18T03:40:53.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.844+0000 7fa5d6e57b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c04d0 0x556c4d9c08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.844+0000 7fa5d6e57b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c0e20 0x556c4d9c16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.844+0000 7fa5d6e57b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c1c70 0x556c4d9c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.844+0000 7fa5d6e57b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4d8c5cb0 con 0x556c4d9c1c70 2023-07-18T03:40:53.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.844+0000 7fa5d6e57b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4d918a40 con 0x556c4d9c04d0 2023-07-18T03:40:53.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.844+0000 7fa5d6e57b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4d911ed0 con 0x556c4d9c0e20 2023-07-18T03:40:53.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.844+0000 7fa5c926e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c1c70 0x556c4d9c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.844+0000 7fa5c926e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c1c70 0x556c4d9c20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33200/0 (socket says 172.21.15.40:33200) 2023-07-18T03:40:53.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.844+0000 7fa5c3fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c0e20 0x556c4d9c16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.844+0000 7fa5c926e700 1 -- 172.21.15.40:0/1251670239 learned_addr learned my addr 172.21.15.40:0/1251670239 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:53.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.845+0000 7fa5c8a6d700 1 --2- 172.21.15.40:0/1251670239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c04d0 0x556c4d9c08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.845+0000 7fa5c926e700 1 -- 172.21.15.40:0/1251670239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c0e20 msgr2=0x556c4d9c16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:53.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.845+0000 7fa5c926e700 1 --2- 172.21.15.40:0/1251670239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c0e20 0x556c4d9c16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.845+0000 7fa5c926e700 1 -- 172.21.15.40:0/1251670239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c04d0 msgr2=0x556c4d9c08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:53.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.845+0000 7fa5c926e700 1 --2- 172.21.15.40:0/1251670239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c04d0 0x556c4d9c08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.845+0000 7fa5c926e700 1 -- 172.21.15.40:0/1251670239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c4d73fde0 con 0x556c4d9c1c70 2023-07-18T03:40:53.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.845+0000 7fa5c8a6d700 1 --2- 172.21.15.40:0/1251670239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c04d0 0x556c4d9c08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:40:53.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.845+0000 7fa5c3fff700 1 --2- 172.21.15.40:0/1251670239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c0e20 0x556c4d9c16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:53.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.845+0000 7fa5c926e700 1 --2- 172.21.15.40:0/1251670239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c1c70 0x556c4d9c20e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc011680 tx=0x7fa5bc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320d5d41ce9671b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:53.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.845+0000 7fa5aeffd700 1 -- 172.21.15.40:0/1251670239 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00e040 con 0x556c4d9c1c70 2023-07-18T03:40:53.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.845+0000 7fa5aeffd700 1 -- 172.21.15.40:0/1251670239 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5bc00f040 con 0x556c4d9c1c70 2023-07-18T03:40:53.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.845+0000 7fa5aeffd700 1 -- 172.21.15.40:0/1251670239 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc004860 con 0x556c4d9c1c70 2023-07-18T03:40:53.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.846+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/1251670239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c1c70 msgr2=0x556c4d9c20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.846+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/1251670239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c1c70 0x556c4d9c20e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc011680 tx=0x7fa5bc009920 comp rx=0 tx=0).stop 2023-07-18T03:40:53.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.846+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/1251670239 shutdown_connections 2023-07-18T03:40:53.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.846+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/1251670239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c1c70 0x556c4d9c20e0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.846+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/1251670239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c04d0 0x556c4d9c08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.846+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/1251670239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c0e20 0x556c4d9c16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.846+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/1251670239 >> 172.21.15.40:0/1251670239 conn(0x556c4d9b5da0 msgr2=0x556c4d9bb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:53.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.846+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/1251670239 shutdown_connections 2023-07-18T03:40:53.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.846+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/1251670239 wait complete. 2023-07-18T03:40:53.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.846+0000 7fa5d6e57b80 1 -- start start 2023-07-18T03:40:53.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.846+0000 7fa5d6e57b80 1 Processor -- start 2023-07-18T03:40:53.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5d6e57b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c04d0 0x556c4d9ca150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5d6e57b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c0e20 0x556c4d9ca690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5d6e57b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c1c70 0x556c4d9d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5d6e57b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4d8b1ac0 con 0x556c4d9c04d0 2023-07-18T03:40:53.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5d6e57b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4d8b17c0 con 0x556c4d9c1c70 2023-07-18T03:40:53.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5d6e57b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4d795920 con 0x556c4d9c0e20 2023-07-18T03:40:53.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5d6e57b80 10 client.0 _ll_register_callbacks cb 0x556c4d8af760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:53.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5c8a6d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c04d0 0x556c4d9ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5c8a6d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c04d0 0x556c4d9ca150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33202/0 (socket says 172.21.15.40:33202) 2023-07-18T03:40:53.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5c8a6d700 1 -- 172.21.15.40:0/3026382391 learned_addr learned my addr 172.21.15.40:0/3026382391 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:53.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5c3fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c0e20 0x556c4d9ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5c8a6d700 1 -- 172.21.15.40:0/3026382391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c0e20 msgr2=0x556c4d9ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:53.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5c8a6d700 1 --2- 172.21.15.40:0/3026382391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c0e20 0x556c4d9ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5c8a6d700 1 -- 172.21.15.40:0/3026382391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c1c70 msgr2=0x556c4d9d16d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-18T03:40:53.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5c8a6d700 1 --2- 172.21.15.40:0/3026382391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c1c70 0x556c4d9d16d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5c8a6d700 1 -- 172.21.15.40:0/3026382391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c419dee0 con 0x556c4d9c04d0 2023-07-18T03:40:53.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5c8a6d700 1 --2- 172.21.15.40:0/3026382391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c04d0 0x556c4d9ca150 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40ffa30 tx=0x7fa5b8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cffacea3772b70 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:53.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:40:53.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5a3fff700 1 -- 172.21.15.40:0/3026382391 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8004a90 con 0x556c4d9c04d0 2023-07-18T03:40:53.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5a3fff700 1 -- 172.21.15.40:0/3026382391 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa5b8004bf0 con 0x556c4d9c04d0 2023-07-18T03:40:53.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.847+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/3026382391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c4d86b4f0 con 0x556c4d9c04d0 2023-07-18T03:40:53.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.848+0000 7fa5a3fff700 1 -- 172.21.15.40:0/3026382391 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5b8004db0 con 0x556c4d9c04d0 2023-07-18T03:40:53.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.848+0000 7fa5d6e57b80 10 client.16836 fetch_fsmap learned FSMap version 68 2023-07-18T03:40:53.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.848+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/3026382391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7fa5bc011330 con 0x556c4d9c04d0 2023-07-18T03:40:53.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.848+0000 7fa5a3fff700 1 -- 172.21.15.40:0/3026382391 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5b803e080 con 0x556c4d9c04d0 2023-07-18T03:40:53.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.848+0000 7fa5a3fff700 1 -- 172.21.15.40:0/3026382391 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 68) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fa5b8038030 con 0x556c4d9c04d0 2023-07-18T03:40:53.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.848+0000 7fa5d6e57b80 10 client.16836 fetch_fsmap finished waiting for FSMap version 68 2023-07-18T03:40:53.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.848+0000 7fa5d6e57b80 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:40:53.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.848+0000 7fa5d6e57b80 20 client.16836 populate_metadata read hostname 'smithi040' 2023-07-18T03:40:53.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.848+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/3026382391 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556c4d9d5ef0 0x556c4d9d8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.848+0000 7fa5d6e57b80 4 client.16836 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:40:53.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.848+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/3026382391 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c4d90aeb0 con 0x556c4d9d5ef0 2023-07-18T03:40:53.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.848+0000 7fa5c3fff700 1 --2- 172.21.15.40:0/3026382391 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556c4d9d5ef0 0x556c4d9d8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.849+0000 7fa5a3fff700 10 client.16836 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:53.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.849+0000 7fa5c3fff700 1 --2- 172.21.15.40:0/3026382391 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556c4d9d5ef0 0x556c4d9d8300 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=171c7ea2c30a5d7a server_cookie=1bf21ecaf9bf25ff in_seq=0 out_seq=0 2023-07-18T03:40:53.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.849+0000 7fa5a3fff700 1 -- 172.21.15.40:0/3026382391 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556c4d90aeb0 con 0x556c4d9d5ef0 2023-07-18T03:40:53.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.850+0000 7fa5a3fff700 10 client.16836 handle_command_reply: tid=0 2023-07-18T03:40:53.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.850+0000 7fa5d6e57b80 1 client.16836 shutdown 2023-07-18T03:40:53.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.850+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/3026382391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c04d0 msgr2=0x556c4d9ca150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.850+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/3026382391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c04d0 0x556c4d9ca150 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40ffa30 tx=0x7fa5b8009ca0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.850+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/3026382391 shutdown_connections 2023-07-18T03:40:53.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.850+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/3026382391 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556c4d9d5ef0 0x556c4d9d8300 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.850+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/3026382391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c1c70 0x556c4d9d16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.850+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/3026382391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c04d0 0x556c4d9ca150 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.850+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/3026382391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c0e20 0x556c4d9ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.850+0000 7fa5a3fff700 0 client.16836 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:53.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.850+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/3026382391 >> 172.21.15.40:0/3026382391 conn(0x556c4d9b5da0 msgr2=0x556c4d9ba8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:53.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.850+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/3026382391 shutdown_connections 2023-07-18T03:40:53.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.850+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/3026382391 wait complete. 2023-07-18T03:40:53.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.850+0000 7fa5d6e57b80 20 client.16836 trim_cache size 0 max 16384 2023-07-18T03:40:53.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.867+0000 7fa5d6e57b80 1 Processor -- start 2023-07-18T03:40:53.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.867+0000 7fa5d6e57b80 1 -- start start 2023-07-18T03:40:53.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.867+0000 7fa5d6e57b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c04d0 0x556c4d9d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.867+0000 7fa5d6e57b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c0e20 0x556c4d9d1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5d6e57b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c1c70 0x556c4d9c8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5d6e57b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4d9d5c40 con 0x556c4d9c04d0 2023-07-18T03:40:53.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5d6e57b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4d9c8b80 con 0x556c4d9c0e20 2023-07-18T03:40:53.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5d6e57b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4d9bc270 con 0x556c4d9c1c70 2023-07-18T03:40:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5c8a6d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c04d0 0x556c4d9d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5c8a6d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c04d0 0x556c4d9d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33208/0 (socket says 172.21.15.40:33208) 2023-07-18T03:40:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5c8a6d700 1 -- 172.21.15.40:0/650365851 learned_addr learned my addr 172.21.15.40:0/650365851 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5c3fff700 1 --2- 172.21.15.40:0/650365851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c0e20 0x556c4d9d1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5c926e700 1 --2- 172.21.15.40:0/650365851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c1c70 0x556c4d9c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5c8a6d700 1 -- 172.21.15.40:0/650365851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c1c70 msgr2=0x556c4d9c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:53.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5c8a6d700 1 --2- 172.21.15.40:0/650365851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c1c70 0x556c4d9c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5c8a6d700 1 -- 172.21.15.40:0/650365851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c0e20 msgr2=0x556c4d9d1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:53.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5c8a6d700 1 --2- 172.21.15.40:0/650365851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c0e20 0x556c4d9d1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5c8a6d700 1 -- 172.21.15.40:0/650365851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c4d728c60 con 0x556c4d9c04d0 2023-07-18T03:40:53.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5c8a6d700 1 --2- 172.21.15.40:0/650365851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c04d0 0x556c4d9d1fc0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa5b802f800 tx=0x7fa5b800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7b59779873685b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:53.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5af7fe700 1 -- 172.21.15.40:0/650365851 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b802fd10 con 0x556c4d9c04d0 2023-07-18T03:40:53.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5af7fe700 1 -- 172.21.15.40:0/650365851 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5b8028030 con 0x556c4d9c04d0 2023-07-18T03:40:53.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.868+0000 7fa5af7fe700 1 -- 172.21.15.40:0/650365851 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b800f7e0 con 0x556c4d9c04d0 2023-07-18T03:40:53.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.869+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/650365851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c04d0 msgr2=0x556c4d9d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.869+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/650365851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c04d0 0x556c4d9d1fc0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa5b802f800 tx=0x7fa5b800bfd0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.869+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/650365851 shutdown_connections 2023-07-18T03:40:53.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.869+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/650365851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c1c70 0x556c4d9c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.869+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/650365851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c04d0 0x556c4d9d1fc0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.869+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/650365851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c0e20 0x556c4d9d1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.869+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/650365851 >> 172.21.15.40:0/650365851 conn(0x556c4d9b5da0 msgr2=0x556c4d9dcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:53.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.869+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/650365851 shutdown_connections 2023-07-18T03:40:53.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.869+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/650365851 wait complete. 2023-07-18T03:40:53.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.869+0000 7fa5d6e57b80 1 -- start start 2023-07-18T03:40:53.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.869+0000 7fa5d6e57b80 1 Processor -- start 2023-07-18T03:40:53.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5d6e57b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c04d0 0x556c4d9cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5d6e57b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c0e20 0x556c4d9cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5d6e57b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c1c70 0x556c4d9d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5d6e57b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4d9d0ff0 con 0x556c4d9c0e20 2023-07-18T03:40:53.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5d6e57b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4d9d1160 con 0x556c4d9c1c70 2023-07-18T03:40:53.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5d6e57b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c4d9d1320 con 0x556c4d9c04d0 2023-07-18T03:40:53.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5d6e57b80 10 client.0 _ll_register_callbacks cb 0x7fa5b800bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:53.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5c3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c0e20 0x556c4d9cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5c3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c0e20 0x556c4d9cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33216/0 (socket says 172.21.15.40:33216) 2023-07-18T03:40:53.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5c3fff700 1 -- 172.21.15.40:0/1216834644 learned_addr learned my addr 172.21.15.40:0/1216834644 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:53.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5c8a6d700 1 --2- 172.21.15.40:0/1216834644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c04d0 0x556c4d9cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5c926e700 1 --2- 172.21.15.40:0/1216834644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c1c70 0x556c4d9d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5c3fff700 1 -- 172.21.15.40:0/1216834644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c04d0 msgr2=0x556c4d9cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:53.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5c3fff700 1 --2- 172.21.15.40:0/1216834644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c04d0 0x556c4d9cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5c3fff700 1 -- 172.21.15.40:0/1216834644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c1c70 msgr2=0x556c4d9d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:53.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5c3fff700 1 --2- 172.21.15.40:0/1216834644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c1c70 0x556c4d9d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5c3fff700 1 -- 172.21.15.40:0/1216834644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5bc011330 con 0x556c4d9c0e20 2023-07-18T03:40:53.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5c3fff700 1 --2- 172.21.15.40:0/1216834644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c0e20 0x556c4d9cde00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0099f50 tx=0x7fa5b000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8641fcde95f42e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:53.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:40:53.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5a3fff700 1 -- 172.21.15.40:0/1216834644 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b00994b0 con 0x556c4d9c0e20 2023-07-18T03:40:53.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5a3fff700 1 -- 172.21.15.40:0/1216834644 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa5b0099610 con 0x556c4d9c0e20 2023-07-18T03:40:53.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.870+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/1216834644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5bc00f040 con 0x556c4d9c0e20 2023-07-18T03:40:53.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.871+0000 7fa5a3fff700 1 -- 172.21.15.40:0/1216834644 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5b0094bd0 con 0x556c4d9c0e20 2023-07-18T03:40:53.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.871+0000 7fa5d6e57b80 10 client.16842 fetch_fsmap learned FSMap version 68 2023-07-18T03:40:53.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.871+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/1216834644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x556c4d766250 con 0x556c4d9c0e20 2023-07-18T03:40:53.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.871+0000 7fa5a3fff700 1 -- 172.21.15.40:0/1216834644 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5bc00f040 con 0x556c4d9c0e20 2023-07-18T03:40:53.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.871+0000 7fa5a3fff700 1 -- 172.21.15.40:0/1216834644 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 68) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fa5b00a20a0 con 0x556c4d9c0e20 2023-07-18T03:40:53.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.871+0000 7fa5d6e57b80 10 client.16842 fetch_fsmap finished waiting for FSMap version 68 2023-07-18T03:40:53.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.871+0000 7fa5d6e57b80 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:40:53.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.871+0000 7fa5d6e57b80 20 client.16842 populate_metadata read hostname 'smithi040' 2023-07-18T03:40:53.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.871+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/1216834644 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556c4da2af00 0x556c4da2d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:53.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.871+0000 7fa5d6e57b80 4 client.16842 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:40:53.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.871+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/1216834644 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556c4d8cd050 con 0x556c4da2af00 2023-07-18T03:40:53.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.871+0000 7fa5c8a6d700 1 --2- 172.21.15.40:0/1216834644 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556c4da2af00 0x556c4da2d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:53.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.872+0000 7fa5a3fff700 10 client.16842 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:53.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.872+0000 7fa5c8a6d700 1 --2- 172.21.15.40:0/1216834644 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556c4da2af00 0x556c4da2d310 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dec0cd010ca67bea server_cookie=85830578a3ae92d3 in_seq=0 out_seq=0 2023-07-18T03:40:53.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.872+0000 7fa5a3fff700 1 -- 172.21.15.40:0/1216834644 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556c4d8cd050 con 0x556c4da2af00 2023-07-18T03:40:53.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.872+0000 7fa5a3fff700 10 client.16842 handle_command_reply: tid=0 2023-07-18T03:40:53.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.872+0000 7fa5d6e57b80 1 client.16842 shutdown 2023-07-18T03:40:53.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.872+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/1216834644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c0e20 msgr2=0x556c4d9cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.872+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/1216834644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c0e20 0x556c4d9cde00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0099f50 tx=0x7fa5b000eef0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.872+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/1216834644 shutdown_connections 2023-07-18T03:40:53.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.872+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/1216834644 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556c4da2af00 0x556c4da2d310 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.872+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/1216834644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c4d9c1c70 0x556c4d9d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.872+0000 7fa5a3fff700 0 client.16842 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:53.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.872+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/1216834644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c4d9c04d0 0x556c4d9cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.872+0000 7fa5d6e57b80 1 --2- 172.21.15.40:0/1216834644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556c4d9c0e20 0x556c4d9cde00 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.872+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/1216834644 >> 172.21.15.40:0/1216834644 conn(0x556c4d9b5da0 msgr2=0x556c4d9d1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:53.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.872+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/1216834644 shutdown_connections 2023-07-18T03:40:53.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.872+0000 7fa5d6e57b80 1 -- 172.21.15.40:0/1216834644 wait complete. 2023-07-18T03:40:53.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.872+0000 7fa5d6e57b80 20 client.16842 trim_cache size 0 max 16384 2023-07-18T03:40:53.893 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:40:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:40:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:40:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:40:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:40:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:40:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:40:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:40:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:40:53.895 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:40:53.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.873+0000 7fa5cacd1700 1 -- 172.21.15.40:0/1703026091 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa5b40a1150 msgr2=0x7fa5b40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.873+0000 7fa5cacd1700 1 --2- 172.21.15.40:0/1703026091 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa5b40a1150 0x7fa5b40a3600 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800b2e0 tx=0x7fa5b8005fb0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.873+0000 7fa5cacd1700 1 -- 172.21.15.40:0/1703026091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5c40ffbd0 msgr2=0x7fa5c4195e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:53.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.873+0000 7fa5cacd1700 1 --2- 172.21.15.40:0/1703026091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5c40ffbd0 0x7fa5c4195e30 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000e9a0 tx=0x7fa5b000ecb0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.873+0000 7fa5cacd1700 1 -- 172.21.15.40:0/1703026091 shutdown_connections 2023-07-18T03:40:53.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.873+0000 7fa5cacd1700 1 --2- 172.21.15.40:0/1703026091 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa5b40a1150 0x7fa5b40a3600 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.873+0000 7fa5cacd1700 1 --2- 172.21.15.40:0/1703026091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5c4100560 0x7fa5c419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.873+0000 7fa5cacd1700 1 --2- 172.21.15.40:0/1703026091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5c40fe9e0 0x7fa5c41958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.873+0000 7fa5cacd1700 1 --2- 172.21.15.40:0/1703026091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5c40ffbd0 0x7fa5c4195e30 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:53.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.873+0000 7fa5cacd1700 1 -- 172.21.15.40:0/1703026091 >> 172.21.15.40:0/1703026091 conn(0x7fa5c40fa000 msgr2=0x7fa5c40fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:53.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.873+0000 7fa5cacd1700 1 -- 172.21.15.40:0/1703026091 shutdown_connections 2023-07-18T03:40:53.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:40:53.873+0000 7fa5cacd1700 1 -- 172.21.15.40:0/1703026091 wait complete. 2023-07-18T03:40:53.898 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:40:55.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:55 smithi040 ceph-mon[96439]: pgmap v595: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 406 op/s 2023-07-18T03:40:55.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:55 smithi064 ceph-mon[98059]: pgmap v595: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 406 op/s 2023-07-18T03:40:55.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:55 smithi134 ceph-mon[98216]: pgmap v595: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 406 op/s 2023-07-18T03:40:56.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:56 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:40:56.605 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:56 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:40:56.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:56 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:40:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.271+0000 7f78594c5700 1 -- 172.21.15.40:0/2160732936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7854068640 msgr2=0x7f7854102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.271+0000 7f78594c5700 1 --2- 172.21.15.40:0/2160732936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7854068640 0x7f7854102690 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7848009990 tx=0x7f7848009ca0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.271+0000 7f78594c5700 1 -- 172.21.15.40:0/2160732936 shutdown_connections 2023-07-18T03:40:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.271+0000 7f78594c5700 1 --2- 172.21.15.40:0/2160732936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f78541056e0 0x7f7854107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.271+0000 7f78594c5700 1 --2- 172.21.15.40:0/2160732936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7854068640 0x7f7854102690 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.271+0000 7f78594c5700 1 --2- 172.21.15.40:0/2160732936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7854102bd0 0x7f7854105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.271+0000 7f78594c5700 1 -- 172.21.15.40:0/2160732936 >> 172.21.15.40:0/2160732936 conn(0x7f78540fa000 msgr2=0x7f78540fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.271+0000 7f78594c5700 1 -- 172.21.15.40:0/2160732936 shutdown_connections 2023-07-18T03:40:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.271+0000 7f78594c5700 1 -- 172.21.15.40:0/2160732936 wait complete. 2023-07-18T03:40:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f78594c5700 1 Processor -- start 2023-07-18T03:40:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f78594c5700 1 -- start start 2023-07-18T03:40:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f78594c5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7854068640 0x7f78541959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f78594c5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7854102bd0 0x7f7854195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f78594c5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78541056e0 0x7f7854196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f78594c5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78541973c0 con 0x7f78541056e0 2023-07-18T03:40:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f78594c5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7854197530 con 0x7f7854068640 2023-07-18T03:40:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f78594c5700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78541976f0 con 0x7f7854102bd0 2023-07-18T03:40:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f78527fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7854102bd0 0x7f7854195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f78537fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78541056e0 0x7f7854196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f78527fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7854102bd0 0x7f7854195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50490/0 (socket says 172.21.15.40:50490) 2023-07-18T03:40:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f78527fc700 1 -- 172.21.15.40:0/2087443377 learned_addr learned my addr 172.21.15.40:0/2087443377 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f7852ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7854068640 0x7f78541959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.283 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f7852ffd700 1 -- 172.21.15.40:0/2087443377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7854102bd0 msgr2=0x7f7854195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f7852ffd700 1 --2- 172.21.15.40:0/2087443377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7854102bd0 0x7f7854195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f7852ffd700 1 -- 172.21.15.40:0/2087443377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78541056e0 msgr2=0x7f7854196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f7852ffd700 1 --2- 172.21.15.40:0/2087443377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78541056e0 0x7f7854196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f7852ffd700 1 -- 172.21.15.40:0/2087443377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7844009520 con 0x7f7854068640 2023-07-18T03:40:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f78537fe700 1 --2- 172.21.15.40:0/2087443377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78541056e0 0x7f7854196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.272+0000 7f78527fc700 1 --2- 172.21.15.40:0/2087443377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7854102bd0 0x7f7854195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:40:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.273+0000 7f7852ffd700 1 --2- 172.21.15.40:0/2087443377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7854068640 0x7f78541959a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f78480098e0 tx=0x7f7848009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.273+0000 7f783bfff700 1 -- 172.21.15.40:0/2087443377 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784801d070 con 0x7f7854068640 2023-07-18T03:40:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.273+0000 7f78594c5700 1 -- 172.21.15.40:0/2087443377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78480095f0 con 0x7f7854068640 2023-07-18T03:40:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.273+0000 7f783bfff700 1 -- 172.21.15.40:0/2087443377 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7848021410 con 0x7f7854068640 2023-07-18T03:40:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.273+0000 7f783bfff700 1 -- 172.21.15.40:0/2087443377 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7848005720 con 0x7f7854068640 2023-07-18T03:40:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.273+0000 7f78594c5700 1 -- 172.21.15.40:0/2087443377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785419dfb0 con 0x7f7854068640 2023-07-18T03:40:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.275+0000 7f783bfff700 1 -- 172.21.15.40:0/2087443377 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7848021580 con 0x7f7854068640 2023-07-18T03:40:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.276+0000 7f783bfff700 1 --2- 172.21.15.40:0/2087443377 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f783c0a1150 0x7f783c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.276+0000 7f78527fc700 1 --2- 172.21.15.40:0/2087443377 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f783c0a1150 0x7f783c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.276+0000 7f783bfff700 1 -- 172.21.15.40:0/2087443377 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f78480cdce0 con 0x7f7854068640 2023-07-18T03:40:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.276+0000 7f78527fc700 1 --2- 172.21.15.40:0/2087443377 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f783c0a1150 0x7f783c0a3600 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7844009fd0 tx=0x7f7844009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f786564bb80 1 Processor -- start 2023-07-18T03:40:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f786564bb80 1 -- start start 2023-07-18T03:40:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f786564bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x556960151300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f786564bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 0x5569601521a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f786564bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569601526e0 0x556960152b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f786564bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x556960056c50 con 0x55695ffad6b0 2023-07-18T03:40:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f786564bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569600a99e0 con 0x556960151840 2023-07-18T03:40:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f786564bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569600a2e70 con 0x5569601526e0 2023-07-18T03:40:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f7852ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x556960151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f7852ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x556960151300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33236/0 (socket says 172.21.15.40:33236) 2023-07-18T03:40:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f7852ffd700 1 -- 172.21.15.40:0/1029172513 learned_addr learned my addr 172.21.15.40:0/1029172513 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f78537fe700 1 --2- 172.21.15.40:0/1029172513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569601526e0 0x556960152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f78527fc700 1 --2- 172.21.15.40:0/1029172513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 0x5569601521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f7852ffd700 1 -- 172.21.15.40:0/1029172513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569601526e0 msgr2=0x556960152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f7852ffd700 1 --2- 172.21.15.40:0/1029172513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569601526e0 0x556960152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f7852ffd700 1 -- 172.21.15.40:0/1029172513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 msgr2=0x5569601521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f7852ffd700 1 --2- 172.21.15.40:0/1029172513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 0x5569601521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f7852ffd700 1 -- 172.21.15.40:0/1029172513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55695fed0d80 con 0x55695ffad6b0 2023-07-18T03:40:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.277+0000 7f7852ffd700 1 --2- 172.21.15.40:0/1029172513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x556960151300 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7848097600 tx=0x7f7848009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7840d02f6d463fd3 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.278+0000 7f78397fa700 1 -- 172.21.15.40:0/1029172513 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78480a34c0 con 0x55695ffad6b0 2023-07-18T03:40:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.278+0000 7f78397fa700 1 -- 172.21.15.40:0/1029172513 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78480994a0 con 0x55695ffad6b0 2023-07-18T03:40:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.278+0000 7f78397fa700 1 -- 172.21.15.40:0/1029172513 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78480a3030 con 0x55695ffad6b0 2023-07-18T03:40:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.278+0000 7f786564bb80 1 -- 172.21.15.40:0/1029172513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 msgr2=0x556960151300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.278+0000 7f786564bb80 1 --2- 172.21.15.40:0/1029172513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x556960151300 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7848097600 tx=0x7f7848009310 comp rx=0 tx=0).stop 2023-07-18T03:40:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.278+0000 7f786564bb80 1 -- 172.21.15.40:0/1029172513 shutdown_connections 2023-07-18T03:40:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.278+0000 7f786564bb80 1 --2- 172.21.15.40:0/1029172513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569601526e0 0x556960152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.278+0000 7f786564bb80 1 --2- 172.21.15.40:0/1029172513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x556960151300 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.278+0000 7f786564bb80 1 --2- 172.21.15.40:0/1029172513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 0x5569601521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.278+0000 7f786564bb80 1 -- 172.21.15.40:0/1029172513 >> 172.21.15.40:0/1029172513 conn(0x556960146be0 msgr2=0x55696014bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.278+0000 7f786564bb80 1 -- 172.21.15.40:0/1029172513 shutdown_connections 2023-07-18T03:40:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.278+0000 7f786564bb80 1 -- 172.21.15.40:0/1029172513 wait complete. 2023-07-18T03:40:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f786564bb80 1 -- start start 2023-07-18T03:40:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f786564bb80 1 Processor -- start 2023-07-18T03:40:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f786564bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x55696015a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f786564bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 0x55696015a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f786564bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569601526e0 0x556960161a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f786564bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x556960042a60 con 0x55695ffad6b0 2023-07-18T03:40:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f786564bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x556960042760 con 0x556960151840 2023-07-18T03:40:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f786564bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695ff268c0 con 0x5569601526e0 2023-07-18T03:40:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f786564bb80 10 client.0 _ll_register_callbacks cb 0x556960040700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f7852ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x55696015a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f7852ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x55696015a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33242/0 (socket says 172.21.15.40:33242) 2023-07-18T03:40:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f7852ffd700 1 -- 172.21.15.40:0/1505735042 learned_addr learned my addr 172.21.15.40:0/1505735042 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f78537fe700 1 --2- 172.21.15.40:0/1505735042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569601526e0 0x556960161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f78527fc700 1 --2- 172.21.15.40:0/1505735042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 0x55696015a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f7852ffd700 1 -- 172.21.15.40:0/1505735042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569601526e0 msgr2=0x556960161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f7852ffd700 1 --2- 172.21.15.40:0/1505735042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569601526e0 0x556960161a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f7852ffd700 1 -- 172.21.15.40:0/1505735042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 msgr2=0x55696015a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f7852ffd700 1 --2- 172.21.15.40:0/1505735042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 0x55696015a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.279+0000 7f7852ffd700 1 -- 172.21.15.40:0/1505735042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785419dfb0 con 0x55695ffad6b0 2023-07-18T03:40:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.280+0000 7f7852ffd700 1 --2- 172.21.15.40:0/1505735042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x55696015a480 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f784809ae00 tx=0x7f784809afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176dd92e7be3eca0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.280+0000 7f7836ff5700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:40:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.280+0000 7f7836ff5700 1 -- 172.21.15.40:0/1505735042 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78480a3230 con 0x55695ffad6b0 2023-07-18T03:40:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.280+0000 7f7836ff5700 1 -- 172.21.15.40:0/1505735042 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f78480a34c0 con 0x55695ffad6b0 2023-07-18T03:40:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.280+0000 7f786564bb80 1 -- 172.21.15.40:0/1505735042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55695fffc490 con 0x55695ffad6b0 2023-07-18T03:40:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.280+0000 7f7836ff5700 1 -- 172.21.15.40:0/1505735042 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f784800fb10 con 0x55695ffad6b0 2023-07-18T03:40:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.280+0000 7f786564bb80 10 client.16854 fetch_fsmap learned FSMap version 68 2023-07-18T03:40:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.280+0000 7f786564bb80 1 -- 172.21.15.40:0/1505735042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55695feb9c00 con 0x55695ffad6b0 2023-07-18T03:40:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.280+0000 7f7836ff5700 1 -- 172.21.15.40:0/1505735042 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55695fffc490 con 0x55695ffad6b0 2023-07-18T03:40:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.280+0000 7f7836ff5700 1 -- 172.21.15.40:0/1505735042 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 68) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f78480b90d0 con 0x55695ffad6b0 2023-07-18T03:40:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.280+0000 7f786564bb80 10 client.16854 fetch_fsmap finished waiting for FSMap version 68 2023-07-18T03:40:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.280+0000 7f786564bb80 10 client.16854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:40:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.280+0000 7f786564bb80 20 client.16854 populate_metadata read hostname 'smithi040' 2023-07-18T03:40:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.280+0000 7f786564bb80 1 --2- 172.21.15.40:0/1505735042 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556960165e50 0x556960168260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.280+0000 7f786564bb80 4 client.16854 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:40:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.280+0000 7f786564bb80 1 -- 172.21.15.40:0/1505735042 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55696009be50 con 0x556960165e50 2023-07-18T03:40:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.281+0000 7f78527fc700 1 --2- 172.21.15.40:0/1505735042 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556960165e50 0x556960168260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.281+0000 7f78527fc700 1 --2- 172.21.15.40:0/1505735042 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556960165e50 0x556960168260 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e83046e5523f6d7f server_cookie=444cfd5723e9a3f in_seq=0 out_seq=0 2023-07-18T03:40:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.281+0000 7f7836ff5700 10 client.16854 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.282+0000 7f7836ff5700 1 -- 172.21.15.40:0/1505735042 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55696009be50 con 0x556960165e50 2023-07-18T03:40:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.282+0000 7f7836ff5700 10 client.16854 handle_command_reply: tid=0 2023-07-18T03:40:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.282+0000 7f786564bb80 1 client.16854 shutdown 2023-07-18T03:40:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.282+0000 7f786564bb80 1 -- 172.21.15.40:0/1505735042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 msgr2=0x55696015a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.282+0000 7f786564bb80 1 --2- 172.21.15.40:0/1505735042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x55696015a480 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f784809ae00 tx=0x7f784809afa0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.282+0000 7f786564bb80 1 -- 172.21.15.40:0/1505735042 shutdown_connections 2023-07-18T03:40:57.298 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.282+0000 7f786564bb80 1 --2- 172.21.15.40:0/1505735042 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x556960165e50 0x556960168260 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.282+0000 7f786564bb80 1 --2- 172.21.15.40:0/1505735042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569601526e0 0x556960161a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.282+0000 7f7836ff5700 0 client.16854 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.282+0000 7f786564bb80 1 --2- 172.21.15.40:0/1505735042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x55696015a480 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.282+0000 7f786564bb80 1 --2- 172.21.15.40:0/1505735042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 0x55696015a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.282+0000 7f786564bb80 1 -- 172.21.15.40:0/1505735042 >> 172.21.15.40:0/1505735042 conn(0x556960146be0 msgr2=0x55696014b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.282+0000 7f786564bb80 1 -- 172.21.15.40:0/1505735042 shutdown_connections 2023-07-18T03:40:57.299 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.282+0000 7f786564bb80 1 -- 172.21.15.40:0/1505735042 wait complete. 2023-07-18T03:40:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.282+0000 7f786564bb80 20 client.16854 trim_cache size 0 max 16384 2023-07-18T03:40:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f786564bb80 1 Processor -- start 2023-07-18T03:40:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f786564bb80 1 -- start start 2023-07-18T03:40:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f786564bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x55696015a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f786564bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 0x556960161990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f786564bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569601526e0 0x55696015a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f786564bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569600a99e0 con 0x55695ffad6b0 2023-07-18T03:40:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f786564bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55696009be50 con 0x556960151840 2023-07-18T03:40:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f786564bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695fffc490 con 0x5569601526e0 2023-07-18T03:40:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f7852ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x55696015a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f7852ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x55696015a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33250/0 (socket says 172.21.15.40:33250) 2023-07-18T03:40:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f7852ffd700 1 -- 172.21.15.40:0/2087971258 learned_addr learned my addr 172.21.15.40:0/2087971258 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f78537fe700 1 --2- 172.21.15.40:0/2087971258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569601526e0 0x55696015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f78527fc700 1 --2- 172.21.15.40:0/2087971258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 0x556960161990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f7852ffd700 1 -- 172.21.15.40:0/2087971258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569601526e0 msgr2=0x55696015a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f7852ffd700 1 --2- 172.21.15.40:0/2087971258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569601526e0 0x55696015a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f7852ffd700 1 -- 172.21.15.40:0/2087971258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 msgr2=0x556960161990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f7852ffd700 1 --2- 172.21.15.40:0/2087971258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 0x556960161990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.297+0000 7f7852ffd700 1 -- 172.21.15.40:0/2087971258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55695fef71f0 con 0x55695ffad6b0 2023-07-18T03:40:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.298+0000 7f7852ffd700 1 --2- 172.21.15.40:0/2087971258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x55696015a230 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f7848097b70 tx=0x7f78480092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fefb6b345d0ade server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.298+0000 7f7839ffb700 1 -- 172.21.15.40:0/2087971258 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78480b92d0 con 0x55695ffad6b0 2023-07-18T03:40:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.298+0000 7f7839ffb700 1 -- 172.21.15.40:0/2087971258 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78480a34c0 con 0x55695ffad6b0 2023-07-18T03:40:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.298+0000 7f7839ffb700 1 -- 172.21.15.40:0/2087971258 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7848096ef0 con 0x55695ffad6b0 2023-07-18T03:40:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.298+0000 7f786564bb80 1 -- 172.21.15.40:0/2087971258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 msgr2=0x55696015a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.298+0000 7f786564bb80 1 --2- 172.21.15.40:0/2087971258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x55696015a230 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f7848097b70 tx=0x7f78480092f0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.298+0000 7f786564bb80 1 -- 172.21.15.40:0/2087971258 shutdown_connections 2023-07-18T03:40:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.298+0000 7f786564bb80 1 --2- 172.21.15.40:0/2087971258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5569601526e0 0x55696015a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.298+0000 7f786564bb80 1 --2- 172.21.15.40:0/2087971258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55695ffad6b0 0x55696015a230 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.298+0000 7f786564bb80 1 --2- 172.21.15.40:0/2087971258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 0x556960161990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.298+0000 7f786564bb80 1 -- 172.21.15.40:0/2087971258 >> 172.21.15.40:0/2087971258 conn(0x556960146be0 msgr2=0x55696016cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.298+0000 7f786564bb80 1 -- 172.21.15.40:0/2087971258 shutdown_connections 2023-07-18T03:40:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.298+0000 7f786564bb80 1 -- 172.21.15.40:0/2087971258 wait complete. 2023-07-18T03:40:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f786564bb80 1 -- start start 2023-07-18T03:40:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f786564bb80 1 Processor -- start 2023-07-18T03:40:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f786564bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55695ffad6b0 0x55696015d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f786564bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 0x55696015dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f786564bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569601526e0 0x556960160d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f786564bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x556960161260 con 0x5569601526e0 2023-07-18T03:40:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f786564bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569601613d0 con 0x556960151840 2023-07-18T03:40:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f786564bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x556960161590 con 0x55695ffad6b0 2023-07-18T03:40:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f786564bb80 10 client.0 _ll_register_callbacks cb 0x7f784c009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:40:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f78537fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569601526e0 0x556960160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f78537fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569601526e0 0x556960160d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33260/0 (socket says 172.21.15.40:33260) 2023-07-18T03:40:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f78537fe700 1 -- 172.21.15.40:0/3139471441 learned_addr learned my addr 172.21.15.40:0/3139471441 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:40:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f7852ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55695ffad6b0 0x55696015d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f78527fc700 1 --2- 172.21.15.40:0/3139471441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 0x55696015dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f78537fe700 1 -- 172.21.15.40:0/3139471441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55695ffad6b0 msgr2=0x55696015d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f78537fe700 1 --2- 172.21.15.40:0/3139471441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55695ffad6b0 0x55696015d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f78537fe700 1 -- 172.21.15.40:0/3139471441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 msgr2=0x55696015dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:40:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f78537fe700 1 --2- 172.21.15.40:0/3139471441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 0x55696015dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.299+0000 7f78537fe700 1 -- 172.21.15.40:0/3139471441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55695feb9c00 con 0x5569601526e0 2023-07-18T03:40:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.300+0000 7f78537fe700 1 --2- 172.21.15.40:0/3139471441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569601526e0 0x556960160d20 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f784c00ef00 tx=0x7f784c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632fe3462d263238 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:40:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.300+0000 7f7836ff5700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:40:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.300+0000 7f7836ff5700 1 -- 172.21.15.40:0/3139471441 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c01c070 con 0x5569601526e0 2023-07-18T03:40:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.300+0000 7f7836ff5700 1 -- 172.21.15.40:0/3139471441 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f784c020410 con 0x5569601526e0 2023-07-18T03:40:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.300+0000 7f786564bb80 1 -- 172.21.15.40:0/3139471441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78480994a0 con 0x5569601526e0 2023-07-18T03:40:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.300+0000 7f7836ff5700 1 -- 172.21.15.40:0/3139471441 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f784c0205d0 con 0x5569601526e0 2023-07-18T03:40:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.300+0000 7f786564bb80 10 client.16860 fetch_fsmap learned FSMap version 68 2023-07-18T03:40:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.300+0000 7f786564bb80 1 -- 172.21.15.40:0/3139471441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x55695fecf250 con 0x5569601526e0 2023-07-18T03:40:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.300+0000 7f7836ff5700 1 -- 172.21.15.40:0/3139471441 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78480994a0 con 0x5569601526e0 2023-07-18T03:40:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.300+0000 7f7836ff5700 1 -- 172.21.15.40:0/3139471441 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 68) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f784c016840 con 0x5569601526e0 2023-07-18T03:40:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.300+0000 7f786564bb80 10 client.16860 fetch_fsmap finished waiting for FSMap version 68 2023-07-18T03:40:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.300+0000 7f786564bb80 10 client.16860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:40:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.300+0000 7f786564bb80 20 client.16860 populate_metadata read hostname 'smithi040' 2023-07-18T03:40:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.300+0000 7f786564bb80 1 --2- 172.21.15.40:0/3139471441 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5569601bad80 0x5569601bd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:40:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.300+0000 7f786564bb80 4 client.16860 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:40:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.300+0000 7f786564bb80 1 -- 172.21.15.40:0/3139471441 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55696005dff0 con 0x5569601bad80 2023-07-18T03:40:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.301+0000 7f7852ffd700 1 --2- 172.21.15.40:0/3139471441 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5569601bad80 0x5569601bd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:40:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.301+0000 7f7836ff5700 10 client.16860 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.301+0000 7f7852ffd700 1 --2- 172.21.15.40:0/3139471441 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5569601bad80 0x5569601bd190 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d2c64bea66548fa server_cookie=bdbfbb867664d834 in_seq=0 out_seq=0 2023-07-18T03:40:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.308+0000 7f7836ff5700 1 -- 172.21.15.40:0/3139471441 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55696005dff0 con 0x5569601bad80 2023-07-18T03:40:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.308+0000 7f7836ff5700 10 client.16860 handle_command_reply: tid=0 2023-07-18T03:40:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.308+0000 7f786564bb80 1 client.16860 shutdown 2023-07-18T03:40:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f786564bb80 1 -- 172.21.15.40:0/3139471441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569601526e0 msgr2=0x556960160d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f786564bb80 1 --2- 172.21.15.40:0/3139471441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569601526e0 0x556960160d20 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f784c00ef00 tx=0x7f784c0077e0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f786564bb80 1 -- 172.21.15.40:0/3139471441 shutdown_connections 2023-07-18T03:40:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f786564bb80 1 --2- 172.21.15.40:0/3139471441 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5569601bad80 0x5569601bd190 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f786564bb80 1 --2- 172.21.15.40:0/3139471441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5569601526e0 0x556960160d20 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f7836ff5700 0 client.16860 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:40:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f786564bb80 1 --2- 172.21.15.40:0/3139471441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55695ffad6b0 0x55696015d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f786564bb80 1 --2- 172.21.15.40:0/3139471441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556960151840 0x55696015dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f786564bb80 1 -- 172.21.15.40:0/3139471441 >> 172.21.15.40:0/3139471441 conn(0x556960146be0 msgr2=0x55696015a230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f786564bb80 1 -- 172.21.15.40:0/3139471441 shutdown_connections 2023-07-18T03:40:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f786564bb80 1 -- 172.21.15.40:0/3139471441 wait complete. 2023-07-18T03:40:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f786564bb80 20 client.16860 trim_cache size 0 max 16384 2023-07-18T03:40:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:40:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:40:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:40:57.318 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:40:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f78594c5700 1 -- 172.21.15.40:0/2087443377 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f783c0a1150 msgr2=0x7f783c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f78594c5700 1 --2- 172.21.15.40:0/2087443377 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f783c0a1150 0x7f783c0a3600 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7844009fd0 tx=0x7f7844009260 comp rx=0 tx=0).stop 2023-07-18T03:40:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f78594c5700 1 -- 172.21.15.40:0/2087443377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7854068640 msgr2=0x7f78541959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:40:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f78594c5700 1 --2- 172.21.15.40:0/2087443377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7854068640 0x7f78541959a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f78480098e0 tx=0x7f7848009580 comp rx=0 tx=0).stop 2023-07-18T03:40:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f78594c5700 1 -- 172.21.15.40:0/2087443377 shutdown_connections 2023-07-18T03:40:57.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f78594c5700 1 --2- 172.21.15.40:0/2087443377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78541056e0 0x7f7854196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f78594c5700 1 --2- 172.21.15.40:0/2087443377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7854068640 0x7f78541959a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f78594c5700 1 --2- 172.21.15.40:0/2087443377 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f783c0a1150 0x7f783c0a3600 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.309+0000 7f78594c5700 1 --2- 172.21.15.40:0/2087443377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7854102bd0 0x7f7854195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:40:57.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.310+0000 7f78594c5700 1 -- 172.21.15.40:0/2087443377 >> 172.21.15.40:0/2087443377 conn(0x7f78540fa000 msgr2=0x7f7854100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:40:57.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.310+0000 7f78594c5700 1 -- 172.21.15.40:0/2087443377 shutdown_connections 2023-07-18T03:40:57.320 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:40:57.310+0000 7f78594c5700 1 -- 172.21.15.40:0/2087443377 wait complete. 2023-07-18T03:40:57.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:57 smithi040 ceph-mon[96439]: pgmap v596: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 446 op/s 2023-07-18T03:40:57.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:57 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:40:57.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:57 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:40:57.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:57 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:40:57.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:57 smithi064 ceph-mon[98059]: pgmap v596: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 446 op/s 2023-07-18T03:40:57.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:57 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:40:57.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:57 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:40:57.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:57 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:40:57.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:57 smithi134 ceph-mon[98216]: pgmap v596: 97 pgs: 97 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 446 op/s 2023-07-18T03:40:57.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:57 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:40:57.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:57 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:40:57.815 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:57 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:40:59.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:40:59 smithi040 ceph-mon[96439]: pgmap v597: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 514 op/s 2023-07-18T03:40:59.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:40:59 smithi064 ceph-mon[98059]: pgmap v597: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 514 op/s 2023-07-18T03:40:59.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:40:59 smithi134 ceph-mon[98216]: pgmap v597: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 514 op/s 2023-07-18T03:41:01.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:01 smithi040 ceph-mon[96439]: pgmap v598: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 423 op/s 2023-07-18T03:41:01.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:01 smithi064 ceph-mon[98059]: pgmap v598: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 423 op/s 2023-07-18T03:41:01.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:01 smithi134 ceph-mon[98216]: pgmap v598: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 423 op/s 2023-07-18T03:41:03.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:02 smithi134 ceph-mon[98216]: pgmap v599: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 551 op/s 2023-07-18T03:41:03.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:02 smithi040 ceph-mon[96439]: pgmap v599: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 551 op/s 2023-07-18T03:41:03.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:02 smithi064 ceph-mon[98059]: pgmap v599: 97 pgs: 97 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 551 op/s 2023-07-18T03:41:05.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:05 smithi040 ceph-mon[96439]: pgmap v600: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 448 op/s 2023-07-18T03:41:05.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:05 smithi064 ceph-mon[98059]: pgmap v600: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 448 op/s 2023-07-18T03:41:05.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:05 smithi134 ceph-mon[98216]: pgmap v600: 97 pgs: 97 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 448 op/s 2023-07-18T03:41:07.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:07 smithi040 ceph-mon[96439]: pgmap v601: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 487 op/s 2023-07-18T03:41:07.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:07 smithi064 ceph-mon[98059]: pgmap v601: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 487 op/s 2023-07-18T03:41:07.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:07 smithi134 ceph-mon[98216]: pgmap v601: 97 pgs: 97 active+clean; 1.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 487 op/s 2023-07-18T03:41:09.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:09 smithi040 ceph-mon[96439]: pgmap v602: 97 pgs: 97 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 492 op/s 2023-07-18T03:41:09.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:09 smithi064 ceph-mon[98059]: pgmap v602: 97 pgs: 97 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 492 op/s 2023-07-18T03:41:09.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:09 smithi134 ceph-mon[98216]: pgmap v602: 97 pgs: 97 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 492 op/s 2023-07-18T03:41:10.422 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.428+0000 7fcc41ba1700 1 -- 172.21.15.40:0/1807269921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc3c0fea00 msgr2=0x7fcc3c0fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.428+0000 7fcc41ba1700 1 --2- 172.21.15.40:0/1807269921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc3c0fea00 0x7fcc3c0fee10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009990 tx=0x7fcc2c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.428+0000 7fcc41ba1700 1 -- 172.21.15.40:0/1807269921 shutdown_connections 2023-07-18T03:41:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.428+0000 7fcc41ba1700 1 --2- 172.21.15.40:0/1807269921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc3c100640 0x7fcc3c106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.428+0000 7fcc41ba1700 1 --2- 172.21.15.40:0/1807269921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc3c0fea00 0x7fcc3c0fee10 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.423 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.428+0000 7fcc41ba1700 1 --2- 172.21.15.40:0/1807269921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc3c0ffc90 0x7fcc3c100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.428+0000 7fcc41ba1700 1 -- 172.21.15.40:0/1807269921 >> 172.21.15.40:0/1807269921 conn(0x7fcc3c0fa000 msgr2=0x7fcc3c0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.428+0000 7fcc41ba1700 1 -- 172.21.15.40:0/1807269921 shutdown_connections 2023-07-18T03:41:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.428+0000 7fcc41ba1700 1 -- 172.21.15.40:0/1807269921 wait complete. 2023-07-18T03:41:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.429+0000 7fcc41ba1700 1 Processor -- start 2023-07-18T03:41:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.429+0000 7fcc41ba1700 1 -- start start 2023-07-18T03:41:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.430+0000 7fcc41ba1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc3c0fea00 0x7fcc3c196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.430+0000 7fcc41ba1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc3c0ffc90 0x7fcc3c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.430+0000 7fcc41ba1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc3c100640 0x7fcc3c198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.430+0000 7fcc41ba1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c1993d0 con 0x7fcc3c0fea00 2023-07-18T03:41:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.430+0000 7fcc41ba1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c199540 con 0x7fcc3c100640 2023-07-18T03:41:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.430+0000 7fcc41ba1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3c199700 con 0x7fcc3c0ffc90 2023-07-18T03:41:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.430+0000 7fcc3b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc3c0fea00 0x7fcc3c196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.430+0000 7fcc3affd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc3c0ffc90 0x7fcc3c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.430+0000 7fcc3b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc3c0fea00 0x7fcc3c196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33274/0 (socket says 172.21.15.40:33274) 2023-07-18T03:41:10.426 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.430+0000 7fcc3b7fe700 1 -- 172.21.15.40:0/1420259777 learned_addr learned my addr 172.21.15.40:0/1420259777 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.431+0000 7fcc3affd700 1 -- 172.21.15.40:0/1420259777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc3c100640 msgr2=0x7fcc3c198dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:41:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.431+0000 7fcc3affd700 1 --2- 172.21.15.40:0/1420259777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc3c100640 0x7fcc3c198dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.431+0000 7fcc3affd700 1 -- 172.21.15.40:0/1420259777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc3c0fea00 msgr2=0x7fcc3c196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.431+0000 7fcc3affd700 1 --2- 172.21.15.40:0/1420259777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc3c0fea00 0x7fcc3c196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.431+0000 7fcc3affd700 1 -- 172.21.15.40:0/1420259777 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c0095f0 con 0x7fcc3c0ffc90 2023-07-18T03:41:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.431+0000 7fcc3b7fe700 1 --2- 172.21.15.40:0/1420259777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc3c0fea00 0x7fcc3c196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.431+0000 7fcc3affd700 1 --2- 172.21.15.40:0/1420259777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc3c0ffc90 0x7fcc3c198890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000e9f0 tx=0x7fcc3000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.432+0000 7fcc38ff9700 1 -- 172.21.15.40:0/1420259777 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3000eee0 con 0x7fcc3c0ffc90 2023-07-18T03:41:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.432+0000 7fcc38ff9700 1 -- 172.21.15.40:0/1420259777 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc3000cbd0 con 0x7fcc3c0ffc90 2023-07-18T03:41:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.432+0000 7fcc41ba1700 1 -- 172.21.15.40:0/1420259777 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc3c1999e0 con 0x7fcc3c0ffc90 2023-07-18T03:41:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.432+0000 7fcc38ff9700 1 -- 172.21.15.40:0/1420259777 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc30018680 con 0x7fcc3c0ffc90 2023-07-18T03:41:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.432+0000 7fcc41ba1700 1 -- 172.21.15.40:0/1420259777 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3c199f70 con 0x7fcc3c0ffc90 2023-07-18T03:41:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.435+0000 7fcc38ff9700 1 -- 172.21.15.40:0/1420259777 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcc3000cd40 con 0x7fcc3c0ffc90 2023-07-18T03:41:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.435+0000 7fcc38ff9700 1 --2- 172.21.15.40:0/1420259777 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcc240a1150 0x7fcc240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.436+0000 7fcc38ff9700 1 -- 172.21.15.40:0/1420259777 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcc30014070 con 0x7fcc3c0ffc90 2023-07-18T03:41:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.436+0000 7fcc3b7fe700 1 --2- 172.21.15.40:0/1420259777 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcc240a1150 0x7fcc240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.436+0000 7fcc4dd27b80 1 Processor -- start 2023-07-18T03:41:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.437+0000 7fcc3b7fe700 1 --2- 172.21.15.40:0/1420259777 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcc240a1150 0x7fcc240a3600 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c068420 tx=0x7fcc2c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.438+0000 7fcc4dd27b80 1 -- start start 2023-07-18T03:41:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.438+0000 7fcc4dd27b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91da66b0 0x55fd91f4a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.438+0000 7fcc4dd27b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91f4a840 0x55fd91f4b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.438+0000 7fcc4dd27b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4b6e0 0x55fd91f4bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.438+0000 7fcc4dd27b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd91e4fc50 con 0x55fd91f4b6e0 2023-07-18T03:41:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.438+0000 7fcc4dd27b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd91ea29e0 con 0x55fd91f4a840 2023-07-18T03:41:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.438+0000 7fcc4dd27b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd91e9be70 con 0x55fd91da66b0 2023-07-18T03:41:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.438+0000 7fcc3b7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91da66b0 0x55fd91f4a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.438+0000 7fcc3b7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91da66b0 0x55fd91f4a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50538/0 (socket says 172.21.15.40:50538) 2023-07-18T03:41:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.438+0000 7fcc3b7fe700 1 -- 172.21.15.40:0/3161066719 learned_addr learned my addr 172.21.15.40:0/3161066719 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.438+0000 7fcc3bfff700 1 --2- 172.21.15.40:0/3161066719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4b6e0 0x55fd91f4bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.438+0000 7fcc3affd700 1 --2- 172.21.15.40:0/3161066719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91f4a840 0x55fd91f4b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.439+0000 7fcc3affd700 1 -- 172.21.15.40:0/3161066719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91da66b0 msgr2=0x55fd91f4a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.439+0000 7fcc3affd700 1 --2- 172.21.15.40:0/3161066719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91da66b0 0x55fd91f4a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.439+0000 7fcc3affd700 1 -- 172.21.15.40:0/3161066719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4b6e0 msgr2=0x55fd91f4bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.439+0000 7fcc3affd700 1 --2- 172.21.15.40:0/3161066719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4b6e0 0x55fd91f4bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.439+0000 7fcc3affd700 1 -- 172.21.15.40:0/3161066719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd91cc9d80 con 0x55fd91f4a840 2023-07-18T03:41:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.439+0000 7fcc3bfff700 1 --2- 172.21.15.40:0/3161066719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4b6e0 0x55fd91f4bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.439+0000 7fcc3affd700 1 --2- 172.21.15.40:0/3161066719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91f4a840 0x55fd91f4b1a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000e9c0 tx=0x7fcc30099880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52931a2124e85c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.439+0000 7fcc21ffb700 1 -- 172.21.15.40:0/3161066719 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc300b1070 con 0x55fd91f4a840 2023-07-18T03:41:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.439+0000 7fcc21ffb700 1 -- 172.21.15.40:0/3161066719 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc3009df50 con 0x55fd91f4a840 2023-07-18T03:41:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.440+0000 7fcc21ffb700 1 -- 172.21.15.40:0/3161066719 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc30099670 con 0x55fd91f4a840 2023-07-18T03:41:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.440+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/3161066719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91f4a840 msgr2=0x55fd91f4b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.440+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/3161066719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91f4a840 0x55fd91f4b1a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000e9c0 tx=0x7fcc30099880 comp rx=0 tx=0).stop 2023-07-18T03:41:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.441+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/3161066719 shutdown_connections 2023-07-18T03:41:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.441+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/3161066719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4b6e0 0x55fd91f4bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.441+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/3161066719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91da66b0 0x55fd91f4a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.441+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/3161066719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91f4a840 0x55fd91f4b1a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.441+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/3161066719 >> 172.21.15.40:0/3161066719 conn(0x55fd91f3fbe0 msgr2=0x55fd91f44cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.441+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/3161066719 shutdown_connections 2023-07-18T03:41:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.441+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/3161066719 wait complete. 2023-07-18T03:41:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.441+0000 7fcc4dd27b80 1 -- start start 2023-07-18T03:41:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc4dd27b80 1 Processor -- start 2023-07-18T03:41:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc4dd27b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91da66b0 0x55fd91f538d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc4dd27b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91f4a840 0x55fd91f53e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc4dd27b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4b6e0 0x55fd91f5ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc4dd27b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd91e3ba60 con 0x55fd91f4b6e0 2023-07-18T03:41:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc4dd27b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd91e3b760 con 0x55fd91da66b0 2023-07-18T03:41:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc4dd27b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd91d1f8c0 con 0x55fd91f4a840 2023-07-18T03:41:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc4dd27b80 10 client.0 _ll_register_callbacks cb 0x55fd91e39700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:10.441 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc3affd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91f4a840 0x55fd91f53e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:10.441 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc3affd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91f4a840 0x55fd91f53e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50544/0 (socket says 172.21.15.40:50544) 2023-07-18T03:41:10.441 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc3affd700 1 -- 172.21.15.40:0/1433596255 learned_addr learned my addr 172.21.15.40:0/1433596255 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:10.441 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc3b7fe700 1 --2- 172.21.15.40:0/1433596255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91da66b0 0x55fd91f538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:10.441 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc3bfff700 1 --2- 172.21.15.40:0/1433596255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4b6e0 0x55fd91f5ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:10.441 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc3affd700 1 -- 172.21.15.40:0/1433596255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91da66b0 msgr2=0x55fd91f538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:10.441 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc3affd700 1 --2- 172.21.15.40:0/1433596255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91da66b0 0x55fd91f538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.441 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc3affd700 1 -- 172.21.15.40:0/1433596255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4b6e0 msgr2=0x55fd91f5ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc3affd700 1 --2- 172.21.15.40:0/1433596255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4b6e0 0x55fd91f5ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc3affd700 1 -- 172.21.15.40:0/1433596255 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3400d040 con 0x55fd91f4a840 2023-07-18T03:41:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.442+0000 7fcc3bfff700 1 --2- 172.21.15.40:0/1433596255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4b6e0 0x55fd91f5ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:41:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.443+0000 7fcc3affd700 1 --2- 172.21.15.40:0/1433596255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91f4a840 0x55fd91f53e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcc300a1e40 tx=0x7fcc300a2240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c9760f51a61c193 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.444+0000 7fcc1affd700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:41:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.444+0000 7fcc1affd700 1 -- 172.21.15.40:0/1433596255 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc300b1070 con 0x55fd91f4a840 2023-07-18T03:41:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.444+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/1433596255 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd91df5490 con 0x55fd91f4a840 2023-07-18T03:41:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.444+0000 7fcc1affd700 1 -- 172.21.15.40:0/1433596255 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcc300b82c0 con 0x55fd91f4a840 2023-07-18T03:41:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.444+0000 7fcc1affd700 1 -- 172.21.15.40:0/1433596255 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc30094d40 con 0x55fd91f4a840 2023-07-18T03:41:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.444+0000 7fcc4dd27b80 10 client.26530 fetch_fsmap learned FSMap version 68 2023-07-18T03:41:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.444+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/1433596255 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7fcc3c199f70 con 0x55fd91f4a840 2023-07-18T03:41:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.444+0000 7fcc1affd700 1 -- 172.21.15.40:0/1433596255 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd91df5490 con 0x55fd91f4a840 2023-07-18T03:41:10.444 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.444+0000 7fcc1affd700 1 -- 172.21.15.40:0/1433596255 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 68) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fcc300ca160 con 0x55fd91f4a840 2023-07-18T03:41:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.444+0000 7fcc4dd27b80 10 client.26530 fetch_fsmap finished waiting for FSMap version 68 2023-07-18T03:41:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.444+0000 7fcc4dd27b80 10 client.26530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:41:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.444+0000 7fcc4dd27b80 20 client.26530 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.444+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/1433596255 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fd91f70450 0x55fd91f72860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.444+0000 7fcc4dd27b80 4 client.26530 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:41:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.444+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/1433596255 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd91e94e50 con 0x55fd91f70450 2023-07-18T03:41:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.445+0000 7fcc3b7fe700 1 --2- 172.21.15.40:0/1433596255 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fd91f70450 0x55fd91f72860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.445+0000 7fcc3b7fe700 1 --2- 172.21.15.40:0/1433596255 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fd91f70450 0x55fd91f72860 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df4a0ef1f1119d7b server_cookie=78e89b657ac3a01b in_seq=0 out_seq=0 2023-07-18T03:41:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.445+0000 7fcc1affd700 10 client.26530 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.446+0000 7fcc1affd700 1 -- 172.21.15.40:0/1433596255 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fd91e94e50 con 0x55fd91f70450 2023-07-18T03:41:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.446+0000 7fcc1affd700 10 client.26530 handle_command_reply: tid=0 2023-07-18T03:41:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.446+0000 7fcc4dd27b80 1 client.26530 shutdown 2023-07-18T03:41:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.446+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/1433596255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91f4a840 msgr2=0x55fd91f53e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.446+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/1433596255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91f4a840 0x55fd91f53e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcc300a1e40 tx=0x7fcc300a2240 comp rx=0 tx=0).stop 2023-07-18T03:41:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.446+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/1433596255 shutdown_connections 2023-07-18T03:41:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.446+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/1433596255 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fd91f70450 0x55fd91f72860 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.446+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/1433596255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4b6e0 0x55fd91f5ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.446+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/1433596255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91da66b0 0x55fd91f538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.446+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/1433596255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91f4a840 0x55fd91f53e10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.446+0000 7fcc1affd700 0 client.26530 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.446+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/1433596255 >> 172.21.15.40:0/1433596255 conn(0x55fd91f3fbe0 msgr2=0x55fd91f444e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:10.447 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.447+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/1433596255 shutdown_connections 2023-07-18T03:41:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.447+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/1433596255 wait complete. 2023-07-18T03:41:10.448 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.447+0000 7fcc4dd27b80 20 client.26530 trim_cache size 0 max 16384 2023-07-18T03:41:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.470+0000 7fcc4dd27b80 1 Processor -- start 2023-07-18T03:41:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.470+0000 7fcc4dd27b80 1 -- start start 2023-07-18T03:41:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.470+0000 7fcc4dd27b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91da66b0 0x55fd91f525a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.470+0000 7fcc4dd27b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4a840 0x55fd91f5ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.470+0000 7fcc4dd27b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91f4b6e0 0x55fd91f4e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.470+0000 7fcc4dd27b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd91f6ff60 con 0x55fd91f4a840 2023-07-18T03:41:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.470+0000 7fcc4dd27b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd91f52d80 con 0x55fd91da66b0 2023-07-18T03:41:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.470+0000 7fcc4dd27b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd91f41d00 con 0x55fd91f4b6e0 2023-07-18T03:41:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.471+0000 7fcc3affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4a840 0x55fd91f5ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.471+0000 7fcc3affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4a840 0x55fd91f5ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33292/0 (socket says 172.21.15.40:33292) 2023-07-18T03:41:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.471+0000 7fcc3affd700 1 -- 172.21.15.40:0/3962557165 learned_addr learned my addr 172.21.15.40:0/3962557165 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.471+0000 7fcc3b7fe700 1 --2- 172.21.15.40:0/3962557165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91da66b0 0x55fd91f525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.471+0000 7fcc3bfff700 1 --2- 172.21.15.40:0/3962557165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91f4b6e0 0x55fd91f4e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.471+0000 7fcc3affd700 1 -- 172.21.15.40:0/3962557165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91f4b6e0 msgr2=0x55fd91f4e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.471+0000 7fcc3affd700 1 --2- 172.21.15.40:0/3962557165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91f4b6e0 0x55fd91f4e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.467 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.471+0000 7fcc3affd700 1 -- 172.21.15.40:0/3962557165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91da66b0 msgr2=0x55fd91f525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.471+0000 7fcc3affd700 1 --2- 172.21.15.40:0/3962557165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91da66b0 0x55fd91f525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.471+0000 7fcc3affd700 1 -- 172.21.15.40:0/3962557165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd91cb2c00 con 0x55fd91f4a840 2023-07-18T03:41:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.471+0000 7fcc3b7fe700 1 --2- 172.21.15.40:0/3962557165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91da66b0 0x55fd91f525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.471+0000 7fcc3affd700 1 --2- 172.21.15.40:0/3962557165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4a840 0x55fd91f5ae10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcc300ca980 tx=0x7fcc30098330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6956ba00d441b9f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.471+0000 7fcc227fc700 1 -- 172.21.15.40:0/3962557165 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc300b1030 con 0x55fd91f4a840 2023-07-18T03:41:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.471+0000 7fcc227fc700 1 -- 172.21.15.40:0/3962557165 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc300a23f0 con 0x55fd91f4a840 2023-07-18T03:41:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.471+0000 7fcc227fc700 1 -- 172.21.15.40:0/3962557165 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc300cad30 con 0x55fd91f4a840 2023-07-18T03:41:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.472+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/3962557165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4a840 msgr2=0x55fd91f5ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.472+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/3962557165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4a840 0x55fd91f5ae10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcc300ca980 tx=0x7fcc30098330 comp rx=0 tx=0).stop 2023-07-18T03:41:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.472+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/3962557165 shutdown_connections 2023-07-18T03:41:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.472+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/3962557165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91f4b6e0 0x55fd91f4e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.472+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/3962557165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91da66b0 0x55fd91f525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.471 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.472+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/3962557165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4a840 0x55fd91f5ae10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.472+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/3962557165 >> 172.21.15.40:0/3962557165 conn(0x55fd91f3fbe0 msgr2=0x55fd91f77180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.472+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/3962557165 shutdown_connections 2023-07-18T03:41:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.472+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/3962557165 wait complete. 2023-07-18T03:41:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.472+0000 7fcc4dd27b80 1 -- start start 2023-07-18T03:41:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc4dd27b80 1 Processor -- start 2023-07-18T03:41:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc4dd27b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91da66b0 0x55fd91f56fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.472 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc4dd27b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4a840 0x55fd91f574e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc4dd27b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91f4b6e0 0x55fd91f59e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc4dd27b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd91f5a340 con 0x55fd91f4a840 2023-07-18T03:41:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc4dd27b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd91f5a4b0 con 0x55fd91da66b0 2023-07-18T03:41:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc4dd27b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd91f5a670 con 0x55fd91f4b6e0 2023-07-18T03:41:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc4dd27b80 10 client.0 _ll_register_callbacks cb 0x55fd91e39700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:10.473 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc3affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4a840 0x55fd91f574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc3b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91da66b0 0x55fd91f56fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc3bfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91f4b6e0 0x55fd91f59e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc3affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4a840 0x55fd91f574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33300/0 (socket says 172.21.15.40:33300) 2023-07-18T03:41:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc3affd700 1 -- 172.21.15.40:0/2382088677 learned_addr learned my addr 172.21.15.40:0/2382088677 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc3b7fe700 1 -- 172.21.15.40:0/2382088677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91f4b6e0 msgr2=0x55fd91f59e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc3b7fe700 1 --2- 172.21.15.40:0/2382088677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91f4b6e0 0x55fd91f59e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc3b7fe700 1 -- 172.21.15.40:0/2382088677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4a840 msgr2=0x55fd91f574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc3b7fe700 1 --2- 172.21.15.40:0/2382088677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4a840 0x55fd91f574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc3b7fe700 1 -- 172.21.15.40:0/2382088677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3c107750 con 0x55fd91da66b0 2023-07-18T03:41:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc3b7fe700 1 --2- 172.21.15.40:0/2382088677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91da66b0 0x55fd91f56fa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c005850 tx=0x7fcc2c01a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f443eb9d5d5ae44 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.473+0000 7fcc1affd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:41:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.474+0000 7fcc1affd700 1 -- 172.21.15.40:0/2382088677 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0352e0 con 0x55fd91da66b0 2023-07-18T03:41:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.474+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/2382088677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc3009df50 con 0x55fd91da66b0 2023-07-18T03:41:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.474+0000 7fcc1affd700 1 -- 172.21.15.40:0/2382088677 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fcc2c035440 con 0x55fd91da66b0 2023-07-18T03:41:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.474+0000 7fcc1affd700 1 -- 172.21.15.40:0/2382088677 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc2c035600 con 0x55fd91da66b0 2023-07-18T03:41:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.474+0000 7fcc4dd27b80 10 client.26516 fetch_fsmap learned FSMap version 68 2023-07-18T03:41:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.474+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/2382088677 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) v3 -- 0x7fcc3c199f70 con 0x55fd91da66b0 2023-07-18T03:41:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.474+0000 7fcc1affd700 1 -- 172.21.15.40:0/2382088677 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc3009df50 con 0x55fd91da66b0 2023-07-18T03:41:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.474+0000 7fcc1affd700 1 -- 172.21.15.40:0/2382088677 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 68) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fcc2c005350 con 0x55fd91da66b0 2023-07-18T03:41:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.474+0000 7fcc4dd27b80 10 client.26516 fetch_fsmap finished waiting for FSMap version 68 2023-07-18T03:41:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.474+0000 7fcc4dd27b80 10 client.26516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:41:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.474+0000 7fcc4dd27b80 20 client.26516 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.474+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/2382088677 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fd91fc5060 0x55fd91fc7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.474+0000 7fcc4dd27b80 4 client.26516 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:41:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.474+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/2382088677 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fd91e56ff0 con 0x55fd91fc5060 2023-07-18T03:41:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.474+0000 7fcc3affd700 1 --2- 172.21.15.40:0/2382088677 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fd91fc5060 0x55fd91fc7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.475+0000 7fcc1affd700 10 client.26516 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.475+0000 7fcc3affd700 1 --2- 172.21.15.40:0/2382088677 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fd91fc5060 0x55fd91fc7470 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad4a7d20b5928bd6 server_cookie=8c88cbbe1fcf7a9e in_seq=0 out_seq=0 2023-07-18T03:41:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.484+0000 7fcc1affd700 1 -- 172.21.15.40:0/2382088677 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fd91e56ff0 con 0x55fd91fc5060 2023-07-18T03:41:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.484+0000 7fcc1affd700 10 client.26516 handle_command_reply: tid=0 2023-07-18T03:41:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.484+0000 7fcc4dd27b80 1 client.26516 shutdown 2023-07-18T03:41:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.484+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/2382088677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91da66b0 msgr2=0x55fd91f56fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.484+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/2382088677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91da66b0 0x55fd91f56fa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c005850 tx=0x7fcc2c01a050 comp rx=0 tx=0).stop 2023-07-18T03:41:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.484+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/2382088677 shutdown_connections 2023-07-18T03:41:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.484+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/2382088677 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55fd91fc5060 0x55fd91fc7470 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.484+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/2382088677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd91f4b6e0 0x55fd91f59e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.479 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.484+0000 7fcc1affd700 0 client.26516 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.484+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/2382088677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd91da66b0 0x55fd91f56fa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.484+0000 7fcc4dd27b80 1 --2- 172.21.15.40:0/2382088677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55fd91f4a840 0x55fd91f574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.484+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/2382088677 >> 172.21.15.40:0/2382088677 conn(0x55fd91f3fbe0 msgr2=0x55fd91f525a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.484+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/2382088677 shutdown_connections 2023-07-18T03:41:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.484+0000 7fcc4dd27b80 1 -- 172.21.15.40:0/2382088677 wait complete. 2023-07-18T03:41:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.484+0000 7fcc4dd27b80 20 client.26516 trim_cache size 0 max 16384 2023-07-18T03:41:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:41:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:41:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:41:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:41:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.485+0000 7fcc41ba1700 1 -- 172.21.15.40:0/1420259777 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcc240a1150 msgr2=0x7fcc240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.485+0000 7fcc41ba1700 1 --2- 172.21.15.40:0/1420259777 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcc240a1150 0x7fcc240a3600 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c068420 tx=0x7fcc2c005960 comp rx=0 tx=0).stop 2023-07-18T03:41:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.485+0000 7fcc41ba1700 1 -- 172.21.15.40:0/1420259777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc3c0ffc90 msgr2=0x7fcc3c198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.485+0000 7fcc41ba1700 1 --2- 172.21.15.40:0/1420259777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc3c0ffc90 0x7fcc3c198890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000e9f0 tx=0x7fcc3000ed00 comp rx=0 tx=0).stop 2023-07-18T03:41:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.485+0000 7fcc41ba1700 1 -- 172.21.15.40:0/1420259777 shutdown_connections 2023-07-18T03:41:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.485+0000 7fcc41ba1700 1 --2- 172.21.15.40:0/1420259777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc3c100640 0x7fcc3c198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.485+0000 7fcc41ba1700 1 --2- 172.21.15.40:0/1420259777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc3c0fea00 0x7fcc3c196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.485+0000 7fcc41ba1700 1 --2- 172.21.15.40:0/1420259777 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fcc240a1150 0x7fcc240a3600 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.485+0000 7fcc41ba1700 1 --2- 172.21.15.40:0/1420259777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc3c0ffc90 0x7fcc3c198890 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.485+0000 7fcc41ba1700 1 -- 172.21.15.40:0/1420259777 >> 172.21.15.40:0/1420259777 conn(0x7fcc3c0fa000 msgr2=0x7fcc3c103150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.485+0000 7fcc41ba1700 1 -- 172.21.15.40:0/1420259777 shutdown_connections 2023-07-18T03:41:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:10.485+0000 7fcc41ba1700 1 -- 172.21.15.40:0/1420259777 wait complete. 2023-07-18T03:41:11.613 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:11 smithi064 ceph-mon[98059]: pgmap v603: 97 pgs: 97 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 375 op/s 2023-07-18T03:41:11.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:11 smithi134 ceph-mon[98216]: pgmap v603: 97 pgs: 97 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 375 op/s 2023-07-18T03:41:11.827 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:11 smithi040 ceph-mon[96439]: pgmap v603: 97 pgs: 97 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 375 op/s 2023-07-18T03:41:13.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:12 smithi134 ceph-mon[98216]: pgmap v604: 97 pgs: 97 active+clean; 1014 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 459 op/s 2023-07-18T03:41:13.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:12 smithi040 ceph-mon[96439]: pgmap v604: 97 pgs: 97 active+clean; 1014 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 459 op/s 2023-07-18T03:41:13.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:12 smithi064 ceph-mon[98059]: pgmap v604: 97 pgs: 97 active+clean; 1014 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 459 op/s 2023-07-18T03:41:15.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:15 smithi040 ceph-mon[96439]: pgmap v605: 97 pgs: 97 active+clean; 1013 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 378 op/s 2023-07-18T03:41:15.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:15 smithi064 ceph-mon[98059]: pgmap v605: 97 pgs: 97 active+clean; 1013 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 378 op/s 2023-07-18T03:41:15.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:15 smithi134 ceph-mon[98216]: pgmap v605: 97 pgs: 97 active+clean; 1013 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 378 op/s 2023-07-18T03:41:17.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:17 smithi040 ceph-mon[96439]: pgmap v606: 97 pgs: 97 active+clean; 994 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 405 op/s 2023-07-18T03:41:17.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:17 smithi064 ceph-mon[98059]: pgmap v606: 97 pgs: 97 active+clean; 994 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 405 op/s 2023-07-18T03:41:17.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:17 smithi134 ceph-mon[98216]: pgmap v606: 97 pgs: 97 active+clean; 994 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 405 op/s 2023-07-18T03:41:19.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:19 smithi040 ceph-mon[96439]: pgmap v607: 97 pgs: 97 active+clean; 975 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 400 op/s 2023-07-18T03:41:19.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:19 smithi064 ceph-mon[98059]: pgmap v607: 97 pgs: 97 active+clean; 975 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 400 op/s 2023-07-18T03:41:19.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:19 smithi134 ceph-mon[98216]: pgmap v607: 97 pgs: 97 active+clean; 975 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 400 op/s 2023-07-18T03:41:20.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:20 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:41:20.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:20 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:41:20.827 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:20 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:41:21.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:21 smithi064 ceph-mon[98059]: pgmap v608: 97 pgs: 97 active+clean; 971 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 335 op/s 2023-07-18T03:41:21.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:21 smithi134 ceph-mon[98216]: pgmap v608: 97 pgs: 97 active+clean; 971 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 335 op/s 2023-07-18T03:41:21.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:21 smithi040 ceph-mon[96439]: pgmap v608: 97 pgs: 97 active+clean; 971 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 335 op/s 2023-07-18T03:41:23.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:22 smithi134 ceph-mon[98216]: pgmap v609: 97 pgs: 97 active+clean; 953 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 439 op/s 2023-07-18T03:41:23.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:22 smithi040 ceph-mon[96439]: pgmap v609: 97 pgs: 97 active+clean; 953 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 439 op/s 2023-07-18T03:41:23.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:22 smithi064 ceph-mon[98059]: pgmap v609: 97 pgs: 97 active+clean; 953 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 439 op/s 2023-07-18T03:41:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.597+0000 7ff4ad0d8700 1 -- 172.21.15.40:0/1294963106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4a80ffc90 msgr2=0x7ff4a8100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.597+0000 7ff4ad0d8700 1 --2- 172.21.15.40:0/1294963106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4a80ffc90 0x7ff4a8100100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0098f0 tx=0x7ff49c009c00 comp rx=0 tx=0).stop 2023-07-18T03:41:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.597+0000 7ff4ad0d8700 1 -- 172.21.15.40:0/1294963106 shutdown_connections 2023-07-18T03:41:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.597+0000 7ff4ad0d8700 1 --2- 172.21.15.40:0/1294963106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a8100640 0x7ff4a8106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.597+0000 7ff4ad0d8700 1 --2- 172.21.15.40:0/1294963106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a80fea00 0x7ff4a80fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.597+0000 7ff4ad0d8700 1 --2- 172.21.15.40:0/1294963106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4a80ffc90 0x7ff4a8100100 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.597+0000 7ff4ad0d8700 1 -- 172.21.15.40:0/1294963106 >> 172.21.15.40:0/1294963106 conn(0x7ff4a80fa000 msgr2=0x7ff4a80fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.597+0000 7ff4ad0d8700 1 -- 172.21.15.40:0/1294963106 shutdown_connections 2023-07-18T03:41:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.597+0000 7ff4ad0d8700 1 -- 172.21.15.40:0/1294963106 wait complete. 2023-07-18T03:41:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.598+0000 7ff4ad0d8700 1 Processor -- start 2023-07-18T03:41:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.598+0000 7ff4ad0d8700 1 -- start start 2023-07-18T03:41:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.598+0000 7ff4ad0d8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4a80fea00 0x7ff4a8194870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.598+0000 7ff4ad0d8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a80ffc90 0x7ff4a8194db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.598+0000 7ff4ad0d8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a8100640 0x7ff4a8195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.598+0000 7ff4ad0d8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a8196290 con 0x7ff4a8100640 2023-07-18T03:41:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.598+0000 7ff4ad0d8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a8196400 con 0x7ff4a80ffc90 2023-07-18T03:41:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.598+0000 7ff4ad0d8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a81965c0 con 0x7ff4a80fea00 2023-07-18T03:41:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.598+0000 7ff4a759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a8100640 0x7ff4a8195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.598+0000 7ff4a659c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a80ffc90 0x7ff4a8194db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.598+0000 7ff4a759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a8100640 0x7ff4a8195c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33316/0 (socket says 172.21.15.40:33316) 2023-07-18T03:41:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.598+0000 7ff4a6d9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4a80fea00 0x7ff4a8194870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.598+0000 7ff4a759e700 1 -- 172.21.15.40:0/2923447789 learned_addr learned my addr 172.21.15.40:0/2923447789 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.599+0000 7ff4a659c700 1 -- 172.21.15.40:0/2923447789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4a80fea00 msgr2=0x7ff4a8194870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.599+0000 7ff4a659c700 1 --2- 172.21.15.40:0/2923447789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4a80fea00 0x7ff4a8194870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.599+0000 7ff4a659c700 1 -- 172.21.15.40:0/2923447789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a8100640 msgr2=0x7ff4a8195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.599+0000 7ff4a659c700 1 --2- 172.21.15.40:0/2923447789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a8100640 0x7ff4a8195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.599+0000 7ff4a659c700 1 -- 172.21.15.40:0/2923447789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a0005e70 con 0x7ff4a80ffc90 2023-07-18T03:41:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.599+0000 7ff4a759e700 1 --2- 172.21.15.40:0/2923447789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a8100640 0x7ff4a8195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.599+0000 7ff4a659c700 1 --2- 172.21.15.40:0/2923447789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a80ffc90 0x7ff4a8194db0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff49c000c00 tx=0x7ff49c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.599+0000 7ff48ffff700 1 -- 172.21.15.40:0/2923447789 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c01e070 con 0x7ff4a80ffc90 2023-07-18T03:41:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.599+0000 7ff48ffff700 1 -- 172.21.15.40:0/2923447789 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff49c00f7d0 con 0x7ff4a80ffc90 2023-07-18T03:41:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.599+0000 7ff4ad0d8700 1 -- 172.21.15.40:0/2923447789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c009520 con 0x7ff4a80ffc90 2023-07-18T03:41:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.599+0000 7ff48ffff700 1 -- 172.21.15.40:0/2923447789 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c003990 con 0x7ff4a80ffc90 2023-07-18T03:41:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.599+0000 7ff4ad0d8700 1 -- 172.21.15.40:0/2923447789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a8067900 con 0x7ff4a80ffc90 2023-07-18T03:41:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.601+0000 7ff48ffff700 1 -- 172.21.15.40:0/2923447789 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff49c022030 con 0x7ff4a80ffc90 2023-07-18T03:41:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.602+0000 7ff48ffff700 1 --2- 172.21.15.40:0/2923447789 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4900a11d0 0x7ff4900a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.602+0000 7ff4a6d9d700 1 --2- 172.21.15.40:0/2923447789 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4900a11d0 0x7ff4900a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.602+0000 7ff48ffff700 1 -- 172.21.15.40:0/2923447789 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff49c0cfa00 con 0x7ff4a80ffc90 2023-07-18T03:41:23.604 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4a6d9d700 1 --2- 172.21.15.40:0/2923447789 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4900a11d0 0x7ff4900a3680 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff498009c60 tx=0x7ff498009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4b925eb80 1 Processor -- start 2023-07-18T03:41:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4b925eb80 1 -- start start 2023-07-18T03:41:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4b925eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f07b56b0 0x55c5f0959300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4b925eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f0959840 0x55c5f095a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4b925eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f095a6e0 0x55c5f095ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4b925eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f085ec50 con 0x55c5f0959840 2023-07-18T03:41:23.605 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4b925eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f08b19e0 con 0x55c5f07b56b0 2023-07-18T03:41:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4b925eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f08aae70 con 0x55c5f095a6e0 2023-07-18T03:41:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4a659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f0959840 0x55c5f095a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4a659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f0959840 0x55c5f095a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33322/0 (socket says 172.21.15.40:33322) 2023-07-18T03:41:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4a659c700 1 -- 172.21.15.40:0/683948751 learned_addr learned my addr 172.21.15.40:0/683948751 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4a759e700 1 --2- 172.21.15.40:0/683948751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f095a6e0 0x55c5f095ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4a6d9d700 1 --2- 172.21.15.40:0/683948751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f07b56b0 0x55c5f0959300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4a659c700 1 -- 172.21.15.40:0/683948751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f095a6e0 msgr2=0x55c5f095ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4a659c700 1 --2- 172.21.15.40:0/683948751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f095a6e0 0x55c5f095ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4a659c700 1 -- 172.21.15.40:0/683948751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f07b56b0 msgr2=0x55c5f0959300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4a659c700 1 --2- 172.21.15.40:0/683948751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f07b56b0 0x55c5f0959300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.603+0000 7ff4a659c700 1 -- 172.21.15.40:0/683948751 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5f06d8d80 con 0x55c5f0959840 2023-07-18T03:41:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.604+0000 7ff4a659c700 1 --2- 172.21.15.40:0/683948751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f0959840 0x55c5f095a1a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00fa40 tx=0x7ff49c09c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ed8fa9bf2fd92f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.604+0000 7ff48d7fa700 1 -- 172.21.15.40:0/683948751 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c0a0eb0 con 0x55c5f0959840 2023-07-18T03:41:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.604+0000 7ff48d7fa700 1 -- 172.21.15.40:0/683948751 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff49c0a5240 con 0x55c5f0959840 2023-07-18T03:41:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.604+0000 7ff48d7fa700 1 -- 172.21.15.40:0/683948751 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c023e70 con 0x55c5f0959840 2023-07-18T03:41:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.604+0000 7ff4b925eb80 1 -- 172.21.15.40:0/683948751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f0959840 msgr2=0x55c5f095a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.604+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/683948751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f0959840 0x55c5f095a1a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00fa40 tx=0x7ff49c09c9c0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.604+0000 7ff4b925eb80 1 -- 172.21.15.40:0/683948751 shutdown_connections 2023-07-18T03:41:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.604+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/683948751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f095a6e0 0x55c5f095ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.604+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/683948751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f07b56b0 0x55c5f0959300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.604+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/683948751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f0959840 0x55c5f095a1a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.604+0000 7ff4b925eb80 1 -- 172.21.15.40:0/683948751 >> 172.21.15.40:0/683948751 conn(0x55c5f094ebe0 msgr2=0x55c5f0953cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.604+0000 7ff4b925eb80 1 -- 172.21.15.40:0/683948751 shutdown_connections 2023-07-18T03:41:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.605+0000 7ff4b925eb80 1 -- 172.21.15.40:0/683948751 wait complete. 2023-07-18T03:41:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.605+0000 7ff4b925eb80 1 -- start start 2023-07-18T03:41:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.605+0000 7ff4b925eb80 1 Processor -- start 2023-07-18T03:41:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.605+0000 7ff4b925eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f07b56b0 0x55c5f09624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.605+0000 7ff4b925eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f0959840 0x55c5f0962a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.605+0000 7ff4b925eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f095a6e0 0x55c5f0969a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.605+0000 7ff4b925eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f084aa60 con 0x55c5f0959840 2023-07-18T03:41:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.605+0000 7ff4b925eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f084a760 con 0x55c5f07b56b0 2023-07-18T03:41:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.605+0000 7ff4b925eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f072e8c0 con 0x55c5f095a6e0 2023-07-18T03:41:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.605+0000 7ff4b925eb80 10 client.0 _ll_register_callbacks cb 0x55c5f0848700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.605+0000 7ff4a659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f0959840 0x55c5f0962a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.605+0000 7ff4a659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f0959840 0x55c5f0962a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33328/0 (socket says 172.21.15.40:33328) 2023-07-18T03:41:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.605+0000 7ff4a659c700 1 -- 172.21.15.40:0/3310080554 learned_addr learned my addr 172.21.15.40:0/3310080554 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.605+0000 7ff4a759e700 1 --2- 172.21.15.40:0/3310080554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f095a6e0 0x55c5f0969a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.605+0000 7ff4a6d9d700 1 --2- 172.21.15.40:0/3310080554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f07b56b0 0x55c5f09624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.606+0000 7ff4a659c700 1 -- 172.21.15.40:0/3310080554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f095a6e0 msgr2=0x55c5f0969a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.606+0000 7ff4a659c700 1 --2- 172.21.15.40:0/3310080554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f095a6e0 0x55c5f0969a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.606+0000 7ff4a659c700 1 -- 172.21.15.40:0/3310080554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f07b56b0 msgr2=0x55c5f09624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.606+0000 7ff4a659c700 1 --2- 172.21.15.40:0/3310080554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f07b56b0 0x55c5f09624c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.606+0000 7ff4a659c700 1 -- 172.21.15.40:0/3310080554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a8067900 con 0x55c5f0959840 2023-07-18T03:41:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.606+0000 7ff4a759e700 1 --2- 172.21.15.40:0/3310080554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f095a6e0 0x55c5f0969a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:23.612 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.606+0000 7ff4a6d9d700 1 --2- 172.21.15.40:0/3310080554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f07b56b0 0x55c5f09624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.606+0000 7ff4a659c700 1 --2- 172.21.15.40:0/3310080554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f0959840 0x55c5f0962a00 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009810 tx=0x7ff49c09b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362eb4d210087594 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.606+0000 7ff48aff5700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:41:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.606+0000 7ff48aff5700 1 -- 172.21.15.40:0/3310080554 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c0cd370 con 0x55c5f0959840 2023-07-18T03:41:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.606+0000 7ff48aff5700 1 -- 172.21.15.40:0/3310080554 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff49c023c10 con 0x55c5f0959840 2023-07-18T03:41:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.606+0000 7ff4b925eb80 1 -- 172.21.15.40:0/3310080554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5f0804490 con 0x55c5f0959840 2023-07-18T03:41:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.606+0000 7ff48aff5700 1 -- 172.21.15.40:0/3310080554 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff49c0bb1f0 con 0x55c5f0959840 2023-07-18T03:41:23.613 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.606+0000 7ff4b925eb80 10 client.16887 fetch_fsmap learned FSMap version 69 2023-07-18T03:41:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.606+0000 7ff4b925eb80 1 -- 172.21.15.40:0/3310080554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x55c5f06c1c00 con 0x55c5f0959840 2023-07-18T03:41:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.606+0000 7ff48aff5700 1 -- 172.21.15.40:0/3310080554 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5f0804490 con 0x55c5f0959840 2023-07-18T03:41:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.607+0000 7ff48aff5700 1 -- 172.21.15.40:0/3310080554 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 69) v1 ==== 7542+0+0 (secure 0 0 0) 0x7ff49c0a4db0 con 0x55c5f0959840 2023-07-18T03:41:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.607+0000 7ff4b925eb80 10 client.16887 fetch_fsmap finished waiting for FSMap version 69 2023-07-18T03:41:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.607+0000 7ff4b925eb80 10 client.16887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:41:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.607+0000 7ff4b925eb80 20 client.16887 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:23.614 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.607+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/3310080554 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c5f096dd30 0x55c5f0970140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.607+0000 7ff4b925eb80 4 client.16887 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:41:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.607+0000 7ff4b925eb80 1 -- 172.21.15.40:0/3310080554 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5f08a3e50 con 0x55c5f096dd30 2023-07-18T03:41:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.607+0000 7ff4a6d9d700 1 --2- 172.21.15.40:0/3310080554 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c5f096dd30 0x55c5f0970140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.607+0000 7ff48aff5700 10 client.16887 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.607+0000 7ff4a6d9d700 1 --2- 172.21.15.40:0/3310080554 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c5f096dd30 0x55c5f0970140 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54372ad812e0476b server_cookie=2a5d7b2f950f98fc in_seq=0 out_seq=0 2023-07-18T03:41:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.608+0000 7ff48aff5700 1 -- 172.21.15.40:0/3310080554 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c5f08a3e50 con 0x55c5f096dd30 2023-07-18T03:41:23.615 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.608+0000 7ff48aff5700 10 client.16887 handle_command_reply: tid=0 2023-07-18T03:41:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.608+0000 7ff4b925eb80 1 client.16887 shutdown 2023-07-18T03:41:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.608+0000 7ff4b925eb80 1 -- 172.21.15.40:0/3310080554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f0959840 msgr2=0x55c5f0962a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.608+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/3310080554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f0959840 0x55c5f0962a00 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009810 tx=0x7ff49c09b040 comp rx=0 tx=0).stop 2023-07-18T03:41:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.608+0000 7ff4b925eb80 1 -- 172.21.15.40:0/3310080554 shutdown_connections 2023-07-18T03:41:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.608+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/3310080554 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c5f096dd30 0x55c5f0970140 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.608+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/3310080554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f095a6e0 0x55c5f0969a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.616 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.608+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/3310080554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f07b56b0 0x55c5f09624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.608+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/3310080554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f0959840 0x55c5f0962a00 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.608+0000 7ff4b925eb80 1 -- 172.21.15.40:0/3310080554 >> 172.21.15.40:0/3310080554 conn(0x55c5f094ebe0 msgr2=0x55c5f09534e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.608+0000 7ff48aff5700 0 client.16887 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.608+0000 7ff4b925eb80 1 -- 172.21.15.40:0/3310080554 shutdown_connections 2023-07-18T03:41:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.608+0000 7ff4b925eb80 1 -- 172.21.15.40:0/3310080554 wait complete. 2023-07-18T03:41:23.617 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.609+0000 7ff4b925eb80 20 client.16887 trim_cache size 0 max 16384 2023-07-18T03:41:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.623+0000 7ff4b925eb80 1 Processor -- start 2023-07-18T03:41:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.623+0000 7ff4b925eb80 1 -- start start 2023-07-18T03:41:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.623+0000 7ff4b925eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f07b56b0 0x55c5f0974a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff4b925eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f0959840 0x55c5f095d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff4b925eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f095a6e0 0x55c5f095d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff4b925eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f0962030 con 0x55c5f095a6e0 2023-07-18T03:41:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff4b925eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f09547a0 con 0x55c5f07b56b0 2023-07-18T03:41:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff4b925eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f096daf0 con 0x55c5f0959840 2023-07-18T03:41:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff4a759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f095a6e0 0x55c5f095d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff4a759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f095a6e0 0x55c5f095d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33338/0 (socket says 172.21.15.40:33338) 2023-07-18T03:41:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff4a759e700 1 -- 172.21.15.40:0/349189966 learned_addr learned my addr 172.21.15.40:0/349189966 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff4a659c700 1 --2- 172.21.15.40:0/349189966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f0959840 0x55c5f095d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff4a6d9d700 1 --2- 172.21.15.40:0/349189966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f07b56b0 0x55c5f0974a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff4a759e700 1 -- 172.21.15.40:0/349189966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f0959840 msgr2=0x55c5f095d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff4a759e700 1 --2- 172.21.15.40:0/349189966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f0959840 0x55c5f095d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff4a759e700 1 -- 172.21.15.40:0/349189966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f07b56b0 msgr2=0x55c5f0974a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff4a759e700 1 --2- 172.21.15.40:0/349189966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f07b56b0 0x55c5f0974a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff4a759e700 1 -- 172.21.15.40:0/349189966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5f06ff1f0 con 0x55c5f095a6e0 2023-07-18T03:41:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff4a759e700 1 --2- 172.21.15.40:0/349189966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f095a6e0 0x55c5f095d8f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0007af0 tx=0x7ff4a00099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52de5a57bcb6d16 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff48dffb700 1 -- 172.21.15.40:0/349189966 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a000eb60 con 0x55c5f095a6e0 2023-07-18T03:41:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff48dffb700 1 -- 172.21.15.40:0/349189966 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4a001b070 con 0x55c5f095a6e0 2023-07-18T03:41:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.624+0000 7ff48dffb700 1 -- 172.21.15.40:0/349189966 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0017670 con 0x55c5f095a6e0 2023-07-18T03:41:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.625+0000 7ff4b925eb80 1 -- 172.21.15.40:0/349189966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f095a6e0 msgr2=0x55c5f095d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.625+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/349189966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f095a6e0 0x55c5f095d8f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0007af0 tx=0x7ff4a00099e0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.625+0000 7ff4b925eb80 1 -- 172.21.15.40:0/349189966 shutdown_connections 2023-07-18T03:41:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.625+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/349189966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f095a6e0 0x55c5f095d8f0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.625+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/349189966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f07b56b0 0x55c5f0974a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.625+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/349189966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f0959840 0x55c5f095d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.625+0000 7ff4b925eb80 1 -- 172.21.15.40:0/349189966 >> 172.21.15.40:0/349189966 conn(0x55c5f094ebe0 msgr2=0x55c5f0953c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.625+0000 7ff4b925eb80 1 -- 172.21.15.40:0/349189966 shutdown_connections 2023-07-18T03:41:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.625+0000 7ff4b925eb80 1 -- 172.21.15.40:0/349189966 wait complete. 2023-07-18T03:41:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.625+0000 7ff4b925eb80 1 -- start start 2023-07-18T03:41:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.625+0000 7ff4b925eb80 1 Processor -- start 2023-07-18T03:41:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4b925eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f07b56b0 0x55c5f09bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4b925eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f0959840 0x55c5f09c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4b925eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f095a6e0 0x55c5f09c4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4b925eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f09c4d60 con 0x55c5f07b56b0 2023-07-18T03:41:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4b925eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f09c4ed0 con 0x55c5f0959840 2023-07-18T03:41:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4b925eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f09c5090 con 0x55c5f095a6e0 2023-07-18T03:41:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4b925eb80 10 client.0 _ll_register_callbacks cb 0x7ff4980096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4a6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f07b56b0 0x55c5f09bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4a6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f07b56b0 0x55c5f09bfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33340/0 (socket says 172.21.15.40:33340) 2023-07-18T03:41:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4a6d9d700 1 -- 172.21.15.40:0/1073498583 learned_addr learned my addr 172.21.15.40:0/1073498583 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4a759e700 1 --2- 172.21.15.40:0/1073498583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f095a6e0 0x55c5f09c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4a659c700 1 --2- 172.21.15.40:0/1073498583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f0959840 0x55c5f09c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4a6d9d700 1 -- 172.21.15.40:0/1073498583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f095a6e0 msgr2=0x55c5f09c4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4a6d9d700 1 --2- 172.21.15.40:0/1073498583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f095a6e0 0x55c5f09c4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4a6d9d700 1 -- 172.21.15.40:0/1073498583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f0959840 msgr2=0x55c5f09c20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4a6d9d700 1 --2- 172.21.15.40:0/1073498583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f0959840 0x55c5f09c20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4a6d9d700 1 -- 172.21.15.40:0/1073498583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a00077a0 con 0x55c5f07b56b0 2023-07-18T03:41:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4a6d9d700 1 --2- 172.21.15.40:0/1073498583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f07b56b0 0x55c5f09bfb90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff4980097c0 tx=0x7ff4980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3addb7de798c5c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff48aff5700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:41:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff48aff5700 1 -- 172.21.15.40:0/1073498583 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498022040 con 0x55c5f07b56b0 2023-07-18T03:41:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff4b925eb80 1 -- 172.21.15.40:0/1073498583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff49c0a5240 con 0x55c5f07b56b0 2023-07-18T03:41:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.626+0000 7ff48aff5700 1 -- 172.21.15.40:0/1073498583 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7ff498003c60 con 0x55c5f07b56b0 2023-07-18T03:41:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.627+0000 7ff48aff5700 1 -- 172.21.15.40:0/1073498583 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff49802be50 con 0x55c5f07b56b0 2023-07-18T03:41:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.627+0000 7ff4b925eb80 10 client.16893 fetch_fsmap learned FSMap version 69 2023-07-18T03:41:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.627+0000 7ff4b925eb80 1 -- 172.21.15.40:0/1073498583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x55c5f06d7250 con 0x55c5f07b56b0 2023-07-18T03:41:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.627+0000 7ff48aff5700 1 -- 172.21.15.40:0/1073498583 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff49c0a5240 con 0x55c5f07b56b0 2023-07-18T03:41:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.627+0000 7ff48aff5700 1 -- 172.21.15.40:0/1073498583 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 69) v1 ==== 7542+0+0 (secure 0 0 0) 0x7ff49803c020 con 0x55c5f07b56b0 2023-07-18T03:41:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.627+0000 7ff4b925eb80 10 client.16893 fetch_fsmap finished waiting for FSMap version 69 2023-07-18T03:41:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.627+0000 7ff4b925eb80 10 client.16893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:41:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.627+0000 7ff4b925eb80 20 client.16893 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.627+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/1073498583 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c5f09c6b30 0x55c5f09c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.627+0000 7ff4b925eb80 4 client.16893 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:41:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.627+0000 7ff4b925eb80 1 -- 172.21.15.40:0/1073498583 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c5f0865ff0 con 0x55c5f09c6b30 2023-07-18T03:41:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.627+0000 7ff4a659c700 1 --2- 172.21.15.40:0/1073498583 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c5f09c6b30 0x55c5f09c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.628+0000 7ff48aff5700 10 client.16893 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.628+0000 7ff4a659c700 1 --2- 172.21.15.40:0/1073498583 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c5f09c6b30 0x55c5f09c8f40 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a902171a6c6ff7b6 server_cookie=57b0695a1ee8e28f in_seq=0 out_seq=0 2023-07-18T03:41:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.634+0000 7ff48aff5700 1 -- 172.21.15.40:0/1073498583 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c5f0865ff0 con 0x55c5f09c6b30 2023-07-18T03:41:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.634+0000 7ff48aff5700 10 client.16893 handle_command_reply: tid=0 2023-07-18T03:41:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.634+0000 7ff4b925eb80 1 client.16893 shutdown 2023-07-18T03:41:23.633 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.634+0000 7ff4b925eb80 1 -- 172.21.15.40:0/1073498583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f07b56b0 msgr2=0x55c5f09bfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.634+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/1073498583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f07b56b0 0x55c5f09bfb90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff4980097c0 tx=0x7ff4980047c0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.634+0000 7ff4b925eb80 1 -- 172.21.15.40:0/1073498583 shutdown_connections 2023-07-18T03:41:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.634+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/1073498583 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x55c5f09c6b30 0x55c5f09c8f40 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.634+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/1073498583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5f095a6e0 0x55c5f09c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.634+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/1073498583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c5f07b56b0 0x55c5f09bfb90 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.634+0000 7ff48aff5700 0 client.16893 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:23.634 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.634+0000 7ff4b925eb80 1 --2- 172.21.15.40:0/1073498583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5f0959840 0x55c5f09c20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.634+0000 7ff4b925eb80 1 -- 172.21.15.40:0/1073498583 >> 172.21.15.40:0/1073498583 conn(0x55c5f094ebe0 msgr2=0x55c5f095d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.634+0000 7ff4b925eb80 1 -- 172.21.15.40:0/1073498583 shutdown_connections 2023-07-18T03:41:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.634+0000 7ff4b925eb80 1 -- 172.21.15.40:0/1073498583 wait complete. 2023-07-18T03:41:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.634+0000 7ff4b925eb80 20 client.16893 trim_cache size 0 max 16384 2023-07-18T03:41:23.635 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:41:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:41:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:41:23.636 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:41:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.635+0000 7ff4ad0d8700 1 -- 172.21.15.40:0/2923447789 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4900a11d0 msgr2=0x7ff4900a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.635+0000 7ff4ad0d8700 1 --2- 172.21.15.40:0/2923447789 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4900a11d0 0x7ff4900a3680 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff498009c60 tx=0x7ff498009260 comp rx=0 tx=0).stop 2023-07-18T03:41:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.635+0000 7ff4ad0d8700 1 -- 172.21.15.40:0/2923447789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a80ffc90 msgr2=0x7ff4a8194db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.635+0000 7ff4ad0d8700 1 --2- 172.21.15.40:0/2923447789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a80ffc90 0x7ff4a8194db0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff49c000c00 tx=0x7ff49c00b790 comp rx=0 tx=0).stop 2023-07-18T03:41:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.635+0000 7ff4ad0d8700 1 -- 172.21.15.40:0/2923447789 shutdown_connections 2023-07-18T03:41:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.635+0000 7ff4ad0d8700 1 --2- 172.21.15.40:0/2923447789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4a8100640 0x7ff4a8195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.635+0000 7ff4ad0d8700 1 --2- 172.21.15.40:0/2923447789 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7ff4900a11d0 0x7ff4900a3680 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.635+0000 7ff4ad0d8700 1 --2- 172.21.15.40:0/2923447789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4a80fea00 0x7ff4a8194870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.635+0000 7ff4ad0d8700 1 --2- 172.21.15.40:0/2923447789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4a80ffc90 0x7ff4a8194db0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.635+0000 7ff4ad0d8700 1 -- 172.21.15.40:0/2923447789 >> 172.21.15.40:0/2923447789 conn(0x7ff4a80fa000 msgr2=0x7ff4a80fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:23.638 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.635+0000 7ff4ad0d8700 1 -- 172.21.15.40:0/2923447789 shutdown_connections 2023-07-18T03:41:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:23.635+0000 7ff4ad0d8700 1 -- 172.21.15.40:0/2923447789 wait complete. 2023-07-18T03:41:23.899 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:41:24.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.039+0000 7f921466f700 1 -- 172.21.15.40:0/3060171984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f920c100600 msgr2=0x7f920c1029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:24.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.039+0000 7f921466f700 1 --2- 172.21.15.40:0/3060171984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f920c100600 0x7f920c1029e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9208009990 tx=0x7f9208009ca0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.039+0000 7f921466f700 1 -- 172.21.15.40:0/3060171984 shutdown_connections 2023-07-18T03:41:24.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.039+0000 7f921466f700 1 --2- 172.21.15.40:0/3060171984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f920c105840 0x7f920c107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.039+0000 7f921466f700 1 --2- 172.21.15.40:0/3060171984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f920c100600 0x7f920c1029e0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.039+0000 7f921466f700 1 --2- 172.21.15.40:0/3060171984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920c102f20 0x7f920c105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.039+0000 7f921466f700 1 -- 172.21.15.40:0/3060171984 >> 172.21.15.40:0/3060171984 conn(0x7f920c0fa000 msgr2=0x7f920c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:24.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.039+0000 7f921466f700 1 -- 172.21.15.40:0/3060171984 shutdown_connections 2023-07-18T03:41:24.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.039+0000 7f921466f700 1 -- 172.21.15.40:0/3060171984 wait complete. 2023-07-18T03:41:24.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.040+0000 7f921466f700 1 Processor -- start 2023-07-18T03:41:24.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.040+0000 7f921466f700 1 -- start start 2023-07-18T03:41:24.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f921466f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f920c100600 0x7f920c195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f921466f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920c102f20 0x7f920c195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f921466f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f920c105840 0x7f920c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f921466f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c19d3d0 con 0x7f920c105840 2023-07-18T03:41:24.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f921466f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c19d510 con 0x7f920c102f20 2023-07-18T03:41:24.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f921466f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c19d6a0 con 0x7f920c100600 2023-07-18T03:41:24.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f9212c0c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f920c105840 0x7f920c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f921240b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f920c100600 0x7f920c195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f9212c0c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f920c105840 0x7f920c19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33358/0 (socket says 172.21.15.40:33358) 2023-07-18T03:41:24.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f9212c0c700 1 -- 172.21.15.40:0/694700660 learned_addr learned my addr 172.21.15.40:0/694700660 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:24.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f9211c0a700 1 --2- 172.21.15.40:0/694700660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920c102f20 0x7f920c195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f9212c0c700 1 -- 172.21.15.40:0/694700660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f920c100600 msgr2=0x7f920c195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:24.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f9212c0c700 1 --2- 172.21.15.40:0/694700660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f920c100600 0x7f920c195850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f9212c0c700 1 -- 172.21.15.40:0/694700660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920c102f20 msgr2=0x7f920c195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:24.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f9212c0c700 1 --2- 172.21.15.40:0/694700660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920c102f20 0x7f920c195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f9212c0c700 1 -- 172.21.15.40:0/694700660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92080095f0 con 0x7f920c105840 2023-07-18T03:41:24.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f921240b700 1 --2- 172.21.15.40:0/694700660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f920c100600 0x7f920c195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:24.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f9212c0c700 1 --2- 172.21.15.40:0/694700660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f920c105840 0x7f920c19ce00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f920400ef30 tx=0x7f920400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:24.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f91ff7fe700 1 -- 172.21.15.40:0/694700660 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9204018070 con 0x7f920c105840 2023-07-18T03:41:24.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f91ff7fe700 1 -- 172.21.15.40:0/694700660 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f920400f040 con 0x7f920c105840 2023-07-18T03:41:24.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f921466f700 1 -- 172.21.15.40:0/694700660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920c19d920 con 0x7f920c105840 2023-07-18T03:41:24.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.041+0000 7f91ff7fe700 1 -- 172.21.15.40:0/694700660 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920400cd00 con 0x7f920c105840 2023-07-18T03:41:24.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.042+0000 7f921466f700 1 -- 172.21.15.40:0/694700660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920c19dde0 con 0x7f920c105840 2023-07-18T03:41:24.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.044+0000 7f91ff7fe700 1 -- 172.21.15.40:0/694700660 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9204003a00 con 0x7f920c105840 2023-07-18T03:41:24.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.044+0000 7f91ff7fe700 1 --2- 172.21.15.40:0/694700660 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f91f80a1150 0x7f91f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.044+0000 7f921240b700 1 --2- 172.21.15.40:0/694700660 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f91f80a1150 0x7f91f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.044+0000 7f91ff7fe700 1 -- 172.21.15.40:0/694700660 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f92040cc480 con 0x7f920c105840 2023-07-18T03:41:24.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.045+0000 7f921240b700 1 --2- 172.21.15.40:0/694700660 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f91f80a1150 0x7f91f80a3600 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f920800b2e0 tx=0x7f9208005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:24.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.045+0000 7f92207f5b80 1 Processor -- start 2023-07-18T03:41:24.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.045+0000 7f92207f5b80 1 -- start start 2023-07-18T03:41:24.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f92207f5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf274d0 0x557a0bf278e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f92207f5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf27e20 0x557a0bf286e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f92207f5b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf28c70 0x557a0bf290e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f92207f5b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0be2ccb0 con 0x557a0bf28c70 2023-07-18T03:41:24.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f92207f5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0be7fa40 con 0x557a0bf274d0 2023-07-18T03:41:24.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f92207f5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0be78ed0 con 0x557a0bf27e20 2023-07-18T03:41:24.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f9211c0a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf27e20 0x557a0bf286e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f9212c0c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf28c70 0x557a0bf290e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f9211c0a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf27e20 0x557a0bf286e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50624/0 (socket says 172.21.15.40:50624) 2023-07-18T03:41:24.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f9211c0a700 1 -- 172.21.15.40:0/2188779530 learned_addr learned my addr 172.21.15.40:0/2188779530 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:24.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f921240b700 1 --2- 172.21.15.40:0/2188779530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf274d0 0x557a0bf278e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f9212c0c700 1 -- 172.21.15.40:0/2188779530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf27e20 msgr2=0x557a0bf286e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:24.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f9212c0c700 1 --2- 172.21.15.40:0/2188779530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf27e20 0x557a0bf286e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f9212c0c700 1 -- 172.21.15.40:0/2188779530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf274d0 msgr2=0x557a0bf278e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:24.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f9212c0c700 1 --2- 172.21.15.40:0/2188779530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf274d0 0x557a0bf278e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f9212c0c700 1 -- 172.21.15.40:0/2188779530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a0bca6de0 con 0x557a0bf28c70 2023-07-18T03:41:24.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f921240b700 1 --2- 172.21.15.40:0/2188779530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf274d0 0x557a0bf278e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:24.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f9212c0c700 1 --2- 172.21.15.40:0/2188779530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf28c70 0x557a0bf290e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f920400c6f0 tx=0x7f9204095d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746b9398b3a1b4e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:24.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f91fcff9700 1 -- 172.21.15.40:0/2188779530 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92040b7ea0 con 0x557a0bf28c70 2023-07-18T03:41:24.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.046+0000 7f91fcff9700 1 -- 172.21.15.40:0/2188779530 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9204004ad0 con 0x557a0bf28c70 2023-07-18T03:41:24.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.047+0000 7f91fcff9700 1 -- 172.21.15.40:0/2188779530 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9204099b60 con 0x557a0bf28c70 2023-07-18T03:41:24.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.047+0000 7f92207f5b80 1 -- 172.21.15.40:0/2188779530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf28c70 msgr2=0x557a0bf290e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:24.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.047+0000 7f92207f5b80 1 --2- 172.21.15.40:0/2188779530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf28c70 0x557a0bf290e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f920400c6f0 tx=0x7f9204095d70 comp rx=0 tx=0).stop 2023-07-18T03:41:24.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.047+0000 7f92207f5b80 1 -- 172.21.15.40:0/2188779530 shutdown_connections 2023-07-18T03:41:24.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.047+0000 7f92207f5b80 1 --2- 172.21.15.40:0/2188779530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf28c70 0x557a0bf290e0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.047+0000 7f92207f5b80 1 --2- 172.21.15.40:0/2188779530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf274d0 0x557a0bf278e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.047+0000 7f92207f5b80 1 --2- 172.21.15.40:0/2188779530 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf27e20 0x557a0bf286e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.047+0000 7f92207f5b80 1 -- 172.21.15.40:0/2188779530 >> 172.21.15.40:0/2188779530 conn(0x557a0bf1cda0 msgr2=0x557a0bf221d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:24.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.047+0000 7f92207f5b80 1 -- 172.21.15.40:0/2188779530 shutdown_connections 2023-07-18T03:41:24.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.047+0000 7f92207f5b80 1 -- 172.21.15.40:0/2188779530 wait complete. 2023-07-18T03:41:24.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.048+0000 7f92207f5b80 1 -- start start 2023-07-18T03:41:24.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.048+0000 7f92207f5b80 1 Processor -- start 2023-07-18T03:41:24.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.048+0000 7f92207f5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf274d0 0x557a0bf311b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.048+0000 7f92207f5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf27e20 0x557a0bf316f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.048+0000 7f92207f5b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf28c70 0x557a0bf38730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.048+0000 7f92207f5b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0be18ac0 con 0x557a0bf28c70 2023-07-18T03:41:24.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.048+0000 7f92207f5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0be187c0 con 0x557a0bf27e20 2023-07-18T03:41:24.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.048+0000 7f92207f5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0bcfc920 con 0x557a0bf274d0 2023-07-18T03:41:24.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.048+0000 7f92207f5b80 10 client.0 _ll_register_callbacks cb 0x557a0be16760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:24.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.048+0000 7f9212c0c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf28c70 0x557a0bf38730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.048+0000 7f921240b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf274d0 0x557a0bf311b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.048+0000 7f921240b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf274d0 0x557a0bf311b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50628/0 (socket says 172.21.15.40:50628) 2023-07-18T03:41:24.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.048+0000 7f921240b700 1 -- 172.21.15.40:0/3056921880 learned_addr learned my addr 172.21.15.40:0/3056921880 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:24.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.048+0000 7f9212c0c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf28c70 0x557a0bf38730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33372/0 (socket says 172.21.15.40:33372) 2023-07-18T03:41:24.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.048+0000 7f9211c0a700 1 --2- 172.21.15.40:0/3056921880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf27e20 0x557a0bf316f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.048+0000 7f9212c0c700 1 -- 172.21.15.40:0/3056921880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf274d0 msgr2=0x557a0bf311b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:24.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.048+0000 7f9212c0c700 1 --2- 172.21.15.40:0/3056921880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf274d0 0x557a0bf311b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.049+0000 7f9212c0c700 1 -- 172.21.15.40:0/3056921880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf27e20 msgr2=0x557a0bf316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:24.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.049+0000 7f9212c0c700 1 --2- 172.21.15.40:0/3056921880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf27e20 0x557a0bf316f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.049+0000 7f9212c0c700 1 -- 172.21.15.40:0/3056921880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920c19dde0 con 0x557a0bf28c70 2023-07-18T03:41:24.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.049+0000 7f921240b700 1 --2- 172.21.15.40:0/3056921880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf274d0 0x557a0bf311b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:24.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.049+0000 7f9212c0c700 1 --2- 172.21.15.40:0/3056921880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf28c70 0x557a0bf38730 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f92040993d0 tx=0x7f9204095640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e080cb62cec74b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:24.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.049+0000 7f91f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:41:24.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.049+0000 7f91f5ffb700 1 -- 172.21.15.40:0/3056921880 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92040b7ea0 con 0x557a0bf28c70 2023-07-18T03:41:24.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.049+0000 7f91f5ffb700 1 -- 172.21.15.40:0/3056921880 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f92040a1cc0 con 0x557a0bf28c70 2023-07-18T03:41:24.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.049+0000 7f92207f5b80 1 -- 172.21.15.40:0/3056921880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a0bdd24f0 con 0x557a0bf28c70 2023-07-18T03:41:24.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.049+0000 7f91f5ffb700 1 -- 172.21.15.40:0/3056921880 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92040b90d0 con 0x557a0bf28c70 2023-07-18T03:41:24.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.049+0000 7f92207f5b80 10 client.16905 fetch_fsmap learned FSMap version 69 2023-07-18T03:41:24.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.049+0000 7f92207f5b80 1 -- 172.21.15.40:0/3056921880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x557a0bc8fc60 con 0x557a0bf28c70 2023-07-18T03:41:24.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.049+0000 7f91f5ffb700 1 -- 172.21.15.40:0/3056921880 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a0bdd24f0 con 0x557a0bf28c70 2023-07-18T03:41:24.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.049+0000 7f91f5ffb700 1 -- 172.21.15.40:0/3056921880 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 69) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f92040b4070 con 0x557a0bf28c70 2023-07-18T03:41:24.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.049+0000 7f92207f5b80 10 client.16905 fetch_fsmap finished waiting for FSMap version 69 2023-07-18T03:41:24.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.049+0000 7f92207f5b80 10 client.16905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:41:24.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.049+0000 7f92207f5b80 20 client.16905 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:24.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.050+0000 7f92207f5b80 1 --2- 172.21.15.40:0/3056921880 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557a0bf3ca60 0x557a0bf3ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.050+0000 7f92207f5b80 4 client.16905 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:41:24.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.050+0000 7f92207f5b80 1 -- 172.21.15.40:0/3056921880 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a0be71eb0 con 0x557a0bf3ca60 2023-07-18T03:41:24.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.050+0000 7f921240b700 1 --2- 172.21.15.40:0/3056921880 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557a0bf3ca60 0x557a0bf3ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.050+0000 7f91f5ffb700 10 client.16905 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:24.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.050+0000 7f921240b700 1 --2- 172.21.15.40:0/3056921880 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557a0bf3ca60 0x557a0bf3ee70 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f846651eca1be8c4 server_cookie=ae59429c92e616c4 in_seq=0 out_seq=0 2023-07-18T03:41:24.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.051+0000 7f91f5ffb700 1 -- 172.21.15.40:0/3056921880 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557a0be71eb0 con 0x557a0bf3ca60 2023-07-18T03:41:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.051+0000 7f91f5ffb700 10 client.16905 handle_command_reply: tid=0 2023-07-18T03:41:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.051+0000 7f92207f5b80 1 client.16905 shutdown 2023-07-18T03:41:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.051+0000 7f92207f5b80 1 -- 172.21.15.40:0/3056921880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf28c70 msgr2=0x557a0bf38730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.051+0000 7f92207f5b80 1 --2- 172.21.15.40:0/3056921880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf28c70 0x557a0bf38730 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f92040993d0 tx=0x7f9204095640 comp rx=0 tx=0).stop 2023-07-18T03:41:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.051+0000 7f92207f5b80 1 -- 172.21.15.40:0/3056921880 shutdown_connections 2023-07-18T03:41:24.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.051+0000 7f92207f5b80 1 --2- 172.21.15.40:0/3056921880 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557a0bf3ca60 0x557a0bf3ee70 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.051+0000 7f92207f5b80 1 --2- 172.21.15.40:0/3056921880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf28c70 0x557a0bf38730 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.051+0000 7f91f5ffb700 0 client.16905 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:24.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.051+0000 7f92207f5b80 1 --2- 172.21.15.40:0/3056921880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf274d0 0x557a0bf311b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.051+0000 7f92207f5b80 1 --2- 172.21.15.40:0/3056921880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf27e20 0x557a0bf316f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.051+0000 7f92207f5b80 1 -- 172.21.15.40:0/3056921880 >> 172.21.15.40:0/3056921880 conn(0x557a0bf1cda0 msgr2=0x557a0bf218b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:24.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.051+0000 7f92207f5b80 1 -- 172.21.15.40:0/3056921880 shutdown_connections 2023-07-18T03:41:24.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.051+0000 7f92207f5b80 1 -- 172.21.15.40:0/3056921880 wait complete. 2023-07-18T03:41:24.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.051+0000 7f92207f5b80 20 client.16905 trim_cache size 0 max 16384 2023-07-18T03:41:24.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.069+0000 7f92207f5b80 1 Processor -- start 2023-07-18T03:41:24.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.069+0000 7f92207f5b80 1 -- start start 2023-07-18T03:41:24.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.069+0000 7f92207f5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf1cda0 0x557a0bf39020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.069+0000 7f92207f5b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf274d0 0x557a0bf386f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.069+0000 7f92207f5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf27e20 0x557a0bf2bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.069+0000 7f92207f5b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0bf1c9c0 con 0x557a0bf274d0 2023-07-18T03:41:24.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.069+0000 7f92207f5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0bf30660 con 0x557a0bf1cda0 2023-07-18T03:41:24.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.069+0000 7f92207f5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0bf30d20 con 0x557a0bf27e20 2023-07-18T03:41:24.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.070+0000 7f9211c0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf274d0 0x557a0bf386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.070+0000 7f9211c0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf274d0 0x557a0bf386f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33378/0 (socket says 172.21.15.40:33378) 2023-07-18T03:41:24.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.070+0000 7f9211c0a700 1 -- 172.21.15.40:0/1612305288 learned_addr learned my addr 172.21.15.40:0/1612305288 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:24.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.070+0000 7f9212c0c700 1 --2- 172.21.15.40:0/1612305288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf27e20 0x557a0bf2bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.070+0000 7f921240b700 1 --2- 172.21.15.40:0/1612305288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf1cda0 0x557a0bf39020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.070+0000 7f9211c0a700 1 -- 172.21.15.40:0/1612305288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf27e20 msgr2=0x557a0bf2bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:24.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.070+0000 7f9211c0a700 1 --2- 172.21.15.40:0/1612305288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf27e20 0x557a0bf2bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.070+0000 7f9211c0a700 1 -- 172.21.15.40:0/1612305288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf1cda0 msgr2=0x557a0bf39020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:24.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.070+0000 7f9211c0a700 1 --2- 172.21.15.40:0/1612305288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf1cda0 0x557a0bf39020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.070+0000 7f9211c0a700 1 -- 172.21.15.40:0/1612305288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a0bccd250 con 0x557a0bf274d0 2023-07-18T03:41:24.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.070+0000 7f9211c0a700 1 --2- 172.21.15.40:0/1612305288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf274d0 0x557a0bf386f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9200009ee0 tx=0x7f9200010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f784d78006b09b07 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:24.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.070+0000 7f91fd7fa700 1 -- 172.21.15.40:0/1612305288 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200010d50 con 0x557a0bf274d0 2023-07-18T03:41:24.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.070+0000 7f91fd7fa700 1 -- 172.21.15.40:0/1612305288 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9200018040 con 0x557a0bf274d0 2023-07-18T03:41:24.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.070+0000 7f91fd7fa700 1 -- 172.21.15.40:0/1612305288 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920001c3f0 con 0x557a0bf274d0 2023-07-18T03:41:24.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.071+0000 7f92207f5b80 1 -- 172.21.15.40:0/1612305288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf274d0 msgr2=0x557a0bf386f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:24.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.071+0000 7f92207f5b80 1 --2- 172.21.15.40:0/1612305288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf274d0 0x557a0bf386f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9200009ee0 tx=0x7f9200010710 comp rx=0 tx=0).stop 2023-07-18T03:41:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.071+0000 7f92207f5b80 1 -- 172.21.15.40:0/1612305288 shutdown_connections 2023-07-18T03:41:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.071+0000 7f92207f5b80 1 --2- 172.21.15.40:0/1612305288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf27e20 0x557a0bf2bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.071+0000 7f92207f5b80 1 --2- 172.21.15.40:0/1612305288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf1cda0 0x557a0bf39020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.071+0000 7f92207f5b80 1 --2- 172.21.15.40:0/1612305288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf274d0 0x557a0bf386f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.071+0000 7f92207f5b80 1 -- 172.21.15.40:0/1612305288 >> 172.21.15.40:0/1612305288 conn(0x557a0bf1e600 msgr2=0x557a0bf2fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.071+0000 7f92207f5b80 1 -- 172.21.15.40:0/1612305288 shutdown_connections 2023-07-18T03:41:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.071+0000 7f92207f5b80 1 -- 172.21.15.40:0/1612305288 wait complete. 2023-07-18T03:41:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.071+0000 7f92207f5b80 1 -- start start 2023-07-18T03:41:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.071+0000 7f92207f5b80 1 Processor -- start 2023-07-18T03:41:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f92207f5b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf1cda0 0x557a0bf38410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f92207f5b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf274d0 0x557a0bf34eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f92207f5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf27e20 0x557a0bf353f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f92207f5b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0bf35930 con 0x557a0bf1cda0 2023-07-18T03:41:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f92207f5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0bf35aa0 con 0x557a0bf27e20 2023-07-18T03:41:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f92207f5b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0bf90910 con 0x557a0bf274d0 2023-07-18T03:41:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f92207f5b80 10 client.0 _ll_register_callbacks cb 0x7f9200009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f921240b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf1cda0 0x557a0bf38410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f921240b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf1cda0 0x557a0bf38410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33382/0 (socket says 172.21.15.40:33382) 2023-07-18T03:41:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f921240b700 1 -- 172.21.15.40:0/225024494 learned_addr learned my addr 172.21.15.40:0/225024494 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f9211c0a700 1 --2- 172.21.15.40:0/225024494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf274d0 0x557a0bf34eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f9212c0c700 1 --2- 172.21.15.40:0/225024494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf27e20 0x557a0bf353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f921240b700 1 -- 172.21.15.40:0/225024494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf274d0 msgr2=0x557a0bf34eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f921240b700 1 --2- 172.21.15.40:0/225024494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf274d0 0x557a0bf34eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f921240b700 1 -- 172.21.15.40:0/225024494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf27e20 msgr2=0x557a0bf353f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f921240b700 1 --2- 172.21.15.40:0/225024494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf27e20 0x557a0bf353f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f921240b700 1 -- 172.21.15.40:0/225024494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9200009b90 con 0x557a0bf1cda0 2023-07-18T03:41:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f921240b700 1 --2- 172.21.15.40:0/225024494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf1cda0 0x557a0bf38410 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x557a0bf3f750 tx=0x7f9208009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f498aff7d63292b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f91f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:41:24.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f91f5ffb700 1 -- 172.21.15.40:0/225024494 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208035ea0 con 0x557a0bf1cda0 2023-07-18T03:41:24.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f91f5ffb700 1 -- 172.21.15.40:0/225024494 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f92080350e0 con 0x557a0bf1cda0 2023-07-18T03:41:24.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.072+0000 7f92207f5b80 1 -- 172.21.15.40:0/225024494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9204004ad0 con 0x557a0bf1cda0 2023-07-18T03:41:24.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.073+0000 7f91f5ffb700 1 -- 172.21.15.40:0/225024494 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9208005450 con 0x557a0bf1cda0 2023-07-18T03:41:24.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.073+0000 7f92207f5b80 10 client.16911 fetch_fsmap learned FSMap version 69 2023-07-18T03:41:24.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.073+0000 7f92207f5b80 1 -- 172.21.15.40:0/225024494 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x557a0bca52b0 con 0x557a0bf1cda0 2023-07-18T03:41:24.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.073+0000 7f91f5ffb700 1 -- 172.21.15.40:0/225024494 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9204004ad0 con 0x557a0bf1cda0 2023-07-18T03:41:24.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.073+0000 7f91f5ffb700 1 -- 172.21.15.40:0/225024494 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 69) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f9208004ca0 con 0x557a0bf1cda0 2023-07-18T03:41:24.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.073+0000 7f92207f5b80 10 client.16911 fetch_fsmap finished waiting for FSMap version 69 2023-07-18T03:41:24.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.073+0000 7f92207f5b80 10 client.16911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:41:24.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.073+0000 7f92207f5b80 20 client.16911 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.073+0000 7f92207f5b80 1 --2- 172.21.15.40:0/225024494 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557a0bf28c70 0x557a0bf94420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.073+0000 7f92207f5b80 4 client.16911 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:41:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.073+0000 7f92207f5b80 1 -- 172.21.15.40:0/225024494 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557a0be34050 con 0x557a0bf28c70 2023-07-18T03:41:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.073+0000 7f9211c0a700 1 --2- 172.21.15.40:0/225024494 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557a0bf28c70 0x557a0bf94420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.074+0000 7f91f5ffb700 10 client.16911 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.074+0000 7f9211c0a700 1 --2- 172.21.15.40:0/225024494 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557a0bf28c70 0x557a0bf94420 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccd52fb94d0f1ab8 server_cookie=d6e5f10bc8d43e64 in_seq=0 out_seq=0 2023-07-18T03:41:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.074+0000 7f91f5ffb700 1 -- 172.21.15.40:0/225024494 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557a0be34050 con 0x557a0bf28c70 2023-07-18T03:41:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.074+0000 7f91f5ffb700 10 client.16911 handle_command_reply: tid=0 2023-07-18T03:41:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.074+0000 7f92207f5b80 1 client.16911 shutdown 2023-07-18T03:41:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.074+0000 7f92207f5b80 1 -- 172.21.15.40:0/225024494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf1cda0 msgr2=0x557a0bf38410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.074+0000 7f92207f5b80 1 --2- 172.21.15.40:0/225024494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf1cda0 0x557a0bf38410 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x557a0bf3f750 tx=0x7f9208009ca0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.074+0000 7f92207f5b80 1 -- 172.21.15.40:0/225024494 shutdown_connections 2023-07-18T03:41:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.074+0000 7f92207f5b80 1 --2- 172.21.15.40:0/225024494 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557a0bf28c70 0x557a0bf94420 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.074+0000 7f92207f5b80 1 --2- 172.21.15.40:0/225024494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a0bf27e20 0x557a0bf353f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.074+0000 7f91f5ffb700 0 client.16911 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.074+0000 7f92207f5b80 1 --2- 172.21.15.40:0/225024494 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557a0bf1cda0 0x557a0bf38410 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.074+0000 7f92207f5b80 1 --2- 172.21.15.40:0/225024494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a0bf274d0 0x557a0bf34eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.074+0000 7f92207f5b80 1 -- 172.21.15.40:0/225024494 >> 172.21.15.40:0/225024494 conn(0x557a0bf1e600 msgr2=0x557a0bf39020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.074+0000 7f92207f5b80 1 -- 172.21.15.40:0/225024494 shutdown_connections 2023-07-18T03:41:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.074+0000 7f92207f5b80 1 -- 172.21.15.40:0/225024494 wait complete. 2023-07-18T03:41:24.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.074+0000 7f92207f5b80 20 client.16911 trim_cache size 0 max 16384 2023-07-18T03:41:24.080 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:41:24.080 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-18T03:41:24.080 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": { 2023-07-18T03:41:24.080 INFO:teuthology.orchestra.run.smithi040.stdout: "0007b3bc-541e-4967-8d88-ee78c77c313f": { 2023-07-18T03:41:24.080 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/", 2023-07-18T03:41:24.080 INFO:teuthology.orchestra.run.smithi040.stdout: "tag": "0007b3bc-541e-4967-8d88-ee78c77c313f", 2023-07-18T03:41:24.081 INFO:teuthology.orchestra.run.smithi040.stdout: "options": "recursive,force" 2023-07-18T03:41:24.081 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:41:24.081 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:41:24.081 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:41:24.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.075+0000 7f921466f700 1 -- 172.21.15.40:0/694700660 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f91f80a1150 msgr2=0x7f91f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:24.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.075+0000 7f921466f700 1 --2- 172.21.15.40:0/694700660 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f91f80a1150 0x7f91f80a3600 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f920800b2e0 tx=0x7f9208005fb0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.075+0000 7f921466f700 1 -- 172.21.15.40:0/694700660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f920c105840 msgr2=0x7f920c19ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:24.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.075+0000 7f921466f700 1 --2- 172.21.15.40:0/694700660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f920c105840 0x7f920c19ce00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f920400ef30 tx=0x7f920400c3a0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.075+0000 7f921466f700 1 -- 172.21.15.40:0/694700660 shutdown_connections 2023-07-18T03:41:24.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.075+0000 7f921466f700 1 --2- 172.21.15.40:0/694700660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f920c105840 0x7f920c19ce00 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.075+0000 7f921466f700 1 --2- 172.21.15.40:0/694700660 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f91f80a1150 0x7f91f80a3600 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.075+0000 7f921466f700 1 --2- 172.21.15.40:0/694700660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f920c100600 0x7f920c195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.075+0000 7f921466f700 1 --2- 172.21.15.40:0/694700660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f920c102f20 0x7f920c195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:24.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.075+0000 7f921466f700 1 -- 172.21.15.40:0/694700660 >> 172.21.15.40:0/694700660 conn(0x7f920c0fa000 msgr2=0x7f920c101320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:24.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.075+0000 7f921466f700 1 -- 172.21.15.40:0/694700660 shutdown_connections 2023-07-18T03:41:24.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:24.075+0000 7f921466f700 1 -- 172.21.15.40:0/694700660 wait complete. 2023-07-18T03:41:24.084 INFO:tasks.cephfs.filesystem:scrub status for tag:0007b3bc-541e-4967-8d88-ee78c77c313f - {'path': '/', 'tag': '0007b3bc-541e-4967-8d88-ee78c77c313f', 'options': 'recursive,force'} 2023-07-18T03:41:24.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:24 smithi134 ceph-mon[98216]: scrub summary: idle+waiting paths [/] 2023-07-18T03:41:24.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:24 smithi040 ceph-mon[96439]: scrub summary: idle+waiting paths [/] 2023-07-18T03:41:24.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:24 smithi064 ceph-mon[98059]: scrub summary: idle+waiting paths [/] 2023-07-18T03:41:25.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:25 smithi134 ceph-mon[98216]: pgmap v610: 97 pgs: 97 active+clean; 945 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 389 op/s 2023-07-18T03:41:25.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:25 smithi040 ceph-mon[96439]: pgmap v610: 97 pgs: 97 active+clean; 945 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 389 op/s 2023-07-18T03:41:25.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:25 smithi064 ceph-mon[98059]: pgmap v610: 97 pgs: 97 active+clean; 945 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 389 op/s 2023-07-18T03:41:27.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:27 smithi040 ceph-mon[96439]: pgmap v611: 97 pgs: 97 active+clean; 929 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 435 op/s 2023-07-18T03:41:27.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:27 smithi064 ceph-mon[98059]: pgmap v611: 97 pgs: 97 active+clean; 929 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 435 op/s 2023-07-18T03:41:27.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:27 smithi134 ceph-mon[98216]: pgmap v611: 97 pgs: 97 active+clean; 929 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 435 op/s 2023-07-18T03:41:29.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:29 smithi064 ceph-mon[98059]: pgmap v612: 97 pgs: 97 active+clean; 901 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 456 op/s 2023-07-18T03:41:29.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:29 smithi134 ceph-mon[98216]: pgmap v612: 97 pgs: 97 active+clean; 901 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 456 op/s 2023-07-18T03:41:29.827 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:29 smithi040 ceph-mon[96439]: pgmap v612: 97 pgs: 97 active+clean; 901 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 456 op/s 2023-07-18T03:41:31.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:31 smithi064 ceph-mon[98059]: pgmap v613: 97 pgs: 97 active+clean; 898 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 397 op/s 2023-07-18T03:41:31.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:31 smithi134 ceph-mon[98216]: pgmap v613: 97 pgs: 97 active+clean; 898 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 397 op/s 2023-07-18T03:41:31.827 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:31 smithi040 ceph-mon[96439]: pgmap v613: 97 pgs: 97 active+clean; 898 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 397 op/s 2023-07-18T03:41:33.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:32 smithi134 ceph-mon[98216]: pgmap v614: 97 pgs: 97 active+clean; 875 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 497 op/s 2023-07-18T03:41:33.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:32 smithi040 ceph-mon[96439]: pgmap v614: 97 pgs: 97 active+clean; 875 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 497 op/s 2023-07-18T03:41:33.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:32 smithi064 ceph-mon[98059]: pgmap v614: 97 pgs: 97 active+clean; 875 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 497 op/s 2023-07-18T03:41:35.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:35 smithi040 ceph-mon[96439]: pgmap v615: 97 pgs: 97 active+clean; 854 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 422 op/s 2023-07-18T03:41:35.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:35 smithi064 ceph-mon[98059]: pgmap v615: 97 pgs: 97 active+clean; 854 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 422 op/s 2023-07-18T03:41:35.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:35 smithi134 ceph-mon[98216]: pgmap v615: 97 pgs: 97 active+clean; 854 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 422 op/s 2023-07-18T03:41:36.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:36 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:41:36.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:36 smithi064 ceph-mon[98059]: scrub summary: idle 2023-07-18T03:41:36.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:36 smithi064 ceph-mon[98059]: scrub summary: idle 2023-07-18T03:41:36.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:36 smithi064 ceph-mon[98059]: scrub summary: idle 2023-07-18T03:41:36.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:36 smithi064 ceph-mon[98059]: scrub summary: idle 2023-07-18T03:41:36.600 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:36 smithi064 ceph-mon[98059]: scrub summary: idle 2023-07-18T03:41:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.746+0000 7f33568e8700 1 -- 172.21.15.40:0/2832117553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33500ff610 msgr2=0x7f3350106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.746+0000 7f33568e8700 1 --2- 172.21.15.40:0/2832117553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33500ff610 0x7f3350106ec0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f334800b0a0 tx=0x7f334800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.746+0000 7f33568e8700 1 -- 172.21.15.40:0/2832117553 shutdown_connections 2023-07-18T03:41:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.746+0000 7f33568e8700 1 --2- 172.21.15.40:0/2832117553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33500ff610 0x7f3350106ec0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.746+0000 7f33568e8700 1 --2- 172.21.15.40:0/2832117553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3350100010 0x7f3350100420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.746+0000 7f33568e8700 1 --2- 172.21.15.40:0/2832117553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f33500fec60 0x7f33500ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.746+0000 7f33568e8700 1 -- 172.21.15.40:0/2832117553 >> 172.21.15.40:0/2832117553 conn(0x7f33500f9780 msgr2=0x7f33500fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.746+0000 7f33568e8700 1 -- 172.21.15.40:0/2832117553 shutdown_connections 2023-07-18T03:41:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.746+0000 7f33568e8700 1 -- 172.21.15.40:0/2832117553 wait complete. 2023-07-18T03:41:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.747+0000 7f33568e8700 1 Processor -- start 2023-07-18T03:41:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.747+0000 7f33568e8700 1 -- start start 2023-07-18T03:41:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.747+0000 7f33568e8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33500fec60 0x7f3350198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.747+0000 7f33568e8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33500ff610 0x7f335019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.747+0000 7f33568e8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3350100010 0x7f335019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.747+0000 7f33568e8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335019d5f0 con 0x7f33500fec60 2023-07-18T03:41:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.747+0000 7f33568e8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335019d730 con 0x7f3350100010 2023-07-18T03:41:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.747+0000 7f33568e8700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335019d8f0 con 0x7f33500ff610 2023-07-18T03:41:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.747+0000 7f334ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33500fec60 0x7f3350198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.747+0000 7f334ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33500fec60 0x7f3350198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33396/0 (socket says 172.21.15.40:33396) 2023-07-18T03:41:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.747+0000 7f334ffff700 1 -- 172.21.15.40:0/1802891641 learned_addr learned my addr 172.21.15.40:0/1802891641 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.747+0000 7f334f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33500ff610 0x7f335019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.748+0000 7f3354e85700 1 --2- 172.21.15.40:0/1802891641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3350100010 0x7f335019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.748+0000 7f334ffff700 1 -- 172.21.15.40:0/1802891641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33500ff610 msgr2=0x7f335019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.748+0000 7f334ffff700 1 --2- 172.21.15.40:0/1802891641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33500ff610 0x7f335019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.748+0000 7f334ffff700 1 -- 172.21.15.40:0/1802891641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3350100010 msgr2=0x7f335019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.748+0000 7f334ffff700 1 --2- 172.21.15.40:0/1802891641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3350100010 0x7f335019d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.748+0000 7f334ffff700 1 -- 172.21.15.40:0/1802891641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3348009d00 con 0x7f33500fec60 2023-07-18T03:41:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.748+0000 7f3354e85700 1 --2- 172.21.15.40:0/1802891641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3350100010 0x7f335019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.748+0000 7f334ffff700 1 --2- 172.21.15.40:0/1802891641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33500fec60 0x7f3350198580 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f334000e970 tx=0x7f334000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.748+0000 7f334d7fa700 1 -- 172.21.15.40:0/1802891641 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3340009790 con 0x7f33500fec60 2023-07-18T03:41:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.748+0000 7f334d7fa700 1 -- 172.21.15.40:0/1802891641 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f334000eec0 con 0x7f33500fec60 2023-07-18T03:41:36.748 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.748+0000 7f334d7fa700 1 -- 172.21.15.40:0/1802891641 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3340003cf0 con 0x7f33500fec60 2023-07-18T03:41:36.748 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.748+0000 7f33568e8700 1 -- 172.21.15.40:0/1802891641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f335019dbd0 con 0x7f33500fec60 2023-07-18T03:41:36.748 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.748+0000 7f33568e8700 1 -- 172.21.15.40:0/1802891641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335019e090 con 0x7f33500fec60 2023-07-18T03:41:36.752 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.751+0000 7f334d7fa700 1 -- 172.21.15.40:0/1802891641 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f334000ccd0 con 0x7f33500fec60 2023-07-18T03:41:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.751+0000 7f334d7fa700 1 --2- 172.21.15.40:0/1802891641 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33380a1150 0x7f33380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.752+0000 7f334f7fe700 1 --2- 172.21.15.40:0/1802891641 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33380a1150 0x7f33380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.752+0000 7f334d7fa700 1 -- 172.21.15.40:0/1802891641 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3340014070 con 0x7f33500fec60 2023-07-18T03:41:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.752+0000 7f334f7fe700 1 --2- 172.21.15.40:0/1802891641 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33380a1150 0x7f33380a3600 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f33500fe020 tx=0x7f334400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.752+0000 7f3362a6eb80 1 Processor -- start 2023-07-18T03:41:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.752+0000 7f3362a6eb80 1 -- start start 2023-07-18T03:41:36.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.752+0000 7f3362a6eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e662b6b0 0x5609e67cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f3362a6eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e67cf840 0x5609e67d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f3362a6eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e67d06e0 0x5609e67d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f3362a6eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e66d4c50 con 0x5609e662b6b0 2023-07-18T03:41:36.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f3362a6eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e67279e0 con 0x5609e67d06e0 2023-07-18T03:41:36.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f3362a6eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e6720e70 con 0x5609e67cf840 2023-07-18T03:41:36.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f334ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e662b6b0 0x5609e67cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.754 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f334ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e662b6b0 0x5609e67cf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33404/0 (socket says 172.21.15.40:33404) 2023-07-18T03:41:36.755 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f334ffff700 1 -- 172.21.15.40:0/2235456711 learned_addr learned my addr 172.21.15.40:0/2235456711 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:36.755 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f334f7fe700 1 --2- 172.21.15.40:0/2235456711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e67cf840 0x5609e67d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.755 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f3354e85700 1 --2- 172.21.15.40:0/2235456711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e67d06e0 0x5609e67d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.755 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f334ffff700 1 -- 172.21.15.40:0/2235456711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e67cf840 msgr2=0x5609e67d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:36.755 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f334ffff700 1 --2- 172.21.15.40:0/2235456711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e67cf840 0x5609e67d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.755 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f334ffff700 1 -- 172.21.15.40:0/2235456711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e67d06e0 msgr2=0x5609e67d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f334ffff700 1 --2- 172.21.15.40:0/2235456711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e67d06e0 0x5609e67d0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f334ffff700 1 -- 172.21.15.40:0/2235456711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609e654ed80 con 0x5609e662b6b0 2023-07-18T03:41:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f334ffff700 1 --2- 172.21.15.40:0/2235456711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e662b6b0 0x5609e67cf300 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f33400969e0 tx=0x7f33400998d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5984633c9f0e490e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f33367fc700 1 -- 172.21.15.40:0/2235456711 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334009dfa0 con 0x5609e662b6b0 2023-07-18T03:41:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f33367fc700 1 -- 172.21.15.40:0/2235456711 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33400a23c0 con 0x5609e662b6b0 2023-07-18T03:41:36.756 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.753+0000 7f33367fc700 1 -- 172.21.15.40:0/2235456711 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33400996c0 con 0x5609e662b6b0 2023-07-18T03:41:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.754+0000 7f3362a6eb80 1 -- 172.21.15.40:0/2235456711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e662b6b0 msgr2=0x5609e67cf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.754+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/2235456711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e662b6b0 0x5609e67cf300 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f33400969e0 tx=0x7f33400998d0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.754+0000 7f3362a6eb80 1 -- 172.21.15.40:0/2235456711 shutdown_connections 2023-07-18T03:41:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.754+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/2235456711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e67d06e0 0x5609e67d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.754+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/2235456711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e662b6b0 0x5609e67cf300 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.754+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/2235456711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e67cf840 0x5609e67d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.754+0000 7f3362a6eb80 1 -- 172.21.15.40:0/2235456711 >> 172.21.15.40:0/2235456711 conn(0x5609e67c4be0 msgr2=0x5609e67c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.754+0000 7f3362a6eb80 1 -- 172.21.15.40:0/2235456711 shutdown_connections 2023-07-18T03:41:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.754+0000 7f3362a6eb80 1 -- 172.21.15.40:0/2235456711 wait complete. 2023-07-18T03:41:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.754+0000 7f3362a6eb80 1 -- start start 2023-07-18T03:41:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f3362a6eb80 1 Processor -- start 2023-07-18T03:41:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f3362a6eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e662b6b0 0x5609e67d8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f3362a6eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e67cf840 0x5609e67d8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f3362a6eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67d06e0 0x5609e67dfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f3362a6eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e66c0a60 con 0x5609e67d06e0 2023-07-18T03:41:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f3362a6eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e66c0760 con 0x5609e67cf840 2023-07-18T03:41:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f3362a6eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e65a48c0 con 0x5609e662b6b0 2023-07-18T03:41:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f3362a6eb80 10 client.0 _ll_register_callbacks cb 0x5609e66be700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f3354e85700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67d06e0 0x5609e67dfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f334ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e662b6b0 0x5609e67d8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f3354e85700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67d06e0 0x5609e67dfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33414/0 (socket says 172.21.15.40:33414) 2023-07-18T03:41:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f3354e85700 1 -- 172.21.15.40:0/1792230471 learned_addr learned my addr 172.21.15.40:0/1792230471 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f334f7fe700 1 --2- 172.21.15.40:0/1792230471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e67cf840 0x5609e67d8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f334f7fe700 1 -- 172.21.15.40:0/1792230471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e662b6b0 msgr2=0x5609e67d8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f334f7fe700 1 --2- 172.21.15.40:0/1792230471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e662b6b0 0x5609e67d8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f334f7fe700 1 -- 172.21.15.40:0/1792230471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67d06e0 msgr2=0x5609e67dfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f334f7fe700 1 --2- 172.21.15.40:0/1792230471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67d06e0 0x5609e67dfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f334f7fe700 1 -- 172.21.15.40:0/1792230471 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335019e090 con 0x5609e67cf840 2023-07-18T03:41:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.755+0000 7f3354e85700 1 --2- 172.21.15.40:0/1792230471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67d06e0 0x5609e67dfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.756+0000 7f334f7fe700 1 --2- 172.21.15.40:0/1792230471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e67cf840 0x5609e67d8dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f334400f750 tx=0x7f33440047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f539bf496fb3eac server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.756+0000 7f332f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:41:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.756+0000 7f332f7fe700 1 -- 172.21.15.40:0/1792230471 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344010840 con 0x5609e67cf840 2023-07-18T03:41:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.756+0000 7f332f7fe700 1 -- 172.21.15.40:0/1792230471 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f33440109a0 con 0x5609e67cf840 2023-07-18T03:41:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.756+0000 7f3362a6eb80 1 -- 172.21.15.40:0/1792230471 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609e667a490 con 0x5609e67cf840 2023-07-18T03:41:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.756+0000 7f332f7fe700 1 -- 172.21.15.40:0/1792230471 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f334400fec0 con 0x5609e67cf840 2023-07-18T03:41:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.756+0000 7f3362a6eb80 10 client.26561 fetch_fsmap learned FSMap version 70 2023-07-18T03:41:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.756+0000 7f3362a6eb80 1 -- 172.21.15.40:0/1792230471 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x5609e6537c00 con 0x5609e67cf840 2023-07-18T03:41:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.756+0000 7f332f7fe700 1 -- 172.21.15.40:0/1792230471 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609e667a490 con 0x5609e67cf840 2023-07-18T03:41:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.757+0000 7f332f7fe700 1 -- 172.21.15.40:0/1792230471 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 70) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f334401e410 con 0x5609e67cf840 2023-07-18T03:41:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.757+0000 7f3362a6eb80 10 client.26561 fetch_fsmap finished waiting for FSMap version 70 2023-07-18T03:41:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.757+0000 7f3362a6eb80 10 client.26561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:41:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.757+0000 7f3362a6eb80 20 client.26561 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.757+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/1792230471 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5609e67e4790 0x5609e67e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.757+0000 7f3362a6eb80 4 client.26561 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:41:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.757+0000 7f3362a6eb80 1 -- 172.21.15.40:0/1792230471 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609e6719e50 con 0x5609e67e4790 2023-07-18T03:41:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.757+0000 7f334ffff700 1 --2- 172.21.15.40:0/1792230471 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5609e67e4790 0x5609e67e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.757+0000 7f332f7fe700 10 client.26561 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.757+0000 7f334ffff700 1 --2- 172.21.15.40:0/1792230471 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5609e67e4790 0x5609e67e6ba0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e572c72b0f1c50ec server_cookie=7c6897237b94a29e in_seq=0 out_seq=0 2023-07-18T03:41:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.758+0000 7f332f7fe700 1 -- 172.21.15.40:0/1792230471 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5609e6719e50 con 0x5609e67e4790 2023-07-18T03:41:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.758+0000 7f332f7fe700 10 client.26561 handle_command_reply: tid=0 2023-07-18T03:41:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.758+0000 7f3362a6eb80 1 client.26561 shutdown 2023-07-18T03:41:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.758+0000 7f3362a6eb80 1 -- 172.21.15.40:0/1792230471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e67cf840 msgr2=0x5609e67d8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.758+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/1792230471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e67cf840 0x5609e67d8dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f334400f750 tx=0x7f33440047c0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.758+0000 7f3362a6eb80 1 -- 172.21.15.40:0/1792230471 shutdown_connections 2023-07-18T03:41:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.758+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/1792230471 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5609e67e4790 0x5609e67e6ba0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.758+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/1792230471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67d06e0 0x5609e67dfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.758+0000 7f332f7fe700 0 client.26561 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.758+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/1792230471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e662b6b0 0x5609e67d8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.758+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/1792230471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e67cf840 0x5609e67d8dc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.758+0000 7f3362a6eb80 1 -- 172.21.15.40:0/1792230471 >> 172.21.15.40:0/1792230471 conn(0x5609e67c4be0 msgr2=0x5609e67c94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.758+0000 7f3362a6eb80 1 -- 172.21.15.40:0/1792230471 shutdown_connections 2023-07-18T03:41:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.758+0000 7f3362a6eb80 1 -- 172.21.15.40:0/1792230471 wait complete. 2023-07-18T03:41:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.758+0000 7f3362a6eb80 20 client.26561 trim_cache size 0 max 16384 2023-07-18T03:41:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.773+0000 7f3362a6eb80 1 Processor -- start 2023-07-18T03:41:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.773+0000 7f3362a6eb80 1 -- start start 2023-07-18T03:41:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.773+0000 7f3362a6eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e662b6b0 0x5609e67d8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.773+0000 7f3362a6eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67cf840 0x5609e67d8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.773+0000 7f3362a6eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e67d06e0 0x5609e67dfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.773+0000 7f3362a6eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e67e45a0 con 0x5609e67cf840 2023-07-18T03:41:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.773+0000 7f3362a6eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e67d9780 con 0x5609e662b6b0 2023-07-18T03:41:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.773+0000 7f3362a6eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e67d8420 con 0x5609e67d06e0 2023-07-18T03:41:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.773+0000 7f334f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67cf840 0x5609e67d8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.773+0000 7f334f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67cf840 0x5609e67d8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33420/0 (socket says 172.21.15.40:33420) 2023-07-18T03:41:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.773+0000 7f334f7fe700 1 -- 172.21.15.40:0/2386305750 learned_addr learned my addr 172.21.15.40:0/2386305750 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.773+0000 7f3354e85700 1 --2- 172.21.15.40:0/2386305750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e67d06e0 0x5609e67dfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.773+0000 7f334ffff700 1 --2- 172.21.15.40:0/2386305750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e662b6b0 0x5609e67d8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.774+0000 7f334f7fe700 1 -- 172.21.15.40:0/2386305750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e67d06e0 msgr2=0x5609e67dfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.774+0000 7f334f7fe700 1 --2- 172.21.15.40:0/2386305750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e67d06e0 0x5609e67dfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.774+0000 7f334f7fe700 1 -- 172.21.15.40:0/2386305750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e662b6b0 msgr2=0x5609e67d8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.774+0000 7f334f7fe700 1 --2- 172.21.15.40:0/2386305750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e662b6b0 0x5609e67d8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.771 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.774+0000 7f334f7fe700 1 -- 172.21.15.40:0/2386305750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609e65751f0 con 0x5609e67cf840 2023-07-18T03:41:36.771 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.774+0000 7f334f7fe700 1 --2- 172.21.15.40:0/2386305750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67cf840 0x5609e67d8dc0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f334401d020 tx=0x7f33440046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c545f55b52c9d825 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:36.771 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.774+0000 7f3336ffd700 1 -- 172.21.15.40:0/2386305750 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334401e410 con 0x5609e67cf840 2023-07-18T03:41:36.771 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.774+0000 7f3336ffd700 1 -- 172.21.15.40:0/2386305750 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f334401e570 con 0x5609e67cf840 2023-07-18T03:41:36.771 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.774+0000 7f3336ffd700 1 -- 172.21.15.40:0/2386305750 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33440326e0 con 0x5609e67cf840 2023-07-18T03:41:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.774+0000 7f3362a6eb80 1 -- 172.21.15.40:0/2386305750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67cf840 msgr2=0x5609e67d8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.774+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/2386305750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67cf840 0x5609e67d8dc0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f334401d020 tx=0x7f33440046f0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.775+0000 7f3362a6eb80 1 -- 172.21.15.40:0/2386305750 shutdown_connections 2023-07-18T03:41:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.775+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/2386305750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e67d06e0 0x5609e67dfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.775+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/2386305750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e662b6b0 0x5609e67d8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.775+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/2386305750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67cf840 0x5609e67d8dc0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.775+0000 7f3362a6eb80 1 -- 172.21.15.40:0/2386305750 >> 172.21.15.40:0/2386305750 conn(0x5609e67c4be0 msgr2=0x5609e67d6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.775+0000 7f3362a6eb80 1 -- 172.21.15.40:0/2386305750 shutdown_connections 2023-07-18T03:41:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.775+0000 7f3362a6eb80 1 -- 172.21.15.40:0/2386305750 wait complete. 2023-07-18T03:41:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.775+0000 7f3362a6eb80 1 -- start start 2023-07-18T03:41:36.775 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.775+0000 7f3362a6eb80 1 Processor -- start 2023-07-18T03:41:36.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.775+0000 7f3362a6eb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e662b6b0 0x5609e67ddd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.775+0000 7f3362a6eb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67cf840 0x5609e6834620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.775+0000 7f3362a6eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e67d06e0 0x5609e6836bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.775+0000 7f3362a6eb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e6837330 con 0x5609e67cf840 2023-07-18T03:41:36.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.775+0000 7f3362a6eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e68374a0 con 0x5609e67d06e0 2023-07-18T03:41:36.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.775+0000 7f3362a6eb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e6837660 con 0x5609e662b6b0 2023-07-18T03:41:36.776 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.775+0000 7f3362a6eb80 10 client.0 _ll_register_callbacks cb 0x5609e66be700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:36.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f334f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67cf840 0x5609e6834620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f334ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e662b6b0 0x5609e67ddd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f334f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67cf840 0x5609e6834620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33426/0 (socket says 172.21.15.40:33426) 2023-07-18T03:41:36.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f334ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e662b6b0 0x5609e67ddd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50684/0 (socket says 172.21.15.40:50684) 2023-07-18T03:41:36.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f334f7fe700 1 -- 172.21.15.40:0/3234402228 learned_addr learned my addr 172.21.15.40:0/3234402228 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:36.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f3354e85700 1 --2- 172.21.15.40:0/3234402228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e67d06e0 0x5609e6836bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.777 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f334ffff700 1 -- 172.21.15.40:0/3234402228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e67d06e0 msgr2=0x5609e6836bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f334ffff700 1 --2- 172.21.15.40:0/3234402228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e67d06e0 0x5609e6836bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f334ffff700 1 -- 172.21.15.40:0/3234402228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67cf840 msgr2=0x5609e6834620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f334ffff700 1 --2- 172.21.15.40:0/3234402228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67cf840 0x5609e6834620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.778 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f334ffff700 1 -- 172.21.15.40:0/3234402228 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609e6537c00 con 0x5609e662b6b0 2023-07-18T03:41:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f334f7fe700 1 --2- 172.21.15.40:0/3234402228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67cf840 0x5609e6834620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f3354e85700 1 --2- 172.21.15.40:0/3234402228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e67d06e0 0x5609e6836bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f334ffff700 1 --2- 172.21.15.40:0/3234402228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e662b6b0 0x5609e67ddd10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f33400a1e90 tx=0x7f3340095c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aab5002ff785f87 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f332f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2023-07-18T03:41:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f332f7fe700 1 -- 172.21.15.40:0/3234402228 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3340098170 con 0x5609e662b6b0 2023-07-18T03:41:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f332f7fe700 1 -- 172.21.15.40:0/3234402228 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f334009a250 con 0x5609e662b6b0 2023-07-18T03:41:36.780 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f3362a6eb80 1 -- 172.21.15.40:0/3234402228 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33400a23c0 con 0x5609e662b6b0 2023-07-18T03:41:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f332f7fe700 1 -- 172.21.15.40:0/3234402228 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f334009a410 con 0x5609e662b6b0 2023-07-18T03:41:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f3362a6eb80 10 client.26590 fetch_fsmap learned FSMap version 70 2023-07-18T03:41:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.776+0000 7f3362a6eb80 1 -- 172.21.15.40:0/3234402228 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x5609e654d250 con 0x5609e662b6b0 2023-07-18T03:41:36.781 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.777+0000 7f332f7fe700 1 -- 172.21.15.40:0/3234402228 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33400a23c0 con 0x5609e662b6b0 2023-07-18T03:41:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.777+0000 7f332f7fe700 1 -- 172.21.15.40:0/3234402228 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 70) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f33400a1f30 con 0x5609e662b6b0 2023-07-18T03:41:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.777+0000 7f3362a6eb80 10 client.26590 fetch_fsmap finished waiting for FSMap version 70 2023-07-18T03:41:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.777+0000 7f3362a6eb80 10 client.26590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:41:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.777+0000 7f3362a6eb80 20 client.26590 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.777+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/3234402228 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5609e67e4790 0x5609e683ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.777+0000 7f3362a6eb80 4 client.26590 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:41:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.777+0000 7f3362a6eb80 1 -- 172.21.15.40:0/3234402228 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5609e66dbff0 con 0x5609e67e4790 2023-07-18T03:41:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.777+0000 7f334f7fe700 1 --2- 172.21.15.40:0/3234402228 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5609e67e4790 0x5609e683ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.777+0000 7f332f7fe700 10 client.26590 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.777+0000 7f334f7fe700 1 --2- 172.21.15.40:0/3234402228 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5609e67e4790 0x5609e683ae30 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d0d5121365efe3b server_cookie=a4609e1574d56884 in_seq=0 out_seq=0 2023-07-18T03:41:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.784+0000 7f332f7fe700 1 -- 172.21.15.40:0/3234402228 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5609e66dbff0 con 0x5609e67e4790 2023-07-18T03:41:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.784+0000 7f332f7fe700 10 client.26590 handle_command_reply: tid=0 2023-07-18T03:41:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.784+0000 7f3362a6eb80 1 client.26590 shutdown 2023-07-18T03:41:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.784+0000 7f3362a6eb80 1 -- 172.21.15.40:0/3234402228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e662b6b0 msgr2=0x5609e67ddd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.784+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/3234402228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e662b6b0 0x5609e67ddd10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f33400a1e90 tx=0x7f3340095c40 comp rx=0 tx=0).stop 2023-07-18T03:41:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.784+0000 7f3362a6eb80 1 -- 172.21.15.40:0/3234402228 shutdown_connections 2023-07-18T03:41:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.784+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/3234402228 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5609e67e4790 0x5609e683ae30 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.784+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/3234402228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609e67d06e0 0x5609e6836bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.784+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/3234402228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609e662b6b0 0x5609e67ddd10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.784+0000 7f3362a6eb80 1 --2- 172.21.15.40:0/3234402228 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5609e67cf840 0x5609e6834620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.784+0000 7f332f7fe700 0 client.26590 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.784+0000 7f3362a6eb80 1 -- 172.21.15.40:0/3234402228 >> 172.21.15.40:0/3234402228 conn(0x5609e67c4be0 msgr2=0x5609e67d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.784+0000 7f3362a6eb80 1 -- 172.21.15.40:0/3234402228 shutdown_connections 2023-07-18T03:41:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.785+0000 7f3362a6eb80 1 -- 172.21.15.40:0/3234402228 wait complete. 2023-07-18T03:41:36.787 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.785+0000 7f3362a6eb80 20 client.26590 trim_cache size 0 max 16384 2023-07-18T03:41:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:41:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:41:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:41:36.788 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:41:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.785+0000 7f33568e8700 1 -- 172.21.15.40:0/1802891641 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33380a1150 msgr2=0x7f33380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.785+0000 7f33568e8700 1 --2- 172.21.15.40:0/1802891641 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33380a1150 0x7f33380a3600 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f33500fe020 tx=0x7f334400a380 comp rx=0 tx=0).stop 2023-07-18T03:41:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.785+0000 7f33568e8700 1 -- 172.21.15.40:0/1802891641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33500fec60 msgr2=0x7f3350198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.785+0000 7f33568e8700 1 --2- 172.21.15.40:0/1802891641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33500fec60 0x7f3350198580 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f334000e970 tx=0x7f334000ec80 comp rx=0 tx=0).stop 2023-07-18T03:41:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.785+0000 7f33568e8700 1 -- 172.21.15.40:0/1802891641 shutdown_connections 2023-07-18T03:41:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.785+0000 7f33568e8700 1 --2- 172.21.15.40:0/1802891641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3350100010 0x7f335019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.785+0000 7f33568e8700 1 --2- 172.21.15.40:0/1802891641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33500fec60 0x7f3350198580 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.785+0000 7f33568e8700 1 --2- 172.21.15.40:0/1802891641 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f33380a1150 0x7f33380a3600 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.785+0000 7f33568e8700 1 --2- 172.21.15.40:0/1802891641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33500ff610 0x7f335019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.785+0000 7f33568e8700 1 -- 172.21.15.40:0/1802891641 >> 172.21.15.40:0/1802891641 conn(0x7f33500f9780 msgr2=0x7f33500fb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.785+0000 7f33568e8700 1 -- 172.21.15.40:0/1802891641 shutdown_connections 2023-07-18T03:41:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:36.785+0000 7f33568e8700 1 -- 172.21.15.40:0/1802891641 wait complete. 2023-07-18T03:41:36.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:36 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:41:36.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:36 smithi134 ceph-mon[98216]: scrub summary: idle 2023-07-18T03:41:36.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:36 smithi134 ceph-mon[98216]: scrub summary: idle 2023-07-18T03:41:36.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:36 smithi134 ceph-mon[98216]: scrub summary: idle 2023-07-18T03:41:36.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:36 smithi134 ceph-mon[98216]: scrub summary: idle 2023-07-18T03:41:36.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:36 smithi134 ceph-mon[98216]: scrub summary: idle 2023-07-18T03:41:36.826 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:36 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:41:36.827 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:36 smithi040 ceph-mon[96439]: scrub summary: idle 2023-07-18T03:41:36.827 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:36 smithi040 ceph-mon[96439]: scrub summary: idle 2023-07-18T03:41:36.827 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:36 smithi040 ceph-mon[96439]: scrub summary: idle 2023-07-18T03:41:36.827 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:36 smithi040 ceph-mon[96439]: scrub summary: idle 2023-07-18T03:41:36.827 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:36 smithi040 ceph-mon[96439]: scrub summary: idle 2023-07-18T03:41:37.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:37 smithi064 ceph-mon[98059]: pgmap v616: 97 pgs: 97 active+clean; 836 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 491 op/s 2023-07-18T03:41:37.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:37 smithi134 ceph-mon[98216]: pgmap v616: 97 pgs: 97 active+clean; 836 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 491 op/s 2023-07-18T03:41:37.827 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:37 smithi040 ceph-mon[96439]: pgmap v616: 97 pgs: 97 active+clean; 836 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 491 op/s 2023-07-18T03:41:39.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:39 smithi064 ceph-mon[98059]: pgmap v617: 97 pgs: 97 active+clean; 824 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 529 op/s 2023-07-18T03:41:39.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:39 smithi134 ceph-mon[98216]: pgmap v617: 97 pgs: 97 active+clean; 824 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 529 op/s 2023-07-18T03:41:39.826 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:39 smithi040 ceph-mon[96439]: pgmap v617: 97 pgs: 97 active+clean; 824 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 529 op/s 2023-07-18T03:41:41.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:41 smithi064 ceph-mon[98059]: pgmap v618: 97 pgs: 97 active+clean; 823 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 452 op/s 2023-07-18T03:41:41.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:41 smithi134 ceph-mon[98216]: pgmap v618: 97 pgs: 97 active+clean; 823 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 452 op/s 2023-07-18T03:41:41.827 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:41 smithi040 ceph-mon[96439]: pgmap v618: 97 pgs: 97 active+clean; 823 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 452 op/s 2023-07-18T03:41:43.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:42 smithi134 ceph-mon[98216]: pgmap v619: 97 pgs: 97 active+clean; 794 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 556 op/s 2023-07-18T03:41:43.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:42 smithi040 ceph-mon[96439]: pgmap v619: 97 pgs: 97 active+clean; 794 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 556 op/s 2023-07-18T03:41:43.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:42 smithi064 ceph-mon[98059]: pgmap v619: 97 pgs: 97 active+clean; 794 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 556 op/s 2023-07-18T03:41:45.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:45 smithi064 ceph-mon[98059]: pgmap v620: 97 pgs: 97 active+clean; 782 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 486 op/s 2023-07-18T03:41:45.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:45 smithi134 ceph-mon[98216]: pgmap v620: 97 pgs: 97 active+clean; 782 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 486 op/s 2023-07-18T03:41:45.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:45 smithi040 ceph-mon[96439]: pgmap v620: 97 pgs: 97 active+clean; 782 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 486 op/s 2023-07-18T03:41:47.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:47 smithi064 ceph-mon[98059]: pgmap v621: 97 pgs: 97 active+clean; 759 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 534 op/s 2023-07-18T03:41:47.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:47 smithi134 ceph-mon[98216]: pgmap v621: 97 pgs: 97 active+clean; 759 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 534 op/s 2023-07-18T03:41:47.826 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:47 smithi040 ceph-mon[96439]: pgmap v621: 97 pgs: 97 active+clean; 759 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 534 op/s 2023-07-18T03:41:49.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:49 smithi064 ceph-mon[98059]: pgmap v622: 97 pgs: 97 active+clean; 748 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 508 op/s 2023-07-18T03:41:49.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:49 smithi134 ceph-mon[98216]: pgmap v622: 97 pgs: 97 active+clean; 748 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 508 op/s 2023-07-18T03:41:49.826 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:49 smithi040 ceph-mon[96439]: pgmap v622: 97 pgs: 97 active+clean; 748 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 508 op/s 2023-07-18T03:41:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.894+0000 7f478a1ef700 1 -- 172.21.15.40:0/3041898972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47840685e0 msgr2=0x7f47841026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.895+0000 7f478a1ef700 1 --2- 172.21.15.40:0/3041898972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47840685e0 0x7f47841026f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4774009990 tx=0x7f4774009ca0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.895+0000 7f478a1ef700 1 -- 172.21.15.40:0/3041898972 shutdown_connections 2023-07-18T03:41:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.895+0000 7f478a1ef700 1 --2- 172.21.15.40:0/3041898972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47841056e0 0x7f4784107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.895+0000 7f478a1ef700 1 --2- 172.21.15.40:0/3041898972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47840685e0 0x7f47841026f0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.895+0000 7f478a1ef700 1 --2- 172.21.15.40:0/3041898972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4784102c30 0x7f4784105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.895+0000 7f478a1ef700 1 -- 172.21.15.40:0/3041898972 >> 172.21.15.40:0/3041898972 conn(0x7f47840fa000 msgr2=0x7f47840fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.895+0000 7f478a1ef700 1 -- 172.21.15.40:0/3041898972 shutdown_connections 2023-07-18T03:41:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.895+0000 7f478a1ef700 1 -- 172.21.15.40:0/3041898972 wait complete. 2023-07-18T03:41:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.895+0000 7f478a1ef700 1 Processor -- start 2023-07-18T03:41:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.895+0000 7f478a1ef700 1 -- start start 2023-07-18T03:41:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f478a1ef700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47840685e0 0x7f4784195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f478a1ef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4784102c30 0x7f4784195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f478a1ef700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47841056e0 0x7f4784196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f478a1ef700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47841973b0 con 0x7f4784102c30 2023-07-18T03:41:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f478a1ef700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4784197520 con 0x7f47841056e0 2023-07-18T03:41:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f478a1ef700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47841976e0 con 0x7f47840685e0 2023-07-18T03:41:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f4782ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4784102c30 0x7f4784195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f47837fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47840685e0 0x7f4784195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f4782ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4784102c30 0x7f4784195ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33440/0 (socket says 172.21.15.40:33440) 2023-07-18T03:41:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f4782ffd700 1 -- 172.21.15.40:0/1193967394 learned_addr learned my addr 172.21.15.40:0/1193967394 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f4783fff700 1 --2- 172.21.15.40:0/1193967394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47841056e0 0x7f4784196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f4782ffd700 1 -- 172.21.15.40:0/1193967394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47840685e0 msgr2=0x7f4784195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f4782ffd700 1 --2- 172.21.15.40:0/1193967394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47840685e0 0x7f4784195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f4782ffd700 1 -- 172.21.15.40:0/1193967394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47841056e0 msgr2=0x7f4784196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f4782ffd700 1 --2- 172.21.15.40:0/1193967394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47841056e0 0x7f4784196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f4782ffd700 1 -- 172.21.15.40:0/1193967394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47740095f0 con 0x7f4784102c30 2023-07-18T03:41:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f47837fe700 1 --2- 172.21.15.40:0/1193967394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47840685e0 0x7f4784195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f4782ffd700 1 --2- 172.21.15.40:0/1193967394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4784102c30 0x7f4784195ed0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f477800e9a0 tx=0x7f477800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f4780ff9700 1 -- 172.21.15.40:0/1193967394 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778009790 con 0x7f4784102c30 2023-07-18T03:41:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f478a1ef700 1 -- 172.21.15.40:0/1193967394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47841979c0 con 0x7f4784102c30 2023-07-18T03:41:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f478a1ef700 1 -- 172.21.15.40:0/1193967394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478419e180 con 0x7f4784102c30 2023-07-18T03:41:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.896+0000 7f4780ff9700 1 -- 172.21.15.40:0/1193967394 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4778010430 con 0x7f4784102c30 2023-07-18T03:41:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.897+0000 7f4780ff9700 1 -- 172.21.15.40:0/1193967394 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778018680 con 0x7f4784102c30 2023-07-18T03:41:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.899+0000 7f4780ff9700 1 -- 172.21.15.40:0/1193967394 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f477800ca10 con 0x7f4784102c30 2023-07-18T03:41:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.900+0000 7f4780ff9700 1 --2- 172.21.15.40:0/1193967394 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f476c0a11d0 0x7f476c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.900+0000 7f4780ff9700 1 -- 172.21.15.40:0/1193967394 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4778014070 con 0x7f4784102c30 2023-07-18T03:41:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.900+0000 7f47837fe700 1 --2- 172.21.15.40:0/1193967394 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f476c0a11d0 0x7f476c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.900+0000 7f47837fe700 1 --2- 172.21.15.40:0/1193967394 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f476c0a11d0 0x7f476c0a3680 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f477400b2e0 tx=0x7f4774005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f4796375b80 1 Processor -- start 2023-07-18T03:41:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f4796375b80 1 -- start start 2023-07-18T03:41:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f4796375b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631599ba6b0 0x563159b5e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f4796375b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5e840 0x563159b5f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f4796375b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563159b5f6e0 0x563159b5fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f4796375b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563159a63c50 con 0x563159b5e840 2023-07-18T03:41:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f4796375b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563159ab69e0 con 0x563159b5f6e0 2023-07-18T03:41:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f4796375b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563159aafe70 con 0x5631599ba6b0 2023-07-18T03:41:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f4782ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5e840 0x563159b5f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f47837fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631599ba6b0 0x563159b5e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f4782ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5e840 0x563159b5f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33448/0 (socket says 172.21.15.40:33448) 2023-07-18T03:41:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f4782ffd700 1 -- 172.21.15.40:0/3033084000 learned_addr learned my addr 172.21.15.40:0/3033084000 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f4783fff700 1 --2- 172.21.15.40:0/3033084000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563159b5f6e0 0x563159b5fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f47837fe700 1 -- 172.21.15.40:0/3033084000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563159b5f6e0 msgr2=0x563159b5fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f47837fe700 1 --2- 172.21.15.40:0/3033084000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563159b5f6e0 0x563159b5fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f47837fe700 1 -- 172.21.15.40:0/3033084000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5e840 msgr2=0x563159b5f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f47837fe700 1 --2- 172.21.15.40:0/3033084000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5e840 0x563159b5f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.901+0000 7f47837fe700 1 -- 172.21.15.40:0/3033084000 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631598ddd80 con 0x5631599ba6b0 2023-07-18T03:41:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.902+0000 7f47837fe700 1 --2- 172.21.15.40:0/3033084000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631599ba6b0 0x563159b5e300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f47740098e0 tx=0x7f4774020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d08ba028bb546b9c server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.902+0000 7f4769ffb700 1 -- 172.21.15.40:0/3033084000 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477402b070 con 0x5631599ba6b0 2023-07-18T03:41:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.902+0000 7f4769ffb700 1 -- 172.21.15.40:0/3033084000 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f477401e040 con 0x5631599ba6b0 2023-07-18T03:41:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.902+0000 7f4769ffb700 1 -- 172.21.15.40:0/3033084000 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774024d90 con 0x5631599ba6b0 2023-07-18T03:41:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.902+0000 7f4796375b80 1 -- 172.21.15.40:0/3033084000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631599ba6b0 msgr2=0x563159b5e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.902+0000 7f4796375b80 1 --2- 172.21.15.40:0/3033084000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631599ba6b0 0x563159b5e300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f47740098e0 tx=0x7f4774020000 comp rx=0 tx=0).stop 2023-07-18T03:41:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.902+0000 7f4796375b80 1 -- 172.21.15.40:0/3033084000 shutdown_connections 2023-07-18T03:41:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.902+0000 7f4796375b80 1 --2- 172.21.15.40:0/3033084000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563159b5f6e0 0x563159b5fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.902+0000 7f4796375b80 1 --2- 172.21.15.40:0/3033084000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631599ba6b0 0x563159b5e300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.902+0000 7f4796375b80 1 --2- 172.21.15.40:0/3033084000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5e840 0x563159b5f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.902+0000 7f4796375b80 1 -- 172.21.15.40:0/3033084000 >> 172.21.15.40:0/3033084000 conn(0x563159b53be0 msgr2=0x563159b58cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.903+0000 7f4796375b80 1 -- 172.21.15.40:0/3033084000 shutdown_connections 2023-07-18T03:41:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.903+0000 7f4796375b80 1 -- 172.21.15.40:0/3033084000 wait complete. 2023-07-18T03:41:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.903+0000 7f4796375b80 1 -- start start 2023-07-18T03:41:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.903+0000 7f4796375b80 1 Processor -- start 2023-07-18T03:41:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.903+0000 7f4796375b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631599ba6b0 0x563159b677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.903+0000 7f4796375b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5e840 0x563159b67cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.903+0000 7f4796375b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563159b5f6e0 0x563159b6ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.903+0000 7f4796375b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563159a4fa60 con 0x563159b5e840 2023-07-18T03:41:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.903+0000 7f4796375b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563159a4f760 con 0x5631599ba6b0 2023-07-18T03:41:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.903+0000 7f4796375b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631599338c0 con 0x563159b5f6e0 2023-07-18T03:41:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.903+0000 7f4796375b80 10 client.0 _ll_register_callbacks cb 0x563159a4d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.903+0000 7f4782ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5e840 0x563159b67cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.903+0000 7f47837fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631599ba6b0 0x563159b677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f47837fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631599ba6b0 0x563159b677b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:45010/0 (socket says 172.21.15.40:45010) 2023-07-18T03:41:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f4782ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5e840 0x563159b67cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33454/0 (socket says 172.21.15.40:33454) 2023-07-18T03:41:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f47837fe700 1 -- 172.21.15.40:0/973921199 learned_addr learned my addr 172.21.15.40:0/973921199 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f4783fff700 1 --2- 172.21.15.40:0/973921199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563159b5f6e0 0x563159b6ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f47837fe700 1 -- 172.21.15.40:0/973921199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563159b5f6e0 msgr2=0x563159b6ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f47837fe700 1 --2- 172.21.15.40:0/973921199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563159b5f6e0 0x563159b6ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f47837fe700 1 -- 172.21.15.40:0/973921199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5e840 msgr2=0x563159b67cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f47837fe700 1 --2- 172.21.15.40:0/973921199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5e840 0x563159b67cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f47837fe700 1 -- 172.21.15.40:0/973921199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4784108850 con 0x5631599ba6b0 2023-07-18T03:41:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f4782ffd700 1 --2- 172.21.15.40:0/973921199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5e840 0x563159b67cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f47837fe700 1 --2- 172.21.15.40:0/973921199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631599ba6b0 0x563159b677b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4774009350 tx=0x7f4774025f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc36988fd7dc97e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f4762ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:41:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f4762ffd700 1 -- 172.21.15.40:0/973921199 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477402b040 con 0x5631599ba6b0 2023-07-18T03:41:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f4762ffd700 1 -- 172.21.15.40:0/973921199 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f4774017490 con 0x5631599ba6b0 2023-07-18T03:41:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f4796375b80 1 -- 172.21.15.40:0/973921199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563159a09490 con 0x5631599ba6b0 2023-07-18T03:41:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f4762ffd700 1 -- 172.21.15.40:0/973921199 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4774017650 con 0x5631599ba6b0 2023-07-18T03:41:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f4796375b80 10 client.26579 fetch_fsmap learned FSMap version 70 2023-07-18T03:41:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.904+0000 7f4796375b80 1 -- 172.21.15.40:0/973921199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x5631598c6c00 con 0x5631599ba6b0 2023-07-18T03:41:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.905+0000 7f4762ffd700 1 -- 172.21.15.40:0/973921199 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563159a09490 con 0x5631599ba6b0 2023-07-18T03:41:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.905+0000 7f4762ffd700 1 -- 172.21.15.40:0/973921199 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 70) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f477402f700 con 0x5631599ba6b0 2023-07-18T03:41:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.905+0000 7f4796375b80 10 client.26579 fetch_fsmap finished waiting for FSMap version 70 2023-07-18T03:41:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.905+0000 7f4796375b80 10 client.26579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:41:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.905+0000 7f4796375b80 20 client.26579 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.905+0000 7f4796375b80 1 --2- 172.21.15.40:0/973921199 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563159b73250 0x563159b75660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.905+0000 7f4796375b80 4 client.26579 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:41:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.905+0000 7f4796375b80 1 -- 172.21.15.40:0/973921199 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563159aa8e50 con 0x563159b73250 2023-07-18T03:41:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.905+0000 7f4782ffd700 1 --2- 172.21.15.40:0/973921199 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563159b73250 0x563159b75660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.905+0000 7f4782ffd700 1 --2- 172.21.15.40:0/973921199 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563159b73250 0x563159b75660 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c5d382ce6b1a2c1 server_cookie=aa8930117f4395ad in_seq=0 out_seq=0 2023-07-18T03:41:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.905+0000 7f4762ffd700 10 client.26579 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.906+0000 7f4762ffd700 1 -- 172.21.15.40:0/973921199 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563159aa8e50 con 0x563159b73250 2023-07-18T03:41:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.906+0000 7f4762ffd700 10 client.26579 handle_command_reply: tid=0 2023-07-18T03:41:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.906+0000 7f4796375b80 1 client.26579 shutdown 2023-07-18T03:41:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.906+0000 7f4796375b80 1 -- 172.21.15.40:0/973921199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631599ba6b0 msgr2=0x563159b677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.906+0000 7f4796375b80 1 --2- 172.21.15.40:0/973921199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631599ba6b0 0x563159b677b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4774009350 tx=0x7f4774025f90 comp rx=0 tx=0).stop 2023-07-18T03:41:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.906+0000 7f4796375b80 1 -- 172.21.15.40:0/973921199 shutdown_connections 2023-07-18T03:41:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.906+0000 7f4796375b80 1 --2- 172.21.15.40:0/973921199 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563159b73250 0x563159b75660 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.906+0000 7f4796375b80 1 --2- 172.21.15.40:0/973921199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563159b5f6e0 0x563159b6ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.906+0000 7f4796375b80 1 --2- 172.21.15.40:0/973921199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5631599ba6b0 0x563159b677b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.906+0000 7f4762ffd700 0 client.26579 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.906+0000 7f4796375b80 1 --2- 172.21.15.40:0/973921199 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5e840 0x563159b67cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.906+0000 7f4796375b80 1 -- 172.21.15.40:0/973921199 >> 172.21.15.40:0/973921199 conn(0x563159b53be0 msgr2=0x563159b584e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.906+0000 7f4796375b80 1 -- 172.21.15.40:0/973921199 shutdown_connections 2023-07-18T03:41:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.906+0000 7f4796375b80 1 -- 172.21.15.40:0/973921199 wait complete. 2023-07-18T03:41:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.906+0000 7f4796375b80 20 client.26579 trim_cache size 0 max 16384 2023-07-18T03:41:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4796375b80 1 Processor -- start 2023-07-18T03:41:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4796375b80 1 -- start start 2023-07-18T03:41:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4796375b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631599ba6b0 0x563159b65a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4796375b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563159b5e840 0x563159b65f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4796375b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5f6e0 0x563159b6ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4796375b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563159b50c30 con 0x563159b5f6e0 2023-07-18T03:41:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4796375b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563159b59930 con 0x563159b5e840 2023-07-18T03:41:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4796375b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563159b657b0 con 0x5631599ba6b0 2023-07-18T03:41:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4783fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5f6e0 0x563159b6ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4783fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5f6e0 0x563159b6ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33464/0 (socket says 172.21.15.40:33464) 2023-07-18T03:41:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4783fff700 1 -- 172.21.15.40:0/1159853526 learned_addr learned my addr 172.21.15.40:0/1159853526 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f47837fe700 1 --2- 172.21.15.40:0/1159853526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631599ba6b0 0x563159b65a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4782ffd700 1 --2- 172.21.15.40:0/1159853526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563159b5e840 0x563159b65f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4783fff700 1 -- 172.21.15.40:0/1159853526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631599ba6b0 msgr2=0x563159b65a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4783fff700 1 --2- 172.21.15.40:0/1159853526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631599ba6b0 0x563159b65a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.930 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4783fff700 1 -- 172.21.15.40:0/1159853526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563159b5e840 msgr2=0x563159b65f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:49.930 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4783fff700 1 --2- 172.21.15.40:0/1159853526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563159b5e840 0x563159b65f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.930 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4783fff700 1 -- 172.21.15.40:0/1159853526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631599041f0 con 0x563159b5f6e0 2023-07-18T03:41:49.930 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.922+0000 7f4783fff700 1 --2- 172.21.15.40:0/1159853526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5f6e0 0x563159b6ecc0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f477c015660 tx=0x7f477c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9786652392c96c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:49.930 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.923+0000 7f476a7fc700 1 -- 172.21.15.40:0/1159853526 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c00e040 con 0x563159b5f6e0 2023-07-18T03:41:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.923+0000 7f476a7fc700 1 -- 172.21.15.40:0/1159853526 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f477c00f040 con 0x563159b5f6e0 2023-07-18T03:41:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.923+0000 7f476a7fc700 1 -- 172.21.15.40:0/1159853526 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c020d20 con 0x563159b5f6e0 2023-07-18T03:41:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.923+0000 7f4796375b80 1 -- 172.21.15.40:0/1159853526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5f6e0 msgr2=0x563159b6ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:49.931 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.923+0000 7f4796375b80 1 --2- 172.21.15.40:0/1159853526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5f6e0 0x563159b6ecc0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f477c015660 tx=0x7f477c009920 comp rx=0 tx=0).stop 2023-07-18T03:41:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.923+0000 7f4796375b80 1 -- 172.21.15.40:0/1159853526 shutdown_connections 2023-07-18T03:41:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.923+0000 7f4796375b80 1 --2- 172.21.15.40:0/1159853526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x563159b5f6e0 0x563159b6ecc0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.923+0000 7f4796375b80 1 --2- 172.21.15.40:0/1159853526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631599ba6b0 0x563159b65a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.923+0000 7f4796375b80 1 --2- 172.21.15.40:0/1159853526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563159b5e840 0x563159b65f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.923+0000 7f4796375b80 1 -- 172.21.15.40:0/1159853526 >> 172.21.15.40:0/1159853526 conn(0x563159b53be0 msgr2=0x563159b79f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:49.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.923+0000 7f4796375b80 1 -- 172.21.15.40:0/1159853526 shutdown_connections 2023-07-18T03:41:49.934 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.923+0000 7f4796375b80 1 -- 172.21.15.40:0/1159853526 wait complete. 2023-07-18T03:41:49.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.923+0000 7f4796375b80 1 -- start start 2023-07-18T03:41:49.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f4796375b80 1 Processor -- start 2023-07-18T03:41:49.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f4796375b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631599ba6b0 0x563159b6aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.935 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f4796375b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563159b5e840 0x563159b6b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f4796375b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563159b5f6e0 0x563159b6df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f4796375b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563159b6e470 con 0x5631599ba6b0 2023-07-18T03:41:49.936 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f4796375b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563159b6e5e0 con 0x563159b5f6e0 2023-07-18T03:41:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f4796375b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563159b6e7a0 con 0x563159b5e840 2023-07-18T03:41:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f4796375b80 10 client.0 _ll_register_callbacks cb 0x7f4778011ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f47837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631599ba6b0 0x563159b6aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.937 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f47837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631599ba6b0 0x563159b6aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33466/0 (socket says 172.21.15.40:33466) 2023-07-18T03:41:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f47837fe700 1 -- 172.21.15.40:0/3080025715 learned_addr learned my addr 172.21.15.40:0/3080025715 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f4782ffd700 1 --2- 172.21.15.40:0/3080025715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563159b5e840 0x563159b6b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f4783fff700 1 --2- 172.21.15.40:0/3080025715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563159b5f6e0 0x563159b6df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f47837fe700 1 -- 172.21.15.40:0/3080025715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563159b5e840 msgr2=0x563159b6b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f47837fe700 1 --2- 172.21.15.40:0/3080025715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563159b5e840 0x563159b6b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f47837fe700 1 -- 172.21.15.40:0/3080025715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563159b5f6e0 msgr2=0x563159b6df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f47837fe700 1 --2- 172.21.15.40:0/3080025715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563159b5f6e0 0x563159b6df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.924+0000 7f47837fe700 1 -- 172.21.15.40:0/3080025715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c005e70 con 0x5631599ba6b0 2023-07-18T03:41:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.925+0000 7f47837fe700 1 --2- 172.21.15.40:0/3080025715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631599ba6b0 0x563159b6aef0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4774009ce0 tx=0x7f4774025e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354341414fd61b5e server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.925+0000 7f4762ffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:41:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.925+0000 7f4762ffd700 1 -- 172.21.15.40:0/3080025715 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477402b070 con 0x5631599ba6b0 2023-07-18T03:41:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.925+0000 7f4762ffd700 1 -- 172.21.15.40:0/3080025715 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f477402f830 con 0x5631599ba6b0 2023-07-18T03:41:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.925+0000 7f4796375b80 1 -- 172.21.15.40:0/3080025715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f477401e040 con 0x5631599ba6b0 2023-07-18T03:41:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.925+0000 7f4762ffd700 1 -- 172.21.15.40:0/3080025715 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4774024bd0 con 0x5631599ba6b0 2023-07-18T03:41:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.925+0000 7f4796375b80 10 client.16947 fetch_fsmap learned FSMap version 70 2023-07-18T03:41:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.925+0000 7f4796375b80 1 -- 172.21.15.40:0/3080025715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x5631598dc250 con 0x5631599ba6b0 2023-07-18T03:41:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.925+0000 7f4762ffd700 1 -- 172.21.15.40:0/3080025715 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f477401e040 con 0x5631599ba6b0 2023-07-18T03:41:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.925+0000 7f4762ffd700 1 -- 172.21.15.40:0/3080025715 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 70) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f4774038460 con 0x5631599ba6b0 2023-07-18T03:41:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.925+0000 7f4796375b80 10 client.16947 fetch_fsmap finished waiting for FSMap version 70 2023-07-18T03:41:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.925+0000 7f4796375b80 10 client.16947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:41:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.925+0000 7f4796375b80 20 client.16947 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.925+0000 7f4796375b80 1 --2- 172.21.15.40:0/3080025715 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563159bc7b90 0x563159bc9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.925+0000 7f4796375b80 4 client.16947 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:41:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.925+0000 7f4796375b80 1 -- 172.21.15.40:0/3080025715 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563159a6aff0 con 0x563159bc7b90 2023-07-18T03:41:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.926+0000 7f4782ffd700 1 --2- 172.21.15.40:0/3080025715 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563159bc7b90 0x563159bc9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.926+0000 7f4762ffd700 10 client.16947 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.926+0000 7f4782ffd700 1 --2- 172.21.15.40:0/3080025715 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563159bc7b90 0x563159bc9fa0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=392b1397917aee3b server_cookie=2ad24eb57f125030 in_seq=0 out_seq=0 2023-07-18T03:41:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.933+0000 7f4762ffd700 1 -- 172.21.15.40:0/3080025715 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563159a6aff0 con 0x563159bc7b90 2023-07-18T03:41:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.933+0000 7f4762ffd700 10 client.16947 handle_command_reply: tid=0 2023-07-18T03:41:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.933+0000 7f4796375b80 1 client.16947 shutdown 2023-07-18T03:41:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.933+0000 7f4796375b80 1 -- 172.21.15.40:0/3080025715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631599ba6b0 msgr2=0x563159b6aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.933+0000 7f4796375b80 1 --2- 172.21.15.40:0/3080025715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631599ba6b0 0x563159b6aef0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4774009ce0 tx=0x7f4774025e70 comp rx=0 tx=0).stop 2023-07-18T03:41:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.933+0000 7f4796375b80 1 -- 172.21.15.40:0/3080025715 shutdown_connections 2023-07-18T03:41:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.933+0000 7f4796375b80 1 --2- 172.21.15.40:0/3080025715 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x563159bc7b90 0x563159bc9fa0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.933+0000 7f4796375b80 1 --2- 172.21.15.40:0/3080025715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563159b5f6e0 0x563159b6df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.933+0000 7f4796375b80 1 --2- 172.21.15.40:0/3080025715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5631599ba6b0 0x563159b6aef0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.933+0000 7f4762ffd700 0 client.16947 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.933+0000 7f4796375b80 1 --2- 172.21.15.40:0/3080025715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563159b5e840 0x563159b6b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.933+0000 7f4796375b80 1 -- 172.21.15.40:0/3080025715 >> 172.21.15.40:0/3080025715 conn(0x563159b53be0 msgr2=0x563159b65a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.933+0000 7f4796375b80 1 -- 172.21.15.40:0/3080025715 shutdown_connections 2023-07-18T03:41:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.933+0000 7f4796375b80 1 -- 172.21.15.40:0/3080025715 wait complete. 2023-07-18T03:41:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.933+0000 7f4796375b80 20 client.16947 trim_cache size 0 max 16384 2023-07-18T03:41:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:41:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:41:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:41:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:41:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.934+0000 7f478a1ef700 1 -- 172.21.15.40:0/1193967394 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f476c0a11d0 msgr2=0x7f476c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.934+0000 7f478a1ef700 1 --2- 172.21.15.40:0/1193967394 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f476c0a11d0 0x7f476c0a3680 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f477400b2e0 tx=0x7f4774005fb0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.934+0000 7f478a1ef700 1 -- 172.21.15.40:0/1193967394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4784102c30 msgr2=0x7f4784195ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.934+0000 7f478a1ef700 1 --2- 172.21.15.40:0/1193967394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4784102c30 0x7f4784195ed0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f477800e9a0 tx=0x7f477800ecb0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.934+0000 7f478a1ef700 1 -- 172.21.15.40:0/1193967394 shutdown_connections 2023-07-18T03:41:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.934+0000 7f478a1ef700 1 --2- 172.21.15.40:0/1193967394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47841056e0 0x7f4784196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.934+0000 7f478a1ef700 1 --2- 172.21.15.40:0/1193967394 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f476c0a11d0 0x7f476c0a3680 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.934+0000 7f478a1ef700 1 --2- 172.21.15.40:0/1193967394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47840685e0 0x7f4784195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.952 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.934+0000 7f478a1ef700 1 --2- 172.21.15.40:0/1193967394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4784102c30 0x7f4784195ed0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:49.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.934+0000 7f478a1ef700 1 -- 172.21.15.40:0/1193967394 >> 172.21.15.40:0/1193967394 conn(0x7f47840fa000 msgr2=0x7f4784100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:49.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.934+0000 7f478a1ef700 1 -- 172.21.15.40:0/1193967394 shutdown_connections 2023-07-18T03:41:49.953 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:41:49.934+0000 7f478a1ef700 1 -- 172.21.15.40:0/1193967394 wait complete. 2023-07-18T03:41:51.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:51 smithi064 ceph-mon[98059]: pgmap v623: 97 pgs: 97 active+clean; 744 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 404 op/s 2023-07-18T03:41:51.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:51 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:41:51.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:51 smithi134 ceph-mon[98216]: pgmap v623: 97 pgs: 97 active+clean; 744 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 404 op/s 2023-07-18T03:41:51.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:51 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:41:51.826 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:51 smithi040 ceph-mon[96439]: pgmap v623: 97 pgs: 97 active+clean; 744 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 404 op/s 2023-07-18T03:41:51.827 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:51 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:41:53.313 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:52 smithi134 ceph-mon[98216]: pgmap v624: 97 pgs: 97 active+clean; 720 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 541 op/s 2023-07-18T03:41:53.326 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:52 smithi040 ceph-mon[96439]: pgmap v624: 97 pgs: 97 active+clean; 720 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 541 op/s 2023-07-18T03:41:53.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:52 smithi064 ceph-mon[98059]: pgmap v624: 97 pgs: 97 active+clean; 720 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 541 op/s 2023-07-18T03:41:54.085 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-18T03:41:54.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.251+0000 7f80c7b54700 1 -- 172.21.15.40:0/4012018575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80c00fe350 msgr2=0x7f80c0100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.251+0000 7f80c7b54700 1 --2- 172.21.15.40:0/4012018575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80c00fe350 0x7f80c0100770 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009990 tx=0x7f80bc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.252+0000 7f80c7b54700 1 -- 172.21.15.40:0/4012018575 shutdown_connections 2023-07-18T03:41:54.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.252+0000 7f80c7b54700 1 --2- 172.21.15.40:0/4012018575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80c0103670 0x7f80c0103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.252+0000 7f80c7b54700 1 --2- 172.21.15.40:0/4012018575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80c00fe350 0x7f80c0100770 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.252+0000 7f80c7b54700 1 --2- 172.21.15.40:0/4012018575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80c0100cb0 0x7f80c0103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.252+0000 7f80c7b54700 1 -- 172.21.15.40:0/4012018575 >> 172.21.15.40:0/4012018575 conn(0x7f80c00fa000 msgr2=0x7f80c00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.252+0000 7f80c7b54700 1 -- 172.21.15.40:0/4012018575 shutdown_connections 2023-07-18T03:41:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.252+0000 7f80c7b54700 1 -- 172.21.15.40:0/4012018575 wait complete. 2023-07-18T03:41:54.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.253+0000 7f80c7b54700 1 Processor -- start 2023-07-18T03:41:54.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.253+0000 7f80c7b54700 1 -- start start 2023-07-18T03:41:54.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.253+0000 7f80c7b54700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80c00fe350 0x7f80c019be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.253+0000 7f80c7b54700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80c0100cb0 0x7f80c019c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.253+0000 7f80c7b54700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80c0103670 0x7f80c01a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.253+0000 7f80c7b54700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c01a3a10 con 0x7f80c0103670 2023-07-18T03:41:54.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.253+0000 7f80c7b54700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c01a3b50 con 0x7f80c0100cb0 2023-07-18T03:41:54.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.253+0000 7f80c7b54700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c01a3ce0 con 0x7f80c00fe350 2023-07-18T03:41:54.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.253+0000 7f80c60f1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80c0103670 0x7f80c01a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.253+0000 7f80c60f1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80c0103670 0x7f80c01a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33480/0 (socket says 172.21.15.40:33480) 2023-07-18T03:41:54.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.253+0000 7f80c60f1700 1 -- 172.21.15.40:0/1082594693 learned_addr learned my addr 172.21.15.40:0/1082594693 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:54.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.254+0000 7f80c60f1700 1 -- 172.21.15.40:0/1082594693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80c00fe350 msgr2=0x7f80c019be90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-18T03:41:54.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.254+0000 7f80c60f1700 1 --2- 172.21.15.40:0/1082594693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80c00fe350 0x7f80c019be90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.254+0000 7f80c50ef700 1 --2- 172.21.15.40:0/1082594693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80c0100cb0 0x7f80c019c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.254+0000 7f80c60f1700 1 -- 172.21.15.40:0/1082594693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80c0100cb0 msgr2=0x7f80c019c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.254+0000 7f80c60f1700 1 --2- 172.21.15.40:0/1082594693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80c0100cb0 0x7f80c019c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.254+0000 7f80c60f1700 1 -- 172.21.15.40:0/1082594693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80bc0095f0 con 0x7f80c0103670 2023-07-18T03:41:54.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.254+0000 7f80c50ef700 1 --2- 172.21.15.40:0/1082594693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80c0100cb0 0x7f80c019c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:41:54.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.254+0000 7f80c60f1700 1 --2- 172.21.15.40:0/1082594693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80c0103670 0x7f80c01a3440 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f80b800b970 tx=0x7f80b800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:54.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.255+0000 7f80b6ffd700 1 -- 172.21.15.40:0/1082594693 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b800c9b0 con 0x7f80c0103670 2023-07-18T03:41:54.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.256+0000 7f80b6ffd700 1 -- 172.21.15.40:0/1082594693 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80b8014440 con 0x7f80c0103670 2023-07-18T03:41:54.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.256+0000 7f80b6ffd700 1 -- 172.21.15.40:0/1082594693 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b80075d0 con 0x7f80c0103670 2023-07-18T03:41:54.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.256+0000 7f80c7b54700 1 -- 172.21.15.40:0/1082594693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c01a3fc0 con 0x7f80c0103670 2023-07-18T03:41:54.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.256+0000 7f80c7b54700 1 -- 172.21.15.40:0/1082594693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c01a4390 con 0x7f80c0103670 2023-07-18T03:41:54.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.259+0000 7f80b6ffd700 1 -- 172.21.15.40:0/1082594693 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f80b80145b0 con 0x7f80c0103670 2023-07-18T03:41:54.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.260+0000 7f80b6ffd700 1 --2- 172.21.15.40:0/1082594693 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f80ac0a12a0 0x7f80ac0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.260+0000 7f80b6ffd700 1 -- 172.21.15.40:0/1082594693 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f80b8006e40 con 0x7f80c0103670 2023-07-18T03:41:54.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.260+0000 7f80c58f0700 1 --2- 172.21.15.40:0/1082594693 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f80ac0a12a0 0x7f80ac0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.260+0000 7f80c58f0700 1 --2- 172.21.15.40:0/1082594693 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f80ac0a12a0 0x7f80ac0a3750 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0098e0 tx=0x7f80bc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:54.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.260+0000 7f80d3cdab80 1 Processor -- start 2023-07-18T03:41:54.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.261+0000 7f80d3cdab80 1 -- start start 2023-07-18T03:41:54.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.261+0000 7f80d3cdab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bef4d0 0x5599a5bef8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.261+0000 7f80d3cdab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5befe20 0x5599a5bf06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.261+0000 7f80d3cdab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5bf0c70 0x5599a5bf10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.261+0000 7f80d3cdab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a5af4cb0 con 0x5599a5bf0c70 2023-07-18T03:41:54.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.261+0000 7f80d3cdab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a5b47a40 con 0x5599a5bef4d0 2023-07-18T03:41:54.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.261+0000 7f80d3cdab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a5b40ed0 con 0x5599a5befe20 2023-07-18T03:41:54.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.261+0000 7f80c58f0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bef4d0 0x5599a5bef8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.261+0000 7f80c58f0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bef4d0 0x5599a5bef8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:45044/0 (socket says 172.21.15.40:45044) 2023-07-18T03:41:54.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.261+0000 7f80c58f0700 1 -- 172.21.15.40:0/79267978 learned_addr learned my addr 172.21.15.40:0/79267978 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:54.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.262+0000 7f80c60f1700 1 --2- 172.21.15.40:0/79267978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5bf0c70 0x5599a5bf10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.262+0000 7f80c60f1700 1 -- 172.21.15.40:0/79267978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5befe20 msgr2=0x5599a5bf06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:54.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.262+0000 7f80c60f1700 1 --2- 172.21.15.40:0/79267978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5befe20 0x5599a5bf06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.262+0000 7f80c60f1700 1 -- 172.21.15.40:0/79267978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bef4d0 msgr2=0x5599a5bef8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:54.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.262+0000 7f80c60f1700 1 --2- 172.21.15.40:0/79267978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bef4d0 0x5599a5bef8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.262+0000 7f80c60f1700 1 -- 172.21.15.40:0/79267978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599a596ede0 con 0x5599a5bf0c70 2023-07-18T03:41:54.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.262+0000 7f80c60f1700 1 --2- 172.21.15.40:0/79267978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5bf0c70 0x5599a5bf10e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f80b80c9ae0 tx=0x7f80b8014ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f66d83f3468c56 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:54.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.262+0000 7f80abfff700 1 -- 172.21.15.40:0/79267978 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b80a1840 con 0x5599a5bf0c70 2023-07-18T03:41:54.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.262+0000 7f80abfff700 1 -- 172.21.15.40:0/79267978 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80b8097790 con 0x5599a5bf0c70 2023-07-18T03:41:54.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.262+0000 7f80abfff700 1 -- 172.21.15.40:0/79267978 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b80a13b0 con 0x5599a5bf0c70 2023-07-18T03:41:54.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.263+0000 7f80d3cdab80 1 -- 172.21.15.40:0/79267978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5bf0c70 msgr2=0x5599a5bf10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.263+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/79267978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5bf0c70 0x5599a5bf10e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f80b80c9ae0 tx=0x7f80b8014ca0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.263+0000 7f80d3cdab80 1 -- 172.21.15.40:0/79267978 shutdown_connections 2023-07-18T03:41:54.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.263+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/79267978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5bf0c70 0x5599a5bf10e0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.263+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/79267978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bef4d0 0x5599a5bef8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.263+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/79267978 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5befe20 0x5599a5bf06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.263+0000 7f80d3cdab80 1 -- 172.21.15.40:0/79267978 >> 172.21.15.40:0/79267978 conn(0x5599a5be4da0 msgr2=0x5599a5bea1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:54.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.263+0000 7f80d3cdab80 1 -- 172.21.15.40:0/79267978 shutdown_connections 2023-07-18T03:41:54.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.263+0000 7f80d3cdab80 1 -- 172.21.15.40:0/79267978 wait complete. 2023-07-18T03:41:54.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.264+0000 7f80d3cdab80 1 -- start start 2023-07-18T03:41:54.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.264+0000 7f80d3cdab80 1 Processor -- start 2023-07-18T03:41:54.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.264+0000 7f80d3cdab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bef4d0 0x5599a5bf91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.264+0000 7f80d3cdab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5befe20 0x5599a5bf9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.264+0000 7f80d3cdab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5bf0c70 0x5599a5c00740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.264+0000 7f80d3cdab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a5ae0ac0 con 0x5599a5befe20 2023-07-18T03:41:54.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.264+0000 7f80d3cdab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a5ae07c0 con 0x5599a5bef4d0 2023-07-18T03:41:54.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.264+0000 7f80d3cdab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a59c4920 con 0x5599a5bf0c70 2023-07-18T03:41:54.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.264+0000 7f80d3cdab80 10 client.0 _ll_register_callbacks cb 0x5599a5ade760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:54.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.264+0000 7f80c50ef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5befe20 0x5599a5bf9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.264+0000 7f80c60f1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5bf0c70 0x5599a5c00740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.264+0000 7f80c50ef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5befe20 0x5599a5bf9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33494/0 (socket says 172.21.15.40:33494) 2023-07-18T03:41:54.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.264+0000 7f80c50ef700 1 -- 172.21.15.40:0/1122719222 learned_addr learned my addr 172.21.15.40:0/1122719222 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:54.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.264+0000 7f80c58f0700 1 --2- 172.21.15.40:0/1122719222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bef4d0 0x5599a5bf91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.265+0000 7f80c50ef700 1 -- 172.21.15.40:0/1122719222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5bf0c70 msgr2=0x5599a5c00740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:54.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.265+0000 7f80c50ef700 1 --2- 172.21.15.40:0/1122719222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5bf0c70 0x5599a5c00740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.265+0000 7f80c50ef700 1 -- 172.21.15.40:0/1122719222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bef4d0 msgr2=0x5599a5bf91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:54.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.265+0000 7f80c50ef700 1 --2- 172.21.15.40:0/1122719222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bef4d0 0x5599a5bf91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.265+0000 7f80c50ef700 1 -- 172.21.15.40:0/1122719222 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c01a4390 con 0x5599a5befe20 2023-07-18T03:41:54.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.265+0000 7f80c50ef700 1 --2- 172.21.15.40:0/1122719222 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5befe20 0x5599a5bf9700 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f80b000e620 tx=0x7f80b000e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51bb23226347320 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:54.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.265+0000 7f80a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:41:54.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.265+0000 7f80a97fa700 1 -- 172.21.15.40:0/1122719222 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b0011880 con 0x5599a5befe20 2023-07-18T03:41:54.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.265+0000 7f80a97fa700 1 -- 172.21.15.40:0/1122719222 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f80b00119e0 con 0x5599a5befe20 2023-07-18T03:41:54.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.265+0000 7f80d3cdab80 1 -- 172.21.15.40:0/1122719222 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599a5a9a4f0 con 0x5599a5befe20 2023-07-18T03:41:54.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.265+0000 7f80a97fa700 1 -- 172.21.15.40:0/1122719222 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80b0009e60 con 0x5599a5befe20 2023-07-18T03:41:54.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.265+0000 7f80d3cdab80 10 client.16959 fetch_fsmap learned FSMap version 72 2023-07-18T03:41:54.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.265+0000 7f80d3cdab80 1 -- 172.21.15.40:0/1122719222 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5599a5957c60 con 0x5599a5befe20 2023-07-18T03:41:54.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.266+0000 7f80a97fa700 1 -- 172.21.15.40:0/1122719222 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5599a5a9a4f0 con 0x5599a5befe20 2023-07-18T03:41:54.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.266+0000 7f80a97fa700 1 -- 172.21.15.40:0/1122719222 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 72) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f80b000c070 con 0x5599a5befe20 2023-07-18T03:41:54.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.266+0000 7f80d3cdab80 10 client.16959 fetch_fsmap finished waiting for FSMap version 72 2023-07-18T03:41:54.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.266+0000 7f80d3cdab80 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:41:54.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.266+0000 7f80d3cdab80 20 client.16959 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:54.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.266+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/1122719222 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5599a5c05040 0x5599a5c07450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.266+0000 7f80d3cdab80 4 client.16959 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:41:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.266+0000 7f80d3cdab80 1 -- 172.21.15.40:0/1122719222 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599a5b39eb0 con 0x5599a5c05040 2023-07-18T03:41:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.266+0000 7f80c58f0700 1 --2- 172.21.15.40:0/1122719222 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5599a5c05040 0x5599a5c07450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.266+0000 7f80a97fa700 10 client.16959 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.266+0000 7f80c58f0700 1 --2- 172.21.15.40:0/1122719222 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5599a5c05040 0x5599a5c07450 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8cb6f0ee4a581ab server_cookie=edf8043433db9943 in_seq=0 out_seq=0 2023-07-18T03:41:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.267+0000 7f80a97fa700 1 -- 172.21.15.40:0/1122719222 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5599a5b39eb0 con 0x5599a5c05040 2023-07-18T03:41:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.267+0000 7f80a97fa700 10 client.16959 handle_command_reply: tid=0 2023-07-18T03:41:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.267+0000 7f80d3cdab80 1 client.16959 shutdown 2023-07-18T03:41:54.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.268+0000 7f80d3cdab80 1 -- 172.21.15.40:0/1122719222 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5befe20 msgr2=0x5599a5bf9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.268+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/1122719222 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5befe20 0x5599a5bf9700 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f80b000e620 tx=0x7f80b000e930 comp rx=0 tx=0).stop 2023-07-18T03:41:54.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.268+0000 7f80d3cdab80 1 -- 172.21.15.40:0/1122719222 shutdown_connections 2023-07-18T03:41:54.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.268+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/1122719222 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5599a5c05040 0x5599a5c07450 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.268+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/1122719222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5bf0c70 0x5599a5c00740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.268+0000 7f80a97fa700 0 client.16959 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:54.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.268+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/1122719222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bef4d0 0x5599a5bf91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.268+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/1122719222 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5befe20 0x5599a5bf9700 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.268+0000 7f80d3cdab80 1 -- 172.21.15.40:0/1122719222 >> 172.21.15.40:0/1122719222 conn(0x5599a5be4da0 msgr2=0x5599a5be98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.268+0000 7f80d3cdab80 1 -- 172.21.15.40:0/1122719222 shutdown_connections 2023-07-18T03:41:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.268+0000 7f80d3cdab80 1 -- 172.21.15.40:0/1122719222 wait complete. 2023-07-18T03:41:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.268+0000 7f80d3cdab80 20 client.16959 trim_cache size 0 max 16384 2023-07-18T03:41:54.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.295+0000 7f80d3cdab80 1 Processor -- start 2023-07-18T03:41:54.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.295+0000 7f80d3cdab80 1 -- start start 2023-07-18T03:41:54.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.296+0000 7f80d3cdab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5bef4d0 0x5599a5c01030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.296+0000 7f80d3cdab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5befe20 0x5599a5c00700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.296+0000 7f80d3cdab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bf0c70 0x5599a5bf3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.296+0000 7f80d3cdab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a5bf8d30 con 0x5599a5befe20 2023-07-18T03:41:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.296+0000 7f80d3cdab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a5bf01f0 con 0x5599a5bf0c70 2023-07-18T03:41:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.296+0000 7f80d3cdab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a5c023b0 con 0x5599a5bef4d0 2023-07-18T03:41:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.296+0000 7f80c50ef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5befe20 0x5599a5c00700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.296+0000 7f80c50ef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5befe20 0x5599a5c00700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33502/0 (socket says 172.21.15.40:33502) 2023-07-18T03:41:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.296+0000 7f80c50ef700 1 -- 172.21.15.40:0/2938843906 learned_addr learned my addr 172.21.15.40:0/2938843906 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.296+0000 7f80c60f1700 1 --2- 172.21.15.40:0/2938843906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bf0c70 0x5599a5bf3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.296+0000 7f80c58f0700 1 --2- 172.21.15.40:0/2938843906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5bef4d0 0x5599a5c01030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.296+0000 7f80c50ef700 1 -- 172.21.15.40:0/2938843906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5bef4d0 msgr2=0x5599a5c01030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.296+0000 7f80c50ef700 1 --2- 172.21.15.40:0/2938843906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5bef4d0 0x5599a5c01030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.296+0000 7f80c50ef700 1 -- 172.21.15.40:0/2938843906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bf0c70 msgr2=0x5599a5bf3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.296+0000 7f80c50ef700 1 --2- 172.21.15.40:0/2938843906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bf0c70 0x5599a5bf3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.296+0000 7f80c50ef700 1 -- 172.21.15.40:0/2938843906 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599a5995250 con 0x5599a5befe20 2023-07-18T03:41:54.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.296+0000 7f80c50ef700 1 --2- 172.21.15.40:0/2938843906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5befe20 0x5599a5c00700 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f80b0000c00 tx=0x7f80b0010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd8ae26b5c0bf01 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:54.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.297+0000 7f80b4dac700 1 -- 172.21.15.40:0/2938843906 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b0005240 con 0x5599a5befe20 2023-07-18T03:41:54.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.297+0000 7f80b4dac700 1 -- 172.21.15.40:0/2938843906 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80b001a480 con 0x5599a5befe20 2023-07-18T03:41:54.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.297+0000 7f80b4dac700 1 -- 172.21.15.40:0/2938843906 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b00103d0 con 0x5599a5befe20 2023-07-18T03:41:54.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.298+0000 7f80d3cdab80 1 -- 172.21.15.40:0/2938843906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5befe20 msgr2=0x5599a5c00700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.298+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/2938843906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5befe20 0x5599a5c00700 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f80b0000c00 tx=0x7f80b0010fa0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.298+0000 7f80d3cdab80 1 -- 172.21.15.40:0/2938843906 shutdown_connections 2023-07-18T03:41:54.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.298+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/2938843906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bf0c70 0x5599a5bf3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.298+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/2938843906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5bef4d0 0x5599a5c01030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.298+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/2938843906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5befe20 0x5599a5c00700 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.298+0000 7f80d3cdab80 1 -- 172.21.15.40:0/2938843906 >> 172.21.15.40:0/2938843906 conn(0x5599a5be4da0 msgr2=0x5599a5bf7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:54.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.298+0000 7f80d3cdab80 1 -- 172.21.15.40:0/2938843906 shutdown_connections 2023-07-18T03:41:54.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.298+0000 7f80d3cdab80 1 -- 172.21.15.40:0/2938843906 wait complete. 2023-07-18T03:41:54.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.299+0000 7f80d3cdab80 1 -- start start 2023-07-18T03:41:54.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.299+0000 7f80d3cdab80 1 Processor -- start 2023-07-18T03:41:54.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.299+0000 7f80d3cdab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bef4d0 0x5599a5c001e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.299+0000 7f80d3cdab80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5befe20 0x5599a5c00720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.299+0000 7f80d3cdab80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5bf0c70 0x5599a5bfcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.299+0000 7f80d3cdab80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a5bfd4a0 con 0x5599a5bf0c70 2023-07-18T03:41:54.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.299+0000 7f80d3cdab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a5bfd610 con 0x5599a5bef4d0 2023-07-18T03:41:54.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.299+0000 7f80d3cdab80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a5bfd7d0 con 0x5599a5befe20 2023-07-18T03:41:54.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.299+0000 7f80d3cdab80 10 client.0 _ll_register_callbacks cb 0x7f80b0009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:54.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.300+0000 7f80c60f1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5bf0c70 0x5599a5bfcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.300+0000 7f80c60f1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5bf0c70 0x5599a5bfcdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33508/0 (socket says 172.21.15.40:33508) 2023-07-18T03:41:54.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.300+0000 7f80c60f1700 1 -- 172.21.15.40:0/4166994478 learned_addr learned my addr 172.21.15.40:0/4166994478 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:54.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.300+0000 7f80c58f0700 1 --2- 172.21.15.40:0/4166994478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bef4d0 0x5599a5c001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.300+0000 7f80c60f1700 1 -- 172.21.15.40:0/4166994478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5befe20 msgr2=0x5599a5c00720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:54.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.300+0000 7f80c60f1700 1 --2- 172.21.15.40:0/4166994478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5befe20 0x5599a5c00720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.300+0000 7f80c60f1700 1 -- 172.21.15.40:0/4166994478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bef4d0 msgr2=0x5599a5c001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:54.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.300+0000 7f80c60f1700 1 --2- 172.21.15.40:0/4166994478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bef4d0 0x5599a5c001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.300+0000 7f80c60f1700 1 -- 172.21.15.40:0/4166994478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599a5957c60 con 0x5599a5bf0c70 2023-07-18T03:41:54.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.300+0000 7f80c60f1700 1 --2- 172.21.15.40:0/4166994478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5bf0c70 0x5599a5bfcdc0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f80b80b9050 tx=0x7f80b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e569882153203d server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:54.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.300+0000 7f80a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:41:54.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.301+0000 7f80a97fa700 1 -- 172.21.15.40:0/4166994478 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b80a1840 con 0x5599a5bf0c70 2023-07-18T03:41:54.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.301+0000 7f80a97fa700 1 -- 172.21.15.40:0/4166994478 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f80b80a13b0 con 0x5599a5bf0c70 2023-07-18T03:41:54.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.301+0000 7f80d3cdab80 1 -- 172.21.15.40:0/4166994478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80b8097790 con 0x5599a5bf0c70 2023-07-18T03:41:54.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.301+0000 7f80a97fa700 1 -- 172.21.15.40:0/4166994478 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80b80a1570 con 0x5599a5bf0c70 2023-07-18T03:41:54.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.301+0000 7f80d3cdab80 10 client.16965 fetch_fsmap learned FSMap version 72 2023-07-18T03:41:54.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.301+0000 7f80d3cdab80 1 -- 172.21.15.40:0/4166994478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5599a596d2b0 con 0x5599a5bf0c70 2023-07-18T03:41:54.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.301+0000 7f80a97fa700 1 -- 172.21.15.40:0/4166994478 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80b8097790 con 0x5599a5bf0c70 2023-07-18T03:41:54.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.302+0000 7f80a97fa700 1 -- 172.21.15.40:0/4166994478 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 72) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f80b80983e0 con 0x5599a5bf0c70 2023-07-18T03:41:54.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.302+0000 7f80d3cdab80 10 client.16965 fetch_fsmap finished waiting for FSMap version 72 2023-07-18T03:41:54.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.302+0000 7f80d3cdab80 10 client.16965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:41:54.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.302+0000 7f80d3cdab80 20 client.16965 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:54.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.302+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/4166994478 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5599a5c5a420 0x5599a5c5c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.302+0000 7f80d3cdab80 4 client.16965 mds_command: new command op to 24527 tid=0[{"prefix": "scrub status"}] 2023-07-18T03:41:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.302+0000 7f80d3cdab80 1 -- 172.21.15.40:0/4166994478 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5599a5afc050 con 0x5599a5c5a420 2023-07-18T03:41:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.302+0000 7f80c58f0700 1 --2- 172.21.15.40:0/4166994478 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5599a5c5a420 0x5599a5c5c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.303+0000 7f80a97fa700 10 client.16965 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.303+0000 7f80c58f0700 1 --2- 172.21.15.40:0/4166994478 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5599a5c5a420 0x5599a5c5c850 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eec7f2e9572425f server_cookie=b6c0f0090e9cc640 in_seq=0 out_seq=0 2023-07-18T03:41:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.303+0000 7f80a97fa700 1 -- 172.21.15.40:0/4166994478 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5599a5afc050 con 0x5599a5c5a420 2023-07-18T03:41:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.303+0000 7f80a97fa700 10 client.16965 handle_command_reply: tid=0 2023-07-18T03:41:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.303+0000 7f80d3cdab80 1 client.16965 shutdown 2023-07-18T03:41:54.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.303+0000 7f80d3cdab80 1 -- 172.21.15.40:0/4166994478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5bf0c70 msgr2=0x5599a5bfcdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.303+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/4166994478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5bf0c70 0x5599a5bfcdc0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f80b80b9050 tx=0x7f80b8004ad0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.303+0000 7f80d3cdab80 1 -- 172.21.15.40:0/4166994478 shutdown_connections 2023-07-18T03:41:54.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.303+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/4166994478 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x5599a5c5a420 0x5599a5c5c850 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.303+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/4166994478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5599a5bf0c70 0x5599a5bfcdc0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.303+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/4166994478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599a5bef4d0 0x5599a5c001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.303+0000 7f80a97fa700 0 client.16965 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:54.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.303+0000 7f80d3cdab80 1 --2- 172.21.15.40:0/4166994478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5599a5befe20 0x5599a5c00720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.303+0000 7f80d3cdab80 1 -- 172.21.15.40:0/4166994478 >> 172.21.15.40:0/4166994478 conn(0x5599a5be4da0 msgr2=0x5599a5c01030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:54.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.303+0000 7f80d3cdab80 1 -- 172.21.15.40:0/4166994478 shutdown_connections 2023-07-18T03:41:54.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.303+0000 7f80d3cdab80 1 -- 172.21.15.40:0/4166994478 wait complete. 2023-07-18T03:41:54.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.304+0000 7f80d3cdab80 20 client.16965 trim_cache size 0 max 16384 2023-07-18T03:41:54.305 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:41:54.306 INFO:teuthology.orchestra.run.smithi040.stdout: "status": "no active scrubs running", 2023-07-18T03:41:54.306 INFO:teuthology.orchestra.run.smithi040.stdout: "scrubs": {} 2023-07-18T03:41:54.306 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:41:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.304+0000 7f80c7b54700 1 -- 172.21.15.40:0/1082594693 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f80ac0a12a0 msgr2=0x7f80ac0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.304+0000 7f80c7b54700 1 --2- 172.21.15.40:0/1082594693 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f80ac0a12a0 0x7f80ac0a3750 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0098e0 tx=0x7f80bc000bc0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.304+0000 7f80c7b54700 1 -- 172.21.15.40:0/1082594693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80c0103670 msgr2=0x7f80c01a3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.304+0000 7f80c7b54700 1 --2- 172.21.15.40:0/1082594693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80c0103670 0x7f80c01a3440 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f80b800b970 tx=0x7f80b800bd30 comp rx=0 tx=0).stop 2023-07-18T03:41:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.304+0000 7f80c7b54700 1 -- 172.21.15.40:0/1082594693 shutdown_connections 2023-07-18T03:41:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.304+0000 7f80c7b54700 1 --2- 172.21.15.40:0/1082594693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80c0103670 0x7f80c01a3440 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.304+0000 7f80c7b54700 1 --2- 172.21.15.40:0/1082594693 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f80ac0a12a0 0x7f80ac0a3750 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.304+0000 7f80c7b54700 1 --2- 172.21.15.40:0/1082594693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80c00fe350 0x7f80c019be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.304+0000 7f80c7b54700 1 --2- 172.21.15.40:0/1082594693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80c0100cb0 0x7f80c019c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.304+0000 7f80c7b54700 1 -- 172.21.15.40:0/1082594693 >> 172.21.15.40:0/1082594693 conn(0x7f80c00fa000 msgr2=0x7f80c01019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.305+0000 7f80c7b54700 1 -- 172.21.15.40:0/1082594693 shutdown_connections 2023-07-18T03:41:54.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.305+0000 7f80c7b54700 1 -- 172.21.15.40:0/1082594693 wait complete. 2023-07-18T03:41:54.311 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:53 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:41:54.313 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-18T03:41:54.313 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:41:54.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:53 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:41:54.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:53 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:41:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.451+0000 7f469fe6f700 1 -- 172.21.15.40:0/785768059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46980ffbc0 msgr2=0x7f4698100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.451+0000 7f469fe6f700 1 --2- 172.21.15.40:0/785768059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46980ffbc0 0x7f4698100010 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4694009990 tx=0x7f4694009ca0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.451+0000 7f469fe6f700 1 -- 172.21.15.40:0/785768059 shutdown_connections 2023-07-18T03:41:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.451+0000 7f469fe6f700 1 --2- 172.21.15.40:0/785768059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4698100550 0x7f4698106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.451+0000 7f469fe6f700 1 --2- 172.21.15.40:0/785768059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46980fe9d0 0x7f46980fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.451+0000 7f469fe6f700 1 --2- 172.21.15.40:0/785768059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46980ffbc0 0x7f4698100010 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.451+0000 7f469fe6f700 1 -- 172.21.15.40:0/785768059 >> 172.21.15.40:0/785768059 conn(0x7f46980f9fd0 msgr2=0x7f46980fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.451+0000 7f469fe6f700 1 -- 172.21.15.40:0/785768059 shutdown_connections 2023-07-18T03:41:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.451+0000 7f469fe6f700 1 -- 172.21.15.40:0/785768059 wait complete. 2023-07-18T03:41:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469fe6f700 1 Processor -- start 2023-07-18T03:41:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469fe6f700 1 -- start start 2023-07-18T03:41:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469fe6f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46980fe9d0 0x7f4698195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469fe6f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46980ffbc0 0x7f4698195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469fe6f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4698100550 0x7f469819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469fe6f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469819d380 con 0x7f4698100550 2023-07-18T03:41:54.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469fe6f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469819d4c0 con 0x7f46980fe9d0 2023-07-18T03:41:54.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469fe6f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469819d650 con 0x7f46980ffbc0 2023-07-18T03:41:54.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469e40c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4698100550 0x7f469819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469e40c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4698100550 0x7f469819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33524/0 (socket says 172.21.15.40:33524) 2023-07-18T03:41:54.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469e40c700 1 -- 172.21.15.40:0/3368095513 learned_addr learned my addr 172.21.15.40:0/3368095513 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469dc0b700 1 --2- 172.21.15.40:0/3368095513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46980fe9d0 0x7f4698195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469d40a700 1 --2- 172.21.15.40:0/3368095513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46980ffbc0 0x7f4698195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469e40c700 1 -- 172.21.15.40:0/3368095513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46980ffbc0 msgr2=0x7f4698195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469e40c700 1 --2- 172.21.15.40:0/3368095513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46980ffbc0 0x7f4698195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469e40c700 1 -- 172.21.15.40:0/3368095513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46980fe9d0 msgr2=0x7f4698195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469e40c700 1 --2- 172.21.15.40:0/3368095513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46980fe9d0 0x7f4698195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.452+0000 7f469e40c700 1 -- 172.21.15.40:0/3368095513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46940095f0 con 0x7f4698100550 2023-07-18T03:41:54.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.453+0000 7f469e40c700 1 --2- 172.21.15.40:0/3368095513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4698100550 0x7f469819cdb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4690007ee0 tx=0x7f469000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:54.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.453+0000 7f468effd700 1 -- 172.21.15.40:0/3368095513 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690017070 con 0x7f4698100550 2023-07-18T03:41:54.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.453+0000 7f469fe6f700 1 -- 172.21.15.40:0/3368095513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469819d930 con 0x7f4698100550 2023-07-18T03:41:54.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.453+0000 7f468effd700 1 -- 172.21.15.40:0/3368095513 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f469000f040 con 0x7f4698100550 2023-07-18T03:41:54.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.453+0000 7f468effd700 1 -- 172.21.15.40:0/3368095513 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690005580 con 0x7f4698100550 2023-07-18T03:41:54.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.453+0000 7f469fe6f700 1 -- 172.21.15.40:0/3368095513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469819de40 con 0x7f4698100550 2023-07-18T03:41:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.454+0000 7f469fe6f700 1 -- 172.21.15.40:0/3368095513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469810b3d0 con 0x7f4698100550 2023-07-18T03:41:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.455+0000 7f468effd700 1 -- 172.21.15.40:0/3368095513 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f469000db20 con 0x7f4698100550 2023-07-18T03:41:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.455+0000 7f468effd700 1 --2- 172.21.15.40:0/3368095513 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f46840a11a0 0x7f46840a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.456+0000 7f468effd700 1 -- 172.21.15.40:0/3368095513 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f46900cc6c0 con 0x7f4698100550 2023-07-18T03:41:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.456+0000 7f469dc0b700 1 --2- 172.21.15.40:0/3368095513 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f46840a11a0 0x7f46840a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.456+0000 7f469dc0b700 1 --2- 172.21.15.40:0/3368095513 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f46840a11a0 0x7f46840a3650 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f46980ffa20 tx=0x7f4688009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:54.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.459+0000 7f468effd700 1 -- 172.21.15.40:0/3368095513 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4690095df0 con 0x7f4698100550 2023-07-18T03:41:54.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.710+0000 7f469fe6f700 1 -- 172.21.15.40:0/3368095513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f469819e510 con 0x7f4698100550 2023-07-18T03:41:54.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.711+0000 7f468effd700 1 -- 172.21.15.40:0/3368095513 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+10922 (secure 0 0 0) 0x7f4690099410 con 0x7f4698100550 2023-07-18T03:41:54.706 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:41:54.706 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":72,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:29:26.718789+0000","modified":"2023-07-18T03:41:52.964776+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24523,"mds_2":24482,"mds_3":24452,"mds_4":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14601":{"gid":14601,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1818167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1818167554},{"type":"v1","addr":"172.21.15.40:6839","nonce":1818167554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14613":{"gid":14613,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/3953074960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":3953074960},{"type":"v1","addr":"172.21.15.40:6841","nonce":3953074960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":52,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/3007039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3007039043},{"type":"v1","addr":"172.21.15.64:6839","nonce":3007039043}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25972":{"gid":25972,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3831344228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3831344228},{"type":"v1","addr":"172.21.15.134:6837","nonce":3831344228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:41:54.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.714+0000 7f469fe6f700 1 -- 172.21.15.40:0/3368095513 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f46840a11a0 msgr2=0x7f46840a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.714+0000 7f469fe6f700 1 --2- 172.21.15.40:0/3368095513 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f46840a11a0 0x7f46840a3650 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f46980ffa20 tx=0x7f4688009260 comp rx=0 tx=0).stop 2023-07-18T03:41:54.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.714+0000 7f469fe6f700 1 -- 172.21.15.40:0/3368095513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4698100550 msgr2=0x7f469819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.714+0000 7f469fe6f700 1 --2- 172.21.15.40:0/3368095513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4698100550 0x7f469819cdb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4690007ee0 tx=0x7f469000d3b0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.714+0000 7f469fe6f700 1 -- 172.21.15.40:0/3368095513 shutdown_connections 2023-07-18T03:41:54.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.714+0000 7f469fe6f700 1 --2- 172.21.15.40:0/3368095513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4698100550 0x7f469819cdb0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.714+0000 7f469fe6f700 1 --2- 172.21.15.40:0/3368095513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46980fe9d0 0x7f4698195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.714+0000 7f469fe6f700 1 --2- 172.21.15.40:0/3368095513 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f46840a11a0 0x7f46840a3650 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.714+0000 7f469fe6f700 1 --2- 172.21.15.40:0/3368095513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46980ffbc0 0x7f4698195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.714+0000 7f469fe6f700 1 -- 172.21.15.40:0/3368095513 >> 172.21.15.40:0/3368095513 conn(0x7f46980f9fd0 msgr2=0x7f46980fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:54.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.714+0000 7f469fe6f700 1 -- 172.21.15.40:0/3368095513 shutdown_connections 2023-07-18T03:41:54.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.714+0000 7f469fe6f700 1 -- 172.21.15.40:0/3368095513 wait complete. 2023-07-18T03:41:54.711 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 72 2023-07-18T03:41:54.721 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-18T03:41:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.860+0000 7fe09234d700 1 -- 172.21.15.40:0/2099852022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe08c0fe9e0 msgr2=0x7fe08c0fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.860+0000 7fe09234d700 1 --2- 172.21.15.40:0/2099852022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe08c0fe9e0 0x7fe08c0fedf0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe07c009990 tx=0x7fe07c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.860+0000 7fe09234d700 1 -- 172.21.15.40:0/2099852022 shutdown_connections 2023-07-18T03:41:54.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.860+0000 7fe09234d700 1 --2- 172.21.15.40:0/2099852022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe08c100560 0x7fe08c106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.860+0000 7fe09234d700 1 --2- 172.21.15.40:0/2099852022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe08c0fe9e0 0x7fe08c0fedf0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.860+0000 7fe09234d700 1 --2- 172.21.15.40:0/2099852022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe08c0ffbd0 0x7fe08c100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.860+0000 7fe09234d700 1 -- 172.21.15.40:0/2099852022 >> 172.21.15.40:0/2099852022 conn(0x7fe08c0fa000 msgr2=0x7fe08c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:54.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.860+0000 7fe09234d700 1 -- 172.21.15.40:0/2099852022 shutdown_connections 2023-07-18T03:41:54.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.860+0000 7fe09234d700 1 -- 172.21.15.40:0/2099852022 wait complete. 2023-07-18T03:41:54.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.860+0000 7fe09234d700 1 Processor -- start 2023-07-18T03:41:54.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe09234d700 1 -- start start 2023-07-18T03:41:54.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe09234d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe08c0fe9e0 0x7fe08c194740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe09234d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe08c0ffbd0 0x7fe08c194c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe09234d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe08c100560 0x7fe08c19bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe09234d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c19c2c0 con 0x7fe08c0ffbd0 2023-07-18T03:41:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe09234d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c19c400 con 0x7fe08c100560 2023-07-18T03:41:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe09234d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c19c590 con 0x7fe08c0fe9e0 2023-07-18T03:41:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe08b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe08c0ffbd0 0x7fe08c194c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe08b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe08c0ffbd0 0x7fe08c194c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33536/0 (socket says 172.21.15.40:33536) 2023-07-18T03:41:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe08b7fe700 1 -- 172.21.15.40:0/287744252 learned_addr learned my addr 172.21.15.40:0/287744252 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:54.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe08bfff700 1 --2- 172.21.15.40:0/287744252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe08c0fe9e0 0x7fe08c194740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe0908ea700 1 --2- 172.21.15.40:0/287744252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe08c100560 0x7fe08c19bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe08b7fe700 1 -- 172.21.15.40:0/287744252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe08c0fe9e0 msgr2=0x7fe08c194740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe08b7fe700 1 --2- 172.21.15.40:0/287744252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe08c0fe9e0 0x7fe08c194740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe08b7fe700 1 -- 172.21.15.40:0/287744252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe08c100560 msgr2=0x7fe08c19bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe08b7fe700 1 --2- 172.21.15.40:0/287744252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe08c100560 0x7fe08c19bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe08b7fe700 1 -- 172.21.15.40:0/287744252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07c0095f0 con 0x7fe08c0ffbd0 2023-07-18T03:41:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.861+0000 7fe08b7fe700 1 --2- 172.21.15.40:0/287744252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe08c0ffbd0 0x7fe08c194c80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe080007910 tx=0x7fe080007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.862+0000 7fe0897fa700 1 -- 172.21.15.40:0/287744252 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe080012070 con 0x7fe08c0ffbd0 2023-07-18T03:41:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.862+0000 7fe09234d700 1 -- 172.21.15.40:0/287744252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08c0676d0 con 0x7fe08c0ffbd0 2023-07-18T03:41:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.862+0000 7fe0897fa700 1 -- 172.21.15.40:0/287744252 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe08000d810 con 0x7fe08c0ffbd0 2023-07-18T03:41:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.862+0000 7fe0897fa700 1 -- 172.21.15.40:0/287744252 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe080005350 con 0x7fe08c0ffbd0 2023-07-18T03:41:54.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.862+0000 7fe09234d700 1 -- 172.21.15.40:0/287744252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08c067be0 con 0x7fe08c0ffbd0 2023-07-18T03:41:54.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.864+0000 7fe0897fa700 1 -- 172.21.15.40:0/287744252 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe080016070 con 0x7fe08c0ffbd0 2023-07-18T03:41:54.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.865+0000 7fe0897fa700 1 --2- 172.21.15.40:0/287744252 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0740a1150 0x7fe0740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.865+0000 7fe08bfff700 1 --2- 172.21.15.40:0/287744252 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0740a1150 0x7fe0740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.865+0000 7fe0897fa700 1 -- 172.21.15.40:0/287744252 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe0800cd890 con 0x7fe08c0ffbd0 2023-07-18T03:41:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.865+0000 7fe08bfff700 1 --2- 172.21.15.40:0/287744252 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0740a1150 0x7fe0740a3600 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe07c006010 tx=0x7fe07c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe09e4d3b80 1 Processor -- start 2023-07-18T03:41:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe09e4d3b80 1 -- start start 2023-07-18T03:41:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe09e4d3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f88e0 0x5638819f8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe09e4d3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f9230 0x5638819f9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe09e4d3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x5638819fa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe09e4d3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638818f9cb0 con 0x5638819fa080 2023-07-18T03:41:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe09e4d3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x56388194ca40 con 0x5638819f88e0 2023-07-18T03:41:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe09e4d3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563881945ed0 con 0x5638819f9230 2023-07-18T03:41:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe0908ea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x5638819fa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe08b7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f9230 0x5638819f9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe0908ea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x5638819fa4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33546/0 (socket says 172.21.15.40:33546) 2023-07-18T03:41:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe0908ea700 1 -- 172.21.15.40:0/1251320348 learned_addr learned my addr 172.21.15.40:0/1251320348 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:54.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe08bfff700 1 --2- 172.21.15.40:0/1251320348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f88e0 0x5638819f8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe08b7fe700 1 -- 172.21.15.40:0/1251320348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f88e0 msgr2=0x5638819f8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:54.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe08b7fe700 1 --2- 172.21.15.40:0/1251320348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f88e0 0x5638819f8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe08b7fe700 1 -- 172.21.15.40:0/1251320348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 msgr2=0x5638819fa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:54.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe08b7fe700 1 --2- 172.21.15.40:0/1251320348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x5638819fa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe08b7fe700 1 -- 172.21.15.40:0/1251320348 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563881773de0 con 0x5638819f9230 2023-07-18T03:41:54.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.866+0000 7fe0908ea700 1 --2- 172.21.15.40:0/1251320348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x5638819fa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:54.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.867+0000 7fe08b7fe700 1 --2- 172.21.15.40:0/1251320348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f9230 0x5638819f9af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe08009a920 tx=0x7fe0800047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3127bd558b6030ef server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:54.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.867+0000 7fe0727fc700 1 -- 172.21.15.40:0/1251320348 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08009ec50 con 0x5638819f9230 2023-07-18T03:41:54.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.867+0000 7fe0727fc700 1 -- 172.21.15.40:0/1251320348 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0800a3070 con 0x5638819f9230 2023-07-18T03:41:54.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.867+0000 7fe0727fc700 1 -- 172.21.15.40:0/1251320348 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0800a2be0 con 0x5638819f9230 2023-07-18T03:41:54.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.867+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/1251320348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f9230 msgr2=0x5638819f9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.867+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/1251320348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f9230 0x5638819f9af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe08009a920 tx=0x7fe0800047d0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.867+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/1251320348 shutdown_connections 2023-07-18T03:41:54.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.867+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/1251320348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x5638819fa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.867+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/1251320348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f88e0 0x5638819f8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.867+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/1251320348 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f9230 0x5638819f9af0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.867+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/1251320348 >> 172.21.15.40:0/1251320348 conn(0x5638819e9da0 msgr2=0x5638819ef1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:54.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.868+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/1251320348 shutdown_connections 2023-07-18T03:41:54.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.868+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/1251320348 wait complete. 2023-07-18T03:41:54.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.868+0000 7fe09e4d3b80 1 -- start start 2023-07-18T03:41:54.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.868+0000 7fe09e4d3b80 1 Processor -- start 2023-07-18T03:41:54.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.868+0000 7fe09e4d3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f88e0 0x563881a02580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.868+0000 7fe09e4d3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f9230 0x563881a02ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.868+0000 7fe09e4d3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x563881a09b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.868+0000 7fe09e4d3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638818e5ac0 con 0x5638819fa080 2023-07-18T03:41:54.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.868+0000 7fe09e4d3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638818e57c0 con 0x5638819f88e0 2023-07-18T03:41:54.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.868+0000 7fe09e4d3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638817c9920 con 0x5638819f9230 2023-07-18T03:41:54.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.868+0000 7fe09e4d3b80 10 client.0 _ll_register_callbacks cb 0x5638818e3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:54.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe0908ea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x563881a09b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe08b7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f9230 0x563881a02ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe0908ea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x563881a09b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33552/0 (socket says 172.21.15.40:33552) 2023-07-18T03:41:54.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe0908ea700 1 -- 172.21.15.40:0/3422921085 learned_addr learned my addr 172.21.15.40:0/3422921085 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:54.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe08bfff700 1 --2- 172.21.15.40:0/3422921085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f88e0 0x563881a02580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe0908ea700 1 -- 172.21.15.40:0/3422921085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f9230 msgr2=0x563881a02ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:54.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe0908ea700 1 --2- 172.21.15.40:0/3422921085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f9230 0x563881a02ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe0908ea700 1 -- 172.21.15.40:0/3422921085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f88e0 msgr2=0x563881a02580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:54.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe0908ea700 1 --2- 172.21.15.40:0/3422921085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f88e0 0x563881a02580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe0908ea700 1 -- 172.21.15.40:0/3422921085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe084011330 con 0x5638819fa080 2023-07-18T03:41:54.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe08b7fe700 1 --2- 172.21.15.40:0/3422921085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f9230 0x563881a02ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:54.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe0908ea700 1 --2- 172.21.15.40:0/3422921085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x563881a09b00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe084014370 tx=0x7fe084014680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c18d2da1313839b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:54.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe06b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:41:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe06b7fe700 1 -- 172.21.15.40:0/3422921085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08400e040 con 0x5638819fa080 2023-07-18T03:41:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe06b7fe700 1 -- 172.21.15.40:0/3422921085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe08401a070 con 0x5638819fa080 2023-07-18T03:41:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/3422921085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56388189f4f0 con 0x5638819fa080 2023-07-18T03:41:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe06b7fe700 1 -- 172.21.15.40:0/3422921085 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0840092d0 con 0x5638819fa080 2023-07-18T03:41:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe09e4d3b80 10 client.16983 fetch_fsmap learned FSMap version 72 2023-07-18T03:41:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.869+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/3422921085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x7fe08c067be0 con 0x5638819fa080 2023-07-18T03:41:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.870+0000 7fe06b7fe700 1 -- 172.21.15.40:0/3422921085 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56388189f4f0 con 0x5638819fa080 2023-07-18T03:41:54.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.870+0000 7fe06b7fe700 1 -- 172.21.15.40:0/3422921085 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 72) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fe084028050 con 0x5638819fa080 2023-07-18T03:41:54.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.870+0000 7fe09e4d3b80 10 client.16983 fetch_fsmap finished waiting for FSMap version 72 2023-07-18T03:41:54.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.870+0000 7fe09e4d3b80 10 client.16983 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2023-07-18T03:41:54.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.870+0000 7fe09e4d3b80 20 client.16983 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:54.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.870+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/3422921085 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x563881a0e460 0x563881a10870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.870+0000 7fe09e4d3b80 4 client.16983 mds_command: new command op to 24442 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:41:54.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.870+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/3422921085 --> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56388193eeb0 con 0x563881a0e460 2023-07-18T03:41:54.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.870+0000 7fe08bfff700 1 --2- 172.21.15.40:0/3422921085 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x563881a0e460 0x563881a10870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.870+0000 7fe06b7fe700 10 client.16983 ms_handle_connect on v2:172.21.15.134:6832/2345264306 2023-07-18T03:41:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.870+0000 7fe08bfff700 1 --2- 172.21.15.40:0/3422921085 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x563881a0e460 0x563881a10870 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=52941006a30150ae server_cookie=8227e11807baf8e in_seq=0 out_seq=0 2023-07-18T03:41:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.871+0000 7fe06b7fe700 1 -- 172.21.15.40:0/3422921085 <== mds.4 v2:172.21.15.134:6832/2345264306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56388193eeb0 con 0x563881a0e460 2023-07-18T03:41:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.871+0000 7fe06b7fe700 10 client.16983 handle_command_reply: tid=0 2023-07-18T03:41:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.871+0000 7fe09e4d3b80 1 client.16983 shutdown 2023-07-18T03:41:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.871+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/3422921085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 msgr2=0x563881a09b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.871+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/3422921085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x563881a09b00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe084014370 tx=0x7fe084014680 comp rx=0 tx=0).stop 2023-07-18T03:41:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.871+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/3422921085 shutdown_connections 2023-07-18T03:41:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.871+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/3422921085 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x563881a0e460 0x563881a10870 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.871+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/3422921085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x563881a09b00 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.871+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/3422921085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f88e0 0x563881a02580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.871+0000 7fe06b7fe700 0 client.16983 ms_handle_reset on v2:172.21.15.134:6832/2345264306 2023-07-18T03:41:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.871+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/3422921085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f9230 0x563881a02ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.871+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/3422921085 >> 172.21.15.40:0/3422921085 conn(0x5638819e9da0 msgr2=0x5638819ee8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.871+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/3422921085 shutdown_connections 2023-07-18T03:41:54.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.871+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/3422921085 wait complete. 2023-07-18T03:41:54.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.871+0000 7fe09e4d3b80 20 client.16983 trim_cache size 0 max 16384 2023-07-18T03:41:54.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.885+0000 7fe09e4d3b80 1 Processor -- start 2023-07-18T03:41:54.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.885+0000 7fe09e4d3b80 1 -- start start 2023-07-18T03:41:54.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe09e4d3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f88e0 0x5638819f8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe09e4d3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f9230 0x563881a09a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe09e4d3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x563881a01260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe09e4d3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638819f0a00 con 0x5638819fa080 2023-07-18T03:41:54.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe09e4d3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563881a00220 con 0x5638819f9230 2023-07-18T03:41:54.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe09e4d3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563881a0e1b0 con 0x5638819f88e0 2023-07-18T03:41:54.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe0908ea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x563881a01260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe0908ea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x563881a01260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33560/0 (socket says 172.21.15.40:33560) 2023-07-18T03:41:54.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe08bfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f88e0 0x5638819f8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe0908ea700 1 -- 172.21.15.40:0/2078633549 learned_addr learned my addr 172.21.15.40:0/2078633549 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:54.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe08b7fe700 1 --2- 172.21.15.40:0/2078633549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f9230 0x563881a09a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe0908ea700 1 -- 172.21.15.40:0/2078633549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f88e0 msgr2=0x5638819f8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe0908ea700 1 --2- 172.21.15.40:0/2078633549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f88e0 0x5638819f8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe0908ea700 1 -- 172.21.15.40:0/2078633549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f9230 msgr2=0x563881a09a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe0908ea700 1 --2- 172.21.15.40:0/2078633549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f9230 0x563881a09a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe0908ea700 1 -- 172.21.15.40:0/2078633549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56388175cc60 con 0x5638819fa080 2023-07-18T03:41:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe08b7fe700 1 --2- 172.21.15.40:0/2078633549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f9230 0x563881a09a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe08bfff700 1 --2- 172.21.15.40:0/2078633549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f88e0 0x5638819f8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.886+0000 7fe0908ea700 1 --2- 172.21.15.40:0/2078633549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x563881a01260 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe0840115e0 tx=0x7fe084009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430a40e9a031da4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.887+0000 7fe072ffd700 1 -- 172.21.15.40:0/2078633549 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08400e030 con 0x5638819fa080 2023-07-18T03:41:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.887+0000 7fe072ffd700 1 -- 172.21.15.40:0/2078633549 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe08400f040 con 0x5638819fa080 2023-07-18T03:41:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.887+0000 7fe072ffd700 1 -- 172.21.15.40:0/2078633549 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084007760 con 0x5638819fa080 2023-07-18T03:41:54.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.887+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/2078633549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 msgr2=0x563881a01260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.887+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/2078633549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x563881a01260 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe0840115e0 tx=0x7fe084009b20 comp rx=0 tx=0).stop 2023-07-18T03:41:54.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.887+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/2078633549 shutdown_connections 2023-07-18T03:41:54.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.887+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/2078633549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819fa080 0x563881a01260 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.887+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/2078633549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819f88e0 0x5638819f8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.887+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/2078633549 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f9230 0x563881a09a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.887+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/2078633549 >> 172.21.15.40:0/2078633549 conn(0x5638819e9da0 msgr2=0x563881a151d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:54.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.887+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/2078633549 shutdown_connections 2023-07-18T03:41:54.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.887+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/2078633549 wait complete. 2023-07-18T03:41:54.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.888+0000 7fe09e4d3b80 1 -- start start 2023-07-18T03:41:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.888+0000 7fe09e4d3b80 1 Processor -- start 2023-07-18T03:41:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.888+0000 7fe09e4d3b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819f88e0 0x563881a05c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.888+0000 7fe09e4d3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f9230 0x563881a06190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.888+0000 7fe09e4d3b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819fa080 0x563881a08c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.888+0000 7fe09e4d3b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x563881a09160 con 0x5638819f88e0 2023-07-18T03:41:54.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.888+0000 7fe09e4d3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x563881a092d0 con 0x5638819f9230 2023-07-18T03:41:54.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.888+0000 7fe09e4d3b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x563881a09490 con 0x5638819fa080 2023-07-18T03:41:54.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.888+0000 7fe09e4d3b80 10 client.0 _ll_register_callbacks cb 0x7fe0840119b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:54.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.889+0000 7fe08bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819f88e0 0x563881a05c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.889+0000 7fe08b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f9230 0x563881a06190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.889+0000 7fe08bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819f88e0 0x563881a05c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33562/0 (socket says 172.21.15.40:33562) 2023-07-18T03:41:54.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.889+0000 7fe08b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f9230 0x563881a06190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:45120/0 (socket says 172.21.15.40:45120) 2023-07-18T03:41:54.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.889+0000 7fe08bfff700 1 -- 172.21.15.40:0/2637782372 learned_addr learned my addr 172.21.15.40:0/2637782372 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:54.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.889+0000 7fe0908ea700 1 --2- 172.21.15.40:0/2637782372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819fa080 0x563881a08c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.889+0000 7fe08b7fe700 1 -- 172.21.15.40:0/2637782372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819fa080 msgr2=0x563881a08c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:54.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.889+0000 7fe08b7fe700 1 --2- 172.21.15.40:0/2637782372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819fa080 0x563881a08c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.889+0000 7fe08b7fe700 1 -- 172.21.15.40:0/2637782372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819f88e0 msgr2=0x563881a05c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:54.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.889+0000 7fe08b7fe700 1 --2- 172.21.15.40:0/2637782372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819f88e0 0x563881a05c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.889+0000 7fe08b7fe700 1 -- 172.21.15.40:0/2637782372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07c02c040 con 0x5638819f9230 2023-07-18T03:41:54.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.889+0000 7fe08bfff700 1 --2- 172.21.15.40:0/2637782372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819f88e0 0x563881a05c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:54.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.889+0000 7fe08b7fe700 1 --2- 172.21.15.40:0/2637782372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f9230 0x563881a06190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe08009b280 tx=0x7fe080097030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b34beaf12d2e01c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:54.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.889+0000 7fe06b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:41:54.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.889+0000 7fe06b7fe700 1 -- 172.21.15.40:0/2637782372 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08009a110 con 0x5638819f9230 2023-07-18T03:41:54.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.890+0000 7fe06b7fe700 1 -- 172.21.15.40:0/2637782372 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fe08009a270 con 0x5638819f9230 2023-07-18T03:41:54.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.890+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/2637782372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0800a3070 con 0x5638819f9230 2023-07-18T03:41:54.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.890+0000 7fe06b7fe700 1 -- 172.21.15.40:0/2637782372 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0800cb0c0 con 0x5638819f9230 2023-07-18T03:41:54.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.890+0000 7fe09e4d3b80 10 client.26624 fetch_fsmap learned FSMap version 72 2023-07-18T03:41:54.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.890+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/2637782372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x563881773de0 con 0x5638819f9230 2023-07-18T03:41:54.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.890+0000 7fe06b7fe700 1 -- 172.21.15.40:0/2637782372 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0800a3070 con 0x5638819f9230 2023-07-18T03:41:54.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.890+0000 7fe06b7fe700 1 -- 172.21.15.40:0/2637782372 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 72) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fe080096a80 con 0x5638819f9230 2023-07-18T03:41:54.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.890+0000 7fe09e4d3b80 10 client.26624 fetch_fsmap finished waiting for FSMap version 72 2023-07-18T03:41:54.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.890+0000 7fe09e4d3b80 10 client.26624 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2023-07-18T03:41:54.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.890+0000 7fe09e4d3b80 20 client.26624 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:54.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.890+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/2637782372 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x563881a62ec0 0x563881a652d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:54.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.890+0000 7fe09e4d3b80 4 client.26624 mds_command: new command op to 24442 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:41:54.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.890+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/2637782372 --> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563881901050 con 0x563881a62ec0 2023-07-18T03:41:54.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.890+0000 7fe08bfff700 1 --2- 172.21.15.40:0/2637782372 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x563881a62ec0 0x563881a652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:54.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.891+0000 7fe06b7fe700 10 client.26624 ms_handle_connect on v2:172.21.15.134:6832/2345264306 2023-07-18T03:41:54.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.891+0000 7fe08bfff700 1 --2- 172.21.15.40:0/2637782372 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x563881a62ec0 0x563881a652d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=56dba010fe66579b server_cookie=f76a5ef3f35a1e74 in_seq=0 out_seq=0 2023-07-18T03:41:54.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.891+0000 7fe06b7fe700 1 -- 172.21.15.40:0/2637782372 <== mds.4 v2:172.21.15.134:6832/2345264306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563881901050 con 0x563881a62ec0 2023-07-18T03:41:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.891+0000 7fe06b7fe700 10 client.26624 handle_command_reply: tid=0 2023-07-18T03:41:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.891+0000 7fe09e4d3b80 1 client.26624 shutdown 2023-07-18T03:41:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.891+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/2637782372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f9230 msgr2=0x563881a06190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.891+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/2637782372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f9230 0x563881a06190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe08009b280 tx=0x7fe080097030 comp rx=0 tx=0).stop 2023-07-18T03:41:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.891+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/2637782372 shutdown_connections 2023-07-18T03:41:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.891+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/2637782372 >> [v2:172.21.15.134:6832/2345264306,v1:172.21.15.134:6833/2345264306] conn(0x563881a62ec0 0x563881a652d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.891+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/2637782372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638819fa080 0x563881a08c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.891+0000 7fe06b7fe700 0 client.26624 ms_handle_reset on v2:172.21.15.134:6832/2345264306 2023-07-18T03:41:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.891+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/2637782372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5638819f88e0 0x563881a05c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.891+0000 7fe09e4d3b80 1 --2- 172.21.15.40:0/2637782372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638819f9230 0x563881a06190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.891+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/2637782372 >> 172.21.15.40:0/2637782372 conn(0x5638819e9da0 msgr2=0x563881a01260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.891+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/2637782372 shutdown_connections 2023-07-18T03:41:54.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.891+0000 7fe09e4d3b80 1 -- 172.21.15.40:0/2637782372 wait complete. 2023-07-18T03:41:54.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.891+0000 7fe09e4d3b80 20 client.26624 trim_cache size 0 max 16384 2023-07-18T03:41:54.900 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:41:54.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.892+0000 7fe09234d700 1 -- 172.21.15.40:0/287744252 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0740a1150 msgr2=0x7fe0740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.892+0000 7fe09234d700 1 --2- 172.21.15.40:0/287744252 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0740a1150 0x7fe0740a3600 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe07c006010 tx=0x7fe07c005960 comp rx=0 tx=0).stop 2023-07-18T03:41:54.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.892+0000 7fe09234d700 1 -- 172.21.15.40:0/287744252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe08c0ffbd0 msgr2=0x7fe08c194c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:54.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.892+0000 7fe09234d700 1 --2- 172.21.15.40:0/287744252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe08c0ffbd0 0x7fe08c194c80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe080007910 tx=0x7fe080007c20 comp rx=0 tx=0).stop 2023-07-18T03:41:54.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.892+0000 7fe09234d700 1 -- 172.21.15.40:0/287744252 shutdown_connections 2023-07-18T03:41:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.892+0000 7fe09234d700 1 --2- 172.21.15.40:0/287744252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe08c100560 0x7fe08c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.892+0000 7fe09234d700 1 --2- 172.21.15.40:0/287744252 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fe0740a1150 0x7fe0740a3600 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.892+0000 7fe09234d700 1 --2- 172.21.15.40:0/287744252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe08c0fe9e0 0x7fe08c194740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.892+0000 7fe09234d700 1 --2- 172.21.15.40:0/287744252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe08c0ffbd0 0x7fe08c194c80 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.892+0000 7fe09234d700 1 -- 172.21.15.40:0/287744252 >> 172.21.15.40:0/287744252 conn(0x7fe08c0fa000 msgr2=0x7fe08c0fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:54.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.892+0000 7fe09234d700 1 -- 172.21.15.40:0/287744252 shutdown_connections 2023-07-18T03:41:54.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:54.892+0000 7fe09234d700 1 -- 172.21.15.40:0/287744252 wait complete. 2023-07-18T03:41:54.904 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-18T03:41:55.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.039+0000 7fa932b05700 1 -- 172.21.15.40:0/1631904158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa92c0ffbd0 msgr2=0x7fa92c100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.039+0000 7fa932b05700 1 --2- 172.21.15.40:0/1631904158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa92c0ffbd0 0x7fa92c100020 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa918009990 tx=0x7fa918009ca0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.039+0000 7fa932b05700 1 -- 172.21.15.40:0/1631904158 shutdown_connections 2023-07-18T03:41:55.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.039+0000 7fa932b05700 1 --2- 172.21.15.40:0/1631904158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa92c100560 0x7fa92c106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.039+0000 7fa932b05700 1 --2- 172.21.15.40:0/1631904158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa92c0fe9e0 0x7fa92c0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.039+0000 7fa932b05700 1 --2- 172.21.15.40:0/1631904158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa92c0ffbd0 0x7fa92c100020 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.039+0000 7fa932b05700 1 -- 172.21.15.40:0/1631904158 >> 172.21.15.40:0/1631904158 conn(0x7fa92c0fa000 msgr2=0x7fa92c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.039+0000 7fa932b05700 1 -- 172.21.15.40:0/1631904158 shutdown_connections 2023-07-18T03:41:55.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.039+0000 7fa932b05700 1 -- 172.21.15.40:0/1631904158 wait complete. 2023-07-18T03:41:55.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa932b05700 1 Processor -- start 2023-07-18T03:41:55.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa932b05700 1 -- start start 2023-07-18T03:41:55.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa932b05700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa92c0fe9e0 0x7fa92c1957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa932b05700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa92c0ffbd0 0x7fa92c195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa932b05700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa92c100560 0x7fa92c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa932b05700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19d370 con 0x7fa92c0fe9e0 2023-07-18T03:41:55.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa932b05700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19d4b0 con 0x7fa92c100560 2023-07-18T03:41:55.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa932b05700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19d640 con 0x7fa92c0ffbd0 2023-07-18T03:41:55.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa9308a1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa92c0fe9e0 0x7fa92c1957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa9308a1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa92c0fe9e0 0x7fa92c1957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33576/0 (socket says 172.21.15.40:33576) 2023-07-18T03:41:55.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa9308a1700 1 -- 172.21.15.40:0/636850271 learned_addr learned my addr 172.21.15.40:0/636850271 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa92bfff700 1 --2- 172.21.15.40:0/636850271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa92c0ffbd0 0x7fa92c195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa9310a2700 1 --2- 172.21.15.40:0/636850271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa92c100560 0x7fa92c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa9308a1700 1 -- 172.21.15.40:0/636850271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa92c0ffbd0 msgr2=0x7fa92c195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa9308a1700 1 --2- 172.21.15.40:0/636850271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa92c0ffbd0 0x7fa92c195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa9308a1700 1 -- 172.21.15.40:0/636850271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa92c100560 msgr2=0x7fa92c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa9308a1700 1 --2- 172.21.15.40:0/636850271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa92c100560 0x7fa92c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.040+0000 7fa9308a1700 1 -- 172.21.15.40:0/636850271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa920009520 con 0x7fa92c0fe9e0 2023-07-18T03:41:55.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.041+0000 7fa9308a1700 1 --2- 172.21.15.40:0/636850271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa92c0fe9e0 0x7fa92c1957f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa92000ea50 tx=0x7fa92000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.041+0000 7fa929ffb700 1 -- 172.21.15.40:0/636850271 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92000eee0 con 0x7fa92c0fe9e0 2023-07-18T03:41:55.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.041+0000 7fa932b05700 1 -- 172.21.15.40:0/636850271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa924009c40 con 0x7fa92c0fe9e0 2023-07-18T03:41:55.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.041+0000 7fa929ffb700 1 -- 172.21.15.40:0/636850271 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa92000cbd0 con 0x7fa92c0fe9e0 2023-07-18T03:41:55.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.041+0000 7fa929ffb700 1 -- 172.21.15.40:0/636850271 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa920018680 con 0x7fa92c0fe9e0 2023-07-18T03:41:55.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.041+0000 7fa932b05700 1 -- 172.21.15.40:0/636850271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9180095f0 con 0x7fa92c0fe9e0 2023-07-18T03:41:55.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.043+0000 7fa929ffb700 1 -- 172.21.15.40:0/636850271 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa920010430 con 0x7fa92c0fe9e0 2023-07-18T03:41:55.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.044+0000 7fa929ffb700 1 --2- 172.21.15.40:0/636850271 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa91c0a1150 0x7fa91c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.044+0000 7fa92bfff700 1 --2- 172.21.15.40:0/636850271 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa91c0a1150 0x7fa91c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.044+0000 7fa929ffb700 1 -- 172.21.15.40:0/636850271 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa920014070 con 0x7fa92c0fe9e0 2023-07-18T03:41:55.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.044+0000 7fa92bfff700 1 --2- 172.21.15.40:0/636850271 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa91c0a1150 0x7fa91c0a3600 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa91800b680 tx=0x7fa918005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.045+0000 7fa93ec8bb80 1 Processor -- start 2023-07-18T03:41:55.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.045+0000 7fa93ec8bb80 1 -- start start 2023-07-18T03:41:55.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.045+0000 7fa93ec8bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad18c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.045+0000 7fa93ec8bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18ce20 0x5648ad18d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.045+0000 7fa93ec8bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18dc70 0x5648ad18e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.045+0000 7fa93ec8bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648ad091cb0 con 0x5648ad18c4d0 2023-07-18T03:41:55.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.045+0000 7fa93ec8bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648ad0e4a40 con 0x5648ad18dc70 2023-07-18T03:41:55.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.045+0000 7fa93ec8bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648ad0dded0 con 0x5648ad18ce20 2023-07-18T03:41:55.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.045+0000 7fa9308a1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad18c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.045+0000 7fa9308a1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad18c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33584/0 (socket says 172.21.15.40:33584) 2023-07-18T03:41:55.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.045+0000 7fa9308a1700 1 -- 172.21.15.40:0/2623189411 learned_addr learned my addr 172.21.15.40:0/2623189411 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.045+0000 7fa92bfff700 1 --2- 172.21.15.40:0/2623189411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18ce20 0x5648ad18d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.045+0000 7fa9310a2700 1 --2- 172.21.15.40:0/2623189411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18dc70 0x5648ad18e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.046+0000 7fa9308a1700 1 -- 172.21.15.40:0/2623189411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18ce20 msgr2=0x5648ad18d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.046+0000 7fa9308a1700 1 --2- 172.21.15.40:0/2623189411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18ce20 0x5648ad18d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.046+0000 7fa9308a1700 1 -- 172.21.15.40:0/2623189411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18dc70 msgr2=0x5648ad18e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.046+0000 7fa9308a1700 1 --2- 172.21.15.40:0/2623189411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18dc70 0x5648ad18e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.046+0000 7fa9308a1700 1 -- 172.21.15.40:0/2623189411 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648acf0bde0 con 0x5648ad18c4d0 2023-07-18T03:41:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.046+0000 7fa9308a1700 1 --2- 172.21.15.40:0/2623189411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad18c8e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa920004800 tx=0x7fa920099ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6be3e90cd1d3cf server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.046+0000 7fa916ffd700 1 -- 172.21.15.40:0/2623189411 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa920098610 con 0x5648ad18c4d0 2023-07-18T03:41:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.046+0000 7fa916ffd700 1 -- 172.21.15.40:0/2623189411 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa92000cd40 con 0x5648ad18c4d0 2023-07-18T03:41:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.046+0000 7fa916ffd700 1 -- 172.21.15.40:0/2623189411 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9200a2490 con 0x5648ad18c4d0 2023-07-18T03:41:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.047+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/2623189411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 msgr2=0x5648ad18c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.047+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/2623189411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad18c8e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa920004800 tx=0x7fa920099ce0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.047+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/2623189411 shutdown_connections 2023-07-18T03:41:55.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.047+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/2623189411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18dc70 0x5648ad18e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.047+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/2623189411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad18c8e0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.047+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/2623189411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18ce20 0x5648ad18d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.047+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/2623189411 >> 172.21.15.40:0/2623189411 conn(0x5648ad181da0 msgr2=0x5648ad1871d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.047+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/2623189411 shutdown_connections 2023-07-18T03:41:55.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.047+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/2623189411 wait complete. 2023-07-18T03:41:55.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.047+0000 7fa93ec8bb80 1 -- start start 2023-07-18T03:41:55.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.047+0000 7fa93ec8bb80 1 Processor -- start 2023-07-18T03:41:55.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa93ec8bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad1960a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa93ec8bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18ce20 0x5648ad1965e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa93ec8bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18dc70 0x5648ad19d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa93ec8bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648ad07dac0 con 0x5648ad18c4d0 2023-07-18T03:41:55.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa93ec8bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648ad07d7c0 con 0x5648ad18dc70 2023-07-18T03:41:55.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa93ec8bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648acf61920 con 0x5648ad18ce20 2023-07-18T03:41:55.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa93ec8bb80 10 client.0 _ll_register_callbacks cb 0x5648ad07b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:55.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa9308a1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad1960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa9308a1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad1960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33590/0 (socket says 172.21.15.40:33590) 2023-07-18T03:41:55.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa9308a1700 1 -- 172.21.15.40:0/21454592 learned_addr learned my addr 172.21.15.40:0/21454592 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa92bfff700 1 --2- 172.21.15.40:0/21454592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18ce20 0x5648ad1965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa9310a2700 1 --2- 172.21.15.40:0/21454592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18dc70 0x5648ad19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa9308a1700 1 -- 172.21.15.40:0/21454592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18ce20 msgr2=0x5648ad1965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa9308a1700 1 --2- 172.21.15.40:0/21454592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18ce20 0x5648ad1965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa9308a1700 1 -- 172.21.15.40:0/21454592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18dc70 msgr2=0x5648ad19d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa9308a1700 1 --2- 172.21.15.40:0/21454592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18dc70 0x5648ad19d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa9308a1700 1 -- 172.21.15.40:0/21454592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9180095f0 con 0x5648ad18c4d0 2023-07-18T03:41:55.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa9308a1700 1 --2- 172.21.15.40:0/21454592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad1960a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa9200a2190 tx=0x7fa920099760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a303350bc45a59 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa90bfff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:41:55.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa90bfff700 1 -- 172.21.15.40:0/21454592 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9200108e0 con 0x5648ad18c4d0 2023-07-18T03:41:55.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa90bfff700 1 -- 172.21.15.40:0/21454592 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa920096060 con 0x5648ad18c4d0 2023-07-18T03:41:55.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/21454592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648ad0374f0 con 0x5648ad18c4d0 2023-07-18T03:41:55.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa90bfff700 1 -- 172.21.15.40:0/21454592 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa920099a20 con 0x5648ad18c4d0 2023-07-18T03:41:55.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa93ec8bb80 10 client.17001 fetch_fsmap learned FSMap version 72 2023-07-18T03:41:55.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.048+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/21454592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5648acef4c60 con 0x5648ad18c4d0 2023-07-18T03:41:55.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.049+0000 7fa90bfff700 1 -- 172.21.15.40:0/21454592 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5648ad0374f0 con 0x5648ad18c4d0 2023-07-18T03:41:55.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.049+0000 7fa90bfff700 1 -- 172.21.15.40:0/21454592 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 72) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fa920099240 con 0x5648ad18c4d0 2023-07-18T03:41:55.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.049+0000 7fa93ec8bb80 10 client.17001 fetch_fsmap finished waiting for FSMap version 72 2023-07-18T03:41:55.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.049+0000 7fa93ec8bb80 10 client.17001 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-18T03:41:55.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.049+0000 7fa93ec8bb80 20 client.17001 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:55.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.049+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/21454592 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x5648ad1a1b90 0x5648ad1a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.049+0000 7fa93ec8bb80 4 client.17001 mds_command: new command op to 24452 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:41:55.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.049+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/21454592 --> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648ad0d6eb0 con 0x5648ad1a1b90 2023-07-18T03:41:55.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.049+0000 7fa92bfff700 1 --2- 172.21.15.40:0/21454592 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x5648ad1a1b90 0x5648ad1a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.050+0000 7fa90bfff700 10 client.17001 ms_handle_connect on v2:172.21.15.64:6834/1864495009 2023-07-18T03:41:55.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.050+0000 7fa92bfff700 1 --2- 172.21.15.40:0/21454592 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x5648ad1a1b90 0x5648ad1a3fa0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=95c4f04311e22189 server_cookie=edeab5f960fb0ccd in_seq=0 out_seq=0 2023-07-18T03:41:55.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.050+0000 7fa90bfff700 1 -- 172.21.15.40:0/21454592 <== mds.3 v2:172.21.15.64:6834/1864495009 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5648ad0d6eb0 con 0x5648ad1a1b90 2023-07-18T03:41:55.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.050+0000 7fa90bfff700 10 client.17001 handle_command_reply: tid=0 2023-07-18T03:41:55.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.050+0000 7fa93ec8bb80 1 client.17001 shutdown 2023-07-18T03:41:55.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.050+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/21454592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 msgr2=0x5648ad1960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.050+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/21454592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad1960a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa9200a2190 tx=0x7fa920099760 comp rx=0 tx=0).stop 2023-07-18T03:41:55.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.050+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/21454592 shutdown_connections 2023-07-18T03:41:55.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.050+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/21454592 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x5648ad1a1b90 0x5648ad1a3fa0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.050+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/21454592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18dc70 0x5648ad19d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.050+0000 7fa90bfff700 0 client.17001 ms_handle_reset on v2:172.21.15.64:6834/1864495009 2023-07-18T03:41:55.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.050+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/21454592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad1960a0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.050+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/21454592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18ce20 0x5648ad1965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.050+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/21454592 >> 172.21.15.40:0/21454592 conn(0x5648ad181da0 msgr2=0x5648ad1868b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.051+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/21454592 shutdown_connections 2023-07-18T03:41:55.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.051+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/21454592 wait complete. 2023-07-18T03:41:55.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.051+0000 7fa93ec8bb80 20 client.17001 trim_cache size 0 max 16384 2023-07-18T03:41:55.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.064+0000 7fa93ec8bb80 1 Processor -- start 2023-07-18T03:41:55.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.064+0000 7fa93ec8bb80 1 -- start start 2023-07-18T03:41:55.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.064+0000 7fa93ec8bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad195750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.064+0000 7fa93ec8bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18ce20 0x5648ad19d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.064+0000 7fa93ec8bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18dc70 0x5648ad194d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.064+0000 7fa93ec8bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648ad188070 con 0x5648ad18c4d0 2023-07-18T03:41:55.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.064+0000 7fa93ec8bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648ad1a1890 con 0x5648ad18dc70 2023-07-18T03:41:55.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.064+0000 7fa93ec8bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648ad1952d0 con 0x5648ad18ce20 2023-07-18T03:41:55.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.064+0000 7fa9308a1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.064+0000 7fa9308a1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad195750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33598/0 (socket says 172.21.15.40:33598) 2023-07-18T03:41:55.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.064+0000 7fa92bfff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18ce20 0x5648ad19d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.064+0000 7fa9308a1700 1 -- 172.21.15.40:0/836471383 learned_addr learned my addr 172.21.15.40:0/836471383 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.064+0000 7fa9310a2700 1 --2- 172.21.15.40:0/836471383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18dc70 0x5648ad194d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.065+0000 7fa92bfff700 1 -- 172.21.15.40:0/836471383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18dc70 msgr2=0x5648ad194d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.065+0000 7fa92bfff700 1 --2- 172.21.15.40:0/836471383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18dc70 0x5648ad194d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.065+0000 7fa92bfff700 1 -- 172.21.15.40:0/836471383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 msgr2=0x5648ad195750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.065+0000 7fa92bfff700 1 --2- 172.21.15.40:0/836471383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad195750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.065+0000 7fa92bfff700 1 -- 172.21.15.40:0/836471383 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648acf32250 con 0x5648ad18ce20 2023-07-18T03:41:55.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.065+0000 7fa9310a2700 1 --2- 172.21.15.40:0/836471383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18dc70 0x5648ad194d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:55.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.065+0000 7fa92bfff700 1 --2- 172.21.15.40:0/836471383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18ce20 0x5648ad19d5e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa918006010 tx=0x7fa918023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ea45ff2789d34ae server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.065+0000 7fa9177fe700 1 -- 172.21.15.40:0/836471383 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91802d050 con 0x5648ad18ce20 2023-07-18T03:41:55.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.065+0000 7fa9177fe700 1 -- 172.21.15.40:0/836471383 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa91802e040 con 0x5648ad18ce20 2023-07-18T03:41:55.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.065+0000 7fa9177fe700 1 -- 172.21.15.40:0/836471383 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa918034290 con 0x5648ad18ce20 2023-07-18T03:41:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.066+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/836471383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18ce20 msgr2=0x5648ad19d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.066+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/836471383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18ce20 0x5648ad19d5e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa918006010 tx=0x7fa918023000 comp rx=0 tx=0).stop 2023-07-18T03:41:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.066+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/836471383 shutdown_connections 2023-07-18T03:41:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.066+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/836471383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18dc70 0x5648ad194d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.066+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/836471383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad195750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.066+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/836471383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18ce20 0x5648ad19d5e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.066+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/836471383 >> 172.21.15.40:0/836471383 conn(0x5648ad181da0 msgr2=0x5648ad1a8900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.066+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/836471383 shutdown_connections 2023-07-18T03:41:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.066+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/836471383 wait complete. 2023-07-18T03:41:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.066+0000 7fa93ec8bb80 1 -- start start 2023-07-18T03:41:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.066+0000 7fa93ec8bb80 1 Processor -- start 2023-07-18T03:41:55.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa93ec8bb80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa93ec8bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18ce20 0x5648ad19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa93ec8bb80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18dc70 0x5648ad19ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa93ec8bb80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648ad19cf50 con 0x5648ad18c4d0 2023-07-18T03:41:55.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa93ec8bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648ad19d0c0 con 0x5648ad18ce20 2023-07-18T03:41:55.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa93ec8bb80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648ad19d280 con 0x5648ad18dc70 2023-07-18T03:41:55.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa93ec8bb80 10 client.0 _ll_register_callbacks cb 0x5648ad07b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:55.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa9308a1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa9308a1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad19b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33604/0 (socket says 172.21.15.40:33604) 2023-07-18T03:41:55.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa9308a1700 1 -- 172.21.15.40:0/1222783168 learned_addr learned my addr 172.21.15.40:0/1222783168 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa9310a2700 1 --2- 172.21.15.40:0/1222783168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18dc70 0x5648ad19ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa92bfff700 1 --2- 172.21.15.40:0/1222783168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18ce20 0x5648ad19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa9308a1700 1 -- 172.21.15.40:0/1222783168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18dc70 msgr2=0x5648ad19ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa9308a1700 1 --2- 172.21.15.40:0/1222783168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18dc70 0x5648ad19ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa9308a1700 1 -- 172.21.15.40:0/1222783168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18ce20 msgr2=0x5648ad19ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa9308a1700 1 --2- 172.21.15.40:0/1222783168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18ce20 0x5648ad19ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa9308a1700 1 -- 172.21.15.40:0/1222783168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92c107750 con 0x5648ad18c4d0 2023-07-18T03:41:55.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa9308a1700 1 --2- 172.21.15.40:0/1222783168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad19b4c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa9200cacd0 tx=0x7fa920098820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc3c3b7c7c0288f server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa90bfff700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:41:55.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa90bfff700 1 -- 172.21.15.40:0/1222783168 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9200999d0 con 0x5648ad18c4d0 2023-07-18T03:41:55.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa90bfff700 1 -- 172.21.15.40:0/1222783168 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fa9200106e0 con 0x5648ad18c4d0 2023-07-18T03:41:55.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.067+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/1222783168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa92000cd40 con 0x5648ad18c4d0 2023-07-18T03:41:55.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.068+0000 7fa90bfff700 1 -- 172.21.15.40:0/1222783168 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa920061020 con 0x5648ad18c4d0 2023-07-18T03:41:55.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.068+0000 7fa93ec8bb80 10 client.17007 fetch_fsmap learned FSMap version 72 2023-07-18T03:41:55.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.068+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/1222783168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5648acf0a2b0 con 0x5648ad18c4d0 2023-07-18T03:41:55.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.068+0000 7fa90bfff700 1 -- 172.21.15.40:0/1222783168 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa92000cd40 con 0x5648ad18c4d0 2023-07-18T03:41:55.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.068+0000 7fa90bfff700 1 -- 172.21.15.40:0/1222783168 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 72) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fa920083c50 con 0x5648ad18c4d0 2023-07-18T03:41:55.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.068+0000 7fa93ec8bb80 10 client.17007 fetch_fsmap finished waiting for FSMap version 72 2023-07-18T03:41:55.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.068+0000 7fa93ec8bb80 10 client.17007 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2023-07-18T03:41:55.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.068+0000 7fa93ec8bb80 20 client.17007 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.068+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/1222783168 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x5648ad1f6640 0x5648ad1f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.068+0000 7fa93ec8bb80 4 client.17007 mds_command: new command op to 24452 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:41:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.068+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/1222783168 --> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5648ad099050 con 0x5648ad1f6640 2023-07-18T03:41:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.068+0000 7fa92bfff700 1 --2- 172.21.15.40:0/1222783168 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x5648ad1f6640 0x5648ad1f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.068+0000 7fa90bfff700 10 client.17007 ms_handle_connect on v2:172.21.15.64:6834/1864495009 2023-07-18T03:41:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.068+0000 7fa92bfff700 1 --2- 172.21.15.40:0/1222783168 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x5648ad1f6640 0x5648ad1f8a50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c9e5a833617858f9 server_cookie=8b4a087a25bf3baf in_seq=0 out_seq=0 2023-07-18T03:41:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa90bfff700 1 -- 172.21.15.40:0/1222783168 <== mds.3 v2:172.21.15.64:6834/1864495009 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5648ad099050 con 0x5648ad1f6640 2023-07-18T03:41:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa90bfff700 10 client.17007 handle_command_reply: tid=0 2023-07-18T03:41:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa93ec8bb80 1 client.17007 shutdown 2023-07-18T03:41:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/1222783168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 msgr2=0x5648ad19b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/1222783168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad19b4c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa9200cacd0 tx=0x7fa920098820 comp rx=0 tx=0).stop 2023-07-18T03:41:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/1222783168 shutdown_connections 2023-07-18T03:41:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/1222783168 >> [v2:172.21.15.64:6834/1864495009,v1:172.21.15.64:6835/1864495009] conn(0x5648ad1f6640 0x5648ad1f8a50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/1222783168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648ad18dc70 0x5648ad19ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa90bfff700 0 client.17007 ms_handle_reset on v2:172.21.15.64:6834/1864495009 2023-07-18T03:41:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/1222783168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648ad18c4d0 0x5648ad19b4c0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa93ec8bb80 1 --2- 172.21.15.40:0/1222783168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648ad18ce20 0x5648ad19ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/1222783168 >> 172.21.15.40:0/1222783168 conn(0x5648ad181da0 msgr2=0x5648ad190c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/1222783168 shutdown_connections 2023-07-18T03:41:55.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa93ec8bb80 1 -- 172.21.15.40:0/1222783168 wait complete. 2023-07-18T03:41:55.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa93ec8bb80 20 client.17007 trim_cache size 0 max 16384 2023-07-18T03:41:55.078 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:41:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa932b05700 1 -- 172.21.15.40:0/636850271 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa91c0a1150 msgr2=0x7fa91c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa932b05700 1 --2- 172.21.15.40:0/636850271 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa91c0a1150 0x7fa91c0a3600 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa91800b680 tx=0x7fa918005960 comp rx=0 tx=0).stop 2023-07-18T03:41:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa932b05700 1 -- 172.21.15.40:0/636850271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa92c0fe9e0 msgr2=0x7fa92c1957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.069+0000 7fa932b05700 1 --2- 172.21.15.40:0/636850271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa92c0fe9e0 0x7fa92c1957f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa92000ea50 tx=0x7fa92000ed60 comp rx=0 tx=0).stop 2023-07-18T03:41:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.070+0000 7fa932b05700 1 -- 172.21.15.40:0/636850271 shutdown_connections 2023-07-18T03:41:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.070+0000 7fa932b05700 1 --2- 172.21.15.40:0/636850271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa92c100560 0x7fa92c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.070+0000 7fa932b05700 1 --2- 172.21.15.40:0/636850271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa92c0fe9e0 0x7fa92c1957f0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.070+0000 7fa932b05700 1 --2- 172.21.15.40:0/636850271 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fa91c0a1150 0x7fa91c0a3600 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.070+0000 7fa932b05700 1 --2- 172.21.15.40:0/636850271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa92c0ffbd0 0x7fa92c195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.070+0000 7fa932b05700 1 -- 172.21.15.40:0/636850271 >> 172.21.15.40:0/636850271 conn(0x7fa92c0fa000 msgr2=0x7fa92c0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.070+0000 7fa932b05700 1 -- 172.21.15.40:0/636850271 shutdown_connections 2023-07-18T03:41:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.070+0000 7fa932b05700 1 -- 172.21.15.40:0/636850271 wait complete. 2023-07-18T03:41:55.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:54 smithi040 ceph-mon[96439]: pgmap v625: 97 pgs: 97 active+clean; 710 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 486 op/s 2023-07-18T03:41:55.081 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:54 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3368095513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:41:55.082 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-18T03:41:55.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.217+0000 7f35f1dca700 1 -- 172.21.15.40:0/2944252368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35ec105840 msgr2=0x7f35ec107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.217+0000 7f35f1dca700 1 --2- 172.21.15.40:0/2944252368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35ec105840 0x7f35ec107c20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009990 tx=0x7f35dc009ca0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.217+0000 7f35f1dca700 1 -- 172.21.15.40:0/2944252368 shutdown_connections 2023-07-18T03:41:55.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.217+0000 7f35f1dca700 1 --2- 172.21.15.40:0/2944252368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35ec105840 0x7f35ec107c20 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.217+0000 7f35f1dca700 1 --2- 172.21.15.40:0/2944252368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35ec100600 0x7f35ec1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.217+0000 7f35f1dca700 1 --2- 172.21.15.40:0/2944252368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35ec102f20 0x7f35ec105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.217+0000 7f35f1dca700 1 -- 172.21.15.40:0/2944252368 >> 172.21.15.40:0/2944252368 conn(0x7f35ec0fa000 msgr2=0x7f35ec0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.217+0000 7f35f1dca700 1 -- 172.21.15.40:0/2944252368 shutdown_connections 2023-07-18T03:41:55.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.217+0000 7f35f1dca700 1 -- 172.21.15.40:0/2944252368 wait complete. 2023-07-18T03:41:55.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.218+0000 7f35f1dca700 1 Processor -- start 2023-07-18T03:41:55.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.218+0000 7f35f1dca700 1 -- start start 2023-07-18T03:41:55.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.218+0000 7f35f1dca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35ec100600 0x7f35ec195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.218+0000 7f35f1dca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35ec102f20 0x7f35ec195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.218+0000 7f35f1dca700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35ec105840 0x7f35ec19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.218+0000 7f35f1dca700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec19d300 con 0x7f35ec100600 2023-07-18T03:41:55.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.218+0000 7f35f1dca700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec19d440 con 0x7f35ec102f20 2023-07-18T03:41:55.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.218+0000 7f35f1dca700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec19d5d0 con 0x7f35ec105840 2023-07-18T03:41:55.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.218+0000 7f35eb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35ec100600 0x7f35ec195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.218+0000 7f35eb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35ec100600 0x7f35ec195780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33618/0 (socket says 172.21.15.40:33618) 2023-07-18T03:41:55.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.218+0000 7f35eb7fe700 1 -- 172.21.15.40:0/3366513544 learned_addr learned my addr 172.21.15.40:0/3366513544 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.218+0000 7f35eaffd700 1 --2- 172.21.15.40:0/3366513544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35ec102f20 0x7f35ec195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.218+0000 7f35ebfff700 1 --2- 172.21.15.40:0/3366513544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35ec105840 0x7f35ec19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.219+0000 7f35eb7fe700 1 -- 172.21.15.40:0/3366513544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35ec105840 msgr2=0x7f35ec19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.219+0000 7f35eb7fe700 1 --2- 172.21.15.40:0/3366513544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35ec105840 0x7f35ec19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.219+0000 7f35eb7fe700 1 -- 172.21.15.40:0/3366513544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35ec102f20 msgr2=0x7f35ec195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.219+0000 7f35eb7fe700 1 --2- 172.21.15.40:0/3366513544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35ec102f20 0x7f35ec195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.219+0000 7f35eb7fe700 1 -- 172.21.15.40:0/3366513544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc0095f0 con 0x7f35ec100600 2023-07-18T03:41:55.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.219+0000 7f35eb7fe700 1 --2- 172.21.15.40:0/3366513544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35ec100600 0x7f35ec195780 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f35e400ef10 tx=0x7f35e400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.219+0000 7f35e8ff9700 1 -- 172.21.15.40:0/3366513544 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e400c9c0 con 0x7f35ec100600 2023-07-18T03:41:55.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.219+0000 7f35f1dca700 1 -- 172.21.15.40:0/3366513544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ec19d850 con 0x7f35ec100600 2023-07-18T03:41:55.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.219+0000 7f35e8ff9700 1 -- 172.21.15.40:0/3366513544 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e400f040 con 0x7f35ec100600 2023-07-18T03:41:55.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.219+0000 7f35e8ff9700 1 -- 172.21.15.40:0/3366513544 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e4014480 con 0x7f35ec100600 2023-07-18T03:41:55.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.219+0000 7f35f1dca700 1 -- 172.21.15.40:0/3366513544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ec19ddb0 con 0x7f35ec100600 2023-07-18T03:41:55.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.222+0000 7f35e8ff9700 1 -- 172.21.15.40:0/3366513544 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f35e4007690 con 0x7f35ec100600 2023-07-18T03:41:55.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.223+0000 7f35e8ff9700 1 --2- 172.21.15.40:0/3366513544 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35d40a1150 0x7f35d40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.223+0000 7f35e8ff9700 1 -- 172.21.15.40:0/3366513544 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f35e40ccc50 con 0x7f35ec100600 2023-07-18T03:41:55.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.223+0000 7f35eaffd700 1 --2- 172.21.15.40:0/3366513544 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35d40a1150 0x7f35d40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.223+0000 7f35eaffd700 1 --2- 172.21.15.40:0/3366513544 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35d40a1150 0x7f35d40a3600 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009c60 tx=0x7f35e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.223+0000 7f35fdf50b80 1 Processor -- start 2023-07-18T03:41:55.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.223+0000 7f35fdf50b80 1 -- start start 2023-07-18T03:41:55.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35fdf50b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c68e0 0x55c1067c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35fdf50b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c7230 0x55c1067c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35fdf50b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c8080 0x55c1067c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35fdf50b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1066c7cb0 con 0x55c1067c68e0 2023-07-18T03:41:55.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35fdf50b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10671aa40 con 0x55c1067c7230 2023-07-18T03:41:55.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35fdf50b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c106713ed0 con 0x55c1067c8080 2023-07-18T03:41:55.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35eb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c68e0 0x55c1067c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35eb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c68e0 0x55c1067c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33626/0 (socket says 172.21.15.40:33626) 2023-07-18T03:41:55.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35eb7fe700 1 -- 172.21.15.40:0/3950490652 learned_addr learned my addr 172.21.15.40:0/3950490652 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35ebfff700 1 --2- 172.21.15.40:0/3950490652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c8080 0x55c1067c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35eaffd700 1 --2- 172.21.15.40:0/3950490652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c7230 0x55c1067c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35eb7fe700 1 -- 172.21.15.40:0/3950490652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c8080 msgr2=0x55c1067c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35eb7fe700 1 --2- 172.21.15.40:0/3950490652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c8080 0x55c1067c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35eb7fe700 1 -- 172.21.15.40:0/3950490652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c7230 msgr2=0x55c1067c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35eb7fe700 1 --2- 172.21.15.40:0/3950490652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c7230 0x55c1067c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35eb7fe700 1 -- 172.21.15.40:0/3950490652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c106541de0 con 0x55c1067c68e0 2023-07-18T03:41:55.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35eaffd700 1 --2- 172.21.15.40:0/3950490652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c7230 0x55c1067c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:55.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35eb7fe700 1 --2- 172.21.15.40:0/3950490652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c68e0 0x55c1067c6cf0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f35e4099f70 tx=0x7f35e4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77034ead983948f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35d1ffb700 1 -- 172.21.15.40:0/3950490652 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e40112c0 con 0x55c1067c68e0 2023-07-18T03:41:55.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35d1ffb700 1 -- 172.21.15.40:0/3950490652 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e4099fe0 con 0x55c1067c68e0 2023-07-18T03:41:55.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.224+0000 7f35d1ffb700 1 -- 172.21.15.40:0/3950490652 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e4099730 con 0x55c1067c68e0 2023-07-18T03:41:55.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.225+0000 7f35fdf50b80 1 -- 172.21.15.40:0/3950490652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c68e0 msgr2=0x55c1067c6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.225+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/3950490652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c68e0 0x55c1067c6cf0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f35e4099f70 tx=0x7f35e4007f90 comp rx=0 tx=0).stop 2023-07-18T03:41:55.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.225+0000 7f35fdf50b80 1 -- 172.21.15.40:0/3950490652 shutdown_connections 2023-07-18T03:41:55.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.225+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/3950490652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c8080 0x55c1067c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.225+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/3950490652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c68e0 0x55c1067c6cf0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.225+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/3950490652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c7230 0x55c1067c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.225+0000 7f35fdf50b80 1 -- 172.21.15.40:0/3950490652 >> 172.21.15.40:0/3950490652 conn(0x55c1067b7da0 msgr2=0x55c1067bd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.225+0000 7f35fdf50b80 1 -- 172.21.15.40:0/3950490652 shutdown_connections 2023-07-18T03:41:55.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.225+0000 7f35fdf50b80 1 -- 172.21.15.40:0/3950490652 wait complete. 2023-07-18T03:41:55.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.225+0000 7f35fdf50b80 1 -- start start 2023-07-18T03:41:55.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35fdf50b80 1 Processor -- start 2023-07-18T03:41:55.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35fdf50b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c68e0 0x55c1067d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35fdf50b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c7230 0x55c1067d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35fdf50b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c8080 0x55c1067d7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35fdf50b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1066b3ac0 con 0x55c1067c68e0 2023-07-18T03:41:55.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35fdf50b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1066b37c0 con 0x55c1067c7230 2023-07-18T03:41:55.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35fdf50b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c106597920 con 0x55c1067c8080 2023-07-18T03:41:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35eb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c68e0 0x55c1067d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35fdf50b80 10 client.0 _ll_register_callbacks cb 0x55c1066b1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35eb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c68e0 0x55c1067d0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33632/0 (socket says 172.21.15.40:33632) 2023-07-18T03:41:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35eb7fe700 1 -- 172.21.15.40:0/1907146533 learned_addr learned my addr 172.21.15.40:0/1907146533 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35ebfff700 1 --2- 172.21.15.40:0/1907146533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c8080 0x55c1067d7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35eaffd700 1 --2- 172.21.15.40:0/1907146533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c7230 0x55c1067d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35eb7fe700 1 -- 172.21.15.40:0/1907146533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c8080 msgr2=0x55c1067d7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35eb7fe700 1 --2- 172.21.15.40:0/1907146533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c8080 0x55c1067d7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35eb7fe700 1 -- 172.21.15.40:0/1907146533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c7230 msgr2=0x55c1067d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35eb7fe700 1 --2- 172.21.15.40:0/1907146533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c7230 0x55c1067d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35eb7fe700 1 -- 172.21.15.40:0/1907146533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ec19ddb0 con 0x55c1067c68e0 2023-07-18T03:41:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.226+0000 7f35ebfff700 1 --2- 172.21.15.40:0/1907146533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c8080 0x55c1067d7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.227+0000 7f35eb7fe700 1 --2- 172.21.15.40:0/1907146533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c68e0 0x55c1067d0590 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f35e400c6f0 tx=0x7f35e400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc6724467d8e527 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.227+0000 7f35caffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:41:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.227+0000 7f35caffd700 1 -- 172.21.15.40:0/1907146533 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e40a2430 con 0x55c1067c68e0 2023-07-18T03:41:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.227+0000 7f35caffd700 1 -- 172.21.15.40:0/1907146533 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f35e40b82c0 con 0x55c1067c68e0 2023-07-18T03:41:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.227+0000 7f35fdf50b80 1 -- 172.21.15.40:0/1907146533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c10666d4f0 con 0x55c1067c68e0 2023-07-18T03:41:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.227+0000 7f35caffd700 1 -- 172.21.15.40:0/1907146533 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35e405fea0 con 0x55c1067c68e0 2023-07-18T03:41:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.227+0000 7f35fdf50b80 10 client.17019 fetch_fsmap learned FSMap version 72 2023-07-18T03:41:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.227+0000 7f35fdf50b80 1 -- 172.21.15.40:0/1907146533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55c10652ac60 con 0x55c1067c68e0 2023-07-18T03:41:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.227+0000 7f35caffd700 1 -- 172.21.15.40:0/1907146533 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c10666d4f0 con 0x55c1067c68e0 2023-07-18T03:41:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.227+0000 7f35caffd700 1 -- 172.21.15.40:0/1907146533 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 72) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f35e4095ed0 con 0x55c1067c68e0 2023-07-18T03:41:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.227+0000 7f35fdf50b80 10 client.17019 fetch_fsmap finished waiting for FSMap version 72 2023-07-18T03:41:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.227+0000 7f35fdf50b80 10 client.17019 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-18T03:41:55.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.227+0000 7f35fdf50b80 20 client.17019 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:55.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.227+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/1907146533 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55c1067dc000 0x55c1067de410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.227+0000 7f35fdf50b80 4 client.17019 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:41:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.227+0000 7f35fdf50b80 1 -- 172.21.15.40:0/1907146533 --> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c10670ceb0 con 0x55c1067dc000 2023-07-18T03:41:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.228+0000 7f35eaffd700 1 --2- 172.21.15.40:0/1907146533 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55c1067dc000 0x55c1067de410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.228+0000 7f35caffd700 10 client.17019 ms_handle_connect on v2:172.21.15.64:6836/3077792296 2023-07-18T03:41:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.228+0000 7f35eaffd700 1 --2- 172.21.15.40:0/1907146533 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55c1067dc000 0x55c1067de410 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=77acae3375107863 server_cookie=1ec474316c3f4c05 in_seq=0 out_seq=0 2023-07-18T03:41:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.229+0000 7f35caffd700 1 -- 172.21.15.40:0/1907146533 <== mds.2 v2:172.21.15.64:6836/3077792296 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c10670ceb0 con 0x55c1067dc000 2023-07-18T03:41:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.229+0000 7f35caffd700 10 client.17019 handle_command_reply: tid=0 2023-07-18T03:41:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.229+0000 7f35fdf50b80 1 client.17019 shutdown 2023-07-18T03:41:55.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.229+0000 7f35fdf50b80 1 -- 172.21.15.40:0/1907146533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c68e0 msgr2=0x55c1067d0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.229+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/1907146533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c68e0 0x55c1067d0590 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f35e400c6f0 tx=0x7f35e400ef40 comp rx=0 tx=0).stop 2023-07-18T03:41:55.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.229+0000 7f35fdf50b80 1 -- 172.21.15.40:0/1907146533 shutdown_connections 2023-07-18T03:41:55.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.229+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/1907146533 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55c1067dc000 0x55c1067de410 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.229+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/1907146533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c8080 0x55c1067d7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.229+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/1907146533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c68e0 0x55c1067d0590 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.229+0000 7f35caffd700 0 client.17019 ms_handle_reset on v2:172.21.15.64:6836/3077792296 2023-07-18T03:41:55.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.229+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/1907146533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c7230 0x55c1067d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.229+0000 7f35fdf50b80 1 -- 172.21.15.40:0/1907146533 >> 172.21.15.40:0/1907146533 conn(0x55c1067b7da0 msgr2=0x55c1067bc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.229+0000 7f35fdf50b80 1 -- 172.21.15.40:0/1907146533 shutdown_connections 2023-07-18T03:41:55.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.229+0000 7f35fdf50b80 1 -- 172.21.15.40:0/1907146533 wait complete. 2023-07-18T03:41:55.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.229+0000 7f35fdf50b80 20 client.17019 trim_cache size 0 max 16384 2023-07-18T03:41:55.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.242+0000 7f35fdf50b80 1 Processor -- start 2023-07-18T03:41:55.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.242+0000 7f35fdf50b80 1 -- start start 2023-07-18T03:41:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.242+0000 7f35fdf50b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c68e0 0x55c1067cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.242+0000 7f35fdf50b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c7230 0x55c1067d7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.242+0000 7f35fdf50b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c8080 0x55c1067cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.242+0000 7f35fdf50b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1067cef90 con 0x55c1067c7230 2023-07-18T03:41:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.242+0000 7f35fdf50b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1067c8810 con 0x55c1067c8080 2023-07-18T03:41:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.242+0000 7f35fdf50b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1067cf7a0 con 0x55c1067c68e0 2023-07-18T03:41:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.242+0000 7f35eaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c7230 0x55c1067d7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.242+0000 7f35eaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c7230 0x55c1067d7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33642/0 (socket says 172.21.15.40:33642) 2023-07-18T03:41:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.242+0000 7f35eaffd700 1 -- 172.21.15.40:0/14975774 learned_addr learned my addr 172.21.15.40:0/14975774 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.242+0000 7f35eb7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c68e0 0x55c1067cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35ebfff700 1 --2- 172.21.15.40:0/14975774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c8080 0x55c1067cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35eaffd700 1 -- 172.21.15.40:0/14975774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c68e0 msgr2=0x55c1067cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35eaffd700 1 --2- 172.21.15.40:0/14975774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c68e0 0x55c1067cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35eaffd700 1 -- 172.21.15.40:0/14975774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c8080 msgr2=0x55c1067cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35eaffd700 1 --2- 172.21.15.40:0/14975774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c8080 0x55c1067cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35eaffd700 1 -- 172.21.15.40:0/14975774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c106568250 con 0x55c1067c7230 2023-07-18T03:41:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35ebfff700 1 --2- 172.21.15.40:0/14975774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c8080 0x55c1067cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:41:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35eaffd700 1 --2- 172.21.15.40:0/14975774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c7230 0x55c1067d7ad0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f35e0013010 tx=0x7f35e0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ba44ec5b1f6453 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35d27fc700 1 -- 172.21.15.40:0/14975774 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e0023040 con 0x55c1067c7230 2023-07-18T03:41:55.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35d27fc700 1 -- 172.21.15.40:0/14975774 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e00285b0 con 0x55c1067c7230 2023-07-18T03:41:55.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35d27fc700 1 -- 172.21.15.40:0/14975774 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e001d670 con 0x55c1067c7230 2023-07-18T03:41:55.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35fdf50b80 1 -- 172.21.15.40:0/14975774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c7230 msgr2=0x55c1067d7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/14975774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c7230 0x55c1067d7ad0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f35e0013010 tx=0x7f35e0014000 comp rx=0 tx=0).stop 2023-07-18T03:41:55.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35fdf50b80 1 -- 172.21.15.40:0/14975774 shutdown_connections 2023-07-18T03:41:55.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/14975774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c8080 0x55c1067cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/14975774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c68e0 0x55c1067cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/14975774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c7230 0x55c1067d7ad0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.243+0000 7f35fdf50b80 1 -- 172.21.15.40:0/14975774 >> 172.21.15.40:0/14975774 conn(0x55c1067b7da0 msgr2=0x55c1067e2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.244+0000 7f35fdf50b80 1 -- 172.21.15.40:0/14975774 shutdown_connections 2023-07-18T03:41:55.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.244+0000 7f35fdf50b80 1 -- 172.21.15.40:0/14975774 wait complete. 2023-07-18T03:41:55.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.244+0000 7f35fdf50b80 1 -- start start 2023-07-18T03:41:55.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.244+0000 7f35fdf50b80 1 Processor -- start 2023-07-18T03:41:55.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.244+0000 7f35fdf50b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c68e0 0x55c1067c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.244+0000 7f35fdf50b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c7230 0x55c1067c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.244+0000 7f35fdf50b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c8080 0x55c1067d7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.244+0000 7f35fdf50b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1067c6350 con 0x55c1067c8080 2023-07-18T03:41:55.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.244+0000 7f35fdf50b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1067c64c0 con 0x55c1067c7230 2023-07-18T03:41:55.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.244+0000 7f35fdf50b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1067c4790 con 0x55c1067c68e0 2023-07-18T03:41:55.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.244+0000 7f35fdf50b80 10 client.0 _ll_register_callbacks cb 0x7f35e0005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:55.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.244+0000 7f35ebfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c8080 0x55c1067d7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.244+0000 7f35eb7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c68e0 0x55c1067c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.244+0000 7f35ebfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c8080 0x55c1067d7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33650/0 (socket says 172.21.15.40:33650) 2023-07-18T03:41:55.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.244+0000 7f35ebfff700 1 -- 172.21.15.40:0/1632867952 learned_addr learned my addr 172.21.15.40:0/1632867952 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.245+0000 7f35eaffd700 1 --2- 172.21.15.40:0/1632867952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c7230 0x55c1067c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.245+0000 7f35ebfff700 1 -- 172.21.15.40:0/1632867952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c68e0 msgr2=0x55c1067c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.245+0000 7f35ebfff700 1 --2- 172.21.15.40:0/1632867952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c68e0 0x55c1067c5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.245+0000 7f35ebfff700 1 -- 172.21.15.40:0/1632867952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c7230 msgr2=0x55c1067c41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.245+0000 7f35ebfff700 1 --2- 172.21.15.40:0/1632867952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c7230 0x55c1067c41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.245+0000 7f35ebfff700 1 -- 172.21.15.40:0/1632867952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e0017070 con 0x55c1067c8080 2023-07-18T03:41:55.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.245+0000 7f35eb7fe700 1 --2- 172.21.15.40:0/1632867952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c68e0 0x55c1067c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:55.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.245+0000 7f35ebfff700 1 --2- 172.21.15.40:0/1632867952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c8080 0x55c1067d7a30 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009990 tx=0x7f35dc005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3cb50a9a341685 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.245+0000 7f35caffd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:41:55.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.245+0000 7f35caffd700 1 -- 172.21.15.40:0/1632867952 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc01d030 con 0x55c1067c8080 2023-07-18T03:41:55.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.245+0000 7f35caffd700 1 -- 172.21.15.40:0/1632867952 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f35dc004700 con 0x55c1067c8080 2023-07-18T03:41:55.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.245+0000 7f35fdf50b80 1 -- 172.21.15.40:0/1632867952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35e4099fe0 con 0x55c1067c8080 2023-07-18T03:41:55.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.245+0000 7f35caffd700 1 -- 172.21.15.40:0/1632867952 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35dc0048c0 con 0x55c1067c8080 2023-07-18T03:41:55.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.245+0000 7f35fdf50b80 10 client.17025 fetch_fsmap learned FSMap version 72 2023-07-18T03:41:55.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.245+0000 7f35fdf50b80 1 -- 172.21.15.40:0/1632867952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55c1065402b0 con 0x55c1067c8080 2023-07-18T03:41:55.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.245+0000 7f35caffd700 1 -- 172.21.15.40:0/1632867952 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e4099fe0 con 0x55c1067c8080 2023-07-18T03:41:55.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.246+0000 7f35caffd700 1 -- 172.21.15.40:0/1632867952 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 72) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f35dc00fb90 con 0x55c1067c8080 2023-07-18T03:41:55.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.246+0000 7f35fdf50b80 10 client.17025 fetch_fsmap finished waiting for FSMap version 72 2023-07-18T03:41:55.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.246+0000 7f35fdf50b80 10 client.17025 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-18T03:41:55.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.246+0000 7f35fdf50b80 20 client.17025 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:55.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.246+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/1632867952 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55c1068356e0 0x55c106837af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.246+0000 7f35fdf50b80 4 client.17025 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:41:55.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.246+0000 7f35fdf50b80 1 -- 172.21.15.40:0/1632867952 --> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c1066cf050 con 0x55c1068356e0 2023-07-18T03:41:55.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.246+0000 7f35eb7fe700 1 --2- 172.21.15.40:0/1632867952 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55c1068356e0 0x55c106837af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.246+0000 7f35caffd700 10 client.17025 ms_handle_connect on v2:172.21.15.64:6836/3077792296 2023-07-18T03:41:55.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.246+0000 7f35eb7fe700 1 --2- 172.21.15.40:0/1632867952 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55c1068356e0 0x55c106837af0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0a69a0cd0fff126 server_cookie=24c9106e0587b2d4 in_seq=0 out_seq=0 2023-07-18T03:41:55.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.246+0000 7f35caffd700 1 -- 172.21.15.40:0/1632867952 <== mds.2 v2:172.21.15.64:6836/3077792296 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c1066cf050 con 0x55c1068356e0 2023-07-18T03:41:55.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.246+0000 7f35caffd700 10 client.17025 handle_command_reply: tid=0 2023-07-18T03:41:55.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.246+0000 7f35fdf50b80 1 client.17025 shutdown 2023-07-18T03:41:55.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.246+0000 7f35fdf50b80 1 -- 172.21.15.40:0/1632867952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c8080 msgr2=0x55c1067d7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/1632867952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c8080 0x55c1067d7a30 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009990 tx=0x7f35dc005c20 comp rx=0 tx=0).stop 2023-07-18T03:41:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35fdf50b80 1 -- 172.21.15.40:0/1632867952 shutdown_connections 2023-07-18T03:41:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/1632867952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c1067c8080 0x55c1067d7a30 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/1632867952 >> [v2:172.21.15.64:6836/3077792296,v1:172.21.15.64:6837/3077792296] conn(0x55c1068356e0 0x55c106837af0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/1632867952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1067c68e0 0x55c1067c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35caffd700 0 client.17025 ms_handle_reset on v2:172.21.15.64:6836/3077792296 2023-07-18T03:41:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35fdf50b80 1 --2- 172.21.15.40:0/1632867952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c1067c7230 0x55c1067c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35fdf50b80 1 -- 172.21.15.40:0/1632867952 >> 172.21.15.40:0/1632867952 conn(0x55c1067b7da0 msgr2=0x55c1067ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35fdf50b80 1 -- 172.21.15.40:0/1632867952 shutdown_connections 2023-07-18T03:41:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35fdf50b80 1 -- 172.21.15.40:0/1632867952 wait complete. 2023-07-18T03:41:55.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35fdf50b80 20 client.17025 trim_cache size 0 max 16384 2023-07-18T03:41:55.252 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:41:55.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35f1dca700 1 -- 172.21.15.40:0/3366513544 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35d40a1150 msgr2=0x7f35d40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35f1dca700 1 --2- 172.21.15.40:0/3366513544 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35d40a1150 0x7f35d40a3600 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009c60 tx=0x7f35e0009260 comp rx=0 tx=0).stop 2023-07-18T03:41:55.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35f1dca700 1 -- 172.21.15.40:0/3366513544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35ec100600 msgr2=0x7f35ec195780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35f1dca700 1 --2- 172.21.15.40:0/3366513544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35ec100600 0x7f35ec195780 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f35e400ef10 tx=0x7f35e400c3a0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35f1dca700 1 -- 172.21.15.40:0/3366513544 shutdown_connections 2023-07-18T03:41:55.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35f1dca700 1 --2- 172.21.15.40:0/3366513544 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35d40a1150 0x7f35d40a3600 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35f1dca700 1 --2- 172.21.15.40:0/3366513544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35ec105840 0x7f35ec19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35f1dca700 1 --2- 172.21.15.40:0/3366513544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35ec100600 0x7f35ec195780 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35f1dca700 1 --2- 172.21.15.40:0/3366513544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35ec102f20 0x7f35ec195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35f1dca700 1 -- 172.21.15.40:0/3366513544 >> 172.21.15.40:0/3366513544 conn(0x7f35ec0fa000 msgr2=0x7f35ec101280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35f1dca700 1 -- 172.21.15.40:0/3366513544 shutdown_connections 2023-07-18T03:41:55.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.247+0000 7f35f1dca700 1 -- 172.21.15.40:0/3366513544 wait complete. 2023-07-18T03:41:55.255 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-18T03:41:55.313 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:54 smithi134 ceph-mon[98216]: pgmap v625: 97 pgs: 97 active+clean; 710 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 486 op/s 2023-07-18T03:41:55.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:54 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3368095513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:41:55.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:54 smithi064 ceph-mon[98059]: pgmap v625: 97 pgs: 97 active+clean; 710 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 486 op/s 2023-07-18T03:41:55.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:54 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3368095513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:41:55.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.389+0000 7f918650b700 1 -- 172.21.15.40:0/706162647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9180100ce0 msgr2=0x7f9180101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.389+0000 7f918650b700 1 --2- 172.21.15.40:0/706162647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9180100ce0 0x7f9180101130 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f9174009990 tx=0x7f9174009ca0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.389+0000 7f918650b700 1 -- 172.21.15.40:0/706162647 shutdown_connections 2023-07-18T03:41:55.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.389+0000 7f918650b700 1 --2- 172.21.15.40:0/706162647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9180101670 0x7f9180107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.389+0000 7f918650b700 1 --2- 172.21.15.40:0/706162647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91800ffaf0 0x7f91800fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.389+0000 7f918650b700 1 --2- 172.21.15.40:0/706162647 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9180100ce0 0x7f9180101130 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.389+0000 7f918650b700 1 -- 172.21.15.40:0/706162647 >> 172.21.15.40:0/706162647 conn(0x7f91800fb130 msgr2=0x7f91800fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.390+0000 7f918650b700 1 -- 172.21.15.40:0/706162647 shutdown_connections 2023-07-18T03:41:55.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.390+0000 7f918650b700 1 -- 172.21.15.40:0/706162647 wait complete. 2023-07-18T03:41:55.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.390+0000 7f918650b700 1 Processor -- start 2023-07-18T03:41:55.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.390+0000 7f918650b700 1 -- start start 2023-07-18T03:41:55.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f918650b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91800ffaf0 0x7f91800742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f918650b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9180100ce0 0x7f9180072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f918650b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9180101670 0x7f9180072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f918650b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91800735e0 con 0x7f91800ffaf0 2023-07-18T03:41:55.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f918650b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9180073750 con 0x7f9180100ce0 2023-07-18T03:41:55.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f918650b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918019f890 con 0x7f9180101670 2023-07-18T03:41:55.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f917ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91800ffaf0 0x7f91800742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f917ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91800ffaf0 0x7f91800742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33660/0 (socket says 172.21.15.40:33660) 2023-07-18T03:41:55.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f917ffff700 1 -- 172.21.15.40:0/2307369576 learned_addr learned my addr 172.21.15.40:0/2307369576 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f9184aa8700 1 --2- 172.21.15.40:0/2307369576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9180101670 0x7f9180072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f917f7fe700 1 --2- 172.21.15.40:0/2307369576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9180100ce0 0x7f9180072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f917ffff700 1 -- 172.21.15.40:0/2307369576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9180101670 msgr2=0x7f9180072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f917ffff700 1 --2- 172.21.15.40:0/2307369576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9180101670 0x7f9180072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f917ffff700 1 -- 172.21.15.40:0/2307369576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9180100ce0 msgr2=0x7f9180072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f917ffff700 1 --2- 172.21.15.40:0/2307369576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9180100ce0 0x7f9180072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f917ffff700 1 -- 172.21.15.40:0/2307369576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91740095f0 con 0x7f91800ffaf0 2023-07-18T03:41:55.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f917f7fe700 1 --2- 172.21.15.40:0/2307369576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9180100ce0 0x7f9180072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:41:55.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f917ffff700 1 --2- 172.21.15.40:0/2307369576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91800ffaf0 0x7f91800742c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f917000d980 tx=0x7f917000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f917d7fa700 1 -- 172.21.15.40:0/2307369576 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917000b950 con 0x7f91800ffaf0 2023-07-18T03:41:55.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f917d7fa700 1 -- 172.21.15.40:0/2307369576 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9170007620 con 0x7f91800ffaf0 2023-07-18T03:41:55.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f918650b700 1 -- 172.21.15.40:0/2307369576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918019fb70 con 0x7f91800ffaf0 2023-07-18T03:41:55.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f917d7fa700 1 -- 172.21.15.40:0/2307369576 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917000f3c0 con 0x7f91800ffaf0 2023-07-18T03:41:55.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.391+0000 7f918650b700 1 -- 172.21.15.40:0/2307369576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91801a00d0 con 0x7f91800ffaf0 2023-07-18T03:41:55.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.394+0000 7f917d7fa700 1 -- 172.21.15.40:0/2307369576 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9170010640 con 0x7f91800ffaf0 2023-07-18T03:41:55.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.395+0000 7f917d7fa700 1 --2- 172.21.15.40:0/2307369576 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f91680a11d0 0x7f91680a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.395+0000 7f917f7fe700 1 --2- 172.21.15.40:0/2307369576 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f91680a11d0 0x7f91680a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.395+0000 7f917d7fa700 1 -- 172.21.15.40:0/2307369576 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f91700cc760 con 0x7f91800ffaf0 2023-07-18T03:41:55.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.395+0000 7f917f7fe700 1 --2- 172.21.15.40:0/2307369576 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f91680a11d0 0x7f91680a3680 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f917400b2e0 tx=0x7f9174005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.395+0000 7f9192691b80 1 Processor -- start 2023-07-18T03:41:55.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.395+0000 7f9192691b80 1 -- start start 2023-07-18T03:41:55.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f9192691b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a952974d0 0x555a952978e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f9192691b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a95297e20 0x555a952986e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f9192691b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a95298c70 0x555a952990e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f9192691b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a9519ccb0 con 0x555a952974d0 2023-07-18T03:41:55.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f9192691b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a951efa40 con 0x555a95297e20 2023-07-18T03:41:55.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f9192691b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a951e8ed0 con 0x555a95298c70 2023-07-18T03:41:55.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f917ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a952974d0 0x555a952978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f917ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a952974d0 0x555a952978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33668/0 (socket says 172.21.15.40:33668) 2023-07-18T03:41:55.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f917ffff700 1 -- 172.21.15.40:0/1484157960 learned_addr learned my addr 172.21.15.40:0/1484157960 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f9184aa8700 1 --2- 172.21.15.40:0/1484157960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a95298c70 0x555a952990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f917f7fe700 1 --2- 172.21.15.40:0/1484157960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a95297e20 0x555a952986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f917ffff700 1 -- 172.21.15.40:0/1484157960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a95298c70 msgr2=0x555a952990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f917ffff700 1 --2- 172.21.15.40:0/1484157960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a95298c70 0x555a952990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f917ffff700 1 -- 172.21.15.40:0/1484157960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a95297e20 msgr2=0x555a952986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f917ffff700 1 --2- 172.21.15.40:0/1484157960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a95297e20 0x555a952986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f917ffff700 1 -- 172.21.15.40:0/1484157960 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a95016de0 con 0x555a952974d0 2023-07-18T03:41:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f917ffff700 1 --2- 172.21.15.40:0/1484157960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a952974d0 0x555a952978e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9170096050 tx=0x7f9170010530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa23f24d937e552 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f91667fc700 1 -- 172.21.15.40:0/1484157960 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91700a1fa0 con 0x555a952974d0 2023-07-18T03:41:55.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f91667fc700 1 -- 172.21.15.40:0/1484157960 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9170022e10 con 0x555a952974d0 2023-07-18T03:41:55.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.396+0000 7f91667fc700 1 -- 172.21.15.40:0/1484157960 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9170099280 con 0x555a952974d0 2023-07-18T03:41:55.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.397+0000 7f9192691b80 1 -- 172.21.15.40:0/1484157960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a952974d0 msgr2=0x555a952978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.397+0000 7f9192691b80 1 --2- 172.21.15.40:0/1484157960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a952974d0 0x555a952978e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9170096050 tx=0x7f9170010530 comp rx=0 tx=0).stop 2023-07-18T03:41:55.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.397+0000 7f9192691b80 1 -- 172.21.15.40:0/1484157960 shutdown_connections 2023-07-18T03:41:55.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.397+0000 7f9192691b80 1 --2- 172.21.15.40:0/1484157960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a95298c70 0x555a952990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.397+0000 7f9192691b80 1 --2- 172.21.15.40:0/1484157960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a952974d0 0x555a952978e0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.397+0000 7f9192691b80 1 --2- 172.21.15.40:0/1484157960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a95297e20 0x555a952986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.397+0000 7f9192691b80 1 -- 172.21.15.40:0/1484157960 >> 172.21.15.40:0/1484157960 conn(0x555a9528cda0 msgr2=0x555a952921d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.397+0000 7f9192691b80 1 -- 172.21.15.40:0/1484157960 shutdown_connections 2023-07-18T03:41:55.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.397+0000 7f9192691b80 1 -- 172.21.15.40:0/1484157960 wait complete. 2023-07-18T03:41:55.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.397+0000 7f9192691b80 1 -- start start 2023-07-18T03:41:55.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f9192691b80 1 Processor -- start 2023-07-18T03:41:55.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f9192691b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a952974d0 0x555a952a1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f9192691b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a95297e20 0x555a952a1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f9192691b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a95298c70 0x555a952a86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f9192691b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a95188ac0 con 0x555a952974d0 2023-07-18T03:41:55.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f9192691b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a951887c0 con 0x555a95298c70 2023-07-18T03:41:55.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f9192691b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a9506c920 con 0x555a95297e20 2023-07-18T03:41:55.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f9192691b80 10 client.0 _ll_register_callbacks cb 0x555a95186760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:55.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f917f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a95297e20 0x555a952a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f917ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a952974d0 0x555a952a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f917f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a95297e20 0x555a952a1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50936/0 (socket says 172.21.15.40:50936) 2023-07-18T03:41:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f917f7fe700 1 -- 172.21.15.40:0/2620023273 learned_addr learned my addr 172.21.15.40:0/2620023273 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f9184aa8700 1 --2- 172.21.15.40:0/2620023273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a95298c70 0x555a952a86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f917ffff700 1 -- 172.21.15.40:0/2620023273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a95297e20 msgr2=0x555a952a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f917ffff700 1 --2- 172.21.15.40:0/2620023273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a95297e20 0x555a952a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f917ffff700 1 -- 172.21.15.40:0/2620023273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a95298c70 msgr2=0x555a952a86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f917ffff700 1 --2- 172.21.15.40:0/2620023273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a95298c70 0x555a952a86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.398+0000 7f917ffff700 1 -- 172.21.15.40:0/2620023273 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91801a00d0 con 0x555a952974d0 2023-07-18T03:41:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.399+0000 7f917ffff700 1 --2- 172.21.15.40:0/2620023273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a952974d0 0x555a952a1130 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9170099800 tx=0x7f917009dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cec5db3b0484c7b server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.399+0000 7f915f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:41:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.399+0000 7f915f7fe700 1 -- 172.21.15.40:0/2620023273 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91700a1fa0 con 0x555a952974d0 2023-07-18T03:41:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.399+0000 7f9192691b80 1 -- 172.21.15.40:0/2620023273 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a951424f0 con 0x555a952974d0 2023-07-18T03:41:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.399+0000 7f915f7fe700 1 -- 172.21.15.40:0/2620023273 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f9170099040 con 0x555a952974d0 2023-07-18T03:41:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.399+0000 7f915f7fe700 1 -- 172.21.15.40:0/2620023273 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9170099240 con 0x555a952974d0 2023-07-18T03:41:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.399+0000 7f9192691b80 10 client.17037 fetch_fsmap learned FSMap version 72 2023-07-18T03:41:55.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.399+0000 7f9192691b80 1 -- 172.21.15.40:0/2620023273 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x555a94fffc60 con 0x555a952974d0 2023-07-18T03:41:55.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.399+0000 7f915f7fe700 1 -- 172.21.15.40:0/2620023273 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f917007c080 con 0x555a952974d0 2023-07-18T03:41:55.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.399+0000 7f915f7fe700 1 -- 172.21.15.40:0/2620023273 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 72) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f9170098ab0 con 0x555a952974d0 2023-07-18T03:41:55.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.399+0000 7f9192691b80 10 client.17037 fetch_fsmap finished waiting for FSMap version 72 2023-07-18T03:41:55.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.399+0000 7f9192691b80 10 client.17037 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-18T03:41:55.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.399+0000 7f9192691b80 20 client.17037 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:55.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.399+0000 7f9192691b80 1 --2- 172.21.15.40:0/2620023273 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x555a952acb00 0x555a952aef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.399+0000 7f9192691b80 4 client.17037 mds_command: new command op to 24523 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:41:55.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.399+0000 7f9192691b80 1 -- 172.21.15.40:0/2620023273 --> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a951e1eb0 con 0x555a952acb00 2023-07-18T03:41:55.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.400+0000 7f917f7fe700 1 --2- 172.21.15.40:0/2620023273 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x555a952acb00 0x555a952aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.400+0000 7f915f7fe700 10 client.17037 ms_handle_connect on v2:172.21.15.134:6838/3192254101 2023-07-18T03:41:55.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.400+0000 7f917f7fe700 1 --2- 172.21.15.40:0/2620023273 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x555a952acb00 0x555a952aef10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75a7da0d671bb155 server_cookie=28d408d5cdd07b64 in_seq=0 out_seq=0 2023-07-18T03:41:55.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.401+0000 7f915f7fe700 1 -- 172.21.15.40:0/2620023273 <== mds.1 v2:172.21.15.134:6838/3192254101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555a951e1eb0 con 0x555a952acb00 2023-07-18T03:41:55.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.401+0000 7f915f7fe700 10 client.17037 handle_command_reply: tid=0 2023-07-18T03:41:55.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.401+0000 7f9192691b80 1 client.17037 shutdown 2023-07-18T03:41:55.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.401+0000 7f9192691b80 1 -- 172.21.15.40:0/2620023273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a952974d0 msgr2=0x555a952a1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.401+0000 7f9192691b80 1 --2- 172.21.15.40:0/2620023273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a952974d0 0x555a952a1130 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9170099800 tx=0x7f917009dcb0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.401+0000 7f9192691b80 1 -- 172.21.15.40:0/2620023273 shutdown_connections 2023-07-18T03:41:55.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.401+0000 7f9192691b80 1 --2- 172.21.15.40:0/2620023273 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x555a952acb00 0x555a952aef10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.401+0000 7f9192691b80 1 --2- 172.21.15.40:0/2620023273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a95298c70 0x555a952a86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.401+0000 7f9192691b80 1 --2- 172.21.15.40:0/2620023273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a952974d0 0x555a952a1130 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.401+0000 7f9192691b80 1 --2- 172.21.15.40:0/2620023273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a95297e20 0x555a952a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.402+0000 7f9192691b80 1 -- 172.21.15.40:0/2620023273 >> 172.21.15.40:0/2620023273 conn(0x555a9528cda0 msgr2=0x555a952918b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.402+0000 7f915f7fe700 0 client.17037 ms_handle_reset on v2:172.21.15.134:6838/3192254101 2023-07-18T03:41:55.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.402+0000 7f9192691b80 1 -- 172.21.15.40:0/2620023273 shutdown_connections 2023-07-18T03:41:55.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.402+0000 7f9192691b80 1 -- 172.21.15.40:0/2620023273 wait complete. 2023-07-18T03:41:55.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.402+0000 7f9192691b80 20 client.17037 trim_cache size 0 max 16384 2023-07-18T03:41:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.415+0000 7f9192691b80 1 Processor -- start 2023-07-18T03:41:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.415+0000 7f9192691b80 1 -- start start 2023-07-18T03:41:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.415+0000 7f9192691b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a952974d0 0x555a952990c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.415+0000 7f9192691b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a95297e20 0x555a952a8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.415+0000 7f9192691b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a95298c70 0x555a952b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.415+0000 7f9192691b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a952b3db0 con 0x555a95298c70 2023-07-18T03:41:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.416+0000 7f9192691b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a9529f9c0 con 0x555a95297e20 2023-07-18T03:41:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.416+0000 7f9192691b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a9529fb80 con 0x555a952974d0 2023-07-18T03:41:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.416+0000 7f9184aa8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a95298c70 0x555a952b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.416+0000 7f917ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a952974d0 0x555a952990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.416+0000 7f917ffff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a952974d0 0x555a952990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:50942/0 (socket says 172.21.15.40:50942) 2023-07-18T03:41:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.416+0000 7f917f7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a95297e20 0x555a952a8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.416+0000 7f917ffff700 1 -- 172.21.15.40:0/3533979774 learned_addr learned my addr 172.21.15.40:0/3533979774 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.416+0000 7f917f7fe700 1 -- 172.21.15.40:0/3533979774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a952974d0 msgr2=0x555a952990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.416+0000 7f917f7fe700 1 --2- 172.21.15.40:0/3533979774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a952974d0 0x555a952990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.416+0000 7f917f7fe700 1 -- 172.21.15.40:0/3533979774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a95298c70 msgr2=0x555a952b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.416+0000 7f917f7fe700 1 --2- 172.21.15.40:0/3533979774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a95298c70 0x555a952b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.416+0000 7f917f7fe700 1 -- 172.21.15.40:0/3533979774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a9503d250 con 0x555a95297e20 2023-07-18T03:41:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.416+0000 7f917f7fe700 1 --2- 172.21.15.40:0/3533979774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a95297e20 0x555a952a8670 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f917400b620 tx=0x7f9174022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=236439dd8495ba21 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.416+0000 7f9166ffd700 1 -- 172.21.15.40:0/3533979774 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917402d070 con 0x555a95297e20 2023-07-18T03:41:55.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.416+0000 7f9166ffd700 1 -- 172.21.15.40:0/3533979774 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f917402e040 con 0x555a95297e20 2023-07-18T03:41:55.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.416+0000 7f9166ffd700 1 -- 172.21.15.40:0/3533979774 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174025d70 con 0x555a95297e20 2023-07-18T03:41:55.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.417+0000 7f9192691b80 1 -- 172.21.15.40:0/3533979774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a95297e20 msgr2=0x555a952a8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.417+0000 7f9192691b80 1 --2- 172.21.15.40:0/3533979774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a95297e20 0x555a952a8670 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f917400b620 tx=0x7f9174022000 comp rx=0 tx=0).stop 2023-07-18T03:41:55.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.417+0000 7f9192691b80 1 -- 172.21.15.40:0/3533979774 shutdown_connections 2023-07-18T03:41:55.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.417+0000 7f9192691b80 1 --2- 172.21.15.40:0/3533979774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a95298c70 0x555a952b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.417+0000 7f9192691b80 1 --2- 172.21.15.40:0/3533979774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a952974d0 0x555a952990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.417+0000 7f9192691b80 1 --2- 172.21.15.40:0/3533979774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a95297e20 0x555a952a8670 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.417+0000 7f9192691b80 1 -- 172.21.15.40:0/3533979774 >> 172.21.15.40:0/3533979774 conn(0x555a9528cda0 msgr2=0x555a9528e310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.417+0000 7f9192691b80 1 -- 172.21.15.40:0/3533979774 shutdown_connections 2023-07-18T03:41:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.417+0000 7f9192691b80 1 -- 172.21.15.40:0/3533979774 wait complete. 2023-07-18T03:41:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.417+0000 7f9192691b80 1 -- start start 2023-07-18T03:41:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f9192691b80 1 Processor -- start 2023-07-18T03:41:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f9192691b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a952974d0 0x555a952a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f9192691b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a95297e20 0x555a952a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f9192691b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a95298c70 0x555a952a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f9192691b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a952a7f60 con 0x555a95297e20 2023-07-18T03:41:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f9192691b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a952a80d0 con 0x555a952974d0 2023-07-18T03:41:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f9192691b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a952a8290 con 0x555a95298c70 2023-07-18T03:41:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f9192691b80 10 client.0 _ll_register_callbacks cb 0x7f9174005630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f917f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a95297e20 0x555a952a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f917ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a952974d0 0x555a952a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f9184aa8700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a95298c70 0x555a952a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f917f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a95297e20 0x555a952a4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33690/0 (socket says 172.21.15.40:33690) 2023-07-18T03:41:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f917ffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a952974d0 0x555a952a44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:45246/0 (socket says 172.21.15.40:45246) 2023-07-18T03:41:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f917f7fe700 1 -- 172.21.15.40:0/971397415 learned_addr learned my addr 172.21.15.40:0/971397415 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f917ffff700 1 -- 172.21.15.40:0/971397415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a95298c70 msgr2=0x555a952a7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f917ffff700 1 --2- 172.21.15.40:0/971397415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a95298c70 0x555a952a7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f917ffff700 1 -- 172.21.15.40:0/971397415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a95297e20 msgr2=0x555a952a4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f917ffff700 1 --2- 172.21.15.40:0/971397415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a95297e20 0x555a952a4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.418+0000 7f917ffff700 1 -- 172.21.15.40:0/971397415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9180108860 con 0x555a952974d0 2023-07-18T03:41:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.419+0000 7f917f7fe700 1 --2- 172.21.15.40:0/971397415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a95297e20 0x555a952a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.419+0000 7f917ffff700 1 --2- 172.21.15.40:0/971397415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a952974d0 0x555a952a44c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9170096050 tx=0x7f9170010a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5580ab9c8f040f0a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.419+0000 7f915f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:41:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.419+0000 7f915f7fe700 1 -- 172.21.15.40:0/971397415 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91700a1d10 con 0x555a952974d0 2023-07-18T03:41:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.419+0000 7f915f7fe700 1 -- 172.21.15.40:0/971397415 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f91700ca090 con 0x555a952974d0 2023-07-18T03:41:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.419+0000 7f9192691b80 1 -- 172.21.15.40:0/971397415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9170022e10 con 0x555a952974d0 2023-07-18T03:41:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.419+0000 7f915f7fe700 1 -- 172.21.15.40:0/971397415 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9170099fc0 con 0x555a952974d0 2023-07-18T03:41:55.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.419+0000 7f9192691b80 10 client.26672 fetch_fsmap learned FSMap version 72 2023-07-18T03:41:55.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.419+0000 7f9192691b80 1 -- 172.21.15.40:0/971397415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x555a950152b0 con 0x555a952974d0 2023-07-18T03:41:55.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.419+0000 7f915f7fe700 1 -- 172.21.15.40:0/971397415 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9170022e10 con 0x555a952974d0 2023-07-18T03:41:55.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.419+0000 7f915f7fe700 1 -- 172.21.15.40:0/971397415 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 72) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f917000bab0 con 0x555a952974d0 2023-07-18T03:41:55.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.419+0000 7f9192691b80 10 client.26672 fetch_fsmap finished waiting for FSMap version 72 2023-07-18T03:41:55.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.419+0000 7f9192691b80 10 client.26672 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-07-18T03:41:55.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.419+0000 7f9192691b80 20 client.26672 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:55.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.419+0000 7f9192691b80 1 --2- 172.21.15.40:0/971397415 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x555a95301600 0x555a95303a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.419+0000 7f9192691b80 4 client.26672 mds_command: new command op to 24523 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:41:55.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.419+0000 7f9192691b80 1 -- 172.21.15.40:0/971397415 --> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555a951a4050 con 0x555a95301600 2023-07-18T03:41:55.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.420+0000 7f917f7fe700 1 --2- 172.21.15.40:0/971397415 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x555a95301600 0x555a95303a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.420+0000 7f917f7fe700 1 --2- 172.21.15.40:0/971397415 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x555a95301600 0x555a95303a10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b88b80a7bb2bf30d server_cookie=31dab8b8d5c0c6ea in_seq=0 out_seq=0 2023-07-18T03:41:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.420+0000 7f915f7fe700 10 client.26672 ms_handle_connect on v2:172.21.15.134:6838/3192254101 2023-07-18T03:41:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.422+0000 7f915f7fe700 1 -- 172.21.15.40:0/971397415 <== mds.1 v2:172.21.15.134:6838/3192254101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+47588 (crc 0 0 0) 0x555a951a4050 con 0x555a95301600 2023-07-18T03:41:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.422+0000 7f915f7fe700 10 client.26672 handle_command_reply: tid=0 2023-07-18T03:41:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.422+0000 7f9192691b80 1 client.26672 shutdown 2023-07-18T03:41:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.422+0000 7f9192691b80 1 -- 172.21.15.40:0/971397415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a952974d0 msgr2=0x555a952a44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.422+0000 7f9192691b80 1 --2- 172.21.15.40:0/971397415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a952974d0 0x555a952a44c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9170096050 tx=0x7f9170010a40 comp rx=0 tx=0).stop 2023-07-18T03:41:55.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f9192691b80 1 -- 172.21.15.40:0/971397415 shutdown_connections 2023-07-18T03:41:55.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f9192691b80 1 --2- 172.21.15.40:0/971397415 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a95298c70 0x555a952a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f9192691b80 1 --2- 172.21.15.40:0/971397415 >> [v2:172.21.15.134:6838/3192254101,v1:172.21.15.134:6839/3192254101] conn(0x555a95301600 0x555a95303a10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f9192691b80 1 --2- 172.21.15.40:0/971397415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555a952974d0 0x555a952a44c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f9192691b80 1 --2- 172.21.15.40:0/971397415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x555a95297e20 0x555a952a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f915f7fe700 0 client.26672 ms_handle_reset on v2:172.21.15.134:6838/3192254101 2023-07-18T03:41:55.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f9192691b80 1 -- 172.21.15.40:0/971397415 >> 172.21.15.40:0/971397415 conn(0x555a9528cda0 msgr2=0x555a9529f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f9192691b80 1 -- 172.21.15.40:0/971397415 shutdown_connections 2023-07-18T03:41:55.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f9192691b80 1 -- 172.21.15.40:0/971397415 wait complete. 2023-07-18T03:41:55.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f9192691b80 20 client.26672 trim_cache size 0 max 16384 2023-07-18T03:41:55.427 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-07-18T03:41:55.428 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.428 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.428 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 11741206, 2023-07-18T03:41:55.428 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628660, 2023-07-18T03:41:55.428 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-27.xml" 2023-07-18T03:41:55.428 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.428 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.429 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.429 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 13494289, 2023-07-18T03:41:55.429 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628681, 2023-07-18T03:41:55.429 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-6.xml" 2023-07-18T03:41:55.429 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.429 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.429 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.430 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24528367, 2023-07-18T03:41:55.430 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259906, 2023-07-18T03:41:55.430 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-13.xml" 2023-07-18T03:41:55.430 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.430 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.430 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.430 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 77849323, 2023-07-18T03:41:55.430 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628636, 2023-07-18T03:41:55.431 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-51.xml" 2023-07-18T03:41:55.431 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.431 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.431 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.431 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 148316563, 2023-07-18T03:41:55.431 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260220, 2023-07-18T03:41:55.431 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-19.jpg" 2023-07-18T03:41:55.432 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.432 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.432 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.432 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 158417010, 2023-07-18T03:41:55.432 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260133, 2023-07-18T03:41:55.432 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-39.jpg" 2023-07-18T03:41:55.432 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.432 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.433 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.433 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 201558950, 2023-07-18T03:41:55.433 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628662, 2023-07-18T03:41:55.433 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-25.xml" 2023-07-18T03:41:55.433 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.433 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.433 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.434 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 237222145, 2023-07-18T03:41:55.434 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260018, 2023-07-18T03:41:55.434 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-87.jpg" 2023-07-18T03:41:55.434 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.434 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.434 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.434 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 308462129, 2023-07-18T03:41:55.434 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260055, 2023-07-18T03:41:55.435 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-65.jpg" 2023-07-18T03:41:55.435 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.435 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.435 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.435 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 333789564, 2023-07-18T03:41:55.435 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260046, 2023-07-18T03:41:55.435 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-70.jpg" 2023-07-18T03:41:55.435 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.436 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.436 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.436 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 339575889, 2023-07-18T03:41:55.436 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259900, 2023-07-18T03:41:55.436 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-15.xml" 2023-07-18T03:41:55.436 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.436 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.436 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.437 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 349062042, 2023-07-18T03:41:55.437 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260253, 2023-07-18T03:41:55.437 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-12.jpg" 2023-07-18T03:41:55.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.437 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.437 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.437 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 370265129, 2023-07-18T03:41:55.438 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260026, 2023-07-18T03:41:55.438 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-82.jpg" 2023-07-18T03:41:55.438 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.438 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.438 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.438 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 379202467, 2023-07-18T03:41:55.438 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628676, 2023-07-18T03:41:55.438 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-11.xml" 2023-07-18T03:41:55.439 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.439 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.439 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.439 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 446192629, 2023-07-18T03:41:55.439 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260295, 2023-07-18T03:41:55.439 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-1.jpg" 2023-07-18T03:41:55.439 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.439 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.440 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.440 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 509020740, 2023-07-18T03:41:55.440 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628651, 2023-07-18T03:41:55.440 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-36.xml" 2023-07-18T03:41:55.440 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.440 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.440 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.441 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 542126001, 2023-07-18T03:41:55.441 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260069, 2023-07-18T03:41:55.441 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-56.jpg" 2023-07-18T03:41:55.441 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.441 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.441 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.441 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 543991973, 2023-07-18T03:41:55.441 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260177, 2023-07-18T03:41:55.442 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-27.jpg" 2023-07-18T03:41:55.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.442 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.442 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.442 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 564941701, 2023-07-18T03:41:55.442 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260203, 2023-07-18T03:41:55.442 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-23.jpg" 2023-07-18T03:41:55.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.443 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.443 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.443 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 576188789, 2023-07-18T03:41:55.443 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629293, 2023-07-18T03:41:55.443 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c" 2023-07-18T03:41:55.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.443 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.444 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.444 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 591146075, 2023-07-18T03:41:55.444 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260214, 2023-07-18T03:41:55.444 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-21.jpg" 2023-07-18T03:41:55.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.444 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.444 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.444 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 613330314, 2023-07-18T03:41:55.445 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260154, 2023-07-18T03:41:55.445 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-32.jpg" 2023-07-18T03:41:55.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.445 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.445 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.445 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 657408747, 2023-07-18T03:41:55.445 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628680, 2023-07-18T03:41:55.446 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-7.xml" 2023-07-18T03:41:55.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.446 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.446 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.446 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 672447047, 2023-07-18T03:41:55.446 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260197, 2023-07-18T03:41:55.446 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-24.jpg" 2023-07-18T03:41:55.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.447 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.447 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 684236234, 2023-07-18T03:41:55.447 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260009, 2023-07-18T03:41:55.447 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-2.xml" 2023-07-18T03:41:55.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.447 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.448 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 705475298, 2023-07-18T03:41:55.448 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260267, 2023-07-18T03:41:55.448 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-8.jpg" 2023-07-18T03:41:55.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.448 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.448 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 738826528, 2023-07-18T03:41:55.448 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260208, 2023-07-18T03:41:55.449 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-22.jpg" 2023-07-18T03:41:55.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.449 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.449 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.449 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 826667852, 2023-07-18T03:41:55.449 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260058, 2023-07-18T03:41:55.449 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-63.jpg" 2023-07-18T03:41:55.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.450 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.450 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 902155430, 2023-07-18T03:41:55.450 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260017, 2023-07-18T03:41:55.450 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-88.jpg" 2023-07-18T03:41:55.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.451 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.451 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 921751974, 2023-07-18T03:41:55.451 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260063, 2023-07-18T03:41:55.451 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-60.jpg" 2023-07-18T03:41:55.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.451 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.451 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 931321988, 2023-07-18T03:41:55.452 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259996, 2023-07-18T03:41:55.452 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-9.xml" 2023-07-18T03:41:55.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.452 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.452 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.452 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 949854085, 2023-07-18T03:41:55.452 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260030, 2023-07-18T03:41:55.452 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-80.jpg" 2023-07-18T03:41:55.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.453 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.453 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 952637168, 2023-07-18T03:41:55.453 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259889, 2023-07-18T03:41:55.453 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-21.xml" 2023-07-18T03:41:55.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.454 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.454 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 958105991, 2023-07-18T03:41:55.454 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259992, 2023-07-18T03:41:55.454 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-11.xml" 2023-07-18T03:41:55.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.454 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.454 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.455 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 980260575, 2023-07-18T03:41:55.455 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259887, 2023-07-18T03:41:55.455 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-22.xml" 2023-07-18T03:41:55.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.455 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.455 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 980561836, 2023-07-18T03:41:55.456 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260263, 2023-07-18T03:41:55.456 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-9.jpg" 2023-07-18T03:41:55.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.456 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.456 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1001805524, 2023-07-18T03:41:55.456 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628646, 2023-07-18T03:41:55.456 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-41.xml" 2023-07-18T03:41:55.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.457 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.457 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1001963806, 2023-07-18T03:41:55.457 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260136, 2023-07-18T03:41:55.457 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-38.jpg" 2023-07-18T03:41:55.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.458 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.458 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1028228729, 2023-07-18T03:41:55.458 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260067, 2023-07-18T03:41:55.458 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-57.jpg" 2023-07-18T03:41:55.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.458 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.458 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1088297899, 2023-07-18T03:41:55.459 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260040, 2023-07-18T03:41:55.459 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-74.jpg" 2023-07-18T03:41:55.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.459 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.459 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.459 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1098561916, 2023-07-18T03:41:55.459 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260159, 2023-07-18T03:41:55.460 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-30.jpg" 2023-07-18T03:41:55.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.460 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.460 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.460 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1102135148, 2023-07-18T03:41:55.460 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260077, 2023-07-18T03:41:55.460 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-53.jpg" 2023-07-18T03:41:55.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.461 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.461 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1116213449, 2023-07-18T03:41:55.461 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629300, 2023-07-18T03:41:55.461 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/rewriter.c" 2023-07-18T03:41:55.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.461 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.462 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1143673065, 2023-07-18T03:41:55.462 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259894, 2023-07-18T03:41:55.462 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-18.xml" 2023-07-18T03:41:55.462 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.462 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.462 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1173197411, 2023-07-18T03:41:55.463 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629297, 2023-07-18T03:41:55.463 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/process.c" 2023-07-18T03:41:55.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.463 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.463 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1198432771, 2023-07-18T03:41:55.463 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628679, 2023-07-18T03:41:55.463 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-8.xml" 2023-07-18T03:41:55.464 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.464 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.464 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.464 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1222272670, 2023-07-18T03:41:55.464 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628657, 2023-07-18T03:41:55.464 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-30.xml" 2023-07-18T03:41:55.464 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.464 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.465 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.465 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1240764602, 2023-07-18T03:41:55.465 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628670, 2023-07-18T03:41:55.465 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-17.xml" 2023-07-18T03:41:55.465 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.465 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.465 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.465 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1256191811, 2023-07-18T03:41:55.466 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260117, 2023-07-18T03:41:55.466 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-43.jpg" 2023-07-18T03:41:55.466 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.466 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.466 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.466 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1256386456, 2023-07-18T03:41:55.466 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260240, 2023-07-18T03:41:55.467 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-14.jpg" 2023-07-18T03:41:55.467 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.467 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.467 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.467 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1272685557, 2023-07-18T03:41:55.467 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260121, 2023-07-18T03:41:55.467 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-42.jpg" 2023-07-18T03:41:55.467 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.468 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.468 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.468 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1301928178, 2023-07-18T03:41:55.468 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260064, 2023-07-18T03:41:55.468 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-59.jpg" 2023-07-18T03:41:55.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.468 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.468 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.469 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1326904727, 2023-07-18T03:41:55.469 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628672, 2023-07-18T03:41:55.469 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-15.xml" 2023-07-18T03:41:55.469 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.469 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.469 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.469 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1336066704, 2023-07-18T03:41:55.470 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260232, 2023-07-18T03:41:55.470 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-16.jpg" 2023-07-18T03:41:55.470 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.470 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.470 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.470 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1343480827, 2023-07-18T03:41:55.470 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260283, 2023-07-18T03:41:55.470 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-4.jpg" 2023-07-18T03:41:55.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.471 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.471 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.471 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1344405975, 2023-07-18T03:41:55.471 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628666, 2023-07-18T03:41:55.471 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-21.xml" 2023-07-18T03:41:55.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.471 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.472 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.472 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1354177727, 2023-07-18T03:41:55.472 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628653, 2023-07-18T03:41:55.472 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-34.xml" 2023-07-18T03:41:55.472 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.472 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.472 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.473 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1365761292, 2023-07-18T03:41:55.473 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628671, 2023-07-18T03:41:55.473 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-16.xml" 2023-07-18T03:41:55.473 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.473 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.473 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.473 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1372739999, 2023-07-18T03:41:55.473 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628685, 2023-07-18T03:41:55.474 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-2.xml" 2023-07-18T03:41:55.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.474 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.474 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.474 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1385197194, 2023-07-18T03:41:55.474 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628658, 2023-07-18T03:41:55.474 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-29.xml" 2023-07-18T03:41:55.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.475 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.475 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.475 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1397819486, 2023-07-18T03:41:55.475 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260105, 2023-07-18T03:41:55.475 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-46.jpg" 2023-07-18T03:41:55.475 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.475 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.475 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.476 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1454150126, 2023-07-18T03:41:55.476 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259891, 2023-07-18T03:41:55.476 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-20.xml" 2023-07-18T03:41:55.476 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.476 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.476 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.476 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1465546320, 2023-07-18T03:41:55.477 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628633, 2023-07-18T03:41:55.477 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-54.xml" 2023-07-18T03:41:55.477 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.477 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.477 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.477 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1495901065, 2023-07-18T03:41:55.477 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629302, 2023-07-18T03:41:55.477 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/writer.c" 2023-07-18T03:41:55.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.478 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.478 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.478 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1575697613, 2023-07-18T03:41:55.478 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260224, 2023-07-18T03:41:55.478 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-18.jpg" 2023-07-18T03:41:55.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.478 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.479 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.479 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1596502432, 2023-07-18T03:41:55.479 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628686, 2023-07-18T03:41:55.479 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-1.xml" 2023-07-18T03:41:55.479 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.479 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.479 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.479 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1614663084, 2023-07-18T03:41:55.480 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259896, 2023-07-18T03:41:55.480 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-17.xml" 2023-07-18T03:41:55.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.480 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.480 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.480 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1658922089, 2023-07-18T03:41:55.480 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511633460, 2023-07-18T03:41:55.481 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-84.xml" 2023-07-18T03:41:55.481 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.481 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.481 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.481 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1670484460, 2023-07-18T03:41:55.481 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259998, 2023-07-18T03:41:55.481 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-8.xml" 2023-07-18T03:41:55.481 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.482 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.482 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.482 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1676047763, 2023-07-18T03:41:55.482 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629295, 2023-07-18T03:41:55.482 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/Makefile.am" 2023-07-18T03:41:55.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.482 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.483 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.483 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1772848277, 2023-07-18T03:41:55.483 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260141, 2023-07-18T03:41:55.483 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-36.jpg" 2023-07-18T03:41:55.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.483 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.483 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.483 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1780453218, 2023-07-18T03:41:55.484 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628663, 2023-07-18T03:41:55.484 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-24.xml" 2023-07-18T03:41:55.484 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.484 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.484 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.484 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1792486769, 2023-07-18T03:41:55.484 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628639, 2023-07-18T03:41:55.484 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-48.xml" 2023-07-18T03:41:55.485 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.485 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.485 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.485 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1818668934, 2023-07-18T03:41:55.485 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260056, 2023-07-18T03:41:55.485 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-64.jpg" 2023-07-18T03:41:55.485 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.485 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.486 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.486 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1824194765, 2023-07-18T03:41:55.486 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628628, 2023-07-18T03:41:55.486 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-7.xml" 2023-07-18T03:41:55.486 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.486 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.486 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.486 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1853214936, 2023-07-18T03:41:55.487 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629288, 2023-07-18T03:41:55.487 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench.c" 2023-07-18T03:41:55.487 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.487 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.487 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.487 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1882285566, 2023-07-18T03:41:55.487 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628684, 2023-07-18T03:41:55.488 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-3.xml" 2023-07-18T03:41:55.488 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.488 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.488 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.488 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1905678506, 2023-07-18T03:41:55.488 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260097, 2023-07-18T03:41:55.488 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-48.jpg" 2023-07-18T03:41:55.488 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.489 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.489 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.489 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1974198274, 2023-07-18T03:41:55.489 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628673, 2023-07-18T03:41:55.489 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-14.xml" 2023-07-18T03:41:55.489 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.489 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.489 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.490 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1977614919, 2023-07-18T03:41:55.490 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260275, 2023-07-18T03:41:55.490 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-6.jpg" 2023-07-18T03:41:55.490 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.490 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.490 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.490 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2006715941, 2023-07-18T03:41:55.490 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260053, 2023-07-18T03:41:55.491 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-66.jpg" 2023-07-18T03:41:55.491 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.491 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.491 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.491 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2015791459, 2023-07-18T03:41:55.491 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260287, 2023-07-18T03:41:55.491 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-3.jpg" 2023-07-18T03:41:55.491 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.492 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.492 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.492 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2027910790, 2023-07-18T03:41:55.492 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260162, 2023-07-18T03:41:55.492 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-29.jpg" 2023-07-18T03:41:55.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.492 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.493 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.493 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2032097419, 2023-07-18T03:41:55.493 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260291, 2023-07-18T03:41:55.493 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-2.jpg" 2023-07-18T03:41:55.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.493 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.493 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.494 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2057492094, 2023-07-18T03:41:55.494 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260085, 2023-07-18T03:41:55.494 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-51.jpg" 2023-07-18T03:41:55.494 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.494 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.494 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.494 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2059310753, 2023-07-18T03:41:55.494 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260035, 2023-07-18T03:41:55.495 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-77.jpg" 2023-07-18T03:41:55.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.495 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.495 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.495 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2060557685, 2023-07-18T03:41:55.495 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629289, 2023-07-18T03:41:55.495 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench.h" 2023-07-18T03:41:55.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.496 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.496 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.496 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2091244724, 2023-07-18T03:41:55.496 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260279, 2023-07-18T03:41:55.496 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-5.jpg" 2023-07-18T03:41:55.496 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.496 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.497 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.497 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2091429076, 2023-07-18T03:41:55.497 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628667, 2023-07-18T03:41:55.497 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-20.xml" 2023-07-18T03:41:55.497 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.497 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.497 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.497 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2130087739, 2023-07-18T03:41:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629292, 2023-07-18T03:41:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench_p.h" 2023-07-18T03:41:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2139004694, 2023-07-18T03:41:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260081, 2023-07-18T03:41:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-52.jpg" 2023-07-18T03:41:55.499 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.499 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.499 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.499 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2147051305, 2023-07-18T03:41:55.499 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628668, 2023-07-18T03:41:55.499 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-19.xml" 2023-07-18T03:41:55.499 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.499 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.500 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.500 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2151109360, 2023-07-18T03:41:55.500 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260005, 2023-07-18T03:41:55.500 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-4.xml" 2023-07-18T03:41:55.500 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.500 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.500 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.501 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2181379371, 2023-07-18T03:41:55.501 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628648, 2023-07-18T03:41:55.501 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-39.xml" 2023-07-18T03:41:55.501 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.501 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.501 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.501 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2206602670, 2023-07-18T03:41:55.501 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260217, 2023-07-18T03:41:55.502 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-20.jpg" 2023-07-18T03:41:55.502 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.502 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.502 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.502 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2206743024, 2023-07-18T03:41:55.502 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259990, 2023-07-18T03:41:55.502 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-12.xml" 2023-07-18T03:41:55.502 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.503 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.503 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.503 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2274534032, 2023-07-18T03:41:55.503 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628634, 2023-07-18T03:41:55.503 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-53.xml" 2023-07-18T03:41:55.503 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.503 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.503 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.504 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2291716061, 2023-07-18T03:41:55.504 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628674, 2023-07-18T03:41:55.504 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-13.xml" 2023-07-18T03:41:55.504 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.504 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.504 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.504 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2300854195, 2023-07-18T03:41:55.505 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260113, 2023-07-18T03:41:55.505 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-44.jpg" 2023-07-18T03:41:55.505 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.505 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.505 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.505 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2307659677, 2023-07-18T03:41:55.505 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628650, 2023-07-18T03:41:55.505 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-37.xml" 2023-07-18T03:41:55.506 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.506 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.506 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.506 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2330813503, 2023-07-18T03:41:55.506 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260004, 2023-07-18T03:41:55.506 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-5.xml" 2023-07-18T03:41:55.506 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.506 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.507 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.507 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2332688267, 2023-07-18T03:41:55.507 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260021, 2023-07-18T03:41:55.507 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-85.jpg" 2023-07-18T03:41:55.507 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.507 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.507 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.507 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2346182204, 2023-07-18T03:41:55.508 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260048, 2023-07-18T03:41:55.508 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-69.jpg" 2023-07-18T03:41:55.508 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.508 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.508 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.508 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2356280183, 2023-07-18T03:41:55.508 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629296, 2023-07-18T03:41:55.509 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/Makefile.in" 2023-07-18T03:41:55.509 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.509 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.509 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.509 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2401013816, 2023-07-18T03:41:55.509 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260025, 2023-07-18T03:41:55.509 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-83.jpg" 2023-07-18T03:41:55.509 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.510 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.510 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.510 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2429297234, 2023-07-18T03:41:55.510 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260066, 2023-07-18T03:41:55.510 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-58.jpg" 2023-07-18T03:41:55.510 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.510 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.510 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.511 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2441657868, 2023-07-18T03:41:55.511 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628669, 2023-07-18T03:41:55.511 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-18.xml" 2023-07-18T03:41:55.511 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.511 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.511 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.511 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2447037269, 2023-07-18T03:41:55.511 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260089, 2023-07-18T03:41:55.512 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-50.jpg" 2023-07-18T03:41:55.512 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.512 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.512 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.512 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2448157669, 2023-07-18T03:41:55.512 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628682, 2023-07-18T03:41:55.512 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-5.xml" 2023-07-18T03:41:55.513 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.513 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.513 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.513 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2503423314, 2023-07-18T03:41:55.513 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628635, 2023-07-18T03:41:55.513 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-52.xml" 2023-07-18T03:41:55.513 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.513 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.514 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.514 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2549982617, 2023-07-18T03:41:55.514 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628675, 2023-07-18T03:41:55.514 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-12.xml" 2023-07-18T03:41:55.514 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.514 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.514 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.514 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2583187403, 2023-07-18T03:41:55.515 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260228, 2023-07-18T03:41:55.515 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-17.jpg" 2023-07-18T03:41:55.515 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.515 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.515 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.515 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2617926256, 2023-07-18T03:41:55.515 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628683, 2023-07-18T03:41:55.515 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-4.xml" 2023-07-18T03:41:55.516 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.516 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.516 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.516 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2660051437, 2023-07-18T03:41:55.516 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629294, 2023-07-18T03:41:55.516 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h" 2023-07-18T03:41:55.516 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.517 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.517 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.517 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2684107693, 2023-07-18T03:41:55.517 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260015, 2023-07-18T03:41:55.517 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-89.jpg" 2023-07-18T03:41:55.517 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.517 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.517 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.518 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2686660087, 2023-07-18T03:41:55.518 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628661, 2023-07-18T03:41:55.518 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-26.xml" 2023-07-18T03:41:55.518 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.518 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.518 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.518 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2731161142, 2023-07-18T03:41:55.518 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260074, 2023-07-18T03:41:55.519 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-54.jpg" 2023-07-18T03:41:55.519 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.519 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.519 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.519 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2741765460, 2023-07-18T03:41:55.519 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260151, 2023-07-18T03:41:55.519 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-33.jpg" 2023-07-18T03:41:55.520 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.520 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.520 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.520 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2754653532, 2023-07-18T03:41:55.520 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260109, 2023-07-18T03:41:55.520 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-45.jpg" 2023-07-18T03:41:55.520 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.520 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.521 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.521 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2759990393, 2023-07-18T03:41:55.521 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259994, 2023-07-18T03:41:55.521 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-10.xml" 2023-07-18T03:41:55.521 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.521 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.521 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.521 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2782387244, 2023-07-18T03:41:55.522 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260020, 2023-07-18T03:41:55.522 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-86.jpg" 2023-07-18T03:41:55.522 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.522 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.522 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.522 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2794029366, 2023-07-18T03:41:55.522 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260050, 2023-07-18T03:41:55.522 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-68.jpg" 2023-07-18T03:41:55.523 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.523 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.523 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.523 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2805799160, 2023-07-18T03:41:55.523 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260038, 2023-07-18T03:41:55.523 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-75.jpg" 2023-07-18T03:41:55.523 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.524 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.524 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.524 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2833906110, 2023-07-18T03:41:55.524 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511633459, 2023-07-18T03:41:55.524 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-25.jpg" 2023-07-18T03:41:55.524 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.524 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.524 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.525 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2859956841, 2023-07-18T03:41:55.525 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260007, 2023-07-18T03:41:55.525 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-3.xml" 2023-07-18T03:41:55.525 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.525 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.525 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.525 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2869966147, 2023-07-18T03:41:55.525 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260101, 2023-07-18T03:41:55.526 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-47.jpg" 2023-07-18T03:41:55.526 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.526 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.526 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.526 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2872381266, 2023-07-18T03:41:55.526 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629298, 2023-07-18T03:41:55.526 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/globals.h" 2023-07-18T03:41:55.526 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.527 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.527 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.527 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2908291091, 2023-07-18T03:41:55.527 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628641, 2023-07-18T03:41:55.527 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-46.xml" 2023-07-18T03:41:55.527 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.527 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.528 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.528 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2954346201, 2023-07-18T03:41:55.528 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628632, 2023-07-18T03:41:55.528 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-55.xml" 2023-07-18T03:41:55.528 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.528 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.528 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.528 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 2970861627, 2023-07-18T03:41:55.529 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260023, 2023-07-18T03:41:55.529 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-84.jpg" 2023-07-18T03:41:55.529 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.529 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.529 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.529 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3021537020, 2023-07-18T03:41:55.529 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628645, 2023-07-18T03:41:55.529 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-42.xml" 2023-07-18T03:41:55.530 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.530 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.530 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.530 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3023974638, 2023-07-18T03:41:55.530 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260010, 2023-07-18T03:41:55.530 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-1.xml" 2023-07-18T03:41:55.530 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.530 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.531 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.531 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3028491385, 2023-07-18T03:41:55.531 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260271, 2023-07-18T03:41:55.531 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-7.jpg" 2023-07-18T03:41:55.531 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.531 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.531 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.532 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3035945037, 2023-07-18T03:41:55.532 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628640, 2023-07-18T03:41:55.532 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-47.xml" 2023-07-18T03:41:55.532 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.532 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.532 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.532 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3049764835, 2023-07-18T03:41:55.532 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628654, 2023-07-18T03:41:55.533 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-33.xml" 2023-07-18T03:41:55.533 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.533 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.533 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.533 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3050368581, 2023-07-18T03:41:55.533 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260125, 2023-07-18T03:41:55.533 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-41.jpg" 2023-07-18T03:41:55.534 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.534 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.534 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.534 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3050524386, 2023-07-18T03:41:55.534 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629303, 2023-07-18T03:41:55.534 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/helpers.c" 2023-07-18T03:41:55.534 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.534 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.535 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.535 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3052022439, 2023-07-18T03:41:55.535 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260236, 2023-07-18T03:41:55.535 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-15.jpg" 2023-07-18T03:41:55.535 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.535 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.535 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.535 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3058849593, 2023-07-18T03:41:55.536 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259879, 2023-07-18T03:41:55.536 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-26.xml" 2023-07-18T03:41:55.536 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.536 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.536 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.536 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3079096748, 2023-07-18T03:41:55.536 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260028, 2023-07-18T03:41:55.537 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-81.jpg" 2023-07-18T03:41:55.537 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.537 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.537 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.537 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3079569109, 2023-07-18T03:41:55.537 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259893, 2023-07-18T03:41:55.537 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-19.xml" 2023-07-18T03:41:55.537 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.538 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.538 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.538 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3092425294, 2023-07-18T03:41:55.538 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628643, 2023-07-18T03:41:55.538 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-44.xml" 2023-07-18T03:41:55.538 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.538 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.538 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.539 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3110906268, 2023-07-18T03:41:55.539 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260014, 2023-07-18T03:41:55.539 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-90.jpg" 2023-07-18T03:41:55.539 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.539 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.539 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.539 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3170206668, 2023-07-18T03:41:55.539 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260093, 2023-07-18T03:41:55.540 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-49.jpg" 2023-07-18T03:41:55.540 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.540 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.540 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.540 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3172578788, 2023-07-18T03:41:55.540 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628647, 2023-07-18T03:41:55.540 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-40.xml" 2023-07-18T03:41:55.541 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.541 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.541 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.541 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3191848830, 2023-07-18T03:41:55.541 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260051, 2023-07-18T03:41:55.541 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-67.jpg" 2023-07-18T03:41:55.541 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.541 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.542 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.542 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3224997279, 2023-07-18T03:41:55.542 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628649, 2023-07-18T03:41:55.542 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-38.xml" 2023-07-18T03:41:55.542 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.542 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.542 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.543 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3226904580, 2023-07-18T03:41:55.543 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629791, 2023-07-18T03:41:55.543 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/mysnprintf.c" 2023-07-18T03:41:55.543 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.543 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.543 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.543 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3236405310, 2023-07-18T03:41:55.543 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260041, 2023-07-18T03:41:55.544 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-73.jpg" 2023-07-18T03:41:55.544 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.544 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.544 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.544 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3281074905, 2023-07-18T03:41:55.544 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260144, 2023-07-18T03:41:55.544 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-35.jpg" 2023-07-18T03:41:55.544 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.545 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.545 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.545 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3288951734, 2023-07-18T03:41:55.545 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259881, 2023-07-18T03:41:55.545 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-25.xml" 2023-07-18T03:41:55.545 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.545 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.545 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.546 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3322261553, 2023-07-18T03:41:55.546 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628644, 2023-07-18T03:41:55.546 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-43.xml" 2023-07-18T03:41:55.546 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.546 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.546 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.546 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3344077048, 2023-07-18T03:41:55.546 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628652, 2023-07-18T03:41:55.547 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-35.xml" 2023-07-18T03:41:55.547 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.547 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.547 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.547 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3383008765, 2023-07-18T03:41:55.547 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259904, 2023-07-18T03:41:55.547 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-14.xml" 2023-07-18T03:41:55.548 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.548 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.548 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.548 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3415496708, 2023-07-18T03:41:55.548 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259877, 2023-07-18T03:41:55.548 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-27.xml" 2023-07-18T03:41:55.548 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.548 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.549 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.549 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3425229598, 2023-07-18T03:41:55.549 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260168, 2023-07-18T03:41:55.549 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-28.jpg" 2023-07-18T03:41:55.549 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.549 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.549 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.549 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3430738128, 2023-07-18T03:41:55.550 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260002, 2023-07-18T03:41:55.550 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-6.xml" 2023-07-18T03:41:55.550 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.550 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.550 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.550 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3432099179, 2023-07-18T03:41:55.550 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260249, 2023-07-18T03:41:55.550 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-13.jpg" 2023-07-18T03:41:55.551 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.551 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.551 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.551 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3450689939, 2023-07-18T03:41:55.551 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628638, 2023-07-18T03:41:55.551 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-49.xml" 2023-07-18T03:41:55.551 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.551 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.552 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.552 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3458543954, 2023-07-18T03:41:55.552 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628655, 2023-07-18T03:41:55.552 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-32.xml" 2023-07-18T03:41:55.552 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.552 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.552 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.553 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3484532026, 2023-07-18T03:41:55.553 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260033, 2023-07-18T03:41:55.553 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-78.jpg" 2023-07-18T03:41:55.553 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.553 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.553 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.553 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3507069502, 2023-07-18T03:41:55.553 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629299, 2023-07-18T03:41:55.554 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/commenter.c" 2023-07-18T03:41:55.554 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.554 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.554 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.554 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3542685283, 2023-07-18T03:41:55.554 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260156, 2023-07-18T03:41:55.554 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-31.jpg" 2023-07-18T03:41:55.554 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.555 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.555 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.555 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3571408817, 2023-07-18T03:41:55.555 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260043, 2023-07-18T03:41:55.555 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-72.jpg" 2023-07-18T03:41:55.555 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.555 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.555 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.556 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3578930611, 2023-07-18T03:41:55.556 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259885, 2023-07-18T03:41:55.556 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-23.xml" 2023-07-18T03:41:55.556 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.556 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.556 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.556 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3595544447, 2023-07-18T03:41:55.556 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629287, 2023-07-18T03:41:55.557 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/mysnprintf.h" 2023-07-18T03:41:55.557 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.557 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.557 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.557 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3648272976, 2023-07-18T03:41:55.557 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629290, 2023-07-18T03:41:55.557 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/fakesnprintf.c" 2023-07-18T03:41:55.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.558 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.558 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.558 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3699580343, 2023-07-18T03:41:55.558 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628665, 2023-07-18T03:41:55.558 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-22.xml" 2023-07-18T03:41:55.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.558 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.559 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.559 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3723936952, 2023-07-18T03:41:55.559 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260045, 2023-07-18T03:41:55.559 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-71.jpg" 2023-07-18T03:41:55.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.559 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.559 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3765594294, 2023-07-18T03:41:55.560 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259883, 2023-07-18T03:41:55.560 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-24.xml" 2023-07-18T03:41:55.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.560 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.560 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.560 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3776296919, 2023-07-18T03:41:55.560 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260257, 2023-07-18T03:41:55.561 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-11.jpg" 2023-07-18T03:41:55.561 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.561 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.561 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.561 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3835511632, 2023-07-18T03:41:55.561 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629291, 2023-07-18T03:41:55.561 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/fakesnprintf.h" 2023-07-18T03:41:55.561 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.562 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.562 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.562 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3866709076, 2023-07-18T03:41:55.562 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260147, 2023-07-18T03:41:55.562 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-34.jpg" 2023-07-18T03:41:55.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.562 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.562 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.563 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3867557818, 2023-07-18T03:41:55.563 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260012, 2023-07-18T03:41:55.563 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-91.jpg" 2023-07-18T03:41:55.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.563 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.563 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.563 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3872351215, 2023-07-18T03:41:55.563 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628642, 2023-07-18T03:41:55.564 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-45.xml" 2023-07-18T03:41:55.564 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.564 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.564 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.564 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3910860626, 2023-07-18T03:41:55.564 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628677, 2023-07-18T03:41:55.564 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-10.xml" 2023-07-18T03:41:55.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.565 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.565 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.565 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3941329526, 2023-07-18T03:41:55.565 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260129, 2023-07-18T03:41:55.565 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-40.jpg" 2023-07-18T03:41:55.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.565 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.566 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.566 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 3976916763, 2023-07-18T03:41:55.566 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260245, 2023-07-18T03:41:55.566 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-28.xml" 2023-07-18T03:41:55.566 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.566 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.566 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.566 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4051100562, 2023-07-18T03:41:55.567 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628659, 2023-07-18T03:41:55.567 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-28.xml" 2023-07-18T03:41:55.567 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.567 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.567 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.567 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4055980886, 2023-07-18T03:41:55.567 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260071, 2023-07-18T03:41:55.567 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-55.jpg" 2023-07-18T03:41:55.568 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.568 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.568 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.568 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4097620916, 2023-07-18T03:41:55.568 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260138, 2023-07-18T03:41:55.568 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-37.jpg" 2023-07-18T03:41:55.568 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.569 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.569 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.569 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4103660336, 2023-07-18T03:41:55.569 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260031, 2023-07-18T03:41:55.569 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-79.jpg" 2023-07-18T03:41:55.569 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.569 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.569 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.570 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4108782361, 2023-07-18T03:41:55.570 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628637, 2023-07-18T03:41:55.570 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-50.xml" 2023-07-18T03:41:55.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.570 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.570 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.570 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4110640393, 2023-07-18T03:41:55.570 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260061, 2023-07-18T03:41:55.571 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-61.jpg" 2023-07-18T03:41:55.571 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.571 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.571 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.571 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4114207509, 2023-07-18T03:41:55.571 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628678, 2023-07-18T03:41:55.571 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-9.xml" 2023-07-18T03:41:55.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.572 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.572 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.572 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4181523483, 2023-07-18T03:41:55.572 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260036, 2023-07-18T03:41:55.572 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-76.jpg" 2023-07-18T03:41:55.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.572 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.573 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.573 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4198512701, 2023-07-18T03:41:55.573 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260186, 2023-07-18T03:41:55.573 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-26.jpg" 2023-07-18T03:41:55.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.573 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.573 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.573 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4215143213, 2023-07-18T03:41:55.574 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260260, 2023-07-18T03:41:55.574 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-10.jpg" 2023-07-18T03:41:55.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.574 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.574 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4266878672, 2023-07-18T03:41:55.574 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628656, 2023-07-18T03:41:55.574 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-31.xml" 2023-07-18T03:41:55.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.575 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.575 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4272391449, 2023-07-18T03:41:55.575 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511629301, 2023-07-18T03:41:55.575 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/reader.c" 2023-07-18T03:41:55.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.576 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.576 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4282229529, 2023-07-18T03:41:55.576 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 1099511628664, 2023-07-18T03:41:55.576 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-23.xml" 2023-07-18T03:41:55.576 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.576 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.576 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.577 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4282928779, 2023-07-18T03:41:55.577 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023259898, 2023-07-18T03:41:55.577 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-16.xml" 2023-07-18T03:41:55.577 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:41:55.577 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:41:55.577 INFO:teuthology.orchestra.run.smithi040.stdout: "damage_type": "backtrace", 2023-07-18T03:41:55.577 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4289980767, 2023-07-18T03:41:55.577 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 2199023260060, 2023-07-18T03:41:55.578 INFO:teuthology.orchestra.run.smithi040.stdout: "path": "/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-62.jpg" 2023-07-18T03:41:55.578 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:41:55.578 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-07-18T03:41:55.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f918650b700 1 -- 172.21.15.40:0/2307369576 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f91680a11d0 msgr2=0x7f91680a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f918650b700 1 --2- 172.21.15.40:0/2307369576 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f91680a11d0 0x7f91680a3680 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f917400b2e0 tx=0x7f9174005fd0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f918650b700 1 -- 172.21.15.40:0/2307369576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91800ffaf0 msgr2=0x7f91800742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f918650b700 1 --2- 172.21.15.40:0/2307369576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91800ffaf0 0x7f91800742c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f917000d980 tx=0x7f917000dc90 comp rx=0 tx=0).stop 2023-07-18T03:41:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f918650b700 1 -- 172.21.15.40:0/2307369576 shutdown_connections 2023-07-18T03:41:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f918650b700 1 --2- 172.21.15.40:0/2307369576 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f91680a11d0 0x7f91680a3680 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f918650b700 1 --2- 172.21.15.40:0/2307369576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9180101670 0x7f9180072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f918650b700 1 --2- 172.21.15.40:0/2307369576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91800ffaf0 0x7f91800742c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f918650b700 1 --2- 172.21.15.40:0/2307369576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9180100ce0 0x7f9180072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f918650b700 1 -- 172.21.15.40:0/2307369576 >> 172.21.15.40:0/2307369576 conn(0x7f91800fb130 msgr2=0x7f91800fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f918650b700 1 -- 172.21.15.40:0/2307369576 shutdown_connections 2023-07-18T03:41:55.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.423+0000 7f918650b700 1 -- 172.21.15.40:0/2307369576 wait complete. 2023-07-18T03:41:55.581 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-18T03:41:55.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.762+0000 7f0d38f6b700 1 -- 172.21.15.40:0/137872720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34076110 msgr2=0x7f0d34076580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.762+0000 7f0d38f6b700 1 --2- 172.21.15.40:0/137872720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34076110 0x7f0d34076580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009990 tx=0x7f0d28009ca0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.762+0000 7f0d38f6b700 1 -- 172.21.15.40:0/137872720 shutdown_connections 2023-07-18T03:41:55.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.762+0000 7f0d38f6b700 1 --2- 172.21.15.40:0/137872720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d34076ac0 0x7f0d34077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.762+0000 7f0d38f6b700 1 --2- 172.21.15.40:0/137872720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d34074ed0 0x7f0d340752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.762+0000 7f0d38f6b700 1 --2- 172.21.15.40:0/137872720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34076110 0x7f0d34076580 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.762+0000 7f0d38f6b700 1 -- 172.21.15.40:0/137872720 >> 172.21.15.40:0/137872720 conn(0x7f0d340fd3c0 msgr2=0x7f0d340ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.762+0000 7f0d38f6b700 1 -- 172.21.15.40:0/137872720 shutdown_connections 2023-07-18T03:41:55.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.762+0000 7f0d38f6b700 1 -- 172.21.15.40:0/137872720 wait complete. 2023-07-18T03:41:55.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.763+0000 7f0d38f6b700 1 Processor -- start 2023-07-18T03:41:55.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.763+0000 7f0d38f6b700 1 -- start start 2023-07-18T03:41:55.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.763+0000 7f0d38f6b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34074ed0 0x7f0d3419be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.763+0000 7f0d38f6b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d34076110 0x7f0d3419c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.763+0000 7f0d38f6b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d34076ac0 0x7f0d341a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.763+0000 7f0d38f6b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d341a3a10 con 0x7f0d34076ac0 2023-07-18T03:41:55.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.763+0000 7f0d38f6b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d341a3b50 con 0x7f0d34074ed0 2023-07-18T03:41:55.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.763+0000 7f0d38f6b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d341a3ce0 con 0x7f0d34076110 2023-07-18T03:41:55.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.763+0000 7f0d3259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34074ed0 0x7f0d3419be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.763+0000 7f0d31d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d34076110 0x7f0d3419c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.763+0000 7f0d3259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34074ed0 0x7f0d3419be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:45260/0 (socket says 172.21.15.40:45260) 2023-07-18T03:41:55.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.763+0000 7f0d3259c700 1 -- 172.21.15.40:0/466127870 learned_addr learned my addr 172.21.15.40:0/466127870 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.763+0000 7f0d32d9d700 1 --2- 172.21.15.40:0/466127870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d34076ac0 0x7f0d341a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.764+0000 7f0d32d9d700 1 -- 172.21.15.40:0/466127870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d34076110 msgr2=0x7f0d3419c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.764+0000 7f0d32d9d700 1 --2- 172.21.15.40:0/466127870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d34076110 0x7f0d3419c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.764+0000 7f0d32d9d700 1 -- 172.21.15.40:0/466127870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34074ed0 msgr2=0x7f0d3419be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.764+0000 7f0d32d9d700 1 --2- 172.21.15.40:0/466127870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34074ed0 0x7f0d3419be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.764+0000 7f0d32d9d700 1 -- 172.21.15.40:0/466127870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d280095f0 con 0x7f0d34076ac0 2023-07-18T03:41:55.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.764+0000 7f0d31d9b700 1 --2- 172.21.15.40:0/466127870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d34076110 0x7f0d3419c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:41:55.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.764+0000 7f0d32d9d700 1 --2- 172.21.15.40:0/466127870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d34076ac0 0x7f0d341a3440 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400b3f0 tx=0x7f0d2400b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.765+0000 7f0d237fe700 1 -- 172.21.15.40:0/466127870 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24011630 con 0x7f0d34076ac0 2023-07-18T03:41:55.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.765+0000 7f0d237fe700 1 -- 172.21.15.40:0/466127870 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d24011c50 con 0x7f0d34076ac0 2023-07-18T03:41:55.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.765+0000 7f0d237fe700 1 -- 172.21.15.40:0/466127870 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24010c60 con 0x7f0d34076ac0 2023-07-18T03:41:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.765+0000 7f0d38f6b700 1 -- 172.21.15.40:0/466127870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d341a3fc0 con 0x7f0d34076ac0 2023-07-18T03:41:55.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.765+0000 7f0d38f6b700 1 -- 172.21.15.40:0/466127870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d341a4470 con 0x7f0d34076ac0 2023-07-18T03:41:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.769+0000 7f0d237fe700 1 -- 172.21.15.40:0/466127870 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0d24011790 con 0x7f0d34076ac0 2023-07-18T03:41:55.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.770+0000 7f0d237fe700 1 --2- 172.21.15.40:0/466127870 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0d1c0a12d0 0x7f0d1c0a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.770+0000 7f0d237fe700 1 -- 172.21.15.40:0/466127870 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0d240cc050 con 0x7f0d34076ac0 2023-07-18T03:41:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.770+0000 7f0d3259c700 1 --2- 172.21.15.40:0/466127870 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0d1c0a12d0 0x7f0d1c0a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.770+0000 7f0d450f1b80 1 Processor -- start 2023-07-18T03:41:55.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d450f1b80 1 -- start start 2023-07-18T03:41:55.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d450f1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac94224d0 0x562ac94228e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d450f1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac9422e20 0x562ac94236e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d450f1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9423c70 0x562ac94240e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d450f1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac9327cb0 con 0x562ac9423c70 2023-07-18T03:41:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d450f1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac937aa40 con 0x562ac94224d0 2023-07-18T03:41:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d450f1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac9373ed0 con 0x562ac9422e20 2023-07-18T03:41:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d32d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9423c70 0x562ac94240e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d31d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac9422e20 0x562ac94236e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d3259c700 1 --2- 172.21.15.40:0/466127870 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0d1c0a12d0 0x7f0d1c0a3780 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009c10 tx=0x7f0d2c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d32d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9423c70 0x562ac94240e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33712/0 (socket says 172.21.15.40:33712) 2023-07-18T03:41:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d32d9d700 1 -- 172.21.15.40:0/111112333 learned_addr learned my addr 172.21.15.40:0/111112333 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d3259c700 1 --2- 172.21.15.40:0/111112333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac94224d0 0x562ac94228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d32d9d700 1 -- 172.21.15.40:0/111112333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac9422e20 msgr2=0x562ac94236e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d32d9d700 1 --2- 172.21.15.40:0/111112333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac9422e20 0x562ac94236e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d32d9d700 1 -- 172.21.15.40:0/111112333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac94224d0 msgr2=0x562ac94228e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d32d9d700 1 --2- 172.21.15.40:0/111112333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac94224d0 0x562ac94228e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.771+0000 7f0d32d9d700 1 -- 172.21.15.40:0/111112333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ac91a1de0 con 0x562ac9423c70 2023-07-18T03:41:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.772+0000 7f0d32d9d700 1 --2- 172.21.15.40:0/111112333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9423c70 0x562ac94240e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0d24095940 tx=0x7f0d24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3687b3ab3454337 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.772+0000 7f0d20ff9700 1 -- 172.21.15.40:0/111112333 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24099440 con 0x562ac9423c70 2023-07-18T03:41:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.772+0000 7f0d20ff9700 1 -- 172.21.15.40:0/111112333 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d240ae070 con 0x562ac9423c70 2023-07-18T03:41:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.772+0000 7f0d20ff9700 1 -- 172.21.15.40:0/111112333 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24010de0 con 0x562ac9423c70 2023-07-18T03:41:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.773+0000 7f0d450f1b80 1 -- 172.21.15.40:0/111112333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9423c70 msgr2=0x562ac94240e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.773+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/111112333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9423c70 0x562ac94240e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0d24095940 tx=0x7f0d24004750 comp rx=0 tx=0).stop 2023-07-18T03:41:55.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.773+0000 7f0d450f1b80 1 -- 172.21.15.40:0/111112333 shutdown_connections 2023-07-18T03:41:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.773+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/111112333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9423c70 0x562ac94240e0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.773+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/111112333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac94224d0 0x562ac94228e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.773+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/111112333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac9422e20 0x562ac94236e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.773+0000 7f0d450f1b80 1 -- 172.21.15.40:0/111112333 >> 172.21.15.40:0/111112333 conn(0x562ac9417da0 msgr2=0x562ac941d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.773+0000 7f0d450f1b80 1 -- 172.21.15.40:0/111112333 shutdown_connections 2023-07-18T03:41:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.773+0000 7f0d450f1b80 1 -- 172.21.15.40:0/111112333 wait complete. 2023-07-18T03:41:55.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.773+0000 7f0d450f1b80 1 -- start start 2023-07-18T03:41:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.773+0000 7f0d450f1b80 1 Processor -- start 2023-07-18T03:41:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.774+0000 7f0d450f1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac94224d0 0x562ac943d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.774+0000 7f0d450f1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac9422e20 0x562ac943d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.774+0000 7f0d450f1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9423c70 0x562ac94446c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.774+0000 7f0d450f1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac9313ac0 con 0x562ac9423c70 2023-07-18T03:41:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.774+0000 7f0d450f1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac93137c0 con 0x562ac9422e20 2023-07-18T03:41:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.774+0000 7f0d450f1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac91f7920 con 0x562ac94224d0 2023-07-18T03:41:55.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.774+0000 7f0d450f1b80 10 client.0 _ll_register_callbacks cb 0x562ac9311760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:55.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.774+0000 7f0d32d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9423c70 0x562ac94446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.774+0000 7f0d32d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9423c70 0x562ac94446c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33718/0 (socket says 172.21.15.40:33718) 2023-07-18T03:41:55.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.774+0000 7f0d32d9d700 1 -- 172.21.15.40:0/2906485191 learned_addr learned my addr 172.21.15.40:0/2906485191 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.774+0000 7f0d3259c700 1 --2- 172.21.15.40:0/2906485191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac94224d0 0x562ac943d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.774+0000 7f0d31d9b700 1 --2- 172.21.15.40:0/2906485191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac9422e20 0x562ac943d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.774+0000 7f0d32d9d700 1 -- 172.21.15.40:0/2906485191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac94224d0 msgr2=0x562ac943d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.774+0000 7f0d32d9d700 1 --2- 172.21.15.40:0/2906485191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac94224d0 0x562ac943d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.774+0000 7f0d32d9d700 1 -- 172.21.15.40:0/2906485191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac9422e20 msgr2=0x562ac943d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.774+0000 7f0d32d9d700 1 --2- 172.21.15.40:0/2906485191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac9422e20 0x562ac943d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.774+0000 7f0d32d9d700 1 -- 172.21.15.40:0/2906485191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d341a4470 con 0x562ac9423c70 2023-07-18T03:41:55.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.775+0000 7f0d32d9d700 1 --2- 172.21.15.40:0/2906485191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9423c70 0x562ac94446c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0d24095eb0 tx=0x7f0d24011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd21cf57791d802 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.775+0000 7f0d19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:41:55.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.775+0000 7f0d19ffb700 1 -- 172.21.15.40:0/2906485191 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24099440 con 0x562ac9423c70 2023-07-18T03:41:55.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.775+0000 7f0d19ffb700 1 -- 172.21.15.40:0/2906485191 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0d24097850 con 0x562ac9423c70 2023-07-18T03:41:55.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.775+0000 7f0d450f1b80 1 -- 172.21.15.40:0/2906485191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ac92cd4f0 con 0x562ac9423c70 2023-07-18T03:41:55.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.775+0000 7f0d19ffb700 1 -- 172.21.15.40:0/2906485191 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d2409d170 con 0x562ac9423c70 2023-07-18T03:41:55.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.775+0000 7f0d450f1b80 10 client.17052 fetch_fsmap learned FSMap version 72 2023-07-18T03:41:55.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.775+0000 7f0d450f1b80 1 -- 172.21.15.40:0/2906485191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x562ac918ac60 con 0x562ac9423c70 2023-07-18T03:41:55.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.776+0000 7f0d19ffb700 1 -- 172.21.15.40:0/2906485191 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562ac92cd4f0 con 0x562ac9423c70 2023-07-18T03:41:55.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.776+0000 7f0d19ffb700 1 -- 172.21.15.40:0/2906485191 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 72) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f0d240a1400 con 0x562ac9423c70 2023-07-18T03:41:55.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.776+0000 7f0d450f1b80 10 client.17052 fetch_fsmap finished waiting for FSMap version 72 2023-07-18T03:41:55.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.776+0000 7f0d450f1b80 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:41:55.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.777+0000 7f0d450f1b80 20 client.17052 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:55.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.777+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/2906485191 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x562ac9448a20 0x562ac944ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.777+0000 7f0d450f1b80 4 client.17052 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:41:55.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.777+0000 7f0d450f1b80 1 -- 172.21.15.40:0/2906485191 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ac936ceb0 con 0x562ac9448a20 2023-07-18T03:41:55.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.777+0000 7f0d3259c700 1 --2- 172.21.15.40:0/2906485191 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x562ac9448a20 0x562ac944ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.777+0000 7f0d19ffb700 10 client.17052 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:55.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.777+0000 7f0d3259c700 1 --2- 172.21.15.40:0/2906485191 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x562ac9448a20 0x562ac944ae30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5effaf6fccb06e6b server_cookie=4e3629ad52fd2b7a in_seq=0 out_seq=0 2023-07-18T03:41:55.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.778+0000 7f0d19ffb700 1 -- 172.21.15.40:0/2906485191 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562ac936ceb0 con 0x562ac9448a20 2023-07-18T03:41:55.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.778+0000 7f0d19ffb700 10 client.17052 handle_command_reply: tid=0 2023-07-18T03:41:55.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.778+0000 7f0d450f1b80 1 client.17052 shutdown 2023-07-18T03:41:55.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.778+0000 7f0d450f1b80 1 -- 172.21.15.40:0/2906485191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9423c70 msgr2=0x562ac94446c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.778+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/2906485191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9423c70 0x562ac94446c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0d24095eb0 tx=0x7f0d24011e80 comp rx=0 tx=0).stop 2023-07-18T03:41:55.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.778+0000 7f0d450f1b80 1 -- 172.21.15.40:0/2906485191 shutdown_connections 2023-07-18T03:41:55.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.778+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/2906485191 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x562ac9448a20 0x562ac944ae30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.778+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/2906485191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9423c70 0x562ac94446c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.778+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/2906485191 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac94224d0 0x562ac943d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.778+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/2906485191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac9422e20 0x562ac943d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.778+0000 7f0d19ffb700 0 client.17052 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:55.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.778+0000 7f0d450f1b80 1 -- 172.21.15.40:0/2906485191 >> 172.21.15.40:0/2906485191 conn(0x562ac9417da0 msgr2=0x562ac941c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.778+0000 7f0d450f1b80 1 -- 172.21.15.40:0/2906485191 shutdown_connections 2023-07-18T03:41:55.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.778+0000 7f0d450f1b80 1 -- 172.21.15.40:0/2906485191 wait complete. 2023-07-18T03:41:55.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.779+0000 7f0d450f1b80 20 client.17052 trim_cache size 0 max 16384 2023-07-18T03:41:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.796+0000 7f0d450f1b80 1 Processor -- start 2023-07-18T03:41:55.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.796+0000 7f0d450f1b80 1 -- start start 2023-07-18T03:41:55.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.796+0000 7f0d450f1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac94224d0 0x562ac942a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.796+0000 7f0d450f1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac9422e20 0x562ac9444680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.796+0000 7f0d450f1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac9423c70 0x562ac9426b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.796+0000 7f0d450f1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac9423440 con 0x562ac94224d0 2023-07-18T03:41:55.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.796+0000 7f0d450f1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac943c5f0 con 0x562ac9423c70 2023-07-18T03:41:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.797+0000 7f0d450f1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac943ccb0 con 0x562ac9422e20 2023-07-18T03:41:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.797+0000 7f0d32d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac9423c70 0x562ac9426b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.797+0000 7f0d32d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac9423c70 0x562ac9426b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:45282/0 (socket says 172.21.15.40:45282) 2023-07-18T03:41:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.797+0000 7f0d32d9d700 1 -- 172.21.15.40:0/3559200860 learned_addr learned my addr 172.21.15.40:0/3559200860 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.797+0000 7f0d32d9d700 1 -- 172.21.15.40:0/3559200860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac9422e20 msgr2=0x562ac9444680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:41:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.797+0000 7f0d32d9d700 1 --2- 172.21.15.40:0/3559200860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac9422e20 0x562ac9444680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.797+0000 7f0d32d9d700 1 -- 172.21.15.40:0/3559200860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac94224d0 msgr2=0x562ac942a020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-18T03:41:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.797+0000 7f0d32d9d700 1 --2- 172.21.15.40:0/3559200860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac94224d0 0x562ac942a020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.797+0000 7f0d32d9d700 1 -- 172.21.15.40:0/3559200860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ac91c8250 con 0x562ac9423c70 2023-07-18T03:41:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.797+0000 7f0d32d9d700 1 --2- 172.21.15.40:0/3559200860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac9423c70 0x562ac9426b30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0d240a17e0 tx=0x7f0d24004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1af815f0bf1b580 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.798+0000 7f0d217fa700 1 -- 172.21.15.40:0/3559200860 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d240952d0 con 0x562ac9423c70 2023-07-18T03:41:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.798+0000 7f0d217fa700 1 -- 172.21.15.40:0/3559200860 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d240b5070 con 0x562ac9423c70 2023-07-18T03:41:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.798+0000 7f0d217fa700 1 -- 172.21.15.40:0/3559200860 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24010dc0 con 0x562ac9423c70 2023-07-18T03:41:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.799+0000 7f0d450f1b80 1 -- 172.21.15.40:0/3559200860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac9423c70 msgr2=0x562ac9426b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.799+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/3559200860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac9423c70 0x562ac9426b30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0d240a17e0 tx=0x7f0d24004750 comp rx=0 tx=0).stop 2023-07-18T03:41:55.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.800+0000 7f0d450f1b80 1 -- 172.21.15.40:0/3559200860 shutdown_connections 2023-07-18T03:41:55.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.800+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/3559200860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac9423c70 0x562ac9426b30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.800+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/3559200860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac94224d0 0x562ac942a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.800+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/3559200860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac9422e20 0x562ac9444680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.800+0000 7f0d450f1b80 1 -- 172.21.15.40:0/3559200860 >> 172.21.15.40:0/3559200860 conn(0x562ac9417da0 msgr2=0x562ac9444fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.800+0000 7f0d450f1b80 1 -- 172.21.15.40:0/3559200860 shutdown_connections 2023-07-18T03:41:55.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.800+0000 7f0d450f1b80 1 -- 172.21.15.40:0/3559200860 wait complete. 2023-07-18T03:41:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.801+0000 7f0d450f1b80 1 -- start start 2023-07-18T03:41:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.801+0000 7f0d450f1b80 1 Processor -- start 2023-07-18T03:41:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.801+0000 7f0d450f1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac94224d0 0x562ac943e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.801+0000 7f0d450f1b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9422e20 0x562ac943ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.801+0000 7f0d450f1b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac9423c70 0x562ac9443b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.801+0000 7f0d450f1b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac94440c0 con 0x562ac9422e20 2023-07-18T03:41:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.801+0000 7f0d450f1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac9444230 con 0x562ac94224d0 2023-07-18T03:41:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.801+0000 7f0d450f1b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ac94443f0 con 0x562ac9423c70 2023-07-18T03:41:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.801+0000 7f0d450f1b80 10 client.0 _ll_register_callbacks cb 0x7f0d2c00f310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:41:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.801+0000 7f0d3259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac94224d0 0x562ac943e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.801+0000 7f0d3259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac94224d0 0x562ac943e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:45284/0 (socket says 172.21.15.40:45284) 2023-07-18T03:41:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.801+0000 7f0d3259c700 1 -- 172.21.15.40:0/3105579009 learned_addr learned my addr 172.21.15.40:0/3105579009 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:41:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.802+0000 7f0d31d9b700 1 --2- 172.21.15.40:0/3105579009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9422e20 0x562ac943ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.802+0000 7f0d32d9d700 1 --2- 172.21.15.40:0/3105579009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac9423c70 0x562ac9443b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.802+0000 7f0d31d9b700 1 -- 172.21.15.40:0/3105579009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac9423c70 msgr2=0x562ac9443b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.802+0000 7f0d31d9b700 1 --2- 172.21.15.40:0/3105579009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac9423c70 0x562ac9443b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.802+0000 7f0d31d9b700 1 -- 172.21.15.40:0/3105579009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac94224d0 msgr2=0x562ac943e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:41:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.802+0000 7f0d31d9b700 1 --2- 172.21.15.40:0/3105579009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac94224d0 0x562ac943e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.802+0000 7f0d31d9b700 1 -- 172.21.15.40:0/3105579009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ac918ac60 con 0x562ac9422e20 2023-07-18T03:41:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.802+0000 7f0d31d9b700 1 --2- 172.21.15.40:0/3105579009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9422e20 0x562ac943ed40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0d28006010 tx=0x7f0d28005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ddaffaa32462c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:41:55.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.802+0000 7f0d19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:41:55.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.803+0000 7f0d19ffb700 1 -- 172.21.15.40:0/3105579009 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2801d040 con 0x562ac9422e20 2023-07-18T03:41:55.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.803+0000 7f0d19ffb700 1 -- 172.21.15.40:0/3105579009 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f0d2800fd00 con 0x562ac9422e20 2023-07-18T03:41:55.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.803+0000 7f0d450f1b80 1 -- 172.21.15.40:0/3105579009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d240ae070 con 0x562ac9422e20 2023-07-18T03:41:55.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.803+0000 7f0d19ffb700 1 -- 172.21.15.40:0/3105579009 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d28025430 con 0x562ac9422e20 2023-07-18T03:41:55.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.803+0000 7f0d450f1b80 10 client.17055 fetch_fsmap learned FSMap version 72 2023-07-18T03:41:55.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.803+0000 7f0d450f1b80 1 -- 172.21.15.40:0/3105579009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x562ac91a02b0 con 0x562ac9422e20 2023-07-18T03:41:55.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.803+0000 7f0d19ffb700 1 -- 172.21.15.40:0/3105579009 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d240ae070 con 0x562ac9422e20 2023-07-18T03:41:55.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.803+0000 7f0d19ffb700 1 -- 172.21.15.40:0/3105579009 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 72) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f0d280176f0 con 0x562ac9422e20 2023-07-18T03:41:55.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.803+0000 7f0d450f1b80 10 client.17055 fetch_fsmap finished waiting for FSMap version 72 2023-07-18T03:41:55.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.803+0000 7f0d450f1b80 10 client.17055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-18T03:41:55.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.803+0000 7f0d450f1b80 20 client.17055 populate_metadata read hostname 'smithi040' 2023-07-18T03:41:55.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.803+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/3105579009 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x562ac94aea00 0x562ac94b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:41:55.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.803+0000 7f0d450f1b80 4 client.17055 mds_command: new command op to 24527 tid=0[{"prefix": "damage ls"}] 2023-07-18T03:41:55.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.803+0000 7f0d450f1b80 1 -- 172.21.15.40:0/3105579009 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562ac932f050 con 0x562ac94aea00 2023-07-18T03:41:55.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.804+0000 7f0d32d9d700 1 --2- 172.21.15.40:0/3105579009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac9423c70 0x562ac9443b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:41:55.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.804+0000 7f0d3259c700 1 --2- 172.21.15.40:0/3105579009 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x562ac94aea00 0x562ac94b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:41:55.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.804+0000 7f0d19ffb700 10 client.17055 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:55.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.804+0000 7f0d3259c700 1 --2- 172.21.15.40:0/3105579009 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x562ac94aea00 0x562ac94b0e10 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2e265072f27216a server_cookie=7789a145fb50f18 in_seq=0 out_seq=0 2023-07-18T03:41:55.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.805+0000 7f0d19ffb700 1 -- 172.21.15.40:0/3105579009 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562ac932f050 con 0x562ac94aea00 2023-07-18T03:41:55.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.805+0000 7f0d19ffb700 10 client.17055 handle_command_reply: tid=0 2023-07-18T03:41:55.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.805+0000 7f0d450f1b80 1 client.17055 shutdown 2023-07-18T03:41:55.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.806+0000 7f0d450f1b80 1 -- 172.21.15.40:0/3105579009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9422e20 msgr2=0x562ac943ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.806+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/3105579009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9422e20 0x562ac943ed40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0d28006010 tx=0x7f0d28005810 comp rx=0 tx=0).stop 2023-07-18T03:41:55.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.806+0000 7f0d450f1b80 1 -- 172.21.15.40:0/3105579009 shutdown_connections 2023-07-18T03:41:55.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.806+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/3105579009 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x562ac94aea00 0x562ac94b0e10 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.806+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/3105579009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ac9423c70 0x562ac9443b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.806+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/3105579009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ac94224d0 0x562ac943e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.806+0000 7f0d450f1b80 1 --2- 172.21.15.40:0/3105579009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x562ac9422e20 0x562ac943ed40 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.806+0000 7f0d19ffb700 0 client.17055 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:41:55.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.806+0000 7f0d450f1b80 1 -- 172.21.15.40:0/3105579009 >> 172.21.15.40:0/3105579009 conn(0x562ac9417da0 msgr2=0x562ac942a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.807+0000 7f0d450f1b80 1 -- 172.21.15.40:0/3105579009 shutdown_connections 2023-07-18T03:41:55.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.807+0000 7f0d450f1b80 1 -- 172.21.15.40:0/3105579009 wait complete. 2023-07-18T03:41:55.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.807+0000 7f0d450f1b80 20 client.17055 trim_cache size 0 max 16384 2023-07-18T03:41:55.807 INFO:teuthology.orchestra.run.smithi040.stdout:[] 2023-07-18T03:41:55.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.808+0000 7f0d38f6b700 1 -- 172.21.15.40:0/466127870 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0d1c0a12d0 msgr2=0x7f0d1c0a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.808+0000 7f0d38f6b700 1 --2- 172.21.15.40:0/466127870 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0d1c0a12d0 0x7f0d1c0a3780 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009c10 tx=0x7f0d2c00b040 comp rx=0 tx=0).stop 2023-07-18T03:41:55.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.808+0000 7f0d38f6b700 1 -- 172.21.15.40:0/466127870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d34076ac0 msgr2=0x7f0d341a3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:41:55.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.808+0000 7f0d38f6b700 1 --2- 172.21.15.40:0/466127870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d34076ac0 0x7f0d341a3440 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400b3f0 tx=0x7f0d2400b700 comp rx=0 tx=0).stop 2023-07-18T03:41:55.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.811+0000 7f0d38f6b700 1 -- 172.21.15.40:0/466127870 shutdown_connections 2023-07-18T03:41:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.811+0000 7f0d38f6b700 1 --2- 172.21.15.40:0/466127870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d34076ac0 0x7f0d341a3440 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.811+0000 7f0d38f6b700 1 --2- 172.21.15.40:0/466127870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d34074ed0 0x7f0d3419be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.811+0000 7f0d38f6b700 1 --2- 172.21.15.40:0/466127870 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f0d1c0a12d0 0x7f0d1c0a3780 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.811+0000 7f0d38f6b700 1 --2- 172.21.15.40:0/466127870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d34076110 0x7f0d3419c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:41:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.811+0000 7f0d38f6b700 1 -- 172.21.15.40:0/466127870 >> 172.21.15.40:0/466127870 conn(0x7f0d340fd3c0 msgr2=0x7f0d340ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:41:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.811+0000 7f0d38f6b700 1 -- 172.21.15.40:0/466127870 shutdown_connections 2023-07-18T03:41:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:41:55.811+0000 7f0d38f6b700 1 -- 172.21.15.40:0/466127870 wait complete. 2023-07-18T03:41:55.823 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 11741206, 'ino': 1099511628660, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-27.xml'}, {'damage_type': 'backtrace', 'id': 13494289, 'ino': 1099511628681, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-6.xml'}, {'damage_type': 'backtrace', 'id': 24528367, 'ino': 2199023259906, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-13.xml'}, {'damage_type': 'backtrace', 'id': 77849323, 'ino': 1099511628636, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-51.xml'}, {'damage_type': 'backtrace', 'id': 148316563, 'ino': 2199023260220, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-19.jpg'}, {'damage_type': 'backtrace', 'id': 158417010, 'ino': 2199023260133, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-39.jpg'}, {'damage_type': 'backtrace', 'id': 201558950, 'ino': 1099511628662, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-25.xml'}, {'damage_type': 'backtrace', 'id': 237222145, 'ino': 2199023260018, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-87.jpg'}, {'damage_type': 'backtrace', 'id': 308462129, 'ino': 2199023260055, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-65.jpg'}, {'damage_type': 'backtrace', 'id': 333789564, 'ino': 2199023260046, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-70.jpg'}, {'damage_type': 'backtrace', 'id': 339575889, 'ino': 2199023259900, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-15.xml'}, {'damage_type': 'backtrace', 'id': 349062042, 'ino': 2199023260253, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-12.jpg'}, {'damage_type': 'backtrace', 'id': 370265129, 'ino': 2199023260026, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-82.jpg'}, {'damage_type': 'backtrace', 'id': 379202467, 'ino': 1099511628676, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-11.xml'}, {'damage_type': 'backtrace', 'id': 446192629, 'ino': 2199023260295, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-1.jpg'}, {'damage_type': 'backtrace', 'id': 509020740, 'ino': 1099511628651, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-36.xml'}, {'damage_type': 'backtrace', 'id': 542126001, 'ino': 2199023260069, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-56.jpg'}, {'damage_type': 'backtrace', 'id': 543991973, 'ino': 2199023260177, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-27.jpg'}, {'damage_type': 'backtrace', 'id': 564941701, 'ino': 2199023260203, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-23.jpg'}, {'damage_type': 'backtrace', 'id': 576188789, 'ino': 1099511629293, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.c'}, {'damage_type': 'backtrace', 'id': 591146075, 'ino': 2199023260214, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-21.jpg'}, {'damage_type': 'backtrace', 'id': 613330314, 'ino': 2199023260154, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-32.jpg'}, {'damage_type': 'backtrace', 'id': 657408747, 'ino': 1099511628680, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-7.xml'}, {'damage_type': 'backtrace', 'id': 672447047, 'ino': 2199023260197, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-24.jpg'}, {'damage_type': 'backtrace', 'id': 684236234, 'ino': 2199023260009, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-2.xml'}, {'damage_type': 'backtrace', 'id': 705475298, 'ino': 2199023260267, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-8.jpg'}, {'damage_type': 'backtrace', 'id': 738826528, 'ino': 2199023260208, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-22.jpg'}, {'damage_type': 'backtrace', 'id': 826667852, 'ino': 2199023260058, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-63.jpg'}, {'damage_type': 'backtrace', 'id': 902155430, 'ino': 2199023260017, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-88.jpg'}, {'damage_type': 'backtrace', 'id': 921751974, 'ino': 2199023260063, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-60.jpg'}, {'damage_type': 'backtrace', 'id': 931321988, 'ino': 2199023259996, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-9.xml'}, {'damage_type': 'backtrace', 'id': 949854085, 'ino': 2199023260030, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-80.jpg'}, {'damage_type': 'backtrace', 'id': 952637168, 'ino': 2199023259889, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-21.xml'}, {'damage_type': 'backtrace', 'id': 958105991, 'ino': 2199023259992, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-11.xml'}, {'damage_type': 'backtrace', 'id': 980260575, 'ino': 2199023259887, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-22.xml'}, {'damage_type': 'backtrace', 'id': 980561836, 'ino': 2199023260263, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-9.jpg'}, {'damage_type': 'backtrace', 'id': 1001805524, 'ino': 1099511628646, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-41.xml'}, {'damage_type': 'backtrace', 'id': 1001963806, 'ino': 2199023260136, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-38.jpg'}, {'damage_type': 'backtrace', 'id': 1028228729, 'ino': 2199023260067, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-57.jpg'}, {'damage_type': 'backtrace', 'id': 1088297899, 'ino': 2199023260040, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-74.jpg'}, {'damage_type': 'backtrace', 'id': 1098561916, 'ino': 2199023260159, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-30.jpg'}, {'damage_type': 'backtrace', 'id': 1102135148, 'ino': 2199023260077, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-53.jpg'}, {'damage_type': 'backtrace', 'id': 1116213449, 'ino': 1099511629300, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/rewriter.c'}, {'damage_type': 'backtrace', 'id': 1143673065, 'ino': 2199023259894, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-18.xml'}, {'damage_type': 'backtrace', 'id': 1173197411, 'ino': 1099511629297, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/process.c'}, {'damage_type': 'backtrace', 'id': 1198432771, 'ino': 1099511628679, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-8.xml'}, {'damage_type': 'backtrace', 'id': 1222272670, 'ino': 1099511628657, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-30.xml'}, {'damage_type': 'backtrace', 'id': 1240764602, 'ino': 1099511628670, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-17.xml'}, {'damage_type': 'backtrace', 'id': 1256191811, 'ino': 2199023260117, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-43.jpg'}, {'damage_type': 'backtrace', 'id': 1256386456, 'ino': 2199023260240, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-14.jpg'}, {'damage_type': 'backtrace', 'id': 1272685557, 'ino': 2199023260121, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-42.jpg'}, {'damage_type': 'backtrace', 'id': 1301928178, 'ino': 2199023260064, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-59.jpg'}, {'damage_type': 'backtrace', 'id': 1326904727, 'ino': 1099511628672, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-15.xml'}, {'damage_type': 'backtrace', 'id': 1336066704, 'ino': 2199023260232, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-16.jpg'}, {'damage_type': 'backtrace', 'id': 1343480827, 'ino': 2199023260283, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-4.jpg'}, {'damage_type': 'backtrace', 'id': 1344405975, 'ino': 1099511628666, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-21.xml'}, {'damage_type': 'backtrace', 'id': 1354177727, 'ino': 1099511628653, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-34.xml'}, {'damage_type': 'backtrace', 'id': 1365761292, 'ino': 1099511628671, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-16.xml'}, {'damage_type': 'backtrace', 'id': 1372739999, 'ino': 1099511628685, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-2.xml'}, {'damage_type': 'backtrace', 'id': 1385197194, 'ino': 1099511628658, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-29.xml'}, {'damage_type': 'backtrace', 'id': 1397819486, 'ino': 2199023260105, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-46.jpg'}, {'damage_type': 'backtrace', 'id': 1454150126, 'ino': 2199023259891, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-20.xml'}, {'damage_type': 'backtrace', 'id': 1465546320, 'ino': 1099511628633, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-54.xml'}, {'damage_type': 'backtrace', 'id': 1495901065, 'ino': 1099511629302, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/writer.c'}, {'damage_type': 'backtrace', 'id': 1575697613, 'ino': 2199023260224, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-18.jpg'}, {'damage_type': 'backtrace', 'id': 1596502432, 'ino': 1099511628686, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-1.xml'}, {'damage_type': 'backtrace', 'id': 1614663084, 'ino': 2199023259896, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-17.xml'}, {'damage_type': 'backtrace', 'id': 1658922089, 'ino': 1099511633460, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-84.xml'}, {'damage_type': 'backtrace', 'id': 1670484460, 'ino': 2199023259998, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-8.xml'}, {'damage_type': 'backtrace', 'id': 1676047763, 'ino': 1099511629295, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/Makefile.am'}, {'damage_type': 'backtrace', 'id': 1772848277, 'ino': 2199023260141, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-36.jpg'}, {'damage_type': 'backtrace', 'id': 1780453218, 'ino': 1099511628663, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-24.xml'}, {'damage_type': 'backtrace', 'id': 1792486769, 'ino': 1099511628639, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-48.xml'}, {'damage_type': 'backtrace', 'id': 1818668934, 'ino': 2199023260056, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-64.jpg'}, {'damage_type': 'backtrace', 'id': 1824194765, 'ino': 1099511628628, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-7.xml'}, {'damage_type': 'backtrace', 'id': 1853214936, 'ino': 1099511629288, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench.c'}, {'damage_type': 'backtrace', 'id': 1882285566, 'ino': 1099511628684, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-3.xml'}, {'damage_type': 'backtrace', 'id': 1905678506, 'ino': 2199023260097, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-48.jpg'}, {'damage_type': 'backtrace', 'id': 1974198274, 'ino': 1099511628673, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-14.xml'}, {'damage_type': 'backtrace', 'id': 1977614919, 'ino': 2199023260275, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-6.jpg'}, {'damage_type': 'backtrace', 'id': 2006715941, 'ino': 2199023260053, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-66.jpg'}, {'damage_type': 'backtrace', 'id': 2015791459, 'ino': 2199023260287, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-3.jpg'}, {'damage_type': 'backtrace', 'id': 2027910790, 'ino': 2199023260162, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-29.jpg'}, {'damage_type': 'backtrace', 'id': 2032097419, 'ino': 2199023260291, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-2.jpg'}, {'damage_type': 'backtrace', 'id': 2057492094, 'ino': 2199023260085, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-51.jpg'}, {'damage_type': 'backtrace', 'id': 2059310753, 'ino': 2199023260035, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-77.jpg'}, {'damage_type': 'backtrace', 'id': 2060557685, 'ino': 1099511629289, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench.h'}, {'damage_type': 'backtrace', 'id': 2091244724, 'ino': 2199023260279, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-5.jpg'}, {'damage_type': 'backtrace', 'id': 2091429076, 'ino': 1099511628667, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-20.xml'}, {'damage_type': 'backtrace', 'id': 2130087739, 'ino': 1099511629292, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogbench_p.h'}, {'damage_type': 'backtrace', 'id': 2139004694, 'ino': 2199023260081, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-52.jpg'}, {'damage_type': 'backtrace', 'id': 2147051305, 'ino': 1099511628668, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-19.xml'}, {'damage_type': 'backtrace', 'id': 2151109360, 'ino': 2199023260005, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-4.xml'}, {'damage_type': 'backtrace', 'id': 2181379371, 'ino': 1099511628648, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-39.xml'}, {'damage_type': 'backtrace', 'id': 2206602670, 'ino': 2199023260217, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-20.jpg'}, {'damage_type': 'backtrace', 'id': 2206743024, 'ino': 2199023259990, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-12.xml'}, {'damage_type': 'backtrace', 'id': 2274534032, 'ino': 1099511628634, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-53.xml'}, {'damage_type': 'backtrace', 'id': 2291716061, 'ino': 1099511628674, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-13.xml'}, {'damage_type': 'backtrace', 'id': 2300854195, 'ino': 2199023260113, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-44.jpg'}, {'damage_type': 'backtrace', 'id': 2307659677, 'ino': 1099511628650, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-37.xml'}, {'damage_type': 'backtrace', 'id': 2330813503, 'ino': 2199023260004, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-5.xml'}, {'damage_type': 'backtrace', 'id': 2332688267, 'ino': 2199023260021, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-85.jpg'}, {'damage_type': 'backtrace', 'id': 2346182204, 'ino': 2199023260048, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-69.jpg'}, {'damage_type': 'backtrace', 'id': 2356280183, 'ino': 1099511629296, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/Makefile.in'}, {'damage_type': 'backtrace', 'id': 2401013816, 'ino': 2199023260025, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-83.jpg'}, {'damage_type': 'backtrace', 'id': 2429297234, 'ino': 2199023260066, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-58.jpg'}, {'damage_type': 'backtrace', 'id': 2441657868, 'ino': 1099511628669, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-18.xml'}, {'damage_type': 'backtrace', 'id': 2447037269, 'ino': 2199023260089, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-50.jpg'}, {'damage_type': 'backtrace', 'id': 2448157669, 'ino': 1099511628682, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-5.xml'}, {'damage_type': 'backtrace', 'id': 2503423314, 'ino': 1099511628635, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-52.xml'}, {'damage_type': 'backtrace', 'id': 2549982617, 'ino': 1099511628675, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-12.xml'}, {'damage_type': 'backtrace', 'id': 2583187403, 'ino': 2199023260228, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-17.jpg'}, {'damage_type': 'backtrace', 'id': 2617926256, 'ino': 1099511628683, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-4.xml'}, {'damage_type': 'backtrace', 'id': 2660051437, 'ino': 1099511629294, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/bsd-getopt_long.h'}, {'damage_type': 'backtrace', 'id': 2684107693, 'ino': 2199023260015, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-89.jpg'}, {'damage_type': 'backtrace', 'id': 2686660087, 'ino': 1099511628661, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-26.xml'}, {'damage_type': 'backtrace', 'id': 2731161142, 'ino': 2199023260074, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-54.jpg'}, {'damage_type': 'backtrace', 'id': 2741765460, 'ino': 2199023260151, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-33.jpg'}, {'damage_type': 'backtrace', 'id': 2754653532, 'ino': 2199023260109, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-45.jpg'}, {'damage_type': 'backtrace', 'id': 2759990393, 'ino': 2199023259994, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-10.xml'}, {'damage_type': 'backtrace', 'id': 2782387244, 'ino': 2199023260020, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-86.jpg'}, {'damage_type': 'backtrace', 'id': 2794029366, 'ino': 2199023260050, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-68.jpg'}, {'damage_type': 'backtrace', 'id': 2805799160, 'ino': 2199023260038, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-75.jpg'}, {'damage_type': 'backtrace', 'id': 2833906110, 'ino': 1099511633459, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-25.jpg'}, {'damage_type': 'backtrace', 'id': 2859956841, 'ino': 2199023260007, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-3.xml'}, {'damage_type': 'backtrace', 'id': 2869966147, 'ino': 2199023260101, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-47.jpg'}, {'damage_type': 'backtrace', 'id': 2872381266, 'ino': 1099511629298, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/globals.h'}, {'damage_type': 'backtrace', 'id': 2908291091, 'ino': 1099511628641, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-46.xml'}, {'damage_type': 'backtrace', 'id': 2954346201, 'ino': 1099511628632, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-55.xml'}, {'damage_type': 'backtrace', 'id': 2970861627, 'ino': 2199023260023, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-84.jpg'}, {'damage_type': 'backtrace', 'id': 3021537020, 'ino': 1099511628645, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-42.xml'}, {'damage_type': 'backtrace', 'id': 3023974638, 'ino': 2199023260010, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-1.xml'}, {'damage_type': 'backtrace', 'id': 3028491385, 'ino': 2199023260271, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-7.jpg'}, {'damage_type': 'backtrace', 'id': 3035945037, 'ino': 1099511628640, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-47.xml'}, {'damage_type': 'backtrace', 'id': 3049764835, 'ino': 1099511628654, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-33.xml'}, {'damage_type': 'backtrace', 'id': 3050368581, 'ino': 2199023260125, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-41.jpg'}, {'damage_type': 'backtrace', 'id': 3050524386, 'ino': 1099511629303, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/helpers.c'}, {'damage_type': 'backtrace', 'id': 3052022439, 'ino': 2199023260236, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-15.jpg'}, {'damage_type': 'backtrace', 'id': 3058849593, 'ino': 2199023259879, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-26.xml'}, {'damage_type': 'backtrace', 'id': 3079096748, 'ino': 2199023260028, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-81.jpg'}, {'damage_type': 'backtrace', 'id': 3079569109, 'ino': 2199023259893, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-19.xml'}, {'damage_type': 'backtrace', 'id': 3092425294, 'ino': 1099511628643, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-44.xml'}, {'damage_type': 'backtrace', 'id': 3110906268, 'ino': 2199023260014, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-90.jpg'}, {'damage_type': 'backtrace', 'id': 3170206668, 'ino': 2199023260093, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-49.jpg'}, {'damage_type': 'backtrace', 'id': 3172578788, 'ino': 1099511628647, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-40.xml'}, {'damage_type': 'backtrace', 'id': 3191848830, 'ino': 2199023260051, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-67.jpg'}, {'damage_type': 'backtrace', 'id': 3224997279, 'ino': 1099511628649, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-38.xml'}, {'damage_type': 'backtrace', 'id': 3226904580, 'ino': 1099511629791, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/mysnprintf.c'}, {'damage_type': 'backtrace', 'id': 3236405310, 'ino': 2199023260041, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-73.jpg'}, {'damage_type': 'backtrace', 'id': 3281074905, 'ino': 2199023260144, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-35.jpg'}, {'damage_type': 'backtrace', 'id': 3288951734, 'ino': 2199023259881, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-25.xml'}, {'damage_type': 'backtrace', 'id': 3322261553, 'ino': 1099511628644, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-43.xml'}, {'damage_type': 'backtrace', 'id': 3344077048, 'ino': 1099511628652, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-35.xml'}, {'damage_type': 'backtrace', 'id': 3383008765, 'ino': 2199023259904, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-14.xml'}, {'damage_type': 'backtrace', 'id': 3415496708, 'ino': 2199023259877, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-27.xml'}, {'damage_type': 'backtrace', 'id': 3425229598, 'ino': 2199023260168, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-28.jpg'}, {'damage_type': 'backtrace', 'id': 3430738128, 'ino': 2199023260002, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-6.xml'}, {'damage_type': 'backtrace', 'id': 3432099179, 'ino': 2199023260249, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-13.jpg'}, {'damage_type': 'backtrace', 'id': 3450689939, 'ino': 1099511628638, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-49.xml'}, {'damage_type': 'backtrace', 'id': 3458543954, 'ino': 1099511628655, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-32.xml'}, {'damage_type': 'backtrace', 'id': 3484532026, 'ino': 2199023260033, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-78.jpg'}, {'damage_type': 'backtrace', 'id': 3507069502, 'ino': 1099511629299, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/commenter.c'}, {'damage_type': 'backtrace', 'id': 3542685283, 'ino': 2199023260156, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-31.jpg'}, {'damage_type': 'backtrace', 'id': 3571408817, 'ino': 2199023260043, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-72.jpg'}, {'damage_type': 'backtrace', 'id': 3578930611, 'ino': 2199023259885, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-23.xml'}, {'damage_type': 'backtrace', 'id': 3595544447, 'ino': 1099511629287, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/mysnprintf.h'}, {'damage_type': 'backtrace', 'id': 3648272976, 'ino': 1099511629290, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/fakesnprintf.c'}, {'damage_type': 'backtrace', 'id': 3699580343, 'ino': 1099511628665, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-22.xml'}, {'damage_type': 'backtrace', 'id': 3723936952, 'ino': 2199023260045, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-71.jpg'}, {'damage_type': 'backtrace', 'id': 3765594294, 'ino': 2199023259883, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-24.xml'}, {'damage_type': 'backtrace', 'id': 3776296919, 'ino': 2199023260257, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-11.jpg'}, {'damage_type': 'backtrace', 'id': 3835511632, 'ino': 1099511629291, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/fakesnprintf.h'}, {'damage_type': 'backtrace', 'id': 3866709076, 'ino': 2199023260147, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-34.jpg'}, {'damage_type': 'backtrace', 'id': 3867557818, 'ino': 2199023260012, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-91.jpg'}, {'damage_type': 'backtrace', 'id': 3872351215, 'ino': 1099511628642, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-45.xml'}, {'damage_type': 'backtrace', 'id': 3910860626, 'ino': 1099511628677, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-10.xml'}, {'damage_type': 'backtrace', 'id': 3941329526, 'ino': 2199023260129, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-40.jpg'}, {'damage_type': 'backtrace', 'id': 3976916763, 'ino': 2199023260245, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-28.xml'}, {'damage_type': 'backtrace', 'id': 4051100562, 'ino': 1099511628659, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-28.xml'}, {'damage_type': 'backtrace', 'id': 4055980886, 'ino': 2199023260071, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-55.jpg'}, {'damage_type': 'backtrace', 'id': 4097620916, 'ino': 2199023260138, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-37.jpg'}, {'damage_type': 'backtrace', 'id': 4103660336, 'ino': 2199023260031, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-79.jpg'}, {'damage_type': 'backtrace', 'id': 4108782361, 'ino': 1099511628637, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-50.xml'}, {'damage_type': 'backtrace', 'id': 4110640393, 'ino': 2199023260061, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-61.jpg'}, {'damage_type': 'backtrace', 'id': 4114207509, 'ino': 1099511628678, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-9.xml'}, {'damage_type': 'backtrace', 'id': 4181523483, 'ino': 2199023260036, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-76.jpg'}, {'damage_type': 'backtrace', 'id': 4198512701, 'ino': 2199023260186, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-26.jpg'}, {'damage_type': 'backtrace', 'id': 4215143213, 'ino': 2199023260260, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-10.jpg'}, {'damage_type': 'backtrace', 'id': 4266878672, 'ino': 1099511628656, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-31.xml'}, {'damage_type': 'backtrace', 'id': 4272391449, 'ino': 1099511629301, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/reader.c'}, {'damage_type': 'backtrace', 'id': 4282229529, 'ino': 1099511628664, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/comment-23.xml'}, {'damage_type': 'backtrace', 'id': 4282928779, 'ino': 2199023259898, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/article-16.xml'}, {'damage_type': 'backtrace', 'id': 4289980767, 'ino': 2199023260060, 'path': '/volumes/_nogroup/sv_1/9b5b0ec5-07bb-486b-9501-2d6d36216c9c/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-34/picture-62.jpg'}] 2023-07-18T03:41:55.824 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-07-18T03:41:57.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:57 smithi064 ceph-mon[98059]: pgmap v626: 97 pgs: 97 active+clean; 688 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 540 op/s 2023-07-18T03:41:57.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:57 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:41:57.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:57 smithi134 ceph-mon[98216]: pgmap v626: 97 pgs: 97 active+clean; 688 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 540 op/s 2023-07-18T03:41:57.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:57 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:41:57.828 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:57 smithi040 ceph-mon[96439]: pgmap v626: 97 pgs: 97 active+clean; 688 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 540 op/s 2023-07-18T03:41:57.829 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:57 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-18T03:41:59.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:59 smithi064 ceph-mon[98059]: pgmap v627: 97 pgs: 97 active+clean; 668 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 533 op/s 2023-07-18T03:41:59.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:41:59 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:41:59.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:59 smithi134 ceph-mon[98216]: pgmap v627: 97 pgs: 97 active+clean; 668 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 533 op/s 2023-07-18T03:41:59.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:41:59 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:41:59.827 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:59 smithi040 ceph-mon[96439]: pgmap v627: 97 pgs: 97 active+clean; 668 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 533 op/s 2023-07-18T03:41:59.827 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:41:59 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:42:01.234 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:00 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:01.234 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:00 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:01.234 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:00 smithi134 ceph-mon[98216]: pgmap v628: 97 pgs: 97 active+clean; 671 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 463 op/s 2023-07-18T03:42:01.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:00 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:01.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:00 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:01.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:00 smithi040 ceph-mon[96439]: pgmap v628: 97 pgs: 97 active+clean; 671 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 463 op/s 2023-07-18T03:42:01.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:00 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:01.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:00 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:01.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:00 smithi064 ceph-mon[98059]: pgmap v628: 97 pgs: 97 active+clean; 671 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 463 op/s 2023-07-18T03:42:02.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:02.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:02 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:02.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:02.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:02 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:02.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:02.599 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:02 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.050+0000 7f2461b7e700 1 -- 172.21.15.40:0/3066742552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f245c102bd0 msgr2=0x7f245c105050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.050+0000 7f2461b7e700 1 --2- 172.21.15.40:0/3066742552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f245c102bd0 0x7f245c105050 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f244c009990 tx=0x7f244c009ca0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.050+0000 7f2461b7e700 1 -- 172.21.15.40:0/3066742552 shutdown_connections 2023-07-18T03:42:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.050+0000 7f2461b7e700 1 --2- 172.21.15.40:0/3066742552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f245c1056e0 0x7f245c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.050+0000 7f2461b7e700 1 --2- 172.21.15.40:0/3066742552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f245c068640 0x7f245c102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.050+0000 7f2461b7e700 1 --2- 172.21.15.40:0/3066742552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f245c102bd0 0x7f245c105050 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.050+0000 7f2461b7e700 1 -- 172.21.15.40:0/3066742552 >> 172.21.15.40:0/3066742552 conn(0x7f245c0fa000 msgr2=0x7f245c0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.050+0000 7f2461b7e700 1 -- 172.21.15.40:0/3066742552 shutdown_connections 2023-07-18T03:42:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.050+0000 7f2461b7e700 1 -- 172.21.15.40:0/3066742552 wait complete. 2023-07-18T03:42:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.051+0000 7f2461b7e700 1 Processor -- start 2023-07-18T03:42:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.051+0000 7f2461b7e700 1 -- start start 2023-07-18T03:42:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.051+0000 7f2461b7e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f245c068640 0x7f245c195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.051+0000 7f2461b7e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f245c102bd0 0x7f245c195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.051+0000 7f2461b7e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f245c1056e0 0x7f245c196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.051+0000 7f2461b7e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c197390 con 0x7f245c068640 2023-07-18T03:42:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.051+0000 7f2461b7e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c197500 con 0x7f245c1056e0 2023-07-18T03:42:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.051+0000 7f2461b7e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c1976c0 con 0x7f245c102bd0 2023-07-18T03:42:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.051+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f245c068640 0x7f245c195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.051+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f245c068640 0x7f245c195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33744/0 (socket says 172.21.15.40:33744) 2023-07-18T03:42:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.051+0000 7f245b7fe700 1 -- 172.21.15.40:0/2615315622 learned_addr learned my addr 172.21.15.40:0/2615315622 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.051+0000 7f245bfff700 1 --2- 172.21.15.40:0/2615315622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f245c1056e0 0x7f245c196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.051+0000 7f245affd700 1 --2- 172.21.15.40:0/2615315622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f245c102bd0 0x7f245c195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.052+0000 7f245b7fe700 1 -- 172.21.15.40:0/2615315622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f245c102bd0 msgr2=0x7f245c195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.052+0000 7f245b7fe700 1 --2- 172.21.15.40:0/2615315622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f245c102bd0 0x7f245c195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.052+0000 7f245b7fe700 1 -- 172.21.15.40:0/2615315622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f245c1056e0 msgr2=0x7f245c196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.052+0000 7f245b7fe700 1 --2- 172.21.15.40:0/2615315622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f245c1056e0 0x7f245c196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.052+0000 7f245b7fe700 1 -- 172.21.15.40:0/2615315622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c0095f0 con 0x7f245c068640 2023-07-18T03:42:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.052+0000 7f245b7fe700 1 --2- 172.21.15.40:0/2615315622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f245c068640 0x7f245c195970 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f245000e970 tx=0x7f245000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.052+0000 7f2458ff9700 1 -- 172.21.15.40:0/2615315622 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2450009790 con 0x7f245c068640 2023-07-18T03:42:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.052+0000 7f2461b7e700 1 -- 172.21.15.40:0/2615315622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245c1979a0 con 0x7f245c068640 2023-07-18T03:42:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.052+0000 7f2458ff9700 1 -- 172.21.15.40:0/2615315622 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f245000c7e0 con 0x7f245c068640 2023-07-18T03:42:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.052+0000 7f2458ff9700 1 -- 172.21.15.40:0/2615315622 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24500054e0 con 0x7f245c068640 2023-07-18T03:42:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.052+0000 7f2461b7e700 1 -- 172.21.15.40:0/2615315622 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245c19e160 con 0x7f245c068640 2023-07-18T03:42:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.054+0000 7f2458ff9700 1 -- 172.21.15.40:0/2615315622 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f245000c950 con 0x7f245c068640 2023-07-18T03:42:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.055+0000 7f2458ff9700 1 --2- 172.21.15.40:0/2615315622 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f24440a1150 0x7f24440a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.055+0000 7f245affd700 1 --2- 172.21.15.40:0/2615315622 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f24440a1150 0x7f24440a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.055+0000 7f2458ff9700 1 -- 172.21.15.40:0/2615315622 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2450014070 con 0x7f245c068640 2023-07-18T03:42:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.056+0000 7f245affd700 1 --2- 172.21.15.40:0/2615315622 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f24440a1150 0x7f24440a3600 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f244c00b2e0 tx=0x7f244c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.056+0000 7f246dd04b80 1 Processor -- start 2023-07-18T03:42:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.056+0000 7f246dd04b80 1 -- start start 2023-07-18T03:42:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.056+0000 7f246dd04b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c528e86b0 0x557c52a8c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.058 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.056+0000 7f246dd04b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c52a8c840 0x557c52a8d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.056+0000 7f246dd04b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c52a8d6e0 0x557c52a8db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.056+0000 7f246dd04b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c52991c50 con 0x557c528e86b0 2023-07-18T03:42:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.056+0000 7f246dd04b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c529e49e0 con 0x557c52a8c840 2023-07-18T03:42:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.056+0000 7f246dd04b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c529dde70 con 0x557c52a8d6e0 2023-07-18T03:42:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.057+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c528e86b0 0x557c52a8c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.057+0000 7f245affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c52a8c840 0x557c52a8d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.057+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c528e86b0 0x557c52a8c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33752/0 (socket says 172.21.15.40:33752) 2023-07-18T03:42:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.057+0000 7f245b7fe700 1 -- 172.21.15.40:0/1058283433 learned_addr learned my addr 172.21.15.40:0/1058283433 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.057+0000 7f245bfff700 1 --2- 172.21.15.40:0/1058283433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c52a8d6e0 0x557c52a8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.057+0000 7f245b7fe700 1 -- 172.21.15.40:0/1058283433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c52a8d6e0 msgr2=0x557c52a8db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:42:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.057+0000 7f245b7fe700 1 --2- 172.21.15.40:0/1058283433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c52a8d6e0 0x557c52a8db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.057+0000 7f245b7fe700 1 -- 172.21.15.40:0/1058283433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c52a8c840 msgr2=0x557c52a8d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:42:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.057+0000 7f245b7fe700 1 --2- 172.21.15.40:0/1058283433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c52a8c840 0x557c52a8d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.057+0000 7f245b7fe700 1 -- 172.21.15.40:0/1058283433 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c5280bd80 con 0x557c528e86b0 2023-07-18T03:42:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.057+0000 7f245b7fe700 1 --2- 172.21.15.40:0/1058283433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c528e86b0 0x557c52a8c300 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f24500a1e80 tx=0x7f2450099990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca720ad4e0bad7a server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.057+0000 7f2441ffb700 1 -- 172.21.15.40:0/1058283433 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245009df90 con 0x557c528e86b0 2023-07-18T03:42:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.057+0000 7f2441ffb700 1 -- 172.21.15.40:0/1058283433 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24500a23b0 con 0x557c528e86b0 2023-07-18T03:42:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.057+0000 7f2441ffb700 1 -- 172.21.15.40:0/1058283433 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2450099450 con 0x557c528e86b0 2023-07-18T03:42:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.058+0000 7f246dd04b80 1 -- 172.21.15.40:0/1058283433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c528e86b0 msgr2=0x557c52a8c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.058+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1058283433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c528e86b0 0x557c52a8c300 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f24500a1e80 tx=0x7f2450099990 comp rx=0 tx=0).stop 2023-07-18T03:42:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.058+0000 7f246dd04b80 1 -- 172.21.15.40:0/1058283433 shutdown_connections 2023-07-18T03:42:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.058+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1058283433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c52a8d6e0 0x557c52a8db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.058+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1058283433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c528e86b0 0x557c52a8c300 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.058+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1058283433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c52a8c840 0x557c52a8d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.058+0000 7f246dd04b80 1 -- 172.21.15.40:0/1058283433 >> 172.21.15.40:0/1058283433 conn(0x557c52a81be0 msgr2=0x557c52a86cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.058+0000 7f246dd04b80 1 -- 172.21.15.40:0/1058283433 shutdown_connections 2023-07-18T03:42:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.058+0000 7f246dd04b80 1 -- 172.21.15.40:0/1058283433 wait complete. 2023-07-18T03:42:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.058+0000 7f246dd04b80 1 -- start start 2023-07-18T03:42:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f246dd04b80 1 Processor -- start 2023-07-18T03:42:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f246dd04b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c528e86b0 0x557c52a95880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f246dd04b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c52a8c840 0x557c52a95dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f246dd04b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8d6e0 0x557c52a9ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f246dd04b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c5297da60 con 0x557c52a8d6e0 2023-07-18T03:42:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f246dd04b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c5297d760 con 0x557c528e86b0 2023-07-18T03:42:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f246dd04b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c528618c0 con 0x557c52a8c840 2023-07-18T03:42:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f246dd04b80 10 client.0 _ll_register_callbacks cb 0x557c5297b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:42:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c528e86b0 0x557c52a95880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f245bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8d6e0 0x557c52a9ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f245affd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c52a8c840 0x557c52a95dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f245bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8d6e0 0x557c52a9ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33762/0 (socket says 172.21.15.40:33762) 2023-07-18T03:42:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c528e86b0 0x557c52a95880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.40:45316/0 (socket says 172.21.15.40:45316) 2023-07-18T03:42:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f245bfff700 1 -- 172.21.15.40:0/1159072432 learned_addr learned my addr 172.21.15.40:0/1159072432 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f245bfff700 1 -- 172.21.15.40:0/1159072432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c52a8c840 msgr2=0x557c52a95dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:42:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f245bfff700 1 --2- 172.21.15.40:0/1159072432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c52a8c840 0x557c52a95dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f245bfff700 1 -- 172.21.15.40:0/1159072432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c528e86b0 msgr2=0x557c52a95880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:42:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f245bfff700 1 --2- 172.21.15.40:0/1159072432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c528e86b0 0x557c52a95880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f245bfff700 1 -- 172.21.15.40:0/1159072432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245c19e160 con 0x557c52a8d6e0 2023-07-18T03:42:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f245b7fe700 1 --2- 172.21.15.40:0/1159072432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c528e86b0 0x557c52a95880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:42:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f245bfff700 1 --2- 172.21.15.40:0/1159072432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8d6e0 0x557c52a9ce00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2454012e80 tx=0x7f2454014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3841122b8f1c4013 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.059+0000 7f243affd700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-07-18T03:42:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.060+0000 7f243affd700 1 -- 172.21.15.40:0/1159072432 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245400e040 con 0x557c52a8d6e0 2023-07-18T03:42:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.060+0000 7f246dd04b80 1 -- 172.21.15.40:0/1159072432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c52937490 con 0x557c52a8d6e0 2023-07-18T03:42:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.060+0000 7f243affd700 1 -- 172.21.15.40:0/1159072432 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2454014b90 con 0x557c52a8d6e0 2023-07-18T03:42:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.060+0000 7f243affd700 1 -- 172.21.15.40:0/1159072432 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2454014d50 con 0x557c52a8d6e0 2023-07-18T03:42:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.060+0000 7f246dd04b80 10 client.17067 fetch_fsmap learned FSMap version 73 2023-07-18T03:42:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.060+0000 7f246dd04b80 1 -- 172.21.15.40:0/1159072432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x557c527f4c00 con 0x557c52a8d6e0 2023-07-18T03:42:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.060+0000 7f243affd700 1 -- 172.21.15.40:0/1159072432 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c52937490 con 0x557c52a8d6e0 2023-07-18T03:42:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.060+0000 7f243affd700 1 -- 172.21.15.40:0/1159072432 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 73) v1 ==== 7534+0+0 (secure 0 0 0) 0x7f2454027050 con 0x557c52a8d6e0 2023-07-18T03:42:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.060+0000 7f246dd04b80 10 client.17067 fetch_fsmap finished waiting for FSMap version 73 2023-07-18T03:42:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.060+0000 7f246dd04b80 10 client.17067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:42:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.060+0000 7f246dd04b80 20 client.17067 populate_metadata read hostname 'smithi040' 2023-07-18T03:42:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.060+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1159072432 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557c52aa17b0 0x557c52aa3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.060+0000 7f246dd04b80 4 client.17067 mds_command: new command op to 24527 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-18T03:42:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.060+0000 7f246dd04b80 1 -- 172.21.15.40:0/1159072432 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c529d6e50 con 0x557c52aa17b0 2023-07-18T03:42:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.061+0000 7f245b7fe700 1 --2- 172.21.15.40:0/1159072432 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557c52aa17b0 0x557c52aa3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.061+0000 7f245b7fe700 1 --2- 172.21.15.40:0/1159072432 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557c52aa17b0 0x557c52aa3bc0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee8ec8f40655588 server_cookie=f70d0d6b97a7b31f in_seq=0 out_seq=0 2023-07-18T03:42:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.061+0000 7f243affd700 10 client.17067 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:42:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.062+0000 7f243affd700 1 -- 172.21.15.40:0/1159072432 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557c529d6e50 con 0x557c52aa17b0 2023-07-18T03:42:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.062+0000 7f243affd700 10 client.17067 handle_command_reply: tid=0 2023-07-18T03:42:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.062+0000 7f246dd04b80 1 client.17067 shutdown 2023-07-18T03:42:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.062+0000 7f246dd04b80 1 -- 172.21.15.40:0/1159072432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8d6e0 msgr2=0x557c52a9ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.062+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1159072432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8d6e0 0x557c52a9ce00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2454012e80 tx=0x7f2454014600 comp rx=0 tx=0).stop 2023-07-18T03:42:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.062+0000 7f246dd04b80 1 -- 172.21.15.40:0/1159072432 shutdown_connections 2023-07-18T03:42:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.062+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1159072432 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557c52aa17b0 0x557c52aa3bc0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.062+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1159072432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8d6e0 0x557c52a9ce00 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.062+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1159072432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c528e86b0 0x557c52a95880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.062+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1159072432 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c52a8c840 0x557c52a95dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.062+0000 7f243affd700 0 client.17067 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:42:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.062+0000 7f246dd04b80 1 -- 172.21.15.40:0/1159072432 >> 172.21.15.40:0/1159072432 conn(0x557c52a81be0 msgr2=0x557c52a864e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.062+0000 7f246dd04b80 1 -- 172.21.15.40:0/1159072432 shutdown_connections 2023-07-18T03:42:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.062+0000 7f246dd04b80 1 -- 172.21.15.40:0/1159072432 wait complete. 2023-07-18T03:42:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.062+0000 7f246dd04b80 20 client.17067 trim_cache size 0 max 16384 2023-07-18T03:42:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f246dd04b80 1 Processor -- start 2023-07-18T03:42:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f246dd04b80 1 -- start start 2023-07-18T03:42:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f246dd04b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c528e86b0 0x557c52a945a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f246dd04b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c52a8c840 0x557c52a90630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f246dd04b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8d6e0 0x557c52a90b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f246dd04b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c52a94370 con 0x557c52a8d6e0 2023-07-18T03:42:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f246dd04b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c52a877a0 con 0x557c52a8c840 2023-07-18T03:42:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f246dd04b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c52a887a0 con 0x557c528e86b0 2023-07-18T03:42:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f245bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8d6e0 0x557c52a90b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f245affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c52a8c840 0x557c52a90630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f245bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8d6e0 0x557c52a90b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33770/0 (socket says 172.21.15.40:33770) 2023-07-18T03:42:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f245bfff700 1 -- 172.21.15.40:0/1301099265 learned_addr learned my addr 172.21.15.40:0/1301099265 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f245b7fe700 1 --2- 172.21.15.40:0/1301099265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c528e86b0 0x557c52a945a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f245bfff700 1 -- 172.21.15.40:0/1301099265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c528e86b0 msgr2=0x557c52a945a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:42:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f245bfff700 1 --2- 172.21.15.40:0/1301099265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c528e86b0 0x557c52a945a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f245bfff700 1 -- 172.21.15.40:0/1301099265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c52a8c840 msgr2=0x557c52a90630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:42:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f245bfff700 1 --2- 172.21.15.40:0/1301099265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c52a8c840 0x557c52a90630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f245bfff700 1 -- 172.21.15.40:0/1301099265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c528321f0 con 0x557c52a8d6e0 2023-07-18T03:42:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.077+0000 7f245b7fe700 1 --2- 172.21.15.40:0/1301099265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c528e86b0 0x557c52a945a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:42:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.078+0000 7f245bfff700 1 --2- 172.21.15.40:0/1301099265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8d6e0 0x557c52a90b70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f24540061e0 tx=0x7f2454009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3871bd8a04df2516 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:03.075 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.078+0000 7f24427fc700 1 -- 172.21.15.40:0/1301099265 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245400e040 con 0x557c52a8d6e0 2023-07-18T03:42:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.078+0000 7f24427fc700 1 -- 172.21.15.40:0/1301099265 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f245400f040 con 0x557c52a8d6e0 2023-07-18T03:42:03.076 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.078+0000 7f24427fc700 1 -- 172.21.15.40:0/1301099265 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2454007760 con 0x557c52a8d6e0 2023-07-18T03:42:03.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.078+0000 7f246dd04b80 1 -- 172.21.15.40:0/1301099265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8d6e0 msgr2=0x557c52a90b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:03.078 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.078+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1301099265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8d6e0 0x557c52a90b70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f24540061e0 tx=0x7f2454009ba0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.078+0000 7f246dd04b80 1 -- 172.21.15.40:0/1301099265 shutdown_connections 2023-07-18T03:42:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.078+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1301099265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8d6e0 0x557c52a90b70 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.078+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1301099265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c528e86b0 0x557c52a945a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.078+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1301099265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c52a8c840 0x557c52a90630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.078+0000 7f246dd04b80 1 -- 172.21.15.40:0/1301099265 >> 172.21.15.40:0/1301099265 conn(0x557c52a81be0 msgr2=0x557c52aa8520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.078+0000 7f246dd04b80 1 -- 172.21.15.40:0/1301099265 shutdown_connections 2023-07-18T03:42:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.079+0000 7f246dd04b80 1 -- 172.21.15.40:0/1301099265 wait complete. 2023-07-18T03:42:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.079+0000 7f246dd04b80 1 -- start start 2023-07-18T03:42:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.079+0000 7f246dd04b80 1 Processor -- start 2023-07-18T03:42:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.079+0000 7f246dd04b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c528e86b0 0x557c52af3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.079+0000 7f246dd04b80 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8c840 0x557c52af5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.079+0000 7f246dd04b80 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c52a8d6e0 0x557c52af8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.079+0000 7f246dd04b80 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c52af8920 con 0x557c52a8c840 2023-07-18T03:42:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.079+0000 7f246dd04b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c52af8a90 con 0x557c528e86b0 2023-07-18T03:42:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.079+0000 7f246dd04b80 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c52af8c50 con 0x557c52a8d6e0 2023-07-18T03:42:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.079+0000 7f246dd04b80 10 client.0 _ll_register_callbacks cb 0x7f24540127b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-18T03:42:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.079+0000 7f245affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8c840 0x557c52af5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.079+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c528e86b0 0x557c52af3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.079+0000 7f245affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8c840 0x557c52af5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33774/0 (socket says 172.21.15.40:33774) 2023-07-18T03:42:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.079+0000 7f245affd700 1 -- 172.21.15.40:0/1190377150 learned_addr learned my addr 172.21.15.40:0/1190377150 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.079+0000 7f245bfff700 1 --2- 172.21.15.40:0/1190377150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c52a8d6e0 0x557c52af8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.080+0000 7f245b7fe700 1 -- 172.21.15.40:0/1190377150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c52a8d6e0 msgr2=0x557c52af8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:42:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.080+0000 7f245b7fe700 1 --2- 172.21.15.40:0/1190377150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c52a8d6e0 0x557c52af8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.080+0000 7f245b7fe700 1 -- 172.21.15.40:0/1190377150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8c840 msgr2=0x557c52af5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-18T03:42:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.080+0000 7f245b7fe700 1 --2- 172.21.15.40:0/1190377150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8c840 0x557c52af5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.080+0000 7f245b7fe700 1 -- 172.21.15.40:0/1190377150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c527f4c00 con 0x557c528e86b0 2023-07-18T03:42:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.080+0000 7f245b7fe700 1 --2- 172.21.15.40:0/1190377150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c528e86b0 0x557c52af3650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f24500ca6a0 tx=0x7f2450099e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d3a080664e36e13 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.080+0000 7f243affd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2023-07-18T03:42:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.080+0000 7f243affd700 1 -- 172.21.15.40:0/1190377150 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24500962a0 con 0x557c528e86b0 2023-07-18T03:42:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.080+0000 7f243affd700 1 -- 172.21.15.40:0/1190377150 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f24500ca030 con 0x557c528e86b0 2023-07-18T03:42:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.080+0000 7f246dd04b80 1 -- 172.21.15.40:0/1190377150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24500a23b0 con 0x557c528e86b0 2023-07-18T03:42:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.080+0000 7f243affd700 1 -- 172.21.15.40:0/1190377150 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f245009df90 con 0x557c528e86b0 2023-07-18T03:42:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.080+0000 7f246dd04b80 10 client.26708 fetch_fsmap learned FSMap version 73 2023-07-18T03:42:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.080+0000 7f246dd04b80 1 -- 172.21.15.40:0/1190377150 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x557c5280a250 con 0x557c528e86b0 2023-07-18T03:42:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.080+0000 7f243affd700 1 -- 172.21.15.40:0/1190377150 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24500a23b0 con 0x557c528e86b0 2023-07-18T03:42:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.081+0000 7f243affd700 1 -- 172.21.15.40:0/1190377150 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 73) v1 ==== 7534+0+0 (secure 0 0 0) 0x7f24500a1f20 con 0x557c528e86b0 2023-07-18T03:42:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.081+0000 7f246dd04b80 10 client.26708 fetch_fsmap finished waiting for FSMap version 73 2023-07-18T03:42:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.081+0000 7f246dd04b80 10 client.26708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-18T03:42:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.081+0000 7f246dd04b80 20 client.26708 populate_metadata read hostname 'smithi040' 2023-07-18T03:42:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.081+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1190377150 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557c52afa430 0x557c52afc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.081+0000 7f246dd04b80 4 client.26708 mds_command: new command op to 24527 tid=0[{"prefix": "flush journal"}] 2023-07-18T03:42:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.081+0000 7f246dd04b80 1 -- 172.21.15.40:0/1190377150 --> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557c52998ff0 con 0x557c52afa430 2023-07-18T03:42:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.081+0000 7f245affd700 1 --2- 172.21.15.40:0/1190377150 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557c52afa430 0x557c52afc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.081+0000 7f243affd700 10 client.26708 ms_handle_connect on v2:172.21.15.64:6838/3007039043 2023-07-18T03:42:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.081+0000 7f245affd700 1 --2- 172.21.15.40:0/1190377150 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557c52afa430 0x557c52afc840 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6f3daf98b8eb7a8 server_cookie=e37c7eae76d3a56c in_seq=0 out_seq=0 2023-07-18T03:42:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.088+0000 7f243affd700 1 -- 172.21.15.40:0/1190377150 <== mds.0 v2:172.21.15.64:6838/3007039043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557c52998ff0 con 0x557c52afa430 2023-07-18T03:42:03.086 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.088+0000 7f243affd700 10 client.26708 handle_command_reply: tid=0 2023-07-18T03:42:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.088+0000 7f246dd04b80 1 client.26708 shutdown 2023-07-18T03:42:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f246dd04b80 1 -- 172.21.15.40:0/1190377150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c528e86b0 msgr2=0x557c52af3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1190377150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c528e86b0 0x557c52af3650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f24500ca6a0 tx=0x7f2450099e00 comp rx=0 tx=0).stop 2023-07-18T03:42:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f246dd04b80 1 -- 172.21.15.40:0/1190377150 shutdown_connections 2023-07-18T03:42:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1190377150 >> [v2:172.21.15.64:6838/3007039043,v1:172.21.15.64:6839/3007039043] conn(0x557c52afa430 0x557c52afc840 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1190377150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557c52a8d6e0 0x557c52af8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f243affd700 0 client.26708 ms_handle_reset on v2:172.21.15.64:6838/3007039043 2023-07-18T03:42:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1190377150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c528e86b0 0x557c52af3650 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f246dd04b80 1 --2- 172.21.15.40:0/1190377150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x557c52a8c840 0x557c52af5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f246dd04b80 1 -- 172.21.15.40:0/1190377150 >> 172.21.15.40:0/1190377150 conn(0x557c52a81be0 msgr2=0x557c52a945a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f246dd04b80 1 -- 172.21.15.40:0/1190377150 shutdown_connections 2023-07-18T03:42:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f246dd04b80 1 -- 172.21.15.40:0/1190377150 wait complete. 2023-07-18T03:42:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f246dd04b80 20 client.26708 trim_cache size 0 max 16384 2023-07-18T03:42:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:{ 2023-07-18T03:42:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "message": "", 2023-07-18T03:42:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout: "return_code": 0 2023-07-18T03:42:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stdout:} 2023-07-18T03:42:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f2461b7e700 1 -- 172.21.15.40:0/2615315622 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f24440a1150 msgr2=0x7f24440a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f2461b7e700 1 --2- 172.21.15.40:0/2615315622 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f24440a1150 0x7f24440a3600 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f244c00b2e0 tx=0x7f244c005fb0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f2461b7e700 1 -- 172.21.15.40:0/2615315622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f245c068640 msgr2=0x7f245c195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f2461b7e700 1 --2- 172.21.15.40:0/2615315622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f245c068640 0x7f245c195970 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f245000e970 tx=0x7f245000ec80 comp rx=0 tx=0).stop 2023-07-18T03:42:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f2461b7e700 1 -- 172.21.15.40:0/2615315622 shutdown_connections 2023-07-18T03:42:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f2461b7e700 1 --2- 172.21.15.40:0/2615315622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f245c1056e0 0x7f245c196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f2461b7e700 1 --2- 172.21.15.40:0/2615315622 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f245c068640 0x7f245c195970 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f2461b7e700 1 --2- 172.21.15.40:0/2615315622 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f24440a1150 0x7f24440a3600 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f2461b7e700 1 --2- 172.21.15.40:0/2615315622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f245c102bd0 0x7f245c195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.089+0000 7f2461b7e700 1 -- 172.21.15.40:0/2615315622 >> 172.21.15.40:0/2615315622 conn(0x7f245c0fa000 msgr2=0x7f245c100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.090+0000 7f2461b7e700 1 -- 172.21.15.40:0/2615315622 shutdown_connections 2023-07-18T03:42:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi040.front.sepia.ceph.com.smithi040.stderr:2023-07-18T03:42:03.090+0000 7f2461b7e700 1 -- 172.21.15.40:0/2615315622 wait complete. 2023-07-18T03:42:03.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:03 smithi134 ceph-mon[98216]: pgmap v629: 97 pgs: 97 active+clean; 639 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 552 op/s 2023-07-18T03:42:03.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:03 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:03.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:03 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:03.576 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:03 smithi040 ceph-mon[96439]: pgmap v629: 97 pgs: 97 active+clean; 639 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 552 op/s 2023-07-18T03:42:03.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:03.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:03 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:03.597 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:03 smithi064 ceph-mon[98059]: pgmap v629: 97 pgs: 97 active+clean; 639 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 552 op/s 2023-07-18T03:42:03.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:03 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:03.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:03 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:04.563 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:04 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:42:04.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:04 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:42:04.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:04 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:04.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:04 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:42:04.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:04 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:42:04.578 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:04 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:04.597 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:04 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-18T03:42:04.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:04 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-18T03:42:04.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:04 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' 2023-07-18T03:42:04.669 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2023-07-18T03:42:04.670 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-07-18T03:42:05.059 DEBUG:teuthology.parallel:result is None 2023-07-18T03:42:05.059 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-07-18T03:42:05.098 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-07-18T03:42:05.098 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-07-18T03:42:05.109 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi040.front.sepia.ceph.com 2023-07-18T03:42:05.146 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-07-18T03:42:05.158 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-07-18T03:42:05.158 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-07-18T03:42:05.159 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-07-18T03:42:05.169 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:42:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.320+0000 7f86d7d3e700 1 -- 172.21.15.40:0/195747967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d00fe9d0 msgr2=0x7f86d00fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.320+0000 7f86d7d3e700 1 --2- 172.21.15.40:0/195747967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d00fe9d0 0x7f86d00fede0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009990 tx=0x7f86c4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.320+0000 7f86d7d3e700 1 -- 172.21.15.40:0/195747967 shutdown_connections 2023-07-18T03:42:05.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.320+0000 7f86d7d3e700 1 --2- 172.21.15.40:0/195747967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86d0100550 0x7f86d0106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.320+0000 7f86d7d3e700 1 --2- 172.21.15.40:0/195747967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d00fe9d0 0x7f86d00fede0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.320+0000 7f86d7d3e700 1 --2- 172.21.15.40:0/195747967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86d00ffbc0 0x7f86d0100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.320+0000 7f86d7d3e700 1 -- 172.21.15.40:0/195747967 >> 172.21.15.40:0/195747967 conn(0x7f86d00f9fd0 msgr2=0x7f86d00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.320+0000 7f86d7d3e700 1 -- 172.21.15.40:0/195747967 shutdown_connections 2023-07-18T03:42:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.320+0000 7f86d7d3e700 1 -- 172.21.15.40:0/195747967 wait complete. 2023-07-18T03:42:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d7d3e700 1 Processor -- start 2023-07-18T03:42:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d7d3e700 1 -- start start 2023-07-18T03:42:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d7d3e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86d00fe9d0 0x7f86d0194630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d7d3e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d00ffbc0 0x7f86d0194b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d7d3e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86d0100550 0x7f86d019bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d7d3e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d019c1b0 con 0x7f86d00ffbc0 2023-07-18T03:42:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d7d3e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d019c2f0 con 0x7f86d00fe9d0 2023-07-18T03:42:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d7d3e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d019c480 con 0x7f86d0100550 2023-07-18T03:42:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d52d9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d00ffbc0 0x7f86d0194b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d52d9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d00ffbc0 0x7f86d0194b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33786/0 (socket says 172.21.15.40:33786) 2023-07-18T03:42:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d52d9700 1 -- 172.21.15.40:0/2352747054 learned_addr learned my addr 172.21.15.40:0/2352747054 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d5ada700 1 --2- 172.21.15.40:0/2352747054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86d00fe9d0 0x7f86d0194630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d62db700 1 --2- 172.21.15.40:0/2352747054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86d0100550 0x7f86d019bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:05.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d52d9700 1 -- 172.21.15.40:0/2352747054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86d0100550 msgr2=0x7f86d019bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:05.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d52d9700 1 --2- 172.21.15.40:0/2352747054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86d0100550 0x7f86d019bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d52d9700 1 -- 172.21.15.40:0/2352747054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86d00fe9d0 msgr2=0x7f86d0194630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:05.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d52d9700 1 --2- 172.21.15.40:0/2352747054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86d00fe9d0 0x7f86d0194630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d52d9700 1 -- 172.21.15.40:0/2352747054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c40095f0 con 0x7f86d00ffbc0 2023-07-18T03:42:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.321+0000 7f86d62db700 1 --2- 172.21.15.40:0/2352747054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86d0100550 0x7f86d019bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:42:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.322+0000 7f86d52d9700 1 --2- 172.21.15.40:0/2352747054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d00ffbc0 0x7f86d0194b70 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00c9b0 tx=0x7f86cc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.322+0000 7f86c2ffd700 1 -- 172.21.15.40:0/2352747054 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc007b00 con 0x7f86d00ffbc0 2023-07-18T03:42:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.322+0000 7f86c2ffd700 1 -- 172.21.15.40:0/2352747054 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86cc007c60 con 0x7f86d00ffbc0 2023-07-18T03:42:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.322+0000 7f86d7d3e700 1 -- 172.21.15.40:0/2352747054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d019c760 con 0x7f86d00ffbc0 2023-07-18T03:42:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.322+0000 7f86c2ffd700 1 -- 172.21.15.40:0/2352747054 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc018660 con 0x7f86d00ffbc0 2023-07-18T03:42:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.322+0000 7f86d7d3e700 1 -- 172.21.15.40:0/2352747054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d019cc70 con 0x7f86d00ffbc0 2023-07-18T03:42:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.323+0000 7f86d7d3e700 1 -- 172.21.15.40:0/2352747054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d0105810 con 0x7f86d00ffbc0 2023-07-18T03:42:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.324+0000 7f86c2ffd700 1 -- 172.21.15.40:0/2352747054 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f86cc00f450 con 0x7f86d00ffbc0 2023-07-18T03:42:05.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.324+0000 7f86c2ffd700 1 --2- 172.21.15.40:0/2352747054 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f86bc0a11a0 0x7f86bc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:05.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.325+0000 7f86c2ffd700 1 -- 172.21.15.40:0/2352747054 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f86cc0ccb50 con 0x7f86d00ffbc0 2023-07-18T03:42:05.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.325+0000 7f86d5ada700 1 --2- 172.21.15.40:0/2352747054 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f86bc0a11a0 0x7f86bc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:05.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.325+0000 7f86d5ada700 1 --2- 172.21.15.40:0/2352747054 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f86bc0a11a0 0x7f86bc0a3650 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f86c400b5c0 tx=0x7f86c4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:05.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.328+0000 7f86c2ffd700 1 -- 172.21.15.40:0/2352747054 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f86cc096310 con 0x7f86d00ffbc0 2023-07-18T03:42:05.326 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:05 smithi040 ceph-mon[96439]: pgmap v630: 97 pgs: 97 active+clean; 634 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 444 op/s 2023-07-18T03:42:05.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.537+0000 7f86d7d3e700 1 -- 172.21.15.40:0/2352747054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86d002ca20 con 0x7f86d00ffbc0 2023-07-18T03:42:05.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.539+0000 7f86c2ffd700 1 -- 172.21.15.40:0/2352747054 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22599 (secure 0 0 0) 0x7f86cc099930 con 0x7f86d00ffbc0 2023-07-18T03:42:05.534 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:42:05.535 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":84,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:35:46.881845+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:42:05.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.541+0000 7f86d7d3e700 1 -- 172.21.15.40:0/2352747054 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f86bc0a11a0 msgr2=0x7f86bc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:05.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.541+0000 7f86d7d3e700 1 --2- 172.21.15.40:0/2352747054 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f86bc0a11a0 0x7f86bc0a3650 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f86c400b5c0 tx=0x7f86c4005960 comp rx=0 tx=0).stop 2023-07-18T03:42:05.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.541+0000 7f86d7d3e700 1 -- 172.21.15.40:0/2352747054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d00ffbc0 msgr2=0x7f86d0194b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:05.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.541+0000 7f86d7d3e700 1 --2- 172.21.15.40:0/2352747054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d00ffbc0 0x7f86d0194b70 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00c9b0 tx=0x7f86cc00ccc0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.541+0000 7f86d7d3e700 1 -- 172.21.15.40:0/2352747054 shutdown_connections 2023-07-18T03:42:05.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.541+0000 7f86d7d3e700 1 --2- 172.21.15.40:0/2352747054 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f86bc0a11a0 0x7f86bc0a3650 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.541+0000 7f86d7d3e700 1 --2- 172.21.15.40:0/2352747054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86d0100550 0x7f86d019bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.541+0000 7f86d7d3e700 1 --2- 172.21.15.40:0/2352747054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86d00fe9d0 0x7f86d0194630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.541+0000 7f86d7d3e700 1 --2- 172.21.15.40:0/2352747054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86d00ffbc0 0x7f86d0194b70 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.542+0000 7f86d7d3e700 1 -- 172.21.15.40:0/2352747054 >> 172.21.15.40:0/2352747054 conn(0x7f86d00f9fd0 msgr2=0x7f86d00fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:05.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.542+0000 7f86d7d3e700 1 -- 172.21.15.40:0/2352747054 shutdown_connections 2023-07-18T03:42:05.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.542+0000 7f86d7d3e700 1 -- 172.21.15.40:0/2352747054 wait complete. 2023-07-18T03:42:05.549 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:25:38.267262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:29:24.727790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:29:26.078507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:42:05.550 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:42:05.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:05 smithi134 ceph-mon[98216]: pgmap v630: 97 pgs: 97 active+clean; 634 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 444 op/s 2023-07-18T03:42:05.597 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:05 smithi064 ceph-mon[98059]: pgmap v630: 97 pgs: 97 active+clean; 634 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 444 op/s 2023-07-18T03:42:05.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.685+0000 7f1aa4eb3700 1 -- 172.21.15.40:0/979824917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa00ffb90 msgr2=0x7f1aa00fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:05.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.685+0000 7f1aa4eb3700 1 --2- 172.21.15.40:0/979824917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa00ffb90 0x7f1aa00fffe0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009990 tx=0x7f1a88009ca0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.685+0000 7f1aa4eb3700 1 -- 172.21.15.40:0/979824917 shutdown_connections 2023-07-18T03:42:05.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.685+0000 7f1aa4eb3700 1 --2- 172.21.15.40:0/979824917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa0100520 0x7f1aa0106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.685+0000 7f1aa4eb3700 1 --2- 172.21.15.40:0/979824917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aa00fe9a0 0x7f1aa00fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.685+0000 7f1aa4eb3700 1 --2- 172.21.15.40:0/979824917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa00ffb90 0x7f1aa00fffe0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.685+0000 7f1aa4eb3700 1 -- 172.21.15.40:0/979824917 >> 172.21.15.40:0/979824917 conn(0x7f1aa00f9fa0 msgr2=0x7f1aa00fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:05.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.685+0000 7f1aa4eb3700 1 -- 172.21.15.40:0/979824917 shutdown_connections 2023-07-18T03:42:05.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.685+0000 7f1aa4eb3700 1 -- 172.21.15.40:0/979824917 wait complete. 2023-07-18T03:42:05.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.686+0000 7f1aa4eb3700 1 Processor -- start 2023-07-18T03:42:05.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.686+0000 7f1aa4eb3700 1 -- start start 2023-07-18T03:42:05.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.686+0000 7f1aa4eb3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa00fe9a0 0x7f1aa00681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:05.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.686+0000 7f1aa4eb3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aa00ffb90 0x7f1aa019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:05.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.686+0000 7f1aa4eb3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa0100520 0x7f1aa019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:05.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.686+0000 7f1aa4eb3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa019d340 con 0x7f1aa00fe9a0 2023-07-18T03:42:05.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.686+0000 7f1aa4eb3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa019d4b0 con 0x7f1aa0100520 2023-07-18T03:42:05.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.686+0000 7f1aa4eb3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa019d670 con 0x7f1aa00ffb90 2023-07-18T03:42:05.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.687+0000 7f1a9e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa00fe9a0 0x7f1aa00681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:05.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.687+0000 7f1a9e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa00fe9a0 0x7f1aa00681e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33798/0 (socket says 172.21.15.40:33798) 2023-07-18T03:42:05.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.687+0000 7f1a9e59c700 1 -- 172.21.15.40:0/749259990 learned_addr learned my addr 172.21.15.40:0/749259990 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:05.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.687+0000 7f1a9ed9d700 1 --2- 172.21.15.40:0/749259990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa0100520 0x7f1aa019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:05.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.687+0000 7f1a9dd9b700 1 --2- 172.21.15.40:0/749259990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aa00ffb90 0x7f1aa019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:05.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.687+0000 7f1a9e59c700 1 -- 172.21.15.40:0/749259990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aa00ffb90 msgr2=0x7f1aa019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:05.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.687+0000 7f1a9e59c700 1 --2- 172.21.15.40:0/749259990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aa00ffb90 0x7f1aa019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.687+0000 7f1a9e59c700 1 -- 172.21.15.40:0/749259990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa0100520 msgr2=0x7f1aa019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:05.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.687+0000 7f1a9e59c700 1 --2- 172.21.15.40:0/749259990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa0100520 0x7f1aa019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.687+0000 7f1a9e59c700 1 -- 172.21.15.40:0/749259990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a880095f0 con 0x7f1aa00fe9a0 2023-07-18T03:42:05.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.687+0000 7f1a9dd9b700 1 --2- 172.21.15.40:0/749259990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aa00ffb90 0x7f1aa019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:42:05.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.687+0000 7f1a9e59c700 1 --2- 172.21.15.40:0/749259990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa00fe9a0 0x7f1aa00681e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000e970 tx=0x7f1a9000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:05.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.687+0000 7f1a977fe700 1 -- 172.21.15.40:0/749259990 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a90009790 con 0x7f1aa00fe9a0 2023-07-18T03:42:05.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.687+0000 7f1aa4eb3700 1 -- 172.21.15.40:0/749259990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa019d950 con 0x7f1aa00fe9a0 2023-07-18T03:42:05.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.687+0000 7f1a977fe700 1 -- 172.21.15.40:0/749259990 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a9000eec0 con 0x7f1aa00fe9a0 2023-07-18T03:42:05.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.687+0000 7f1a977fe700 1 -- 172.21.15.40:0/749259990 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a90003cf0 con 0x7f1aa00fe9a0 2023-07-18T03:42:05.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.687+0000 7f1aa4eb3700 1 -- 172.21.15.40:0/749259990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa019de60 con 0x7f1aa00fe9a0 2023-07-18T03:42:05.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.688+0000 7f1aa4eb3700 1 -- 172.21.15.40:0/749259990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa0105f80 con 0x7f1aa00fe9a0 2023-07-18T03:42:05.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.690+0000 7f1a977fe700 1 -- 172.21.15.40:0/749259990 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1a9000ccd0 con 0x7f1aa00fe9a0 2023-07-18T03:42:05.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.690+0000 7f1a977fe700 1 --2- 172.21.15.40:0/749259990 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1a8c0a1150 0x7f1a8c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:05.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.690+0000 7f1a977fe700 1 -- 172.21.15.40:0/749259990 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f1a90014070 con 0x7f1aa00fe9a0 2023-07-18T03:42:05.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.690+0000 7f1a9dd9b700 1 --2- 172.21.15.40:0/749259990 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1a8c0a1150 0x7f1a8c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:05.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.691+0000 7f1a9dd9b700 1 --2- 172.21.15.40:0/749259990 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1a8c0a1150 0x7f1a8c0a3600 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800b2e0 tx=0x7f1a88005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:05.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.693+0000 7f1a977fe700 1 -- 172.21.15.40:0/749259990 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1a900963f0 con 0x7f1aa00fe9a0 2023-07-18T03:42:05.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.892+0000 7f1aa4eb3700 1 -- 172.21.15.40:0/749259990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1aa002caa0 con 0x7f1aa00fe9a0 2023-07-18T03:42:05.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.893+0000 7f1a977fe700 1 -- 172.21.15.40:0/749259990 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1a90099a10 con 0x7f1aa00fe9a0 2023-07-18T03:42:05.888 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-07-18T03:42:05.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.895+0000 7f1aa4eb3700 1 -- 172.21.15.40:0/749259990 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1a8c0a1150 msgr2=0x7f1a8c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:05.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.895+0000 7f1aa4eb3700 1 --2- 172.21.15.40:0/749259990 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1a8c0a1150 0x7f1a8c0a3600 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800b2e0 tx=0x7f1a88005fd0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.895+0000 7f1aa4eb3700 1 -- 172.21.15.40:0/749259990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa00fe9a0 msgr2=0x7f1aa00681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:05.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.895+0000 7f1aa4eb3700 1 --2- 172.21.15.40:0/749259990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa00fe9a0 0x7f1aa00681e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000e970 tx=0x7f1a9000ec80 comp rx=0 tx=0).stop 2023-07-18T03:42:05.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.895+0000 7f1aa4eb3700 1 -- 172.21.15.40:0/749259990 shutdown_connections 2023-07-18T03:42:05.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.895+0000 7f1aa4eb3700 1 --2- 172.21.15.40:0/749259990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1aa0100520 0x7f1aa019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.895+0000 7f1aa4eb3700 1 --2- 172.21.15.40:0/749259990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aa00fe9a0 0x7f1aa00681e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.895+0000 7f1aa4eb3700 1 --2- 172.21.15.40:0/749259990 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f1a8c0a1150 0x7f1a8c0a3600 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.895+0000 7f1aa4eb3700 1 --2- 172.21.15.40:0/749259990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1aa00ffb90 0x7f1aa019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:05.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.895+0000 7f1aa4eb3700 1 -- 172.21.15.40:0/749259990 >> 172.21.15.40:0/749259990 conn(0x7f1aa00f9fa0 msgr2=0x7f1aa01030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:05.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.895+0000 7f1aa4eb3700 1 -- 172.21.15.40:0/749259990 shutdown_connections 2023-07-18T03:42:05.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:05.895+0000 7f1aa4eb3700 1 -- 172.21.15.40:0/749259990 wait complete. 2023-07-18T03:42:05.902 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:42:06.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.036+0000 7f6c2c1be700 1 -- 172.21.15.40:0/424433983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c24100560 msgr2=0x7f6c24106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:06.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.036+0000 7f6c2c1be700 1 --2- 172.21.15.40:0/424433983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c24100560 0x7f6c24106e20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00b0a0 tx=0x7f6c1c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.036+0000 7f6c2c1be700 1 -- 172.21.15.40:0/424433983 shutdown_connections 2023-07-18T03:42:06.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.036+0000 7f6c2c1be700 1 --2- 172.21.15.40:0/424433983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c24100560 0x7f6c24106e20 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.036+0000 7f6c2c1be700 1 --2- 172.21.15.40:0/424433983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240fe9e0 0x7f6c240fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.036+0000 7f6c2c1be700 1 --2- 172.21.15.40:0/424433983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c240ffbd0 0x7f6c24100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.036+0000 7f6c2c1be700 1 -- 172.21.15.40:0/424433983 >> 172.21.15.40:0/424433983 conn(0x7f6c240fa000 msgr2=0x7f6c240fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:06.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.036+0000 7f6c2c1be700 1 -- 172.21.15.40:0/424433983 shutdown_connections 2023-07-18T03:42:06.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.036+0000 7f6c2c1be700 1 -- 172.21.15.40:0/424433983 wait complete. 2023-07-18T03:42:06.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.037+0000 7f6c2c1be700 1 Processor -- start 2023-07-18T03:42:06.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.037+0000 7f6c2c1be700 1 -- start start 2023-07-18T03:42:06.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.037+0000 7f6c2c1be700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c240fe9e0 0x7f6c24194800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:06.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.037+0000 7f6c2c1be700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240ffbd0 0x7f6c24194d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:06.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.037+0000 7f6c2c1be700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c24100560 0x7f6c2419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:06.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.037+0000 7f6c2c1be700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2419c380 con 0x7f6c24100560 2023-07-18T03:42:06.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.037+0000 7f6c2c1be700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2419c4c0 con 0x7f6c240ffbd0 2023-07-18T03:42:06.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.037+0000 7f6c2c1be700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2419c650 con 0x7f6c240fe9e0 2023-07-18T03:42:06.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.037+0000 7f6c2a75b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c24100560 0x7f6c2419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:06.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.037+0000 7f6c2a75b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c24100560 0x7f6c2419bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33816/0 (socket says 172.21.15.40:33816) 2023-07-18T03:42:06.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.037+0000 7f6c2a75b700 1 -- 172.21.15.40:0/1400947614 learned_addr learned my addr 172.21.15.40:0/1400947614 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:06.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.037+0000 7f6c29759700 1 --2- 172.21.15.40:0/1400947614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240ffbd0 0x7f6c24194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:06.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.037+0000 7f6c29f5a700 1 --2- 172.21.15.40:0/1400947614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c240fe9e0 0x7f6c24194800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:06.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.038+0000 7f6c2a75b700 1 -- 172.21.15.40:0/1400947614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c240fe9e0 msgr2=0x7f6c24194800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:06.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.038+0000 7f6c2a75b700 1 --2- 172.21.15.40:0/1400947614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c240fe9e0 0x7f6c24194800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.038+0000 7f6c2a75b700 1 -- 172.21.15.40:0/1400947614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240ffbd0 msgr2=0x7f6c24194d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:06.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.038+0000 7f6c2a75b700 1 --2- 172.21.15.40:0/1400947614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240ffbd0 0x7f6c24194d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.038+0000 7f6c2a75b700 1 -- 172.21.15.40:0/1400947614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c009d00 con 0x7f6c24100560 2023-07-18T03:42:06.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.038+0000 7f6c2a75b700 1 --2- 172.21.15.40:0/1400947614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c24100560 0x7f6c2419bdb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00b780 tx=0x7f6c1c00bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:06.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.038+0000 7f6c16ffd700 1 -- 172.21.15.40:0/1400947614 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c00e040 con 0x7f6c24100560 2023-07-18T03:42:06.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.038+0000 7f6c16ffd700 1 -- 172.21.15.40:0/1400947614 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c1c004c60 con 0x7f6c24100560 2023-07-18T03:42:06.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.038+0000 7f6c2c1be700 1 -- 172.21.15.40:0/1400947614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2419c8d0 con 0x7f6c24100560 2023-07-18T03:42:06.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.038+0000 7f6c16ffd700 1 -- 172.21.15.40:0/1400947614 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c01b6b0 con 0x7f6c24100560 2023-07-18T03:42:06.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.038+0000 7f6c2c1be700 1 -- 172.21.15.40:0/1400947614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2419cd50 con 0x7f6c24100560 2023-07-18T03:42:06.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.039+0000 7f6c2c1be700 1 -- 172.21.15.40:0/1400947614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c24105870 con 0x7f6c24100560 2023-07-18T03:42:06.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.040+0000 7f6c16ffd700 1 -- 172.21.15.40:0/1400947614 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6c1c01d480 con 0x7f6c24100560 2023-07-18T03:42:06.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.041+0000 7f6c16ffd700 1 --2- 172.21.15.40:0/1400947614 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6c100a1150 0x7f6c100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:06.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.041+0000 7f6c16ffd700 1 -- 172.21.15.40:0/1400947614 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f6c1c0ce6a0 con 0x7f6c24100560 2023-07-18T03:42:06.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.041+0000 7f6c29f5a700 1 --2- 172.21.15.40:0/1400947614 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6c100a1150 0x7f6c100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:06.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.041+0000 7f6c29f5a700 1 --2- 172.21.15.40:0/1400947614 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6c100a1150 0x7f6c100a3600 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6c240ffa30 tx=0x7f6c18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:06.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.044+0000 7f6c16ffd700 1 -- 172.21.15.40:0/1400947614 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6c1c097e60 con 0x7f6c24100560 2023-07-18T03:42:06.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.246+0000 7f6c2c1be700 1 -- 172.21.15.40:0/1400947614 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c2402ca60 con 0x7f6c24100560 2023-07-18T03:42:06.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.246+0000 7f6c16ffd700 1 -- 172.21.15.40:0/1400947614 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c1c09b480 con 0x7f6c24100560 2023-07-18T03:42:06.242 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-07-18T03:42:06.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.249+0000 7f6c2c1be700 1 -- 172.21.15.40:0/1400947614 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6c100a1150 msgr2=0x7f6c100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:06.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.249+0000 7f6c2c1be700 1 --2- 172.21.15.40:0/1400947614 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6c100a1150 0x7f6c100a3600 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6c240ffa30 tx=0x7f6c18009260 comp rx=0 tx=0).stop 2023-07-18T03:42:06.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.249+0000 7f6c2c1be700 1 -- 172.21.15.40:0/1400947614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c24100560 msgr2=0x7f6c2419bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:06.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.249+0000 7f6c2c1be700 1 --2- 172.21.15.40:0/1400947614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c24100560 0x7f6c2419bdb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00b780 tx=0x7f6c1c00bd00 comp rx=0 tx=0).stop 2023-07-18T03:42:06.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.249+0000 7f6c2c1be700 1 -- 172.21.15.40:0/1400947614 shutdown_connections 2023-07-18T03:42:06.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.249+0000 7f6c2c1be700 1 --2- 172.21.15.40:0/1400947614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c24100560 0x7f6c2419bdb0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.249+0000 7f6c2c1be700 1 --2- 172.21.15.40:0/1400947614 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6c100a1150 0x7f6c100a3600 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.249+0000 7f6c2c1be700 1 --2- 172.21.15.40:0/1400947614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c240fe9e0 0x7f6c24194800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.249+0000 7f6c2c1be700 1 --2- 172.21.15.40:0/1400947614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c240ffbd0 0x7f6c24194d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.249+0000 7f6c2c1be700 1 -- 172.21.15.40:0/1400947614 >> 172.21.15.40:0/1400947614 conn(0x7f6c240fa000 msgr2=0x7f6c240fc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:06.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.249+0000 7f6c2c1be700 1 -- 172.21.15.40:0/1400947614 shutdown_connections 2023-07-18T03:42:06.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.249+0000 7f6c2c1be700 1 -- 172.21.15.40:0/1400947614 wait complete. 2023-07-18T03:42:06.257 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:42:06.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.393+0000 7f35d0c1c700 1 -- 172.21.15.40:0/3939169862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35cc0ffbd0 msgr2=0x7f35cc100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:06.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.393+0000 7f35d0c1c700 1 --2- 172.21.15.40:0/3939169862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35cc0ffbd0 0x7f35cc100020 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009990 tx=0x7f35c0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.393+0000 7f35d0c1c700 1 -- 172.21.15.40:0/3939169862 shutdown_connections 2023-07-18T03:42:06.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.393+0000 7f35d0c1c700 1 --2- 172.21.15.40:0/3939169862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35cc100560 0x7f35cc106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.393+0000 7f35d0c1c700 1 --2- 172.21.15.40:0/3939169862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35cc0fe9e0 0x7f35cc0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.393+0000 7f35d0c1c700 1 --2- 172.21.15.40:0/3939169862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35cc0ffbd0 0x7f35cc100020 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.393+0000 7f35d0c1c700 1 -- 172.21.15.40:0/3939169862 >> 172.21.15.40:0/3939169862 conn(0x7f35cc0fa000 msgr2=0x7f35cc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.394+0000 7f35d0c1c700 1 -- 172.21.15.40:0/3939169862 shutdown_connections 2023-07-18T03:42:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.394+0000 7f35d0c1c700 1 -- 172.21.15.40:0/3939169862 wait complete. 2023-07-18T03:42:06.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.394+0000 7f35d0c1c700 1 Processor -- start 2023-07-18T03:42:06.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35d0c1c700 1 -- start start 2023-07-18T03:42:06.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35d0c1c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35cc0fe9e0 0x7f35cc194750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:06.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35d0c1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35cc0ffbd0 0x7f35cc194c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:06.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35d0c1c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35cc100560 0x7f35cc19bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:06.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35d0c1c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc19c2d0 con 0x7f35cc100560 2023-07-18T03:42:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35d0c1c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc19c410 con 0x7f35cc0ffbd0 2023-07-18T03:42:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35d0c1c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc19c5a0 con 0x7f35cc0fe9e0 2023-07-18T03:42:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35cad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35cc100560 0x7f35cc19bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35ca59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35cc0fe9e0 0x7f35cc194750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35cad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35cc100560 0x7f35cc19bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33830/0 (socket says 172.21.15.40:33830) 2023-07-18T03:42:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35ca59c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35cc0fe9e0 0x7f35cc194750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:51086/0 (socket says 172.21.15.40:51086) 2023-07-18T03:42:06.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35c9d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35cc0ffbd0 0x7f35cc194c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35cad9d700 1 -- 172.21.15.40:0/3581550512 learned_addr learned my addr 172.21.15.40:0/3581550512 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35cad9d700 1 -- 172.21.15.40:0/3581550512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35cc0fe9e0 msgr2=0x7f35cc194750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35cad9d700 1 --2- 172.21.15.40:0/3581550512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35cc0fe9e0 0x7f35cc194750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35cad9d700 1 -- 172.21.15.40:0/3581550512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35cc0ffbd0 msgr2=0x7f35cc194c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35cad9d700 1 --2- 172.21.15.40:0/3581550512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35cc0ffbd0 0x7f35cc194c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35cad9d700 1 -- 172.21.15.40:0/3581550512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c00095f0 con 0x7f35cc100560 2023-07-18T03:42:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.395+0000 7f35cad9d700 1 --2- 172.21.15.40:0/3581550512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35cc100560 0x7f35cc19bd00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f35c400ada0 tx=0x7f35c400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:06.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.396+0000 7f35bb7fe700 1 -- 172.21.15.40:0/3581550512 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c400f070 con 0x7f35cc100560 2023-07-18T03:42:06.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.396+0000 7f35bb7fe700 1 -- 172.21.15.40:0/3581550512 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35c4007b30 con 0x7f35cc100560 2023-07-18T03:42:06.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.396+0000 7f35bb7fe700 1 -- 172.21.15.40:0/3581550512 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4005280 con 0x7f35cc100560 2023-07-18T03:42:06.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.396+0000 7f35d0c1c700 1 -- 172.21.15.40:0/3581550512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35cc0676d0 con 0x7f35cc100560 2023-07-18T03:42:06.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.396+0000 7f35d0c1c700 1 -- 172.21.15.40:0/3581550512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35cc067b90 con 0x7f35cc100560 2023-07-18T03:42:06.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.397+0000 7f35d0c1c700 1 -- 172.21.15.40:0/3581550512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35cc067290 con 0x7f35cc100560 2023-07-18T03:42:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.398+0000 7f35bb7fe700 1 -- 172.21.15.40:0/3581550512 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f35c400a090 con 0x7f35cc100560 2023-07-18T03:42:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.399+0000 7f35bb7fe700 1 --2- 172.21.15.40:0/3581550512 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35b40a11d0 0x7f35b40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.399+0000 7f35bb7fe700 1 -- 172.21.15.40:0/3581550512 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f35c40cc3c0 con 0x7f35cc100560 2023-07-18T03:42:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.399+0000 7f35ca59c700 1 --2- 172.21.15.40:0/3581550512 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35b40a11d0 0x7f35b40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.399+0000 7f35ca59c700 1 --2- 172.21.15.40:0/3581550512 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35b40a11d0 0x7f35b40a3680 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0ffa30 tx=0x7f35bc010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.402+0000 7f35bb7fe700 1 -- 172.21.15.40:0/3581550512 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f35c4099610 con 0x7f35cc100560 2023-07-18T03:42:06.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:06 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2352747054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:42:06.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:06 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/749259990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:42:06.564 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:06 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1400947614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:42:06.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:06 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2352747054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:42:06.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:06 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/749259990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:42:06.577 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:06 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1400947614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:42:06.597 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:06 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2352747054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:42:06.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:06 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/749259990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:42:06.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:06 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1400947614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:42:06.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.604+0000 7f35d0c1c700 1 -- 172.21.15.40:0/3581550512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f35cc105e20 con 0x7f35cc100560 2023-07-18T03:42:06.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.604+0000 7f35bb7fe700 1 -- 172.21.15.40:0/3581550512 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f35c40991a0 con 0x7f35cc100560 2023-07-18T03:42:06.599 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-07-18T03:42:06.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.606+0000 7f35d0c1c700 1 -- 172.21.15.40:0/3581550512 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35b40a11d0 msgr2=0x7f35b40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:06.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.606+0000 7f35d0c1c700 1 --2- 172.21.15.40:0/3581550512 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35b40a11d0 0x7f35b40a3680 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0ffa30 tx=0x7f35bc010040 comp rx=0 tx=0).stop 2023-07-18T03:42:06.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.606+0000 7f35d0c1c700 1 -- 172.21.15.40:0/3581550512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35cc100560 msgr2=0x7f35cc19bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:06.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.606+0000 7f35d0c1c700 1 --2- 172.21.15.40:0/3581550512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35cc100560 0x7f35cc19bd00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f35c400ada0 tx=0x7f35c400c330 comp rx=0 tx=0).stop 2023-07-18T03:42:06.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.606+0000 7f35d0c1c700 1 -- 172.21.15.40:0/3581550512 shutdown_connections 2023-07-18T03:42:06.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.606+0000 7f35d0c1c700 1 --2- 172.21.15.40:0/3581550512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35cc100560 0x7f35cc19bd00 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.606+0000 7f35d0c1c700 1 --2- 172.21.15.40:0/3581550512 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f35b40a11d0 0x7f35b40a3680 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.606+0000 7f35d0c1c700 1 --2- 172.21.15.40:0/3581550512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35cc0fe9e0 0x7f35cc194750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.606+0000 7f35d0c1c700 1 --2- 172.21.15.40:0/3581550512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35cc0ffbd0 0x7f35cc194c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.606+0000 7f35d0c1c700 1 -- 172.21.15.40:0/3581550512 >> 172.21.15.40:0/3581550512 conn(0x7f35cc0fa000 msgr2=0x7f35cc103100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:06.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.606+0000 7f35d0c1c700 1 -- 172.21.15.40:0/3581550512 shutdown_connections 2023-07-18T03:42:06.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.607+0000 7f35d0c1c700 1 -- 172.21.15.40:0/3581550512 wait complete. 2023-07-18T03:42:06.614 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-07-18T03:42:06.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.747+0000 7fb79f906700 1 -- 172.21.15.40:0/1006793664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb798100550 msgr2=0x7fb798106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:06.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.747+0000 7fb79f906700 1 --2- 172.21.15.40:0/1006793664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb798100550 0x7fb798106e10 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb79000b0a0 tx=0x7fb79000b3b0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.747+0000 7fb79f906700 1 -- 172.21.15.40:0/1006793664 shutdown_connections 2023-07-18T03:42:06.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.747+0000 7fb79f906700 1 --2- 172.21.15.40:0/1006793664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb798100550 0x7fb798106e10 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.747+0000 7fb79f906700 1 --2- 172.21.15.40:0/1006793664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7980fe9d0 0x7fb7980fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.747+0000 7fb79f906700 1 --2- 172.21.15.40:0/1006793664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7980ffbc0 0x7fb798100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.747+0000 7fb79f906700 1 -- 172.21.15.40:0/1006793664 >> 172.21.15.40:0/1006793664 conn(0x7fb7980f9fd0 msgr2=0x7fb7980fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:06.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.747+0000 7fb79f906700 1 -- 172.21.15.40:0/1006793664 shutdown_connections 2023-07-18T03:42:06.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.747+0000 7fb79f906700 1 -- 172.21.15.40:0/1006793664 wait complete. 2023-07-18T03:42:06.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79f906700 1 Processor -- start 2023-07-18T03:42:06.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79f906700 1 -- start start 2023-07-18T03:42:06.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79f906700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7980fe9d0 0x7fb798102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:06.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79f906700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7980ffbc0 0x7fb7981011c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:06.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79f906700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb798100550 0x7fb798101700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:06.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79f906700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb798101ea0 con 0x7fb7980fe9d0 2023-07-18T03:42:06.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79f906700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb798102010 con 0x7fb798100550 2023-07-18T03:42:06.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79f906700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79819f910 con 0x7fb7980ffbc0 2023-07-18T03:42:06.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79d6a2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7980fe9d0 0x7fb798102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:06.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79d6a2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7980fe9d0 0x7fb798102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33840/0 (socket says 172.21.15.40:33840) 2023-07-18T03:42:06.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79d6a2700 1 -- 172.21.15.40:0/1248548659 learned_addr learned my addr 172.21.15.40:0/1248548659 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:06.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79cea1700 1 --2- 172.21.15.40:0/1248548659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7980ffbc0 0x7fb7981011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:06.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79dea3700 1 --2- 172.21.15.40:0/1248548659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb798100550 0x7fb798101700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:06.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79d6a2700 1 -- 172.21.15.40:0/1248548659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7980ffbc0 msgr2=0x7fb7981011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:06.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79d6a2700 1 --2- 172.21.15.40:0/1248548659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7980ffbc0 0x7fb7981011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79d6a2700 1 -- 172.21.15.40:0/1248548659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb798100550 msgr2=0x7fb798101700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:06.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79d6a2700 1 --2- 172.21.15.40:0/1248548659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb798100550 0x7fb798101700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.748+0000 7fb79d6a2700 1 -- 172.21.15.40:0/1248548659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb790009d00 con 0x7fb7980fe9d0 2023-07-18T03:42:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.749+0000 7fb79d6a2700 1 --2- 172.21.15.40:0/1248548659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7980fe9d0 0x7fb798102b80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00d980 tx=0x7fb78c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.749+0000 7fb78a7fc700 1 -- 172.21.15.40:0/1248548659 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c00f890 con 0x7fb7980fe9d0 2023-07-18T03:42:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.749+0000 7fb78a7fc700 1 -- 172.21.15.40:0/1248548659 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb78c010460 con 0x7fb7980fe9d0 2023-07-18T03:42:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.749+0000 7fb78a7fc700 1 -- 172.21.15.40:0/1248548659 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c0195c0 con 0x7fb7980fe9d0 2023-07-18T03:42:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.749+0000 7fb79f906700 1 -- 172.21.15.40:0/1248548659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79819fbf0 con 0x7fb7980fe9d0 2023-07-18T03:42:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.749+0000 7fb79f906700 1 -- 172.21.15.40:0/1248548659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7981a0100 con 0x7fb7980fe9d0 2023-07-18T03:42:06.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.750+0000 7fb79f906700 1 -- 172.21.15.40:0/1248548659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7981069c0 con 0x7fb7980fe9d0 2023-07-18T03:42:06.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.751+0000 7fb78a7fc700 1 -- 172.21.15.40:0/1248548659 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb78c0105d0 con 0x7fb7980fe9d0 2023-07-18T03:42:06.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.752+0000 7fb78a7fc700 1 --2- 172.21.15.40:0/1248548659 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb7840a1220 0x7fb7840a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:06.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.752+0000 7fb79cea1700 1 --2- 172.21.15.40:0/1248548659 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb7840a1220 0x7fb7840a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:06.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.752+0000 7fb78a7fc700 1 -- 172.21.15.40:0/1248548659 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fb78c0cc3f0 con 0x7fb7980fe9d0 2023-07-18T03:42:06.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.752+0000 7fb79cea1700 1 --2- 172.21.15.40:0/1248548659 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb7840a1220 0x7fb7840a36d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb794009be0 tx=0x7fb794009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:06.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:06.756+0000 7fb78a7fc700 1 -- 172.21.15.40:0/1248548659 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb78c095b20 con 0x7fb7980fe9d0 2023-07-18T03:42:07.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:07.012+0000 7fb79f906700 1 -- 172.21.15.40:0/1248548659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fb79802c850 con 0x7fb7980fe9d0 2023-07-18T03:42:07.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:07.020+0000 7fb78a7fc700 1 -- 172.21.15.40:0/1248548659 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733243 (secure 0 0 0) 0x7fb78c099140 con 0x7fb7980fe9d0 2023-07-18T03:42:07.022 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:42:07.023 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:42:07.023 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 18, 2023-07-18T03:42:07.023 INFO:teuthology.orchestra.run.smithi040.stdout: "active_gid": 14152, 2023-07-18T03:42:07.023 INFO:teuthology.orchestra.run.smithi040.stdout: "active_name": "x", 2023-07-18T03:42:07.023 INFO:teuthology.orchestra.run.smithi040.stdout: "active_addrs": { 2023-07-18T03:42:07.023 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-07-18T03:42:07.023 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.024 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-07-18T03:42:07.024 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-07-18T03:42:07.024 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3612011194 2023-07-18T03:42:07.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.024 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.024 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-07-18T03:42:07.024 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-07-18T03:42:07.025 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3612011194 2023-07-18T03:42:07.025 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.025 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-07-18T03:42:07.025 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.025 INFO:teuthology.orchestra.run.smithi040.stdout: "active_addr": "172.21.15.40:6801/3612011194", 2023-07-18T03:42:07.025 INFO:teuthology.orchestra.run.smithi040.stdout: "active_change": "2023-07-18T03:23:31.734747+0000", 2023-07-18T03:42:07.025 INFO:teuthology.orchestra.run.smithi040.stdout: "active_mgr_features": 4540138322906710015, 2023-07-18T03:42:07.025 INFO:teuthology.orchestra.run.smithi040.stdout: "available": true, 2023-07-18T03:42:07.026 INFO:teuthology.orchestra.run.smithi040.stdout: "standbys": [ 2023-07-18T03:42:07.026 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.026 INFO:teuthology.orchestra.run.smithi040.stdout: "gid": 24106, 2023-07-18T03:42:07.026 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "z", 2023-07-18T03:42:07.026 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_features": 4540138322906710015, 2023-07-18T03:42:07.026 INFO:teuthology.orchestra.run.smithi040.stdout: "available_modules": [ 2023-07-18T03:42:07.026 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.026 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "alerts", 2023-07-18T03:42:07.027 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.027 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.027 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.027 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2023-07-18T03:42:07.027 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2023-07-18T03:42:07.027 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.027 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.028 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.028 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2023-07-18T03:42:07.028 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.028 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.028 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.028 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How frequently to reexamine health status", 2023-07-18T03:42:07.028 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.028 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.029 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.029 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.029 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.029 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.029 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.029 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.029 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.030 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.030 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.030 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.030 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.030 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.030 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.030 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.030 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.031 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.031 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.031 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.031 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.031 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.031 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.031 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.031 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.032 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.032 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.032 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.032 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.032 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.032 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.032 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.033 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.033 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.033 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.033 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.033 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.033 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.033 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.033 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.034 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.034 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.034 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.034 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.034 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.034 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.034 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.034 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.035 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.035 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.035 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.035 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.035 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.035 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.035 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.035 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.036 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.036 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.036 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.036 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.036 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.036 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.036 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.037 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.037 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.037 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.037 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.038 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.038 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.039 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.039 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.039 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.039 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.040 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.040 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_destination": { 2023-07-18T03:42:07.040 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_destination", 2023-07-18T03:42:07.040 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.041 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.041 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.041 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.042 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.042 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.042 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.042 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Email address to send alerts to", 2023-07-18T03:42:07.043 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.043 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.043 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.043 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.044 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_from_name": { 2023-07-18T03:42:07.044 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_from_name", 2023-07-18T03:42:07.044 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.045 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.045 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.045 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "Ceph", 2023-07-18T03:42:07.045 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.046 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.046 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.046 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Email From: name", 2023-07-18T03:42:07.046 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.047 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.047 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.047 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.047 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_host": { 2023-07-18T03:42:07.048 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_host", 2023-07-18T03:42:07.048 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.048 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.049 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.049 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.049 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.049 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.050 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.050 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP server", 2023-07-18T03:42:07.050 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.050 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.051 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.051 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.051 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_password": { 2023-07-18T03:42:07.052 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_password", 2023-07-18T03:42:07.052 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.052 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.052 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.053 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.053 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.053 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.053 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.054 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Password to authenticate with", 2023-07-18T03:42:07.054 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.054 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.055 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.055 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.055 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_port": { 2023-07-18T03:42:07.055 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_port", 2023-07-18T03:42:07.056 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.056 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.056 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.056 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "465", 2023-07-18T03:42:07.057 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.057 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.057 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.057 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP port", 2023-07-18T03:42:07.058 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.058 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.058 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.058 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.059 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_sender": { 2023-07-18T03:42:07.059 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_sender", 2023-07-18T03:42:07.059 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.059 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.060 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.060 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.060 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.060 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.061 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.061 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP envelope sender", 2023-07-18T03:42:07.061 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.061 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.062 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.062 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.062 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_ssl": { 2023-07-18T03:42:07.063 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_ssl", 2023-07-18T03:42:07.063 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.063 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.063 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.064 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.064 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.064 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.065 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.065 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-18T03:42:07.065 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.065 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.066 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.066 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.066 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_user": { 2023-07-18T03:42:07.066 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_user", 2023-07-18T03:42:07.067 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.067 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.067 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.068 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.068 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.068 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.068 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.069 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "User to authenticate as", 2023-07-18T03:42:07.069 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.069 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.069 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.070 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.070 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.070 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.070 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.071 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "balancer", 2023-07-18T03:42:07.071 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.071 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.072 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.072 INFO:teuthology.orchestra.run.smithi040.stdout: "active": { 2023-07-18T03:42:07.072 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "active", 2023-07-18T03:42:07.072 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.073 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.073 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.073 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.074 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.074 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.074 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.074 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "automatically balance PGs across cluster", 2023-07-18T03:42:07.075 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.075 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.075 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.075 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.076 INFO:teuthology.orchestra.run.smithi040.stdout: "begin_time": { 2023-07-18T03:42:07.076 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "begin_time", 2023-07-18T03:42:07.076 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.076 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.077 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.077 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0000", 2023-07-18T03:42:07.077 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.077 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.078 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.078 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "beginning time of day to automatically balance", 2023-07-18T03:42:07.078 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-18T03:42:07.078 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.079 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.079 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.079 INFO:teuthology.orchestra.run.smithi040.stdout: "begin_weekday": { 2023-07-18T03:42:07.080 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "begin_weekday", 2023-07-18T03:42:07.080 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2023-07-18T03:42:07.080 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.080 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.080 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2023-07-18T03:42:07.080 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0", 2023-07-18T03:42:07.081 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "6", 2023-07-18T03:42:07.081 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.081 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-18T03:42:07.081 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-18T03:42:07.081 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.081 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.081 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.082 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_max_iterations": { 2023-07-18T03:42:07.082 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_max_iterations", 2023-07-18T03:42:07.082 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2023-07-18T03:42:07.082 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.082 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.082 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "25", 2023-07-18T03:42:07.082 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2023-07-18T03:42:07.083 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "250", 2023-07-18T03:42:07.083 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.083 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-18T03:42:07.083 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.083 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.083 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.083 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.084 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_metrics": { 2023-07-18T03:42:07.084 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_metrics", 2023-07-18T03:42:07.084 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.084 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.084 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.084 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "pgs,objects,bytes", 2023-07-18T03:42:07.084 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.085 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.085 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.085 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-18T03:42:07.085 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-18T03:42:07.085 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.085 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.085 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.086 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_step": { 2023-07-18T03:42:07.086 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_step", 2023-07-18T03:42:07.086 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:07.086 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.086 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.086 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0.5", 2023-07-18T03:42:07.086 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0.001", 2023-07-18T03:42:07.087 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "0.999", 2023-07-18T03:42:07.087 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.087 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "aggressiveness of optimization", 2023-07-18T03:42:07.087 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-18T03:42:07.087 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.087 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.087 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.088 INFO:teuthology.orchestra.run.smithi040.stdout: "end_time": { 2023-07-18T03:42:07.088 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "end_time", 2023-07-18T03:42:07.088 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.088 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.088 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.088 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "2359", 2023-07-18T03:42:07.088 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.089 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.089 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.089 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "ending time of day to automatically balance", 2023-07-18T03:42:07.089 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-18T03:42:07.089 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.089 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.089 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.090 INFO:teuthology.orchestra.run.smithi040.stdout: "end_weekday": { 2023-07-18T03:42:07.090 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "end_weekday", 2023-07-18T03:42:07.090 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2023-07-18T03:42:07.090 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.090 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.090 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2023-07-18T03:42:07.090 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0", 2023-07-18T03:42:07.091 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "6", 2023-07-18T03:42:07.091 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.091 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-18T03:42:07.091 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-18T03:42:07.091 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.091 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.091 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.092 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.092 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.092 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.092 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.092 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.092 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.092 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.093 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.093 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.093 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.093 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.093 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.093 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.093 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.094 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.094 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.094 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.094 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.094 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.094 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.095 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.095 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.095 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.095 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.095 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.095 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.095 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.096 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.096 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.096 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.096 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.096 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.096 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.097 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.097 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.097 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.097 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.097 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.097 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.097 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.098 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.098 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.098 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.098 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.098 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.098 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.098 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.099 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.099 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.099 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.099 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.099 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.099 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.099 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.100 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.100 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.100 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.100 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.100 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.100 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.101 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.101 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.101 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.101 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.101 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.101 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.101 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.102 INFO:teuthology.orchestra.run.smithi040.stdout: "min_score": { 2023-07-18T03:42:07.102 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "min_score", 2023-07-18T03:42:07.102 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:07.102 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.102 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.102 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2023-07-18T03:42:07.102 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.103 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.103 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.103 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-18T03:42:07.103 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.103 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.103 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.104 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": { 2023-07-18T03:42:07.104 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mode", 2023-07-18T03:42:07.104 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.104 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.104 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.104 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "upmap", 2023-07-18T03:42:07.105 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.105 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.105 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.105 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-compat", 2023-07-18T03:42:07.105 INFO:teuthology.orchestra.run.smithi040.stdout: "none", 2023-07-18T03:42:07.106 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap" 2023-07-18T03:42:07.106 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.106 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Balancer mode", 2023-07-18T03:42:07.106 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.106 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.106 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.107 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_ids": { 2023-07-18T03:42:07.107 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pool_ids", 2023-07-18T03:42:07.107 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.107 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.107 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.107 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.107 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.108 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.108 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.108 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-18T03:42:07.108 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.108 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.108 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.109 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2023-07-18T03:42:07.109 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2023-07-18T03:42:07.109 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.109 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.109 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.109 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2023-07-18T03:42:07.109 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.110 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.110 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.110 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-18T03:42:07.110 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.110 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.110 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.111 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap_max_deviation": { 2023-07-18T03:42:07.111 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "upmap_max_deviation", 2023-07-18T03:42:07.111 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.111 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.111 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.111 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2023-07-18T03:42:07.111 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2023-07-18T03:42:07.112 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.112 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.112 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-18T03:42:07.112 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-18T03:42:07.112 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.112 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.113 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap_max_optimizations": { 2023-07-18T03:42:07.113 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "upmap_max_optimizations", 2023-07-18T03:42:07.113 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2023-07-18T03:42:07.113 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.113 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.113 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:07.113 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.113 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.114 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.114 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-18T03:42:07.114 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.114 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.114 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.114 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.114 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.115 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cephadm", 2023-07-18T03:42:07.115 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.115 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.115 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.115 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_down_multiplier": { 2023-07-18T03:42:07.116 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_down_multiplier", 2023-07-18T03:42:07.116 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:07.116 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.116 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.116 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3.0", 2023-07-18T03:42:07.116 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.116 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.117 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.117 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-18T03:42:07.117 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.117 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.117 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.117 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_refresh_rate": { 2023-07-18T03:42:07.118 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_refresh_rate", 2023-07-18T03:42:07.118 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.118 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.118 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.118 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "20", 2023-07-18T03:42:07.118 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.118 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.118 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.119 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-18T03:42:07.119 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.119 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.119 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.119 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_starting_port": { 2023-07-18T03:42:07.119 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_starting_port", 2023-07-18T03:42:07.120 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.120 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.120 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.120 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "4721", 2023-07-18T03:42:07.120 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.120 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.120 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.121 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-18T03:42:07.121 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.121 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.121 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.121 INFO:teuthology.orchestra.run.smithi040.stdout: "alertmanager_web_password": { 2023-07-18T03:42:07.122 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "alertmanager_web_password", 2023-07-18T03:42:07.122 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.122 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.122 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.122 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:07.122 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.122 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.123 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.123 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Alertmanager web password", 2023-07-18T03:42:07.123 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.123 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.123 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.123 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.123 INFO:teuthology.orchestra.run.smithi040.stdout: "alertmanager_web_user": { 2023-07-18T03:42:07.124 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "alertmanager_web_user", 2023-07-18T03:42:07.124 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.124 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.124 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.124 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:07.124 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.124 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.125 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.125 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Alertmanager web user", 2023-07-18T03:42:07.125 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.125 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.125 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.125 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_ptrace": { 2023-07-18T03:42:07.126 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_ptrace", 2023-07-18T03:42:07.126 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.126 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.126 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.126 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.126 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.126 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.127 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.127 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-18T03:42:07.127 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-18T03:42:07.127 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.127 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.127 INFO:teuthology.orchestra.run.smithi040.stdout: "autotune_interval": { 2023-07-18T03:42:07.128 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "autotune_interval", 2023-07-18T03:42:07.128 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.128 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.128 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.128 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2023-07-18T03:42:07.128 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.128 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.129 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.129 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-18T03:42:07.129 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.129 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.129 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.129 INFO:teuthology.orchestra.run.smithi040.stdout: "autotune_memory_target_ratio": { 2023-07-18T03:42:07.130 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "autotune_memory_target_ratio", 2023-07-18T03:42:07.130 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:07.130 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.130 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.130 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0.7", 2023-07-18T03:42:07.130 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.130 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.131 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.131 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-18T03:42:07.131 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.131 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.131 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.131 INFO:teuthology.orchestra.run.smithi040.stdout: "cgroups_split": { 2023-07-18T03:42:07.131 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cgroups_split", 2023-07-18T03:42:07.132 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.132 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.132 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.132 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.132 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.132 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.132 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.133 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-18T03:42:07.133 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.133 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.133 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.133 INFO:teuthology.orchestra.run.smithi040.stdout: "config_checks_enabled": { 2023-07-18T03:42:07.133 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_checks_enabled", 2023-07-18T03:42:07.134 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.134 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.134 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.134 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.134 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.134 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.134 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.135 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-18T03:42:07.135 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.135 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.135 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.135 INFO:teuthology.orchestra.run.smithi040.stdout: "config_dashboard": { 2023-07-18T03:42:07.135 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_dashboard", 2023-07-18T03:42:07.136 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.136 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.136 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.136 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.136 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.136 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.136 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.137 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-18T03:42:07.137 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.137 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.137 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.137 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.137 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_alertmanager": { 2023-07-18T03:42:07.137 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_alertmanager", 2023-07-18T03:42:07.137 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.138 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.138 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.138 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-18T03:42:07.138 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.138 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.138 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.138 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2023-07-18T03:42:07.139 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.139 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.139 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.139 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_base": { 2023-07-18T03:42:07.139 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_base", 2023-07-18T03:42:07.139 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.140 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.140 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.140 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-18T03:42:07.140 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.140 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.140 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.140 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Container image name, without the tag", 2023-07-18T03:42:07.141 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.141 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.141 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.141 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_elasticsearch": { 2023-07-18T03:42:07.141 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_elasticsearch", 2023-07-18T03:42:07.141 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.142 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.142 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.142 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-18T03:42:07.142 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.142 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.142 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.142 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "elasticsearch container image", 2023-07-18T03:42:07.143 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.143 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.143 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.143 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_grafana": { 2023-07-18T03:42:07.143 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_grafana", 2023-07-18T03:42:07.143 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.144 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.144 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.144 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-18T03:42:07.144 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.144 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.144 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.144 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2023-07-18T03:42:07.145 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.145 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.145 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.145 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_haproxy": { 2023-07-18T03:42:07.145 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_haproxy", 2023-07-18T03:42:07.145 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.146 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.146 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.146 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-18T03:42:07.146 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.146 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.146 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.146 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "HAproxy container image", 2023-07-18T03:42:07.147 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.147 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.147 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.147 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_agent": { 2023-07-18T03:42:07.147 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_agent", 2023-07-18T03:42:07.147 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.148 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.148 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.148 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-18T03:42:07.148 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.148 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.148 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.148 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger agent container image", 2023-07-18T03:42:07.149 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.149 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.149 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.149 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_collector": { 2023-07-18T03:42:07.149 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_collector", 2023-07-18T03:42:07.149 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.150 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.150 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.150 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-18T03:42:07.150 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.150 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.150 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.150 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger collector container image", 2023-07-18T03:42:07.150 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.151 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.151 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.151 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_query": { 2023-07-18T03:42:07.151 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_query", 2023-07-18T03:42:07.151 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.151 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.152 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.152 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-18T03:42:07.152 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.152 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.152 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.152 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger query container image", 2023-07-18T03:42:07.152 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.153 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.153 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.153 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_keepalived": { 2023-07-18T03:42:07.153 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_keepalived", 2023-07-18T03:42:07.153 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.153 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.154 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.154 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-18T03:42:07.154 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.154 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.154 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.154 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Keepalived container image", 2023-07-18T03:42:07.154 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.155 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.155 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.155 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_loki": { 2023-07-18T03:42:07.155 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_loki", 2023-07-18T03:42:07.155 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.155 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.155 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.156 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-18T03:42:07.156 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.156 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.156 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.156 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Loki container image", 2023-07-18T03:42:07.156 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.156 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.157 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.157 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_node_exporter": { 2023-07-18T03:42:07.157 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_node_exporter", 2023-07-18T03:42:07.157 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.157 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.157 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.158 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-18T03:42:07.158 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.158 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.158 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.158 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2023-07-18T03:42:07.158 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.158 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.159 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.159 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_prometheus": { 2023-07-18T03:42:07.159 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_prometheus", 2023-07-18T03:42:07.159 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.159 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.159 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.160 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-18T03:42:07.160 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.160 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.160 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.160 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2023-07-18T03:42:07.160 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.160 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.161 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.161 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_promtail": { 2023-07-18T03:42:07.161 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_promtail", 2023-07-18T03:42:07.161 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.161 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.161 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.162 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-18T03:42:07.162 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.162 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.162 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.162 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Promtail container image", 2023-07-18T03:42:07.162 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.162 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.163 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.163 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_snmp_gateway": { 2023-07-18T03:42:07.163 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_snmp_gateway", 2023-07-18T03:42:07.163 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.163 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.163 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.163 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-18T03:42:07.164 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.164 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.164 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.164 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SNMP Gateway container image", 2023-07-18T03:42:07.164 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.164 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.164 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.165 INFO:teuthology.orchestra.run.smithi040.stdout: "container_init": { 2023-07-18T03:42:07.165 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_init", 2023-07-18T03:42:07.165 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.165 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.165 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.165 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.166 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.166 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.166 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.166 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Run podman/docker with `--init`", 2023-07-18T03:42:07.166 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.166 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.166 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.167 INFO:teuthology.orchestra.run.smithi040.stdout: "daemon_cache_timeout": { 2023-07-18T03:42:07.167 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "daemon_cache_timeout", 2023-07-18T03:42:07.167 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.167 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.167 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.167 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2023-07-18T03:42:07.168 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.168 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.168 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.168 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-18T03:42:07.168 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.168 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.168 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: "default_cephadm_command_timeout": { 2023-07-18T03:42:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "default_cephadm_command_timeout", 2023-07-18T03:42:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.170 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "900", 2023-07-18T03:42:07.170 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.170 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.170 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.170 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-18T03:42:07.170 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.170 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: "default_registry": { 2023-07-18T03:42:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "default_registry", 2023-07-18T03:42:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io", 2023-07-18T03:42:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-18T03:42:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: "device_cache_timeout": { 2023-07-18T03:42:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_cache_timeout", 2023-07-18T03:42:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1800", 2023-07-18T03:42:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache device inventory", 2023-07-18T03:42:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: "device_enhanced_scan": { 2023-07-18T03:42:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_enhanced_scan", 2023-07-18T03:42:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-18T03:42:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: "facts_cache_timeout": { 2023-07-18T03:42:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "facts_cache_timeout", 2023-07-18T03:42:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2023-07-18T03:42:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache host facts data", 2023-07-18T03:42:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: "host_check_interval": { 2023-07-18T03:42:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "host_check_interval", 2023-07-18T03:42:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2023-07-18T03:42:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to perform a host check", 2023-07-18T03:42:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: "log_refresh_metadata": { 2023-07-18T03:42:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_refresh_metadata", 2023-07-18T03:42:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-18T03:42:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-18T03:42:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-18T03:42:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-18T03:42:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-18T03:42:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-18T03:42:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-18T03:42:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "*", 2023-07-18T03:42:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-18T03:42:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: "max_count_per_host": { 2023-07-18T03:42:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_count_per_host", 2023-07-18T03:42:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "max number of daemons per service per host", 2023-07-18T03:42:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd_draining_count": { 2023-07-18T03:42:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_osd_draining_count", 2023-07-18T03:42:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-18T03:42:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: "migration_current": { 2023-07-18T03:42:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "migration_current", 2023-07-18T03:42:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "internal - do not modify", 2023-07-18T03:42:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": { 2023-07-18T03:42:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mode", 2023-07-18T03:42:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "root", 2023-07-18T03:42:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm-package", 2023-07-18T03:42:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: "root" 2023-07-18T03:42:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "mode for remote execution of cephadm", 2023-07-18T03:42:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: "prometheus_alerts_path": { 2023-07-18T03:42:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus_alerts_path", 2023-07-18T03:42:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-18T03:42:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-18T03:42:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: "prometheus_web_password": { 2023-07-18T03:42:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus_web_password", 2023-07-18T03:42:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus web password", 2023-07-18T03:42:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: "prometheus_web_user": { 2023-07-18T03:42:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus_web_user", 2023-07-18T03:42:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus web user", 2023-07-18T03:42:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_insecure": { 2023-07-18T03:42:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_insecure", 2023-07-18T03:42:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-18T03:42:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_password": { 2023-07-18T03:42:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_password", 2023-07-18T03:42:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-18T03:42:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_url": { 2023-07-18T03:42:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_url", 2023-07-18T03:42:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-18T03:42:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_username": { 2023-07-18T03:42:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_username", 2023-07-18T03:42:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-18T03:42:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: "secure_monitoring_stack": { 2023-07-18T03:42:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "secure_monitoring_stack", 2023-07-18T03:42:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-18T03:42:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: "service_discovery_port": { 2023-07-18T03:42:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "service_discovery_port", 2023-07-18T03:42:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8765", 2023-07-18T03:42:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "cephadm service discovery port", 2023-07-18T03:42:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: "ssh_config_file": { 2023-07-18T03:42:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssh_config_file", 2023-07-18T03:42:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-18T03:42:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: "use_agent": { 2023-07-18T03:42:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "use_agent", 2023-07-18T03:42:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-18T03:42:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "use_repo_digest": { 2023-07-18T03:42:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "use_repo_digest", 2023-07-18T03:42:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-18T03:42:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_failed_host_check": { 2023-07-18T03:42:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_failed_host_check", 2023-07-18T03:42:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if the host check fails", 2023-07-18T03:42:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_stray_daemons": { 2023-07-18T03:42:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_stray_daemons", 2023-07-18T03:42:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-18T03:42:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_stray_hosts": { 2023-07-18T03:42:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_stray_hosts", 2023-07-18T03:42:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-18T03:42:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crash", 2023-07-18T03:42:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: "retain_interval": { 2023-07-18T03:42:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "retain_interval", 2023-07-18T03:42:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "31536000", 2023-07-18T03:42:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-18T03:42:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_recent_interval": { 2023-07-18T03:42:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_recent_interval", 2023-07-18T03:42:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1209600", 2023-07-18T03:42:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-18T03:42:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "dashboard", 2023-07-18T03:42:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-18T03:42:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-18T03:42:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: "ALERTMANAGER_API_HOST": { 2023-07-18T03:42:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-18T03:42:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-18T03:42:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-18T03:42:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.257 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.257 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.257 INFO:teuthology.orchestra.run.smithi040.stdout: "AUDIT_API_ENABLED": { 2023-07-18T03:42:07.257 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "AUDIT_API_ENABLED", 2023-07-18T03:42:07.257 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.257 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.257 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.258 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.258 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.258 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.258 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.258 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.258 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.258 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.259 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.259 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.259 INFO:teuthology.orchestra.run.smithi040.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-18T03:42:07.259 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-18T03:42:07.259 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.259 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.259 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.260 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.260 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.260 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.260 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.260 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.260 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.260 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.261 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.261 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.261 INFO:teuthology.orchestra.run.smithi040.stdout: "ENABLE_BROWSABLE_API": { 2023-07-18T03:42:07.261 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-18T03:42:07.261 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.261 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.261 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.262 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.262 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.262 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.262 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.262 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.262 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.262 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.263 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.263 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.263 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-18T03:42:07.263 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-18T03:42:07.263 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.263 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.263 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.263 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.264 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.264 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.264 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.264 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.264 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.264 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.265 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.265 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.265 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-18T03:42:07.265 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-18T03:42:07.265 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.265 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.265 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.266 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.266 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.266 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.266 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.266 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.266 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.266 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.267 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.267 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-18T03:42:07.267 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-18T03:42:07.267 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.267 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.267 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.268 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.268 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.268 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.268 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.268 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.268 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.268 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.269 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.269 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-18T03:42:07.269 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-18T03:42:07.269 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.269 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.269 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.270 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.270 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.270 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.270 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.270 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.270 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-18T03:42:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-18T03:42:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-18T03:42:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-18T03:42:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.274 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-18T03:42:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-18T03:42:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.275 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.276 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-18T03:42:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-18T03:42:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.278 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_PASSWORD": { 2023-07-18T03:42:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-18T03:42:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.279 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.280 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-18T03:42:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-18T03:42:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.281 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.282 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_URL": { 2023-07-18T03:42:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_URL", 2023-07-18T03:42:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.283 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.284 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_USERNAME": { 2023-07-18T03:42:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-18T03:42:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.285 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:07.286 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.286 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.286 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.286 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.286 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.286 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.286 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.287 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.287 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-18T03:42:07.287 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-18T03:42:07.287 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.287 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.287 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.287 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.288 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.288 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.288 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.288 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.288 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.288 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.288 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.289 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.289 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-18T03:42:07.289 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-18T03:42:07.289 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.289 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.289 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.289 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.290 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.290 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.290 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.290 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.290 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.290 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.290 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.291 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.291 INFO:teuthology.orchestra.run.smithi040.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-18T03:42:07.291 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-18T03:42:07.291 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.291 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.291 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.291 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.292 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.292 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.292 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.292 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.292 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.292 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.292 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.293 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.293 INFO:teuthology.orchestra.run.smithi040.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-18T03:42:07.293 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-18T03:42:07.293 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.293 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.293 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.293 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.294 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.294 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.294 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.294 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.294 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.294 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.294 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.295 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.295 INFO:teuthology.orchestra.run.smithi040.stdout: "PROMETHEUS_API_HOST": { 2023-07-18T03:42:07.295 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-18T03:42:07.295 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.295 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.295 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.295 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.296 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.296 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.296 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.296 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.296 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.296 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.296 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.297 INFO:teuthology.orchestra.run.smithi040.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-18T03:42:07.297 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-18T03:42:07.297 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.297 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.297 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.298 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.298 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.298 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.298 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.298 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.298 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.298 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.299 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.299 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-18T03:42:07.299 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-18T03:42:07.299 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.299 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.299 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.300 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.300 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.300 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.300 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.300 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.300 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.300 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.301 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.301 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-18T03:42:07.301 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-18T03:42:07.301 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.301 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.301 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.302 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.302 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.302 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.302 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.302 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.302 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.302 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.303 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.303 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-18T03:42:07.303 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-18T03:42:07.303 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.303 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.303 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.303 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.304 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.304 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.304 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.304 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.304 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.304 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.304 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.305 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.305 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-18T03:42:07.305 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-18T03:42:07.305 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.305 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.305 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.305 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.306 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.306 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.306 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.306 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.307 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.307 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.307 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.307 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.307 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-18T03:42:07.307 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-18T03:42:07.307 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.307 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.308 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.308 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.308 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.308 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.308 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.308 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.308 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.309 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.309 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.309 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.309 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-18T03:42:07.309 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-18T03:42:07.309 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.309 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.310 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.310 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.310 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.310 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.310 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.310 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.310 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.311 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.311 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.311 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.311 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-18T03:42:07.311 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-18T03:42:07.311 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.311 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.312 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.312 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.312 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.312 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.312 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.312 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.312 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.313 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.313 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.313 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_ENABLED": { 2023-07-18T03:42:07.313 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-18T03:42:07.313 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.313 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.313 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.314 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.314 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.314 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.314 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.314 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.314 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.314 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.315 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.315 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-18T03:42:07.315 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-18T03:42:07.315 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.315 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.315 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.316 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-18T03:42:07.316 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.316 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.316 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.316 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.316 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.316 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.317 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.317 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.317 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-18T03:42:07.317 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-18T03:42:07.317 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.317 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.317 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.318 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:07.318 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.318 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.318 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.318 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.318 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.318 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.319 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.319 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-18T03:42:07.319 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-18T03:42:07.319 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.319 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.319 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.320 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8", 2023-07-18T03:42:07.320 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.320 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.320 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.320 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.320 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.320 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.321 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.321 INFO:teuthology.orchestra.run.smithi040.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-18T03:42:07.321 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-18T03:42:07.321 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.321 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.321 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.322 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "45", 2023-07-18T03:42:07.322 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.322 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.322 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.322 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.322 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.323 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.323 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.323 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_ACCESS_KEY": { 2023-07-18T03:42:07.323 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-18T03:42:07.323 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.323 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.324 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.324 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.324 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.324 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.324 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.324 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.324 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.325 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.325 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.325 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-18T03:42:07.325 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-18T03:42:07.325 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.325 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.326 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.326 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:07.326 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.326 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.326 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.326 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.326 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.327 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.327 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.327 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.327 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_SECRET_KEY": { 2023-07-18T03:42:07.327 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-18T03:42:07.327 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.327 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.328 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.328 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.328 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.328 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.328 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.328 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.328 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.329 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.329 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.329 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.329 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_SSL_VERIFY": { 2023-07-18T03:42:07.329 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-18T03:42:07.329 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.329 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.330 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.330 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.330 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.330 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.330 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.330 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.330 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.331 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.331 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.331 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.331 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-18T03:42:07.331 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-18T03:42:07.331 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.331 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.332 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.332 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2023-07-18T03:42:07.332 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.332 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.332 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.332 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.332 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.332 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.333 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.333 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-18T03:42:07.333 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-18T03:42:07.333 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.333 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.333 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.334 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:07.334 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.334 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.334 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.334 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.334 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.334 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.335 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.335 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-18T03:42:07.335 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-18T03:42:07.335 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.335 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.335 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.336 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2023-07-18T03:42:07.336 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.336 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.336 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.336 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.336 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.336 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.337 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.337 INFO:teuthology.orchestra.run.smithi040.stdout: "cross_origin_url": { 2023-07-18T03:42:07.337 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cross_origin_url", 2023-07-18T03:42:07.337 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.337 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.337 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.338 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.338 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.338 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.338 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.338 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.338 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.338 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.338 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.339 INFO:teuthology.orchestra.run.smithi040.stdout: "crt_file": { 2023-07-18T03:42:07.339 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crt_file", 2023-07-18T03:42:07.339 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.339 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.339 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.339 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.340 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.340 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.340 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.340 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.340 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.340 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.340 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.341 INFO:teuthology.orchestra.run.smithi040.stdout: "debug": { 2023-07-18T03:42:07.341 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "debug", 2023-07-18T03:42:07.341 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.341 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.341 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.341 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.342 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.342 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.342 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.342 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable/disable debug options", 2023-07-18T03:42:07.342 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.342 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.342 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.343 INFO:teuthology.orchestra.run.smithi040.stdout: "jwt_token_ttl": { 2023-07-18T03:42:07.343 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "jwt_token_ttl", 2023-07-18T03:42:07.343 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.343 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.343 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.343 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "28800", 2023-07-18T03:42:07.344 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.344 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.344 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.344 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.344 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.344 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.344 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.345 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.345 INFO:teuthology.orchestra.run.smithi040.stdout: "key_file": { 2023-07-18T03:42:07.345 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "key_file", 2023-07-18T03:42:07.345 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.345 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.345 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.345 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.346 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.346 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.346 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.346 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.346 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.346 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.346 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.347 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.347 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.347 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.347 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.347 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.347 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.348 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.348 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.348 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.348 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.348 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.348 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.348 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.349 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.349 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.349 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.349 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.349 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.349 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.350 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.350 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.350 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.350 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.350 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.350 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.351 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.351 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.351 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.351 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.351 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.351 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.351 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.351 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.352 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.352 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.352 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.352 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.352 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.353 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.353 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.353 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.353 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.353 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.353 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.353 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.354 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.354 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.354 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.354 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.354 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.354 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.354 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.354 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.355 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.355 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.355 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.355 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.355 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.355 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.356 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.356 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.356 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.356 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.356 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.356 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.356 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.357 INFO:teuthology.orchestra.run.smithi040.stdout: "motd": { 2023-07-18T03:42:07.357 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "motd", 2023-07-18T03:42:07.357 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.357 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.357 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.357 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.358 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.358 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.358 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.358 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "The message of the day", 2023-07-18T03:42:07.358 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.358 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.358 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.359 INFO:teuthology.orchestra.run.smithi040.stdout: "redirect_resolve_ip_addr": { 2023-07-18T03:42:07.359 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "redirect_resolve_ip_addr", 2023-07-18T03:42:07.359 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.359 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.359 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.359 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.359 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.360 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.360 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.360 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.360 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.360 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.360 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.360 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.361 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2023-07-18T03:42:07.361 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2023-07-18T03:42:07.361 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.361 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.361 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.361 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "::", 2023-07-18T03:42:07.361 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.362 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.362 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.362 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.362 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.362 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.362 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.362 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.363 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2023-07-18T03:42:07.363 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2023-07-18T03:42:07.363 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.363 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.363 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.363 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8080", 2023-07-18T03:42:07.363 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.364 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.364 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.364 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.364 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.364 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.364 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.364 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.364 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl": { 2023-07-18T03:42:07.365 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl", 2023-07-18T03:42:07.365 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.365 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.365 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.365 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.365 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.365 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.366 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.366 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.366 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.366 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.366 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.366 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.366 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl_server_port": { 2023-07-18T03:42:07.367 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl_server_port", 2023-07-18T03:42:07.367 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.367 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.367 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.367 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8443", 2023-07-18T03:42:07.367 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.367 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.368 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.368 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.368 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.368 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.368 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.368 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_behaviour": { 2023-07-18T03:42:07.369 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_behaviour", 2023-07-18T03:42:07.369 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.369 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.369 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.369 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "redirect", 2023-07-18T03:42:07.369 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.369 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.370 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.370 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.370 INFO:teuthology.orchestra.run.smithi040.stdout: "redirect" 2023-07-18T03:42:07.370 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.370 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.370 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.370 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.371 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.371 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.371 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_error_status_code": { 2023-07-18T03:42:07.371 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_error_status_code", 2023-07-18T03:42:07.371 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.371 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.371 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.372 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "500", 2023-07-18T03:42:07.372 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "400", 2023-07-18T03:42:07.372 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "599", 2023-07-18T03:42:07.372 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.372 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.372 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.372 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.373 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.373 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.373 INFO:teuthology.orchestra.run.smithi040.stdout: "url_prefix": { 2023-07-18T03:42:07.373 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "url_prefix", 2023-07-18T03:42:07.373 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.373 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.373 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.374 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.374 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.374 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.374 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.374 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.374 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.374 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.375 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.375 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.375 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.375 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.375 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.375 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "devicehealth", 2023-07-18T03:42:07.375 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.376 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.376 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.376 INFO:teuthology.orchestra.run.smithi040.stdout: "enable_monitoring": { 2023-07-18T03:42:07.376 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enable_monitoring", 2023-07-18T03:42:07.376 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.376 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.377 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.377 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.377 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.377 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.377 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.377 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "monitor device health metrics", 2023-07-18T03:42:07.377 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.378 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.378 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.378 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.378 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.378 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.378 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.378 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.379 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.379 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.379 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.379 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.379 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.379 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.379 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.380 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.380 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.380 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.380 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.380 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.380 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.380 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.381 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.381 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.381 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.381 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.381 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.381 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.381 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.382 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.382 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.382 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.382 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.382 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.382 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.382 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.382 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.383 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.383 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.383 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.383 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.383 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.383 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.383 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.384 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.384 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.384 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.384 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.384 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.384 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.384 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.385 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.385 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.385 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.385 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.385 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.385 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.385 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.386 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.386 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.386 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.386 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.386 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.386 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.386 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.387 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.387 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.387 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.387 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.387 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.387 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.387 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.388 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.388 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.388 INFO:teuthology.orchestra.run.smithi040.stdout: "mark_out_threshold": { 2023-07-18T03:42:07.388 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mark_out_threshold", 2023-07-18T03:42:07.388 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.388 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.388 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.389 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "2419200", 2023-07-18T03:42:07.389 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.389 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.389 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.389 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-18T03:42:07.389 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.389 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.389 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.390 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": { 2023-07-18T03:42:07.390 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pool_name", 2023-07-18T03:42:07.390 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.390 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.390 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.390 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "device_health_metrics", 2023-07-18T03:42:07.391 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.391 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.391 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.391 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-18T03:42:07.391 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.391 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.391 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.392 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.392 INFO:teuthology.orchestra.run.smithi040.stdout: "retention_period": { 2023-07-18T03:42:07.392 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "retention_period", 2023-07-18T03:42:07.392 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.392 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.392 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.392 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15552000", 2023-07-18T03:42:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long to retain device health metrics", 2023-07-18T03:42:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.394 INFO:teuthology.orchestra.run.smithi040.stdout: "scrape_frequency": { 2023-07-18T03:42:07.394 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "scrape_frequency", 2023-07-18T03:42:07.394 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.394 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.394 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.394 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "86400", 2023-07-18T03:42:07.394 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.395 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.395 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.395 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-18T03:42:07.395 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.395 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.395 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.396 INFO:teuthology.orchestra.run.smithi040.stdout: "self_heal": { 2023-07-18T03:42:07.396 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "self_heal", 2023-07-18T03:42:07.396 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.396 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.396 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.396 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.396 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.397 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.397 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.397 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-18T03:42:07.397 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.397 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.397 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.397 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.398 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2023-07-18T03:42:07.398 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2023-07-18T03:42:07.398 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.398 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.398 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.398 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2023-07-18T03:42:07.398 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.398 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.399 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.399 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to wake up and check device health", 2023-07-18T03:42:07.399 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.399 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.399 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.399 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.399 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_threshold": { 2023-07-18T03:42:07.400 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_threshold", 2023-07-18T03:42:07.400 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.400 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.400 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.400 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "7257600", 2023-07-18T03:42:07.400 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.400 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.401 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.401 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-18T03:42:07.401 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.401 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.401 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.401 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.401 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.402 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.402 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "diskprediction_local", 2023-07-18T03:42:07.402 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.402 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.402 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.403 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.403 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.403 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.403 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.403 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.403 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.403 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.404 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.404 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.404 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.404 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.404 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.404 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.404 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.405 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.405 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.405 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.405 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.405 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.405 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.405 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.406 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.406 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.406 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.406 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.406 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.407 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.407 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.407 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.407 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.407 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.407 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.407 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.408 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.408 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.408 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.408 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.408 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.408 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.408 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.409 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.409 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.409 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.409 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.409 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.409 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.409 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.410 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.410 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.410 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.410 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.410 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.410 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.410 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.411 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.411 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.411 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.411 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.411 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.411 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.411 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.412 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.412 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.412 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.412 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.412 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.412 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.412 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.413 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.413 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.413 INFO:teuthology.orchestra.run.smithi040.stdout: "predict_interval": { 2023-07-18T03:42:07.413 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "predict_interval", 2023-07-18T03:42:07.413 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.413 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.413 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.414 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "86400", 2023-07-18T03:42:07.414 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.414 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.414 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.414 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.414 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.414 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.414 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.415 INFO:teuthology.orchestra.run.smithi040.stdout: "predictor_model": { 2023-07-18T03:42:07.415 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "predictor_model", 2023-07-18T03:42:07.415 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.415 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.415 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.415 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "prophetstor", 2023-07-18T03:42:07.416 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.416 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.416 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.416 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.416 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.416 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.416 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.417 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.417 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2023-07-18T03:42:07.417 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2023-07-18T03:42:07.417 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.417 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.417 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.417 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2023-07-18T03:42:07.418 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.418 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.418 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.418 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.418 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.418 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.418 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.419 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.419 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.419 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.419 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.419 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "influx", 2023-07-18T03:42:07.419 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": false, 2023-07-18T03:42:07.419 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "influxdb python module not found", 2023-07-18T03:42:07.420 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.420 INFO:teuthology.orchestra.run.smithi040.stdout: "batch_size": { 2023-07-18T03:42:07.420 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "batch_size", 2023-07-18T03:42:07.420 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.420 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.420 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.420 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5000", 2023-07-18T03:42:07.421 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.421 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.421 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.421 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-18T03:42:07.421 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.421 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.421 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.422 INFO:teuthology.orchestra.run.smithi040.stdout: "database": { 2023-07-18T03:42:07.422 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "database", 2023-07-18T03:42:07.422 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.422 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.422 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.422 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "ceph", 2023-07-18T03:42:07.422 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.423 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.423 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.423 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-18T03:42:07.423 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.423 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.423 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.423 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.424 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": { 2023-07-18T03:42:07.424 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "hostname", 2023-07-18T03:42:07.424 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.424 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.424 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.424 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.424 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.425 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.425 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.425 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB server hostname", 2023-07-18T03:42:07.425 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.425 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.425 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.425 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.426 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2023-07-18T03:42:07.426 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2023-07-18T03:42:07.426 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.426 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.426 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.426 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "30", 2023-07-18T03:42:07.426 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "5", 2023-07-18T03:42:07.427 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.427 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.427 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-18T03:42:07.427 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.427 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.427 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.427 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.428 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.428 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.428 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.428 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.428 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.428 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.429 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.429 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.429 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.429 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.429 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.429 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.429 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.430 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.430 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.430 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.430 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.430 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.430 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.430 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.431 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.431 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.431 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.431 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.431 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.431 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.431 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.432 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.432 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.432 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.432 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.432 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.432 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.432 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.433 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.433 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.433 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.433 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.433 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.433 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.433 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.434 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.434 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.434 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.434 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.434 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.434 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.434 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.435 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.435 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.435 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.435 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.435 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.435 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.435 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.436 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.436 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.436 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.436 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.436 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.436 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.436 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.437 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.437 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.437 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.437 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.437 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.437 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.438 INFO:teuthology.orchestra.run.smithi040.stdout: "password": { 2023-07-18T03:42:07.438 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "password", 2023-07-18T03:42:07.438 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.438 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.438 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.438 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.438 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.439 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.439 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.439 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "password of InfluxDB server user", 2023-07-18T03:42:07.439 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.439 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.439 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.439 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.439 INFO:teuthology.orchestra.run.smithi040.stdout: "port": { 2023-07-18T03:42:07.440 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "port", 2023-07-18T03:42:07.440 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.440 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.440 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.440 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8086", 2023-07-18T03:42:07.440 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.440 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.441 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.441 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB server port", 2023-07-18T03:42:07.441 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.441 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.441 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.441 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.441 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl": { 2023-07-18T03:42:07.442 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl", 2023-07-18T03:42:07.442 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.442 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.442 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.442 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "false", 2023-07-18T03:42:07.442 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.442 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.443 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.443 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-18T03:42:07.443 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.443 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.443 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.443 INFO:teuthology.orchestra.run.smithi040.stdout: "threads": { 2023-07-18T03:42:07.443 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "threads", 2023-07-18T03:42:07.444 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.444 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.444 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.444 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2023-07-18T03:42:07.444 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2023-07-18T03:42:07.444 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "32", 2023-07-18T03:42:07.444 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.445 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-18T03:42:07.445 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.445 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.445 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.445 INFO:teuthology.orchestra.run.smithi040.stdout: "username": { 2023-07-18T03:42:07.445 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "username", 2023-07-18T03:42:07.446 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.446 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.446 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.446 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.446 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.446 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.446 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.447 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "username of InfluxDB server user", 2023-07-18T03:42:07.447 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.447 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.447 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.447 INFO:teuthology.orchestra.run.smithi040.stdout: "verify_ssl": { 2023-07-18T03:42:07.447 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "verify_ssl", 2023-07-18T03:42:07.447 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.448 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.448 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.448 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "true", 2023-07-18T03:42:07.448 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.448 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.448 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.448 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-18T03:42:07.449 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.449 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.449 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.449 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.449 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.449 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.450 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "insights", 2023-07-18T03:42:07.450 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.450 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.450 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.450 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.450 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.450 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.451 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.451 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.451 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.451 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.451 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.451 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.451 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.452 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.452 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.452 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.452 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.452 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.452 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.452 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.453 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.453 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.453 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.453 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.453 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.453 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.454 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.454 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.454 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.454 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.454 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.454 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.454 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.454 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.455 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.455 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.455 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.455 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.455 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.456 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.456 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.456 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.456 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.456 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.456 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.456 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.457 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.457 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.457 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.457 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.457 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.457 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.457 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.458 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.458 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.458 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.458 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.458 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.458 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.459 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.459 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.459 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.459 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.459 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.459 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.459 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.460 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.460 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.460 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.460 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.460 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.460 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.461 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "iostat", 2023-07-18T03:42:07.461 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.461 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.461 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.461 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.461 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.461 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.461 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.462 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.462 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.462 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.462 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.462 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.462 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.462 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.463 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.463 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.463 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.463 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.463 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.463 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.463 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.464 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.464 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.464 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.464 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.464 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.464 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.464 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.465 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.465 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.465 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.465 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.465 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.465 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.465 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.466 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.466 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.466 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.466 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.466 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.466 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.466 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.466 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.467 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.467 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.467 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.467 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.467 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.467 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.467 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.468 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.468 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.468 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.468 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.468 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.468 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.468 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.469 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.469 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.469 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.469 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.469 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.469 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.469 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.470 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.470 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.470 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.470 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.470 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.470 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.470 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.471 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.471 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.471 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.471 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.471 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "k8sevents", 2023-07-18T03:42:07.471 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.471 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.472 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.472 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_event_retention_days": { 2023-07-18T03:42:07.472 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ceph_event_retention_days", 2023-07-18T03:42:07.472 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.472 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.472 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.472 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "7", 2023-07-18T03:42:07.473 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.473 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.473 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.473 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-18T03:42:07.473 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.473 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.473 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.474 INFO:teuthology.orchestra.run.smithi040.stdout: "config_check_secs": { 2023-07-18T03:42:07.474 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_check_secs", 2023-07-18T03:42:07.474 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.474 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.474 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.474 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:07.475 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "10", 2023-07-18T03:42:07.475 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.475 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.475 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-18T03:42:07.475 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.475 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.475 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.475 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.476 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.476 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.476 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.476 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.476 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.476 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.476 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.477 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.477 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.477 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.477 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.477 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.477 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.477 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.496 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "localpool", 2023-07-18T03:42:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.506 INFO:teuthology.orchestra.run.smithi040.stdout: "failure_domain": { 2023-07-18T03:42:07.506 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "failure_domain", 2023-07-18T03:42:07.506 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.506 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.506 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.506 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "host", 2023-07-18T03:42:07.506 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.507 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.507 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.507 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "failure domain for any created local pool", 2023-07-18T03:42:07.507 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-18T03:42:07.507 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.507 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.507 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.508 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.508 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.508 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.508 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.508 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.508 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.508 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.509 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.509 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.509 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.509 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.509 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.509 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.509 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.510 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.510 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.510 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.510 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.510 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.510 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.510 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.511 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.511 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.511 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.511 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.511 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.511 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.511 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.512 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.512 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.512 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.512 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.512 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.512 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.512 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.513 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.513 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.513 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.513 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.513 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.513 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.513 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.514 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.514 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.514 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.514 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.514 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.514 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.514 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.514 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.515 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.515 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.515 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.515 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.515 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.515 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.515 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.516 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.516 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.516 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.516 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.516 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.516 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.516 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.517 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.517 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.517 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.517 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.517 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.517 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.517 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": { 2023-07-18T03:42:07.518 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "min_size", 2023-07-18T03:42:07.518 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.518 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.518 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.518 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.518 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.518 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.518 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.519 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default min_size for any created local pool", 2023-07-18T03:42:07.519 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-18T03:42:07.519 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.519 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.519 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.519 INFO:teuthology.orchestra.run.smithi040.stdout: "num_rep": { 2023-07-18T03:42:07.519 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "num_rep", 2023-07-18T03:42:07.520 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.520 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.520 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.520 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3", 2023-07-18T03:42:07.520 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.520 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.520 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.521 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default replica count for any created local pool", 2023-07-18T03:42:07.521 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.521 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.521 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.521 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.521 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": { 2023-07-18T03:42:07.521 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pg_num", 2023-07-18T03:42:07.522 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.522 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.522 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.522 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "128", 2023-07-18T03:42:07.522 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.522 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.522 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.523 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default pg_num for any created local pool", 2023-07-18T03:42:07.523 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.523 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.523 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.523 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.523 INFO:teuthology.orchestra.run.smithi040.stdout: "prefix": { 2023-07-18T03:42:07.523 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prefix", 2023-07-18T03:42:07.524 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.524 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.524 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.524 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.524 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.524 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.524 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.524 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "name prefix for any created local pool", 2023-07-18T03:42:07.525 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.525 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.525 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.525 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.525 INFO:teuthology.orchestra.run.smithi040.stdout: "subtree": { 2023-07-18T03:42:07.525 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "subtree", 2023-07-18T03:42:07.525 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.526 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.526 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.526 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "rack", 2023-07-18T03:42:07.526 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.526 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.526 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.526 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-18T03:42:07.527 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-18T03:42:07.527 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.527 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.527 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.527 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.527 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.527 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.528 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mds_autoscaler", 2023-07-18T03:42:07.528 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.528 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.528 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.528 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.528 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.529 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.529 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.529 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.529 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.529 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.529 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.529 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.530 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.530 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.530 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.530 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.530 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.530 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.530 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.530 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.531 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.531 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.531 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.531 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.531 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.531 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.531 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.532 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.532 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.532 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.532 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.532 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.532 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.532 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.533 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.533 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.533 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.533 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.533 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.533 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.533 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.534 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.534 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.534 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.534 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.534 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.534 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.534 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.535 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.535 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.535 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.535 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.535 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.535 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.535 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.536 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.536 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.536 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.536 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.536 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.536 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.536 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.537 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.537 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.537 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.537 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.537 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.537 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.537 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.537 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.538 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.538 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.538 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.538 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.538 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.538 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.538 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mirroring", 2023-07-18T03:42:07.539 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.539 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.539 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.539 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.539 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.539 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.539 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.540 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.540 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.540 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.540 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.540 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.540 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.540 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.541 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.541 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.541 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.541 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.541 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.541 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.541 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.542 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.542 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.542 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.542 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.542 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.542 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.542 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.543 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.543 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.543 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.543 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.543 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.543 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.543 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.544 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.544 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.544 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.544 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.544 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.544 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.544 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.545 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.545 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.545 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.545 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.545 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.545 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.545 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.546 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.546 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.546 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.546 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.546 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.546 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.546 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.547 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.547 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.547 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.547 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.547 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.547 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.547 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.547 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.548 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.548 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.548 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.548 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.548 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.548 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.548 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.549 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.549 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.549 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.549 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.549 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.549 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "nfs", 2023-07-18T03:42:07.549 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.550 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.550 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.550 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.550 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.550 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.550 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.550 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.551 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.551 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.551 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.551 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.551 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.551 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.551 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.552 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.552 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.552 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.552 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.552 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.552 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.552 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.553 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.553 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.553 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.553 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.553 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.553 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.553 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.554 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.554 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.554 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.554 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.554 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.554 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.554 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.554 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.555 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.555 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.555 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.555 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.555 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.555 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.555 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.556 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.556 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.556 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.556 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.556 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.556 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.557 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.557 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.557 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.557 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.557 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.557 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.557 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.558 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.558 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.558 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.558 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.558 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.558 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.559 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.559 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.559 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.559 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.559 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.559 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.559 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.560 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.560 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.560 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.560 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.560 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "orchestrator", 2023-07-18T03:42:07.560 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.561 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.561 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.561 INFO:teuthology.orchestra.run.smithi040.stdout: "fail_fs": { 2023-07-18T03:42:07.561 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "fail_fs", 2023-07-18T03:42:07.561 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.561 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.561 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.562 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.562 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.562 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.562 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.562 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-18T03:42:07.562 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.562 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.562 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.564 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.564 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.564 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.564 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.564 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.564 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.564 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.565 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.565 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.565 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.565 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.565 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.565 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.565 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.566 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.566 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.566 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.566 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.566 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.566 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.566 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.567 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.567 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.567 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.567 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.567 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.567 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.567 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.567 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.568 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.568 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.568 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.568 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.568 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.568 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.568 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.569 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.569 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.569 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.569 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.569 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.569 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.569 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.570 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.570 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.570 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.570 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.570 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.570 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.571 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.571 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.571 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.571 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.571 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.571 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.571 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.571 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.572 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.572 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.572 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.572 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.572 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.572 INFO:teuthology.orchestra.run.smithi040.stdout: "orchestrator": { 2023-07-18T03:42:07.573 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "orchestrator", 2023-07-18T03:42:07.573 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.573 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.573 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.573 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.573 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.573 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.574 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.574 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm", 2023-07-18T03:42:07.574 INFO:teuthology.orchestra.run.smithi040.stdout: "rook", 2023-07-18T03:42:07.574 INFO:teuthology.orchestra.run.smithi040.stdout: "test_orchestrator" 2023-07-18T03:42:07.574 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.574 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Orchestrator backend", 2023-07-18T03:42:07.574 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.575 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.575 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.575 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.575 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.575 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "osd_perf_query", 2023-07-18T03:42:07.576 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.576 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.576 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.576 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.576 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.576 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.577 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.577 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.577 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.577 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.577 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.577 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.578 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.578 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.578 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.578 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.578 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.578 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.578 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.579 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.579 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.579 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.579 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.579 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.579 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.580 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.580 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.580 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.580 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.580 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.580 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.580 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.581 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.581 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.581 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.581 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.581 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.581 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.581 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.582 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.582 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.582 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.582 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.582 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.582 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.582 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.583 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.583 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.583 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.583 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.583 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.583 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.583 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.583 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.584 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.584 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.584 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.584 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.584 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.584 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.585 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.585 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.585 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.585 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.585 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.585 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.585 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.586 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.586 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.586 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.586 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.586 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.586 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.586 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.587 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.587 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "osd_support", 2023-07-18T03:42:07.587 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.587 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.587 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.587 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.587 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.588 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.588 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.588 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.588 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.588 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.588 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.588 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.589 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.589 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.589 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.589 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.589 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.589 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.589 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.590 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.590 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.590 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.590 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.590 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.590 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.590 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.591 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.591 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.591 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.591 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.591 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.591 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.591 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.592 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.592 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.592 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.592 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.592 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.592 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.593 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.593 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.593 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.593 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.593 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.593 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.593 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.594 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.594 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.594 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.594 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.594 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.594 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.594 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.595 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.595 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.595 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.595 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.595 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.595 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.596 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.596 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.596 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.596 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.596 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.596 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.596 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.597 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.597 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.597 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.597 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.597 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.597 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pg_autoscaler", 2023-07-18T03:42:07.598 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.598 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.598 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.598 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.598 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.598 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.598 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.599 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.599 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.599 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.599 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.599 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.599 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.599 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.600 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.600 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.600 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.600 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.600 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.600 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.600 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.601 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.601 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.601 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.601 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.601 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.601 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.602 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.602 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.602 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.602 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.602 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.602 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.602 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.602 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.603 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.603 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.603 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.603 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.603 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.603 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.604 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.604 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.604 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.604 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.604 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.604 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.604 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.605 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.605 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.605 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.605 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.605 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.605 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.605 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.606 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.606 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.606 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.606 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.606 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.606 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.607 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.607 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.607 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.607 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.607 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.607 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.607 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.607 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.608 INFO:teuthology.orchestra.run.smithi040.stdout: "noautoscale": { 2023-07-18T03:42:07.608 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "noautoscale", 2023-07-18T03:42:07.608 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.608 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.608 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.608 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.609 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.609 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.609 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.609 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "global autoscale flag", 2023-07-18T03:42:07.609 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-18T03:42:07.609 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.609 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.610 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2023-07-18T03:42:07.610 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2023-07-18T03:42:07.610 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.610 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.610 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.610 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2023-07-18T03:42:07.611 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.611 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.611 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.611 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.611 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.611 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.611 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.612 INFO:teuthology.orchestra.run.smithi040.stdout: "threshold": { 2023-07-18T03:42:07.612 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "threshold", 2023-07-18T03:42:07.612 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:07.612 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.612 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.612 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3.0", 2023-07-18T03:42:07.613 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1.0", 2023-07-18T03:42:07.613 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.613 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.613 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "scaling threshold", 2023-07-18T03:42:07.613 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-18T03:42:07.613 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.613 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.614 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.614 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.614 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "progress", 2023-07-18T03:42:07.614 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.614 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.615 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.615 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_pg_recovery_event": { 2023-07-18T03:42:07.615 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_pg_recovery_event", 2023-07-18T03:42:07.615 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.615 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.615 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.615 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.616 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.616 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.616 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.616 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-18T03:42:07.616 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.616 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.616 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.617 INFO:teuthology.orchestra.run.smithi040.stdout: "enabled": { 2023-07-18T03:42:07.617 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enabled", 2023-07-18T03:42:07.617 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.617 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.617 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.617 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.618 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.618 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.618 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.618 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.618 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.618 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.618 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.619 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.619 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.619 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.619 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.619 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.619 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.619 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.620 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.620 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.620 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.620 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.620 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.620 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.620 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.621 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.621 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.621 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.621 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.621 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.621 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.622 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.622 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.622 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.622 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.622 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.622 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.622 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.623 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.623 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.623 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.623 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.623 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.623 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.623 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.624 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.624 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.624 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.624 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.624 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.624 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.624 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.625 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.625 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.625 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.625 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.625 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.625 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.625 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.626 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.626 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.626 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.626 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.626 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.626 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.627 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.627 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.627 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.627 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.627 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.627 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.627 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.628 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.628 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.628 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.628 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.628 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.628 INFO:teuthology.orchestra.run.smithi040.stdout: "max_completed_events": { 2023-07-18T03:42:07.629 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_completed_events", 2023-07-18T03:42:07.629 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.629 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.629 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.629 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "50", 2023-07-18T03:42:07.629 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.629 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.629 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.630 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "number of past completed events to remember", 2023-07-18T03:42:07.630 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.630 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.630 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.630 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2023-07-18T03:42:07.630 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2023-07-18T03:42:07.631 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.631 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.631 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.631 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2023-07-18T03:42:07.631 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.631 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.631 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.632 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long the module is going to sleep", 2023-07-18T03:42:07.632 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.632 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.632 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.632 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.632 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.633 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.633 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus", 2023-07-18T03:42:07.633 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.633 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.633 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.633 INFO:teuthology.orchestra.run.smithi040.stdout: "cache": { 2023-07-18T03:42:07.633 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cache", 2023-07-18T03:42:07.633 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.634 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.634 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.634 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.634 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.634 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.634 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.634 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.635 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.635 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.635 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.635 INFO:teuthology.orchestra.run.smithi040.stdout: "exclude_perf_counters": { 2023-07-18T03:42:07.635 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "exclude_perf_counters", 2023-07-18T03:42:07.635 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.636 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.636 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.636 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.636 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.636 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.636 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.636 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-18T03:42:07.637 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-18T03:42:07.637 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.637 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.637 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.637 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.638 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.638 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.638 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.638 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.638 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.638 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.638 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.638 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_stats_pools": { 2023-07-18T03:42:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_stats_pools", 2023-07-18T03:42:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-18T03:42:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-18T03:42:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "300", 2023-07-18T03:42:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "scrape_interval": { 2023-07-18T03:42:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "scrape_interval", 2023-07-18T03:42:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15.0", 2023-07-18T03:42:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2023-07-18T03:42:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2023-07-18T03:42:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "::", 2023-07-18T03:42:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-18T03:42:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2023-07-18T03:42:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2023-07-18T03:42:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "9283", 2023-07-18T03:42:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-18T03:42:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "stale_cache_strategy": { 2023-07-18T03:42:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "stale_cache_strategy", 2023-07-18T03:42:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "log", 2023-07-18T03:42:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_behaviour": { 2023-07-18T03:42:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_behaviour", 2023-07-18T03:42:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "default", 2023-07-18T03:42:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "default", 2023-07-18T03:42:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "error" 2023-07-18T03:42:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_error_status_code": { 2023-07-18T03:42:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_error_status_code", 2023-07-18T03:42:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "500", 2023-07-18T03:42:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "400", 2023-07-18T03:42:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "599", 2023-07-18T03:42:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_support", 2023-07-18T03:42:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: "max_concurrent_snap_create": { 2023-07-18T03:42:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_concurrent_snap_create", 2023-07-18T03:42:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: "mirror_snapshot_schedule": { 2023-07-18T03:42:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mirror_snapshot_schedule", 2023-07-18T03:42:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "trash_purge_schedule": { 2023-07-18T03:42:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "trash_purge_schedule", 2023-07-18T03:42:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "restful", 2023-07-18T03:42:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "enable_auth": { 2023-07-18T03:42:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enable_auth", 2023-07-18T03:42:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "key_file": { 2023-07-18T03:42:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "key_file", 2023-07-18T03:42:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2023-07-18T03:42:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2023-07-18T03:42:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2023-07-18T03:42:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2023-07-18T03:42:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rgw", 2023-07-18T03:42:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rook", 2023-07-18T03:42:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: "drive_group_interval": { 2023-07-18T03:42:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "drive_group_interval", 2023-07-18T03:42:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "300.0", 2023-07-18T03:42:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-18T03:42:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: "storage_class": { 2023-07-18T03:42:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "storage_class", 2023-07-18T03:42:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "local", 2023-07-18T03:42:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-18T03:42:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "selftest", 2023-07-18T03:42:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: "roption1": { 2023-07-18T03:42:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "roption1", 2023-07-18T03:42:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: "roption2": { 2023-07-18T03:42:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "roption2", 2023-07-18T03:42:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.738 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.738 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.738 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "xyz", 2023-07-18T03:42:07.738 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.738 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.738 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.738 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.739 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.739 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.739 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.739 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.739 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption1": { 2023-07-18T03:42:07.739 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption1", 2023-07-18T03:42:07.739 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.739 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.740 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.740 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.740 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.740 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.740 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.740 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.740 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption2": { 2023-07-18T03:42:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption2", 2023-07-18T03:42:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption3": { 2023-07-18T03:42:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption3", 2023-07-18T03:42:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption4": { 2023-07-18T03:42:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption4", 2023-07-18T03:42:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption5": { 2023-07-18T03:42:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption5", 2023-07-18T03:42:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption6": { 2023-07-18T03:42:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption6", 2023-07-18T03:42:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.750 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.750 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.750 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.750 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.750 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.750 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.750 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.751 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.751 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.751 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption7": { 2023-07-18T03:42:07.751 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption7", 2023-07-18T03:42:07.751 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.751 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.752 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.752 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.752 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2023-07-18T03:42:07.752 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "42", 2023-07-18T03:42:07.752 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.752 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.752 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.753 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.753 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.753 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.753 INFO:teuthology.orchestra.run.smithi040.stdout: "testkey": { 2023-07-18T03:42:07.753 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testkey", 2023-07-18T03:42:07.753 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.753 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.754 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.754 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.754 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.754 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.754 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.754 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.754 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.754 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.755 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.755 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.755 INFO:teuthology.orchestra.run.smithi040.stdout: "testlkey": { 2023-07-18T03:42:07.755 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testlkey", 2023-07-18T03:42:07.755 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.755 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.755 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.756 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.756 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.756 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.756 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.756 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.756 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.756 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.757 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.757 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.757 INFO:teuthology.orchestra.run.smithi040.stdout: "testnewline": { 2023-07-18T03:42:07.757 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testnewline", 2023-07-18T03:42:07.757 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.757 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.757 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.758 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.758 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.758 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.758 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.758 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.758 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.758 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.759 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.759 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.759 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.759 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.759 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.759 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "snap_schedule", 2023-07-18T03:42:07.759 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.759 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.760 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.760 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_m_granularity": { 2023-07-18T03:42:07.760 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_m_granularity", 2023-07-18T03:42:07.760 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.760 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.760 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.760 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.761 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.761 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.761 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.761 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow minute scheduled snapshots", 2023-07-18T03:42:07.761 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.761 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.761 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.762 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.762 INFO:teuthology.orchestra.run.smithi040.stdout: "dump_on_update": { 2023-07-18T03:42:07.762 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "dump_on_update", 2023-07-18T03:42:07.762 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.762 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.762 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.762 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.763 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.763 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.763 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.763 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "dump database to debug log on update", 2023-07-18T03:42:07.763 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.763 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.763 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.764 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.764 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.764 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.764 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.764 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.764 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.764 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.765 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.765 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.765 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.765 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.765 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.765 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.765 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.765 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.766 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.766 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.766 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.766 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.766 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.766 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.766 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.767 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.767 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.767 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.767 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.767 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.767 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.767 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.768 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.768 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.768 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.768 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.768 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.768 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.768 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.769 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.769 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.769 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.769 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.769 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.769 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.769 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.770 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.770 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.770 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.770 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.770 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.770 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.770 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.771 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.771 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.771 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.771 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.771 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.771 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.771 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.772 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.772 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.772 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.772 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.772 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.772 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.772 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.773 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.773 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.773 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.773 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.773 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.773 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.774 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.774 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.774 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.774 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.774 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "stats", 2023-07-18T03:42:07.774 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.774 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.774 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.775 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.775 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.775 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.775 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.775 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.775 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.775 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.776 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.776 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.776 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.776 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.776 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.776 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.776 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.777 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.777 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.777 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.777 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.777 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.777 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.777 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.778 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.778 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.778 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.778 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.778 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.778 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.778 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.779 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.779 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.779 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.779 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.779 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.779 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.779 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.779 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.780 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.780 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.780 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.780 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.780 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.780 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.780 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.781 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.781 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.781 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.781 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.781 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.781 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.781 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.782 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.782 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.782 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.782 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.782 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.782 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.783 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.783 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.783 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.783 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.783 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.783 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.783 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.784 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.784 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.784 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.784 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.784 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.784 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.784 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.785 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.785 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.785 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "status", 2023-07-18T03:42:07.785 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.785 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.785 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.785 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.785 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.786 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.786 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.786 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.786 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.786 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.786 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.786 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.787 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.787 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.787 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.787 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.787 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.787 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.787 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.788 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.788 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.788 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.788 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.788 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.788 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.789 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.789 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.789 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.789 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.789 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.789 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.789 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.789 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.790 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.790 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.790 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.790 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.790 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.790 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.791 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.791 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.791 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.791 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.791 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.791 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.791 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.792 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.792 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.792 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.792 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.792 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.792 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.792 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.793 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.793 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.793 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.793 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.793 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.793 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.794 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.794 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.794 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.794 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.794 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.794 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.794 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.795 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.795 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.795 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.795 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.795 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.795 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.795 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "telegraf", 2023-07-18T03:42:07.796 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.796 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.796 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.796 INFO:teuthology.orchestra.run.smithi040.stdout: "address": { 2023-07-18T03:42:07.796 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "address", 2023-07-18T03:42:07.796 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.796 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.797 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.797 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-18T03:42:07.797 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.797 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.797 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.797 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.797 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.798 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.798 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.798 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.798 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2023-07-18T03:42:07.798 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2023-07-18T03:42:07.798 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.798 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.799 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.799 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15", 2023-07-18T03:42:07.799 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.799 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.799 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.799 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.799 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.800 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.800 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.800 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.800 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.800 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.800 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.800 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.801 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.801 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.801 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.801 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.801 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.801 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.801 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.802 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.802 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.802 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.802 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.802 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.802 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.802 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.803 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.803 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.803 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.803 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.803 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.803 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.803 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.804 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.804 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.804 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.804 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.804 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.804 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.804 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.805 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.805 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.805 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.805 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.805 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.805 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.805 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.805 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.806 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.806 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.806 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.806 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.806 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.806 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.806 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "telemetry", 2023-07-18T03:42:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.811 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.811 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.811 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_basic": { 2023-07-18T03:42:07.811 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_basic", 2023-07-18T03:42:07.811 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.811 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.811 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.811 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-18T03:42:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.813 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_crash": { 2023-07-18T03:42:07.813 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_crash", 2023-07-18T03:42:07.813 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.813 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.813 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.813 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-18T03:42:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_device": { 2023-07-18T03:42:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_device", 2023-07-18T03:42:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-18T03:42:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_ident": { 2023-07-18T03:42:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_ident", 2023-07-18T03:42:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-18T03:42:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.819 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.819 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_perf": { 2023-07-18T03:42:07.819 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_perf", 2023-07-18T03:42:07.819 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.819 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.819 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.819 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.820 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.820 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.820 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.820 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-18T03:42:07.820 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.820 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.820 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.821 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.821 INFO:teuthology.orchestra.run.smithi040.stdout: "contact": { 2023-07-18T03:42:07.821 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "contact", 2023-07-18T03:42:07.821 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.821 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.821 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.821 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.821 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.822 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.822 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.822 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.822 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.822 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.822 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.822 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.823 INFO:teuthology.orchestra.run.smithi040.stdout: "description": { 2023-07-18T03:42:07.823 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "description", 2023-07-18T03:42:07.823 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.823 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.823 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.823 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.823 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.824 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.824 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.824 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.824 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.824 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.824 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.824 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.825 INFO:teuthology.orchestra.run.smithi040.stdout: "device_url": { 2023-07-18T03:42:07.825 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_url", 2023-07-18T03:42:07.825 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.825 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.825 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.825 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-18T03:42:07.825 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.826 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.826 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.826 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.826 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.826 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.826 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.826 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.827 INFO:teuthology.orchestra.run.smithi040.stdout: "enabled": { 2023-07-18T03:42:07.827 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enabled", 2023-07-18T03:42:07.827 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.827 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.827 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.827 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.827 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.828 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.828 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.828 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.828 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.828 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.828 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.828 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.829 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2023-07-18T03:42:07.829 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2023-07-18T03:42:07.829 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.829 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.829 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.829 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "24", 2023-07-18T03:42:07.829 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "8", 2023-07-18T03:42:07.830 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.830 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.830 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.830 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.830 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.830 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.830 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.831 INFO:teuthology.orchestra.run.smithi040.stdout: "last_opt_revision": { 2023-07-18T03:42:07.831 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "last_opt_revision", 2023-07-18T03:42:07.831 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.831 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.831 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.831 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1", 2023-07-18T03:42:07.831 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.831 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.832 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.832 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.832 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.832 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.832 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.832 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.832 INFO:teuthology.orchestra.run.smithi040.stdout: "leaderboard": { 2023-07-18T03:42:07.833 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "leaderboard", 2023-07-18T03:42:07.833 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.833 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.833 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.833 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.833 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.833 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.834 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.834 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.834 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.834 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.834 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.834 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.834 INFO:teuthology.orchestra.run.smithi040.stdout: "leaderboard_description": { 2023-07-18T03:42:07.835 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "leaderboard_description", 2023-07-18T03:42:07.835 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.835 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.835 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.835 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.835 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.835 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.835 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.836 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.836 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.836 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.836 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.836 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.836 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.836 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.837 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.837 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.837 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.837 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.837 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.837 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.837 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.838 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.838 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.838 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.838 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.838 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.838 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.838 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.839 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.839 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.839 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.839 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.839 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.839 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.839 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.840 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.840 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.840 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.840 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.840 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.840 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.840 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.841 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.841 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.841 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.841 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.841 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.841 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.841 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.841 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.842 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.842 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.842 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.842 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.842 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.842 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.842 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.843 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.843 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.843 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.843 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.843 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.843 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.843 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.844 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.844 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.844 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.844 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.844 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.844 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.845 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.845 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.845 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.845 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.845 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.845 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.845 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.845 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.846 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.846 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.846 INFO:teuthology.orchestra.run.smithi040.stdout: "organization": { 2023-07-18T03:42:07.846 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "organization", 2023-07-18T03:42:07.846 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.846 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.847 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.847 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.847 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.847 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.847 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.847 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.847 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.848 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.848 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.848 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.848 INFO:teuthology.orchestra.run.smithi040.stdout: "proxy": { 2023-07-18T03:42:07.848 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "proxy", 2023-07-18T03:42:07.848 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.848 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.849 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.849 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.849 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.849 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.849 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.849 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.849 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.850 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.850 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.850 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.850 INFO:teuthology.orchestra.run.smithi040.stdout: "url": { 2023-07-18T03:42:07.850 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "url", 2023-07-18T03:42:07.850 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.850 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.850 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.851 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-18T03:42:07.851 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.851 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.851 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.851 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.851 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.851 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.852 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.852 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.852 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.852 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.852 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.852 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "test_orchestrator", 2023-07-18T03:42:07.852 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.853 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.853 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.853 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.853 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.853 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.853 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.853 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.854 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.854 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.854 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.854 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.854 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.854 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.854 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.855 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.855 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.855 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.855 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.855 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.855 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.855 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.856 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.856 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.856 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.856 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.856 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.856 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.857 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.857 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.857 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.857 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.857 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.857 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.857 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.858 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.858 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.858 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.858 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.858 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.858 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.858 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.859 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.859 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.859 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.859 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.859 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.859 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.859 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.859 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.860 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.860 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.860 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.860 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.860 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.860 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.860 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.861 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.861 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.861 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.861 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.861 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.861 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.861 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.862 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.862 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.862 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.862 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.862 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.862 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.862 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.863 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.863 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.863 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "volumes", 2023-07-18T03:42:07.863 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.863 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.864 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.864 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.864 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.864 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.864 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.864 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.864 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.864 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.865 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.865 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.865 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.865 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.865 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.865 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.865 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.866 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.866 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.866 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.866 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.866 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.866 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.866 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.867 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.867 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.867 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.867 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.867 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.867 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.867 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.867 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.868 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.868 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.868 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.868 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.868 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.868 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.868 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.869 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.869 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.869 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.869 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.869 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.869 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.869 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.870 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.870 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.870 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.870 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.870 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.870 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.870 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.871 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.871 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.871 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.871 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.871 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.871 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.871 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.871 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.872 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.872 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.872 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.872 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.872 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.872 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.872 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.873 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.873 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.873 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.873 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.873 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.873 INFO:teuthology.orchestra.run.smithi040.stdout: "max_concurrent_clones": { 2023-07-18T03:42:07.873 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_concurrent_clones", 2023-07-18T03:42:07.874 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.874 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.874 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.874 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "4", 2023-07-18T03:42:07.874 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.874 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.874 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.875 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-18T03:42:07.875 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.875 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.875 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.875 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.875 INFO:teuthology.orchestra.run.smithi040.stdout: "snapshot_clone_delay": { 2023-07-18T03:42:07.875 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "snapshot_clone_delay", 2023-07-18T03:42:07.875 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.876 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.876 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.876 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2023-07-18T03:42:07.876 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.876 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.876 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.876 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-18T03:42:07.877 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.877 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.877 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.877 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.877 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.877 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.877 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.878 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix", 2023-07-18T03:42:07.878 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.878 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.878 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.878 INFO:teuthology.orchestra.run.smithi040.stdout: "discovery_interval": { 2023-07-18T03:42:07.878 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "discovery_interval", 2023-07-18T03:42:07.878 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2023-07-18T03:42:07.879 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.879 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.879 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "100", 2023-07-18T03:42:07.879 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.879 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.879 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.879 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.880 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.880 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.880 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.880 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.880 INFO:teuthology.orchestra.run.smithi040.stdout: "identifier": { 2023-07-18T03:42:07.880 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "identifier", 2023-07-18T03:42:07.880 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.881 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.881 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.881 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.881 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.881 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.881 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.881 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.882 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.882 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.882 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.882 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2023-07-18T03:42:07.882 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2023-07-18T03:42:07.882 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.883 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.883 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.883 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2023-07-18T03:42:07.883 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.883 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.883 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.883 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.884 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.884 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.884 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.884 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.884 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.884 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.884 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.885 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.885 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.885 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.885 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.885 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.885 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.885 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.886 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.886 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.886 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.886 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.886 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.886 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.886 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.887 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.887 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.887 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.887 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.887 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.887 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.887 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.887 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.888 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.888 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.888 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.888 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.888 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.888 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.889 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.889 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.889 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.889 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.889 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.889 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.889 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.890 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.890 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.890 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.890 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.890 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.890 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.890 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.890 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.891 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.891 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.891 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.891 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.891 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.891 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.891 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.892 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.892 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.892 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.892 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.892 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.892 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.892 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.893 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.893 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.893 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.893 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.893 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.893 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.893 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.894 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.894 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.894 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.894 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_host": { 2023-07-18T03:42:07.894 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_host", 2023-07-18T03:42:07.894 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.894 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.895 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.895 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.895 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.895 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.895 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.895 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.895 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.895 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.896 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.896 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.896 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_port": { 2023-07-18T03:42:07.896 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_port", 2023-07-18T03:42:07.896 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.896 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.896 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.897 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10051", 2023-07-18T03:42:07.897 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.897 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.897 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.897 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.897 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.897 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.898 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.898 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.898 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_sender": { 2023-07-18T03:42:07.898 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_sender", 2023-07-18T03:42:07.898 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.898 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.898 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.899 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-18T03:42:07.899 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.899 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.899 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.899 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.899 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.899 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.900 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.900 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.900 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.900 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.900 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-07-18T03:42:07.900 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.900 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.901 INFO:teuthology.orchestra.run.smithi040.stdout: "gid": 24110, 2023-07-18T03:42:07.901 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "y", 2023-07-18T03:42:07.901 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_features": 4540138322906710015, 2023-07-18T03:42:07.901 INFO:teuthology.orchestra.run.smithi040.stdout: "available_modules": [ 2023-07-18T03:42:07.901 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.901 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "alerts", 2023-07-18T03:42:07.901 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.901 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.902 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.902 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2023-07-18T03:42:07.902 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2023-07-18T03:42:07.902 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.902 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.902 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.902 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2023-07-18T03:42:07.903 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.903 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.903 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.903 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How frequently to reexamine health status", 2023-07-18T03:42:07.903 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.903 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.903 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.904 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.904 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.904 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.904 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.904 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.904 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.905 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.905 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.905 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.905 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.905 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.905 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.905 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.906 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.906 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.906 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.906 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.906 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.906 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.906 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.906 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.907 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.907 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.907 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.907 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.907 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.907 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.907 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.908 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.908 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.908 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.908 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.908 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.908 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.909 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.909 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.909 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.909 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.909 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.909 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.909 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.910 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.910 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.910 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.910 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.910 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.910 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.910 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.910 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.911 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.911 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.911 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.911 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.911 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.911 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.911 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.912 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.912 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.912 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.912 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.912 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.912 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.912 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.913 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.913 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.913 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.913 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.913 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.913 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.913 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.914 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_destination": { 2023-07-18T03:42:07.914 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_destination", 2023-07-18T03:42:07.914 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.914 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.914 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.914 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.914 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.915 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.915 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.915 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Email address to send alerts to", 2023-07-18T03:42:07.915 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.915 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.915 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.915 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.915 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_from_name": { 2023-07-18T03:42:07.916 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_from_name", 2023-07-18T03:42:07.916 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.916 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.916 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.916 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "Ceph", 2023-07-18T03:42:07.916 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.916 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.917 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.917 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Email From: name", 2023-07-18T03:42:07.917 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.917 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.917 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.917 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.917 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_host": { 2023-07-18T03:42:07.918 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_host", 2023-07-18T03:42:07.918 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.918 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.918 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.918 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.918 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.918 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.919 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.919 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP server", 2023-07-18T03:42:07.919 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.919 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.919 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.919 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.919 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_password": { 2023-07-18T03:42:07.920 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_password", 2023-07-18T03:42:07.920 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.920 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.920 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.920 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.920 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.920 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.921 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.921 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Password to authenticate with", 2023-07-18T03:42:07.921 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.921 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.921 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.921 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.921 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_port": { 2023-07-18T03:42:07.921 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_port", 2023-07-18T03:42:07.922 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.922 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.922 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.922 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "465", 2023-07-18T03:42:07.922 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.922 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.922 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.923 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP port", 2023-07-18T03:42:07.923 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.923 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.923 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.923 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.923 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_sender": { 2023-07-18T03:42:07.923 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_sender", 2023-07-18T03:42:07.924 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.924 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.924 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.924 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.924 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.924 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.924 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.925 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP envelope sender", 2023-07-18T03:42:07.925 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.925 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.925 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.925 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.925 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_ssl": { 2023-07-18T03:42:07.925 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_ssl", 2023-07-18T03:42:07.926 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.926 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.926 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.926 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.926 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.926 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.926 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.926 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-18T03:42:07.927 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.927 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.927 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.927 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.927 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_user": { 2023-07-18T03:42:07.927 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_user", 2023-07-18T03:42:07.927 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.928 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.928 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.928 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.928 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.928 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.928 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.928 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "User to authenticate as", 2023-07-18T03:42:07.929 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.929 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.929 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.929 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.929 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.929 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.929 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.930 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "balancer", 2023-07-18T03:42:07.930 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.930 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.930 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.930 INFO:teuthology.orchestra.run.smithi040.stdout: "active": { 2023-07-18T03:42:07.930 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "active", 2023-07-18T03:42:07.930 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.930 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.931 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.931 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.931 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.931 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.931 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.931 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "automatically balance PGs across cluster", 2023-07-18T03:42:07.931 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.932 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.932 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.932 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.932 INFO:teuthology.orchestra.run.smithi040.stdout: "begin_time": { 2023-07-18T03:42:07.932 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "begin_time", 2023-07-18T03:42:07.932 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.932 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.933 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.933 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0000", 2023-07-18T03:42:07.933 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.933 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.933 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.933 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "beginning time of day to automatically balance", 2023-07-18T03:42:07.933 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-18T03:42:07.934 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.934 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.934 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.934 INFO:teuthology.orchestra.run.smithi040.stdout: "begin_weekday": { 2023-07-18T03:42:07.934 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "begin_weekday", 2023-07-18T03:42:07.934 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2023-07-18T03:42:07.934 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.935 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.935 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2023-07-18T03:42:07.935 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0", 2023-07-18T03:42:07.935 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "6", 2023-07-18T03:42:07.935 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.935 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-18T03:42:07.935 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-18T03:42:07.936 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.936 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.936 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.936 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_max_iterations": { 2023-07-18T03:42:07.936 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_max_iterations", 2023-07-18T03:42:07.936 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2023-07-18T03:42:07.936 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.937 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.937 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "25", 2023-07-18T03:42:07.937 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2023-07-18T03:42:07.937 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "250", 2023-07-18T03:42:07.937 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.937 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-18T03:42:07.937 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.938 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.938 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.938 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.938 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_metrics": { 2023-07-18T03:42:07.938 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_metrics", 2023-07-18T03:42:07.938 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.938 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.938 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.939 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "pgs,objects,bytes", 2023-07-18T03:42:07.939 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.939 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.939 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.939 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-18T03:42:07.939 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-18T03:42:07.939 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.940 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.940 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.940 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_step": { 2023-07-18T03:42:07.940 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_step", 2023-07-18T03:42:07.940 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:07.940 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.940 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.941 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0.5", 2023-07-18T03:42:07.941 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0.001", 2023-07-18T03:42:07.941 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "0.999", 2023-07-18T03:42:07.941 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.941 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "aggressiveness of optimization", 2023-07-18T03:42:07.941 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-18T03:42:07.941 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.942 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.942 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.942 INFO:teuthology.orchestra.run.smithi040.stdout: "end_time": { 2023-07-18T03:42:07.942 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "end_time", 2023-07-18T03:42:07.942 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.942 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.942 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.943 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "2359", 2023-07-18T03:42:07.943 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.943 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.943 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.943 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "ending time of day to automatically balance", 2023-07-18T03:42:07.943 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-18T03:42:07.943 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.943 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.944 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.944 INFO:teuthology.orchestra.run.smithi040.stdout: "end_weekday": { 2023-07-18T03:42:07.944 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "end_weekday", 2023-07-18T03:42:07.944 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2023-07-18T03:42:07.944 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.944 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.944 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2023-07-18T03:42:07.945 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0", 2023-07-18T03:42:07.945 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "6", 2023-07-18T03:42:07.945 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.945 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-18T03:42:07.945 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-18T03:42:07.945 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.945 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.946 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.946 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:07.946 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:07.946 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.946 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.946 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.946 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.947 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.947 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.947 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.947 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.947 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.947 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.947 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.947 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.948 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.948 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.948 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.948 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.948 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.948 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.948 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.949 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:07.949 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:07.949 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.949 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.949 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.949 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.949 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.950 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.950 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.950 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.950 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.950 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.950 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.950 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.951 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:07.951 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:07.951 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.951 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.951 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.951 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:07.951 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.951 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.952 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.952 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:07.952 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:07.952 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:07.952 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:07.952 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:07.952 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:07.953 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.953 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.953 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.953 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.953 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.953 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.953 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:07.954 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:07.954 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.954 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.954 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.954 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.954 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.954 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.955 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.955 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:07.955 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.955 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.955 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.955 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.955 INFO:teuthology.orchestra.run.smithi040.stdout: "min_score": { 2023-07-18T03:42:07.956 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "min_score", 2023-07-18T03:42:07.956 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:07.956 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.956 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.956 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2023-07-18T03:42:07.956 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.956 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.957 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.957 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-18T03:42:07.957 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.957 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.957 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.957 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.957 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": { 2023-07-18T03:42:07.958 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mode", 2023-07-18T03:42:07.958 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.958 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.958 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.958 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "upmap", 2023-07-18T03:42:07.958 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.958 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.959 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:07.959 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-compat", 2023-07-18T03:42:07.959 INFO:teuthology.orchestra.run.smithi040.stdout: "none", 2023-07-18T03:42:07.959 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap" 2023-07-18T03:42:07.959 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:07.959 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Balancer mode", 2023-07-18T03:42:07.959 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.960 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.960 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.960 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.960 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_ids": { 2023-07-18T03:42:07.960 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pool_ids", 2023-07-18T03:42:07.960 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.960 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.961 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.961 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:07.961 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.961 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.961 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.961 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-18T03:42:07.961 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.962 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.962 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.962 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.962 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2023-07-18T03:42:07.962 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2023-07-18T03:42:07.962 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.962 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.963 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.963 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2023-07-18T03:42:07.963 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.963 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.963 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.963 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-18T03:42:07.963 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.964 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.964 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.964 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.964 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap_max_deviation": { 2023-07-18T03:42:07.964 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "upmap_max_deviation", 2023-07-18T03:42:07.964 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.964 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.965 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.965 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2023-07-18T03:42:07.965 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2023-07-18T03:42:07.965 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.965 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.965 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-18T03:42:07.965 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-18T03:42:07.966 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.966 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.966 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.966 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap_max_optimizations": { 2023-07-18T03:42:07.966 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "upmap_max_optimizations", 2023-07-18T03:42:07.966 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2023-07-18T03:42:07.966 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.967 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.967 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:07.967 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.967 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.967 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.967 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-18T03:42:07.967 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.967 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.968 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.968 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.968 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:07.968 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.968 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:07.968 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cephadm", 2023-07-18T03:42:07.968 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:07.969 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:07.969 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:07.969 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_down_multiplier": { 2023-07-18T03:42:07.969 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_down_multiplier", 2023-07-18T03:42:07.969 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:07.969 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.969 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.970 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3.0", 2023-07-18T03:42:07.970 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.970 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.970 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.970 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-18T03:42:07.970 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.970 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.971 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.971 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.971 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_refresh_rate": { 2023-07-18T03:42:07.971 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_refresh_rate", 2023-07-18T03:42:07.971 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.971 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.971 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.972 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "20", 2023-07-18T03:42:07.972 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.972 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.972 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.972 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-18T03:42:07.972 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.972 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.973 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.973 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.973 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_starting_port": { 2023-07-18T03:42:07.973 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_starting_port", 2023-07-18T03:42:07.973 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:07.973 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.973 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.973 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "4721", 2023-07-18T03:42:07.974 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.974 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.974 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.974 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-18T03:42:07.974 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.974 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.974 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.975 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.975 INFO:teuthology.orchestra.run.smithi040.stdout: "alertmanager_web_password": { 2023-07-18T03:42:07.975 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "alertmanager_web_password", 2023-07-18T03:42:07.975 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.975 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.975 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.975 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:07.976 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.976 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.976 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.976 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Alertmanager web password", 2023-07-18T03:42:07.976 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.976 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.976 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.977 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.977 INFO:teuthology.orchestra.run.smithi040.stdout: "alertmanager_web_user": { 2023-07-18T03:42:07.977 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "alertmanager_web_user", 2023-07-18T03:42:07.977 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.977 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.977 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.977 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:07.977 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.978 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.978 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.978 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Alertmanager web user", 2023-07-18T03:42:07.978 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.978 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.978 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.978 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.979 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_ptrace": { 2023-07-18T03:42:07.979 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_ptrace", 2023-07-18T03:42:07.979 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.979 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.979 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.979 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.979 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.980 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.980 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.980 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-18T03:42:07.980 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-18T03:42:07.980 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.981 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.981 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.981 INFO:teuthology.orchestra.run.smithi040.stdout: "autotune_interval": { 2023-07-18T03:42:07.981 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "autotune_interval", 2023-07-18T03:42:07.981 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:07.981 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.981 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.981 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2023-07-18T03:42:07.982 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.982 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.982 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.982 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-18T03:42:07.982 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.982 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.982 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.983 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.983 INFO:teuthology.orchestra.run.smithi040.stdout: "autotune_memory_target_ratio": { 2023-07-18T03:42:07.983 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "autotune_memory_target_ratio", 2023-07-18T03:42:07.983 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:07.983 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.983 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.984 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0.7", 2023-07-18T03:42:07.984 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.984 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.984 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.984 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-18T03:42:07.984 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.984 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.985 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.985 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.985 INFO:teuthology.orchestra.run.smithi040.stdout: "cgroups_split": { 2023-07-18T03:42:07.985 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cgroups_split", 2023-07-18T03:42:07.985 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.986 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.986 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.986 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.986 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.986 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.986 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.987 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-18T03:42:07.987 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.987 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.988 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.988 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.988 INFO:teuthology.orchestra.run.smithi040.stdout: "config_checks_enabled": { 2023-07-18T03:42:07.989 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_checks_enabled", 2023-07-18T03:42:07.989 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.989 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.989 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.990 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:07.990 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.990 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.991 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.991 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-18T03:42:07.991 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.991 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.991 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.992 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.992 INFO:teuthology.orchestra.run.smithi040.stdout: "config_dashboard": { 2023-07-18T03:42:07.992 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_dashboard", 2023-07-18T03:42:07.992 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:07.993 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.993 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.993 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:07.993 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.994 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.994 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.994 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-18T03:42:07.994 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.995 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.995 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.995 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.995 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_alertmanager": { 2023-07-18T03:42:07.995 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_alertmanager", 2023-07-18T03:42:07.995 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.996 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.996 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:07.996 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-18T03:42:07.996 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.996 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.996 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.996 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2023-07-18T03:42:07.997 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.997 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.997 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.997 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.997 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_base": { 2023-07-18T03:42:07.997 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_base", 2023-07-18T03:42:07.997 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.998 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:07.998 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:07.998 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-18T03:42:07.998 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:07.998 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:07.998 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:07.998 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Container image name, without the tag", 2023-07-18T03:42:07.998 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:07.999 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:07.999 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:07.999 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:07.999 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_elasticsearch": { 2023-07-18T03:42:07.999 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_elasticsearch", 2023-07-18T03:42:07.999 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:07.999 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.000 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.000 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-18T03:42:08.000 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.000 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.000 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.000 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "elasticsearch container image", 2023-07-18T03:42:08.000 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.000 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.001 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.001 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.001 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_grafana": { 2023-07-18T03:42:08.001 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_grafana", 2023-07-18T03:42:08.001 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.001 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.001 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.002 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-18T03:42:08.002 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.002 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.002 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.002 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2023-07-18T03:42:08.002 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.002 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.002 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.003 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.003 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_haproxy": { 2023-07-18T03:42:08.003 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_haproxy", 2023-07-18T03:42:08.003 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.003 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.003 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.003 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-18T03:42:08.003 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.004 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.004 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.004 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "HAproxy container image", 2023-07-18T03:42:08.004 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.004 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.004 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.004 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.005 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_agent": { 2023-07-18T03:42:08.005 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_agent", 2023-07-18T03:42:08.005 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.005 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.005 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.005 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-18T03:42:08.005 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.005 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.006 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.006 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger agent container image", 2023-07-18T03:42:08.006 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.006 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.006 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.006 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.006 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_collector": { 2023-07-18T03:42:08.006 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_collector", 2023-07-18T03:42:08.007 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.007 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.007 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.007 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-18T03:42:08.007 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.007 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.007 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.008 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger collector container image", 2023-07-18T03:42:08.008 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.008 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.008 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.008 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.008 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_query": { 2023-07-18T03:42:08.008 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_query", 2023-07-18T03:42:08.008 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.009 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.009 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.009 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-18T03:42:08.009 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.009 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.009 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.009 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger query container image", 2023-07-18T03:42:08.010 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.010 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.010 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.011 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_keepalived": { 2023-07-18T03:42:08.011 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_keepalived", 2023-07-18T03:42:08.011 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.012 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.012 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.012 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-18T03:42:08.012 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.013 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.013 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.013 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Keepalived container image", 2023-07-18T03:42:08.013 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.013 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.014 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.014 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_loki": { 2023-07-18T03:42:08.014 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_loki", 2023-07-18T03:42:08.014 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.015 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.015 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.015 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-18T03:42:08.015 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.016 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.016 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.016 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Loki container image", 2023-07-18T03:42:08.016 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.017 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.017 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.017 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.017 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_node_exporter": { 2023-07-18T03:42:08.017 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_node_exporter", 2023-07-18T03:42:08.018 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.018 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.018 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.018 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-18T03:42:08.019 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.019 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.019 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.019 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2023-07-18T03:42:08.019 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.020 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.020 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.020 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_prometheus": { 2023-07-18T03:42:08.021 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_prometheus", 2023-07-18T03:42:08.021 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.021 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.021 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.021 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-18T03:42:08.022 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.022 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.022 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.022 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2023-07-18T03:42:08.023 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.023 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.023 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.023 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.023 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_promtail": { 2023-07-18T03:42:08.024 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_promtail", 2023-07-18T03:42:08.024 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.024 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.024 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.025 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-18T03:42:08.025 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.025 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.025 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.025 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Promtail container image", 2023-07-18T03:42:08.025 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.025 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.026 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.026 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.026 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_snmp_gateway": { 2023-07-18T03:42:08.026 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_snmp_gateway", 2023-07-18T03:42:08.027 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.027 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.027 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.027 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-18T03:42:08.027 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.027 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.027 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.027 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SNMP Gateway container image", 2023-07-18T03:42:08.028 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.028 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.028 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.028 INFO:teuthology.orchestra.run.smithi040.stdout: "container_init": { 2023-07-18T03:42:08.028 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_init", 2023-07-18T03:42:08.028 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.029 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.029 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.029 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.029 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.029 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.029 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.029 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Run podman/docker with `--init`", 2023-07-18T03:42:08.030 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.030 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.030 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.030 INFO:teuthology.orchestra.run.smithi040.stdout: "daemon_cache_timeout": { 2023-07-18T03:42:08.030 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "daemon_cache_timeout", 2023-07-18T03:42:08.030 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.031 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.031 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.031 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2023-07-18T03:42:08.031 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.031 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.031 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.031 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-18T03:42:08.032 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.032 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.032 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.032 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.032 INFO:teuthology.orchestra.run.smithi040.stdout: "default_cephadm_command_timeout": { 2023-07-18T03:42:08.032 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "default_cephadm_command_timeout", 2023-07-18T03:42:08.033 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.033 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.033 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.034 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "900", 2023-07-18T03:42:08.034 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.034 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.034 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.035 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-18T03:42:08.035 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.035 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.035 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.035 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.035 INFO:teuthology.orchestra.run.smithi040.stdout: "default_registry": { 2023-07-18T03:42:08.035 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "default_registry", 2023-07-18T03:42:08.035 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.036 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.036 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.036 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io", 2023-07-18T03:42:08.036 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.036 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.036 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.036 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-18T03:42:08.037 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.037 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.037 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.037 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.037 INFO:teuthology.orchestra.run.smithi040.stdout: "device_cache_timeout": { 2023-07-18T03:42:08.037 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_cache_timeout", 2023-07-18T03:42:08.037 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.037 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.038 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.038 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1800", 2023-07-18T03:42:08.038 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.038 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.038 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.038 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache device inventory", 2023-07-18T03:42:08.038 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.039 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.039 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.039 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.039 INFO:teuthology.orchestra.run.smithi040.stdout: "device_enhanced_scan": { 2023-07-18T03:42:08.039 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_enhanced_scan", 2023-07-18T03:42:08.039 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.039 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.039 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.040 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.040 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.040 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.040 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.040 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-18T03:42:08.041 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.041 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.041 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.041 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.041 INFO:teuthology.orchestra.run.smithi040.stdout: "facts_cache_timeout": { 2023-07-18T03:42:08.041 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "facts_cache_timeout", 2023-07-18T03:42:08.041 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.042 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.042 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.042 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2023-07-18T03:42:08.042 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.042 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.042 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.043 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache host facts data", 2023-07-18T03:42:08.043 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.043 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.043 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.043 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.043 INFO:teuthology.orchestra.run.smithi040.stdout: "host_check_interval": { 2023-07-18T03:42:08.044 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "host_check_interval", 2023-07-18T03:42:08.044 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.044 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.044 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.044 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2023-07-18T03:42:08.044 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.045 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.045 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.045 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to perform a host check", 2023-07-18T03:42:08.045 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.045 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.045 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.045 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.046 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.046 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.046 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.046 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.046 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.046 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.046 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.047 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.047 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.047 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.047 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.047 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.048 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.048 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.048 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.048 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.049 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.049 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.049 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.049 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.049 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.050 INFO:teuthology.orchestra.run.smithi040.stdout: "log_refresh_metadata": { 2023-07-18T03:42:08.050 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_refresh_metadata", 2023-07-18T03:42:08.050 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.050 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.051 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.051 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.051 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.052 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.052 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.053 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-18T03:42:08.053 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.053 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.053 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.054 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.054 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.054 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.054 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.055 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.055 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.055 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.055 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.056 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.056 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.056 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-18T03:42:08.057 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.057 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.057 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.057 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.058 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.058 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.058 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.058 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.059 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.059 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.060 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.060 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.060 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.060 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.061 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.061 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.061 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.061 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.062 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.062 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.062 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.063 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.063 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.063 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.063 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.063 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.063 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.064 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.064 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.064 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.064 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.064 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.064 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.064 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.065 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.065 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.065 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.065 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.065 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.066 INFO:teuthology.orchestra.run.smithi040.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-18T03:42:08.066 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-18T03:42:08.066 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.066 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.066 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.066 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.066 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.066 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.067 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.067 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-18T03:42:08.067 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.067 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.067 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.068 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.068 INFO:teuthology.orchestra.run.smithi040.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-18T03:42:08.068 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-18T03:42:08.068 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.068 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.068 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.068 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "*", 2023-07-18T03:42:08.068 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.069 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.069 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.069 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-18T03:42:08.069 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.069 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.069 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.070 INFO:teuthology.orchestra.run.smithi040.stdout: "max_count_per_host": { 2023-07-18T03:42:08.070 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_count_per_host", 2023-07-18T03:42:08.070 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.070 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.070 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.071 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:08.071 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.071 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.071 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.071 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "max number of daemons per service per host", 2023-07-18T03:42:08.071 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.071 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.072 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.072 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.072 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd_draining_count": { 2023-07-18T03:42:08.072 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_osd_draining_count", 2023-07-18T03:42:08.072 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.072 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.072 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.073 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:08.073 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.073 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.073 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.073 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-18T03:42:08.074 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.074 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.074 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.074 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.074 INFO:teuthology.orchestra.run.smithi040.stdout: "migration_current": { 2023-07-18T03:42:08.074 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "migration_current", 2023-07-18T03:42:08.075 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.075 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.075 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.075 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.075 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.075 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.075 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.075 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "internal - do not modify", 2023-07-18T03:42:08.076 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.076 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.076 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.076 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.076 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": { 2023-07-18T03:42:08.076 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mode", 2023-07-18T03:42:08.076 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.077 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.077 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.077 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "root", 2023-07-18T03:42:08.077 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.077 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.077 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.077 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm-package", 2023-07-18T03:42:08.078 INFO:teuthology.orchestra.run.smithi040.stdout: "root" 2023-07-18T03:42:08.078 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.078 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "mode for remote execution of cephadm", 2023-07-18T03:42:08.078 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.078 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.078 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.078 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.078 INFO:teuthology.orchestra.run.smithi040.stdout: "prometheus_alerts_path": { 2023-07-18T03:42:08.079 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus_alerts_path", 2023-07-18T03:42:08.079 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.079 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.079 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.079 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-18T03:42:08.079 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.079 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.079 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.080 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-18T03:42:08.080 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.080 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.080 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.080 INFO:teuthology.orchestra.run.smithi040.stdout: "prometheus_web_password": { 2023-07-18T03:42:08.080 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus_web_password", 2023-07-18T03:42:08.081 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.081 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.081 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.081 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:08.081 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.081 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.081 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.082 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus web password", 2023-07-18T03:42:08.082 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.082 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.082 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.082 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.082 INFO:teuthology.orchestra.run.smithi040.stdout: "prometheus_web_user": { 2023-07-18T03:42:08.082 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus_web_user", 2023-07-18T03:42:08.082 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.083 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.083 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.083 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:08.083 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.083 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.083 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.083 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus web user", 2023-07-18T03:42:08.083 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.084 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.084 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.084 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.084 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_insecure": { 2023-07-18T03:42:08.084 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_insecure", 2023-07-18T03:42:08.084 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.084 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.085 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.085 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.085 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.085 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.085 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.085 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-18T03:42:08.085 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.085 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.086 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.086 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.086 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_password": { 2023-07-18T03:42:08.086 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_password", 2023-07-18T03:42:08.086 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.086 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.086 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.086 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.087 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.087 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.087 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.087 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-18T03:42:08.087 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.087 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.087 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.088 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.088 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_url": { 2023-07-18T03:42:08.088 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_url", 2023-07-18T03:42:08.088 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.088 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.088 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.088 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.088 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.089 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.089 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.089 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-18T03:42:08.089 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.089 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.089 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.089 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.089 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_username": { 2023-07-18T03:42:08.090 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_username", 2023-07-18T03:42:08.090 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.090 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.090 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.090 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.090 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.090 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.091 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.091 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-18T03:42:08.091 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.091 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.091 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.091 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.091 INFO:teuthology.orchestra.run.smithi040.stdout: "secure_monitoring_stack": { 2023-07-18T03:42:08.091 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "secure_monitoring_stack", 2023-07-18T03:42:08.092 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.092 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.092 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.092 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.092 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.092 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.092 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.093 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-18T03:42:08.093 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.093 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.093 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.093 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.093 INFO:teuthology.orchestra.run.smithi040.stdout: "service_discovery_port": { 2023-07-18T03:42:08.093 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "service_discovery_port", 2023-07-18T03:42:08.093 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.094 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.094 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.094 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8765", 2023-07-18T03:42:08.094 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.094 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.094 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.094 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "cephadm service discovery port", 2023-07-18T03:42:08.094 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.095 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.095 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.095 INFO:teuthology.orchestra.run.smithi040.stdout: "ssh_config_file": { 2023-07-18T03:42:08.095 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssh_config_file", 2023-07-18T03:42:08.095 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.095 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.095 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.096 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.096 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.096 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.096 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.096 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-18T03:42:08.096 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.096 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.097 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.097 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.097 INFO:teuthology.orchestra.run.smithi040.stdout: "use_agent": { 2023-07-18T03:42:08.097 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "use_agent", 2023-07-18T03:42:08.097 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.097 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.097 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.097 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.098 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.098 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.098 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.098 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-18T03:42:08.098 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.098 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.098 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.099 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.099 INFO:teuthology.orchestra.run.smithi040.stdout: "use_repo_digest": { 2023-07-18T03:42:08.099 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "use_repo_digest", 2023-07-18T03:42:08.099 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.099 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.099 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.100 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.100 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.100 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.100 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.101 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-18T03:42:08.101 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.101 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.101 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.101 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.101 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_failed_host_check": { 2023-07-18T03:42:08.101 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_failed_host_check", 2023-07-18T03:42:08.102 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.102 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.102 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.102 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.102 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.102 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.102 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.102 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if the host check fails", 2023-07-18T03:42:08.103 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.103 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.103 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.103 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_stray_daemons": { 2023-07-18T03:42:08.103 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_stray_daemons", 2023-07-18T03:42:08.103 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.103 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.104 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.104 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.104 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.104 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.104 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.104 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-18T03:42:08.104 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.105 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.105 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.105 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.105 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_stray_hosts": { 2023-07-18T03:42:08.105 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_stray_hosts", 2023-07-18T03:42:08.105 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.105 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.105 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.106 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.106 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.106 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.106 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.106 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-18T03:42:08.106 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.106 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.106 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.107 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.107 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.107 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crash", 2023-07-18T03:42:08.107 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.107 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.107 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.108 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.108 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.108 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.108 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.108 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.108 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.108 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.109 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.109 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.109 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.109 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.109 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.109 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.109 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.110 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.110 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.110 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.110 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.110 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.110 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.110 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.111 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.111 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.111 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.111 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.111 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.111 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.111 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.111 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.112 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.112 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.112 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.112 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.112 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.112 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.112 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.113 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.113 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.113 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.113 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.113 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.113 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.113 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.113 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.114 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.114 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.114 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.114 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.114 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.114 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.114 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.115 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.115 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.115 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.115 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.115 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.115 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.115 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.116 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.116 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.116 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.116 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.116 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.116 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.116 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.116 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.117 INFO:teuthology.orchestra.run.smithi040.stdout: "retain_interval": { 2023-07-18T03:42:08.117 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "retain_interval", 2023-07-18T03:42:08.117 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.117 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.117 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.117 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "31536000", 2023-07-18T03:42:08.117 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.118 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.118 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.118 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-18T03:42:08.118 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.118 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.118 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.118 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_recent_interval": { 2023-07-18T03:42:08.119 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_recent_interval", 2023-07-18T03:42:08.119 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.119 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.119 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.119 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1209600", 2023-07-18T03:42:08.119 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.119 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.119 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.120 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-18T03:42:08.120 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.120 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.120 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.120 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.120 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.121 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "dashboard", 2023-07-18T03:42:08.121 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.121 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.121 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.121 INFO:teuthology.orchestra.run.smithi040.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-18T03:42:08.121 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-18T03:42:08.121 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.122 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.122 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.122 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:08.122 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.122 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.122 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.122 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.123 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.123 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.123 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.123 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.123 INFO:teuthology.orchestra.run.smithi040.stdout: "ALERTMANAGER_API_HOST": { 2023-07-18T03:42:08.123 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-18T03:42:08.123 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.123 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.124 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.124 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.124 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.124 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.124 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.124 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.125 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.125 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.125 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.125 INFO:teuthology.orchestra.run.smithi040.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-18T03:42:08.125 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-18T03:42:08.125 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.125 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.126 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.126 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.126 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.126 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.126 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.126 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.126 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.126 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.127 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.127 INFO:teuthology.orchestra.run.smithi040.stdout: "AUDIT_API_ENABLED": { 2023-07-18T03:42:08.127 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "AUDIT_API_ENABLED", 2023-07-18T03:42:08.127 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.127 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.127 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.128 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.128 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.128 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.128 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.128 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.128 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.129 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.129 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.129 INFO:teuthology.orchestra.run.smithi040.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-18T03:42:08.130 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-18T03:42:08.130 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.130 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.131 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.131 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.131 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.131 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.132 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.132 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.132 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.133 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.133 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.133 INFO:teuthology.orchestra.run.smithi040.stdout: "ENABLE_BROWSABLE_API": { 2023-07-18T03:42:08.133 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-18T03:42:08.134 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.134 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.134 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.134 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.135 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.135 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.135 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.136 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.136 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.136 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.136 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.136 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-18T03:42:08.136 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-18T03:42:08.137 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.137 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.137 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.137 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.137 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.137 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.137 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.137 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.138 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.138 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.138 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.138 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-18T03:42:08.138 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-18T03:42:08.138 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.139 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.139 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.139 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.139 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.139 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.139 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.139 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.139 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.140 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.140 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.140 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-18T03:42:08.140 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-18T03:42:08.140 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.140 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.141 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.141 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.141 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.141 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.141 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.141 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.141 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.141 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.142 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.142 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-18T03:42:08.142 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-18T03:42:08.142 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.142 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.142 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.142 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.143 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.143 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.143 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.143 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.143 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.143 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.143 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.144 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-18T03:42:08.144 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-18T03:42:08.144 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.144 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.144 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.144 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.144 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.145 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.145 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.145 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.145 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.145 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.145 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.145 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-18T03:42:08.146 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-18T03:42:08.146 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.146 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.146 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.146 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.146 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.146 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.146 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.147 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.147 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.147 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.147 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.147 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-18T03:42:08.147 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-18T03:42:08.147 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.148 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.148 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.148 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.148 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.148 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.148 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.148 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.149 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.149 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.149 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.149 INFO:teuthology.orchestra.run.smithi040.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-18T03:42:08.149 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-18T03:42:08.149 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.150 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.150 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.150 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.150 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.150 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.150 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.150 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.151 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.151 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.151 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.151 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_PASSWORD": { 2023-07-18T03:42:08.151 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-18T03:42:08.151 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.151 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.152 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.152 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:08.152 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.152 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.152 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.152 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.153 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.153 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.153 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.153 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-18T03:42:08.153 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-18T03:42:08.153 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.153 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.154 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.154 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.154 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.154 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.154 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.154 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.154 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.154 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.155 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.155 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_URL": { 2023-07-18T03:42:08.155 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_URL", 2023-07-18T03:42:08.155 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.155 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.155 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.156 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.156 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.156 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.156 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.156 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.156 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.156 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.156 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.157 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_USERNAME": { 2023-07-18T03:42:08.157 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-18T03:42:08.157 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.157 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.157 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.157 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:08.157 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.158 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.158 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.158 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.158 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.158 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.158 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.158 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-18T03:42:08.159 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-18T03:42:08.159 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.159 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.159 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.159 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.159 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.159 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.160 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.160 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.160 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.160 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.160 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.160 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-18T03:42:08.160 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-18T03:42:08.161 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.161 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.161 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.161 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.161 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.161 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.161 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.162 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.162 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.162 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.162 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.162 INFO:teuthology.orchestra.run.smithi040.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-18T03:42:08.162 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-18T03:42:08.162 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.163 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.163 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.163 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.163 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.163 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.163 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.163 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.164 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.164 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.164 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.164 INFO:teuthology.orchestra.run.smithi040.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-18T03:42:08.165 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-18T03:42:08.165 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.165 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.165 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.165 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.165 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.165 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.165 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.166 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.166 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.166 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.166 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.166 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.166 INFO:teuthology.orchestra.run.smithi040.stdout: "PROMETHEUS_API_HOST": { 2023-07-18T03:42:08.166 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-18T03:42:08.166 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.167 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.167 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.167 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.167 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.167 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.167 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.167 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.168 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.168 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.168 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.168 INFO:teuthology.orchestra.run.smithi040.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-18T03:42:08.168 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-18T03:42:08.168 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.168 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.169 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.169 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.169 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.169 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.169 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.169 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.169 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.170 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.170 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.170 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.170 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-18T03:42:08.171 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-18T03:42:08.171 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.171 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.171 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.172 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.172 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.172 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.173 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.173 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.173 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.173 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.174 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.174 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.174 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-18T03:42:08.175 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-18T03:42:08.175 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.175 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.175 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.176 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.176 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.176 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.177 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.177 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.177 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.178 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.178 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.178 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.178 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-18T03:42:08.178 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-18T03:42:08.179 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.179 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.179 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.179 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.179 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.180 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.180 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.180 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.181 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.181 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.181 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.181 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-18T03:42:08.182 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-18T03:42:08.182 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.182 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.182 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.182 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.182 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.182 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.183 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.183 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.183 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.183 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.183 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.183 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-18T03:42:08.183 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-18T03:42:08.184 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.184 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.184 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.184 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.184 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.184 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.184 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.185 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.185 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.185 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.185 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.185 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-18T03:42:08.185 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-18T03:42:08.185 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.186 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.186 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.186 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.186 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.186 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.186 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.186 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.187 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.187 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.187 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.187 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-18T03:42:08.187 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-18T03:42:08.187 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.187 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.188 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.188 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.188 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.188 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.188 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.188 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.188 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.188 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.189 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.189 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_ENABLED": { 2023-07-18T03:42:08.189 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-18T03:42:08.189 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.189 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.189 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.190 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.190 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.190 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.190 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.190 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.190 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.190 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.190 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.191 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-18T03:42:08.191 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-18T03:42:08.191 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.191 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.191 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.191 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-18T03:42:08.191 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.192 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.192 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.192 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.192 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.192 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.192 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.193 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-18T03:42:08.193 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-18T03:42:08.193 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.193 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.193 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.193 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:08.193 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.193 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.194 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.194 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.194 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.194 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.194 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.194 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-18T03:42:08.194 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-18T03:42:08.195 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.195 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.195 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.195 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8", 2023-07-18T03:42:08.195 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.195 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.195 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.195 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.196 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.196 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.196 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.196 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.196 INFO:teuthology.orchestra.run.smithi040.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-18T03:42:08.196 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-18T03:42:08.196 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.197 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.197 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.197 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "45", 2023-07-18T03:42:08.197 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.197 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.197 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.197 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.197 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.198 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.198 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.198 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.198 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_ACCESS_KEY": { 2023-07-18T03:42:08.198 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-18T03:42:08.198 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.198 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.199 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.199 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.199 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.199 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.199 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.200 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.200 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.200 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.200 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.200 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.200 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-18T03:42:08.200 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-18T03:42:08.201 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.201 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.201 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.201 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:08.201 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.202 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.202 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.202 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.203 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.203 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.203 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.203 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.204 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_SECRET_KEY": { 2023-07-18T03:42:08.204 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-18T03:42:08.204 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.205 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.205 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.205 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.205 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.206 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.206 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.206 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.206 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.207 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.207 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.207 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.208 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_SSL_VERIFY": { 2023-07-18T03:42:08.208 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-18T03:42:08.208 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.208 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.208 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.208 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.208 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.209 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.209 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.209 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.209 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.209 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.209 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.209 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.209 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-18T03:42:08.210 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-18T03:42:08.210 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.210 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.210 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.210 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2023-07-18T03:42:08.210 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.210 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.211 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.211 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.211 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.211 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.211 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.211 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.211 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-18T03:42:08.211 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-18T03:42:08.212 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.212 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.212 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.212 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:08.212 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.212 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.212 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.213 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.213 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.213 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.213 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.213 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.213 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-18T03:42:08.213 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-18T03:42:08.214 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.214 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.214 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.214 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2023-07-18T03:42:08.214 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.214 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.214 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.214 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.215 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.215 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.215 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.215 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.215 INFO:teuthology.orchestra.run.smithi040.stdout: "cross_origin_url": { 2023-07-18T03:42:08.215 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cross_origin_url", 2023-07-18T03:42:08.215 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.215 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.216 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.216 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.216 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.216 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.216 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.216 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.217 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.217 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.217 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.217 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.218 INFO:teuthology.orchestra.run.smithi040.stdout: "crt_file": { 2023-07-18T03:42:08.218 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crt_file", 2023-07-18T03:42:08.218 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.218 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.219 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.219 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.219 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.219 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.220 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.220 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.220 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.221 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.221 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.221 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.222 INFO:teuthology.orchestra.run.smithi040.stdout: "debug": { 2023-07-18T03:42:08.222 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "debug", 2023-07-18T03:42:08.222 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.222 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.223 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.223 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.223 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.224 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.224 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.224 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable/disable debug options", 2023-07-18T03:42:08.224 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.225 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.225 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.225 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.225 INFO:teuthology.orchestra.run.smithi040.stdout: "jwt_token_ttl": { 2023-07-18T03:42:08.225 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "jwt_token_ttl", 2023-07-18T03:42:08.226 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.226 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.226 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.226 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "28800", 2023-07-18T03:42:08.227 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.227 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.227 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.228 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.228 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.228 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.228 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.228 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.228 INFO:teuthology.orchestra.run.smithi040.stdout: "key_file": { 2023-07-18T03:42:08.228 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "key_file", 2023-07-18T03:42:08.229 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.229 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.229 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.229 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.229 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.229 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.229 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.229 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.230 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.230 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.230 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.230 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.230 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.230 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.230 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.231 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.231 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.231 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.231 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.231 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.231 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.231 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.231 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.232 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.232 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.232 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.232 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.232 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.232 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.232 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.233 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.233 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.233 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.233 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.233 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.233 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.233 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.233 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.234 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.234 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.234 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.234 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.234 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.234 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.234 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.234 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.235 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.235 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.235 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.235 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.235 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.235 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.235 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.235 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.236 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.236 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.236 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.236 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.236 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.236 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.236 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.237 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.237 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.237 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.237 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.237 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.237 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.237 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.237 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.238 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.238 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.238 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.238 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.238 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.238 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.238 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.239 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.239 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.239 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.239 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.239 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.239 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.239 INFO:teuthology.orchestra.run.smithi040.stdout: "motd": { 2023-07-18T03:42:08.239 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "motd", 2023-07-18T03:42:08.240 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.240 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.240 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.240 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.240 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.240 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.240 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.240 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "The message of the day", 2023-07-18T03:42:08.241 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.241 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.241 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.241 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.241 INFO:teuthology.orchestra.run.smithi040.stdout: "redirect_resolve_ip_addr": { 2023-07-18T03:42:08.241 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "redirect_resolve_ip_addr", 2023-07-18T03:42:08.242 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.242 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.242 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.242 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.242 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.242 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.242 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.242 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.243 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.243 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.243 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.243 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.243 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2023-07-18T03:42:08.243 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2023-07-18T03:42:08.243 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.243 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.244 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.244 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "::", 2023-07-18T03:42:08.244 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.244 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.244 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.244 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.244 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.245 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.245 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.245 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.245 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2023-07-18T03:42:08.245 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2023-07-18T03:42:08.245 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.245 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.245 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.246 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8080", 2023-07-18T03:42:08.246 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.246 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.246 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.246 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.246 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.246 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.246 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.247 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.247 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl": { 2023-07-18T03:42:08.247 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl", 2023-07-18T03:42:08.247 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.247 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.247 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.247 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.247 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.248 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.248 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.248 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.248 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.248 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.248 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.248 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.249 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl_server_port": { 2023-07-18T03:42:08.249 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl_server_port", 2023-07-18T03:42:08.249 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.249 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.249 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.249 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8443", 2023-07-18T03:42:08.249 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.249 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.250 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.250 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.250 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.250 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.250 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.250 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.250 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_behaviour": { 2023-07-18T03:42:08.250 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_behaviour", 2023-07-18T03:42:08.251 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.251 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.251 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.251 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "redirect", 2023-07-18T03:42:08.251 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.251 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.251 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.251 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.252 INFO:teuthology.orchestra.run.smithi040.stdout: "redirect" 2023-07-18T03:42:08.252 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.252 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.252 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.252 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.252 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.252 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.253 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_error_status_code": { 2023-07-18T03:42:08.253 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_error_status_code", 2023-07-18T03:42:08.253 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.253 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.253 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.253 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "500", 2023-07-18T03:42:08.253 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "400", 2023-07-18T03:42:08.253 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "599", 2023-07-18T03:42:08.254 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.254 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.254 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.254 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.254 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.254 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.254 INFO:teuthology.orchestra.run.smithi040.stdout: "url_prefix": { 2023-07-18T03:42:08.254 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "url_prefix", 2023-07-18T03:42:08.255 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.255 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.255 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.255 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.255 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.255 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.255 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.256 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.256 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.256 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.256 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.256 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.256 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.256 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.256 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.257 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "devicehealth", 2023-07-18T03:42:08.257 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.257 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.257 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.257 INFO:teuthology.orchestra.run.smithi040.stdout: "enable_monitoring": { 2023-07-18T03:42:08.257 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enable_monitoring", 2023-07-18T03:42:08.257 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.258 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.258 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.258 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.258 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.258 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.258 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.258 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "monitor device health metrics", 2023-07-18T03:42:08.258 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.259 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.259 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.259 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.259 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.259 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.259 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.259 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.259 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.260 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.260 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.260 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.260 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.260 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.260 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.260 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.261 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.261 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.261 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.261 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.261 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.261 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.261 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.261 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.262 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.262 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.262 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.262 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.262 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.262 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.262 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.262 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.263 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.263 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.263 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.263 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.263 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.263 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.263 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.264 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.264 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.264 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.264 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.264 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.264 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.264 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.264 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.265 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.265 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.265 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.265 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.265 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.265 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.265 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.266 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.266 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.266 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.266 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.266 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.266 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.266 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.266 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.267 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.267 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.267 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.267 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.267 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.267 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.267 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.268 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.268 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.268 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.268 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.268 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.268 INFO:teuthology.orchestra.run.smithi040.stdout: "mark_out_threshold": { 2023-07-18T03:42:08.268 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mark_out_threshold", 2023-07-18T03:42:08.269 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.269 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.269 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.269 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "2419200", 2023-07-18T03:42:08.269 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.269 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.269 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.269 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-18T03:42:08.270 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.270 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.270 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.270 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.270 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": { 2023-07-18T03:42:08.270 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pool_name", 2023-07-18T03:42:08.270 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.271 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.271 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.271 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "device_health_metrics", 2023-07-18T03:42:08.271 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.271 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.271 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.271 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-18T03:42:08.271 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.272 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.272 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.272 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.272 INFO:teuthology.orchestra.run.smithi040.stdout: "retention_period": { 2023-07-18T03:42:08.272 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "retention_period", 2023-07-18T03:42:08.272 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.272 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.272 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.273 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15552000", 2023-07-18T03:42:08.273 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.273 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.273 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.273 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long to retain device health metrics", 2023-07-18T03:42:08.273 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.274 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.274 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.274 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.274 INFO:teuthology.orchestra.run.smithi040.stdout: "scrape_frequency": { 2023-07-18T03:42:08.274 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "scrape_frequency", 2023-07-18T03:42:08.274 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.274 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.274 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.275 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "86400", 2023-07-18T03:42:08.275 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.275 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.275 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.275 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-18T03:42:08.275 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.276 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.276 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.276 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.276 INFO:teuthology.orchestra.run.smithi040.stdout: "self_heal": { 2023-07-18T03:42:08.276 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "self_heal", 2023-07-18T03:42:08.276 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.276 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.277 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.277 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.277 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.277 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.277 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.277 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-18T03:42:08.277 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.277 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.278 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.278 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2023-07-18T03:42:08.278 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2023-07-18T03:42:08.278 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.278 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.278 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.278 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2023-07-18T03:42:08.279 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.279 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.279 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.279 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to wake up and check device health", 2023-07-18T03:42:08.279 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.279 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.279 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.279 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.280 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_threshold": { 2023-07-18T03:42:08.280 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_threshold", 2023-07-18T03:42:08.280 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.280 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.280 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.280 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "7257600", 2023-07-18T03:42:08.280 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.281 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.281 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.281 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-18T03:42:08.281 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.281 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.281 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.281 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.281 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.282 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.282 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "diskprediction_local", 2023-07-18T03:42:08.282 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.282 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.282 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.282 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.282 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.283 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.283 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.283 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.283 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.283 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.283 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.283 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.283 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.284 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.284 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.284 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.284 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.284 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.284 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.284 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.285 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.285 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.285 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.285 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.285 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.285 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.285 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.285 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.286 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.286 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.286 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.286 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.286 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.286 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.286 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.286 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.287 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.287 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.287 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.287 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.287 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.287 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.287 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.287 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.288 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.288 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.288 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.288 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.288 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.289 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.289 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.289 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.290 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.290 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.290 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.290 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.291 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.291 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.291 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.291 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.292 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.292 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.292 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.292 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.293 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.293 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.293 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.293 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.294 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.294 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.294 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.295 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.295 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.295 INFO:teuthology.orchestra.run.smithi040.stdout: "predict_interval": { 2023-07-18T03:42:08.295 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "predict_interval", 2023-07-18T03:42:08.295 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.295 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.295 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.296 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "86400", 2023-07-18T03:42:08.296 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.296 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.296 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.296 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.296 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.296 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.297 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.297 INFO:teuthology.orchestra.run.smithi040.stdout: "predictor_model": { 2023-07-18T03:42:08.297 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "predictor_model", 2023-07-18T03:42:08.298 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.298 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.298 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.298 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "prophetstor", 2023-07-18T03:42:08.299 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.299 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.299 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.299 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.299 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.299 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.299 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.300 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2023-07-18T03:42:08.300 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2023-07-18T03:42:08.300 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.301 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.301 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.301 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2023-07-18T03:42:08.302 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.302 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.302 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.302 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.303 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.303 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.303 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.303 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.304 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.304 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.305 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "influx", 2023-07-18T03:42:08.305 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": false, 2023-07-18T03:42:08.305 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "influxdb python module not found", 2023-07-18T03:42:08.305 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.306 INFO:teuthology.orchestra.run.smithi040.stdout: "batch_size": { 2023-07-18T03:42:08.306 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "batch_size", 2023-07-18T03:42:08.306 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.306 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.307 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.307 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5000", 2023-07-18T03:42:08.307 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.307 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.308 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.308 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-18T03:42:08.308 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.309 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.309 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.309 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.309 INFO:teuthology.orchestra.run.smithi040.stdout: "database": { 2023-07-18T03:42:08.310 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "database", 2023-07-18T03:42:08.310 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.310 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.310 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.311 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "ceph", 2023-07-18T03:42:08.311 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.311 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.311 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.312 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-18T03:42:08.312 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.312 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.312 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.313 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": { 2023-07-18T03:42:08.313 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "hostname", 2023-07-18T03:42:08.313 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.314 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.314 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.314 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.315 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.315 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.315 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.315 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB server hostname", 2023-07-18T03:42:08.316 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.316 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.316 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.317 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.317 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2023-07-18T03:42:08.317 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2023-07-18T03:42:08.317 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.318 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.318 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.318 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "30", 2023-07-18T03:42:08.318 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "5", 2023-07-18T03:42:08.319 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.319 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.319 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-18T03:42:08.319 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.320 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.320 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.321 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.321 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.321 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.321 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.321 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.322 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.322 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.322 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.323 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.323 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.323 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.323 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.324 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.324 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.324 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.324 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.325 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.325 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.325 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.326 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.326 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.326 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.326 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.327 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.327 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.327 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.327 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.328 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.328 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.328 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.328 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.328 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.329 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.329 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.329 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.329 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.330 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.330 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.330 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.330 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.331 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.331 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.331 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.331 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.332 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.332 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.332 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.333 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.333 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.333 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.333 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.334 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.334 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.334 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.334 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.335 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.335 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.335 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.336 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.336 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.336 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.336 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.337 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.337 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.337 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.337 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.338 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.338 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.338 INFO:teuthology.orchestra.run.smithi040.stdout: "password": { 2023-07-18T03:42:08.339 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "password", 2023-07-18T03:42:08.339 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.339 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.339 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.340 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.340 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.340 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.340 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.341 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "password of InfluxDB server user", 2023-07-18T03:42:08.341 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.341 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.342 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.342 INFO:teuthology.orchestra.run.smithi040.stdout: "port": { 2023-07-18T03:42:08.342 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "port", 2023-07-18T03:42:08.343 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.343 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.343 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.343 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8086", 2023-07-18T03:42:08.344 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.344 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.344 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.345 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB server port", 2023-07-18T03:42:08.345 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.345 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.345 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.346 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl": { 2023-07-18T03:42:08.346 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl", 2023-07-18T03:42:08.346 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.347 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.347 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.347 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "false", 2023-07-18T03:42:08.348 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.348 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.348 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.348 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-18T03:42:08.349 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.349 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.349 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.350 INFO:teuthology.orchestra.run.smithi040.stdout: "threads": { 2023-07-18T03:42:08.350 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "threads", 2023-07-18T03:42:08.350 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.350 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.351 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.351 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2023-07-18T03:42:08.351 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2023-07-18T03:42:08.351 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "32", 2023-07-18T03:42:08.351 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.351 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-18T03:42:08.352 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.352 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.352 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.352 INFO:teuthology.orchestra.run.smithi040.stdout: "username": { 2023-07-18T03:42:08.352 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "username", 2023-07-18T03:42:08.353 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.353 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.353 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.353 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.353 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.353 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.353 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.353 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "username of InfluxDB server user", 2023-07-18T03:42:08.354 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.354 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.354 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.354 INFO:teuthology.orchestra.run.smithi040.stdout: "verify_ssl": { 2023-07-18T03:42:08.354 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "verify_ssl", 2023-07-18T03:42:08.354 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.354 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.355 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.355 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "true", 2023-07-18T03:42:08.355 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.355 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.355 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.356 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-18T03:42:08.356 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.356 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.356 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.356 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.356 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.357 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "insights", 2023-07-18T03:42:08.357 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.357 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.357 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.357 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.357 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.357 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.358 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.358 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.358 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.358 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.358 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.358 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.358 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.358 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.359 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.359 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.359 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.359 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.359 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.359 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.359 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.359 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.360 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.360 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.360 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.360 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.360 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.361 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.361 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.361 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.361 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.361 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.361 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.361 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.362 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.362 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.362 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.362 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.362 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.362 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.362 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.363 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.363 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.363 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.363 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.363 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.363 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.363 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.363 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.364 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.364 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.364 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.364 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.364 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.364 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.364 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.364 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.365 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.365 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.365 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.365 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.365 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.365 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.365 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.366 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.366 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.366 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.366 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.366 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.366 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.366 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.366 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.367 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.367 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.367 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.367 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.367 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "iostat", 2023-07-18T03:42:08.367 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.367 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.367 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.368 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.368 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.368 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.368 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.368 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.368 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.368 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.369 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.369 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.369 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.369 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.369 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.369 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.369 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.369 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.370 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.370 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.370 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.370 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.370 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.370 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.370 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.370 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.371 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.371 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.371 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.371 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.371 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.371 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.371 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.371 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.372 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.372 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.372 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.372 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.372 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.372 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.373 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.373 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.373 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.373 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.373 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.373 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.373 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.373 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.374 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.374 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.374 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.374 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.374 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.374 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.374 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.374 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.375 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.375 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.375 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.375 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.375 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.375 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.376 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.376 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.376 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.376 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.377 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.377 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.377 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.377 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.378 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.378 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.378 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.378 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.379 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.379 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.379 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "k8sevents", 2023-07-18T03:42:08.379 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.380 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.380 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.380 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_event_retention_days": { 2023-07-18T03:42:08.381 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ceph_event_retention_days", 2023-07-18T03:42:08.381 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.381 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.381 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.382 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "7", 2023-07-18T03:42:08.382 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.382 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.382 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.382 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-18T03:42:08.383 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.383 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.383 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.383 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.383 INFO:teuthology.orchestra.run.smithi040.stdout: "config_check_secs": { 2023-07-18T03:42:08.383 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_check_secs", 2023-07-18T03:42:08.383 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.384 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.384 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.384 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:08.384 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "10", 2023-07-18T03:42:08.384 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.384 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.384 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-18T03:42:08.384 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.385 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.385 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.385 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.385 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.385 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.385 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.386 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.386 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.386 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.386 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.386 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.386 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.386 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.386 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.387 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.387 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.387 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.387 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.387 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.387 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.387 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.387 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.388 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.388 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.388 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.388 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.388 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.388 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.388 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.389 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.389 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.389 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.389 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.389 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.389 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.389 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.390 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.390 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.390 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.390 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.390 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.390 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.390 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.390 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.391 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.391 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.391 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.391 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.391 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.391 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.391 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.391 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.392 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.392 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.392 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.392 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.392 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.392 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.392 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.393 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.393 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.393 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.393 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.393 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.393 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.393 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.394 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.394 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.394 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.394 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.394 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.394 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.395 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "localpool", 2023-07-18T03:42:08.395 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.395 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.395 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.396 INFO:teuthology.orchestra.run.smithi040.stdout: "failure_domain": { 2023-07-18T03:42:08.396 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "failure_domain", 2023-07-18T03:42:08.396 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.396 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.396 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.396 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "host", 2023-07-18T03:42:08.396 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.396 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.397 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.397 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "failure domain for any created local pool", 2023-07-18T03:42:08.397 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-18T03:42:08.397 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.397 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.397 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.397 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.398 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.398 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.398 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.398 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.398 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.398 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.398 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.398 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.399 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.399 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.399 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.399 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.399 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.399 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.399 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.399 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.400 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.400 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.400 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.400 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.400 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.400 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.400 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.401 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.401 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.401 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.401 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.401 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.401 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.401 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.401 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.402 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.402 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.402 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.402 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.402 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.402 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.403 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.403 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.403 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.403 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.403 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.403 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.403 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.403 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.404 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.404 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.404 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.404 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.404 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.404 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.404 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.405 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.405 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.405 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.405 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.405 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.405 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.405 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.405 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.406 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.406 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.406 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.406 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.406 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.406 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.406 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.406 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.407 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": { 2023-07-18T03:42:08.407 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "min_size", 2023-07-18T03:42:08.407 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.407 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.407 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.407 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.407 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.407 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.408 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.408 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default min_size for any created local pool", 2023-07-18T03:42:08.408 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-18T03:42:08.408 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.408 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.408 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.408 INFO:teuthology.orchestra.run.smithi040.stdout: "num_rep": { 2023-07-18T03:42:08.409 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "num_rep", 2023-07-18T03:42:08.409 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.409 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.409 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.409 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3", 2023-07-18T03:42:08.409 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.409 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.409 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.410 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default replica count for any created local pool", 2023-07-18T03:42:08.410 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.410 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.410 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.410 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.410 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": { 2023-07-18T03:42:08.410 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pg_num", 2023-07-18T03:42:08.410 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.411 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.411 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.411 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "128", 2023-07-18T03:42:08.411 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.411 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.411 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.411 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default pg_num for any created local pool", 2023-07-18T03:42:08.411 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.412 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.412 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.412 INFO:teuthology.orchestra.run.smithi040.stdout: "prefix": { 2023-07-18T03:42:08.412 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prefix", 2023-07-18T03:42:08.413 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.413 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.413 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.413 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.414 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.414 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.414 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.414 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "name prefix for any created local pool", 2023-07-18T03:42:08.414 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.415 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.415 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.415 INFO:teuthology.orchestra.run.smithi040.stdout: "subtree": { 2023-07-18T03:42:08.415 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "subtree", 2023-07-18T03:42:08.416 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.416 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.416 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.416 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "rack", 2023-07-18T03:42:08.416 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.417 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.417 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.417 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-18T03:42:08.417 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-18T03:42:08.418 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.418 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.418 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.418 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.418 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.419 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.419 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mds_autoscaler", 2023-07-18T03:42:08.419 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.419 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.420 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.420 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.420 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.420 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.420 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.421 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.421 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.421 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.421 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.421 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.422 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.422 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.422 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.422 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.423 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.423 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.423 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.423 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.423 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.424 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.424 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.424 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.424 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.424 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.425 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.425 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.425 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.425 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.425 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.426 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.426 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.426 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.426 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.426 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.426 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.427 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.427 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.427 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.427 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.427 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.427 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.428 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.428 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.428 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.428 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.428 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.428 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.428 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.428 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.429 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.429 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.429 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.429 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.429 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.429 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.429 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.430 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.430 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.430 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.430 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.430 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.430 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.430 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.430 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.431 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.431 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.431 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.431 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.431 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.431 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.432 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.432 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.432 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.432 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mirroring", 2023-07-18T03:42:08.432 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.433 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.433 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.433 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.433 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.433 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.433 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.433 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.434 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.434 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.434 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.434 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.434 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.434 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.434 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.434 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.435 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.435 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.435 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.435 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.435 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.435 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.435 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.436 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.436 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.436 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.436 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.436 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.436 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.436 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.437 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.437 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.437 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.437 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.437 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.437 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.437 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.438 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.438 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.438 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.438 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.438 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.438 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.439 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.439 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.439 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.439 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.439 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.439 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.439 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.440 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.440 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.440 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.440 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.440 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.440 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.440 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.441 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.441 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.441 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.441 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.441 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.441 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.441 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.441 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.442 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.442 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.442 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.442 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.442 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.442 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.442 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.442 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.443 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.443 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.443 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "nfs", 2023-07-18T03:42:08.443 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.443 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.443 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.443 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.444 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.444 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.444 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.444 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.444 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.444 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.444 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.445 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.445 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.445 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.445 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.445 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.445 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.445 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.445 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.446 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.446 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.446 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.446 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.446 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.446 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.447 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.447 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.447 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.447 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.448 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.448 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.448 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.448 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.448 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.448 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.448 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.449 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.449 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.449 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.449 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.449 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.449 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.449 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.450 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.450 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.450 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.450 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.450 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.450 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.450 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.450 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.451 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.451 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.451 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.451 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.451 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.451 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.451 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.452 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.452 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.452 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.452 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.452 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.452 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.452 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.453 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.453 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.453 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.453 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.453 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.453 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.454 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "orchestrator", 2023-07-18T03:42:08.454 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.454 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.454 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.454 INFO:teuthology.orchestra.run.smithi040.stdout: "fail_fs": { 2023-07-18T03:42:08.454 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "fail_fs", 2023-07-18T03:42:08.454 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.454 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.455 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.455 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.455 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.455 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.455 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.455 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-18T03:42:08.455 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.455 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.456 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.456 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.456 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.456 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.456 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.457 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.457 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.457 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.458 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.458 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.458 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.458 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.459 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.459 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.459 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.459 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.460 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.460 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.460 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.460 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.461 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.461 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.462 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.462 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.462 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.463 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.463 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.463 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.463 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.463 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.463 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.463 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.463 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.464 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.464 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.464 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.464 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.464 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.464 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.464 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.465 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.465 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.465 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.465 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.465 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.465 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.465 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.466 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.466 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.466 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.466 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.466 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.466 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.466 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.466 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.467 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.467 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.467 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.467 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.467 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.467 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.467 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.467 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.468 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.468 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.468 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.468 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.468 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.468 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.469 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.469 INFO:teuthology.orchestra.run.smithi040.stdout: "orchestrator": { 2023-07-18T03:42:08.469 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "orchestrator", 2023-07-18T03:42:08.469 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.469 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.469 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.470 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.470 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.470 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.470 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.470 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm", 2023-07-18T03:42:08.470 INFO:teuthology.orchestra.run.smithi040.stdout: "rook", 2023-07-18T03:42:08.470 INFO:teuthology.orchestra.run.smithi040.stdout: "test_orchestrator" 2023-07-18T03:42:08.470 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.471 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Orchestrator backend", 2023-07-18T03:42:08.471 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.471 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.471 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.471 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.471 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.472 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.472 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "osd_perf_query", 2023-07-18T03:42:08.472 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.472 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.472 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.472 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.472 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.473 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.473 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.473 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.473 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.473 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.473 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.473 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.473 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.474 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.474 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.474 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.474 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.474 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.474 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.474 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.474 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.475 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.475 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.475 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.475 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.475 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.475 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.475 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.475 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.476 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.476 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.476 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.476 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.476 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.476 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.477 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.477 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.477 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.477 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.477 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.477 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.477 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.477 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.478 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.478 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.478 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.478 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.478 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.478 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.478 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.479 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.479 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.479 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.479 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.479 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.479 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.479 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.479 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.480 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.480 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.480 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.480 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.480 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.480 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.481 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.481 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.481 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.481 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.481 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.481 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.481 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.481 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.482 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.482 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.482 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "osd_support", 2023-07-18T03:42:08.482 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.482 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.482 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.482 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.483 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.483 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.483 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.483 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.483 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.483 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.483 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.483 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.484 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.484 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.484 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.484 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.484 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.484 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.485 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.485 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.485 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.485 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.485 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.485 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.485 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.486 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.486 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.486 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.486 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.486 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.486 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.486 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.486 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.487 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.487 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.487 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.487 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.487 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.487 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.487 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.487 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.488 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.488 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.488 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.488 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.488 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.488 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.488 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.489 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.489 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.489 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.489 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.489 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.489 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.489 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.490 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.490 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.490 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.490 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.490 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.490 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.490 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.490 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.491 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.491 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.491 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.491 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.491 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.491 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.491 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.491 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.492 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.492 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.492 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.492 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.492 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pg_autoscaler", 2023-07-18T03:42:08.492 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.493 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.493 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.493 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.493 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.493 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.493 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.493 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.493 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.494 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.494 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.494 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.494 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.494 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.494 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.494 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.494 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.495 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.495 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.495 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.495 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.495 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.495 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.496 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.496 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.496 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.496 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.496 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.496 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.496 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.497 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.497 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.497 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.497 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.497 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.497 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.497 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.497 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.498 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.498 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.498 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.498 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.498 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.498 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.498 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.498 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.499 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.499 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.499 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.499 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.499 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.499 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.499 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.500 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.500 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.500 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.500 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.500 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.500 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.500 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.500 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.501 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.501 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.501 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.501 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.501 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.501 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.501 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.502 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.502 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.502 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.502 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.502 INFO:teuthology.orchestra.run.smithi040.stdout: "noautoscale": { 2023-07-18T03:42:08.502 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "noautoscale", 2023-07-18T03:42:08.502 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.502 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.503 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.503 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.503 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.503 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.503 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.503 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "global autoscale flag", 2023-07-18T03:42:08.503 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-18T03:42:08.503 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.504 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.504 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.504 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2023-07-18T03:42:08.504 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2023-07-18T03:42:08.504 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.504 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.504 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.505 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2023-07-18T03:42:08.505 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.505 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.505 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.505 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.505 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.505 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.505 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.506 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.506 INFO:teuthology.orchestra.run.smithi040.stdout: "threshold": { 2023-07-18T03:42:08.506 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "threshold", 2023-07-18T03:42:08.506 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:08.506 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.506 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.506 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3.0", 2023-07-18T03:42:08.506 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1.0", 2023-07-18T03:42:08.507 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.507 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.507 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "scaling threshold", 2023-07-18T03:42:08.507 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-18T03:42:08.507 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.507 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.507 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.508 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.508 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.508 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.508 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "progress", 2023-07-18T03:42:08.508 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.508 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.508 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.508 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_pg_recovery_event": { 2023-07-18T03:42:08.509 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_pg_recovery_event", 2023-07-18T03:42:08.509 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.509 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.509 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.509 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.509 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.509 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.509 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.510 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-18T03:42:08.510 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.510 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.510 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.510 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.510 INFO:teuthology.orchestra.run.smithi040.stdout: "enabled": { 2023-07-18T03:42:08.510 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enabled", 2023-07-18T03:42:08.510 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.511 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.511 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.511 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.511 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.511 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.511 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.511 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.512 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.512 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.512 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.512 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.512 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.512 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.512 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.512 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.513 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.513 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.513 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.513 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.513 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.513 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.513 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.513 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.514 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.514 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.514 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.514 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.514 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.514 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.515 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.515 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.515 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.515 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.515 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.515 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.515 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.515 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.516 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.516 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.516 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.516 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.516 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.516 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.516 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.516 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.517 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.517 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.517 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.517 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.517 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.517 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.517 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.518 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.518 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.518 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.518 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.518 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.518 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.518 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.518 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.519 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.519 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.519 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.519 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.519 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.519 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.519 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.519 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.520 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.520 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.520 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.520 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.520 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.520 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.520 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.521 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.521 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.521 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.521 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.521 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.521 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.521 INFO:teuthology.orchestra.run.smithi040.stdout: "max_completed_events": { 2023-07-18T03:42:08.521 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_completed_events", 2023-07-18T03:42:08.522 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.522 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.522 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.522 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "50", 2023-07-18T03:42:08.522 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.522 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.522 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.522 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "number of past completed events to remember", 2023-07-18T03:42:08.523 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.523 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.523 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.523 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.523 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2023-07-18T03:42:08.523 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2023-07-18T03:42:08.523 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.523 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.524 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.524 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2023-07-18T03:42:08.524 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.524 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.524 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.524 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long the module is going to sleep", 2023-07-18T03:42:08.524 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.525 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.525 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.525 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.525 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.525 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.525 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.525 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus", 2023-07-18T03:42:08.525 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.526 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.526 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.526 INFO:teuthology.orchestra.run.smithi040.stdout: "cache": { 2023-07-18T03:42:08.526 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cache", 2023-07-18T03:42:08.526 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.527 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.527 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.527 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.527 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.527 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.527 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.527 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.527 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.528 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.528 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.528 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.528 INFO:teuthology.orchestra.run.smithi040.stdout: "exclude_perf_counters": { 2023-07-18T03:42:08.528 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "exclude_perf_counters", 2023-07-18T03:42:08.528 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.528 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.529 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.529 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.529 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.529 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.529 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.529 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-18T03:42:08.529 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-18T03:42:08.529 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.530 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.530 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.530 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.530 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.530 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.530 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.530 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.530 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.531 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.531 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.531 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.531 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.531 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.532 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.532 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.532 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.532 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.532 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.532 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.532 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.533 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.533 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.533 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.533 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.533 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.533 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.533 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.534 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.534 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.534 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.534 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.534 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.534 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.534 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.534 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.535 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.535 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.535 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.535 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.535 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.535 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.535 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.535 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.536 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.536 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.536 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.536 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.536 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.536 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.536 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.537 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.537 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.537 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.537 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.537 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.537 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.537 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.537 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.538 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.538 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.538 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.538 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.538 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.538 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.538 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.538 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.539 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.539 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.539 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.539 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.539 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.539 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.539 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_stats_pools": { 2023-07-18T03:42:08.540 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_stats_pools", 2023-07-18T03:42:08.540 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.540 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.540 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.540 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.540 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.540 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.541 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.541 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.541 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.541 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.541 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.541 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.541 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-18T03:42:08.541 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-18T03:42:08.542 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.542 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.542 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.542 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "300", 2023-07-18T03:42:08.542 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.542 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.542 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.542 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.543 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.543 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.543 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.543 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.543 INFO:teuthology.orchestra.run.smithi040.stdout: "scrape_interval": { 2023-07-18T03:42:08.543 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "scrape_interval", 2023-07-18T03:42:08.543 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:08.543 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.544 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.544 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15.0", 2023-07-18T03:42:08.544 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.544 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.544 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.544 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.544 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.545 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.545 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.545 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.545 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2023-07-18T03:42:08.545 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2023-07-18T03:42:08.545 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.545 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.545 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "::", 2023-07-18T03:42:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-18T03:42:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2023-07-18T03:42:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2023-07-18T03:42:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "9283", 2023-07-18T03:42:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-18T03:42:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.549 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.549 INFO:teuthology.orchestra.run.smithi040.stdout: "stale_cache_strategy": { 2023-07-18T03:42:08.549 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "stale_cache_strategy", 2023-07-18T03:42:08.549 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.549 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.550 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.550 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "log", 2023-07-18T03:42:08.550 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.551 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.551 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.551 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.551 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.552 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.552 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.552 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.552 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_behaviour": { 2023-07-18T03:42:08.553 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_behaviour", 2023-07-18T03:42:08.553 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.553 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "default", 2023-07-18T03:42:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.555 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.555 INFO:teuthology.orchestra.run.smithi040.stdout: "default", 2023-07-18T03:42:08.555 INFO:teuthology.orchestra.run.smithi040.stdout: "error" 2023-07-18T03:42:08.555 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.556 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.556 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.556 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.557 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.557 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.557 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_error_status_code": { 2023-07-18T03:42:08.557 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_error_status_code", 2023-07-18T03:42:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "500", 2023-07-18T03:42:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "400", 2023-07-18T03:42:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "599", 2023-07-18T03:42:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_support", 2023-07-18T03:42:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.561 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.561 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.561 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.561 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.561 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.562 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.562 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.562 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.562 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.562 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.563 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.563 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.563 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.563 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.563 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.563 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.564 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.564 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.564 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.564 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.565 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.565 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.565 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.565 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.565 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.566 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.566 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.566 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.566 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.566 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.566 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.566 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.567 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.567 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.567 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.567 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.567 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.567 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.567 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.567 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.568 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.568 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.568 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.568 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.568 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.568 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.568 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.569 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.569 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.569 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.569 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.569 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.569 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.569 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.569 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.570 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.570 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.570 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.570 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.570 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.570 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.570 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.571 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.571 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.571 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.571 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.571 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.571 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.571 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.571 INFO:teuthology.orchestra.run.smithi040.stdout: "max_concurrent_snap_create": { 2023-07-18T03:42:08.572 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_concurrent_snap_create", 2023-07-18T03:42:08.572 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.572 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.572 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.572 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:08.572 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.572 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.573 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.573 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.573 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.573 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.573 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.573 INFO:teuthology.orchestra.run.smithi040.stdout: "mirror_snapshot_schedule": { 2023-07-18T03:42:08.574 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mirror_snapshot_schedule", 2023-07-18T03:42:08.574 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.574 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.574 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.574 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.574 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.574 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.574 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.575 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.575 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.575 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.575 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.575 INFO:teuthology.orchestra.run.smithi040.stdout: "trash_purge_schedule": { 2023-07-18T03:42:08.575 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "trash_purge_schedule", 2023-07-18T03:42:08.576 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.576 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.576 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.576 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.576 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.576 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.576 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.577 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.577 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.577 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.577 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.577 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.578 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.578 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.578 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "restful", 2023-07-18T03:42:08.578 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.578 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.578 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.578 INFO:teuthology.orchestra.run.smithi040.stdout: "enable_auth": { 2023-07-18T03:42:08.579 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enable_auth", 2023-07-18T03:42:08.579 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.579 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.579 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.579 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.579 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.579 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.579 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.580 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.580 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.580 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.580 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.580 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.580 INFO:teuthology.orchestra.run.smithi040.stdout: "key_file": { 2023-07-18T03:42:08.580 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "key_file", 2023-07-18T03:42:08.581 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.581 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.581 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.581 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.581 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.581 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.581 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.582 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.582 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.582 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.582 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.582 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.583 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.583 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.583 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.583 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.583 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.583 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.583 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.584 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.584 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.584 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.584 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.585 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.585 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.585 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.585 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.586 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.586 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.586 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.586 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.586 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.586 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.587 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.587 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.587 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.587 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.587 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.587 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.587 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.588 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.588 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.588 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.588 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.588 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.588 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.589 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.589 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.589 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.589 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.589 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.589 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.589 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.589 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.590 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.590 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.590 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.590 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.590 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.590 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.590 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.590 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.591 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.591 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.591 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.591 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.591 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.591 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.592 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.592 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.592 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.592 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.592 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.592 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.592 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.592 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.593 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.593 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.593 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2023-07-18T03:42:08.593 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2023-07-18T03:42:08.593 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.593 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.594 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.594 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.594 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.594 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.595 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.595 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.595 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.595 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.596 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.596 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2023-07-18T03:42:08.596 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2023-07-18T03:42:08.597 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.597 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.597 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.597 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.598 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.598 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.598 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.599 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.599 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.599 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.599 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.600 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.600 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.600 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.600 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rgw", 2023-07-18T03:42:08.600 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.600 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.601 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.601 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.601 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.601 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.601 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.601 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.601 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.602 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.602 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.602 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.602 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.602 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.602 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.602 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.602 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.603 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.603 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.603 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.603 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.603 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.603 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.604 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.604 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.604 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.604 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.604 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.604 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.604 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.605 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.605 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.605 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.605 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.605 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.605 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.606 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.606 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.607 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.607 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.607 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.607 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.608 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.608 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.608 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.609 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.609 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.609 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.609 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.610 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.610 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.610 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.611 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.611 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.611 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.611 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.612 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.612 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.613 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.613 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.613 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.613 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.613 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.614 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.614 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.614 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.614 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.615 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.615 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.615 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.616 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.616 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rook", 2023-07-18T03:42:08.616 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.616 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.616 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.617 INFO:teuthology.orchestra.run.smithi040.stdout: "drive_group_interval": { 2023-07-18T03:42:08.617 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "drive_group_interval", 2023-07-18T03:42:08.617 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:08.617 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.617 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.617 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "300.0", 2023-07-18T03:42:08.617 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.618 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.618 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.618 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-18T03:42:08.618 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.618 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.618 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.618 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.619 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.619 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.619 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.619 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.619 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.619 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.619 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.619 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.620 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.620 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.620 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.620 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.620 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.620 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.620 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.621 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.621 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.621 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.621 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.621 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.621 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.621 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.622 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.622 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.622 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.622 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.622 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.622 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.622 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.622 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.623 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.623 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.623 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.623 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.623 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.623 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.624 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.624 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.624 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.624 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.624 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.624 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.624 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.624 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.625 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.625 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.625 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.625 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.625 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.626 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.626 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.626 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.627 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.627 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.627 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.627 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.627 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.627 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.627 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.628 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.628 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.628 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.628 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.628 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.628 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.628 INFO:teuthology.orchestra.run.smithi040.stdout: "storage_class": { 2023-07-18T03:42:08.629 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "storage_class", 2023-07-18T03:42:08.629 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.629 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.629 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.629 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "local", 2023-07-18T03:42:08.629 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.629 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.629 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.630 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-18T03:42:08.630 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.630 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.630 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.630 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.630 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.631 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.631 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "selftest", 2023-07-18T03:42:08.631 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.631 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.631 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.631 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.631 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.631 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.632 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.632 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.632 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.632 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.632 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.632 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.632 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.632 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.633 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.633 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.633 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.633 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.633 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.633 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.633 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.634 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.634 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.634 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.634 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.634 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.634 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.634 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.635 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.635 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.635 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.635 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.635 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.635 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.635 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.636 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.636 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.636 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.636 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.636 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.636 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.636 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.637 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.637 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.637 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.637 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.637 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.637 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.637 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.637 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.638 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.638 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.638 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.638 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.638 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.638 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.639 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.639 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.639 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.639 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.639 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.639 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.639 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.639 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.640 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.640 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.640 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.640 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.640 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.640 INFO:teuthology.orchestra.run.smithi040.stdout: "roption1": { 2023-07-18T03:42:08.640 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "roption1", 2023-07-18T03:42:08.641 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.641 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.641 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.641 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.641 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.641 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.641 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.641 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.642 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.642 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.642 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.642 INFO:teuthology.orchestra.run.smithi040.stdout: "roption2": { 2023-07-18T03:42:08.642 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "roption2", 2023-07-18T03:42:08.642 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.642 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.643 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.643 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "xyz", 2023-07-18T03:42:08.643 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.643 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.643 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.643 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.643 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.644 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.644 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.644 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption1": { 2023-07-18T03:42:08.644 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption1", 2023-07-18T03:42:08.644 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.644 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.644 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.645 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.645 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.645 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.645 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.645 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.645 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.645 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.645 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.646 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption2": { 2023-07-18T03:42:08.646 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption2", 2023-07-18T03:42:08.646 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.646 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.646 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.646 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.646 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.647 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.647 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.647 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.647 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.647 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.647 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.647 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption3": { 2023-07-18T03:42:08.648 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption3", 2023-07-18T03:42:08.648 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:08.648 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.648 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.648 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.648 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.648 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.649 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.649 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.649 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.649 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.649 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.649 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption4": { 2023-07-18T03:42:08.649 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption4", 2023-07-18T03:42:08.650 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.650 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.650 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.650 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.650 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.650 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.650 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.650 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.651 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.651 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.651 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.651 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption5": { 2023-07-18T03:42:08.651 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption5", 2023-07-18T03:42:08.651 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.651 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.652 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.652 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.652 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.652 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.652 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.652 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.652 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.652 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.653 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.653 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.653 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption6": { 2023-07-18T03:42:08.653 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption6", 2023-07-18T03:42:08.653 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.653 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.653 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.654 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.654 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.654 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.654 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.654 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.654 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.654 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.654 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.655 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.655 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption7": { 2023-07-18T03:42:08.655 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption7", 2023-07-18T03:42:08.655 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.655 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.655 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.655 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.655 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2023-07-18T03:42:08.656 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "42", 2023-07-18T03:42:08.656 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.656 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.656 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.656 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.656 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.656 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.657 INFO:teuthology.orchestra.run.smithi040.stdout: "testkey": { 2023-07-18T03:42:08.657 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testkey", 2023-07-18T03:42:08.657 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.657 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.657 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.657 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.657 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.657 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.658 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.658 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.658 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.658 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.658 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.658 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.658 INFO:teuthology.orchestra.run.smithi040.stdout: "testlkey": { 2023-07-18T03:42:08.658 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testlkey", 2023-07-18T03:42:08.659 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.659 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.659 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.659 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.659 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.659 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.659 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.660 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.660 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.660 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.660 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.660 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.660 INFO:teuthology.orchestra.run.smithi040.stdout: "testnewline": { 2023-07-18T03:42:08.660 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testnewline", 2023-07-18T03:42:08.661 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.661 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.661 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.661 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.661 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.661 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.661 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.661 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.662 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.662 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.662 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.662 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.662 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.662 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.662 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.662 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "snap_schedule", 2023-07-18T03:42:08.663 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.663 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.663 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.663 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_m_granularity": { 2023-07-18T03:42:08.663 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_m_granularity", 2023-07-18T03:42:08.663 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.663 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.663 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.664 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.664 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.664 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.664 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.664 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow minute scheduled snapshots", 2023-07-18T03:42:08.664 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.664 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.665 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.665 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.665 INFO:teuthology.orchestra.run.smithi040.stdout: "dump_on_update": { 2023-07-18T03:42:08.665 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "dump_on_update", 2023-07-18T03:42:08.665 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.665 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.665 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.665 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.666 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.666 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.666 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.666 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "dump database to debug log on update", 2023-07-18T03:42:08.666 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.666 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.666 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.666 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.667 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.667 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.667 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.667 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.667 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.667 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.667 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.667 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.668 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.668 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.668 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.668 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.668 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.668 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.668 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.669 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.669 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.669 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.669 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.669 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.669 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.669 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.670 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.670 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.670 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.670 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.670 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.670 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.670 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.670 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.671 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.671 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.671 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.671 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.671 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.671 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.671 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.671 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.672 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.672 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.672 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.672 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.672 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.672 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.672 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.672 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.673 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.673 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.673 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.673 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.673 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.673 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.673 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.673 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.674 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.674 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.674 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.674 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.674 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.674 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.674 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.675 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.675 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.675 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.675 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.675 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.675 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.675 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.675 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.676 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.676 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.676 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.676 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "stats", 2023-07-18T03:42:08.676 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.676 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.676 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.676 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.677 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.677 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.677 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.677 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.677 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.677 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.677 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.677 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.678 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.678 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.678 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.678 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.678 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.678 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.678 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.678 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.679 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.679 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.679 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.679 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.679 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.679 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.679 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.679 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.680 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.680 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.680 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.680 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.680 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.680 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.680 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.681 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.681 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.681 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.681 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.681 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.681 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.681 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.681 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.682 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.682 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.682 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.682 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.682 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.682 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.682 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.682 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.683 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.683 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.683 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.683 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.683 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.683 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.683 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.683 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.684 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.684 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.684 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.684 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.684 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.684 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.684 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.685 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.685 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.685 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.685 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.685 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.685 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.685 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.685 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.686 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.686 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.686 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "status", 2023-07-18T03:42:08.686 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.686 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.686 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.686 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.686 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.687 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.687 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.687 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.687 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.687 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.687 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.687 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.688 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.688 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.688 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.688 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.688 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.688 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.688 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.688 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.689 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.689 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.689 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.689 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.689 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.689 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.689 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.689 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.690 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.690 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.690 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.690 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.690 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.690 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.690 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.690 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.691 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.691 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.691 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.691 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.691 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.691 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.691 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.691 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.692 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.692 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.692 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.692 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.692 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.692 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.692 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.693 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.693 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.693 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.693 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.693 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.693 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.693 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.693 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.694 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.694 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.694 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.694 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.694 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.694 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.694 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.694 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.695 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.695 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.695 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.695 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.695 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.695 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.695 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.695 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.696 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.696 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "telegraf", 2023-07-18T03:42:08.696 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.696 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.696 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.696 INFO:teuthology.orchestra.run.smithi040.stdout: "address": { 2023-07-18T03:42:08.696 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "address", 2023-07-18T03:42:08.696 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.697 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.697 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.697 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-18T03:42:08.697 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.697 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.697 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.697 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.698 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.698 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.698 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.698 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.698 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2023-07-18T03:42:08.698 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2023-07-18T03:42:08.698 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.698 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.699 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.699 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15", 2023-07-18T03:42:08.699 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.699 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.699 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.699 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.699 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.699 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.700 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.700 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.700 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.700 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.700 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.700 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.700 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.701 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.701 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.701 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.701 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.701 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.701 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.701 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.701 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.702 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.702 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.702 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.702 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.702 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.702 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.702 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.702 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.703 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.703 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.703 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.703 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.703 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.703 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.703 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.703 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.704 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.704 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.704 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.704 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.704 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.704 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.704 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.705 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.705 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.705 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.705 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.705 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.705 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.705 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.705 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.706 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.706 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.706 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.706 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.706 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.706 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.706 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.706 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.707 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.707 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.707 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.707 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.707 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.707 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.707 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.707 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.708 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.708 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.708 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.708 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.708 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.708 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.708 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.709 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.709 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.709 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.709 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.709 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.709 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "telemetry", 2023-07-18T03:42:08.709 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.710 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.710 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.710 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_basic": { 2023-07-18T03:42:08.710 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_basic", 2023-07-18T03:42:08.710 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.710 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.710 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.710 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.711 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.711 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.711 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.711 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-18T03:42:08.711 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.711 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.711 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.711 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.712 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_crash": { 2023-07-18T03:42:08.712 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_crash", 2023-07-18T03:42:08.712 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.712 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.712 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.712 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.712 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.712 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.713 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.713 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-18T03:42:08.713 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.713 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.713 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.713 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.713 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_device": { 2023-07-18T03:42:08.714 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_device", 2023-07-18T03:42:08.714 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.714 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.714 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.714 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.714 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.714 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.714 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.715 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-18T03:42:08.715 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.715 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.715 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.715 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.715 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_ident": { 2023-07-18T03:42:08.715 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_ident", 2023-07-18T03:42:08.715 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.716 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.716 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.716 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.716 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.716 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.716 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.716 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-18T03:42:08.716 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.717 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.717 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.717 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.717 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_perf": { 2023-07-18T03:42:08.717 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_perf", 2023-07-18T03:42:08.717 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.717 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.718 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.718 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.718 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.718 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.718 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.718 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-18T03:42:08.718 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.718 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.719 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.719 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.719 INFO:teuthology.orchestra.run.smithi040.stdout: "contact": { 2023-07-18T03:42:08.719 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "contact", 2023-07-18T03:42:08.719 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.719 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.719 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.719 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.720 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.720 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.720 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.720 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.720 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.720 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.720 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.720 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.721 INFO:teuthology.orchestra.run.smithi040.stdout: "description": { 2023-07-18T03:42:08.721 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "description", 2023-07-18T03:42:08.721 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.721 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.721 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.721 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.721 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.722 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.722 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.722 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.722 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.722 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.722 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.722 INFO:teuthology.orchestra.run.smithi040.stdout: "device_url": { 2023-07-18T03:42:08.723 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_url", 2023-07-18T03:42:08.723 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.723 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.723 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.723 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-18T03:42:08.723 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.723 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.723 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.724 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.724 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.724 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.724 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.724 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.724 INFO:teuthology.orchestra.run.smithi040.stdout: "enabled": { 2023-07-18T03:42:08.724 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enabled", 2023-07-18T03:42:08.724 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2023-07-18T03:42:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2023-07-18T03:42:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "24", 2023-07-18T03:42:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "8", 2023-07-18T03:42:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: "last_opt_revision": { 2023-07-18T03:42:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "last_opt_revision", 2023-07-18T03:42:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1", 2023-07-18T03:42:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: "leaderboard": { 2023-07-18T03:42:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "leaderboard", 2023-07-18T03:42:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: "leaderboard_description": { 2023-07-18T03:42:08.732 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "leaderboard_description", 2023-07-18T03:42:08.732 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.732 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.732 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.732 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.732 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.732 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.732 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.733 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.733 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.733 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.733 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.733 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.733 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.733 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.734 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.734 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.734 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.734 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.734 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.734 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.734 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.734 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.735 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.735 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.735 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.735 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.735 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.735 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.735 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.735 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.736 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.736 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.736 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.736 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.736 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.736 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.736 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.736 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.737 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.737 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.737 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.737 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.737 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.737 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.737 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.738 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.738 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.738 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.738 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.738 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.738 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.738 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.738 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.739 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.739 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.739 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.739 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.739 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.739 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.739 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.739 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.740 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.740 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.740 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.740 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.740 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.740 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.740 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.740 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.741 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.741 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.741 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.741 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.741 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.741 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.741 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.741 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.742 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.742 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.742 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.742 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.742 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.742 INFO:teuthology.orchestra.run.smithi040.stdout: "organization": { 2023-07-18T03:42:08.742 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "organization", 2023-07-18T03:42:08.742 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.743 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.743 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.743 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.743 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.743 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.743 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.743 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.744 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.744 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.744 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.744 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.744 INFO:teuthology.orchestra.run.smithi040.stdout: "proxy": { 2023-07-18T03:42:08.744 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "proxy", 2023-07-18T03:42:08.744 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.744 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.745 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.745 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.745 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.745 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.745 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.745 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.745 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.745 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.746 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.746 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.746 INFO:teuthology.orchestra.run.smithi040.stdout: "url": { 2023-07-18T03:42:08.746 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "url", 2023-07-18T03:42:08.746 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.746 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.746 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.746 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-18T03:42:08.747 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.747 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.747 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.747 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.747 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.747 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.747 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.747 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.748 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.748 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.748 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.748 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "test_orchestrator", 2023-07-18T03:42:08.748 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.748 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.748 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.749 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.749 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.749 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.749 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.749 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.749 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.749 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.749 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.750 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.750 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.750 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.750 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.750 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.750 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.750 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.750 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.751 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.751 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.751 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.751 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.751 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.751 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.751 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.752 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.752 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.752 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.752 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.752 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.752 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.752 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.752 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.753 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.753 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.753 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.753 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.753 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.753 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.753 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.753 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.754 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.754 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.754 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.754 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.754 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.754 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.754 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.754 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.755 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.755 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.755 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.755 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.756 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.756 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.756 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.756 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.756 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.756 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.756 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.757 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.757 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.757 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.757 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.757 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.757 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.757 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.757 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.758 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.758 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.758 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.758 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.758 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.758 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.758 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "volumes", 2023-07-18T03:42:08.759 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.759 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.759 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.759 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.759 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.759 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.759 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.760 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.760 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.760 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.760 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.760 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.760 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.760 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.761 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.761 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.761 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.761 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.761 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.761 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.761 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.761 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.762 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.762 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.762 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.762 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.762 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.762 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.762 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.762 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.763 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.763 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.763 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.763 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.763 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.763 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.763 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.763 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.764 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.764 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.764 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.764 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.764 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.764 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.764 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.765 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.765 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.765 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.765 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.765 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.765 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.765 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.766 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.766 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.766 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.766 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.766 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.766 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.766 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.766 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.767 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.767 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.767 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.767 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.767 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.767 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.767 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.767 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.768 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.768 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.768 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.768 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.768 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.768 INFO:teuthology.orchestra.run.smithi040.stdout: "max_concurrent_clones": { 2023-07-18T03:42:08.768 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_concurrent_clones", 2023-07-18T03:42:08.769 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.769 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.769 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.769 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "4", 2023-07-18T03:42:08.769 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.769 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.769 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.769 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-18T03:42:08.770 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.770 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.770 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.770 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.770 INFO:teuthology.orchestra.run.smithi040.stdout: "snapshot_clone_delay": { 2023-07-18T03:42:08.770 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "snapshot_clone_delay", 2023-07-18T03:42:08.770 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.770 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.771 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.771 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2023-07-18T03:42:08.771 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.771 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.771 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.771 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-18T03:42:08.771 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.771 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.772 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.772 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.772 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.772 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.772 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.772 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix", 2023-07-18T03:42:08.772 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.773 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.773 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.773 INFO:teuthology.orchestra.run.smithi040.stdout: "discovery_interval": { 2023-07-18T03:42:08.773 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "discovery_interval", 2023-07-18T03:42:08.773 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2023-07-18T03:42:08.774 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.774 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.774 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "100", 2023-07-18T03:42:08.774 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.774 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.774 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.775 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.775 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.775 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.775 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.775 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.775 INFO:teuthology.orchestra.run.smithi040.stdout: "identifier": { 2023-07-18T03:42:08.775 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "identifier", 2023-07-18T03:42:08.775 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.776 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.776 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.776 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.776 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.776 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.776 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.776 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.777 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.777 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.777 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.777 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.777 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2023-07-18T03:42:08.777 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2023-07-18T03:42:08.777 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.777 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.778 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.778 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2023-07-18T03:42:08.778 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.778 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.778 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.778 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.778 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.778 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.779 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.779 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.779 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.779 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.779 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.779 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.779 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.780 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.780 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.780 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.780 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.780 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.780 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.780 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.780 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.781 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.781 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.781 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.781 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.781 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.781 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.781 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.781 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.782 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.782 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.782 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.782 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.782 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.782 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.782 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.782 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.783 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.783 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.783 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.783 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.783 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.783 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.783 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.783 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.784 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.784 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.784 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.784 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.784 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.784 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.784 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.785 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.785 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.785 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.785 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.785 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.785 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.785 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.786 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.786 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.786 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.786 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.786 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.786 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.786 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.787 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.787 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.787 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.787 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.787 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.787 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.787 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.787 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.788 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.788 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.788 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_host": { 2023-07-18T03:42:08.788 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_host", 2023-07-18T03:42:08.788 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.788 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.789 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.789 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.789 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.789 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.789 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.789 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.789 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.789 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.790 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.790 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_port": { 2023-07-18T03:42:08.790 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_port", 2023-07-18T03:42:08.790 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.790 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.790 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.790 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10051", 2023-07-18T03:42:08.791 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.791 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.791 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.791 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.791 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.791 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.791 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.792 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_sender": { 2023-07-18T03:42:08.792 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_sender", 2023-07-18T03:42:08.792 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.792 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.792 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.792 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-18T03:42:08.792 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.793 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.793 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.793 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.793 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.793 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.793 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.793 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.793 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.794 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.794 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-07-18T03:42:08.794 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.794 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.794 INFO:teuthology.orchestra.run.smithi040.stdout: "modules": [ 2023-07-18T03:42:08.794 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm", 2023-07-18T03:42:08.794 INFO:teuthology.orchestra.run.smithi040.stdout: "dashboard", 2023-07-18T03:42:08.794 INFO:teuthology.orchestra.run.smithi040.stdout: "iostat", 2023-07-18T03:42:08.795 INFO:teuthology.orchestra.run.smithi040.stdout: "nfs", 2023-07-18T03:42:08.795 INFO:teuthology.orchestra.run.smithi040.stdout: "restful" 2023-07-18T03:42:08.795 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.795 INFO:teuthology.orchestra.run.smithi040.stdout: "available_modules": [ 2023-07-18T03:42:08.795 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.795 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "alerts", 2023-07-18T03:42:08.795 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.795 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.796 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.796 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2023-07-18T03:42:08.796 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2023-07-18T03:42:08.796 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.796 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.796 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.796 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2023-07-18T03:42:08.797 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.797 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.797 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.797 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How frequently to reexamine health status", 2023-07-18T03:42:08.797 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.797 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.797 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.797 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.798 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.798 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.798 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.798 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.798 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.798 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.798 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.798 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.799 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.799 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.799 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.799 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.799 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.799 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.799 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.800 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.800 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.800 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.800 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.800 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.800 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.800 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.800 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.801 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.801 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.801 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.801 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.801 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.801 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.801 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.801 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.802 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.802 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.802 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.802 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.802 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.802 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.802 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.802 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.803 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.803 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.803 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.803 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.803 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.803 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.803 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.804 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.804 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.804 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.804 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.804 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.804 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.804 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.804 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.805 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.805 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.805 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.805 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.805 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.805 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.805 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.805 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.806 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.806 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.806 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.806 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.806 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.806 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.806 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.807 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.807 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_destination": { 2023-07-18T03:42:08.807 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_destination", 2023-07-18T03:42:08.807 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.807 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.807 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.807 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.807 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.808 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.808 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.808 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Email address to send alerts to", 2023-07-18T03:42:08.808 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.808 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.808 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.808 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.809 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_from_name": { 2023-07-18T03:42:08.809 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_from_name", 2023-07-18T03:42:08.809 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.809 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.809 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.809 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "Ceph", 2023-07-18T03:42:08.809 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.809 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.810 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.810 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Email From: name", 2023-07-18T03:42:08.810 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.810 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.810 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.810 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.810 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_host": { 2023-07-18T03:42:08.810 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_host", 2023-07-18T03:42:08.811 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.811 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.811 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.811 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.811 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.811 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.811 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.812 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP server", 2023-07-18T03:42:08.812 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.812 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.812 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.812 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.812 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_password": { 2023-07-18T03:42:08.812 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_password", 2023-07-18T03:42:08.812 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.813 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.813 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.813 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.813 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.813 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.813 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.813 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Password to authenticate with", 2023-07-18T03:42:08.814 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.814 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.814 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.814 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_port": { 2023-07-18T03:42:08.814 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_port", 2023-07-18T03:42:08.814 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.815 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.815 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.815 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "465", 2023-07-18T03:42:08.815 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.815 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.815 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.815 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP port", 2023-07-18T03:42:08.816 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.816 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.816 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.816 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.816 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_sender": { 2023-07-18T03:42:08.816 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_sender", 2023-07-18T03:42:08.816 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.816 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.817 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.817 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.817 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.817 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.817 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.817 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SMTP envelope sender", 2023-07-18T03:42:08.817 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.818 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.818 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.818 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.818 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_ssl": { 2023-07-18T03:42:08.818 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_ssl", 2023-07-18T03:42:08.818 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.818 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.818 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.819 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.819 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.819 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.819 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.819 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-18T03:42:08.819 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.819 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.820 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.820 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.820 INFO:teuthology.orchestra.run.smithi040.stdout: "smtp_user": { 2023-07-18T03:42:08.820 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "smtp_user", 2023-07-18T03:42:08.820 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.820 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.820 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.820 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.821 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.821 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.821 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.821 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "User to authenticate as", 2023-07-18T03:42:08.821 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.821 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.821 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.822 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.822 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.822 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.822 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.822 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "balancer", 2023-07-18T03:42:08.822 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.822 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.822 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.823 INFO:teuthology.orchestra.run.smithi040.stdout: "active": { 2023-07-18T03:42:08.823 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "active", 2023-07-18T03:42:08.823 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.823 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.823 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.823 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.823 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.823 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.824 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.824 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "automatically balance PGs across cluster", 2023-07-18T03:42:08.824 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.824 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.824 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.824 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.824 INFO:teuthology.orchestra.run.smithi040.stdout: "begin_time": { 2023-07-18T03:42:08.825 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "begin_time", 2023-07-18T03:42:08.825 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.825 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.825 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.825 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0000", 2023-07-18T03:42:08.825 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.825 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.826 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.826 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "beginning time of day to automatically balance", 2023-07-18T03:42:08.826 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-18T03:42:08.826 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.826 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.826 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.826 INFO:teuthology.orchestra.run.smithi040.stdout: "begin_weekday": { 2023-07-18T03:42:08.826 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "begin_weekday", 2023-07-18T03:42:08.827 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2023-07-18T03:42:08.827 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.827 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.827 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2023-07-18T03:42:08.827 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0", 2023-07-18T03:42:08.827 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "6", 2023-07-18T03:42:08.827 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.827 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-18T03:42:08.828 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-18T03:42:08.828 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.828 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.828 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.828 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_max_iterations": { 2023-07-18T03:42:08.828 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_max_iterations", 2023-07-18T03:42:08.828 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2023-07-18T03:42:08.829 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.829 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.829 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "25", 2023-07-18T03:42:08.829 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2023-07-18T03:42:08.829 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "250", 2023-07-18T03:42:08.829 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.829 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-18T03:42:08.829 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.830 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.830 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.830 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.830 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_metrics": { 2023-07-18T03:42:08.830 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_metrics", 2023-07-18T03:42:08.830 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.830 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.831 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.831 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "pgs,objects,bytes", 2023-07-18T03:42:08.831 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.831 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.831 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.831 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-18T03:42:08.831 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-18T03:42:08.831 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.832 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.832 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.832 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_compat_step": { 2023-07-18T03:42:08.832 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crush_compat_step", 2023-07-18T03:42:08.832 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:08.832 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.832 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.833 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0.5", 2023-07-18T03:42:08.833 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0.001", 2023-07-18T03:42:08.833 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "0.999", 2023-07-18T03:42:08.833 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.833 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "aggressiveness of optimization", 2023-07-18T03:42:08.833 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-18T03:42:08.833 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.834 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.834 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.834 INFO:teuthology.orchestra.run.smithi040.stdout: "end_time": { 2023-07-18T03:42:08.834 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "end_time", 2023-07-18T03:42:08.834 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.834 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.834 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.834 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "2359", 2023-07-18T03:42:08.835 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.835 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.835 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.835 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "ending time of day to automatically balance", 2023-07-18T03:42:08.835 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-18T03:42:08.835 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.835 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.836 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.836 INFO:teuthology.orchestra.run.smithi040.stdout: "end_weekday": { 2023-07-18T03:42:08.836 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "end_weekday", 2023-07-18T03:42:08.836 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2023-07-18T03:42:08.836 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.836 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.836 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2023-07-18T03:42:08.837 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "0", 2023-07-18T03:42:08.837 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "6", 2023-07-18T03:42:08.837 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.837 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-18T03:42:08.837 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-18T03:42:08.837 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.837 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.837 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.838 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.838 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.838 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.838 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.838 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.838 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.838 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.839 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.839 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.839 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.839 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.839 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.839 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.839 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.839 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.840 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.840 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.840 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.840 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.840 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.840 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.840 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.841 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.841 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.841 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.841 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.841 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.841 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.841 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.842 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.842 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.842 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.842 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.842 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.842 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.842 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.843 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.843 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.843 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.843 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.843 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.843 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.843 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.844 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.844 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.844 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.844 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.844 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.844 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.844 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.845 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.845 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.845 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.845 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.845 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.845 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.845 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.845 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.846 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.846 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.846 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.846 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.846 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.846 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.846 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.847 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.847 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.847 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.847 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.847 INFO:teuthology.orchestra.run.smithi040.stdout: "min_score": { 2023-07-18T03:42:08.847 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "min_score", 2023-07-18T03:42:08.847 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:08.848 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.848 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.848 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2023-07-18T03:42:08.848 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.848 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.848 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.848 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-18T03:42:08.849 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.849 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.849 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.854 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.855 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": { 2023-07-18T03:42:08.855 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mode", 2023-07-18T03:42:08.855 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.855 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.856 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.856 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "upmap", 2023-07-18T03:42:08.856 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.856 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.856 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.856 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-compat", 2023-07-18T03:42:08.856 INFO:teuthology.orchestra.run.smithi040.stdout: "none", 2023-07-18T03:42:08.857 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap" 2023-07-18T03:42:08.857 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.857 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Balancer mode", 2023-07-18T03:42:08.857 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.857 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.857 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.858 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_ids": { 2023-07-18T03:42:08.858 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pool_ids", 2023-07-18T03:42:08.858 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.858 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.858 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.858 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.858 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.858 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.859 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.859 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-18T03:42:08.859 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.859 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.859 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.859 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.859 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2023-07-18T03:42:08.860 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2023-07-18T03:42:08.860 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.860 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.860 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.860 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2023-07-18T03:42:08.860 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.860 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.861 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.861 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-18T03:42:08.861 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.861 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.861 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.861 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.861 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap_max_deviation": { 2023-07-18T03:42:08.862 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "upmap_max_deviation", 2023-07-18T03:42:08.862 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.862 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.862 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.862 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2023-07-18T03:42:08.862 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2023-07-18T03:42:08.862 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.863 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.863 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-18T03:42:08.863 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-18T03:42:08.863 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.863 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.863 INFO:teuthology.orchestra.run.smithi040.stdout: "upmap_max_optimizations": { 2023-07-18T03:42:08.863 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "upmap_max_optimizations", 2023-07-18T03:42:08.864 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2023-07-18T03:42:08.864 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.864 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.864 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:08.864 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.864 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.864 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.865 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-18T03:42:08.865 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.865 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.865 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.865 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.865 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.866 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.866 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cephadm", 2023-07-18T03:42:08.866 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.866 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.866 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.866 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_down_multiplier": { 2023-07-18T03:42:08.866 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_down_multiplier", 2023-07-18T03:42:08.867 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:08.867 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.867 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.867 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3.0", 2023-07-18T03:42:08.867 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.867 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.867 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.868 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-18T03:42:08.868 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.868 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.868 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.868 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.868 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_refresh_rate": { 2023-07-18T03:42:08.868 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_refresh_rate", 2023-07-18T03:42:08.869 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.869 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.869 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.869 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "20", 2023-07-18T03:42:08.869 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.869 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.870 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.870 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-18T03:42:08.870 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.870 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.870 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.870 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.870 INFO:teuthology.orchestra.run.smithi040.stdout: "agent_starting_port": { 2023-07-18T03:42:08.871 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "agent_starting_port", 2023-07-18T03:42:08.871 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.871 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.871 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.871 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "4721", 2023-07-18T03:42:08.871 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.871 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.871 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.872 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-18T03:42:08.872 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.872 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.872 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.872 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.872 INFO:teuthology.orchestra.run.smithi040.stdout: "alertmanager_web_password": { 2023-07-18T03:42:08.872 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "alertmanager_web_password", 2023-07-18T03:42:08.873 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.873 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.873 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.873 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:08.873 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.873 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.873 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.873 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Alertmanager web password", 2023-07-18T03:42:08.874 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.874 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.874 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.874 INFO:teuthology.orchestra.run.smithi040.stdout: "alertmanager_web_user": { 2023-07-18T03:42:08.874 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "alertmanager_web_user", 2023-07-18T03:42:08.874 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.874 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.875 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.875 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:08.875 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.875 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.875 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.875 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Alertmanager web user", 2023-07-18T03:42:08.875 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.876 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.876 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.876 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.876 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_ptrace": { 2023-07-18T03:42:08.876 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_ptrace", 2023-07-18T03:42:08.876 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.876 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.876 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.877 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.877 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.877 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.877 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.877 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-18T03:42:08.877 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-18T03:42:08.877 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.878 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.878 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.878 INFO:teuthology.orchestra.run.smithi040.stdout: "autotune_interval": { 2023-07-18T03:42:08.878 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "autotune_interval", 2023-07-18T03:42:08.878 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.878 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.878 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.879 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2023-07-18T03:42:08.879 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.879 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.879 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.879 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-18T03:42:08.879 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.879 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.879 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.880 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.880 INFO:teuthology.orchestra.run.smithi040.stdout: "autotune_memory_target_ratio": { 2023-07-18T03:42:08.880 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "autotune_memory_target_ratio", 2023-07-18T03:42:08.880 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:08.880 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.880 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.880 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0.7", 2023-07-18T03:42:08.881 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.881 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.881 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.881 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-18T03:42:08.881 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.881 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.881 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.882 INFO:teuthology.orchestra.run.smithi040.stdout: "cgroups_split": { 2023-07-18T03:42:08.882 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cgroups_split", 2023-07-18T03:42:08.882 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.882 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.882 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.882 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.882 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.883 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.883 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.883 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-18T03:42:08.883 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.883 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.883 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.883 INFO:teuthology.orchestra.run.smithi040.stdout: "config_checks_enabled": { 2023-07-18T03:42:08.884 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_checks_enabled", 2023-07-18T03:42:08.884 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.884 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.884 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.884 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.884 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.884 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.885 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.885 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-18T03:42:08.885 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.885 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.885 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.885 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.885 INFO:teuthology.orchestra.run.smithi040.stdout: "config_dashboard": { 2023-07-18T03:42:08.885 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_dashboard", 2023-07-18T03:42:08.886 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.886 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.886 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.886 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.886 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.886 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.886 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.887 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-18T03:42:08.887 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.887 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.887 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.887 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.887 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_alertmanager": { 2023-07-18T03:42:08.887 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_alertmanager", 2023-07-18T03:42:08.887 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.888 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.888 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.888 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-18T03:42:08.888 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.888 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.888 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.888 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2023-07-18T03:42:08.889 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.889 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.889 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.889 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.889 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_base": { 2023-07-18T03:42:08.889 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_base", 2023-07-18T03:42:08.889 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.889 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.890 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.890 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-18T03:42:08.890 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.890 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.890 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.890 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Container image name, without the tag", 2023-07-18T03:42:08.890 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.891 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.891 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.891 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.891 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_elasticsearch": { 2023-07-18T03:42:08.891 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_elasticsearch", 2023-07-18T03:42:08.892 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.892 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.892 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.892 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-18T03:42:08.892 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.892 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.892 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.893 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "elasticsearch container image", 2023-07-18T03:42:08.893 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.893 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.893 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.893 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.893 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_grafana": { 2023-07-18T03:42:08.894 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_grafana", 2023-07-18T03:42:08.894 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.894 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.894 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.894 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-18T03:42:08.894 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.894 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.894 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.895 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2023-07-18T03:42:08.895 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.895 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.895 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.895 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.895 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_haproxy": { 2023-07-18T03:42:08.895 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_haproxy", 2023-07-18T03:42:08.895 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.896 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.896 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.896 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-18T03:42:08.896 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.896 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.896 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.896 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "HAproxy container image", 2023-07-18T03:42:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_agent": { 2023-07-18T03:42:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_agent", 2023-07-18T03:42:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.898 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.898 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-18T03:42:08.898 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.898 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.898 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.898 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger agent container image", 2023-07-18T03:42:08.898 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.898 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.899 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.899 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.899 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_collector": { 2023-07-18T03:42:08.899 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_collector", 2023-07-18T03:42:08.899 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.899 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.899 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-18T03:42:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger collector container image", 2023-07-18T03:42:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_jaeger_query": { 2023-07-18T03:42:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_jaeger_query", 2023-07-18T03:42:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-18T03:42:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Jaeger query container image", 2023-07-18T03:42:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.903 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.903 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.903 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.904 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_keepalived": { 2023-07-18T03:42:08.904 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_keepalived", 2023-07-18T03:42:08.904 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.904 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.905 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.905 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-18T03:42:08.905 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.906 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.906 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.906 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Keepalived container image", 2023-07-18T03:42:08.906 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.907 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.907 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.908 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_loki": { 2023-07-18T03:42:08.908 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_loki", 2023-07-18T03:42:08.908 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.908 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.909 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.909 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-18T03:42:08.909 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.910 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.910 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.910 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Loki container image", 2023-07-18T03:42:08.910 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.911 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.911 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.911 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.911 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_node_exporter": { 2023-07-18T03:42:08.911 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_node_exporter", 2023-07-18T03:42:08.912 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.912 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.912 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.913 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-18T03:42:08.913 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.913 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.913 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.913 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2023-07-18T03:42:08.914 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.914 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.914 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.914 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.915 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_prometheus": { 2023-07-18T03:42:08.915 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_prometheus", 2023-07-18T03:42:08.915 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.915 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.915 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.915 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-18T03:42:08.915 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.916 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.916 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.916 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus container image", 2023-07-18T03:42:08.916 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.916 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.916 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.916 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.916 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_promtail": { 2023-07-18T03:42:08.917 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_promtail", 2023-07-18T03:42:08.917 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.917 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.917 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.917 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-18T03:42:08.917 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.917 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.918 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.918 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Promtail container image", 2023-07-18T03:42:08.918 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.918 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.918 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.918 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.918 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image_snmp_gateway": { 2023-07-18T03:42:08.918 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_image_snmp_gateway", 2023-07-18T03:42:08.919 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.919 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.919 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.919 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-18T03:42:08.919 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.919 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.919 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.919 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "SNMP Gateway container image", 2023-07-18T03:42:08.920 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.920 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.920 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.920 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.920 INFO:teuthology.orchestra.run.smithi040.stdout: "container_init": { 2023-07-18T03:42:08.920 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "container_init", 2023-07-18T03:42:08.920 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.921 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.921 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.921 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.921 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.921 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.921 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.921 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Run podman/docker with `--init`", 2023-07-18T03:42:08.921 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.922 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.922 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.922 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.922 INFO:teuthology.orchestra.run.smithi040.stdout: "daemon_cache_timeout": { 2023-07-18T03:42:08.922 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "daemon_cache_timeout", 2023-07-18T03:42:08.922 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.922 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.922 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.923 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2023-07-18T03:42:08.923 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.923 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.923 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.923 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-18T03:42:08.923 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.923 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.924 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.924 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.924 INFO:teuthology.orchestra.run.smithi040.stdout: "default_cephadm_command_timeout": { 2023-07-18T03:42:08.924 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "default_cephadm_command_timeout", 2023-07-18T03:42:08.924 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.924 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.924 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.924 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "900", 2023-07-18T03:42:08.925 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.925 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.925 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.925 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-18T03:42:08.925 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.925 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.925 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.925 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.926 INFO:teuthology.orchestra.run.smithi040.stdout: "default_registry": { 2023-07-18T03:42:08.926 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "default_registry", 2023-07-18T03:42:08.926 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.926 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.926 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.926 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "docker.io", 2023-07-18T03:42:08.926 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.927 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.927 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.927 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-18T03:42:08.927 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.927 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.927 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.927 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.927 INFO:teuthology.orchestra.run.smithi040.stdout: "device_cache_timeout": { 2023-07-18T03:42:08.928 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_cache_timeout", 2023-07-18T03:42:08.928 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.928 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.928 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.928 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1800", 2023-07-18T03:42:08.928 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.928 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.929 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.929 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache device inventory", 2023-07-18T03:42:08.929 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.929 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.929 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.929 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.929 INFO:teuthology.orchestra.run.smithi040.stdout: "device_enhanced_scan": { 2023-07-18T03:42:08.929 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_enhanced_scan", 2023-07-18T03:42:08.930 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.930 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.930 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.930 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.930 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.930 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.930 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.930 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-18T03:42:08.931 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.931 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.931 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.931 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.931 INFO:teuthology.orchestra.run.smithi040.stdout: "facts_cache_timeout": { 2023-07-18T03:42:08.931 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "facts_cache_timeout", 2023-07-18T03:42:08.931 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.931 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.932 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.932 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2023-07-18T03:42:08.932 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.932 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.932 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.932 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "seconds to cache host facts data", 2023-07-18T03:42:08.932 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.933 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.933 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.933 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.933 INFO:teuthology.orchestra.run.smithi040.stdout: "host_check_interval": { 2023-07-18T03:42:08.933 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "host_check_interval", 2023-07-18T03:42:08.933 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.933 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.933 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.934 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2023-07-18T03:42:08.934 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.934 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.934 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.934 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to perform a host check", 2023-07-18T03:42:08.934 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.934 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.934 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.935 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.935 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.935 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.935 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.935 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.935 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.935 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.935 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.936 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.936 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.936 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.936 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.936 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.936 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.936 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.936 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.937 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.937 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.937 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.937 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.937 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.937 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.937 INFO:teuthology.orchestra.run.smithi040.stdout: "log_refresh_metadata": { 2023-07-18T03:42:08.938 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_refresh_metadata", 2023-07-18T03:42:08.938 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.938 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.938 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.938 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.938 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.938 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.938 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.939 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-18T03:42:08.939 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.939 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.939 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.939 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.939 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.939 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.940 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.940 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.940 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.940 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.940 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.940 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.940 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.940 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-18T03:42:08.941 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.941 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.941 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.941 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.941 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.941 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.941 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.941 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.942 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.942 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.942 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.942 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.942 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.942 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.942 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.942 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.943 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.943 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.943 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.943 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.943 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.943 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.943 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.944 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.944 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.944 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.944 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.944 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.944 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.944 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.944 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.945 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.945 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.945 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.945 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.945 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.945 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.945 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.945 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.946 INFO:teuthology.orchestra.run.smithi040.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-18T03:42:08.946 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-18T03:42:08.946 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.946 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.946 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.946 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.947 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.947 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.947 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.947 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-18T03:42:08.947 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.947 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.947 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.948 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.948 INFO:teuthology.orchestra.run.smithi040.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-18T03:42:08.948 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-18T03:42:08.948 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.948 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.948 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.948 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "*", 2023-07-18T03:42:08.948 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.949 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.949 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.949 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-18T03:42:08.949 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.949 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.949 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.949 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.950 INFO:teuthology.orchestra.run.smithi040.stdout: "max_count_per_host": { 2023-07-18T03:42:08.950 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_count_per_host", 2023-07-18T03:42:08.950 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.950 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.950 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.950 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:08.950 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.950 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.951 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.951 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "max number of daemons per service per host", 2023-07-18T03:42:08.951 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.951 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.951 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.951 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.951 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd_draining_count": { 2023-07-18T03:42:08.952 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_osd_draining_count", 2023-07-18T03:42:08.952 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.952 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.952 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.952 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:08.952 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.952 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.952 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.953 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-18T03:42:08.953 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.953 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.953 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.953 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.953 INFO:teuthology.orchestra.run.smithi040.stdout: "migration_current": { 2023-07-18T03:42:08.953 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "migration_current", 2023-07-18T03:42:08.953 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.954 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.954 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.954 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.954 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.954 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.954 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.954 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "internal - do not modify", 2023-07-18T03:42:08.955 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.955 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.955 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.955 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.955 INFO:teuthology.orchestra.run.smithi040.stdout: "mode": { 2023-07-18T03:42:08.955 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mode", 2023-07-18T03:42:08.955 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.955 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.956 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.956 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "root", 2023-07-18T03:42:08.956 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.956 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.956 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.956 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm-package", 2023-07-18T03:42:08.956 INFO:teuthology.orchestra.run.smithi040.stdout: "root" 2023-07-18T03:42:08.957 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.957 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "mode for remote execution of cephadm", 2023-07-18T03:42:08.957 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.957 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.957 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.957 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.957 INFO:teuthology.orchestra.run.smithi040.stdout: "prometheus_alerts_path": { 2023-07-18T03:42:08.957 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus_alerts_path", 2023-07-18T03:42:08.958 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.958 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.958 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.958 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-18T03:42:08.958 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.958 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.958 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.958 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-18T03:42:08.959 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.959 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.959 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.959 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.959 INFO:teuthology.orchestra.run.smithi040.stdout: "prometheus_web_password": { 2023-07-18T03:42:08.959 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus_web_password", 2023-07-18T03:42:08.959 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.960 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.960 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.960 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:08.960 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.960 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.960 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.960 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus web password", 2023-07-18T03:42:08.960 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.961 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.961 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.961 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.961 INFO:teuthology.orchestra.run.smithi040.stdout: "prometheus_web_user": { 2023-07-18T03:42:08.961 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus_web_user", 2023-07-18T03:42:08.961 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.961 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.961 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.962 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:08.962 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.962 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.962 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.962 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Prometheus web user", 2023-07-18T03:42:08.962 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.962 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.962 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.963 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.963 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_insecure": { 2023-07-18T03:42:08.963 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_insecure", 2023-07-18T03:42:08.963 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.963 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.963 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.963 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.964 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.964 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.964 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.964 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-18T03:42:08.964 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.964 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.964 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.964 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.965 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_password": { 2023-07-18T03:42:08.965 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_password", 2023-07-18T03:42:08.965 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.965 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.965 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.965 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.965 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.965 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.966 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.966 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-18T03:42:08.966 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.966 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.966 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.966 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.966 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_url": { 2023-07-18T03:42:08.966 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_url", 2023-07-18T03:42:08.967 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.967 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.967 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.967 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.967 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.967 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.967 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.968 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-18T03:42:08.968 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.968 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.968 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.968 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.968 INFO:teuthology.orchestra.run.smithi040.stdout: "registry_username": { 2023-07-18T03:42:08.968 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "registry_username", 2023-07-18T03:42:08.969 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.969 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.969 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.969 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.969 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.969 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.969 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.969 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-18T03:42:08.970 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.970 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.970 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.970 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.970 INFO:teuthology.orchestra.run.smithi040.stdout: "secure_monitoring_stack": { 2023-07-18T03:42:08.970 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "secure_monitoring_stack", 2023-07-18T03:42:08.970 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.970 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.971 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.971 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.971 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.971 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.971 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.971 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-18T03:42:08.971 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.972 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.972 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.972 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.972 INFO:teuthology.orchestra.run.smithi040.stdout: "service_discovery_port": { 2023-07-18T03:42:08.972 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "service_discovery_port", 2023-07-18T03:42:08.972 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:08.972 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.972 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.973 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8765", 2023-07-18T03:42:08.973 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.973 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.973 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.973 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "cephadm service discovery port", 2023-07-18T03:42:08.973 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.973 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.974 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.974 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.974 INFO:teuthology.orchestra.run.smithi040.stdout: "ssh_config_file": { 2023-07-18T03:42:08.974 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssh_config_file", 2023-07-18T03:42:08.974 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.975 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.975 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.975 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.975 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.975 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.975 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.976 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-18T03:42:08.976 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.976 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.976 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.976 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.977 INFO:teuthology.orchestra.run.smithi040.stdout: "use_agent": { 2023-07-18T03:42:08.977 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "use_agent", 2023-07-18T03:42:08.977 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.977 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.977 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.977 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.977 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.977 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.978 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.978 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-18T03:42:08.978 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.978 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.978 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.978 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.978 INFO:teuthology.orchestra.run.smithi040.stdout: "use_repo_digest": { 2023-07-18T03:42:08.978 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "use_repo_digest", 2023-07-18T03:42:08.979 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.979 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.979 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.979 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.979 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.979 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.979 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.980 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-18T03:42:08.980 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.980 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.980 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.980 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.980 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_failed_host_check": { 2023-07-18T03:42:08.980 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_failed_host_check", 2023-07-18T03:42:08.980 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.981 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.981 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.981 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.981 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.981 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.981 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.981 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if the host check fails", 2023-07-18T03:42:08.981 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.982 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.982 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.982 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.982 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_stray_daemons": { 2023-07-18T03:42:08.982 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_stray_daemons", 2023-07-18T03:42:08.982 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.982 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.983 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.983 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.983 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.983 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.983 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.983 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-18T03:42:08.983 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.983 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.984 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.984 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.984 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_on_stray_hosts": { 2023-07-18T03:42:08.984 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_on_stray_hosts", 2023-07-18T03:42:08.984 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.984 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.984 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:08.984 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:08.985 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.985 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.985 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.985 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-18T03:42:08.985 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.985 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.985 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.986 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.986 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:08.986 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.986 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:08.986 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crash", 2023-07-18T03:42:08.986 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:08.986 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:08.986 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:08.987 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:08.987 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:08.987 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.987 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.987 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.987 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:08.987 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.987 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.988 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.988 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.988 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.988 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.988 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.988 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.988 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.988 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.989 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.989 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.989 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.989 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.989 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.989 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:08.989 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:08.989 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.990 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.990 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.990 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.990 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.990 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.990 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.990 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.991 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.991 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.991 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.991 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.991 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:08.991 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:08.991 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:08.991 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.992 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.992 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:08.992 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.992 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.992 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:08.992 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:08.992 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:08.993 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:08.993 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:08.993 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:08.993 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:08.993 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:08.993 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.994 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.994 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.994 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.994 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.994 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:08.994 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:08.994 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:08.994 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.995 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.995 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:08.995 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.995 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.995 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.995 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:08.996 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.996 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.996 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.996 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.996 INFO:teuthology.orchestra.run.smithi040.stdout: "retain_interval": { 2023-07-18T03:42:08.996 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "retain_interval", 2023-07-18T03:42:08.996 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.997 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.997 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.997 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "31536000", 2023-07-18T03:42:08.997 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:08.997 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:08.997 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:08.998 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-18T03:42:08.998 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:08.998 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:08.998 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:08.998 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:08.998 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_recent_interval": { 2023-07-18T03:42:08.999 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_recent_interval", 2023-07-18T03:42:08.999 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:08.999 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:08.999 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:08.999 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1209600", 2023-07-18T03:42:08.999 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.000 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.000 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.000 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-18T03:42:09.000 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.000 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.000 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.001 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.001 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.001 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.001 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.001 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "dashboard", 2023-07-18T03:42:09.001 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.002 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.002 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.002 INFO:teuthology.orchestra.run.smithi040.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-18T03:42:09.002 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-18T03:42:09.002 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.002 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.003 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.003 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:09.003 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.003 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.003 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.003 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.003 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.004 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.004 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.004 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.004 INFO:teuthology.orchestra.run.smithi040.stdout: "ALERTMANAGER_API_HOST": { 2023-07-18T03:42:09.004 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-18T03:42:09.004 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.004 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.005 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.005 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.005 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.005 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.005 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.005 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.005 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.005 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.006 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.006 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.006 INFO:teuthology.orchestra.run.smithi040.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-18T03:42:09.006 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-18T03:42:09.006 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.006 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.006 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.006 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.007 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.007 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.007 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.007 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.007 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.007 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.007 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.008 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.008 INFO:teuthology.orchestra.run.smithi040.stdout: "AUDIT_API_ENABLED": { 2023-07-18T03:42:09.008 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "AUDIT_API_ENABLED", 2023-07-18T03:42:09.008 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.008 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.008 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.008 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.009 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.009 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.009 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.009 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.009 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.009 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.009 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.009 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.010 INFO:teuthology.orchestra.run.smithi040.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-18T03:42:09.010 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-18T03:42:09.010 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.010 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.010 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.010 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.010 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.011 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.011 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.011 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.011 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.011 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.011 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.011 INFO:teuthology.orchestra.run.smithi040.stdout: "ENABLE_BROWSABLE_API": { 2023-07-18T03:42:09.012 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-18T03:42:09.012 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.012 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.012 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.012 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.012 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.012 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.012 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.013 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.013 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.013 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.013 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.013 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.013 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-18T03:42:09.013 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-18T03:42:09.014 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.014 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.014 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.014 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.014 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.014 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.014 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.014 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.015 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.015 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.015 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.015 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-18T03:42:09.015 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-18T03:42:09.015 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.015 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.016 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.016 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.016 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.016 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.016 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.016 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.016 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.016 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.017 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.017 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.017 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-18T03:42:09.017 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-18T03:42:09.017 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.017 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.017 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.017 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.018 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.018 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.018 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.018 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.018 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.018 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.018 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.019 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-18T03:42:09.019 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-18T03:42:09.019 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.019 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.019 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.021 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.021 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-18T03:42:09.021 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-18T03:42:09.021 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.021 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.021 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.022 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.022 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.022 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.022 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.022 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.022 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.022 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.023 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.023 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.023 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-18T03:42:09.023 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-18T03:42:09.023 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.023 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.024 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.024 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.024 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.024 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.024 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.024 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.025 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.025 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.025 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.025 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.025 INFO:teuthology.orchestra.run.smithi040.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-18T03:42:09.025 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-18T03:42:09.026 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.026 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.026 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.026 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.026 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.026 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.027 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.027 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.027 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.027 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.027 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.028 INFO:teuthology.orchestra.run.smithi040.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-18T03:42:09.028 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-18T03:42:09.028 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.028 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.028 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.028 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.028 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.029 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.029 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.029 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.029 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.029 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.029 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.029 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.030 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_PASSWORD": { 2023-07-18T03:42:09.030 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-18T03:42:09.030 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.030 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.030 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.030 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:09.030 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.030 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.031 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.031 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.031 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.031 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.031 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.031 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.032 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-18T03:42:09.032 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-18T03:42:09.032 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.032 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.032 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.032 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.033 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.033 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.033 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.033 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.033 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.033 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.033 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.033 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.034 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_URL": { 2023-07-18T03:42:09.034 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_URL", 2023-07-18T03:42:09.034 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.034 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.034 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.034 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.034 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.034 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.035 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.035 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.035 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.035 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.035 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.035 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.035 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_API_USERNAME": { 2023-07-18T03:42:09.035 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-18T03:42:09.036 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.036 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.036 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.036 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:09.036 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.036 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.036 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.037 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.037 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.037 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.037 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.037 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.037 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-18T03:42:09.037 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-18T03:42:09.037 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.038 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.038 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.038 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.038 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.038 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.038 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.038 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.038 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.039 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.039 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.039 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.039 INFO:teuthology.orchestra.run.smithi040.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-18T03:42:09.039 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-18T03:42:09.039 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.039 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.039 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.040 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.040 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.040 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.040 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.040 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.040 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.040 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.041 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.041 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.041 INFO:teuthology.orchestra.run.smithi040.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-18T03:42:09.041 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-18T03:42:09.041 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.041 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.041 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.041 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.042 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.042 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.042 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.042 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.042 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.042 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.042 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.042 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.043 INFO:teuthology.orchestra.run.smithi040.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-18T03:42:09.043 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-18T03:42:09.043 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.043 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.043 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.043 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.043 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.043 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.044 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.044 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.044 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.044 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.044 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.044 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.044 INFO:teuthology.orchestra.run.smithi040.stdout: "PROMETHEUS_API_HOST": { 2023-07-18T03:42:09.044 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-18T03:42:09.045 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.045 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.045 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.045 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.045 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.045 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.045 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.045 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.046 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.046 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.046 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.046 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.046 INFO:teuthology.orchestra.run.smithi040.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-18T03:42:09.046 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-18T03:42:09.046 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.047 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.047 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.047 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.047 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.047 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.047 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.047 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.047 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.048 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.048 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.048 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.048 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-18T03:42:09.048 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-18T03:42:09.048 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.048 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.048 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.049 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.049 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.049 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.049 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.049 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.049 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.049 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.049 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.050 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.050 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-18T03:42:09.050 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-18T03:42:09.050 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.050 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.050 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.050 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.050 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.051 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.051 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.051 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.051 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.051 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.051 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.051 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.052 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-18T03:42:09.052 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-18T03:42:09.052 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.052 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.052 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.052 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.052 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.052 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.053 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.053 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.053 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.053 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.053 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.053 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.053 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-18T03:42:09.053 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-18T03:42:09.054 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.054 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.054 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.054 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.054 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.054 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.054 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.054 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.055 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.055 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.055 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.055 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.055 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-18T03:42:09.055 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-18T03:42:09.055 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.055 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.056 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.056 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.056 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.056 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.056 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.056 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.056 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.057 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.057 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.057 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.057 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-18T03:42:09.057 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-18T03:42:09.057 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.057 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.057 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.058 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.058 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.058 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.058 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.058 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.058 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.058 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.058 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.059 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.059 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-18T03:42:09.059 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-18T03:42:09.059 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.059 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.059 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.059 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.060 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.060 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.060 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.060 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.060 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.060 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.060 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.061 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.061 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_ENABLED": { 2023-07-18T03:42:09.061 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-18T03:42:09.061 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.061 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.061 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.061 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.061 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.062 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.062 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.062 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.062 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.062 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.062 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.062 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.062 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-18T03:42:09.063 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-18T03:42:09.063 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.063 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.063 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.063 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-18T03:42:09.063 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.063 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.063 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.064 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.064 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.064 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.064 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.064 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.064 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-18T03:42:09.064 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-18T03:42:09.064 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.065 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.065 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.065 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:09.065 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.065 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.065 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.065 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.066 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.066 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.066 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.066 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.066 INFO:teuthology.orchestra.run.smithi040.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-18T03:42:09.066 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-18T03:42:09.066 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.066 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.067 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.067 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8", 2023-07-18T03:42:09.067 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.067 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.067 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.067 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.067 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.067 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.068 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.068 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.068 INFO:teuthology.orchestra.run.smithi040.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-18T03:42:09.068 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-18T03:42:09.068 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.068 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.068 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.068 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "45", 2023-07-18T03:42:09.069 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.069 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.069 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.069 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.069 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.069 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.069 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.070 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_ACCESS_KEY": { 2023-07-18T03:42:09.070 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-18T03:42:09.070 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.070 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.070 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.070 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.070 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.070 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.071 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.071 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.071 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.071 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.071 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.071 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-18T03:42:09.071 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-18T03:42:09.072 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.072 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.072 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.072 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "admin", 2023-07-18T03:42:09.072 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.072 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.072 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.073 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.073 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.073 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.073 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.073 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_SECRET_KEY": { 2023-07-18T03:42:09.073 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-18T03:42:09.073 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.074 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.074 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.074 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.074 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.074 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.074 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.074 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.074 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.075 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.075 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.075 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.075 INFO:teuthology.orchestra.run.smithi040.stdout: "RGW_API_SSL_VERIFY": { 2023-07-18T03:42:09.075 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-18T03:42:09.075 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.075 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.075 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.076 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.076 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.076 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.076 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.076 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.076 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.076 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.077 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.077 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.077 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-18T03:42:09.077 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-18T03:42:09.077 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.077 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.077 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.077 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2023-07-18T03:42:09.078 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.078 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.078 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.078 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.078 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.078 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.078 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.078 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.079 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-18T03:42:09.079 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-18T03:42:09.079 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.079 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.079 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.079 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:09.079 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.079 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.080 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.080 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.080 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.080 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.080 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.080 INFO:teuthology.orchestra.run.smithi040.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-18T03:42:09.081 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-18T03:42:09.081 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.081 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.081 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.081 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2023-07-18T03:42:09.081 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.081 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.081 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.082 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.082 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.082 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.082 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.082 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.082 INFO:teuthology.orchestra.run.smithi040.stdout: "cross_origin_url": { 2023-07-18T03:42:09.082 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cross_origin_url", 2023-07-18T03:42:09.082 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.083 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.083 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.083 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.083 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.083 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.083 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.083 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.083 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.084 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.084 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.084 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.084 INFO:teuthology.orchestra.run.smithi040.stdout: "crt_file": { 2023-07-18T03:42:09.084 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "crt_file", 2023-07-18T03:42:09.084 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.084 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.085 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.085 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.085 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.085 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.085 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.085 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.085 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.085 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.086 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.086 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.086 INFO:teuthology.orchestra.run.smithi040.stdout: "debug": { 2023-07-18T03:42:09.086 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "debug", 2023-07-18T03:42:09.086 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.086 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.086 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.086 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.087 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.087 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.087 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.087 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Enable/disable debug options", 2023-07-18T03:42:09.087 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.087 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.087 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.088 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.088 INFO:teuthology.orchestra.run.smithi040.stdout: "jwt_token_ttl": { 2023-07-18T03:42:09.088 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "jwt_token_ttl", 2023-07-18T03:42:09.088 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.088 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.088 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.088 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "28800", 2023-07-18T03:42:09.088 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.089 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.089 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.089 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.089 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.089 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.089 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.089 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.090 INFO:teuthology.orchestra.run.smithi040.stdout: "key_file": { 2023-07-18T03:42:09.090 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "key_file", 2023-07-18T03:42:09.090 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.090 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.090 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.090 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.090 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.090 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.091 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.091 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.091 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.091 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.091 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.091 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.091 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.091 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.092 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.092 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.092 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.092 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.092 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.092 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.092 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.093 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.093 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.093 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.093 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.093 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.093 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.093 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.093 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.094 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.094 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.094 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.094 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.094 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.094 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.094 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.095 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.095 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.095 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.095 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.095 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.095 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.095 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.095 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.096 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.096 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.096 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.096 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.096 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.096 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.096 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.097 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.097 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.097 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.097 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.097 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.097 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.097 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.098 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.098 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.098 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.098 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.098 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.098 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.098 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.098 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.099 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.099 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.099 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.099 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.099 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.099 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.099 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.099 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.100 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.100 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.100 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.100 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.100 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.100 INFO:teuthology.orchestra.run.smithi040.stdout: "motd": { 2023-07-18T03:42:09.100 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "motd", 2023-07-18T03:42:09.101 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.101 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.101 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.101 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.101 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.101 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.101 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.102 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "The message of the day", 2023-07-18T03:42:09.102 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.102 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.102 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.102 INFO:teuthology.orchestra.run.smithi040.stdout: "redirect_resolve_ip_addr": { 2023-07-18T03:42:09.102 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "redirect_resolve_ip_addr", 2023-07-18T03:42:09.102 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.103 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.103 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.103 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.103 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.103 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.103 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.103 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.104 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.104 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.104 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.104 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.104 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2023-07-18T03:42:09.104 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2023-07-18T03:42:09.104 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.105 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.105 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.105 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "::", 2023-07-18T03:42:09.105 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.105 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.105 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.105 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.105 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.106 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.106 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.106 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2023-07-18T03:42:09.106 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2023-07-18T03:42:09.106 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.106 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.106 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.107 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8080", 2023-07-18T03:42:09.107 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.107 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.107 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.107 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.107 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.107 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.108 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.108 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl": { 2023-07-18T03:42:09.108 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl", 2023-07-18T03:42:09.108 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.108 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.108 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.108 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.109 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.109 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.109 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.109 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.109 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.109 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.109 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.110 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl_server_port": { 2023-07-18T03:42:09.110 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl_server_port", 2023-07-18T03:42:09.110 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.110 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.110 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.110 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8443", 2023-07-18T03:42:09.110 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.111 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.111 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.111 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.111 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.111 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.111 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.111 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.111 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_behaviour": { 2023-07-18T03:42:09.112 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_behaviour", 2023-07-18T03:42:09.112 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.112 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.112 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.112 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "redirect", 2023-07-18T03:42:09.112 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.112 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.113 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.113 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.113 INFO:teuthology.orchestra.run.smithi040.stdout: "redirect" 2023-07-18T03:42:09.113 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.113 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.113 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.113 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.113 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.114 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_error_status_code": { 2023-07-18T03:42:09.114 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_error_status_code", 2023-07-18T03:42:09.114 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.114 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.114 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.114 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "500", 2023-07-18T03:42:09.114 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "400", 2023-07-18T03:42:09.115 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "599", 2023-07-18T03:42:09.115 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.115 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.115 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.115 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.115 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.115 INFO:teuthology.orchestra.run.smithi040.stdout: "url_prefix": { 2023-07-18T03:42:09.116 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "url_prefix", 2023-07-18T03:42:09.116 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.116 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.116 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.116 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.116 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.116 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.117 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.117 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.117 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.117 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.117 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.117 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.117 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.118 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "devicehealth", 2023-07-18T03:42:09.118 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.118 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.118 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.118 INFO:teuthology.orchestra.run.smithi040.stdout: "enable_monitoring": { 2023-07-18T03:42:09.118 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enable_monitoring", 2023-07-18T03:42:09.118 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.119 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.119 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.119 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.119 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.119 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.119 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.119 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "monitor device health metrics", 2023-07-18T03:42:09.120 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.120 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.120 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.120 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.120 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.120 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.120 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.121 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.121 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.121 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.121 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.121 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.121 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.121 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.121 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.122 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.122 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.122 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.122 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.122 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.122 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.122 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.122 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.123 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.123 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.123 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.123 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.123 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.123 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.123 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.123 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.124 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.124 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.124 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.124 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.124 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.124 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.125 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.125 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.125 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.125 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.125 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.125 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.125 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.125 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.126 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.126 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.126 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.126 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.126 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.126 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.126 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.126 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.127 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.127 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.127 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.127 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.127 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.127 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.127 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.128 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.128 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.128 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.128 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.128 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.128 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.128 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.129 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.129 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.129 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.129 INFO:teuthology.orchestra.run.smithi040.stdout: "mark_out_threshold": { 2023-07-18T03:42:09.129 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mark_out_threshold", 2023-07-18T03:42:09.129 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:09.129 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.130 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.130 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "2419200", 2023-07-18T03:42:09.130 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.130 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.130 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.130 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-18T03:42:09.130 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.130 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.131 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.131 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": { 2023-07-18T03:42:09.131 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pool_name", 2023-07-18T03:42:09.131 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.131 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.131 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.131 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "device_health_metrics", 2023-07-18T03:42:09.132 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.132 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.132 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.132 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-18T03:42:09.132 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.132 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.132 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.133 INFO:teuthology.orchestra.run.smithi040.stdout: "retention_period": { 2023-07-18T03:42:09.133 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "retention_period", 2023-07-18T03:42:09.133 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:09.133 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.133 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.133 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15552000", 2023-07-18T03:42:09.133 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.134 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.134 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.134 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long to retain device health metrics", 2023-07-18T03:42:09.134 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.134 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.134 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.134 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.134 INFO:teuthology.orchestra.run.smithi040.stdout: "scrape_frequency": { 2023-07-18T03:42:09.135 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "scrape_frequency", 2023-07-18T03:42:09.135 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:09.135 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.135 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.135 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "86400", 2023-07-18T03:42:09.135 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.135 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.135 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.136 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-18T03:42:09.136 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.136 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.136 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.136 INFO:teuthology.orchestra.run.smithi040.stdout: "self_heal": { 2023-07-18T03:42:09.136 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "self_heal", 2023-07-18T03:42:09.137 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.137 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.137 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.137 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.137 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.137 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.137 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.137 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-18T03:42:09.138 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.138 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.138 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.138 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2023-07-18T03:42:09.138 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2023-07-18T03:42:09.138 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:09.138 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.139 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.139 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2023-07-18T03:42:09.139 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.139 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.139 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.139 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how frequently to wake up and check device health", 2023-07-18T03:42:09.139 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.139 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.140 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.140 INFO:teuthology.orchestra.run.smithi040.stdout: "warn_threshold": { 2023-07-18T03:42:09.140 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "warn_threshold", 2023-07-18T03:42:09.140 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:09.140 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.140 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.141 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "7257600", 2023-07-18T03:42:09.141 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.141 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.141 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.141 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-18T03:42:09.141 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.141 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.141 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.142 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.142 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.142 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "diskprediction_local", 2023-07-18T03:42:09.142 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.142 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.142 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.143 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.143 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.143 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.143 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.143 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.143 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.143 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.143 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.144 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.144 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.144 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.144 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.144 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.144 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.144 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.145 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.145 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.145 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.145 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.145 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.145 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.145 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.146 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.146 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.146 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.146 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.146 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.146 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.146 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.146 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.147 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.147 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.147 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.147 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.147 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.147 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.148 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.148 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.148 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.148 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.148 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.148 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.148 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.148 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.149 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.149 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.149 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.149 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.149 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.149 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.149 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.149 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.150 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.150 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.150 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.150 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.150 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.150 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.150 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.151 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.151 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.151 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.151 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.151 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.151 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.151 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.152 INFO:teuthology.orchestra.run.smithi040.stdout: "predict_interval": { 2023-07-18T03:42:09.152 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "predict_interval", 2023-07-18T03:42:09.152 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.152 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.152 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.152 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "86400", 2023-07-18T03:42:09.152 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.153 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.153 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.153 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.153 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.153 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.153 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.153 INFO:teuthology.orchestra.run.smithi040.stdout: "predictor_model": { 2023-07-18T03:42:09.154 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "predictor_model", 2023-07-18T03:42:09.154 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.154 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.154 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.154 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "prophetstor", 2023-07-18T03:42:09.154 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.154 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.154 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.155 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.155 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.155 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.155 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.155 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2023-07-18T03:42:09.155 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2023-07-18T03:42:09.155 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.156 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.156 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.156 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "600", 2023-07-18T03:42:09.156 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.156 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.156 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.156 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.156 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.157 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.157 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.157 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.157 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.157 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.157 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "influx", 2023-07-18T03:42:09.157 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": false, 2023-07-18T03:42:09.158 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "influxdb python module not found", 2023-07-18T03:42:09.158 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.158 INFO:teuthology.orchestra.run.smithi040.stdout: "batch_size": { 2023-07-18T03:42:09.158 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "batch_size", 2023-07-18T03:42:09.158 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.158 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.158 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.159 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5000", 2023-07-18T03:42:09.159 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.159 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.159 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.159 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-18T03:42:09.159 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.159 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.159 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.160 INFO:teuthology.orchestra.run.smithi040.stdout: "database": { 2023-07-18T03:42:09.160 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "database", 2023-07-18T03:42:09.160 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.160 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.160 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.160 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "ceph", 2023-07-18T03:42:09.160 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.161 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.161 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.161 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-18T03:42:09.161 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.161 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.161 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.162 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": { 2023-07-18T03:42:09.162 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "hostname", 2023-07-18T03:42:09.162 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.162 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.162 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.162 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.162 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.162 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.163 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.163 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB server hostname", 2023-07-18T03:42:09.163 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.163 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.163 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.163 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2023-07-18T03:42:09.163 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2023-07-18T03:42:09.164 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:09.164 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.164 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.164 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "30", 2023-07-18T03:42:09.164 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "5", 2023-07-18T03:42:09.164 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.164 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.164 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-18T03:42:09.165 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.165 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.165 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.165 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.165 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.165 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.165 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.166 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.166 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.166 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.166 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.166 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.166 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.166 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.166 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.167 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.167 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.167 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.167 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.167 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.167 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.167 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.167 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.168 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.168 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.168 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.168 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.168 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.168 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.169 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.169 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.169 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.169 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.169 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.169 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.169 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.170 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.170 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.170 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.170 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.170 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.170 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.170 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.170 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.171 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.171 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.171 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.171 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.171 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.171 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.171 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.171 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.172 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.172 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.172 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.172 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.172 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.172 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.172 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.173 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.173 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.173 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.173 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.173 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.173 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.173 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.174 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.174 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.174 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.174 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.174 INFO:teuthology.orchestra.run.smithi040.stdout: "password": { 2023-07-18T03:42:09.174 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "password", 2023-07-18T03:42:09.174 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.174 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.175 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.175 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.175 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.175 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.175 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.175 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "password of InfluxDB server user", 2023-07-18T03:42:09.175 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.175 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.176 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.176 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.176 INFO:teuthology.orchestra.run.smithi040.stdout: "port": { 2023-07-18T03:42:09.176 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "port", 2023-07-18T03:42:09.176 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.176 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.176 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.177 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "8086", 2023-07-18T03:42:09.177 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.177 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.177 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.177 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "InfluxDB server port", 2023-07-18T03:42:09.178 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.178 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.178 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.178 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.178 INFO:teuthology.orchestra.run.smithi040.stdout: "ssl": { 2023-07-18T03:42:09.178 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ssl", 2023-07-18T03:42:09.178 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.178 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.179 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.179 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "false", 2023-07-18T03:42:09.179 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.179 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.179 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.179 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-18T03:42:09.179 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.179 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.180 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.180 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.180 INFO:teuthology.orchestra.run.smithi040.stdout: "threads": { 2023-07-18T03:42:09.180 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "threads", 2023-07-18T03:42:09.180 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.180 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.180 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.181 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2023-07-18T03:42:09.181 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2023-07-18T03:42:09.181 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "32", 2023-07-18T03:42:09.181 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.181 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-18T03:42:09.181 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.181 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.181 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.182 INFO:teuthology.orchestra.run.smithi040.stdout: "username": { 2023-07-18T03:42:09.182 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "username", 2023-07-18T03:42:09.182 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.182 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.182 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.182 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.182 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.183 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.183 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.183 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "username of InfluxDB server user", 2023-07-18T03:42:09.183 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.183 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.183 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.183 INFO:teuthology.orchestra.run.smithi040.stdout: "verify_ssl": { 2023-07-18T03:42:09.184 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "verify_ssl", 2023-07-18T03:42:09.184 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.184 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.184 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.184 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "true", 2023-07-18T03:42:09.184 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.184 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.185 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.185 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-18T03:42:09.185 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.185 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.185 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.185 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.185 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.186 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "insights", 2023-07-18T03:42:09.186 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.186 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.186 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.186 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.186 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.186 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.187 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.187 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.187 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.187 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.187 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.187 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.187 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.187 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.188 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.188 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.188 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.188 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.188 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.188 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.188 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.189 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.189 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.189 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.189 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.189 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.189 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.189 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.190 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.190 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.190 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.190 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.190 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.190 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.190 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.190 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.191 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.191 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.191 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.191 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.191 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.191 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.191 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.192 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.192 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.192 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.192 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.192 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.192 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.192 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.192 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.193 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.193 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.193 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.193 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.193 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.193 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.193 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.194 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.194 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.194 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.194 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.194 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.194 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.194 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.195 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.195 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.195 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.195 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.195 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.195 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.195 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.196 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "iostat", 2023-07-18T03:42:09.196 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.196 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.196 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.196 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.196 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.196 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.196 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.197 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.197 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.197 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.197 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.197 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.197 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.197 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.198 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.198 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.198 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.198 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.198 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.198 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.199 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.199 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.199 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.199 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.199 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.200 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.200 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.200 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.200 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.200 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.200 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.201 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.201 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.201 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.201 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.201 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.201 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.201 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.201 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.202 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.202 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.202 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.202 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.202 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.202 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.202 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.202 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.203 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.203 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.203 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.203 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.203 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.203 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.203 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.203 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.204 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.204 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.204 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.204 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.204 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.204 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.204 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.205 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.205 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.205 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.205 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.205 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.205 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.205 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.205 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.206 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.206 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.206 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.206 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.206 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.206 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.206 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "k8sevents", 2023-07-18T03:42:09.206 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.207 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.207 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.207 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_event_retention_days": { 2023-07-18T03:42:09.207 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "ceph_event_retention_days", 2023-07-18T03:42:09.207 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.207 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.207 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.207 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "7", 2023-07-18T03:42:09.208 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.208 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.208 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.208 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-18T03:42:09.208 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.208 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.208 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.209 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.209 INFO:teuthology.orchestra.run.smithi040.stdout: "config_check_secs": { 2023-07-18T03:42:09.209 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "config_check_secs", 2023-07-18T03:42:09.209 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.209 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.209 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.209 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:09.210 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "10", 2023-07-18T03:42:09.210 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.210 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.210 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-18T03:42:09.210 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.210 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.210 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.210 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.211 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.211 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.211 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.211 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.211 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.211 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.211 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.211 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.212 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.212 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.212 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.212 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.212 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.212 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.212 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.212 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.213 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.213 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.213 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.213 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.213 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.213 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.213 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.213 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.214 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.214 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.214 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.214 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.214 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.214 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.214 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.215 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.215 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.215 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.215 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.215 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.215 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.215 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.215 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.216 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.216 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.216 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.216 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.216 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.216 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.216 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.217 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.217 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.217 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.217 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.217 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.217 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.217 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.218 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.218 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.218 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.218 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.218 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.218 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.219 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.219 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.219 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.219 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.219 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.219 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.219 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.219 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.220 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.220 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.220 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.220 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.220 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.220 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.220 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "localpool", 2023-07-18T03:42:09.221 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.221 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.221 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.221 INFO:teuthology.orchestra.run.smithi040.stdout: "failure_domain": { 2023-07-18T03:42:09.221 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "failure_domain", 2023-07-18T03:42:09.221 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.221 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.221 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.222 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "host", 2023-07-18T03:42:09.222 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.222 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.222 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.222 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "failure domain for any created local pool", 2023-07-18T03:42:09.222 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-18T03:42:09.222 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.222 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.223 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.223 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.223 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.223 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.223 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.223 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.223 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.223 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.224 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.224 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.224 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.224 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.224 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.225 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.225 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.225 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.225 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.225 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.225 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.225 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.226 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.226 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.226 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.226 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.226 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.226 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.226 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.227 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.227 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.227 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.227 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.227 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.227 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.227 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.227 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.228 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.228 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.228 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.228 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.228 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.228 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.228 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.229 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.229 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.229 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.229 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.229 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.229 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.229 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.229 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.230 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.230 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.230 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.230 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.230 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.230 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.230 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.230 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.231 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.231 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.231 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.231 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.231 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.231 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.231 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.231 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.232 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.232 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.232 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.232 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.232 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.232 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": { 2023-07-18T03:42:09.232 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "min_size", 2023-07-18T03:42:09.232 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.233 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.233 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.233 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.233 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.233 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.233 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.233 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default min_size for any created local pool", 2023-07-18T03:42:09.233 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-18T03:42:09.234 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.234 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.234 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.234 INFO:teuthology.orchestra.run.smithi040.stdout: "num_rep": { 2023-07-18T03:42:09.234 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "num_rep", 2023-07-18T03:42:09.234 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.234 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.235 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.235 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3", 2023-07-18T03:42:09.235 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.235 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.235 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.235 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default replica count for any created local pool", 2023-07-18T03:42:09.235 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.235 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.236 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.236 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.236 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": { 2023-07-18T03:42:09.236 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pg_num", 2023-07-18T03:42:09.236 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.236 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.236 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.236 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "128", 2023-07-18T03:42:09.237 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.237 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.237 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.237 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "default pg_num for any created local pool", 2023-07-18T03:42:09.237 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.237 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.237 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.237 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.238 INFO:teuthology.orchestra.run.smithi040.stdout: "prefix": { 2023-07-18T03:42:09.238 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prefix", 2023-07-18T03:42:09.238 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.238 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.238 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.238 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.238 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.238 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.239 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.239 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "name prefix for any created local pool", 2023-07-18T03:42:09.239 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.239 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.239 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.239 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.239 INFO:teuthology.orchestra.run.smithi040.stdout: "subtree": { 2023-07-18T03:42:09.240 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "subtree", 2023-07-18T03:42:09.240 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.240 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.240 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.240 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "rack", 2023-07-18T03:42:09.240 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.240 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.240 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.241 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-18T03:42:09.241 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-18T03:42:09.241 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.241 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.241 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.241 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.241 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.242 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.242 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mds_autoscaler", 2023-07-18T03:42:09.242 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.242 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.242 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.242 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.242 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.242 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.243 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.243 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.243 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.243 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.243 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.243 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.243 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.243 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.244 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.244 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.244 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.244 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.244 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.244 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.244 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.244 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.245 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.245 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.245 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.245 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.245 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.245 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.245 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.245 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.246 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.246 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.246 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.246 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.246 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.246 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.246 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.246 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.247 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.247 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.247 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.247 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.247 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.247 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.247 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.248 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.248 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.248 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.248 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.248 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.248 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.248 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.249 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.249 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.249 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.249 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.249 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.249 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.249 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.249 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.250 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.250 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.250 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.250 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.250 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.250 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.250 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.250 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.251 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.251 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.251 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.251 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.251 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.251 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.251 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.252 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.252 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mirroring", 2023-07-18T03:42:09.252 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.252 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.252 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.252 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.252 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.253 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.253 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.253 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.253 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.253 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.253 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.253 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.253 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.254 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.254 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.254 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.254 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.254 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.254 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.254 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.255 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.255 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.255 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.255 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.255 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.255 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.255 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.255 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.256 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.256 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.256 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.256 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.256 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.256 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.256 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.257 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.257 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.257 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.257 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.257 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.257 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.257 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.257 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.258 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.258 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.258 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.258 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.258 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.258 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.258 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.258 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.259 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.259 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.259 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.259 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.259 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.259 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.259 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.259 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.260 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.260 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.260 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.260 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.260 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.260 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.260 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.261 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.261 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.261 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.261 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.261 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.261 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.261 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.261 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.262 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.262 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.262 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "nfs", 2023-07-18T03:42:09.262 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.262 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.262 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.262 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.262 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.263 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.263 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.263 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.263 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.263 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.263 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.263 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.263 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.264 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.264 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.264 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.264 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.264 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.264 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.264 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.265 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.265 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.265 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.265 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.265 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.265 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.265 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.265 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.266 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.266 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.266 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.266 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.266 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.266 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.266 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.266 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.267 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.267 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.267 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.267 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.267 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.267 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.267 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.268 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.268 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.268 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.268 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.268 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.268 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.268 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.268 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.269 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.269 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.269 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.269 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.269 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.269 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.269 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.270 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.270 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.270 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.270 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.270 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.270 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.270 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.271 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.271 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.271 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.271 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.271 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.271 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.271 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.272 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.272 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.272 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "orchestrator", 2023-07-18T03:42:09.272 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.272 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.272 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.272 INFO:teuthology.orchestra.run.smithi040.stdout: "fail_fs": { 2023-07-18T03:42:09.272 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "fail_fs", 2023-07-18T03:42:09.273 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.273 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.273 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.273 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.273 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.273 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.273 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.273 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-18T03:42:09.274 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.274 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.274 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.274 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.274 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.274 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.274 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.274 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.275 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.275 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.275 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.275 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.275 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.275 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.275 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.275 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.276 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.276 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.276 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.276 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.276 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.276 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.276 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.276 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.277 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.277 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.277 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.277 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.277 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.277 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.277 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.278 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.278 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.278 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.278 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.278 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.278 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.278 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.279 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.279 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.279 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.279 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.279 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.279 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.279 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.279 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.280 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.280 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.280 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.280 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.280 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.280 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.280 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.280 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.281 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.281 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.281 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.281 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.281 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.281 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.281 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.282 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.282 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.282 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.282 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.282 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.282 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.282 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.282 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.283 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.283 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.283 INFO:teuthology.orchestra.run.smithi040.stdout: "orchestrator": { 2023-07-18T03:42:09.283 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "orchestrator", 2023-07-18T03:42:09.283 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.283 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.284 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.284 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.284 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.284 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.284 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.284 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm", 2023-07-18T03:42:09.284 INFO:teuthology.orchestra.run.smithi040.stdout: "rook", 2023-07-18T03:42:09.284 INFO:teuthology.orchestra.run.smithi040.stdout: "test_orchestrator" 2023-07-18T03:42:09.285 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.285 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Orchestrator backend", 2023-07-18T03:42:09.285 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.285 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.285 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.285 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.285 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.286 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.286 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.286 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "osd_perf_query", 2023-07-18T03:42:09.286 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.286 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.286 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.286 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.286 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.287 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.287 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.287 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.287 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.287 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.287 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.287 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.287 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.288 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.288 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.288 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.288 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.288 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.288 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.288 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.288 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.289 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.289 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.289 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.289 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.289 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.289 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.289 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.290 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.290 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.290 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.290 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.290 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.290 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.290 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.290 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.291 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.291 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.291 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.291 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.291 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.291 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.291 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.291 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.292 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.292 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.292 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.292 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.292 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.292 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.292 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.292 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.293 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.293 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.293 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.293 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.293 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.293 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.293 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.293 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.294 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.294 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.294 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.294 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.294 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.294 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.294 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.295 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.295 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.295 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.295 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.295 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.295 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.295 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.295 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.296 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.296 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "osd_support", 2023-07-18T03:42:09.296 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.296 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.296 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.296 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.296 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.296 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.297 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.297 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.297 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.297 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.297 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.297 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.297 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.297 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.298 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.298 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.298 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.298 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.298 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.298 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.298 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.298 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.299 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.299 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.299 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.299 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.299 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.299 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.300 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.300 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.300 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.300 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.300 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.300 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.300 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.300 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.301 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.302 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.302 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.302 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.302 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.302 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.302 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.302 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.303 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.303 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.303 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.303 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.303 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.303 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.303 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.304 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.304 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.304 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.304 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.304 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.304 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.304 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.304 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.305 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.305 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.305 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.305 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.305 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.305 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.305 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.305 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.306 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "pg_autoscaler", 2023-07-18T03:42:09.306 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.306 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.306 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.306 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.306 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.306 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.307 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.307 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.307 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.307 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.307 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.307 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.307 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.307 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.308 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.308 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.308 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.308 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.308 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.308 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.309 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.309 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.309 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.309 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.309 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.309 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.309 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.309 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.310 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.310 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.310 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.310 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.310 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.310 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.310 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.310 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.311 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.311 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.311 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.311 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.311 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.311 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.311 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.311 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.312 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.312 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.312 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.312 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.312 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.312 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.312 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.312 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.313 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.313 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.313 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.313 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.313 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.313 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.313 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.314 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.314 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.314 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.314 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.314 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.314 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.314 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.315 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.315 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.315 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.315 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.315 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.315 INFO:teuthology.orchestra.run.smithi040.stdout: "noautoscale": { 2023-07-18T03:42:09.315 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "noautoscale", 2023-07-18T03:42:09.316 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.316 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.316 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.316 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.316 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.316 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.316 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.316 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "global autoscale flag", 2023-07-18T03:42:09.317 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-18T03:42:09.317 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.317 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.317 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.317 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2023-07-18T03:42:09.317 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2023-07-18T03:42:09.317 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:09.317 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.318 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.318 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2023-07-18T03:42:09.318 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.318 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.318 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.318 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.318 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.318 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.319 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.319 INFO:teuthology.orchestra.run.smithi040.stdout: "threshold": { 2023-07-18T03:42:09.319 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "threshold", 2023-07-18T03:42:09.319 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:09.319 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.319 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.319 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "3.0", 2023-07-18T03:42:09.320 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1.0", 2023-07-18T03:42:09.320 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.320 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.320 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "scaling threshold", 2023-07-18T03:42:09.320 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-18T03:42:09.320 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.320 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.321 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.321 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.321 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.321 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "progress", 2023-07-18T03:42:09.321 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.321 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.321 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.322 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_pg_recovery_event": { 2023-07-18T03:42:09.322 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_pg_recovery_event", 2023-07-18T03:42:09.322 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.322 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.322 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.322 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.322 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.322 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.323 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.323 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-18T03:42:09.323 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.323 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.323 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.323 INFO:teuthology.orchestra.run.smithi040.stdout: "enabled": { 2023-07-18T03:42:09.324 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enabled", 2023-07-18T03:42:09.324 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.324 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.324 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.324 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.325 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.325 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.325 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.325 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.325 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.325 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.325 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.326 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.326 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.326 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.326 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.326 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.326 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.326 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.326 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.327 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.327 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.327 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.327 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.327 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.327 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.327 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.328 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.328 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.328 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.328 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.328 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.328 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.328 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.328 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.329 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.329 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.329 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.329 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.329 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.329 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.329 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.329 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.330 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.330 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.330 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.330 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.330 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.330 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.330 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.330 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.331 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.331 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.331 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.331 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.331 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.331 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.331 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.332 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.332 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.332 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.332 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.332 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.332 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.332 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.332 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.333 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.333 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.333 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.333 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.333 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.333 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.334 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.334 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.334 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.334 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.334 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.334 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.334 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.334 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.335 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.335 INFO:teuthology.orchestra.run.smithi040.stdout: "max_completed_events": { 2023-07-18T03:42:09.335 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_completed_events", 2023-07-18T03:42:09.335 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.335 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.335 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.335 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "50", 2023-07-18T03:42:09.336 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.336 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.336 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.336 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "number of past completed events to remember", 2023-07-18T03:42:09.336 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.336 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.336 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.337 INFO:teuthology.orchestra.run.smithi040.stdout: "sleep_interval": { 2023-07-18T03:42:09.337 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "sleep_interval", 2023-07-18T03:42:09.337 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:09.337 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.337 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.337 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "5", 2023-07-18T03:42:09.338 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.338 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.338 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.338 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "how long the module is going to sleep", 2023-07-18T03:42:09.338 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.338 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.338 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.338 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.339 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.339 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.339 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "prometheus", 2023-07-18T03:42:09.339 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.339 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.339 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.339 INFO:teuthology.orchestra.run.smithi040.stdout: "cache": { 2023-07-18T03:42:09.340 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cache", 2023-07-18T03:42:09.340 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.340 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.340 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.340 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.340 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.340 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.340 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.341 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.341 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.341 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.341 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.341 INFO:teuthology.orchestra.run.smithi040.stdout: "exclude_perf_counters": { 2023-07-18T03:42:09.341 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "exclude_perf_counters", 2023-07-18T03:42:09.341 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.342 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.342 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.342 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.342 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.342 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.342 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.342 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-18T03:42:09.343 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-18T03:42:09.343 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.343 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.343 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.343 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.343 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.343 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.344 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.344 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.344 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.344 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.344 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.344 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.344 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.344 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.345 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.345 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.345 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.345 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.345 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.345 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.345 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.345 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.346 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.346 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.346 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.346 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.346 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.346 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.347 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.347 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.347 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.347 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.347 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.347 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.347 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.348 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.348 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.348 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.348 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.348 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.348 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.348 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.348 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.349 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.349 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.349 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.349 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.349 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.349 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.349 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.349 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.350 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.350 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.350 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.350 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.350 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.350 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.350 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.351 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.351 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.351 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.351 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.351 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.351 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.351 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.352 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.352 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.352 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.352 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_stats_pools": { 2023-07-18T03:42:09.352 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_stats_pools", 2023-07-18T03:42:09.352 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.352 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.353 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.353 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.353 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.353 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.353 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.353 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.353 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.353 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.354 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.354 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-18T03:42:09.354 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-18T03:42:09.354 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.354 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.354 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.354 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "300", 2023-07-18T03:42:09.355 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.355 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.355 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.355 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.355 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.355 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.355 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.356 INFO:teuthology.orchestra.run.smithi040.stdout: "scrape_interval": { 2023-07-18T03:42:09.356 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "scrape_interval", 2023-07-18T03:42:09.356 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:09.356 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.356 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.356 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15.0", 2023-07-18T03:42:09.356 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.357 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.357 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.357 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.357 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.357 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.357 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.357 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2023-07-18T03:42:09.358 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2023-07-18T03:42:09.358 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.358 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.358 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.358 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "::", 2023-07-18T03:42:09.358 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.358 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.358 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.359 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-18T03:42:09.359 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.359 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.359 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.359 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2023-07-18T03:42:09.359 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2023-07-18T03:42:09.359 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.360 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.360 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.360 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "9283", 2023-07-18T03:42:09.360 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.360 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.360 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.360 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-18T03:42:09.360 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.361 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.361 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.361 INFO:teuthology.orchestra.run.smithi040.stdout: "stale_cache_strategy": { 2023-07-18T03:42:09.361 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "stale_cache_strategy", 2023-07-18T03:42:09.361 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.361 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.362 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.362 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "log", 2023-07-18T03:42:09.362 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.362 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.362 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.362 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.362 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.363 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.363 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.363 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.363 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_behaviour": { 2023-07-18T03:42:09.363 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_behaviour", 2023-07-18T03:42:09.363 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.363 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.363 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.364 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "default", 2023-07-18T03:42:09.364 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.364 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.364 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.364 INFO:teuthology.orchestra.run.smithi040.stdout: "default", 2023-07-18T03:42:09.364 INFO:teuthology.orchestra.run.smithi040.stdout: "error" 2023-07-18T03:42:09.364 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.364 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.365 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.365 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.365 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.365 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.365 INFO:teuthology.orchestra.run.smithi040.stdout: "standby_error_status_code": { 2023-07-18T03:42:09.365 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "standby_error_status_code", 2023-07-18T03:42:09.365 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.365 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.366 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.366 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "500", 2023-07-18T03:42:09.366 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "400", 2023-07-18T03:42:09.366 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "599", 2023-07-18T03:42:09.366 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.366 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.366 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.367 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.367 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.367 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.367 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.367 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.367 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.367 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_support", 2023-07-18T03:42:09.367 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.368 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.368 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.368 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.368 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.368 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.368 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.368 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.368 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.369 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.369 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.369 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.369 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.369 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.369 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.369 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.369 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.370 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.370 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.370 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.370 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.370 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.370 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.370 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.370 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.371 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.371 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.371 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.371 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.371 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.371 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.371 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.371 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.372 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.372 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.372 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.372 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.372 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.372 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.372 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.373 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.373 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.373 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.373 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.373 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.373 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.373 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.373 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.374 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.374 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.374 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.374 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.374 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.374 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.374 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.374 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.375 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.375 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.375 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.375 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.375 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.375 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.375 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.375 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.376 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.376 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.376 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.376 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.376 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.376 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.376 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.376 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.377 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.377 INFO:teuthology.orchestra.run.smithi040.stdout: "max_concurrent_snap_create": { 2023-07-18T03:42:09.377 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_concurrent_snap_create", 2023-07-18T03:42:09.377 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.377 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.377 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.377 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10", 2023-07-18T03:42:09.377 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.378 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.378 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.378 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.378 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.378 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.378 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.378 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.378 INFO:teuthology.orchestra.run.smithi040.stdout: "mirror_snapshot_schedule": { 2023-07-18T03:42:09.379 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mirror_snapshot_schedule", 2023-07-18T03:42:09.379 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.379 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.379 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.379 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.379 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.379 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.379 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.380 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.380 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.380 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.380 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.380 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.380 INFO:teuthology.orchestra.run.smithi040.stdout: "trash_purge_schedule": { 2023-07-18T03:42:09.380 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "trash_purge_schedule", 2023-07-18T03:42:09.381 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.381 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.381 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.381 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.381 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.381 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.381 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.381 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.382 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.382 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.382 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.382 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.382 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.382 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.382 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.382 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "restful", 2023-07-18T03:42:09.383 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.383 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.383 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.383 INFO:teuthology.orchestra.run.smithi040.stdout: "enable_auth": { 2023-07-18T03:42:09.383 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enable_auth", 2023-07-18T03:42:09.383 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.383 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.383 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.384 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.384 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.384 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.384 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.384 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.384 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.384 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.384 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.385 INFO:teuthology.orchestra.run.smithi040.stdout: "key_file": { 2023-07-18T03:42:09.385 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "key_file", 2023-07-18T03:42:09.385 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.385 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.385 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.385 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.385 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.386 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.386 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.386 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.386 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.386 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.386 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.386 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.387 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.387 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.387 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.387 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.387 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.387 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.387 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.387 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.388 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.388 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.388 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.388 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.388 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.388 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.388 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.388 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.389 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.389 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.389 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.389 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.389 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.389 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.389 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.390 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.390 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.390 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.390 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.390 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.390 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.390 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.390 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.391 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.391 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.391 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.391 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.391 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.391 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.391 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.392 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.392 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.392 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.392 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.392 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.392 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.392 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.393 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.393 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.393 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.393 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.393 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.393 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.393 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.393 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.394 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.394 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.394 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.394 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.394 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.394 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.394 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.395 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.395 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.395 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.395 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.395 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.395 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.395 INFO:teuthology.orchestra.run.smithi040.stdout: "server_addr": { 2023-07-18T03:42:09.396 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_addr", 2023-07-18T03:42:09.396 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.396 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.396 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.396 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.396 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.396 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.396 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.397 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.397 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.397 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.397 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.397 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.397 INFO:teuthology.orchestra.run.smithi040.stdout: "server_port": { 2023-07-18T03:42:09.397 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "server_port", 2023-07-18T03:42:09.397 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.398 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.398 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.398 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.398 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.398 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.398 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.398 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.398 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.399 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.399 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.399 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.399 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.399 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.399 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.399 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rgw", 2023-07-18T03:42:09.400 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.400 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.400 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.400 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.400 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.400 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.400 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.400 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.401 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.401 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.401 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.401 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.401 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.401 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.401 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.401 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.402 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.402 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.402 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.402 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.402 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.402 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.402 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.403 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.403 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.403 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.403 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.403 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.403 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.403 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.403 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.404 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.404 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.404 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.404 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.404 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.404 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.404 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.405 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.405 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.405 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.405 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.405 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.405 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.405 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.406 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.406 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.406 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.406 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.406 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.406 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.407 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.407 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.407 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.407 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.407 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.407 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.407 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.408 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.408 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.408 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.408 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.408 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.408 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.408 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.408 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.409 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.409 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.409 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.409 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.409 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.409 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.409 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.410 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.410 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rook", 2023-07-18T03:42:09.410 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.410 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.410 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.410 INFO:teuthology.orchestra.run.smithi040.stdout: "drive_group_interval": { 2023-07-18T03:42:09.410 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "drive_group_interval", 2023-07-18T03:42:09.410 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:09.411 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.411 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.411 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "300.0", 2023-07-18T03:42:09.411 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.411 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.411 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.411 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-18T03:42:09.411 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.412 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.412 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.412 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.412 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.412 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.412 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.413 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.413 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.413 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.413 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.413 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.413 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.413 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.413 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.414 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.414 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.414 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.414 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.414 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.414 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.414 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.414 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.415 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.415 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.415 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.415 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.415 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.415 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.415 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.416 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.416 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.416 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.416 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.416 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.416 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.416 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.417 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.417 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.417 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.417 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.417 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.417 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.417 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.417 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.418 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.418 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.418 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.418 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.418 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.418 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.419 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.419 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.419 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.419 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.419 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.419 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.419 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.419 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.420 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.420 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.420 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.420 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.420 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.420 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.420 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.421 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.421 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.421 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.421 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.421 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.421 INFO:teuthology.orchestra.run.smithi040.stdout: "storage_class": { 2023-07-18T03:42:09.421 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "storage_class", 2023-07-18T03:42:09.422 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.422 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.422 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.422 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "local", 2023-07-18T03:42:09.422 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.422 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.422 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.422 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-18T03:42:09.423 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.423 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.423 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.423 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.423 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.423 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.423 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.423 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "selftest", 2023-07-18T03:42:09.424 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.424 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.424 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.424 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.424 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.424 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.424 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.425 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.425 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.425 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.425 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.425 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.425 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.425 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.425 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.426 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.426 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.426 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.426 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.426 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.426 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.426 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.426 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.427 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.427 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.427 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.427 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.427 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.427 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.427 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.428 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.428 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.428 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.428 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.428 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.428 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.428 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.428 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.429 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.429 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.429 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.429 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.429 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.429 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.429 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.429 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.430 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.430 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.430 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.430 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.430 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.430 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.430 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.430 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.431 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.431 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.431 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.431 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.431 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.431 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.431 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.432 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.432 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.432 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.432 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.432 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.432 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.432 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.432 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.433 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.433 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.433 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.433 INFO:teuthology.orchestra.run.smithi040.stdout: "roption1": { 2023-07-18T03:42:09.433 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "roption1", 2023-07-18T03:42:09.433 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.433 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.433 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.434 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.434 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.434 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.434 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.434 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.434 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.434 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.434 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.435 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.435 INFO:teuthology.orchestra.run.smithi040.stdout: "roption2": { 2023-07-18T03:42:09.435 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "roption2", 2023-07-18T03:42:09.435 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.435 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.435 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.435 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "xyz", 2023-07-18T03:42:09.435 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.436 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.436 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.436 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.436 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.436 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.436 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.436 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.437 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption1": { 2023-07-18T03:42:09.437 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption1", 2023-07-18T03:42:09.437 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.437 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.437 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.437 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.437 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.437 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.438 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.438 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.438 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.438 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.438 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.438 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.438 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption2": { 2023-07-18T03:42:09.438 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption2", 2023-07-18T03:42:09.439 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.439 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.439 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.439 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.439 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.439 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.439 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.439 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.440 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.440 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.440 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.440 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.440 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption3": { 2023-07-18T03:42:09.440 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption3", 2023-07-18T03:42:09.440 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "float", 2023-07-18T03:42:09.440 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.441 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.441 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.441 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.441 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.441 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.441 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.441 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.441 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.442 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.442 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption4": { 2023-07-18T03:42:09.442 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption4", 2023-07-18T03:42:09.442 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.442 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.442 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.443 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.443 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.443 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.443 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.443 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.443 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.443 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.443 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.444 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption5": { 2023-07-18T03:42:09.444 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption5", 2023-07-18T03:42:09.444 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.444 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.444 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.444 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.444 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.445 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.445 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.445 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.445 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.445 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.445 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.445 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption6": { 2023-07-18T03:42:09.446 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption6", 2023-07-18T03:42:09.446 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.446 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.446 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.446 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.446 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.446 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.446 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.447 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.447 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.447 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.447 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.447 INFO:teuthology.orchestra.run.smithi040.stdout: "rwoption7": { 2023-07-18T03:42:09.447 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rwoption7", 2023-07-18T03:42:09.447 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.448 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.448 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.448 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.448 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "1", 2023-07-18T03:42:09.448 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "42", 2023-07-18T03:42:09.448 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.448 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.449 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.449 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.449 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.449 INFO:teuthology.orchestra.run.smithi040.stdout: "testkey": { 2023-07-18T03:42:09.449 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testkey", 2023-07-18T03:42:09.449 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.449 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.450 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.450 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.450 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.450 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.450 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.450 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.450 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.450 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.451 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.451 INFO:teuthology.orchestra.run.smithi040.stdout: "testlkey": { 2023-07-18T03:42:09.451 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testlkey", 2023-07-18T03:42:09.451 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.451 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.451 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.451 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.452 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.452 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.452 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.452 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.452 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.452 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.452 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.453 INFO:teuthology.orchestra.run.smithi040.stdout: "testnewline": { 2023-07-18T03:42:09.453 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "testnewline", 2023-07-18T03:42:09.453 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.453 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.453 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.453 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.453 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.453 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.454 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.454 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.454 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.454 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.454 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.454 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.454 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.455 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "snap_schedule", 2023-07-18T03:42:09.455 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.455 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.455 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.455 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_m_granularity": { 2023-07-18T03:42:09.455 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "allow_m_granularity", 2023-07-18T03:42:09.456 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.456 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.456 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.456 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.456 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.456 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.456 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.456 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "allow minute scheduled snapshots", 2023-07-18T03:42:09.457 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.457 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.457 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.457 INFO:teuthology.orchestra.run.smithi040.stdout: "dump_on_update": { 2023-07-18T03:42:09.457 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "dump_on_update", 2023-07-18T03:42:09.457 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.457 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.458 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.458 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.458 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.458 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.458 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.458 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "dump database to debug log on update", 2023-07-18T03:42:09.458 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.459 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.459 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.459 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.459 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.459 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.459 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.459 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.460 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.460 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.460 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.460 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.460 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.460 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.460 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.460 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.461 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.461 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.461 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.461 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.461 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.461 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.461 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.462 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.462 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.462 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.462 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.462 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.462 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.462 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.463 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.463 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.463 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.463 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.463 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.463 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.463 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.464 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.464 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.464 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.464 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.464 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.464 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.464 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.464 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.465 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.465 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.465 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.465 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.465 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.465 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.465 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.465 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.466 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.466 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.466 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.466 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.466 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.466 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.466 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.466 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.467 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.467 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.467 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.467 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.467 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.467 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.467 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.467 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.468 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.468 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.468 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.468 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.468 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "stats", 2023-07-18T03:42:09.468 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.469 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.469 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.469 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.469 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.469 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.469 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.469 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.469 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.475 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.475 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.475 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.475 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.475 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.475 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.475 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.475 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "status", 2023-07-18T03:42:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.482 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.482 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.482 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.482 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.482 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.482 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.482 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.482 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.483 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.483 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.483 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.483 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.483 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.483 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.483 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.484 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.484 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.484 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.484 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.484 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.484 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.484 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.485 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.485 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.485 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.485 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.485 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.485 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.485 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.485 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.486 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.486 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.486 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.486 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.486 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.486 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.486 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.486 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "telegraf", 2023-07-18T03:42:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: "address": { 2023-07-18T03:42:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "address", 2023-07-18T03:42:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-18T03:42:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2023-07-18T03:42:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2023-07-18T03:42:09.491 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:09.491 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.491 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.491 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "15", 2023-07-18T03:42:09.491 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.491 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.491 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.491 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.499 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.499 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.499 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.499 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.499 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.499 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.499 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.500 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.500 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.500 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.500 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.500 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.500 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.500 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.500 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.501 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.501 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.501 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.501 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.501 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.501 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.501 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.501 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "telemetry", 2023-07-18T03:42:09.502 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.502 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.502 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.502 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_basic": { 2023-07-18T03:42:09.502 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_basic", 2023-07-18T03:42:09.502 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.502 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.502 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.503 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.503 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.503 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.503 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.503 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-18T03:42:09.503 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.503 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.503 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.504 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.504 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_crash": { 2023-07-18T03:42:09.504 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_crash", 2023-07-18T03:42:09.504 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.504 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.504 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.504 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.505 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.505 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.505 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.505 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-18T03:42:09.505 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.505 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.505 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.505 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.506 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_device": { 2023-07-18T03:42:09.506 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_device", 2023-07-18T03:42:09.506 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.506 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.506 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.506 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "True", 2023-07-18T03:42:09.506 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.506 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.507 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.507 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-18T03:42:09.507 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.507 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.507 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.507 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.507 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_ident": { 2023-07-18T03:42:09.507 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_ident", 2023-07-18T03:42:09.508 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.508 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.508 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.508 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.508 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.508 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.508 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.508 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-18T03:42:09.509 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.509 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.509 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.509 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.509 INFO:teuthology.orchestra.run.smithi040.stdout: "channel_perf": { 2023-07-18T03:42:09.509 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "channel_perf", 2023-07-18T03:42:09.509 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.510 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.510 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.510 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.510 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.510 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.510 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.510 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-18T03:42:09.510 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.511 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.511 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.511 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.511 INFO:teuthology.orchestra.run.smithi040.stdout: "contact": { 2023-07-18T03:42:09.511 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "contact", 2023-07-18T03:42:09.511 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.511 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.511 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.512 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.512 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.512 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.512 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.512 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.512 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.512 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.512 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.513 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.513 INFO:teuthology.orchestra.run.smithi040.stdout: "description": { 2023-07-18T03:42:09.513 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "description", 2023-07-18T03:42:09.513 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.513 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.513 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.513 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.513 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.514 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.514 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.514 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.514 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.514 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.514 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.514 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.515 INFO:teuthology.orchestra.run.smithi040.stdout: "device_url": { 2023-07-18T03:42:09.515 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "device_url", 2023-07-18T03:42:09.515 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.515 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.515 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.515 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-18T03:42:09.515 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.515 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.516 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.516 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.516 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.516 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.516 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.516 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.516 INFO:teuthology.orchestra.run.smithi040.stdout: "enabled": { 2023-07-18T03:42:09.517 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "enabled", 2023-07-18T03:42:09.517 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.517 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.517 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.517 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.517 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.517 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.517 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.518 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.518 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.518 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.518 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.518 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.518 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2023-07-18T03:42:09.518 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2023-07-18T03:42:09.518 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.519 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.519 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.519 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "24", 2023-07-18T03:42:09.519 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "8", 2023-07-18T03:42:09.519 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.519 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.519 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.519 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.520 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.520 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.520 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.520 INFO:teuthology.orchestra.run.smithi040.stdout: "last_opt_revision": { 2023-07-18T03:42:09.520 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "last_opt_revision", 2023-07-18T03:42:09.520 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.520 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.520 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.521 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "1", 2023-07-18T03:42:09.521 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.521 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.521 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.521 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.521 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.521 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.521 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.522 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.522 INFO:teuthology.orchestra.run.smithi040.stdout: "leaderboard": { 2023-07-18T03:42:09.522 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "leaderboard", 2023-07-18T03:42:09.522 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.522 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.522 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.522 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.522 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.523 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.523 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.523 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.523 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.523 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.523 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.523 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.524 INFO:teuthology.orchestra.run.smithi040.stdout: "leaderboard_description": { 2023-07-18T03:42:09.524 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "leaderboard_description", 2023-07-18T03:42:09.524 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.524 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.524 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.524 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.524 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.524 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.525 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.525 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.525 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.525 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.525 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.525 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.525 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.525 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.526 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.526 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.526 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.526 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.526 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.526 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.526 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.526 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.527 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.527 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.527 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.527 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.527 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.527 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.527 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.528 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.528 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.528 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.528 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.528 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.528 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.528 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.528 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.529 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.529 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.529 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.529 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.529 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.529 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.529 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.529 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.530 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.530 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.530 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.530 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.530 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.530 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.530 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.530 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.531 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.531 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.531 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.531 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.531 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.531 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.531 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.531 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.532 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.532 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.532 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.532 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.532 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.532 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.532 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.533 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.533 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.533 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.533 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.533 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.533 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.533 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.533 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.534 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.534 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.534 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.534 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.534 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.534 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.534 INFO:teuthology.orchestra.run.smithi040.stdout: "organization": { 2023-07-18T03:42:09.534 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "organization", 2023-07-18T03:42:09.535 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.535 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.535 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.535 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.535 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.535 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.535 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.535 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.536 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.536 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.536 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.536 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.536 INFO:teuthology.orchestra.run.smithi040.stdout: "proxy": { 2023-07-18T03:42:09.536 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "proxy", 2023-07-18T03:42:09.536 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.536 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.537 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.537 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.537 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.537 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.537 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.537 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.537 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.537 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.538 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.538 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.538 INFO:teuthology.orchestra.run.smithi040.stdout: "url": { 2023-07-18T03:42:09.538 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "url", 2023-07-18T03:42:09.538 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.538 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.538 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.538 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-18T03:42:09.539 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.539 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.539 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.539 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.539 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.539 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.539 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.540 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.540 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.540 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.540 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.540 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "test_orchestrator", 2023-07-18T03:42:09.540 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.540 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.540 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.541 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.541 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.541 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.541 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.541 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.541 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.541 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.541 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.542 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.542 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.542 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.542 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.542 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.542 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.542 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.542 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.543 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.543 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.543 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.543 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.543 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.543 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.543 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.543 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.544 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.544 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.544 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.544 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.544 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.544 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.544 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.544 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.545 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.545 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.545 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.545 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.545 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.545 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.545 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.546 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.546 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.546 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.546 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.546 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.546 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.546 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.546 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.547 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.547 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.547 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.547 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.547 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.547 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.547 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.547 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.548 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.548 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.548 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.548 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.548 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.548 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.548 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.548 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.549 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.549 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.549 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.549 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.549 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.549 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.549 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.549 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.550 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.550 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.550 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "volumes", 2023-07-18T03:42:09.550 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.550 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.550 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.550 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.550 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.551 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.551 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.551 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.551 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.551 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.551 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.551 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.552 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.552 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.552 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.552 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.552 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.552 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.552 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.552 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.553 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.553 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.553 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.553 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.553 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.553 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.553 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.553 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.554 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.554 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.554 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.554 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.554 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.554 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.554 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.554 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.555 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.555 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.555 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.555 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.555 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.555 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.555 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.555 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.556 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.556 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.556 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.556 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.556 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.556 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.556 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.557 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.557 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.557 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.557 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.557 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.557 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.557 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.557 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.558 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.558 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.558 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.558 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.558 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.558 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.558 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.558 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.559 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.559 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.559 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.559 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.559 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.559 INFO:teuthology.orchestra.run.smithi040.stdout: "max_concurrent_clones": { 2023-07-18T03:42:09.559 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "max_concurrent_clones", 2023-07-18T03:42:09.560 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.560 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.560 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.560 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "4", 2023-07-18T03:42:09.560 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.560 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.560 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.560 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-18T03:42:09.561 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.561 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.561 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.561 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.561 INFO:teuthology.orchestra.run.smithi040.stdout: "snapshot_clone_delay": { 2023-07-18T03:42:09.561 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "snapshot_clone_delay", 2023-07-18T03:42:09.561 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.561 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.562 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.562 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "0", 2023-07-18T03:42:09.562 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.562 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.562 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.562 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-18T03:42:09.562 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.563 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.563 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.563 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.563 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.563 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.563 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix", 2023-07-18T03:42:09.563 INFO:teuthology.orchestra.run.smithi040.stdout: "can_run": true, 2023-07-18T03:42:09.564 INFO:teuthology.orchestra.run.smithi040.stdout: "error_string": "", 2023-07-18T03:42:09.564 INFO:teuthology.orchestra.run.smithi040.stdout: "module_options": { 2023-07-18T03:42:09.564 INFO:teuthology.orchestra.run.smithi040.stdout: "discovery_interval": { 2023-07-18T03:42:09.564 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "discovery_interval", 2023-07-18T03:42:09.564 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "uint", 2023-07-18T03:42:09.564 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.564 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.564 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "100", 2023-07-18T03:42:09.565 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.565 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.565 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.565 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.565 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.565 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.565 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.566 INFO:teuthology.orchestra.run.smithi040.stdout: "identifier": { 2023-07-18T03:42:09.566 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "identifier", 2023-07-18T03:42:09.566 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.566 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.566 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.566 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.566 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.566 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.567 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.567 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.567 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.567 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.567 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.567 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.567 INFO:teuthology.orchestra.run.smithi040.stdout: "interval": { 2023-07-18T03:42:09.567 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "interval", 2023-07-18T03:42:09.568 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "secs", 2023-07-18T03:42:09.568 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.568 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.568 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "60", 2023-07-18T03:42:09.568 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.568 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.568 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.569 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.569 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.569 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.569 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.569 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.569 INFO:teuthology.orchestra.run.smithi040.stdout: "log_level": { 2023-07-18T03:42:09.569 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_level", 2023-07-18T03:42:09.569 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.570 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.570 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.570 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.570 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.570 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.570 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.570 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.570 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.571 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.571 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.571 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.571 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.571 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.571 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.571 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.571 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.572 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.572 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster": { 2023-07-18T03:42:09.572 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster", 2023-07-18T03:42:09.572 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.572 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.572 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.572 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.573 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.573 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.573 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.573 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.573 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.573 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.573 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.574 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_cluster_level": { 2023-07-18T03:42:09.574 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_cluster_level", 2023-07-18T03:42:09.574 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.574 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.574 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.574 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "info", 2023-07-18T03:42:09.574 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.575 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.575 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [ 2023-07-18T03:42:09.575 INFO:teuthology.orchestra.run.smithi040.stdout: "", 2023-07-18T03:42:09.575 INFO:teuthology.orchestra.run.smithi040.stdout: "critical", 2023-07-18T03:42:09.575 INFO:teuthology.orchestra.run.smithi040.stdout: "debug", 2023-07-18T03:42:09.575 INFO:teuthology.orchestra.run.smithi040.stdout: "error", 2023-07-18T03:42:09.575 INFO:teuthology.orchestra.run.smithi040.stdout: "info", 2023-07-18T03:42:09.575 INFO:teuthology.orchestra.run.smithi040.stdout: "warning" 2023-07-18T03:42:09.576 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.576 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.576 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.576 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.576 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.576 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.576 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": { 2023-07-18T03:42:09.576 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "log_to_file", 2023-07-18T03:42:09.577 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "bool", 2023-07-18T03:42:09.577 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.577 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-07-18T03:42:09.577 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "False", 2023-07-18T03:42:09.577 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.577 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.577 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.578 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.578 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.578 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.578 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.578 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_host": { 2023-07-18T03:42:09.578 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_host", 2023-07-18T03:42:09.578 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.579 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.579 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.579 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "", 2023-07-18T03:42:09.579 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.579 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.579 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.579 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.579 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.580 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.580 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.580 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.580 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_port": { 2023-07-18T03:42:09.580 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_port", 2023-07-18T03:42:09.580 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "int", 2023-07-18T03:42:09.580 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.580 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.581 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "10051", 2023-07-18T03:42:09.581 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.581 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.581 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.581 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.581 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.581 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.581 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.582 INFO:teuthology.orchestra.run.smithi040.stdout: "zabbix_sender": { 2023-07-18T03:42:09.582 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "zabbix_sender", 2023-07-18T03:42:09.582 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "str", 2023-07-18T03:42:09.582 INFO:teuthology.orchestra.run.smithi040.stdout: "level": "advanced", 2023-07-18T03:42:09.582 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 0, 2023-07-18T03:42:09.582 INFO:teuthology.orchestra.run.smithi040.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-18T03:42:09.582 INFO:teuthology.orchestra.run.smithi040.stdout: "min": "", 2023-07-18T03:42:09.583 INFO:teuthology.orchestra.run.smithi040.stdout: "max": "", 2023-07-18T03:42:09.583 INFO:teuthology.orchestra.run.smithi040.stdout: "enum_allowed": [], 2023-07-18T03:42:09.583 INFO:teuthology.orchestra.run.smithi040.stdout: "desc": "", 2023-07-18T03:42:09.583 INFO:teuthology.orchestra.run.smithi040.stdout: "long_desc": "", 2023-07-18T03:42:09.583 INFO:teuthology.orchestra.run.smithi040.stdout: "tags": [], 2023-07-18T03:42:09.583 INFO:teuthology.orchestra.run.smithi040.stdout: "see_also": [] 2023-07-18T03:42:09.583 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.584 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.584 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.584 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.584 INFO:teuthology.orchestra.run.smithi040.stdout: "services": { 2023-07-18T03:42:09.584 INFO:teuthology.orchestra.run.smithi040.stdout: "dashboard": "https://172.21.15.40:8443/" 2023-07-18T03:42:09.584 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.584 INFO:teuthology.orchestra.run.smithi040.stdout: "always_on_modules": { 2023-07-18T03:42:09.584 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus": [ 2023-07-18T03:42:09.585 INFO:teuthology.orchestra.run.smithi040.stdout: "balancer", 2023-07-18T03:42:09.585 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2023-07-18T03:42:09.585 INFO:teuthology.orchestra.run.smithi040.stdout: "devicehealth", 2023-07-18T03:42:09.585 INFO:teuthology.orchestra.run.smithi040.stdout: "orchestrator", 2023-07-18T03:42:09.585 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscaler", 2023-07-18T03:42:09.585 INFO:teuthology.orchestra.run.smithi040.stdout: "progress", 2023-07-18T03:42:09.585 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_support", 2023-07-18T03:42:09.585 INFO:teuthology.orchestra.run.smithi040.stdout: "status", 2023-07-18T03:42:09.586 INFO:teuthology.orchestra.run.smithi040.stdout: "telemetry", 2023-07-18T03:42:09.586 INFO:teuthology.orchestra.run.smithi040.stdout: "volumes" 2023-07-18T03:42:09.586 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.586 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific": [ 2023-07-18T03:42:09.586 INFO:teuthology.orchestra.run.smithi040.stdout: "balancer", 2023-07-18T03:42:09.586 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2023-07-18T03:42:09.586 INFO:teuthology.orchestra.run.smithi040.stdout: "devicehealth", 2023-07-18T03:42:09.586 INFO:teuthology.orchestra.run.smithi040.stdout: "orchestrator", 2023-07-18T03:42:09.587 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscaler", 2023-07-18T03:42:09.587 INFO:teuthology.orchestra.run.smithi040.stdout: "progress", 2023-07-18T03:42:09.587 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_support", 2023-07-18T03:42:09.587 INFO:teuthology.orchestra.run.smithi040.stdout: "status", 2023-07-18T03:42:09.587 INFO:teuthology.orchestra.run.smithi040.stdout: "telemetry", 2023-07-18T03:42:09.587 INFO:teuthology.orchestra.run.smithi040.stdout: "volumes" 2023-07-18T03:42:09.587 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.588 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy": [ 2023-07-18T03:42:09.588 INFO:teuthology.orchestra.run.smithi040.stdout: "balancer", 2023-07-18T03:42:09.588 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2023-07-18T03:42:09.588 INFO:teuthology.orchestra.run.smithi040.stdout: "devicehealth", 2023-07-18T03:42:09.588 INFO:teuthology.orchestra.run.smithi040.stdout: "orchestrator", 2023-07-18T03:42:09.588 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscaler", 2023-07-18T03:42:09.588 INFO:teuthology.orchestra.run.smithi040.stdout: "progress", 2023-07-18T03:42:09.588 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_support", 2023-07-18T03:42:09.589 INFO:teuthology.orchestra.run.smithi040.stdout: "status", 2023-07-18T03:42:09.589 INFO:teuthology.orchestra.run.smithi040.stdout: "telemetry", 2023-07-18T03:42:09.589 INFO:teuthology.orchestra.run.smithi040.stdout: "volumes" 2023-07-18T03:42:09.589 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-07-18T03:42:09.589 INFO:teuthology.orchestra.run.smithi040.stdout: "reef": [ 2023-07-18T03:42:09.589 INFO:teuthology.orchestra.run.smithi040.stdout: "balancer", 2023-07-18T03:42:09.589 INFO:teuthology.orchestra.run.smithi040.stdout: "crash", 2023-07-18T03:42:09.589 INFO:teuthology.orchestra.run.smithi040.stdout: "devicehealth", 2023-07-18T03:42:09.590 INFO:teuthology.orchestra.run.smithi040.stdout: "orchestrator", 2023-07-18T03:42:09.590 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscaler", 2023-07-18T03:42:09.590 INFO:teuthology.orchestra.run.smithi040.stdout: "progress", 2023-07-18T03:42:09.590 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_support", 2023-07-18T03:42:09.590 INFO:teuthology.orchestra.run.smithi040.stdout: "status", 2023-07-18T03:42:09.590 INFO:teuthology.orchestra.run.smithi040.stdout: "telemetry", 2023-07-18T03:42:09.590 INFO:teuthology.orchestra.run.smithi040.stdout: "volumes" 2023-07-18T03:42:09.590 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-07-18T03:42:09.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.591 INFO:teuthology.orchestra.run.smithi040.stdout: "last_failure_osd_epoch": 3, 2023-07-18T03:42:09.591 INFO:teuthology.orchestra.run.smithi040.stdout: "active_clients": [ 2023-07-18T03:42:09.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.591 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "devicehealth", 2023-07-18T03:42:09.591 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-07-18T03:42:09.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.591 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-07-18T03:42:09.592 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:0", 2023-07-18T03:42:09.592 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 4139476041 2023-07-18T03:42:09.592 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.592 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-07-18T03:42:09.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.592 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.592 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "libcephsqlite", 2023-07-18T03:42:09.593 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-07-18T03:42:09.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.593 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-07-18T03:42:09.593 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:0", 2023-07-18T03:42:09.593 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 755430971 2023-07-18T03:42:09.593 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.593 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-07-18T03:42:09.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.594 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.594 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "rbd_support", 2023-07-18T03:42:09.594 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-07-18T03:42:09.594 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.594 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-07-18T03:42:09.594 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:0", 2023-07-18T03:42:09.594 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3730969862 2023-07-18T03:42:09.594 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.595 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-07-18T03:42:09.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:09.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.595 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "volumes", 2023-07-18T03:42:09.595 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-07-18T03:42:09.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-07-18T03:42:09.595 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-07-18T03:42:09.595 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:0", 2023-07-18T03:42:09.596 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3309852071 2023-07-18T03:42:09.596 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.596 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-07-18T03:42:09.596 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:09.596 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-07-18T03:42:09.596 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:42:09.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:07 smithi064 ceph-mon[98059]: pgmap v631: 97 pgs: 97 active+clean; 618 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 448 op/s 2023-07-18T03:42:09.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:07 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3581550512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:42:09.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:07 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1248548659' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-18T03:42:09.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:09 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:42:09.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:09 smithi064 ceph-mon[98059]: pgmap v632: 97 pgs: 97 active+clean; 594 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 426 op/s 2023-07-18T03:42:09.598 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:07 smithi134 ceph-mon[98216]: pgmap v631: 97 pgs: 97 active+clean; 618 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 448 op/s 2023-07-18T03:42:09.599 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:07 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3581550512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:42:09.599 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:07 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1248548659' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-18T03:42:09.599 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:09 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:42:09.599 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:09 smithi134 ceph-mon[98216]: pgmap v632: 97 pgs: 97 active+clean; 594 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 426 op/s 2023-07-18T03:42:09.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:07.025+0000 7fb79f906700 1 -- 172.21.15.40:0/1248548659 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb7840a1220 msgr2=0x7fb7840a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:09.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:07.025+0000 7fb79f906700 1 --2- 172.21.15.40:0/1248548659 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb7840a1220 0x7fb7840a36d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb794009be0 tx=0x7fb794009260 comp rx=0 tx=0).stop 2023-07-18T03:42:09.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:07.025+0000 7fb79f906700 1 -- 172.21.15.40:0/1248548659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7980fe9d0 msgr2=0x7fb798102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:09.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:07.025+0000 7fb79f906700 1 --2- 172.21.15.40:0/1248548659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7980fe9d0 0x7fb798102b80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00d980 tx=0x7fb78c00dc90 comp rx=0 tx=0).stop 2023-07-18T03:42:09.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:07.025+0000 7fb79f906700 1 -- 172.21.15.40:0/1248548659 shutdown_connections 2023-07-18T03:42:09.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:07.025+0000 7fb79f906700 1 --2- 172.21.15.40:0/1248548659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb798100550 0x7fb798101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:09.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:07.025+0000 7fb79f906700 1 --2- 172.21.15.40:0/1248548659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7980fe9d0 0x7fb798102b80 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:09.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:07.025+0000 7fb79f906700 1 --2- 172.21.15.40:0/1248548659 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fb7840a1220 0x7fb7840a36d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:09.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:07.025+0000 7fb79f906700 1 --2- 172.21.15.40:0/1248548659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7980ffbc0 0x7fb7981011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:09.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:07.025+0000 7fb79f906700 1 -- 172.21.15.40:0/1248548659 >> 172.21.15.40:0/1248548659 conn(0x7fb7980f9fd0 msgr2=0x7fb7980fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:09.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:07.026+0000 7fb79f906700 1 -- 172.21.15.40:0/1248548659 shutdown_connections 2023-07-18T03:42:09.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:07.026+0000 7fb79f906700 1 -- 172.21.15.40:0/1248548659 wait complete. 2023-07-18T03:42:09.607 DEBUG:tasks.check_counter:Getting stats from a 2023-07-18T03:42:09.608 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph daemon mds.a perf dump 2023-07-18T03:42:09.608 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:07 smithi040 ceph-mon[96439]: pgmap v631: 97 pgs: 97 active+clean; 618 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 448 op/s 2023-07-18T03:42:09.609 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:07 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3581550512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:42:09.609 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:07 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1248548659' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-18T03:42:09.609 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:09 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:42:09.609 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:09 smithi040 ceph-mon[96439]: pgmap v632: 97 pgs: 97 active+clean; 594 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 426 op/s 2023-07-18T03:42:10.179 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:42:11.385 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:42:11.386 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:42:11.386 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 500, 2023-07-18T03:42:11.386 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 450, 2023-07-18T03:42:11.386 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 42779890, 2023-07-18T03:42:11.386 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 192700, 2023-07-18T03:42:11.386 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 7, 2023-07-18T03:42:11.387 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 3, 2023-07-18T03:42:11.387 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.084189305, 2023-07-18T03:42:11.387 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.019895326, 2023-07-18T03:42:11.387 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.048608923, 2023-07-18T03:42:11.387 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.014154432, 2023-07-18T03:42:11.387 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:11.387 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 449, 2023-07-18T03:42:11.388 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.131476519, 2023-07-18T03:42:11.388 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000292820 2023-07-18T03:42:11.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.389 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:11.389 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.390 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.390 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.390 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 771504, 2023-07-18T03:42:11.391 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 119920 2023-07-18T03:42:11.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.391 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:42:11.391 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 1024, 2023-07-18T03:42:11.392 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 1181, 2023-07-18T03:42:11.392 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 76874893, 2023-07-18T03:42:11.392 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 664686, 2023-07-18T03:42:11.393 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 6, 2023-07-18T03:42:11.393 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 5, 2023-07-18T03:42:11.393 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.149098526, 2023-07-18T03:42:11.394 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.031843657, 2023-07-18T03:42:11.394 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.075880048, 2023-07-18T03:42:11.394 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.033222630, 2023-07-18T03:42:11.395 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:11.395 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1179, 2023-07-18T03:42:11.395 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.198660334, 2023-07-18T03:42:11.396 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000168499 2023-07-18T03:42:11.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.396 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:11.397 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.397 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.397 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.397 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.397 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-18T03:42:11.398 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 346528 2023-07-18T03:42:11.398 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.398 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:42:11.398 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 1235, 2023-07-18T03:42:11.399 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 1235, 2023-07-18T03:42:11.399 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 42329423, 2023-07-18T03:42:11.399 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 383347, 2023-07-18T03:42:11.399 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 6, 2023-07-18T03:42:11.399 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 5, 2023-07-18T03:42:11.399 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.159809860, 2023-07-18T03:42:11.400 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.043718868, 2023-07-18T03:42:11.400 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.061317285, 2023-07-18T03:42:11.400 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.043833485, 2023-07-18T03:42:11.400 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:11.400 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1234, 2023-07-18T03:42:11.400 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.304698138, 2023-07-18T03:42:11.400 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000246919 2023-07-18T03:42:11.401 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.401 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:11.401 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.401 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.402 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.402 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-18T03:42:11.402 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 448 2023-07-18T03:42:11.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.402 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": { 2023-07-18T03:42:11.403 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 1, 2023-07-18T03:42:11.403 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2023-07-18T03:42:11.403 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.403 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-MDSRank": { 2023-07-18T03:42:11.403 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2023-07-18T03:42:11.403 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2023-07-18T03:42:11.403 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 4155, 2023-07-18T03:42:11.404 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.134828191, 2023-07-18T03:42:11.404 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000032449 2023-07-18T03:42:11.404 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:11.404 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.404 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-PurgeQueue": { 2023-07-18T03:42:11.404 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2023-07-18T03:42:11.404 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2023-07-18T03:42:11.405 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.405 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.405 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.405 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:11.405 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.405 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2023-07-18T03:42:11.405 INFO:teuthology.orchestra.run.smithi040.stdout: "request": 0, 2023-07-18T03:42:11.406 INFO:teuthology.orchestra.run.smithi040.stdout: "reply": 0, 2023-07-18T03:42:11.406 INFO:teuthology.orchestra.run.smithi040.stdout: "reply_latency": { 2023-07-18T03:42:11.406 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.406 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.406 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.406 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.407 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_reply": 0, 2023-07-18T03:42:11.407 INFO:teuthology.orchestra.run.smithi040.stdout: "forward": 0, 2023-07-18T03:42:11.407 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_complete": 0, 2023-07-18T03:42:11.407 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_keys": 0, 2023-07-18T03:42:11.407 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_commit": 0, 2023-07-18T03:42:11.407 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_split": 0, 2023-07-18T03:42:11.407 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_merge": 0, 2023-07-18T03:42:11.408 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes": 8372, 2023-07-18T03:42:11.408 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_top": 0, 2023-07-18T03:42:11.408 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_bottom": 0, 2023-07-18T03:42:11.408 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pin_tail": 8372, 2023-07-18T03:42:11.408 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pinned": 8372, 2023-07-18T03:42:11.408 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_expired": 804, 2023-07-18T03:42:11.409 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_with_caps": 0, 2023-07-18T03:42:11.409 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": 0, 2023-07-18T03:42:11.409 INFO:teuthology.orchestra.run.smithi040.stdout: "subtrees": 1, 2023-07-18T03:42:11.409 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse": 0, 2023-07-18T03:42:11.409 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_hit": 0, 2023-07-18T03:42:11.409 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_forward": 0, 2023-07-18T03:42:11.410 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_discover": 0, 2023-07-18T03:42:11.410 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_dir_fetch": 0, 2023-07-18T03:42:11.410 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_remote_ino": 0, 2023-07-18T03:42:11.410 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_lock": 0, 2023-07-18T03:42:11.410 INFO:teuthology.orchestra.run.smithi040.stdout: "load_cent": 0, 2023-07-18T03:42:11.410 INFO:teuthology.orchestra.run.smithi040.stdout: "q": 0, 2023-07-18T03:42:11.411 INFO:teuthology.orchestra.run.smithi040.stdout: "exported": 0, 2023-07-18T03:42:11.411 INFO:teuthology.orchestra.run.smithi040.stdout: "exported_inodes": 0, 2023-07-18T03:42:11.411 INFO:teuthology.orchestra.run.smithi040.stdout: "imported": 0, 2023-07-18T03:42:11.411 INFO:teuthology.orchestra.run.smithi040.stdout: "imported_inodes": 0, 2023-07-18T03:42:11.411 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_dir_fetch": 0, 2023-07-18T03:42:11.411 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_backtrace_fetch": 0, 2023-07-18T03:42:11.412 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_peer_discover": 0, 2023-07-18T03:42:11.412 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rfiles": 0, 2023-07-18T03:42:11.412 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rbytes": 0, 2023-07-18T03:42:11.412 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rsnaps": 0, 2023-07-18T03:42:11.412 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_fetch": 0, 2023-07-18T03:42:11.413 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_set_tag": 0, 2023-07-18T03:42:11.413 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_repaired": 0, 2023-07-18T03:42:11.413 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_inotable_repaired": 0, 2023-07-18T03:42:11.413 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_inodes": 0, 2023-07-18T03:42:11.413 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_base_inodes": 0, 2023-07-18T03:42:11.413 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dirfrag_rstats": 0, 2023-07-18T03:42:11.414 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_file_inodes": 0, 2023-07-18T03:42:11.414 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_inode_file_caps": 0, 2023-07-18T03:42:11.414 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_revoke": 0, 2023-07-18T03:42:11.414 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_grant": 0, 2023-07-18T03:42:11.414 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_trunc": 0, 2023-07-18T03:42:11.414 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-18T03:42:11.414 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-18T03:42:11.414 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps": 0, 2023-07-18T03:42:11.415 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps_dirty": 0, 2023-07-18T03:42:11.415 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_cap_release": 0, 2023-07-18T03:42:11.415 INFO:teuthology.orchestra.run.smithi040.stdout: "process_request_cap_release": 0 2023-07-18T03:42:11.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.415 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache": { 2023-07-18T03:42:11.415 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update": 0, 2023-07-18T03:42:11.415 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update_receipt": 0, 2023-07-18T03:42:11.415 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_try_discover": 0, 2023-07-18T03:42:11.416 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_send_discover": 0, 2023-07-18T03:42:11.416 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_handle_discover": 0, 2023-07-18T03:42:11.416 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays": 0, 2023-07-18T03:42:11.416 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_delayed": 0, 2023-07-18T03:42:11.416 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_enqueuing": 0, 2023-07-18T03:42:11.416 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_created": 0, 2023-07-18T03:42:11.416 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_enqueued": 0, 2023-07-18T03:42:11.417 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_reintegrated": 0, 2023-07-18T03:42:11.417 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_migrated": 0, 2023-07-18T03:42:11.417 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_processing": 0, 2023-07-18T03:42:11.417 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_enqueued": 0, 2023-07-18T03:42:11.417 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_prioritized": 0, 2023-07-18T03:42:11.417 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_started": 0, 2023-07-18T03:42:11.417 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_completed": 0, 2023-07-18T03:42:11.417 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_enqueue_scrub": 0, 2023-07-18T03:42:11.418 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_exportdir": 0, 2023-07-18T03:42:11.418 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_flush": 0, 2023-07-18T03:42:11.418 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragmentdir": 0, 2023-07-18T03:42:11.418 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragstats": 0, 2023-07-18T03:42:11.418 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_inodestats": 0 2023-07-18T03:42:11.418 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.418 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log": { 2023-07-18T03:42:11.418 INFO:teuthology.orchestra.run.smithi040.stdout: "evadd": 0, 2023-07-18T03:42:11.419 INFO:teuthology.orchestra.run.smithi040.stdout: "evex": 0, 2023-07-18T03:42:11.419 INFO:teuthology.orchestra.run.smithi040.stdout: "evtrm": 0, 2023-07-18T03:42:11.419 INFO:teuthology.orchestra.run.smithi040.stdout: "ev": 0, 2023-07-18T03:42:11.419 INFO:teuthology.orchestra.run.smithi040.stdout: "evexg": 0, 2023-07-18T03:42:11.419 INFO:teuthology.orchestra.run.smithi040.stdout: "evexd": 0, 2023-07-18T03:42:11.419 INFO:teuthology.orchestra.run.smithi040.stdout: "segadd": 0, 2023-07-18T03:42:11.419 INFO:teuthology.orchestra.run.smithi040.stdout: "segex": 0, 2023-07-18T03:42:11.419 INFO:teuthology.orchestra.run.smithi040.stdout: "segtrm": 0, 2023-07-18T03:42:11.420 INFO:teuthology.orchestra.run.smithi040.stdout: "seg": 84, 2023-07-18T03:42:11.420 INFO:teuthology.orchestra.run.smithi040.stdout: "segexg": 0, 2023-07-18T03:42:11.420 INFO:teuthology.orchestra.run.smithi040.stdout: "segexd": 0, 2023-07-18T03:42:11.420 INFO:teuthology.orchestra.run.smithi040.stdout: "expos": 4194304, 2023-07-18T03:42:11.420 INFO:teuthology.orchestra.run.smithi040.stdout: "wrpos": 0, 2023-07-18T03:42:11.420 INFO:teuthology.orchestra.run.smithi040.stdout: "rdpos": 164687138, 2023-07-18T03:42:11.420 INFO:teuthology.orchestra.run.smithi040.stdout: "jlat": { 2023-07-18T03:42:11.420 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.421 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.421 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.421 INFO:teuthology.orchestra.run.smithi040.stdout: "replayed": 69018 2023-07-18T03:42:11.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.421 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_mem": { 2023-07-18T03:42:11.421 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 14, 2023-07-18T03:42:11.422 INFO:teuthology.orchestra.run.smithi040.stdout: "ino+": 19848, 2023-07-18T03:42:11.422 INFO:teuthology.orchestra.run.smithi040.stdout: "ino-": 19834, 2023-07-18T03:42:11.422 INFO:teuthology.orchestra.run.smithi040.stdout: "dir": 19, 2023-07-18T03:42:11.422 INFO:teuthology.orchestra.run.smithi040.stdout: "dir+": 839, 2023-07-18T03:42:11.422 INFO:teuthology.orchestra.run.smithi040.stdout: "dir-": 820, 2023-07-18T03:42:11.422 INFO:teuthology.orchestra.run.smithi040.stdout: "dn": 8372, 2023-07-18T03:42:11.422 INFO:teuthology.orchestra.run.smithi040.stdout: "dn+": 42768, 2023-07-18T03:42:11.422 INFO:teuthology.orchestra.run.smithi040.stdout: "dn-": 34396, 2023-07-18T03:42:11.423 INFO:teuthology.orchestra.run.smithi040.stdout: "cap": 0, 2023-07-18T03:42:11.423 INFO:teuthology.orchestra.run.smithi040.stdout: "cap+": 0, 2023-07-18T03:42:11.423 INFO:teuthology.orchestra.run.smithi040.stdout: "cap-": 0, 2023-07-18T03:42:11.423 INFO:teuthology.orchestra.run.smithi040.stdout: "rss": 75544, 2023-07-18T03:42:11.423 INFO:teuthology.orchestra.run.smithi040.stdout: "heap": 207156 2023-07-18T03:42:11.423 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.423 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_server": { 2023-07-18T03:42:11.423 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_client_request": 0, 2023-07-18T03:42:11.424 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_server_request": 0, 2023-07-18T03:42:11.424 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_request": 0, 2023-07-18T03:42:11.424 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_session": 0, 2023-07-18T03:42:11.424 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_peer_request": 0, 2023-07-18T03:42:11.424 INFO:teuthology.orchestra.run.smithi040.stdout: "req_create_latency": { 2023-07-18T03:42:11.424 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.424 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.425 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.425 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.425 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getattr_latency": { 2023-07-18T03:42:11.425 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.425 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.425 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.425 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.425 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getfilelock_latency": { 2023-07-18T03:42:11.426 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.426 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.426 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.426 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.426 INFO:teuthology.orchestra.run.smithi040.stdout: "req_link_latency": { 2023-07-18T03:42:11.426 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.426 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.426 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.427 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookup_latency": { 2023-07-18T03:42:11.427 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.427 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.427 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.427 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookuphash_latency": { 2023-07-18T03:42:11.428 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.428 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.428 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.428 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.428 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupino_latency": { 2023-07-18T03:42:11.428 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.428 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.429 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.429 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.429 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupname_latency": { 2023-07-18T03:42:11.429 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.429 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.429 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.429 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.430 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupparent_latency": { 2023-07-18T03:42:11.430 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.430 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.430 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.430 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.430 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupsnap_latency": { 2023-07-18T03:42:11.431 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.431 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.431 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.431 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.431 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lssnap_latency": { 2023-07-18T03:42:11.431 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.432 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.432 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.432 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.432 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mkdir_latency": { 2023-07-18T03:42:11.432 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.432 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.432 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.433 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.433 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mknod_latency": { 2023-07-18T03:42:11.433 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.433 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.433 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.433 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.433 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mksnap_latency": { 2023-07-18T03:42:11.434 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.434 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.434 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.434 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.434 INFO:teuthology.orchestra.run.smithi040.stdout: "req_open_latency": { 2023-07-18T03:42:11.434 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.435 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.435 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.435 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.435 INFO:teuthology.orchestra.run.smithi040.stdout: "req_readdir_latency": { 2023-07-18T03:42:11.435 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.435 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.436 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.436 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.436 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rename_latency": { 2023-07-18T03:42:11.436 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.436 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.436 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.437 INFO:teuthology.orchestra.run.smithi040.stdout: "req_renamesnap_latency": { 2023-07-18T03:42:11.437 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.437 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.437 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.437 INFO:teuthology.orchestra.run.smithi040.stdout: "req_snapdiff_latency": { 2023-07-18T03:42:11.437 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.438 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.438 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.438 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.438 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmdir_latency": { 2023-07-18T03:42:11.438 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.438 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.438 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.438 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.439 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmsnap_latency": { 2023-07-18T03:42:11.439 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.439 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.439 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.439 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.439 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmxattr_latency": { 2023-07-18T03:42:11.440 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.440 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.440 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.440 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.440 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setattr_latency": { 2023-07-18T03:42:11.441 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.441 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.441 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.441 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.441 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setdirlayout_latency": { 2023-07-18T03:42:11.441 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.442 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.442 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.442 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setfilelock_latency": { 2023-07-18T03:42:11.442 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.442 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.442 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.443 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setlayout_latency": { 2023-07-18T03:42:11.443 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.443 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.443 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.443 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setxattr_latency": { 2023-07-18T03:42:11.443 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.444 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.444 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.444 INFO:teuthology.orchestra.run.smithi040.stdout: "req_symlink_latency": { 2023-07-18T03:42:11.444 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.444 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.444 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.445 INFO:teuthology.orchestra.run.smithi040.stdout: "req_unlink_latency": { 2023-07-18T03:42:11.445 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.445 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.445 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.445 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_revoke_eviction": 0, 2023-07-18T03:42:11.445 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_acquisition_throttle": 0, 2023-07-18T03:42:11.446 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getvxattr_latency": { 2023-07-18T03:42:11.446 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.446 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.446 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.446 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:11.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.446 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_sessions": { 2023-07-18T03:42:11.446 INFO:teuthology.orchestra.run.smithi040.stdout: "session_count": 1, 2023-07-18T03:42:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: "session_add": 1, 2023-07-18T03:42:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: "session_remove": 0, 2023-07-18T03:42:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_open": 1, 2023-07-18T03:42:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_stale": 0, 2023-07-18T03:42:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: "total_load": 0, 2023-07-18T03:42:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: "average_load": 0, 2023-07-18T03:42:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_session_uptime": 2104 2023-07-18T03:42:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": { 2023-07-18T03:42:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:42:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2023-07-18T03:42:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:42:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:42:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:42:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:42:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:42:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:42:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:42:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:42:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:42:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:42:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:42:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:42:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:42:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2023-07-18T03:42:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:42:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2023-07-18T03:42:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:42:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:42:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:42:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:42:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:42:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:42:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:42:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2023-07-18T03:42:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:42:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:42:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:42:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2023-07-18T03:42:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2023-07-18T03:42:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2023-07-18T03:42:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:42:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:42:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:42:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:42:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 231385, 2023-07-18T03:42:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 75, 2023-07-18T03:42:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 0, 2023-07-18T03:42:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 0, 2023-07-18T03:42:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2023-07-18T03:42:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2023-07-18T03:42:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:42:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2023-07-18T03:42:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:42:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2023-07-18T03:42:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 7720, 2023-07-18T03:42:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 162, 2023-07-18T03:42:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:42:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:42:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2023-07-18T03:42:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2023-07-18T03:42:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 5950554, 2023-07-18T03:42:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 34117, 2023-07-18T03:42:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2023-07-18T03:42:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2023-07-18T03:42:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2023-07-18T03:42:11.456 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2023-07-18T03:42:11.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.456 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter": { 2023-07-18T03:42:11.456 INFO:teuthology.orchestra.run.smithi040.stdout: "op_active": 0, 2023-07-18T03:42:11.456 INFO:teuthology.orchestra.run.smithi040.stdout: "op_laggy": 0, 2023-07-18T03:42:11.456 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send": 2488, 2023-07-18T03:42:11.456 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send_bytes": 21, 2023-07-18T03:42:11.456 INFO:teuthology.orchestra.run.smithi040.stdout: "op_resend": 3, 2023-07-18T03:42:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: "op_reply": 2488, 2023-07-18T03:42:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: "op_latency": { 2023-07-18T03:42:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2488, 2023-07-18T03:42:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 12.416833490, 2023-07-18T03:42:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.004990688 2023-07-18T03:42:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: "op_inflight": 0, 2023-07-18T03:42:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: "oplen_avg": { 2023-07-18T03:42:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2488, 2023-07-18T03:42:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 2489 2023-07-18T03:42:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: "op": 2488, 2023-07-18T03:42:11.459 INFO:teuthology.orchestra.run.smithi040.stdout: "op_r": 2488, 2023-07-18T03:42:11.459 INFO:teuthology.orchestra.run.smithi040.stdout: "op_w": 0, 2023-07-18T03:42:11.459 INFO:teuthology.orchestra.run.smithi040.stdout: "op_rmw": 0, 2023-07-18T03:42:11.459 INFO:teuthology.orchestra.run.smithi040.stdout: "op_pg": 0, 2023-07-18T03:42:11.459 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_stat": 1441, 2023-07-18T03:42:11.459 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_create": 0, 2023-07-18T03:42:11.460 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_read": 1045, 2023-07-18T03:42:11.460 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_write": 0, 2023-07-18T03:42:11.460 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writefull": 0, 2023-07-18T03:42:11.460 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writesame": 0, 2023-07-18T03:42:11.460 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_append": 0, 2023-07-18T03:42:11.460 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_zero": 0, 2023-07-18T03:42:11.461 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_truncate": 0, 2023-07-18T03:42:11.461 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_delete": 0, 2023-07-18T03:42:11.461 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_mapext": 0, 2023-07-18T03:42:11.461 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_sparse_read": 0, 2023-07-18T03:42:11.461 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_clonerange": 0, 2023-07-18T03:42:11.461 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_getxattr": 1, 2023-07-18T03:42:11.461 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_setxattr": 0, 2023-07-18T03:42:11.462 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_cmpxattr": 0, 2023-07-18T03:42:11.462 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_rmxattr": 0, 2023-07-18T03:42:11.462 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_resetxattrs": 0, 2023-07-18T03:42:11.462 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_call": 0, 2023-07-18T03:42:11.462 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_watch": 0, 2023-07-18T03:42:11.462 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_notify": 0, 2023-07-18T03:42:11.462 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_src_cmpxattr": 0, 2023-07-18T03:42:11.463 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls": 0, 2023-07-18T03:42:11.463 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls_filter": 0, 2023-07-18T03:42:11.463 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_other": 0, 2023-07-18T03:42:11.463 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_active": 0, 2023-07-18T03:42:11.463 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_send": 0, 2023-07-18T03:42:11.463 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_resend": 0, 2023-07-18T03:42:11.464 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_ping": 0, 2023-07-18T03:42:11.464 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_active": 0, 2023-07-18T03:42:11.464 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_send": 0, 2023-07-18T03:42:11.464 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_resend": 0, 2023-07-18T03:42:11.464 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_active": 0, 2023-07-18T03:42:11.464 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_send": 0, 2023-07-18T03:42:11.465 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_resend": 0, 2023-07-18T03:42:11.465 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_active": 0, 2023-07-18T03:42:11.465 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_send": 0, 2023-07-18T03:42:11.465 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_resend": 0, 2023-07-18T03:42:11.465 INFO:teuthology.orchestra.run.smithi040.stdout: "command_active": 0, 2023-07-18T03:42:11.465 INFO:teuthology.orchestra.run.smithi040.stdout: "command_send": 0, 2023-07-18T03:42:11.465 INFO:teuthology.orchestra.run.smithi040.stdout: "command_resend": 0, 2023-07-18T03:42:11.466 INFO:teuthology.orchestra.run.smithi040.stdout: "map_epoch": 84, 2023-07-18T03:42:11.466 INFO:teuthology.orchestra.run.smithi040.stdout: "map_full": 0, 2023-07-18T03:42:11.466 INFO:teuthology.orchestra.run.smithi040.stdout: "map_inc": 3, 2023-07-18T03:42:11.466 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_sessions": 10, 2023-07-18T03:42:11.466 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_open": 10, 2023-07-18T03:42:11.466 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_close": 0, 2023-07-18T03:42:11.467 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_laggy": 0, 2023-07-18T03:42:11.467 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_wr": 0, 2023-07-18T03:42:11.467 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_rd": 2, 2023-07-18T03:42:11.467 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_del": 0 2023-07-18T03:42:11.467 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.467 INFO:teuthology.orchestra.run.smithi040.stdout: "oft": { 2023-07-18T03:42:11.468 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_objs": 0, 2023-07-18T03:42:11.468 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_kv_pairs": 0, 2023-07-18T03:42:11.468 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_updates": 0, 2023-07-18T03:42:11.468 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_removes": 0 2023-07-18T03:42:11.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.468 INFO:teuthology.orchestra.run.smithi040.stdout: "osd-slow-ops": { 2023-07-18T03:42:11.468 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_ops_count": 0 2023-07-18T03:42:11.469 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.469 INFO:teuthology.orchestra.run.smithi040.stdout: "purge_queue": { 2023-07-18T03:42:11.469 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops": 0, 2023-07-18T03:42:11.469 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops_high_water": 0, 2023-07-18T03:42:11.469 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing": 0, 2023-07-18T03:42:11.469 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_high_water": 0, 2023-07-18T03:42:11.469 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed": 0, 2023-07-18T03:42:11.470 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_item_in_journal": 0 2023-07-18T03:42:11.470 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.470 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:42:11.470 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2023-07-18T03:42:11.470 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2023-07-18T03:42:11.470 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2023-07-18T03:42:11.470 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 2755, 2023-07-18T03:42:11.470 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 161719825, 2023-07-18T03:42:11.471 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:11.471 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 2755, 2023-07-18T03:42:11.471 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2023-07-18T03:42:11.471 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2023-07-18T03:42:11.471 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2755, 2023-07-18T03:42:11.471 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 161719825, 2023-07-18T03:42:11.471 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2023-07-18T03:42:11.471 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.472 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.472 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.472 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:11.472 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.472 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_bytes": { 2023-07-18T03:42:11.472 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2023-07-18T03:42:11.472 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2023-07-18T03:42:11.473 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2023-07-18T03:42:11.473 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2023-07-18T03:42:11.473 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2023-07-18T03:42:11.473 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:11.473 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:11.473 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2488, 2023-07-18T03:42:11.473 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 160515310, 2023-07-18T03:42:11.473 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2488, 2023-07-18T03:42:11.474 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 160515310, 2023-07-18T03:42:11.474 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2023-07-18T03:42:11.474 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.474 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.474 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.474 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:11.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.474 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_ops": { 2023-07-18T03:42:11.475 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2023-07-18T03:42:11.475 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 1024, 2023-07-18T03:42:11.475 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2023-07-18T03:42:11.475 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2023-07-18T03:42:11.475 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2023-07-18T03:42:11.475 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:11.475 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:11.476 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2488, 2023-07-18T03:42:11.476 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 2488, 2023-07-18T03:42:11.476 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2488, 2023-07-18T03:42:11.476 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 2488, 2023-07-18T03:42:11.476 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2023-07-18T03:42:11.476 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.476 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.476 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.477 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:11.477 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.477 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle": { 2023-07-18T03:42:11.477 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2023-07-18T03:42:11.477 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2023-07-18T03:42:11.477 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2023-07-18T03:42:11.477 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2023-07-18T03:42:11.477 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2023-07-18T03:42:11.478 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:11.478 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:11.478 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2023-07-18T03:42:11.478 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2023-07-18T03:42:11.478 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2023-07-18T03:42:11.478 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2023-07-18T03:42:11.478 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2023-07-18T03:42:11.478 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.479 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.479 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.479 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:11.479 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:11.479 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle-0x55d3186c6460": { 2023-07-18T03:42:11.479 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2023-07-18T03:42:11.479 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2023-07-18T03:42:11.480 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2023-07-18T03:42:11.480 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2023-07-18T03:42:11.480 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2023-07-18T03:42:11.480 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:11.480 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:11.480 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2023-07-18T03:42:11.480 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2023-07-18T03:42:11.481 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2023-07-18T03:42:11.481 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2023-07-18T03:42:11.481 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2023-07-18T03:42:11.481 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:11.481 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:11.481 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:11.481 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:11.481 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:11.482 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:42:11.597 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:11 smithi064 ceph-mon[98059]: pgmap v633: 97 pgs: 97 active+clean; 593 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 344 op/s 2023-07-18T03:42:11.693 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:11 smithi040 ceph-mon[96439]: pgmap v633: 97 pgs: 97 active+clean; 593 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 344 op/s 2023-07-18T03:42:11.731 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-07-18T03:42:11.732 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2023-07-18T03:42:11.732 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2023-07-18T03:42:11.732 DEBUG:tasks.check_counter:Getting stats from d 2023-07-18T03:42:11.733 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph daemon mds.d perf dump 2023-07-18T03:42:11.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:11 smithi134 ceph-mon[98216]: pgmap v633: 97 pgs: 97 active+clean; 593 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 344 op/s 2023-07-18T03:42:12.292 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:42:13.313 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:12 smithi134 ceph-mon[98216]: pgmap v634: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 899 KiB/s wr, 330 op/s 2023-07-18T03:42:13.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:12 smithi040 ceph-mon[96439]: pgmap v634: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 899 KiB/s wr, 330 op/s 2023-07-18T03:42:13.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:12 smithi064 ceph-mon[98059]: pgmap v634: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 899 KiB/s wr, 330 op/s 2023-07-18T03:42:14.977 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:42:14.977 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:42:14.977 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 1198, 2023-07-18T03:42:14.977 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 1148, 2023-07-18T03:42:14.978 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 26237559, 2023-07-18T03:42:14.978 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 410896, 2023-07-18T03:42:14.978 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 7, 2023-07-18T03:42:14.978 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 3, 2023-07-18T03:42:14.978 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.156582300, 2023-07-18T03:42:14.978 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.052874391, 2023-07-18T03:42:14.978 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.055324430, 2023-07-18T03:42:14.979 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.040265490, 2023-07-18T03:42:14.979 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:14.979 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1147, 2023-07-18T03:42:14.979 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.508310057, 2023-07-18T03:42:14.979 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000443164 2023-07-18T03:42:14.979 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:14.979 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:14.980 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:14.980 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:14.980 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:14.980 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:14.980 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 772592, 2023-07-18T03:42:14.980 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 122288 2023-07-18T03:42:14.980 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:14.981 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:42:14.981 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 899, 2023-07-18T03:42:14.981 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 1060, 2023-07-18T03:42:14.981 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 45576648, 2023-07-18T03:42:14.981 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 628119, 2023-07-18T03:42:14.981 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 6, 2023-07-18T03:42:14.981 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 5, 2023-07-18T03:42:14.982 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.123090631, 2023-07-18T03:42:14.982 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.028296514, 2023-07-18T03:42:14.982 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.057953370, 2023-07-18T03:42:14.982 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.029132189, 2023-07-18T03:42:14.982 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:14.982 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1058, 2023-07-18T03:42:14.982 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.246651630, 2023-07-18T03:42:14.983 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000233130 2023-07-18T03:42:14.983 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:14.983 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:14.983 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:14.983 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:14.983 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:14.983 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:14.984 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-18T03:42:14.984 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 348784 2023-07-18T03:42:14.984 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:14.984 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:42:14.984 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 616, 2023-07-18T03:42:14.984 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 616, 2023-07-18T03:42:14.984 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 50640338, 2023-07-18T03:42:14.985 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 190792, 2023-07-18T03:42:14.985 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 6, 2023-07-18T03:42:14.985 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 5, 2023-07-18T03:42:14.985 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.090295372, 2023-07-18T03:42:14.985 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.013838950, 2023-07-18T03:42:14.985 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.051946690, 2023-07-18T03:42:14.985 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.020019526, 2023-07-18T03:42:14.986 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:14.986 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 615, 2023-07-18T03:42:14.986 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.124164022, 2023-07-18T03:42:14.986 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000201892 2023-07-18T03:42:14.986 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:14.986 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:14.986 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:14.987 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:14.987 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:14.987 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:14.987 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-18T03:42:14.987 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 448 2023-07-18T03:42:14.987 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:14.987 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": { 2023-07-18T03:42:14.987 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 1, 2023-07-18T03:42:14.988 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2023-07-18T03:42:14.988 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:14.988 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-MDSRank": { 2023-07-18T03:42:14.988 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2023-07-18T03:42:14.988 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2023-07-18T03:42:14.988 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 4163, 2023-07-18T03:42:14.988 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.140720391, 2023-07-18T03:42:14.989 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000033802 2023-07-18T03:42:14.989 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:14.989 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:14.989 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-PurgeQueue": { 2023-07-18T03:42:14.989 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2023-07-18T03:42:14.989 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2023-07-18T03:42:14.989 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:14.990 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:14.990 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:14.990 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:14.990 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:14.990 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2023-07-18T03:42:14.990 INFO:teuthology.orchestra.run.smithi040.stdout: "request": 0, 2023-07-18T03:42:14.990 INFO:teuthology.orchestra.run.smithi040.stdout: "reply": 0, 2023-07-18T03:42:14.991 INFO:teuthology.orchestra.run.smithi040.stdout: "reply_latency": { 2023-07-18T03:42:14.991 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:14.991 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:14.991 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:14.991 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:14.991 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_reply": 0, 2023-07-18T03:42:14.991 INFO:teuthology.orchestra.run.smithi040.stdout: "forward": 0, 2023-07-18T03:42:14.992 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_complete": 0, 2023-07-18T03:42:14.992 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_keys": 0, 2023-07-18T03:42:14.992 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_commit": 0, 2023-07-18T03:42:14.992 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_split": 0, 2023-07-18T03:42:14.992 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_merge": 0, 2023-07-18T03:42:14.992 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes": 10078, 2023-07-18T03:42:14.992 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_top": 0, 2023-07-18T03:42:14.993 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_bottom": 0, 2023-07-18T03:42:14.993 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pin_tail": 10078, 2023-07-18T03:42:14.993 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pinned": 10078, 2023-07-18T03:42:14.993 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_expired": 726, 2023-07-18T03:42:14.993 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_with_caps": 0, 2023-07-18T03:42:14.993 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": 0, 2023-07-18T03:42:14.993 INFO:teuthology.orchestra.run.smithi040.stdout: "subtrees": 1, 2023-07-18T03:42:14.994 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse": 0, 2023-07-18T03:42:14.994 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_hit": 0, 2023-07-18T03:42:14.994 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_forward": 0, 2023-07-18T03:42:14.994 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_discover": 0, 2023-07-18T03:42:14.994 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_dir_fetch": 0, 2023-07-18T03:42:14.994 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_remote_ino": 0, 2023-07-18T03:42:14.994 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_lock": 0, 2023-07-18T03:42:14.995 INFO:teuthology.orchestra.run.smithi040.stdout: "load_cent": 0, 2023-07-18T03:42:14.995 INFO:teuthology.orchestra.run.smithi040.stdout: "q": 0, 2023-07-18T03:42:14.995 INFO:teuthology.orchestra.run.smithi040.stdout: "exported": 0, 2023-07-18T03:42:14.995 INFO:teuthology.orchestra.run.smithi040.stdout: "exported_inodes": 0, 2023-07-18T03:42:14.995 INFO:teuthology.orchestra.run.smithi040.stdout: "imported": 0, 2023-07-18T03:42:14.995 INFO:teuthology.orchestra.run.smithi040.stdout: "imported_inodes": 0, 2023-07-18T03:42:14.995 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_dir_fetch": 0, 2023-07-18T03:42:14.996 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_backtrace_fetch": 0, 2023-07-18T03:42:14.996 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_peer_discover": 0, 2023-07-18T03:42:14.996 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rfiles": 0, 2023-07-18T03:42:14.996 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rbytes": 0, 2023-07-18T03:42:14.996 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rsnaps": 0, 2023-07-18T03:42:14.996 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_fetch": 0, 2023-07-18T03:42:14.996 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_set_tag": 0, 2023-07-18T03:42:14.997 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_repaired": 0, 2023-07-18T03:42:14.997 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_inotable_repaired": 0, 2023-07-18T03:42:14.997 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_inodes": 0, 2023-07-18T03:42:14.997 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_base_inodes": 0, 2023-07-18T03:42:14.997 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dirfrag_rstats": 0, 2023-07-18T03:42:14.997 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_file_inodes": 0, 2023-07-18T03:42:14.997 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_inode_file_caps": 0, 2023-07-18T03:42:14.998 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_revoke": 0, 2023-07-18T03:42:14.998 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_grant": 0, 2023-07-18T03:42:14.998 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_trunc": 0, 2023-07-18T03:42:14.998 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-18T03:42:14.998 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-18T03:42:14.998 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps": 0, 2023-07-18T03:42:14.998 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps_dirty": 0, 2023-07-18T03:42:14.999 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_cap_release": 0, 2023-07-18T03:42:14.999 INFO:teuthology.orchestra.run.smithi040.stdout: "process_request_cap_release": 0 2023-07-18T03:42:14.999 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:14.999 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache": { 2023-07-18T03:42:14.999 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update": 0, 2023-07-18T03:42:14.999 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update_receipt": 0, 2023-07-18T03:42:14.999 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_try_discover": 0, 2023-07-18T03:42:14.999 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_send_discover": 0, 2023-07-18T03:42:15.000 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_handle_discover": 0, 2023-07-18T03:42:15.000 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays": 0, 2023-07-18T03:42:15.000 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_delayed": 0, 2023-07-18T03:42:15.000 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_enqueuing": 0, 2023-07-18T03:42:15.000 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_created": 0, 2023-07-18T03:42:15.000 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_enqueued": 0, 2023-07-18T03:42:15.000 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_reintegrated": 0, 2023-07-18T03:42:15.001 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_migrated": 0, 2023-07-18T03:42:15.001 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_processing": 0, 2023-07-18T03:42:15.001 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_enqueued": 0, 2023-07-18T03:42:15.001 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_prioritized": 0, 2023-07-18T03:42:15.001 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_started": 0, 2023-07-18T03:42:15.001 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_completed": 0, 2023-07-18T03:42:15.001 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_enqueue_scrub": 0, 2023-07-18T03:42:15.002 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_exportdir": 0, 2023-07-18T03:42:15.002 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_flush": 0, 2023-07-18T03:42:15.002 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragmentdir": 0, 2023-07-18T03:42:15.002 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragstats": 0, 2023-07-18T03:42:15.002 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_inodestats": 0 2023-07-18T03:42:15.002 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.002 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log": { 2023-07-18T03:42:15.003 INFO:teuthology.orchestra.run.smithi040.stdout: "evadd": 0, 2023-07-18T03:42:15.003 INFO:teuthology.orchestra.run.smithi040.stdout: "evex": 0, 2023-07-18T03:42:15.003 INFO:teuthology.orchestra.run.smithi040.stdout: "evtrm": 0, 2023-07-18T03:42:15.003 INFO:teuthology.orchestra.run.smithi040.stdout: "ev": 0, 2023-07-18T03:42:15.003 INFO:teuthology.orchestra.run.smithi040.stdout: "evexg": 0, 2023-07-18T03:42:15.003 INFO:teuthology.orchestra.run.smithi040.stdout: "evexd": 0, 2023-07-18T03:42:15.003 INFO:teuthology.orchestra.run.smithi040.stdout: "segadd": 0, 2023-07-18T03:42:15.004 INFO:teuthology.orchestra.run.smithi040.stdout: "segex": 0, 2023-07-18T03:42:15.004 INFO:teuthology.orchestra.run.smithi040.stdout: "segtrm": 0, 2023-07-18T03:42:15.004 INFO:teuthology.orchestra.run.smithi040.stdout: "seg": 72, 2023-07-18T03:42:15.004 INFO:teuthology.orchestra.run.smithi040.stdout: "segexg": 0, 2023-07-18T03:42:15.004 INFO:teuthology.orchestra.run.smithi040.stdout: "segexd": 0, 2023-07-18T03:42:15.004 INFO:teuthology.orchestra.run.smithi040.stdout: "expos": 4194304, 2023-07-18T03:42:15.004 INFO:teuthology.orchestra.run.smithi040.stdout: "wrpos": 0, 2023-07-18T03:42:15.005 INFO:teuthology.orchestra.run.smithi040.stdout: "rdpos": 125036140, 2023-07-18T03:42:15.005 INFO:teuthology.orchestra.run.smithi040.stdout: "jlat": { 2023-07-18T03:42:15.005 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.005 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.005 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.005 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.005 INFO:teuthology.orchestra.run.smithi040.stdout: "replayed": 57660 2023-07-18T03:42:15.006 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.006 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_mem": { 2023-07-18T03:42:15.006 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 13, 2023-07-18T03:42:15.006 INFO:teuthology.orchestra.run.smithi040.stdout: "ino+": 13152, 2023-07-18T03:42:15.006 INFO:teuthology.orchestra.run.smithi040.stdout: "ino-": 13139, 2023-07-18T03:42:15.006 INFO:teuthology.orchestra.run.smithi040.stdout: "dir": 11, 2023-07-18T03:42:15.006 INFO:teuthology.orchestra.run.smithi040.stdout: "dir+": 649, 2023-07-18T03:42:15.007 INFO:teuthology.orchestra.run.smithi040.stdout: "dir-": 638, 2023-07-18T03:42:15.007 INFO:teuthology.orchestra.run.smithi040.stdout: "dn": 10078, 2023-07-18T03:42:15.007 INFO:teuthology.orchestra.run.smithi040.stdout: "dn+": 31530, 2023-07-18T03:42:15.007 INFO:teuthology.orchestra.run.smithi040.stdout: "dn-": 21452, 2023-07-18T03:42:15.007 INFO:teuthology.orchestra.run.smithi040.stdout: "cap": 0, 2023-07-18T03:42:15.007 INFO:teuthology.orchestra.run.smithi040.stdout: "cap+": 0, 2023-07-18T03:42:15.007 INFO:teuthology.orchestra.run.smithi040.stdout: "cap-": 0, 2023-07-18T03:42:15.008 INFO:teuthology.orchestra.run.smithi040.stdout: "rss": 74880, 2023-07-18T03:42:15.008 INFO:teuthology.orchestra.run.smithi040.stdout: "heap": 207156 2023-07-18T03:42:15.008 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.008 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_server": { 2023-07-18T03:42:15.008 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_client_request": 0, 2023-07-18T03:42:15.008 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_server_request": 0, 2023-07-18T03:42:15.008 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_request": 0, 2023-07-18T03:42:15.009 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_session": 0, 2023-07-18T03:42:15.009 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_peer_request": 0, 2023-07-18T03:42:15.009 INFO:teuthology.orchestra.run.smithi040.stdout: "req_create_latency": { 2023-07-18T03:42:15.010 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.010 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.010 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.011 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getattr_latency": { 2023-07-18T03:42:15.011 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.011 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.011 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.011 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getfilelock_latency": { 2023-07-18T03:42:15.012 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.012 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.012 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.012 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.012 INFO:teuthology.orchestra.run.smithi040.stdout: "req_link_latency": { 2023-07-18T03:42:15.012 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.012 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.013 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.013 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.013 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookup_latency": { 2023-07-18T03:42:15.013 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.013 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.013 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.013 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.013 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookuphash_latency": { 2023-07-18T03:42:15.014 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.014 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.014 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.014 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupino_latency": { 2023-07-18T03:42:15.014 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.014 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.015 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.015 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupname_latency": { 2023-07-18T03:42:15.015 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.015 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.015 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.016 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupparent_latency": { 2023-07-18T03:42:15.016 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.016 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.016 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.016 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.016 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupsnap_latency": { 2023-07-18T03:42:15.016 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.017 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.017 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.017 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.017 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lssnap_latency": { 2023-07-18T03:42:15.017 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.017 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.017 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.018 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mkdir_latency": { 2023-07-18T03:42:15.018 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.018 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.018 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.018 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mknod_latency": { 2023-07-18T03:42:15.019 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.019 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.019 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.019 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mksnap_latency": { 2023-07-18T03:42:15.019 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.019 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.020 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.020 INFO:teuthology.orchestra.run.smithi040.stdout: "req_open_latency": { 2023-07-18T03:42:15.020 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.020 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.020 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.021 INFO:teuthology.orchestra.run.smithi040.stdout: "req_readdir_latency": { 2023-07-18T03:42:15.021 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.021 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.021 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.021 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.021 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rename_latency": { 2023-07-18T03:42:15.021 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.021 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.022 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.022 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.022 INFO:teuthology.orchestra.run.smithi040.stdout: "req_renamesnap_latency": { 2023-07-18T03:42:15.022 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.022 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.022 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.022 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.023 INFO:teuthology.orchestra.run.smithi040.stdout: "req_snapdiff_latency": { 2023-07-18T03:42:15.023 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.023 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.023 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.023 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.023 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmdir_latency": { 2023-07-18T03:42:15.023 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.024 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.024 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.024 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmsnap_latency": { 2023-07-18T03:42:15.024 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.024 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.024 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.025 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.025 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmxattr_latency": { 2023-07-18T03:42:15.025 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.025 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.025 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.025 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.025 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setattr_latency": { 2023-07-18T03:42:15.026 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.026 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.026 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.026 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.026 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setdirlayout_latency": { 2023-07-18T03:42:15.026 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.026 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.027 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.027 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setfilelock_latency": { 2023-07-18T03:42:15.027 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.027 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.027 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.028 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setlayout_latency": { 2023-07-18T03:42:15.028 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.028 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.028 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.028 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setxattr_latency": { 2023-07-18T03:42:15.028 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.029 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.029 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.029 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.029 INFO:teuthology.orchestra.run.smithi040.stdout: "req_symlink_latency": { 2023-07-18T03:42:15.029 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.029 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.029 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.030 INFO:teuthology.orchestra.run.smithi040.stdout: "req_unlink_latency": { 2023-07-18T03:42:15.030 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.030 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.030 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.030 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_revoke_eviction": 0, 2023-07-18T03:42:15.031 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_acquisition_throttle": 0, 2023-07-18T03:42:15.031 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getvxattr_latency": { 2023-07-18T03:42:15.031 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.031 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.031 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.031 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:15.031 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.032 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_sessions": { 2023-07-18T03:42:15.032 INFO:teuthology.orchestra.run.smithi040.stdout: "session_count": 1, 2023-07-18T03:42:15.032 INFO:teuthology.orchestra.run.smithi040.stdout: "session_add": 1, 2023-07-18T03:42:15.032 INFO:teuthology.orchestra.run.smithi040.stdout: "session_remove": 0, 2023-07-18T03:42:15.032 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_open": 1, 2023-07-18T03:42:15.032 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_stale": 0, 2023-07-18T03:42:15.032 INFO:teuthology.orchestra.run.smithi040.stdout: "total_load": 0, 2023-07-18T03:42:15.033 INFO:teuthology.orchestra.run.smithi040.stdout: "average_load": 0, 2023-07-18T03:42:15.033 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_session_uptime": 2109 2023-07-18T03:42:15.033 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.033 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": { 2023-07-18T03:42:15.033 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:42:15.033 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2023-07-18T03:42:15.033 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:42:15.034 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:42:15.034 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:42:15.034 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:42:15.034 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:42:15.034 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:42:15.034 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:42:15.034 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:42:15.035 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:42:15.035 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:42:15.035 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:42:15.035 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:42:15.035 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:42:15.035 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2023-07-18T03:42:15.035 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:42:15.036 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2023-07-18T03:42:15.036 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:42:15.036 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:42:15.036 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:42:15.036 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:42:15.036 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:42:15.036 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:42:15.037 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:42:15.037 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2023-07-18T03:42:15.037 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:42:15.037 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:42:15.037 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:42:15.037 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2023-07-18T03:42:15.037 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2023-07-18T03:42:15.038 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2023-07-18T03:42:15.038 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:42:15.038 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:42:15.038 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:42:15.038 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:42:15.038 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 232382, 2023-07-18T03:42:15.038 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 75, 2023-07-18T03:42:15.038 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 0, 2023-07-18T03:42:15.039 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 0, 2023-07-18T03:42:15.039 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2023-07-18T03:42:15.039 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2023-07-18T03:42:15.039 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:42:15.039 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2023-07-18T03:42:15.039 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:42:15.039 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2023-07-18T03:42:15.040 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 7720, 2023-07-18T03:42:15.040 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 162, 2023-07-18T03:42:15.040 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:42:15.040 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:42:15.040 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2023-07-18T03:42:15.040 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2023-07-18T03:42:15.040 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 7137672, 2023-07-18T03:42:15.041 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 40440, 2023-07-18T03:42:15.041 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2023-07-18T03:42:15.041 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2023-07-18T03:42:15.041 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2023-07-18T03:42:15.041 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2023-07-18T03:42:15.041 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.041 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter": { 2023-07-18T03:42:15.042 INFO:teuthology.orchestra.run.smithi040.stdout: "op_active": 0, 2023-07-18T03:42:15.042 INFO:teuthology.orchestra.run.smithi040.stdout: "op_laggy": 0, 2023-07-18T03:42:15.042 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send": 2438, 2023-07-18T03:42:15.042 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send_bytes": 21, 2023-07-18T03:42:15.042 INFO:teuthology.orchestra.run.smithi040.stdout: "op_resend": 3, 2023-07-18T03:42:15.042 INFO:teuthology.orchestra.run.smithi040.stdout: "op_reply": 2438, 2023-07-18T03:42:15.042 INFO:teuthology.orchestra.run.smithi040.stdout: "op_latency": { 2023-07-18T03:42:15.043 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2438, 2023-07-18T03:42:15.043 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 11.629843744, 2023-07-18T03:42:15.043 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.004770239 2023-07-18T03:42:15.043 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.043 INFO:teuthology.orchestra.run.smithi040.stdout: "op_inflight": 0, 2023-07-18T03:42:15.043 INFO:teuthology.orchestra.run.smithi040.stdout: "oplen_avg": { 2023-07-18T03:42:15.043 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2438, 2023-07-18T03:42:15.043 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 2439 2023-07-18T03:42:15.044 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.044 INFO:teuthology.orchestra.run.smithi040.stdout: "op": 2438, 2023-07-18T03:42:15.044 INFO:teuthology.orchestra.run.smithi040.stdout: "op_r": 2438, 2023-07-18T03:42:15.044 INFO:teuthology.orchestra.run.smithi040.stdout: "op_w": 0, 2023-07-18T03:42:15.044 INFO:teuthology.orchestra.run.smithi040.stdout: "op_rmw": 0, 2023-07-18T03:42:15.044 INFO:teuthology.orchestra.run.smithi040.stdout: "op_pg": 0, 2023-07-18T03:42:15.045 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_stat": 1354, 2023-07-18T03:42:15.045 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_create": 0, 2023-07-18T03:42:15.045 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_read": 1082, 2023-07-18T03:42:15.045 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_write": 0, 2023-07-18T03:42:15.045 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writefull": 0, 2023-07-18T03:42:15.045 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writesame": 0, 2023-07-18T03:42:15.045 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_append": 0, 2023-07-18T03:42:15.046 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_zero": 0, 2023-07-18T03:42:15.046 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_truncate": 0, 2023-07-18T03:42:15.046 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_delete": 0, 2023-07-18T03:42:15.046 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_mapext": 0, 2023-07-18T03:42:15.046 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_sparse_read": 0, 2023-07-18T03:42:15.046 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_clonerange": 0, 2023-07-18T03:42:15.046 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_getxattr": 1, 2023-07-18T03:42:15.046 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_setxattr": 0, 2023-07-18T03:42:15.047 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_cmpxattr": 0, 2023-07-18T03:42:15.047 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_rmxattr": 0, 2023-07-18T03:42:15.047 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_resetxattrs": 0, 2023-07-18T03:42:15.047 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_call": 0, 2023-07-18T03:42:15.047 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_watch": 0, 2023-07-18T03:42:15.047 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_notify": 0, 2023-07-18T03:42:15.047 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_src_cmpxattr": 0, 2023-07-18T03:42:15.048 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls": 0, 2023-07-18T03:42:15.048 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls_filter": 0, 2023-07-18T03:42:15.048 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_other": 0, 2023-07-18T03:42:15.048 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_active": 0, 2023-07-18T03:42:15.048 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_send": 0, 2023-07-18T03:42:15.048 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_resend": 0, 2023-07-18T03:42:15.048 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_ping": 0, 2023-07-18T03:42:15.049 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_active": 0, 2023-07-18T03:42:15.049 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_send": 0, 2023-07-18T03:42:15.049 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_resend": 0, 2023-07-18T03:42:15.049 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_active": 0, 2023-07-18T03:42:15.049 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_send": 0, 2023-07-18T03:42:15.049 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_resend": 0, 2023-07-18T03:42:15.049 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_active": 0, 2023-07-18T03:42:15.050 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_send": 0, 2023-07-18T03:42:15.050 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_resend": 0, 2023-07-18T03:42:15.050 INFO:teuthology.orchestra.run.smithi040.stdout: "command_active": 0, 2023-07-18T03:42:15.050 INFO:teuthology.orchestra.run.smithi040.stdout: "command_send": 0, 2023-07-18T03:42:15.050 INFO:teuthology.orchestra.run.smithi040.stdout: "command_resend": 0, 2023-07-18T03:42:15.050 INFO:teuthology.orchestra.run.smithi040.stdout: "map_epoch": 84, 2023-07-18T03:42:15.050 INFO:teuthology.orchestra.run.smithi040.stdout: "map_full": 0, 2023-07-18T03:42:15.051 INFO:teuthology.orchestra.run.smithi040.stdout: "map_inc": 3, 2023-07-18T03:42:15.051 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_sessions": 10, 2023-07-18T03:42:15.051 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_open": 10, 2023-07-18T03:42:15.051 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_close": 0, 2023-07-18T03:42:15.051 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_laggy": 0, 2023-07-18T03:42:15.051 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_wr": 0, 2023-07-18T03:42:15.051 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_rd": 2, 2023-07-18T03:42:15.052 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_del": 0 2023-07-18T03:42:15.052 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.052 INFO:teuthology.orchestra.run.smithi040.stdout: "oft": { 2023-07-18T03:42:15.052 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_objs": 0, 2023-07-18T03:42:15.052 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_kv_pairs": 0, 2023-07-18T03:42:15.052 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_updates": 0, 2023-07-18T03:42:15.052 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_removes": 0 2023-07-18T03:42:15.053 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.053 INFO:teuthology.orchestra.run.smithi040.stdout: "osd-slow-ops": { 2023-07-18T03:42:15.053 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_ops_count": 0 2023-07-18T03:42:15.053 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.053 INFO:teuthology.orchestra.run.smithi040.stdout: "purge_queue": { 2023-07-18T03:42:15.053 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops": 0, 2023-07-18T03:42:15.053 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops_high_water": 0, 2023-07-18T03:42:15.053 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing": 0, 2023-07-18T03:42:15.054 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_high_water": 0, 2023-07-18T03:42:15.054 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed": 0, 2023-07-18T03:42:15.054 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_item_in_journal": 0 2023-07-18T03:42:15.054 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.054 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:42:15.054 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2023-07-18T03:42:15.054 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2023-07-18T03:42:15.055 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2023-07-18T03:42:15.055 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 2709, 2023-07-18T03:42:15.055 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 122194096, 2023-07-18T03:42:15.055 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:15.055 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 2709, 2023-07-18T03:42:15.055 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2023-07-18T03:42:15.055 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2023-07-18T03:42:15.056 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2709, 2023-07-18T03:42:15.056 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 122194096, 2023-07-18T03:42:15.056 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2023-07-18T03:42:15.056 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.056 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.056 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.056 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:15.057 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.057 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_bytes": { 2023-07-18T03:42:15.057 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2023-07-18T03:42:15.057 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2023-07-18T03:42:15.057 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2023-07-18T03:42:15.057 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2023-07-18T03:42:15.057 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2023-07-18T03:42:15.058 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:15.058 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:15.058 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2438, 2023-07-18T03:42:15.058 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 120996819, 2023-07-18T03:42:15.058 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2438, 2023-07-18T03:42:15.058 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 120996819, 2023-07-18T03:42:15.058 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2023-07-18T03:42:15.059 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.059 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.059 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.059 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:15.059 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.059 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_ops": { 2023-07-18T03:42:15.059 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2023-07-18T03:42:15.060 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 1024, 2023-07-18T03:42:15.060 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2023-07-18T03:42:15.060 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2023-07-18T03:42:15.060 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2023-07-18T03:42:15.060 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:15.060 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:15.060 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2438, 2023-07-18T03:42:15.061 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 2438, 2023-07-18T03:42:15.061 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2438, 2023-07-18T03:42:15.061 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 2438, 2023-07-18T03:42:15.061 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2023-07-18T03:42:15.061 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.061 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.061 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.062 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:15.062 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.062 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle": { 2023-07-18T03:42:15.062 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2023-07-18T03:42:15.062 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2023-07-18T03:42:15.062 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2023-07-18T03:42:15.062 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2023-07-18T03:42:15.062 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2023-07-18T03:42:15.063 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:15.063 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:15.063 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2023-07-18T03:42:15.063 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2023-07-18T03:42:15.063 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2023-07-18T03:42:15.063 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2023-07-18T03:42:15.063 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2023-07-18T03:42:15.064 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.064 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.064 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.064 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:15.064 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:15.064 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle-0x55a9c17e4460": { 2023-07-18T03:42:15.065 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2023-07-18T03:42:15.065 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2023-07-18T03:42:15.065 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2023-07-18T03:42:15.065 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2023-07-18T03:42:15.065 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2023-07-18T03:42:15.065 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:15.065 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:15.066 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2023-07-18T03:42:15.066 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2023-07-18T03:42:15.066 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2023-07-18T03:42:15.066 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2023-07-18T03:42:15.066 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2023-07-18T03:42:15.066 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:15.066 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:15.066 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:15.067 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:15.067 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:15.067 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:42:15.453 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2023-07-18T03:42:15.454 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2023-07-18T03:42:15.454 INFO:tasks.check_counter:Daemon mds.d mds.dir_split=0 2023-07-18T03:42:15.454 DEBUG:tasks.check_counter:Getting stats from g 2023-07-18T03:42:15.455 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph daemon mds.g perf dump 2023-07-18T03:42:15.576 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:15 smithi040 ceph-mon[96439]: pgmap v635: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 874 KiB/s wr, 216 op/s 2023-07-18T03:42:15.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:15 smithi064 ceph-mon[98059]: pgmap v635: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 874 KiB/s wr, 216 op/s 2023-07-18T03:42:15.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:15 smithi134 ceph-mon[98216]: pgmap v635: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 874 KiB/s wr, 216 op/s 2023-07-18T03:42:16.017 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:42:17.209 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:42:17.210 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:42:17.210 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 1300, 2023-07-18T03:42:17.210 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 1248, 2023-07-18T03:42:17.210 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 18055787, 2023-07-18T03:42:17.210 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 441260, 2023-07-18T03:42:17.210 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 7, 2023-07-18T03:42:17.210 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 3, 2023-07-18T03:42:17.211 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.157126969, 2023-07-18T03:42:17.211 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.045400652, 2023-07-18T03:42:17.211 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.054152430, 2023-07-18T03:42:17.211 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.047966723, 2023-07-18T03:42:17.211 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:17.211 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1248, 2023-07-18T03:42:17.211 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.277753673, 2023-07-18T03:42:17.212 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000222559 2023-07-18T03:42:17.212 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.212 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:17.212 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.212 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.212 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.212 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.212 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 774992, 2023-07-18T03:42:17.213 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 121552 2023-07-18T03:42:17.213 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.213 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:42:17.213 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 865, 2023-07-18T03:42:17.213 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 1023, 2023-07-18T03:42:17.213 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 68861329, 2023-07-18T03:42:17.213 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 617028, 2023-07-18T03:42:17.213 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 6, 2023-07-18T03:42:17.214 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 5, 2023-07-18T03:42:17.214 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.147477280, 2023-07-18T03:42:17.214 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.031188246, 2023-07-18T03:42:17.214 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.074302992, 2023-07-18T03:42:17.214 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.032901068, 2023-07-18T03:42:17.214 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:17.214 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 1021, 2023-07-18T03:42:17.215 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.350369559, 2023-07-18T03:42:17.215 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000343163 2023-07-18T03:42:17.215 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.215 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:17.215 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.215 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.215 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.215 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.216 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-18T03:42:17.216 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 348256 2023-07-18T03:42:17.216 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.216 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:42:17.216 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 546, 2023-07-18T03:42:17.216 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 548, 2023-07-18T03:42:17.216 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 29501185, 2023-07-18T03:42:17.217 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 169807, 2023-07-18T03:42:17.217 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 6, 2023-07-18T03:42:17.217 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 5, 2023-07-18T03:42:17.217 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.082090592, 2023-07-18T03:42:17.217 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.014960168, 2023-07-18T03:42:17.217 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.041560028, 2023-07-18T03:42:17.217 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.020377935, 2023-07-18T03:42:17.217 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:17.218 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 546, 2023-07-18T03:42:17.218 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.238943619, 2023-07-18T03:42:17.218 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000437625 2023-07-18T03:42:17.218 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.218 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:17.218 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.218 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.218 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.219 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.219 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-18T03:42:17.219 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 0 2023-07-18T03:42:17.219 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.219 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": { 2023-07-18T03:42:17.219 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 1, 2023-07-18T03:42:17.220 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2023-07-18T03:42:17.220 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.220 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-MDSRank": { 2023-07-18T03:42:17.220 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2023-07-18T03:42:17.220 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2023-07-18T03:42:17.220 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 4115, 2023-07-18T03:42:17.221 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.159993636, 2023-07-18T03:42:17.221 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000038880 2023-07-18T03:42:17.221 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:17.221 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.221 INFO:teuthology.orchestra.run.smithi040.stdout: "finisher-PurgeQueue": { 2023-07-18T03:42:17.221 INFO:teuthology.orchestra.run.smithi040.stdout: "queue_len": 0, 2023-07-18T03:42:17.221 INFO:teuthology.orchestra.run.smithi040.stdout: "complete_latency": { 2023-07-18T03:42:17.221 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.222 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.222 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.222 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:17.222 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.222 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": { 2023-07-18T03:42:17.222 INFO:teuthology.orchestra.run.smithi040.stdout: "request": 0, 2023-07-18T03:42:17.222 INFO:teuthology.orchestra.run.smithi040.stdout: "reply": 0, 2023-07-18T03:42:17.222 INFO:teuthology.orchestra.run.smithi040.stdout: "reply_latency": { 2023-07-18T03:42:17.223 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.223 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.223 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.223 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.223 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_reply": 0, 2023-07-18T03:42:17.223 INFO:teuthology.orchestra.run.smithi040.stdout: "forward": 0, 2023-07-18T03:42:17.223 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_complete": 0, 2023-07-18T03:42:17.224 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_fetch_keys": 0, 2023-07-18T03:42:17.224 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_commit": 0, 2023-07-18T03:42:17.224 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_split": 0, 2023-07-18T03:42:17.224 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_merge": 0, 2023-07-18T03:42:17.224 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes": 7710, 2023-07-18T03:42:17.224 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_top": 0, 2023-07-18T03:42:17.224 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_bottom": 0, 2023-07-18T03:42:17.224 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pin_tail": 7710, 2023-07-18T03:42:17.225 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_pinned": 7710, 2023-07-18T03:42:17.225 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_expired": 1838, 2023-07-18T03:42:17.225 INFO:teuthology.orchestra.run.smithi040.stdout: "inodes_with_caps": 0, 2023-07-18T03:42:17.225 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": 0, 2023-07-18T03:42:17.225 INFO:teuthology.orchestra.run.smithi040.stdout: "subtrees": 1, 2023-07-18T03:42:17.225 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse": 0, 2023-07-18T03:42:17.225 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_hit": 0, 2023-07-18T03:42:17.225 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_forward": 0, 2023-07-18T03:42:17.226 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_discover": 0, 2023-07-18T03:42:17.226 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_dir_fetch": 0, 2023-07-18T03:42:17.226 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_remote_ino": 0, 2023-07-18T03:42:17.226 INFO:teuthology.orchestra.run.smithi040.stdout: "traverse_lock": 0, 2023-07-18T03:42:17.226 INFO:teuthology.orchestra.run.smithi040.stdout: "load_cent": 0, 2023-07-18T03:42:17.226 INFO:teuthology.orchestra.run.smithi040.stdout: "q": 0, 2023-07-18T03:42:17.226 INFO:teuthology.orchestra.run.smithi040.stdout: "exported": 0, 2023-07-18T03:42:17.227 INFO:teuthology.orchestra.run.smithi040.stdout: "exported_inodes": 0, 2023-07-18T03:42:17.227 INFO:teuthology.orchestra.run.smithi040.stdout: "imported": 0, 2023-07-18T03:42:17.227 INFO:teuthology.orchestra.run.smithi040.stdout: "imported_inodes": 0, 2023-07-18T03:42:17.227 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_dir_fetch": 0, 2023-07-18T03:42:17.227 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_backtrace_fetch": 0, 2023-07-18T03:42:17.227 INFO:teuthology.orchestra.run.smithi040.stdout: "openino_peer_discover": 0, 2023-07-18T03:42:17.227 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rfiles": 0, 2023-07-18T03:42:17.227 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rbytes": 0, 2023-07-18T03:42:17.228 INFO:teuthology.orchestra.run.smithi040.stdout: "root_rsnaps": 0, 2023-07-18T03:42:17.228 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_fetch": 0, 2023-07-18T03:42:17.228 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_set_tag": 0, 2023-07-18T03:42:17.228 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_backtrace_repaired": 0, 2023-07-18T03:42:17.228 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_inotable_repaired": 0, 2023-07-18T03:42:17.228 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_inodes": 0, 2023-07-18T03:42:17.228 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dir_base_inodes": 0, 2023-07-18T03:42:17.228 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_dirfrag_rstats": 0, 2023-07-18T03:42:17.229 INFO:teuthology.orchestra.run.smithi040.stdout: "scrub_file_inodes": 0, 2023-07-18T03:42:17.229 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_inode_file_caps": 0, 2023-07-18T03:42:17.229 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_revoke": 0, 2023-07-18T03:42:17.229 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_grant": 0, 2023-07-18T03:42:17.229 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_trunc": 0, 2023-07-18T03:42:17.229 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-18T03:42:17.229 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-18T03:42:17.230 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps": 0, 2023-07-18T03:42:17.230 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_caps_dirty": 0, 2023-07-18T03:42:17.230 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_cap_release": 0, 2023-07-18T03:42:17.230 INFO:teuthology.orchestra.run.smithi040.stdout: "process_request_cap_release": 0 2023-07-18T03:42:17.230 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.230 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache": { 2023-07-18T03:42:17.230 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update": 0, 2023-07-18T03:42:17.230 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_update_receipt": 0, 2023-07-18T03:42:17.231 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_try_discover": 0, 2023-07-18T03:42:17.231 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_send_discover": 0, 2023-07-18T03:42:17.231 INFO:teuthology.orchestra.run.smithi040.stdout: "dir_handle_discover": 0, 2023-07-18T03:42:17.231 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays": 0, 2023-07-18T03:42:17.231 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_delayed": 0, 2023-07-18T03:42:17.231 INFO:teuthology.orchestra.run.smithi040.stdout: "num_strays_enqueuing": 0, 2023-07-18T03:42:17.231 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_created": 0, 2023-07-18T03:42:17.231 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_enqueued": 0, 2023-07-18T03:42:17.232 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_reintegrated": 0, 2023-07-18T03:42:17.232 INFO:teuthology.orchestra.run.smithi040.stdout: "strays_migrated": 0, 2023-07-18T03:42:17.232 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_processing": 0, 2023-07-18T03:42:17.232 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_enqueued": 0, 2023-07-18T03:42:17.232 INFO:teuthology.orchestra.run.smithi040.stdout: "num_recovering_prioritized": 0, 2023-07-18T03:42:17.232 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_started": 0, 2023-07-18T03:42:17.232 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_completed": 0, 2023-07-18T03:42:17.233 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_enqueue_scrub": 0, 2023-07-18T03:42:17.233 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_exportdir": 0, 2023-07-18T03:42:17.233 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_flush": 0, 2023-07-18T03:42:17.233 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragmentdir": 0, 2023-07-18T03:42:17.233 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_fragstats": 0, 2023-07-18T03:42:17.233 INFO:teuthology.orchestra.run.smithi040.stdout: "ireq_inodestats": 0 2023-07-18T03:42:17.233 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.233 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log": { 2023-07-18T03:42:17.234 INFO:teuthology.orchestra.run.smithi040.stdout: "evadd": 0, 2023-07-18T03:42:17.234 INFO:teuthology.orchestra.run.smithi040.stdout: "evex": 0, 2023-07-18T03:42:17.234 INFO:teuthology.orchestra.run.smithi040.stdout: "evtrm": 0, 2023-07-18T03:42:17.234 INFO:teuthology.orchestra.run.smithi040.stdout: "ev": 0, 2023-07-18T03:42:17.234 INFO:teuthology.orchestra.run.smithi040.stdout: "evexg": 0, 2023-07-18T03:42:17.234 INFO:teuthology.orchestra.run.smithi040.stdout: "evexd": 0, 2023-07-18T03:42:17.234 INFO:teuthology.orchestra.run.smithi040.stdout: "segadd": 0, 2023-07-18T03:42:17.234 INFO:teuthology.orchestra.run.smithi040.stdout: "segex": 0, 2023-07-18T03:42:17.235 INFO:teuthology.orchestra.run.smithi040.stdout: "segtrm": 0, 2023-07-18T03:42:17.235 INFO:teuthology.orchestra.run.smithi040.stdout: "seg": 63, 2023-07-18T03:42:17.235 INFO:teuthology.orchestra.run.smithi040.stdout: "segexg": 0, 2023-07-18T03:42:17.235 INFO:teuthology.orchestra.run.smithi040.stdout: "segexd": 0, 2023-07-18T03:42:17.235 INFO:teuthology.orchestra.run.smithi040.stdout: "expos": 4194304, 2023-07-18T03:42:17.235 INFO:teuthology.orchestra.run.smithi040.stdout: "wrpos": 0, 2023-07-18T03:42:17.235 INFO:teuthology.orchestra.run.smithi040.stdout: "rdpos": 118964390, 2023-07-18T03:42:17.235 INFO:teuthology.orchestra.run.smithi040.stdout: "jlat": { 2023-07-18T03:42:17.236 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.236 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.236 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.236 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.236 INFO:teuthology.orchestra.run.smithi040.stdout: "replayed": 51554 2023-07-18T03:42:17.236 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.236 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_mem": { 2023-07-18T03:42:17.236 INFO:teuthology.orchestra.run.smithi040.stdout: "ino": 13, 2023-07-18T03:42:17.237 INFO:teuthology.orchestra.run.smithi040.stdout: "ino+": 14416, 2023-07-18T03:42:17.237 INFO:teuthology.orchestra.run.smithi040.stdout: "ino-": 14403, 2023-07-18T03:42:17.237 INFO:teuthology.orchestra.run.smithi040.stdout: "dir": 11, 2023-07-18T03:42:17.237 INFO:teuthology.orchestra.run.smithi040.stdout: "dir+": 666, 2023-07-18T03:42:17.237 INFO:teuthology.orchestra.run.smithi040.stdout: "dir-": 655, 2023-07-18T03:42:17.237 INFO:teuthology.orchestra.run.smithi040.stdout: "dn": 7710, 2023-07-18T03:42:17.237 INFO:teuthology.orchestra.run.smithi040.stdout: "dn+": 32082, 2023-07-18T03:42:17.237 INFO:teuthology.orchestra.run.smithi040.stdout: "dn-": 24372, 2023-07-18T03:42:17.238 INFO:teuthology.orchestra.run.smithi040.stdout: "cap": 0, 2023-07-18T03:42:17.238 INFO:teuthology.orchestra.run.smithi040.stdout: "cap+": 0, 2023-07-18T03:42:17.238 INFO:teuthology.orchestra.run.smithi040.stdout: "cap-": 0, 2023-07-18T03:42:17.238 INFO:teuthology.orchestra.run.smithi040.stdout: "rss": 71284, 2023-07-18T03:42:17.238 INFO:teuthology.orchestra.run.smithi040.stdout: "heap": 207156 2023-07-18T03:42:17.238 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.238 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_server": { 2023-07-18T03:42:17.239 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_client_request": 0, 2023-07-18T03:42:17.239 INFO:teuthology.orchestra.run.smithi040.stdout: "dispatch_server_request": 0, 2023-07-18T03:42:17.239 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_request": 0, 2023-07-18T03:42:17.239 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_client_session": 0, 2023-07-18T03:42:17.239 INFO:teuthology.orchestra.run.smithi040.stdout: "handle_peer_request": 0, 2023-07-18T03:42:17.239 INFO:teuthology.orchestra.run.smithi040.stdout: "req_create_latency": { 2023-07-18T03:42:17.239 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.239 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.240 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.240 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.240 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getattr_latency": { 2023-07-18T03:42:17.240 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.240 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.240 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.240 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.240 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getfilelock_latency": { 2023-07-18T03:42:17.241 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.241 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.241 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.241 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.241 INFO:teuthology.orchestra.run.smithi040.stdout: "req_link_latency": { 2023-07-18T03:42:17.241 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.241 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.241 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.242 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.242 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookup_latency": { 2023-07-18T03:42:17.242 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.242 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.242 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.242 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.242 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookuphash_latency": { 2023-07-18T03:42:17.242 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.243 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.243 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.243 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.243 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupino_latency": { 2023-07-18T03:42:17.243 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.243 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.243 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.243 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.244 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupname_latency": { 2023-07-18T03:42:17.244 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.244 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.244 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.244 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.244 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupparent_latency": { 2023-07-18T03:42:17.244 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.245 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.245 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.245 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.245 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lookupsnap_latency": { 2023-07-18T03:42:17.245 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.245 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.245 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.245 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.246 INFO:teuthology.orchestra.run.smithi040.stdout: "req_lssnap_latency": { 2023-07-18T03:42:17.246 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.246 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.246 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.246 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.246 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mkdir_latency": { 2023-07-18T03:42:17.246 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.246 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.247 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.247 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.247 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mknod_latency": { 2023-07-18T03:42:17.247 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.247 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.247 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.247 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.248 INFO:teuthology.orchestra.run.smithi040.stdout: "req_mksnap_latency": { 2023-07-18T03:42:17.248 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.248 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.248 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.248 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.248 INFO:teuthology.orchestra.run.smithi040.stdout: "req_open_latency": { 2023-07-18T03:42:17.248 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.248 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.249 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.249 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.249 INFO:teuthology.orchestra.run.smithi040.stdout: "req_readdir_latency": { 2023-07-18T03:42:17.249 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.249 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.249 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.249 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.249 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rename_latency": { 2023-07-18T03:42:17.250 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.250 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.250 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.250 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.250 INFO:teuthology.orchestra.run.smithi040.stdout: "req_renamesnap_latency": { 2023-07-18T03:42:17.250 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.250 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.250 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.251 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.251 INFO:teuthology.orchestra.run.smithi040.stdout: "req_snapdiff_latency": { 2023-07-18T03:42:17.251 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.251 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.251 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.251 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.251 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmdir_latency": { 2023-07-18T03:42:17.251 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.252 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.252 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.252 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.252 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmsnap_latency": { 2023-07-18T03:42:17.252 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.252 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.252 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.253 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.253 INFO:teuthology.orchestra.run.smithi040.stdout: "req_rmxattr_latency": { 2023-07-18T03:42:17.253 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.253 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.253 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.253 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.253 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setattr_latency": { 2023-07-18T03:42:17.253 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.254 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.254 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.254 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.254 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setdirlayout_latency": { 2023-07-18T03:42:17.254 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.254 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.254 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.254 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.255 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setfilelock_latency": { 2023-07-18T03:42:17.255 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.255 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.255 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.255 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.255 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setlayout_latency": { 2023-07-18T03:42:17.255 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.255 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.256 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.256 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.256 INFO:teuthology.orchestra.run.smithi040.stdout: "req_setxattr_latency": { 2023-07-18T03:42:17.256 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.256 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.256 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.256 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.257 INFO:teuthology.orchestra.run.smithi040.stdout: "req_symlink_latency": { 2023-07-18T03:42:17.257 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.257 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.257 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.257 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.257 INFO:teuthology.orchestra.run.smithi040.stdout: "req_unlink_latency": { 2023-07-18T03:42:17.257 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.257 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.258 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.258 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.258 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_revoke_eviction": 0, 2023-07-18T03:42:17.258 INFO:teuthology.orchestra.run.smithi040.stdout: "cap_acquisition_throttle": 0, 2023-07-18T03:42:17.258 INFO:teuthology.orchestra.run.smithi040.stdout: "req_getvxattr_latency": { 2023-07-18T03:42:17.258 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.258 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.259 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.259 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:17.259 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.259 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_sessions": { 2023-07-18T03:42:17.259 INFO:teuthology.orchestra.run.smithi040.stdout: "session_count": 1, 2023-07-18T03:42:17.259 INFO:teuthology.orchestra.run.smithi040.stdout: "session_add": 1, 2023-07-18T03:42:17.259 INFO:teuthology.orchestra.run.smithi040.stdout: "session_remove": 0, 2023-07-18T03:42:17.259 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_open": 1, 2023-07-18T03:42:17.260 INFO:teuthology.orchestra.run.smithi040.stdout: "sessions_stale": 0, 2023-07-18T03:42:17.260 INFO:teuthology.orchestra.run.smithi040.stdout: "total_load": 0, 2023-07-18T03:42:17.260 INFO:teuthology.orchestra.run.smithi040.stdout: "average_load": 0, 2023-07-18T03:42:17.260 INFO:teuthology.orchestra.run.smithi040.stdout: "avg_session_uptime": 2112 2023-07-18T03:42:17.260 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.260 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": { 2023-07-18T03:42:17.260 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:42:17.260 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2023-07-18T03:42:17.261 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:42:17.261 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:42:17.261 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:42:17.261 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:42:17.261 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:42:17.261 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:42:17.261 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:42:17.261 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:42:17.262 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:42:17.262 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:42:17.262 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:42:17.262 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:42:17.262 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:42:17.262 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2023-07-18T03:42:17.262 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:42:17.263 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2023-07-18T03:42:17.263 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:42:17.263 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:42:17.263 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:42:17.263 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:42:17.263 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:42:17.263 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:42:17.263 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:42:17.264 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2023-07-18T03:42:17.264 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:42:17.264 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:42:17.264 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:42:17.264 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2023-07-18T03:42:17.264 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2023-07-18T03:42:17.264 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2023-07-18T03:42:17.264 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:42:17.265 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:42:17.265 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:42:17.265 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:42:17.265 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 232135, 2023-07-18T03:42:17.265 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 76, 2023-07-18T03:42:17.265 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 0, 2023-07-18T03:42:17.265 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 0, 2023-07-18T03:42:17.265 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2023-07-18T03:42:17.266 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2023-07-18T03:42:17.266 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:42:17.266 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2023-07-18T03:42:17.266 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:42:17.266 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2023-07-18T03:42:17.266 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 7720, 2023-07-18T03:42:17.266 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 162, 2023-07-18T03:42:17.266 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:42:17.267 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:42:17.267 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2023-07-18T03:42:17.267 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2023-07-18T03:42:17.267 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 5470600, 2023-07-18T03:42:17.267 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 30968, 2023-07-18T03:42:17.267 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2023-07-18T03:42:17.267 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2023-07-18T03:42:17.268 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2023-07-18T03:42:17.268 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2023-07-18T03:42:17.268 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.268 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter": { 2023-07-18T03:42:17.268 INFO:teuthology.orchestra.run.smithi040.stdout: "op_active": 0, 2023-07-18T03:42:17.268 INFO:teuthology.orchestra.run.smithi040.stdout: "op_laggy": 0, 2023-07-18T03:42:17.268 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send": 2438, 2023-07-18T03:42:17.268 INFO:teuthology.orchestra.run.smithi040.stdout: "op_send_bytes": 21, 2023-07-18T03:42:17.269 INFO:teuthology.orchestra.run.smithi040.stdout: "op_resend": 3, 2023-07-18T03:42:17.269 INFO:teuthology.orchestra.run.smithi040.stdout: "op_reply": 2438, 2023-07-18T03:42:17.269 INFO:teuthology.orchestra.run.smithi040.stdout: "op_latency": { 2023-07-18T03:42:17.269 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2438, 2023-07-18T03:42:17.269 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 7.684897302, 2023-07-18T03:42:17.269 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.003152131 2023-07-18T03:42:17.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.269 INFO:teuthology.orchestra.run.smithi040.stdout: "op_inflight": 0, 2023-07-18T03:42:17.270 INFO:teuthology.orchestra.run.smithi040.stdout: "oplen_avg": { 2023-07-18T03:42:17.270 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 2438, 2023-07-18T03:42:17.270 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 2439 2023-07-18T03:42:17.270 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.270 INFO:teuthology.orchestra.run.smithi040.stdout: "op": 2438, 2023-07-18T03:42:17.270 INFO:teuthology.orchestra.run.smithi040.stdout: "op_r": 2438, 2023-07-18T03:42:17.270 INFO:teuthology.orchestra.run.smithi040.stdout: "op_w": 0, 2023-07-18T03:42:17.270 INFO:teuthology.orchestra.run.smithi040.stdout: "op_rmw": 0, 2023-07-18T03:42:17.271 INFO:teuthology.orchestra.run.smithi040.stdout: "op_pg": 0, 2023-07-18T03:42:17.271 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_stat": 1375, 2023-07-18T03:42:17.271 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_create": 0, 2023-07-18T03:42:17.271 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_read": 1061, 2023-07-18T03:42:17.271 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_write": 0, 2023-07-18T03:42:17.271 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writefull": 0, 2023-07-18T03:42:17.271 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_writesame": 0, 2023-07-18T03:42:17.271 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_append": 0, 2023-07-18T03:42:17.272 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_zero": 0, 2023-07-18T03:42:17.272 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_truncate": 0, 2023-07-18T03:42:17.272 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_delete": 0, 2023-07-18T03:42:17.272 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_mapext": 0, 2023-07-18T03:42:17.272 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_sparse_read": 0, 2023-07-18T03:42:17.272 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_clonerange": 0, 2023-07-18T03:42:17.272 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_getxattr": 1, 2023-07-18T03:42:17.273 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_setxattr": 0, 2023-07-18T03:42:17.273 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_cmpxattr": 0, 2023-07-18T03:42:17.273 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_rmxattr": 0, 2023-07-18T03:42:17.273 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_resetxattrs": 0, 2023-07-18T03:42:17.273 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_call": 0, 2023-07-18T03:42:17.273 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_watch": 0, 2023-07-18T03:42:17.273 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_notify": 0, 2023-07-18T03:42:17.273 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_src_cmpxattr": 0, 2023-07-18T03:42:17.274 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls": 0, 2023-07-18T03:42:17.274 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_pgls_filter": 0, 2023-07-18T03:42:17.274 INFO:teuthology.orchestra.run.smithi040.stdout: "osdop_other": 0, 2023-07-18T03:42:17.274 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_active": 0, 2023-07-18T03:42:17.274 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_send": 0, 2023-07-18T03:42:17.274 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_resend": 0, 2023-07-18T03:42:17.274 INFO:teuthology.orchestra.run.smithi040.stdout: "linger_ping": 0, 2023-07-18T03:42:17.274 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_active": 0, 2023-07-18T03:42:17.275 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_send": 0, 2023-07-18T03:42:17.275 INFO:teuthology.orchestra.run.smithi040.stdout: "poolop_resend": 0, 2023-07-18T03:42:17.275 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_active": 0, 2023-07-18T03:42:17.275 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_send": 0, 2023-07-18T03:42:17.275 INFO:teuthology.orchestra.run.smithi040.stdout: "poolstat_resend": 0, 2023-07-18T03:42:17.275 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_active": 0, 2023-07-18T03:42:17.275 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_send": 0, 2023-07-18T03:42:17.275 INFO:teuthology.orchestra.run.smithi040.stdout: "statfs_resend": 0, 2023-07-18T03:42:17.276 INFO:teuthology.orchestra.run.smithi040.stdout: "command_active": 0, 2023-07-18T03:42:17.276 INFO:teuthology.orchestra.run.smithi040.stdout: "command_send": 0, 2023-07-18T03:42:17.276 INFO:teuthology.orchestra.run.smithi040.stdout: "command_resend": 0, 2023-07-18T03:42:17.276 INFO:teuthology.orchestra.run.smithi040.stdout: "map_epoch": 84, 2023-07-18T03:42:17.276 INFO:teuthology.orchestra.run.smithi040.stdout: "map_full": 0, 2023-07-18T03:42:17.276 INFO:teuthology.orchestra.run.smithi040.stdout: "map_inc": 3, 2023-07-18T03:42:17.276 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_sessions": 10, 2023-07-18T03:42:17.276 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_open": 10, 2023-07-18T03:42:17.277 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_session_close": 0, 2023-07-18T03:42:17.277 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_laggy": 0, 2023-07-18T03:42:17.277 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_wr": 0, 2023-07-18T03:42:17.277 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_rd": 2, 2023-07-18T03:42:17.277 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_del": 0 2023-07-18T03:42:17.277 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.277 INFO:teuthology.orchestra.run.smithi040.stdout: "oft": { 2023-07-18T03:42:17.278 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_objs": 0, 2023-07-18T03:42:17.278 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_kv_pairs": 0, 2023-07-18T03:42:17.278 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_updates": 0, 2023-07-18T03:42:17.278 INFO:teuthology.orchestra.run.smithi040.stdout: "omap_total_removes": 0 2023-07-18T03:42:17.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.278 INFO:teuthology.orchestra.run.smithi040.stdout: "osd-slow-ops": { 2023-07-18T03:42:17.278 INFO:teuthology.orchestra.run.smithi040.stdout: "slow_ops_count": 0 2023-07-18T03:42:17.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.279 INFO:teuthology.orchestra.run.smithi040.stdout: "purge_queue": { 2023-07-18T03:42:17.279 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops": 0, 2023-07-18T03:42:17.279 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_ops_high_water": 0, 2023-07-18T03:42:17.279 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing": 0, 2023-07-18T03:42:17.279 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executing_high_water": 0, 2023-07-18T03:42:17.279 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_executed": 0, 2023-07-18T03:42:17.279 INFO:teuthology.orchestra.run.smithi040.stdout: "pq_item_in_journal": 0 2023-07-18T03:42:17.279 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.280 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:42:17.280 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2023-07-18T03:42:17.280 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2023-07-18T03:42:17.280 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2023-07-18T03:42:17.280 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 2707, 2023-07-18T03:42:17.280 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 116158136, 2023-07-18T03:42:17.280 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:17.280 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 2707, 2023-07-18T03:42:17.281 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2023-07-18T03:42:17.281 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2023-07-18T03:42:17.281 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2707, 2023-07-18T03:42:17.281 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 116158136, 2023-07-18T03:42:17.281 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2023-07-18T03:42:17.281 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.281 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.281 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.282 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:17.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.282 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_bytes": { 2023-07-18T03:42:17.282 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2023-07-18T03:42:17.282 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2023-07-18T03:42:17.282 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2023-07-18T03:42:17.282 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2023-07-18T03:42:17.282 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2023-07-18T03:42:17.283 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:17.283 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:17.283 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2438, 2023-07-18T03:42:17.283 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 114960435, 2023-07-18T03:42:17.283 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2438, 2023-07-18T03:42:17.283 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 114960435, 2023-07-18T03:42:17.283 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2023-07-18T03:42:17.284 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.284 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.284 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.284 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:17.284 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.284 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-objecter_ops": { 2023-07-18T03:42:17.284 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2023-07-18T03:42:17.284 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 1024, 2023-07-18T03:42:17.285 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2023-07-18T03:42:17.285 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2023-07-18T03:42:17.285 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2023-07-18T03:42:17.285 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:17.285 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:17.285 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 2438, 2023-07-18T03:42:17.285 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 2438, 2023-07-18T03:42:17.286 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 2438, 2023-07-18T03:42:17.286 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 2438, 2023-07-18T03:42:17.286 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2023-07-18T03:42:17.286 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.286 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.286 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.286 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:17.286 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.287 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle": { 2023-07-18T03:42:17.287 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2023-07-18T03:42:17.287 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2023-07-18T03:42:17.287 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2023-07-18T03:42:17.287 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2023-07-18T03:42:17.287 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2023-07-18T03:42:17.287 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:17.287 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:17.288 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2023-07-18T03:42:17.288 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2023-07-18T03:42:17.288 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2023-07-18T03:42:17.288 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2023-07-18T03:42:17.288 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2023-07-18T03:42:17.288 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.288 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.288 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.289 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:17.289 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:17.289 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-write_buf_throttle-0x55a540446460": { 2023-07-18T03:42:17.289 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2023-07-18T03:42:17.289 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 3758096384, 2023-07-18T03:42:17.289 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2023-07-18T03:42:17.289 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 0, 2023-07-18T03:42:17.289 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 0, 2023-07-18T03:42:17.290 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:17.290 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:17.290 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2023-07-18T03:42:17.290 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2023-07-18T03:42:17.290 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 0, 2023-07-18T03:42:17.290 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 0, 2023-07-18T03:42:17.290 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2023-07-18T03:42:17.291 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:17.291 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:17.291 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:17.291 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:17.291 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:17.291 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:42:17.581 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2023-07-18T03:42:17.582 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2023-07-18T03:42:17.582 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2023-07-18T03:42:17.582 DEBUG:tasks.check_counter:Getting stats from j 2023-07-18T03:42:17.582 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph daemon mds.j perf dump 2023-07-18T03:42:17.597 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:17 smithi064 ceph-mon[98059]: pgmap v636: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 530 KiB/s wr, 168 op/s 2023-07-18T03:42:17.608 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:17 smithi040 ceph-mon[96439]: pgmap v636: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 530 KiB/s wr, 168 op/s 2023-07-18T03:42:17.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:17 smithi134 ceph-mon[98216]: pgmap v636: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 530 KiB/s wr, 168 op/s 2023-07-18T03:42:18.143 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.a/config 2023-07-18T03:42:18.671 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:18 smithi040 ceph-mon[96439]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:42:18.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:18 smithi134 ceph-mon[98216]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:42:18.848 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:18 smithi064 ceph-mon[98059]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-18T03:42:19.329 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-07-18T03:42:19.330 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:42:19.330 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 0, 2023-07-18T03:42:19.330 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 2, 2023-07-18T03:42:19.330 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 0, 2023-07-18T03:42:19.330 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 0, 2023-07-18T03:42:19.331 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 4, 2023-07-18T03:42:19.331 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 0, 2023-07-18T03:42:19.331 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.001923210, 2023-07-18T03:42:19.331 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.000000000, 2023-07-18T03:42:19.331 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.000421022, 2023-07-18T03:42:19.332 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-18T03:42:19.332 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:19.332 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:19.332 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:19.332 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:19.332 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:19.333 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:19.333 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:19.333 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:19.333 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:19.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:19.334 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-18T03:42:19.334 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 0 2023-07-18T03:42:19.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:19.334 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:42:19.334 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 1, 2023-07-18T03:42:19.334 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 163, 2023-07-18T03:42:19.335 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 160, 2023-07-18T03:42:19.335 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 104256, 2023-07-18T03:42:19.335 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 3, 2023-07-18T03:42:19.335 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 2, 2023-07-18T03:42:19.335 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.011638851, 2023-07-18T03:42:19.335 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.008777518, 2023-07-18T03:42:19.335 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.001296116, 2023-07-18T03:42:19.335 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-18T03:42:19.336 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:19.336 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 161, 2023-07-18T03:42:19.336 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.013945451, 2023-07-18T03:42:19.336 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000086617 2023-07-18T03:42:19.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:19.336 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:19.336 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:19.337 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:19.337 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:19.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:19.337 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-18T03:42:19.337 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 104256 2023-07-18T03:42:19.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:19.337 INFO:teuthology.orchestra.run.smithi040.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:42:19.338 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_messages": 221, 2023-07-18T03:42:19.338 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages": 216, 2023-07-18T03:42:19.338 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_bytes": 474112, 2023-07-18T03:42:19.338 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_bytes": 122560, 2023-07-18T03:42:19.338 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_created_connections": 2, 2023-07-18T03:42:19.338 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_active_connections": 1, 2023-07-18T03:42:19.338 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_total_time": 0.033810688, 2023-07-18T03:42:19.338 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_send_time": 0.013406034, 2023-07-18T03:42:19.339 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_recv_time": 0.014616592, 2023-07-18T03:42:19.339 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_running_fast_dispatch_time": 0.005758377, 2023-07-18T03:42:19.339 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:19.339 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 216, 2023-07-18T03:42:19.339 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.050425223, 2023-07-18T03:42:19.339 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000233450 2023-07-18T03:42:19.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:19.339 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:19.340 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:19.340 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:19.340 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:19.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:19.340 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_recv_encrypted_bytes": 474112, 2023-07-18T03:42:19.340 INFO:teuthology.orchestra.run.smithi040.stdout: "msgr_send_encrypted_bytes": 122560 2023-07-18T03:42:19.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:19.340 INFO:teuthology.orchestra.run.smithi040.stdout: "cct": { 2023-07-18T03:42:19.341 INFO:teuthology.orchestra.run.smithi040.stdout: "total_workers": 0, 2023-07-18T03:42:19.341 INFO:teuthology.orchestra.run.smithi040.stdout: "unhealthy_workers": 0 2023-07-18T03:42:19.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:19.341 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool": { 2023-07-18T03:42:19.341 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:42:19.341 INFO:teuthology.orchestra.run.smithi040.stdout: "bloom_filter_items": 0, 2023-07-18T03:42:19.341 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:42:19.342 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:42:19.342 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:42:19.342 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:42:19.342 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:42:19.342 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:42:19.342 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:42:19.342 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:42:19.342 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:42:19.343 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:42:19.343 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:42:19.343 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:42:19.343 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:42:19.343 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_items": 0, 2023-07-18T03:42:19.343 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:42:19.343 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blob_items": 0, 2023-07-18T03:42:19.344 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:42:19.344 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:42:19.344 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:42:19.344 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:42:19.344 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:42:19.344 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:42:19.344 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:42:19.344 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_txc_items": 0, 2023-07-18T03:42:19.345 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:42:19.345 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:42:19.345 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:42:19.345 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_writing_items": 0, 2023-07-18T03:42:19.345 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_bytes": 0, 2023-07-18T03:42:19.345 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_items": 0, 2023-07-18T03:42:19.345 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:42:19.345 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:42:19.346 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:42:19.346 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:42:19.346 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_bytes": 148967, 2023-07-18T03:42:19.346 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_anon_items": 29, 2023-07-18T03:42:19.346 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_bytes": 0, 2023-07-18T03:42:19.346 INFO:teuthology.orchestra.run.smithi040.stdout: "buffer_meta_items": 0, 2023-07-18T03:42:19.346 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bytes": 0, 2023-07-18T03:42:19.347 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_items": 0, 2023-07-18T03:42:19.347 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:42:19.347 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mapbl_items": 0, 2023-07-18T03:42:19.347 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:42:19.347 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pglog_items": 0, 2023-07-18T03:42:19.347 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_bytes": 0, 2023-07-18T03:42:19.347 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_items": 0, 2023-07-18T03:42:19.347 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:42:19.348 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:42:19.348 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_bytes": 0, 2023-07-18T03:42:19.348 INFO:teuthology.orchestra.run.smithi040.stdout: "pgmap_items": 0, 2023-07-18T03:42:19.348 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_bytes": 928, 2023-07-18T03:42:19.348 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_co_items": 2, 2023-07-18T03:42:19.348 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_bytes": 0, 2023-07-18T03:42:19.348 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_1_items": 0, 2023-07-18T03:42:19.348 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_bytes": 0, 2023-07-18T03:42:19.349 INFO:teuthology.orchestra.run.smithi040.stdout: "unittest_2_items": 0 2023-07-18T03:42:19.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-07-18T03:42:19.349 INFO:teuthology.orchestra.run.smithi040.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:42:19.349 INFO:teuthology.orchestra.run.smithi040.stdout: "val": 0, 2023-07-18T03:42:19.349 INFO:teuthology.orchestra.run.smithi040.stdout: "max": 104857600, 2023-07-18T03:42:19.349 INFO:teuthology.orchestra.run.smithi040.stdout: "get_started": 0, 2023-07-18T03:42:19.349 INFO:teuthology.orchestra.run.smithi040.stdout: "get": 218, 2023-07-18T03:42:19.349 INFO:teuthology.orchestra.run.smithi040.stdout: "get_sum": 440491, 2023-07-18T03:42:19.350 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:19.350 INFO:teuthology.orchestra.run.smithi040.stdout: "get_or_fail_success": 218, 2023-07-18T03:42:19.350 INFO:teuthology.orchestra.run.smithi040.stdout: "take": 0, 2023-07-18T03:42:19.350 INFO:teuthology.orchestra.run.smithi040.stdout: "take_sum": 0, 2023-07-18T03:42:19.350 INFO:teuthology.orchestra.run.smithi040.stdout: "put": 218, 2023-07-18T03:42:19.350 INFO:teuthology.orchestra.run.smithi040.stdout: "put_sum": 440491, 2023-07-18T03:42:19.350 INFO:teuthology.orchestra.run.smithi040.stdout: "wait": { 2023-07-18T03:42:19.351 INFO:teuthology.orchestra.run.smithi040.stdout: "avgcount": 0, 2023-07-18T03:42:19.351 INFO:teuthology.orchestra.run.smithi040.stdout: "sum": 0.000000000, 2023-07-18T03:42:19.351 INFO:teuthology.orchestra.run.smithi040.stdout: "avgtime": 0.000000000 2023-07-18T03:42:19.351 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:19.351 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-07-18T03:42:19.351 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-07-18T03:42:19.454 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:19 smithi040 ceph-mon[96439]: pgmap v637: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 100 op/s 2023-07-18T03:42:19.708 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-07-18T03:42:19.708 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-07-18T03:42:19.709 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-07-18T03:42:19.709 DEBUG:tasks.check_counter:Getting stats from b 2023-07-18T03:42:19.709 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph daemon mds.b perf dump 2023-07-18T03:42:19.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:19 smithi134 ceph-mon[98216]: pgmap v637: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 100 op/s 2023-07-18T03:42:19.816 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:19 smithi064 ceph-mon[98059]: pgmap v637: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 100 op/s 2023-07-18T03:42:20.358 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:42:21.598 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:21 smithi064 ceph-mon[98059]: pgmap v638: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 19 op/s 2023-07-18T03:42:21.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:21 smithi134 ceph-mon[98216]: pgmap v638: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 19 op/s 2023-07-18T03:42:21.826 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:21 smithi040 ceph-mon[96439]: pgmap v638: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 KiB/s wr, 19 op/s 2023-07-18T03:42:22.440 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-07-18T03:42:22.440 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:42:22.440 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 101151, 2023-07-18T03:42:22.441 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 141111, 2023-07-18T03:42:22.441 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 32439015, 2023-07-18T03:42:22.441 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 109485807, 2023-07-18T03:42:22.441 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 175, 2023-07-18T03:42:22.442 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 5, 2023-07-18T03:42:22.442 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 4.383715712, 2023-07-18T03:42:22.442 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 2.127389798, 2023-07-18T03:42:22.442 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 25.256698930, 2023-07-18T03:42:22.442 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.293854368, 2023-07-18T03:42:22.442 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:22.442 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 141109, 2023-07-18T03:42:22.443 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 32.214129427, 2023-07-18T03:42:22.443 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000228292 2023-07-18T03:42:22.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.443 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:22.443 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 75586, 2023-07-18T03:42:22.443 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.025123981, 2023-07-18T03:42:22.444 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000332 2023-07-18T03:42:22.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.444 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 879520, 2023-07-18T03:42:22.444 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 165248 2023-07-18T03:42:22.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.444 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:42:22.444 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2467850, 2023-07-18T03:42:22.445 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2465909, 2023-07-18T03:42:22.446 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 394306370, 2023-07-18T03:42:22.447 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 438149156, 2023-07-18T03:42:22.447 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 11, 2023-07-18T03:42:22.447 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 6, 2023-07-18T03:42:22.447 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 86.598349905, 2023-07-18T03:42:22.447 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 30.876906555, 2023-07-18T03:42:22.448 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 64.437968423, 2023-07-18T03:42:22.448 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.307539407, 2023-07-18T03:42:22.448 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:22.448 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2465909, 2023-07-18T03:42:22.448 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 64.026243267, 2023-07-18T03:42:22.448 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000025964 2023-07-18T03:42:22.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.449 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:22.449 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 4896454, 2023-07-18T03:42:22.449 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 1.386277007, 2023-07-18T03:42:22.449 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000283 2023-07-18T03:42:22.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.449 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-18T03:42:22.450 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 361632 2023-07-18T03:42:22.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.450 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:42:22.450 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 783110, 2023-07-18T03:42:22.450 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 786119, 2023-07-18T03:42:22.450 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 140370210, 2023-07-18T03:42:22.450 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 172911651, 2023-07-18T03:42:22.450 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 11, 2023-07-18T03:42:22.451 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 7, 2023-07-18T03:42:22.451 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 34.700781059, 2023-07-18T03:42:22.451 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 13.099752221, 2023-07-18T03:42:22.451 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 25.155890254, 2023-07-18T03:42:22.451 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.347784351, 2023-07-18T03:42:22.452 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:22.452 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 786117, 2023-07-18T03:42:22.452 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 41.361876306, 2023-07-18T03:42:22.452 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000052615 2023-07-18T03:42:22.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.452 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:22.452 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1548899, 2023-07-18T03:42:22.452 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.224053896, 2023-07-18T03:42:22.453 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000144 2023-07-18T03:42:22.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.453 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-18T03:42:22.453 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2023-07-18T03:42:22.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.453 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2023-07-18T03:42:22.454 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 1, 2023-07-18T03:42:22.454 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2023-07-18T03:42:22.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.454 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-MDSRank": { 2023-07-18T03:42:22.454 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-07-18T03:42:22.454 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-07-18T03:42:22.455 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 5047, 2023-07-18T03:42:22.455 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 39.570185230, 2023-07-18T03:42:22.455 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.007840337 2023-07-18T03:42:22.455 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:22.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.455 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-PurgeQueue": { 2023-07-18T03:42:22.456 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-07-18T03:42:22.456 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-07-18T03:42:22.456 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 9369, 2023-07-18T03:42:22.456 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 2.849507423, 2023-07-18T03:42:22.456 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000304142 2023-07-18T03:42:22.456 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:22.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.457 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2023-07-18T03:42:22.457 INFO:teuthology.orchestra.run.smithi064.stdout: "request": 48817, 2023-07-18T03:42:22.457 INFO:teuthology.orchestra.run.smithi064.stdout: "reply": 46666, 2023-07-18T03:42:22.457 INFO:teuthology.orchestra.run.smithi064.stdout: "reply_latency": { 2023-07-18T03:42:22.457 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 46666, 2023-07-18T03:42:22.458 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 1069.023703875, 2023-07-18T03:42:22.458 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.022907978 2023-07-18T03:42:22.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.458 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_reply": 0, 2023-07-18T03:42:22.458 INFO:teuthology.orchestra.run.smithi064.stdout: "forward": 2150, 2023-07-18T03:42:22.458 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_complete": 62, 2023-07-18T03:42:22.459 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_keys": 2, 2023-07-18T03:42:22.459 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_commit": 3436, 2023-07-18T03:42:22.459 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_split": 34, 2023-07-18T03:42:22.459 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_merge": 0, 2023-07-18T03:42:22.459 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes": 132, 2023-07-18T03:42:22.459 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_top": 81, 2023-07-18T03:42:22.460 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_bottom": 40, 2023-07-18T03:42:22.460 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pin_tail": 11, 2023-07-18T03:42:22.460 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pinned": 14, 2023-07-18T03:42:22.460 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_expired": 21575, 2023-07-18T03:42:22.460 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_with_caps": 0, 2023-07-18T03:42:22.460 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": 0, 2023-07-18T03:42:22.460 INFO:teuthology.orchestra.run.smithi064.stdout: "subtrees": 4, 2023-07-18T03:42:22.461 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse": 84186, 2023-07-18T03:42:22.461 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_hit": 72661, 2023-07-18T03:42:22.461 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_forward": 1974, 2023-07-18T03:42:22.461 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_discover": 210, 2023-07-18T03:42:22.461 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_dir_fetch": 3, 2023-07-18T03:42:22.461 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_remote_ino": 0, 2023-07-18T03:42:22.461 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_lock": 346, 2023-07-18T03:42:22.461 INFO:teuthology.orchestra.run.smithi064.stdout: "load_cent": 176, 2023-07-18T03:42:22.462 INFO:teuthology.orchestra.run.smithi064.stdout: "q": 0, 2023-07-18T03:42:22.462 INFO:teuthology.orchestra.run.smithi064.stdout: "exported": 108, 2023-07-18T03:42:22.462 INFO:teuthology.orchestra.run.smithi064.stdout: "exported_inodes": 17176, 2023-07-18T03:42:22.462 INFO:teuthology.orchestra.run.smithi064.stdout: "imported": 231, 2023-07-18T03:42:22.462 INFO:teuthology.orchestra.run.smithi064.stdout: "imported_inodes": 13881, 2023-07-18T03:42:22.462 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_dir_fetch": 0, 2023-07-18T03:42:22.462 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_backtrace_fetch": 487, 2023-07-18T03:42:22.463 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_peer_discover": 0, 2023-07-18T03:42:22.463 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rfiles": 4, 2023-07-18T03:42:22.463 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rbytes": 246, 2023-07-18T03:42:22.463 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rsnaps": 0, 2023-07-18T03:42:22.463 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_fetch": 487, 2023-07-18T03:42:22.463 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_set_tag": 0, 2023-07-18T03:42:22.463 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_repaired": 0, 2023-07-18T03:42:22.463 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_inotable_repaired": 0, 2023-07-18T03:42:22.464 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_inodes": 43, 2023-07-18T03:42:22.464 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_base_inodes": 4, 2023-07-18T03:42:22.464 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dirfrag_rstats": 39, 2023-07-18T03:42:22.464 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_file_inodes": 444, 2023-07-18T03:42:22.464 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_inode_file_caps": 230, 2023-07-18T03:42:22.464 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_revoke": 7417, 2023-07-18T03:42:22.464 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_grant": 13010, 2023-07-18T03:42:22.464 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_trunc": 5, 2023-07-18T03:42:22.465 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-18T03:42:22.465 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-18T03:42:22.465 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps": 16929, 2023-07-18T03:42:22.465 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps_dirty": 6720, 2023-07-18T03:42:22.465 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_cap_release": 243, 2023-07-18T03:42:22.465 INFO:teuthology.orchestra.run.smithi064.stdout: "process_request_cap_release": 14022 2023-07-18T03:42:22.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.466 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_cache": { 2023-07-18T03:42:22.466 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update": 0, 2023-07-18T03:42:22.466 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update_receipt": 5, 2023-07-18T03:42:22.466 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_try_discover": 3, 2023-07-18T03:42:22.466 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_send_discover": 486, 2023-07-18T03:42:22.466 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_handle_discover": 412, 2023-07-18T03:42:22.466 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays": 1, 2023-07-18T03:42:22.466 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_delayed": 0, 2023-07-18T03:42:22.467 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_enqueuing": 0, 2023-07-18T03:42:22.467 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_created": 4954, 2023-07-18T03:42:22.467 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_enqueued": 4953, 2023-07-18T03:42:22.467 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_reintegrated": 0, 2023-07-18T03:42:22.467 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_migrated": 0, 2023-07-18T03:42:22.467 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_processing": 0, 2023-07-18T03:42:22.467 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_enqueued": 0, 2023-07-18T03:42:22.467 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_prioritized": 0, 2023-07-18T03:42:22.468 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_started": 0, 2023-07-18T03:42:22.468 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_completed": 0, 2023-07-18T03:42:22.468 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_enqueue_scrub": 4, 2023-07-18T03:42:22.468 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_exportdir": 108, 2023-07-18T03:42:22.468 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_flush": 0, 2023-07-18T03:42:22.468 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragmentdir": 34, 2023-07-18T03:42:22.468 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragstats": 0, 2023-07-18T03:42:22.469 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_inodestats": 0 2023-07-18T03:42:22.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.469 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_log": { 2023-07-18T03:42:22.469 INFO:teuthology.orchestra.run.smithi064.stdout: "evadd": 41350, 2023-07-18T03:42:22.469 INFO:teuthology.orchestra.run.smithi064.stdout: "evex": 41347, 2023-07-18T03:42:22.469 INFO:teuthology.orchestra.run.smithi064.stdout: "evtrm": 41347, 2023-07-18T03:42:22.469 INFO:teuthology.orchestra.run.smithi064.stdout: "ev": 3, 2023-07-18T03:42:22.469 INFO:teuthology.orchestra.run.smithi064.stdout: "evexg": 0, 2023-07-18T03:42:22.470 INFO:teuthology.orchestra.run.smithi064.stdout: "evexd": 2, 2023-07-18T03:42:22.470 INFO:teuthology.orchestra.run.smithi064.stdout: "segadd": 90, 2023-07-18T03:42:22.470 INFO:teuthology.orchestra.run.smithi064.stdout: "segex": 89, 2023-07-18T03:42:22.470 INFO:teuthology.orchestra.run.smithi064.stdout: "segtrm": 89, 2023-07-18T03:42:22.470 INFO:teuthology.orchestra.run.smithi064.stdout: "seg": 2, 2023-07-18T03:42:22.470 INFO:teuthology.orchestra.run.smithi064.stdout: "segexg": 0, 2023-07-18T03:42:22.470 INFO:teuthology.orchestra.run.smithi064.stdout: "segexd": 1, 2023-07-18T03:42:22.470 INFO:teuthology.orchestra.run.smithi064.stdout: "expos": 103365854, 2023-07-18T03:42:22.471 INFO:teuthology.orchestra.run.smithi064.stdout: "wrpos": 103370912, 2023-07-18T03:42:22.471 INFO:teuthology.orchestra.run.smithi064.stdout: "rdpos": 0, 2023-07-18T03:42:22.471 INFO:teuthology.orchestra.run.smithi064.stdout: "jlat": { 2023-07-18T03:42:22.471 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1790, 2023-07-18T03:42:22.471 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 79.932620369, 2023-07-18T03:42:22.471 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.044655095 2023-07-18T03:42:22.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.471 INFO:teuthology.orchestra.run.smithi064.stdout: "replayed": 0 2023-07-18T03:42:22.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.472 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_mem": { 2023-07-18T03:42:22.472 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 32, 2023-07-18T03:42:22.472 INFO:teuthology.orchestra.run.smithi064.stdout: "ino+": 15615, 2023-07-18T03:42:22.472 INFO:teuthology.orchestra.run.smithi064.stdout: "ino-": 15583, 2023-07-18T03:42:22.472 INFO:teuthology.orchestra.run.smithi064.stdout: "dir": 35, 2023-07-18T03:42:22.472 INFO:teuthology.orchestra.run.smithi064.stdout: "dir+": 797, 2023-07-18T03:42:22.472 INFO:teuthology.orchestra.run.smithi064.stdout: "dir-": 762, 2023-07-18T03:42:22.473 INFO:teuthology.orchestra.run.smithi064.stdout: "dn": 132, 2023-07-18T03:42:22.473 INFO:teuthology.orchestra.run.smithi064.stdout: "dn+": 32293, 2023-07-18T03:42:22.473 INFO:teuthology.orchestra.run.smithi064.stdout: "dn-": 32161, 2023-07-18T03:42:22.473 INFO:teuthology.orchestra.run.smithi064.stdout: "cap": 0, 2023-07-18T03:42:22.473 INFO:teuthology.orchestra.run.smithi064.stdout: "cap+": 16692, 2023-07-18T03:42:22.473 INFO:teuthology.orchestra.run.smithi064.stdout: "cap-": 16692, 2023-07-18T03:42:22.473 INFO:teuthology.orchestra.run.smithi064.stdout: "rss": 110956, 2023-07-18T03:42:22.474 INFO:teuthology.orchestra.run.smithi064.stdout: "heap": 207156 2023-07-18T03:42:22.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.474 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_server": { 2023-07-18T03:42:22.474 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_client_request": 57012, 2023-07-18T03:42:22.474 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_server_request": 1050, 2023-07-18T03:42:22.474 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_request": 48817, 2023-07-18T03:42:22.474 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_session": 155, 2023-07-18T03:42:22.474 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_peer_request": 1895, 2023-07-18T03:42:22.475 INFO:teuthology.orchestra.run.smithi064.stdout: "req_create_latency": { 2023-07-18T03:42:22.475 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 6678, 2023-07-18T03:42:22.475 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 30.480923094, 2023-07-18T03:42:22.475 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.004564379 2023-07-18T03:42:22.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.475 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getattr_latency": { 2023-07-18T03:42:22.475 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1497, 2023-07-18T03:42:22.476 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 35.900300082, 2023-07-18T03:42:22.476 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.023981496 2023-07-18T03:42:22.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.476 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getfilelock_latency": { 2023-07-18T03:42:22.476 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.476 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.476 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.477 INFO:teuthology.orchestra.run.smithi064.stdout: "req_link_latency": { 2023-07-18T03:42:22.477 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.477 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.477 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.478 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookup_latency": { 2023-07-18T03:42:22.478 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 25692, 2023-07-18T03:42:22.478 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 847.594475969, 2023-07-18T03:42:22.478 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.032990599 2023-07-18T03:42:22.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.478 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookuphash_latency": { 2023-07-18T03:42:22.478 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.479 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.479 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.479 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.479 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupino_latency": { 2023-07-18T03:42:22.479 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.479 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.480 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.480 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupname_latency": { 2023-07-18T03:42:22.480 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.480 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.480 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.480 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.481 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupparent_latency": { 2023-07-18T03:42:22.481 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.481 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.481 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.481 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.481 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupsnap_latency": { 2023-07-18T03:42:22.482 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.482 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.482 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.482 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.482 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lssnap_latency": { 2023-07-18T03:42:22.482 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.483 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.483 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.483 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mkdir_latency": { 2023-07-18T03:42:22.483 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 52, 2023-07-18T03:42:22.483 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.951891388, 2023-07-18T03:42:22.484 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.018305603 2023-07-18T03:42:22.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.484 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mknod_latency": { 2023-07-18T03:42:22.484 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.484 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.485 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.485 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mksnap_latency": { 2023-07-18T03:42:22.485 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.485 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.485 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.486 INFO:teuthology.orchestra.run.smithi064.stdout: "req_open_latency": { 2023-07-18T03:42:22.486 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2090, 2023-07-18T03:42:22.486 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 6.809052377, 2023-07-18T03:42:22.486 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.003257919 2023-07-18T03:42:22.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.486 INFO:teuthology.orchestra.run.smithi064.stdout: "req_readdir_latency": { 2023-07-18T03:42:22.486 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 237, 2023-07-18T03:42:22.486 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 1.217509389, 2023-07-18T03:42:22.487 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.005137170 2023-07-18T03:42:22.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.487 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rename_latency": { 2023-07-18T03:42:22.487 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 6271, 2023-07-18T03:42:22.487 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 129.714957097, 2023-07-18T03:42:22.487 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.020684891 2023-07-18T03:42:22.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.488 INFO:teuthology.orchestra.run.smithi064.stdout: "req_renamesnap_latency": { 2023-07-18T03:42:22.488 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.488 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.488 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.488 INFO:teuthology.orchestra.run.smithi064.stdout: "req_snapdiff_latency": { 2023-07-18T03:42:22.489 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.489 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.489 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.489 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmdir_latency": { 2023-07-18T03:42:22.489 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 36, 2023-07-18T03:42:22.489 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 7.957902005, 2023-07-18T03:42:22.489 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.221052833 2023-07-18T03:42:22.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.490 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmsnap_latency": { 2023-07-18T03:42:22.490 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.490 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.490 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.490 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmxattr_latency": { 2023-07-18T03:42:22.490 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.491 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.491 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.491 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setattr_latency": { 2023-07-18T03:42:22.491 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 6, 2023-07-18T03:42:22.491 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.038738670, 2023-07-18T03:42:22.491 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.006456445 2023-07-18T03:42:22.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.492 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setdirlayout_latency": { 2023-07-18T03:42:22.492 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.492 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.492 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.492 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setfilelock_latency": { 2023-07-18T03:42:22.492 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.493 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.493 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.493 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setlayout_latency": { 2023-07-18T03:42:22.493 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.493 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.493 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.494 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setxattr_latency": { 2023-07-18T03:42:22.494 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 6, 2023-07-18T03:42:22.494 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.019668605, 2023-07-18T03:42:22.494 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.003278100 2023-07-18T03:42:22.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.494 INFO:teuthology.orchestra.run.smithi064.stdout: "req_symlink_latency": { 2023-07-18T03:42:22.494 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3, 2023-07-18T03:42:22.494 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.003704809, 2023-07-18T03:42:22.495 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001234936 2023-07-18T03:42:22.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.495 INFO:teuthology.orchestra.run.smithi064.stdout: "req_unlink_latency": { 2023-07-18T03:42:22.495 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 4098, 2023-07-18T03:42:22.495 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 8.334580390, 2023-07-18T03:42:22.495 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.002033816 2023-07-18T03:42:22.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.496 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_revoke_eviction": 0, 2023-07-18T03:42:22.496 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_acquisition_throttle": 0, 2023-07-18T03:42:22.496 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getvxattr_latency": { 2023-07-18T03:42:22.496 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.496 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.496 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.496 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:22.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.497 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_sessions": { 2023-07-18T03:42:22.497 INFO:teuthology.orchestra.run.smithi064.stdout: "session_count": 1, 2023-07-18T03:42:22.497 INFO:teuthology.orchestra.run.smithi064.stdout: "session_add": 2, 2023-07-18T03:42:22.497 INFO:teuthology.orchestra.run.smithi064.stdout: "session_remove": 1, 2023-07-18T03:42:22.497 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_open": 1, 2023-07-18T03:42:22.497 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_stale": 0, 2023-07-18T03:42:22.497 INFO:teuthology.orchestra.run.smithi064.stdout: "total_load": 3317, 2023-07-18T03:42:22.497 INFO:teuthology.orchestra.run.smithi064.stdout: "average_load": 3317, 2023-07-18T03:42:22.498 INFO:teuthology.orchestra.run.smithi064.stdout: "avg_session_uptime": 696 2023-07-18T03:42:22.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.498 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2023-07-18T03:42:22.498 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:42:22.498 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2023-07-18T03:42:22.498 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:42:22.498 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:42:22.498 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:42:22.499 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:42:22.499 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:42:22.499 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:42:22.499 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:42:22.499 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:42:22.499 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:42:22.499 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:42:22.499 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:42:22.500 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:42:22.500 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:42:22.500 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2023-07-18T03:42:22.500 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:42:22.500 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2023-07-18T03:42:22.500 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:42:22.500 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:42:22.501 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:42:22.501 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:42:22.501 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:42:22.501 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:42:22.501 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:42:22.501 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2023-07-18T03:42:22.501 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:42:22.501 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:42:22.502 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:42:22.502 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2023-07-18T03:42:22.502 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2023-07-18T03:42:22.502 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2023-07-18T03:42:22.502 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:42:22.502 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:42:22.502 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:42:22.503 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:42:22.503 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 306132, 2023-07-18T03:42:22.503 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 125, 2023-07-18T03:42:22.503 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 352, 2023-07-18T03:42:22.503 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 4, 2023-07-18T03:42:22.503 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2023-07-18T03:42:22.503 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2023-07-18T03:42:22.503 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:42:22.504 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2023-07-18T03:42:22.504 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:42:22.504 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2023-07-18T03:42:22.504 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 7720, 2023-07-18T03:42:22.504 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 162, 2023-07-18T03:42:22.504 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:42:22.504 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:42:22.504 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2023-07-18T03:42:22.505 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2023-07-18T03:42:22.505 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 213141, 2023-07-18T03:42:22.505 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 1431, 2023-07-18T03:42:22.505 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2023-07-18T03:42:22.505 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2023-07-18T03:42:22.505 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2023-07-18T03:42:22.506 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2023-07-18T03:42:22.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.506 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter": { 2023-07-18T03:42:22.506 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2023-07-18T03:42:22.506 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2023-07-18T03:42:22.506 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 26188, 2023-07-18T03:42:22.506 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 125977572, 2023-07-18T03:42:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 31, 2023-07-18T03:42:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 26188, 2023-07-18T03:42:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2023-07-18T03:42:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 26188, 2023-07-18T03:42:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 3657.683569335, 2023-07-18T03:42:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.139670214 2023-07-18T03:42:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.508 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2023-07-18T03:42:22.508 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2023-07-18T03:42:22.508 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 26188, 2023-07-18T03:42:22.508 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 60359 2023-07-18T03:42:22.508 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.508 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 26188, 2023-07-18T03:42:22.508 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 909, 2023-07-18T03:42:22.509 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 25279, 2023-07-18T03:42:22.509 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2023-07-18T03:42:22.509 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2023-07-18T03:42:22.509 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 3084, 2023-07-18T03:42:22.509 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 13562, 2023-07-18T03:42:22.509 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 254, 2023-07-18T03:42:22.509 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 2041, 2023-07-18T03:42:22.510 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 326, 2023-07-18T03:42:22.510 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2023-07-18T03:42:22.510 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2023-07-18T03:42:22.510 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2023-07-18T03:42:22.510 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 34, 2023-07-18T03:42:22.510 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 5695, 2023-07-18T03:42:22.511 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2023-07-18T03:42:22.511 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2023-07-18T03:42:22.511 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2023-07-18T03:42:22.511 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 501, 2023-07-18T03:42:22.511 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 27124, 2023-07-18T03:42:22.511 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2023-07-18T03:42:22.511 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2023-07-18T03:42:22.512 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2023-07-18T03:42:22.512 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2023-07-18T03:42:22.512 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2023-07-18T03:42:22.512 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2023-07-18T03:42:22.512 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2023-07-18T03:42:22.512 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2023-07-18T03:42:22.513 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2023-07-18T03:42:22.513 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 5, 2023-07-18T03:42:22.513 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2023-07-18T03:42:22.513 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2023-07-18T03:42:22.513 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2023-07-18T03:42:22.513 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2023-07-18T03:42:22.514 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2023-07-18T03:42:22.514 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2023-07-18T03:42:22.514 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2023-07-18T03:42:22.514 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2023-07-18T03:42:22.514 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2023-07-18T03:42:22.514 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2023-07-18T03:42:22.515 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2023-07-18T03:42:22.515 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2023-07-18T03:42:22.515 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2023-07-18T03:42:22.515 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2023-07-18T03:42:22.515 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2023-07-18T03:42:22.515 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2023-07-18T03:42:22.516 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 84, 2023-07-18T03:42:22.516 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2023-07-18T03:42:22.516 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 4, 2023-07-18T03:42:22.516 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 12, 2023-07-18T03:42:22.516 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 12, 2023-07-18T03:42:22.516 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 0, 2023-07-18T03:42:22.516 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2023-07-18T03:42:22.517 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 6534, 2023-07-18T03:42:22.517 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 228, 2023-07-18T03:42:22.517 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 971 2023-07-18T03:42:22.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.517 INFO:teuthology.orchestra.run.smithi064.stdout: "oft": { 2023-07-18T03:42:22.517 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_objs": 1, 2023-07-18T03:42:22.517 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_kv_pairs": 0, 2023-07-18T03:42:22.518 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_updates": 0, 2023-07-18T03:42:22.518 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_removes": 0 2023-07-18T03:42:22.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.518 INFO:teuthology.orchestra.run.smithi064.stdout: "osd-slow-ops": { 2023-07-18T03:42:22.518 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_ops_count": 49249 2023-07-18T03:42:22.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.518 INFO:teuthology.orchestra.run.smithi064.stdout: "purge_queue": { 2023-07-18T03:42:22.518 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops": 0, 2023-07-18T03:42:22.519 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops_high_water": 14, 2023-07-18T03:42:22.519 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing": 0, 2023-07-18T03:42:22.519 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_high_water": 6, 2023-07-18T03:42:22.519 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed": 4953, 2023-07-18T03:42:22.519 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_item_in_journal": 0 2023-07-18T03:42:22.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.519 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:42:22.519 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-07-18T03:42:22.520 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-07-18T03:42:22.520 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-07-18T03:42:22.520 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 3352107, 2023-07-18T03:42:22.520 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 266898618, 2023-07-18T03:42:22.520 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:22.520 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 3352107, 2023-07-18T03:42:22.520 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-07-18T03:42:22.521 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-07-18T03:42:22.521 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3352107, 2023-07-18T03:42:22.521 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 266898618, 2023-07-18T03:42:22.521 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-07-18T03:42:22.521 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.521 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.521 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.521 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:22.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.522 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes": { 2023-07-18T03:42:22.522 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-07-18T03:42:22.522 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-07-18T03:42:22.522 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-07-18T03:42:22.522 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-07-18T03:42:22.522 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-07-18T03:42:22.522 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:22.523 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:22.523 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 26188, 2023-07-18T03:42:22.523 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 126473529, 2023-07-18T03:42:22.523 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 26188, 2023-07-18T03:42:22.523 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 126473529, 2023-07-18T03:42:22.523 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-07-18T03:42:22.523 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.524 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.524 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.524 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:22.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.524 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops": { 2023-07-18T03:42:22.524 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-07-18T03:42:22.524 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2023-07-18T03:42:22.524 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-07-18T03:42:22.525 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-07-18T03:42:22.525 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-07-18T03:42:22.525 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:22.525 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:22.525 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 26188, 2023-07-18T03:42:22.525 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 26188, 2023-07-18T03:42:22.525 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 26188, 2023-07-18T03:42:22.526 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 26188, 2023-07-18T03:42:22.526 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-07-18T03:42:22.526 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.526 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.526 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.526 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:22.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.526 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle": { 2023-07-18T03:42:22.527 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-07-18T03:42:22.527 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2023-07-18T03:42:22.527 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-07-18T03:42:22.527 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 4953, 2023-07-18T03:42:22.527 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 500437, 2023-07-18T03:42:22.527 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:22.527 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 4953, 2023-07-18T03:42:22.527 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-07-18T03:42:22.528 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-07-18T03:42:22.528 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 251, 2023-07-18T03:42:22.528 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 500437, 2023-07-18T03:42:22.528 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-07-18T03:42:22.528 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.528 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.528 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.529 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:22.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:22.529 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle-0x55e49b4440a0": { 2023-07-18T03:42:22.529 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-07-18T03:42:22.529 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2023-07-18T03:42:22.529 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-07-18T03:42:22.529 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 41350, 2023-07-18T03:42:22.529 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 99176608, 2023-07-18T03:42:22.530 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:22.530 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 41350, 2023-07-18T03:42:22.530 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-07-18T03:42:22.530 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-07-18T03:42:22.530 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 1790, 2023-07-18T03:42:22.530 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 99176608, 2023-07-18T03:42:22.530 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-07-18T03:42:22.531 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:22.531 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:22.531 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:22.531 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:22.531 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:22.531 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-07-18T03:42:22.950 INFO:tasks.check_counter:Daemon mds.b mds.exported=108 2023-07-18T03:42:22.951 INFO:tasks.check_counter:Daemon mds.b mds.imported=231 2023-07-18T03:42:22.951 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=34 2023-07-18T03:42:22.951 DEBUG:tasks.check_counter:Getting stats from e 2023-07-18T03:42:22.951 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph daemon mds.e perf dump 2023-07-18T03:42:23.313 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:22 smithi134 ceph-mon[98216]: pgmap v639: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2023-07-18T03:42:23.326 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:22 smithi040 ceph-mon[96439]: pgmap v639: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2023-07-18T03:42:23.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:22 smithi064 ceph-mon[98059]: pgmap v639: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2023-07-18T03:42:23.543 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:42:24.891 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-07-18T03:42:24.891 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:42:24.891 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 922603, 2023-07-18T03:42:24.891 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 918445, 2023-07-18T03:42:24.892 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 159795366, 2023-07-18T03:42:24.892 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 179631931, 2023-07-18T03:42:24.892 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 10, 2023-07-18T03:42:24.892 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 3, 2023-07-18T03:42:24.892 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 39.741641664, 2023-07-18T03:42:24.892 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 14.751092188, 2023-07-18T03:42:24.892 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 28.846893964, 2023-07-18T03:42:24.893 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.120031988, 2023-07-18T03:42:24.893 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:24.893 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 918448, 2023-07-18T03:42:24.893 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 15.060680077, 2023-07-18T03:42:24.893 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000016397 2023-07-18T03:42:24.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.893 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:24.893 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1832117, 2023-07-18T03:42:24.894 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.264342440, 2023-07-18T03:42:24.894 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000144 2023-07-18T03:42:24.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.894 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-18T03:42:24.894 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 448 2023-07-18T03:42:24.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.894 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:42:24.894 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 138394, 2023-07-18T03:42:24.895 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 182511, 2023-07-18T03:42:24.895 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 43418088, 2023-07-18T03:42:24.895 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 146177971, 2023-07-18T03:42:24.895 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 21, 2023-07-18T03:42:24.895 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 8, 2023-07-18T03:42:24.895 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 5.717450095, 2023-07-18T03:42:24.895 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 2.933122180, 2023-07-18T03:42:24.896 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 19.176247233, 2023-07-18T03:42:24.896 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.182775193, 2023-07-18T03:42:24.896 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:24.896 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 182507, 2023-07-18T03:42:24.896 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 18.164584550, 2023-07-18T03:42:24.896 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000099528 2023-07-18T03:42:24.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.896 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:24.897 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 94717, 2023-07-18T03:42:24.897 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.032950255, 2023-07-18T03:42:24.897 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000347 2023-07-18T03:42:24.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.897 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-18T03:42:24.897 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 357296 2023-07-18T03:42:24.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.897 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:42:24.898 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 362060, 2023-07-18T03:42:24.898 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 365691, 2023-07-18T03:42:24.898 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 65291784, 2023-07-18T03:42:24.898 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 90884179, 2023-07-18T03:42:24.898 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 11, 2023-07-18T03:42:24.898 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 9, 2023-07-18T03:42:24.898 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 17.556669499, 2023-07-18T03:42:24.899 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 6.574777720, 2023-07-18T03:42:24.899 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 12.655929677, 2023-07-18T03:42:24.899 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.106799936, 2023-07-18T03:42:24.899 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:24.899 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 365688, 2023-07-18T03:42:24.899 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 7.508348145, 2023-07-18T03:42:24.899 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000020532 2023-07-18T03:42:24.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.900 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:24.900 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 719931, 2023-07-18T03:42:24.900 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.120991465, 2023-07-18T03:42:24.900 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000168 2023-07-18T03:42:24.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.900 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 802176, 2023-07-18T03:42:24.900 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 136144 2023-07-18T03:42:24.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.901 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2023-07-18T03:42:24.901 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 1, 2023-07-18T03:42:24.901 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2023-07-18T03:42:24.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.901 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-MDSRank": { 2023-07-18T03:42:24.901 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-07-18T03:42:24.901 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-07-18T03:42:24.901 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 4907, 2023-07-18T03:42:24.902 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 43.545500190, 2023-07-18T03:42:24.902 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.008874159 2023-07-18T03:42:24.902 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:24.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.902 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-PurgeQueue": { 2023-07-18T03:42:24.902 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-07-18T03:42:24.903 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-07-18T03:42:24.903 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 13876, 2023-07-18T03:42:24.903 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 5.293282916, 2023-07-18T03:42:24.903 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000381470 2023-07-18T03:42:24.903 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:24.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.903 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2023-07-18T03:42:24.904 INFO:teuthology.orchestra.run.smithi064.stdout: "request": 69360, 2023-07-18T03:42:24.904 INFO:teuthology.orchestra.run.smithi064.stdout: "reply": 68882, 2023-07-18T03:42:24.904 INFO:teuthology.orchestra.run.smithi064.stdout: "reply_latency": { 2023-07-18T03:42:24.904 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 68882, 2023-07-18T03:42:24.904 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 3009.398156968, 2023-07-18T03:42:24.904 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.043689180 2023-07-18T03:42:24.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.904 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_reply": 0, 2023-07-18T03:42:24.905 INFO:teuthology.orchestra.run.smithi064.stdout: "forward": 478, 2023-07-18T03:42:24.905 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_complete": 1, 2023-07-18T03:42:24.905 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_keys": 0, 2023-07-18T03:42:24.905 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_commit": 308, 2023-07-18T03:42:24.905 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_split": 33, 2023-07-18T03:42:24.905 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_merge": 0, 2023-07-18T03:42:24.905 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes": 18, 2023-07-18T03:42:24.905 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_top": 3, 2023-07-18T03:42:24.906 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_bottom": 2, 2023-07-18T03:42:24.906 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pin_tail": 13, 2023-07-18T03:42:24.906 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pinned": 13, 2023-07-18T03:42:24.906 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_expired": 15875, 2023-07-18T03:42:24.906 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_with_caps": 0, 2023-07-18T03:42:24.906 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": 0, 2023-07-18T03:42:24.906 INFO:teuthology.orchestra.run.smithi064.stdout: "subtrees": 4, 2023-07-18T03:42:24.906 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse": 127099, 2023-07-18T03:42:24.907 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_hit": 100488, 2023-07-18T03:42:24.907 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_forward": 323, 2023-07-18T03:42:24.907 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_discover": 539, 2023-07-18T03:42:24.907 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_dir_fetch": 0, 2023-07-18T03:42:24.907 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_remote_ino": 0, 2023-07-18T03:42:24.907 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_lock": 1007, 2023-07-18T03:42:24.907 INFO:teuthology.orchestra.run.smithi064.stdout: "load_cent": 0, 2023-07-18T03:42:24.907 INFO:teuthology.orchestra.run.smithi064.stdout: "q": 0, 2023-07-18T03:42:24.908 INFO:teuthology.orchestra.run.smithi064.stdout: "exported": 274, 2023-07-18T03:42:24.908 INFO:teuthology.orchestra.run.smithi064.stdout: "exported_inodes": 11484, 2023-07-18T03:42:24.908 INFO:teuthology.orchestra.run.smithi064.stdout: "imported": 325, 2023-07-18T03:42:24.908 INFO:teuthology.orchestra.run.smithi064.stdout: "imported_inodes": 13971, 2023-07-18T03:42:24.908 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_dir_fetch": 0, 2023-07-18T03:42:24.908 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_backtrace_fetch": 499, 2023-07-18T03:42:24.908 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_peer_discover": 0, 2023-07-18T03:42:24.909 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rfiles": 3846, 2023-07-18T03:42:24.909 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rbytes": 72892698, 2023-07-18T03:42:24.909 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rsnaps": 0, 2023-07-18T03:42:24.909 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_fetch": 499, 2023-07-18T03:42:24.909 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_set_tag": 0, 2023-07-18T03:42:24.909 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_repaired": 0, 2023-07-18T03:42:24.909 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_inotable_repaired": 0, 2023-07-18T03:42:24.909 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_inodes": 0, 2023-07-18T03:42:24.910 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_base_inodes": 0, 2023-07-18T03:42:24.910 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dirfrag_rstats": 0, 2023-07-18T03:42:24.910 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_file_inodes": 499, 2023-07-18T03:42:24.910 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_inode_file_caps": 283, 2023-07-18T03:42:24.910 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_revoke": 8375, 2023-07-18T03:42:24.910 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_grant": 17125, 2023-07-18T03:42:24.910 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_trunc": 0, 2023-07-18T03:42:24.910 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-18T03:42:24.911 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-18T03:42:24.911 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps": 18973, 2023-07-18T03:42:24.911 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps_dirty": 6915, 2023-07-18T03:42:24.911 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_cap_release": 363, 2023-07-18T03:42:24.911 INFO:teuthology.orchestra.run.smithi064.stdout: "process_request_cap_release": 19310 2023-07-18T03:42:24.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.911 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_cache": { 2023-07-18T03:42:24.911 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update": 3, 2023-07-18T03:42:24.912 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update_receipt": 7, 2023-07-18T03:42:24.912 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_try_discover": 5, 2023-07-18T03:42:24.912 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_send_discover": 641, 2023-07-18T03:42:24.912 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_handle_discover": 674, 2023-07-18T03:42:24.912 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays": 0, 2023-07-18T03:42:24.912 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_delayed": 0, 2023-07-18T03:42:24.912 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_enqueuing": 0, 2023-07-18T03:42:24.913 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_created": 7700, 2023-07-18T03:42:24.913 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_enqueued": 7700, 2023-07-18T03:42:24.913 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_reintegrated": 0, 2023-07-18T03:42:24.913 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_migrated": 0, 2023-07-18T03:42:24.913 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_processing": 0, 2023-07-18T03:42:24.913 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_enqueued": 0, 2023-07-18T03:42:24.913 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_prioritized": 0, 2023-07-18T03:42:24.913 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_started": 0, 2023-07-18T03:42:24.914 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_completed": 0, 2023-07-18T03:42:24.914 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_enqueue_scrub": 0, 2023-07-18T03:42:24.914 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_exportdir": 274, 2023-07-18T03:42:24.914 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_flush": 0, 2023-07-18T03:42:24.914 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragmentdir": 33, 2023-07-18T03:42:24.914 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragstats": 0, 2023-07-18T03:42:24.914 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_inodestats": 0 2023-07-18T03:42:24.914 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.915 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_log": { 2023-07-18T03:42:24.915 INFO:teuthology.orchestra.run.smithi064.stdout: "evadd": 51554, 2023-07-18T03:42:24.915 INFO:teuthology.orchestra.run.smithi064.stdout: "evex": 0, 2023-07-18T03:42:24.915 INFO:teuthology.orchestra.run.smithi064.stdout: "evtrm": 0, 2023-07-18T03:42:24.915 INFO:teuthology.orchestra.run.smithi064.stdout: "ev": 51554, 2023-07-18T03:42:24.915 INFO:teuthology.orchestra.run.smithi064.stdout: "evexg": 0, 2023-07-18T03:42:24.915 INFO:teuthology.orchestra.run.smithi064.stdout: "evexd": 0, 2023-07-18T03:42:24.915 INFO:teuthology.orchestra.run.smithi064.stdout: "segadd": 63, 2023-07-18T03:42:24.916 INFO:teuthology.orchestra.run.smithi064.stdout: "segex": 0, 2023-07-18T03:42:24.916 INFO:teuthology.orchestra.run.smithi064.stdout: "segtrm": 0, 2023-07-18T03:42:24.916 INFO:teuthology.orchestra.run.smithi064.stdout: "seg": 63, 2023-07-18T03:42:24.916 INFO:teuthology.orchestra.run.smithi064.stdout: "segexg": 0, 2023-07-18T03:42:24.916 INFO:teuthology.orchestra.run.smithi064.stdout: "segexd": 0, 2023-07-18T03:42:24.916 INFO:teuthology.orchestra.run.smithi064.stdout: "expos": 4194304, 2023-07-18T03:42:24.916 INFO:teuthology.orchestra.run.smithi064.stdout: "wrpos": 118965528, 2023-07-18T03:42:24.916 INFO:teuthology.orchestra.run.smithi064.stdout: "rdpos": 0, 2023-07-18T03:42:24.917 INFO:teuthology.orchestra.run.smithi064.stdout: "jlat": { 2023-07-18T03:42:24.917 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2288, 2023-07-18T03:42:24.917 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 60.132364001, 2023-07-18T03:42:24.917 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.026281627 2023-07-18T03:42:24.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.917 INFO:teuthology.orchestra.run.smithi064.stdout: "replayed": 0 2023-07-18T03:42:24.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.917 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_mem": { 2023-07-18T03:42:24.918 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 25, 2023-07-18T03:42:24.918 INFO:teuthology.orchestra.run.smithi064.stdout: "ino+": 14661, 2023-07-18T03:42:24.918 INFO:teuthology.orchestra.run.smithi064.stdout: "ino-": 14636, 2023-07-18T03:42:24.918 INFO:teuthology.orchestra.run.smithi064.stdout: "dir": 21, 2023-07-18T03:42:24.918 INFO:teuthology.orchestra.run.smithi064.stdout: "dir+": 737, 2023-07-18T03:42:24.918 INFO:teuthology.orchestra.run.smithi064.stdout: "dir-": 716, 2023-07-18T03:42:24.918 INFO:teuthology.orchestra.run.smithi064.stdout: "dn": 18, 2023-07-18T03:42:24.918 INFO:teuthology.orchestra.run.smithi064.stdout: "dn+": 33767, 2023-07-18T03:42:24.919 INFO:teuthology.orchestra.run.smithi064.stdout: "dn-": 33749, 2023-07-18T03:42:24.919 INFO:teuthology.orchestra.run.smithi064.stdout: "cap": 0, 2023-07-18T03:42:24.919 INFO:teuthology.orchestra.run.smithi064.stdout: "cap+": 15592, 2023-07-18T03:42:24.919 INFO:teuthology.orchestra.run.smithi064.stdout: "cap-": 15592, 2023-07-18T03:42:24.919 INFO:teuthology.orchestra.run.smithi064.stdout: "rss": 105236, 2023-07-18T03:42:24.919 INFO:teuthology.orchestra.run.smithi064.stdout: "heap": 198964 2023-07-18T03:42:24.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.920 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_server": { 2023-07-18T03:42:24.920 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_client_request": 78598, 2023-07-18T03:42:24.920 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_server_request": 3154, 2023-07-18T03:42:24.920 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_request": 69360, 2023-07-18T03:42:24.920 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_session": 357, 2023-07-18T03:42:24.920 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_peer_request": 5485, 2023-07-18T03:42:24.920 INFO:teuthology.orchestra.run.smithi064.stdout: "req_create_latency": { 2023-07-18T03:42:24.920 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 6666, 2023-07-18T03:42:24.921 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 66.646069960, 2023-07-18T03:42:24.921 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.009997910 2023-07-18T03:42:24.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.921 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getattr_latency": { 2023-07-18T03:42:24.921 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 152, 2023-07-18T03:42:24.921 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 13.428713754, 2023-07-18T03:42:24.921 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.088346801 2023-07-18T03:42:24.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.922 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getfilelock_latency": { 2023-07-18T03:42:24.922 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.922 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.922 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.922 INFO:teuthology.orchestra.run.smithi064.stdout: "req_link_latency": { 2023-07-18T03:42:24.922 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.923 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.923 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.923 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookup_latency": { 2023-07-18T03:42:24.923 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 45975, 2023-07-18T03:42:24.923 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 2627.662743051, 2023-07-18T03:42:24.923 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.057154165 2023-07-18T03:42:24.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.924 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookuphash_latency": { 2023-07-18T03:42:24.924 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.924 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.924 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.924 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupino_latency": { 2023-07-18T03:42:24.924 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.924 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.925 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.925 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupname_latency": { 2023-07-18T03:42:24.925 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.925 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.925 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.925 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupparent_latency": { 2023-07-18T03:42:24.926 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.926 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.926 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.926 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupsnap_latency": { 2023-07-18T03:42:24.926 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.926 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.926 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.927 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lssnap_latency": { 2023-07-18T03:42:24.927 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.927 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.927 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.927 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mkdir_latency": { 2023-07-18T03:42:24.928 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 29, 2023-07-18T03:42:24.928 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 1.594726299, 2023-07-18T03:42:24.928 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.054990562 2023-07-18T03:42:24.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.928 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mknod_latency": { 2023-07-18T03:42:24.928 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.928 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.928 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.929 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mksnap_latency": { 2023-07-18T03:42:24.929 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.929 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.929 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.929 INFO:teuthology.orchestra.run.smithi064.stdout: "req_open_latency": { 2023-07-18T03:42:24.929 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2940, 2023-07-18T03:42:24.930 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 33.390615287, 2023-07-18T03:42:24.930 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.011357352 2023-07-18T03:42:24.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.930 INFO:teuthology.orchestra.run.smithi064.stdout: "req_readdir_latency": { 2023-07-18T03:42:24.930 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 278, 2023-07-18T03:42:24.930 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.713054042, 2023-07-18T03:42:24.930 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.002564942 2023-07-18T03:42:24.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.931 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rename_latency": { 2023-07-18T03:42:24.931 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 6653, 2023-07-18T03:42:24.931 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 241.918854633, 2023-07-18T03:42:24.931 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.036362371 2023-07-18T03:42:24.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.931 INFO:teuthology.orchestra.run.smithi064.stdout: "req_renamesnap_latency": { 2023-07-18T03:42:24.931 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.932 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.932 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.932 INFO:teuthology.orchestra.run.smithi064.stdout: "req_snapdiff_latency": { 2023-07-18T03:42:24.932 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.932 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.932 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.933 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmdir_latency": { 2023-07-18T03:42:24.933 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 52, 2023-07-18T03:42:24.933 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 8.489689846, 2023-07-18T03:42:24.933 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.163263266 2023-07-18T03:42:24.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.933 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmsnap_latency": { 2023-07-18T03:42:24.933 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.933 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.934 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.934 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmxattr_latency": { 2023-07-18T03:42:24.934 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.934 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.934 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.934 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setattr_latency": { 2023-07-18T03:42:24.935 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.935 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.935 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.935 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setdirlayout_latency": { 2023-07-18T03:42:24.935 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.935 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.935 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.936 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setfilelock_latency": { 2023-07-18T03:42:24.936 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.936 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.936 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.936 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setlayout_latency": { 2023-07-18T03:42:24.937 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.937 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.937 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.937 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setxattr_latency": { 2023-07-18T03:42:24.937 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.937 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.937 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.938 INFO:teuthology.orchestra.run.smithi064.stdout: "req_symlink_latency": { 2023-07-18T03:42:24.938 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.938 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.938 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.938 INFO:teuthology.orchestra.run.smithi064.stdout: "req_unlink_latency": { 2023-07-18T03:42:24.938 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 6137, 2023-07-18T03:42:24.939 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 15.553690096, 2023-07-18T03:42:24.939 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.002534412 2023-07-18T03:42:24.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.939 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_revoke_eviction": 0, 2023-07-18T03:42:24.939 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_acquisition_throttle": 0, 2023-07-18T03:42:24.939 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getvxattr_latency": { 2023-07-18T03:42:24.939 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.939 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.940 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.940 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:24.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.940 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_sessions": { 2023-07-18T03:42:24.940 INFO:teuthology.orchestra.run.smithi064.stdout: "session_count": 1, 2023-07-18T03:42:24.940 INFO:teuthology.orchestra.run.smithi064.stdout: "session_add": 1, 2023-07-18T03:42:24.940 INFO:teuthology.orchestra.run.smithi064.stdout: "session_remove": 0, 2023-07-18T03:42:24.941 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_open": 1, 2023-07-18T03:42:24.941 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_stale": 0, 2023-07-18T03:42:24.941 INFO:teuthology.orchestra.run.smithi064.stdout: "total_load": 2439, 2023-07-18T03:42:24.941 INFO:teuthology.orchestra.run.smithi064.stdout: "average_load": 2439, 2023-07-18T03:42:24.941 INFO:teuthology.orchestra.run.smithi064.stdout: "avg_session_uptime": 600 2023-07-18T03:42:24.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.941 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2023-07-18T03:42:24.941 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:42:24.942 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2023-07-18T03:42:24.942 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:42:24.942 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:42:24.942 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:42:24.942 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:42:24.942 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:42:24.942 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:42:24.942 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:42:24.943 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:42:24.943 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:42:24.943 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:42:24.943 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:42:24.943 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:42:24.943 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:42:24.943 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2023-07-18T03:42:24.943 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:42:24.944 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2023-07-18T03:42:24.944 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:42:24.944 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:42:24.944 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:42:24.944 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:42:24.944 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:42:24.944 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:42:24.944 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:42:24.945 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2023-07-18T03:42:24.945 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:42:24.945 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:42:24.945 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:42:24.945 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2023-07-18T03:42:24.945 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2023-07-18T03:42:24.945 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2023-07-18T03:42:24.946 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:42:24.946 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:42:24.946 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:42:24.946 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:42:24.946 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 311933, 2023-07-18T03:42:24.946 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 130, 2023-07-18T03:42:24.946 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 440, 2023-07-18T03:42:24.946 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 5, 2023-07-18T03:42:24.947 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2023-07-18T03:42:24.947 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2023-07-18T03:42:24.947 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:42:24.947 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2023-07-18T03:42:24.947 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:42:24.947 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2023-07-18T03:42:24.947 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 7720, 2023-07-18T03:42:24.947 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 162, 2023-07-18T03:42:24.948 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:42:24.948 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:42:24.948 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2023-07-18T03:42:24.948 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2023-07-18T03:42:24.948 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 96245, 2023-07-18T03:42:24.948 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 510, 2023-07-18T03:42:24.948 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2023-07-18T03:42:24.948 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2023-07-18T03:42:24.949 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2023-07-18T03:42:24.949 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2023-07-18T03:42:24.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.949 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter": { 2023-07-18T03:42:24.949 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2023-07-18T03:42:24.949 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2023-07-18T03:42:24.949 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 12103, 2023-07-18T03:42:24.949 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 121933691, 2023-07-18T03:42:24.950 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 33, 2023-07-18T03:42:24.950 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 12103, 2023-07-18T03:42:24.950 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2023-07-18T03:42:24.950 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 12103, 2023-07-18T03:42:24.950 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 94.651073023, 2023-07-18T03:42:24.950 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.007820463 2023-07-18T03:42:24.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.951 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2023-07-18T03:42:24.951 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2023-07-18T03:42:24.951 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 12103, 2023-07-18T03:42:24.951 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 12638 2023-07-18T03:42:24.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.951 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 12103, 2023-07-18T03:42:24.951 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 816, 2023-07-18T03:42:24.951 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 11287, 2023-07-18T03:42:24.952 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2023-07-18T03:42:24.952 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2023-07-18T03:42:24.952 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 0, 2023-07-18T03:42:24.952 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 10, 2023-07-18T03:42:24.952 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 314, 2023-07-18T03:42:24.952 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 2603, 2023-07-18T03:42:24.952 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 190, 2023-07-18T03:42:24.952 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2023-07-18T03:42:24.953 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2023-07-18T03:42:24.953 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2023-07-18T03:42:24.953 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 33, 2023-07-18T03:42:24.953 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 8035, 2023-07-18T03:42:24.953 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2023-07-18T03:42:24.953 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2023-07-18T03:42:24.953 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2023-07-18T03:42:24.953 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 499, 2023-07-18T03:42:24.954 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 20, 2023-07-18T03:42:24.954 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2023-07-18T03:42:24.954 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2023-07-18T03:42:24.954 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2023-07-18T03:42:24.954 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2023-07-18T03:42:24.954 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2023-07-18T03:42:24.954 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2023-07-18T03:42:24.954 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2023-07-18T03:42:24.955 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2023-07-18T03:42:24.955 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2023-07-18T03:42:24.955 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2023-07-18T03:42:24.955 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2023-07-18T03:42:24.955 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2023-07-18T03:42:24.955 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2023-07-18T03:42:24.955 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2023-07-18T03:42:24.955 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2023-07-18T03:42:24.956 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2023-07-18T03:42:24.956 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2023-07-18T03:42:24.956 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2023-07-18T03:42:24.956 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2023-07-18T03:42:24.956 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2023-07-18T03:42:24.956 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2023-07-18T03:42:24.956 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2023-07-18T03:42:24.957 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2023-07-18T03:42:24.957 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2023-07-18T03:42:24.957 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2023-07-18T03:42:24.957 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2023-07-18T03:42:24.957 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 84, 2023-07-18T03:42:24.957 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2023-07-18T03:42:24.957 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 3, 2023-07-18T03:42:24.957 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 12, 2023-07-18T03:42:24.958 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 12, 2023-07-18T03:42:24.958 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 0, 2023-07-18T03:42:24.958 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2023-07-18T03:42:24.958 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 805, 2023-07-18T03:42:24.958 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 4, 2023-07-18T03:42:24.958 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 125 2023-07-18T03:42:24.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.958 INFO:teuthology.orchestra.run.smithi064.stdout: "oft": { 2023-07-18T03:42:24.959 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_objs": 1, 2023-07-18T03:42:24.959 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_kv_pairs": 0, 2023-07-18T03:42:24.959 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_updates": 0, 2023-07-18T03:42:24.959 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_removes": 0 2023-07-18T03:42:24.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.959 INFO:teuthology.orchestra.run.smithi064.stdout: "osd-slow-ops": { 2023-07-18T03:42:24.959 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_ops_count": 71164 2023-07-18T03:42:24.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.960 INFO:teuthology.orchestra.run.smithi064.stdout: "purge_queue": { 2023-07-18T03:42:24.960 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops": 0, 2023-07-18T03:42:24.960 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops_high_water": 14, 2023-07-18T03:42:24.960 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing": 0, 2023-07-18T03:42:24.960 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_high_water": 6, 2023-07-18T03:42:24.960 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed": 7700, 2023-07-18T03:42:24.960 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_item_in_journal": 0 2023-07-18T03:42:24.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.961 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:42:24.961 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-07-18T03:42:24.961 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-07-18T03:42:24.961 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-07-18T03:42:24.961 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 1423053, 2023-07-18T03:42:24.961 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 142541492, 2023-07-18T03:42:24.961 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:24.961 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 1423053, 2023-07-18T03:42:24.962 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-07-18T03:42:24.962 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-07-18T03:42:24.962 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 1423053, 2023-07-18T03:42:24.962 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 142541492, 2023-07-18T03:42:24.962 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-07-18T03:42:24.962 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.962 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.962 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.963 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:24.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.963 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes": { 2023-07-18T03:42:24.963 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-07-18T03:42:24.963 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-07-18T03:42:24.963 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-07-18T03:42:24.963 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-07-18T03:42:24.964 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-07-18T03:42:24.964 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:24.964 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:24.964 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 12103, 2023-07-18T03:42:24.964 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 122711601, 2023-07-18T03:42:24.964 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 12103, 2023-07-18T03:42:24.964 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 122711601, 2023-07-18T03:42:24.964 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-07-18T03:42:24.965 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.965 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.965 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.965 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:24.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.965 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops": { 2023-07-18T03:42:24.965 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-07-18T03:42:24.965 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2023-07-18T03:42:24.966 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-07-18T03:42:24.966 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-07-18T03:42:24.966 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-07-18T03:42:24.966 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:24.966 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:24.966 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 12103, 2023-07-18T03:42:24.966 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 12103, 2023-07-18T03:42:24.967 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 12103, 2023-07-18T03:42:24.967 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 12103, 2023-07-18T03:42:24.967 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-07-18T03:42:24.967 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.967 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.967 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.967 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:24.967 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.968 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle": { 2023-07-18T03:42:24.968 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-07-18T03:42:24.968 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2023-07-18T03:42:24.968 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-07-18T03:42:24.968 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 7700, 2023-07-18T03:42:24.968 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 777996, 2023-07-18T03:42:24.968 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:24.968 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 7700, 2023-07-18T03:42:24.969 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-07-18T03:42:24.969 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-07-18T03:42:24.969 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 315, 2023-07-18T03:42:24.969 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 777996, 2023-07-18T03:42:24.969 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-07-18T03:42:24.969 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.969 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.969 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.970 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:24.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:24.970 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle-0x560fccdfa0a0": { 2023-07-18T03:42:24.970 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-07-18T03:42:24.970 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2023-07-18T03:42:24.970 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-07-18T03:42:24.970 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 51554, 2023-07-18T03:42:24.970 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 114771224, 2023-07-18T03:42:24.971 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:24.971 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 51554, 2023-07-18T03:42:24.971 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-07-18T03:42:24.971 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-07-18T03:42:24.971 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 2288, 2023-07-18T03:42:24.971 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 114771224, 2023-07-18T03:42:24.971 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-07-18T03:42:24.971 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:24.972 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:24.972 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:24.972 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:24.972 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:24.972 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-07-18T03:42:25.290 INFO:tasks.check_counter:Daemon mds.e mds.exported=274 2023-07-18T03:42:25.291 INFO:tasks.check_counter:Daemon mds.e mds.imported=325 2023-07-18T03:42:25.291 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=33 2023-07-18T03:42:25.291 DEBUG:tasks.check_counter:Getting stats from h 2023-07-18T03:42:25.291 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph daemon mds.h perf dump 2023-07-18T03:42:25.597 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:25 smithi064 ceph-mon[98059]: pgmap v640: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-07-18T03:42:25.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:25 smithi134 ceph-mon[98216]: pgmap v640: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-07-18T03:42:25.826 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:25 smithi040 ceph-mon[96439]: pgmap v640: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-07-18T03:42:25.850 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:42:27.074 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-07-18T03:42:27.074 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:42:27.074 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 210748, 2023-07-18T03:42:27.074 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 203272, 2023-07-18T03:42:27.075 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 43602504, 2023-07-18T03:42:27.075 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 119844689, 2023-07-18T03:42:27.075 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 11, 2023-07-18T03:42:27.075 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 4, 2023-07-18T03:42:27.075 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 10.590838470, 2023-07-18T03:42:27.075 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 3.948403981, 2023-07-18T03:42:27.075 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 7.722225473, 2023-07-18T03:42:27.076 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.158455874, 2023-07-18T03:42:27.076 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:27.076 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 203270, 2023-07-18T03:42:27.076 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 4.716478528, 2023-07-18T03:42:27.076 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000023203 2023-07-18T03:42:27.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.076 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:27.077 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 405963, 2023-07-18T03:42:27.077 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.067802373, 2023-07-18T03:42:27.077 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000167 2023-07-18T03:42:27.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.077 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-18T03:42:27.077 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2023-07-18T03:42:27.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.077 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:42:27.078 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 117738, 2023-07-18T03:42:27.078 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 184355, 2023-07-18T03:42:27.078 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 35678257, 2023-07-18T03:42:27.078 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 139740825, 2023-07-18T03:42:27.078 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 21, 2023-07-18T03:42:27.078 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 6, 2023-07-18T03:42:27.078 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 5.852341946, 2023-07-18T03:42:27.079 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 3.063968465, 2023-07-18T03:42:27.079 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 27.371616937, 2023-07-18T03:42:27.079 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.185269417, 2023-07-18T03:42:27.079 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:27.079 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 184354, 2023-07-18T03:42:27.079 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 18.224019819, 2023-07-18T03:42:27.079 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000098853 2023-07-18T03:42:27.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.080 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:27.080 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 105625, 2023-07-18T03:42:27.080 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.035469661, 2023-07-18T03:42:27.080 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000335 2023-07-18T03:42:27.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.080 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-07-18T03:42:27.080 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 358496 2023-07-18T03:42:27.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.081 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:42:27.081 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 15327, 2023-07-18T03:42:27.081 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 11012, 2023-07-18T03:42:27.081 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 8843607, 2023-07-18T03:42:27.081 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 21358125, 2023-07-18T03:42:27.081 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 10, 2023-07-18T03:42:27.081 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 7, 2023-07-18T03:42:27.082 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 1.026908903, 2023-07-18T03:42:27.082 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.450977820, 2023-07-18T03:42:27.082 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.570048350, 2023-07-18T03:42:27.082 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.108425766, 2023-07-18T03:42:27.082 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:27.083 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 11011, 2023-07-18T03:42:27.083 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.722013989, 2023-07-18T03:42:27.083 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000065572 2023-07-18T03:42:27.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.083 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:27.084 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 18310, 2023-07-18T03:42:27.084 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.003550561, 2023-07-18T03:42:27.084 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000193 2023-07-18T03:42:27.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.084 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 807552, 2023-07-18T03:42:27.084 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 357728 2023-07-18T03:42:27.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.085 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2023-07-18T03:42:27.085 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 1, 2023-07-18T03:42:27.085 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2023-07-18T03:42:27.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.085 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-MDSRank": { 2023-07-18T03:42:27.085 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-07-18T03:42:27.085 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-07-18T03:42:27.085 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3724, 2023-07-18T03:42:27.086 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 73.087201915, 2023-07-18T03:42:27.086 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.019625994 2023-07-18T03:42:27.086 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:27.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.086 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-PurgeQueue": { 2023-07-18T03:42:27.086 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2023-07-18T03:42:27.086 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2023-07-18T03:42:27.087 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 14903, 2023-07-18T03:42:27.087 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 5.757920421, 2023-07-18T03:42:27.087 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000386359 2023-07-18T03:42:27.087 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:27.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.087 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2023-07-18T03:42:27.087 INFO:teuthology.orchestra.run.smithi064.stdout: "request": 66283, 2023-07-18T03:42:27.087 INFO:teuthology.orchestra.run.smithi064.stdout: "reply": 64412, 2023-07-18T03:42:27.088 INFO:teuthology.orchestra.run.smithi064.stdout: "reply_latency": { 2023-07-18T03:42:27.088 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 64412, 2023-07-18T03:42:27.088 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 1461.118311540, 2023-07-18T03:42:27.088 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.022683945 2023-07-18T03:42:27.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.088 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_reply": 0, 2023-07-18T03:42:27.088 INFO:teuthology.orchestra.run.smithi064.stdout: "forward": 1871, 2023-07-18T03:42:27.088 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_complete": 0, 2023-07-18T03:42:27.089 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_keys": 0, 2023-07-18T03:42:27.089 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_commit": 578, 2023-07-18T03:42:27.089 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_split": 63, 2023-07-18T03:42:27.089 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_merge": 0, 2023-07-18T03:42:27.089 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes": 33, 2023-07-18T03:42:27.089 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_top": 11, 2023-07-18T03:42:27.089 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_bottom": 11, 2023-07-18T03:42:27.089 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pin_tail": 11, 2023-07-18T03:42:27.090 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pinned": 14, 2023-07-18T03:42:27.090 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_expired": 26367, 2023-07-18T03:42:27.090 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_with_caps": 1, 2023-07-18T03:42:27.090 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": 1, 2023-07-18T03:42:27.090 INFO:teuthology.orchestra.run.smithi064.stdout: "subtrees": 5, 2023-07-18T03:42:27.090 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse": 110198, 2023-07-18T03:42:27.090 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_hit": 91759, 2023-07-18T03:42:27.091 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_forward": 1778, 2023-07-18T03:42:27.091 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_discover": 322, 2023-07-18T03:42:27.091 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_dir_fetch": 0, 2023-07-18T03:42:27.091 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_remote_ino": 0, 2023-07-18T03:42:27.091 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_lock": 104, 2023-07-18T03:42:27.091 INFO:teuthology.orchestra.run.smithi064.stdout: "load_cent": 0, 2023-07-18T03:42:27.091 INFO:teuthology.orchestra.run.smithi064.stdout: "q": 0, 2023-07-18T03:42:27.091 INFO:teuthology.orchestra.run.smithi064.stdout: "exported": 183, 2023-07-18T03:42:27.092 INFO:teuthology.orchestra.run.smithi064.stdout: "exported_inodes": 20490, 2023-07-18T03:42:27.092 INFO:teuthology.orchestra.run.smithi064.stdout: "imported": 225, 2023-07-18T03:42:27.092 INFO:teuthology.orchestra.run.smithi064.stdout: "imported_inodes": 12947, 2023-07-18T03:42:27.092 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_dir_fetch": 0, 2023-07-18T03:42:27.092 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_backtrace_fetch": 796, 2023-07-18T03:42:27.092 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_peer_discover": 0, 2023-07-18T03:42:27.092 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rfiles": 3846, 2023-07-18T03:42:27.092 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rbytes": 72892698, 2023-07-18T03:42:27.093 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rsnaps": 0, 2023-07-18T03:42:27.093 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_fetch": 796, 2023-07-18T03:42:27.093 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_set_tag": 0, 2023-07-18T03:42:27.093 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_repaired": 0, 2023-07-18T03:42:27.093 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_inotable_repaired": 0, 2023-07-18T03:42:27.093 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_inodes": 0, 2023-07-18T03:42:27.093 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_base_inodes": 0, 2023-07-18T03:42:27.093 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dirfrag_rstats": 0, 2023-07-18T03:42:27.094 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_file_inodes": 796, 2023-07-18T03:42:27.094 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_inode_file_caps": 102, 2023-07-18T03:42:27.094 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_revoke": 13536, 2023-07-18T03:42:27.094 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_grant": 22617, 2023-07-18T03:42:27.094 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_trunc": 0, 2023-07-18T03:42:27.095 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-18T03:42:27.095 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-18T03:42:27.095 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps": 27485, 2023-07-18T03:42:27.095 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps_dirty": 12165, 2023-07-18T03:42:27.095 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_cap_release": 314, 2023-07-18T03:42:27.095 INFO:teuthology.orchestra.run.smithi064.stdout: "process_request_cap_release": 25045 2023-07-18T03:42:27.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.095 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_cache": { 2023-07-18T03:42:27.096 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update": 3, 2023-07-18T03:42:27.096 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update_receipt": 8, 2023-07-18T03:42:27.096 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_try_discover": 6, 2023-07-18T03:42:27.096 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_send_discover": 413, 2023-07-18T03:42:27.096 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_handle_discover": 180, 2023-07-18T03:42:27.096 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays": 1, 2023-07-18T03:42:27.096 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_delayed": 0, 2023-07-18T03:42:27.097 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_enqueuing": 0, 2023-07-18T03:42:27.097 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_created": 8282, 2023-07-18T03:42:27.097 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_enqueued": 8281, 2023-07-18T03:42:27.097 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_reintegrated": 0, 2023-07-18T03:42:27.097 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_migrated": 0, 2023-07-18T03:42:27.097 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_processing": 0, 2023-07-18T03:42:27.097 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_enqueued": 0, 2023-07-18T03:42:27.097 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_prioritized": 0, 2023-07-18T03:42:27.098 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_started": 0, 2023-07-18T03:42:27.098 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_completed": 0, 2023-07-18T03:42:27.098 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_enqueue_scrub": 0, 2023-07-18T03:42:27.098 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_exportdir": 183, 2023-07-18T03:42:27.098 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_flush": 0, 2023-07-18T03:42:27.098 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragmentdir": 63, 2023-07-18T03:42:27.098 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragstats": 0, 2023-07-18T03:42:27.098 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_inodestats": 0 2023-07-18T03:42:27.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.099 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_log": { 2023-07-18T03:42:27.099 INFO:teuthology.orchestra.run.smithi064.stdout: "evadd": 69019, 2023-07-18T03:42:27.099 INFO:teuthology.orchestra.run.smithi064.stdout: "evex": 0, 2023-07-18T03:42:27.099 INFO:teuthology.orchestra.run.smithi064.stdout: "evtrm": 0, 2023-07-18T03:42:27.099 INFO:teuthology.orchestra.run.smithi064.stdout: "ev": 69019, 2023-07-18T03:42:27.099 INFO:teuthology.orchestra.run.smithi064.stdout: "evexg": 0, 2023-07-18T03:42:27.099 INFO:teuthology.orchestra.run.smithi064.stdout: "evexd": 0, 2023-07-18T03:42:27.100 INFO:teuthology.orchestra.run.smithi064.stdout: "segadd": 84, 2023-07-18T03:42:27.100 INFO:teuthology.orchestra.run.smithi064.stdout: "segex": 0, 2023-07-18T03:42:27.100 INFO:teuthology.orchestra.run.smithi064.stdout: "segtrm": 0, 2023-07-18T03:42:27.100 INFO:teuthology.orchestra.run.smithi064.stdout: "seg": 84, 2023-07-18T03:42:27.100 INFO:teuthology.orchestra.run.smithi064.stdout: "segexg": 0, 2023-07-18T03:42:27.100 INFO:teuthology.orchestra.run.smithi064.stdout: "segexd": 0, 2023-07-18T03:42:27.101 INFO:teuthology.orchestra.run.smithi064.stdout: "expos": 4194304, 2023-07-18T03:42:27.101 INFO:teuthology.orchestra.run.smithi064.stdout: "wrpos": 164688875, 2023-07-18T03:42:27.101 INFO:teuthology.orchestra.run.smithi064.stdout: "rdpos": 0, 2023-07-18T03:42:27.101 INFO:teuthology.orchestra.run.smithi064.stdout: "jlat": { 2023-07-18T03:42:27.101 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1420, 2023-07-18T03:42:27.101 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 100.345702817, 2023-07-18T03:42:27.101 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.070665987 2023-07-18T03:42:27.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.102 INFO:teuthology.orchestra.run.smithi064.stdout: "replayed": 0 2023-07-18T03:42:27.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.102 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_mem": { 2023-07-18T03:42:27.102 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 29, 2023-07-18T03:42:27.102 INFO:teuthology.orchestra.run.smithi064.stdout: "ino+": 20096, 2023-07-18T03:42:27.102 INFO:teuthology.orchestra.run.smithi064.stdout: "ino-": 20067, 2023-07-18T03:42:27.102 INFO:teuthology.orchestra.run.smithi064.stdout: "dir": 33, 2023-07-18T03:42:27.103 INFO:teuthology.orchestra.run.smithi064.stdout: "dir+": 884, 2023-07-18T03:42:27.103 INFO:teuthology.orchestra.run.smithi064.stdout: "dir-": 851, 2023-07-18T03:42:27.103 INFO:teuthology.orchestra.run.smithi064.stdout: "dn": 33, 2023-07-18T03:42:27.103 INFO:teuthology.orchestra.run.smithi064.stdout: "dn+": 44449, 2023-07-18T03:42:27.103 INFO:teuthology.orchestra.run.smithi064.stdout: "dn-": 44416, 2023-07-18T03:42:27.103 INFO:teuthology.orchestra.run.smithi064.stdout: "cap": 1, 2023-07-18T03:42:27.103 INFO:teuthology.orchestra.run.smithi064.stdout: "cap+": 21670, 2023-07-18T03:42:27.103 INFO:teuthology.orchestra.run.smithi064.stdout: "cap-": 21669, 2023-07-18T03:42:27.104 INFO:teuthology.orchestra.run.smithi064.stdout: "rss": 112800, 2023-07-18T03:42:27.104 INFO:teuthology.orchestra.run.smithi064.stdout: "heap": 198964 2023-07-18T03:42:27.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.104 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_server": { 2023-07-18T03:42:27.104 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_client_request": 81456, 2023-07-18T03:42:27.104 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_server_request": 1134, 2023-07-18T03:42:27.104 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_request": 66283, 2023-07-18T03:42:27.105 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_session": 198, 2023-07-18T03:42:27.105 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_peer_request": 2794, 2023-07-18T03:42:27.105 INFO:teuthology.orchestra.run.smithi064.stdout: "req_create_latency": { 2023-07-18T03:42:27.105 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 12264, 2023-07-18T03:42:27.105 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 46.320298475, 2023-07-18T03:42:27.105 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.003776932 2023-07-18T03:42:27.105 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.105 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getattr_latency": { 2023-07-18T03:42:27.106 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1648, 2023-07-18T03:42:27.106 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 121.250172403, 2023-07-18T03:42:27.106 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.073574133 2023-07-18T03:42:27.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.106 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getfilelock_latency": { 2023-07-18T03:42:27.106 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.106 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.106 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.107 INFO:teuthology.orchestra.run.smithi064.stdout: "req_link_latency": { 2023-07-18T03:42:27.107 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.107 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.107 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.107 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookup_latency": { 2023-07-18T03:42:27.107 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 28387, 2023-07-18T03:42:27.108 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 1063.463136712, 2023-07-18T03:42:27.108 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.037463033 2023-07-18T03:42:27.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.108 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookuphash_latency": { 2023-07-18T03:42:27.108 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.108 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.108 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.109 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupino_latency": { 2023-07-18T03:42:27.109 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1, 2023-07-18T03:42:27.109 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000165266, 2023-07-18T03:42:27.109 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000165266 2023-07-18T03:42:27.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.109 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupname_latency": { 2023-07-18T03:42:27.109 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupparent_latency": { 2023-07-18T03:42:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupsnap_latency": { 2023-07-18T03:42:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lssnap_latency": { 2023-07-18T03:42:27.111 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.112 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.112 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.112 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mkdir_latency": { 2023-07-18T03:42:27.112 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 72, 2023-07-18T03:42:27.112 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 2.799485562, 2023-07-18T03:42:27.112 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.038881743 2023-07-18T03:42:27.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.113 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mknod_latency": { 2023-07-18T03:42:27.113 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.113 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.113 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.113 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mksnap_latency": { 2023-07-18T03:42:27.113 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.114 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.114 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.114 INFO:teuthology.orchestra.run.smithi064.stdout: "req_open_latency": { 2023-07-18T03:42:27.114 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2799, 2023-07-18T03:42:27.114 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 9.967297139, 2023-07-18T03:42:27.114 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.003561020 2023-07-18T03:42:27.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.115 INFO:teuthology.orchestra.run.smithi064.stdout: "req_readdir_latency": { 2023-07-18T03:42:27.115 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 305, 2023-07-18T03:42:27.115 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.661574470, 2023-07-18T03:42:27.115 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.002169096 2023-07-18T03:42:27.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.115 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rename_latency": { 2023-07-18T03:42:27.115 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 12262, 2023-07-18T03:42:27.115 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 191.023600669, 2023-07-18T03:42:27.116 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.015578502 2023-07-18T03:42:27.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.116 INFO:teuthology.orchestra.run.smithi064.stdout: "req_renamesnap_latency": { 2023-07-18T03:42:27.116 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.116 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.116 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.117 INFO:teuthology.orchestra.run.smithi064.stdout: "req_snapdiff_latency": { 2023-07-18T03:42:27.117 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.117 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.117 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.117 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmdir_latency": { 2023-07-18T03:42:27.117 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 46, 2023-07-18T03:42:27.117 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 8.943669009, 2023-07-18T03:42:27.118 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.194427587 2023-07-18T03:42:27.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.118 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmsnap_latency": { 2023-07-18T03:42:27.118 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.118 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.118 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.118 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmxattr_latency": { 2023-07-18T03:42:27.119 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.119 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.119 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.119 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setattr_latency": { 2023-07-18T03:42:27.119 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.119 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.119 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.120 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setdirlayout_latency": { 2023-07-18T03:42:27.120 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.120 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.120 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.120 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setfilelock_latency": { 2023-07-18T03:42:27.120 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.121 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.121 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.121 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setlayout_latency": { 2023-07-18T03:42:27.121 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.121 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.121 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.122 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setxattr_latency": { 2023-07-18T03:42:27.122 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.122 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.122 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.122 INFO:teuthology.orchestra.run.smithi064.stdout: "req_symlink_latency": { 2023-07-18T03:42:27.122 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.122 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.123 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.123 INFO:teuthology.orchestra.run.smithi064.stdout: "req_unlink_latency": { 2023-07-18T03:42:27.123 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 6628, 2023-07-18T03:42:27.123 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 16.688911835, 2023-07-18T03:42:27.123 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.002517940 2023-07-18T03:42:27.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.123 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_revoke_eviction": 0, 2023-07-18T03:42:27.124 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_acquisition_throttle": 0, 2023-07-18T03:42:27.124 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getvxattr_latency": { 2023-07-18T03:42:27.124 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.124 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.124 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.124 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:27.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.124 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_sessions": { 2023-07-18T03:42:27.125 INFO:teuthology.orchestra.run.smithi064.stdout: "session_count": 1, 2023-07-18T03:42:27.125 INFO:teuthology.orchestra.run.smithi064.stdout: "session_add": 1, 2023-07-18T03:42:27.125 INFO:teuthology.orchestra.run.smithi064.stdout: "session_remove": 0, 2023-07-18T03:42:27.125 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_open": 1, 2023-07-18T03:42:27.125 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_stale": 0, 2023-07-18T03:42:27.125 INFO:teuthology.orchestra.run.smithi064.stdout: "total_load": 3448, 2023-07-18T03:42:27.125 INFO:teuthology.orchestra.run.smithi064.stdout: "average_load": 3448, 2023-07-18T03:42:27.125 INFO:teuthology.orchestra.run.smithi064.stdout: "avg_session_uptime": 691 2023-07-18T03:42:27.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.126 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2023-07-18T03:42:27.126 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:42:27.126 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2023-07-18T03:42:27.126 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:42:27.126 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:42:27.126 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:42:27.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:42:27.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:42:27.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:42:27.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:42:27.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:42:27.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:42:27.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:42:27.127 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:42:27.128 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:42:27.128 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:42:27.128 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2023-07-18T03:42:27.128 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:42:27.128 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2023-07-18T03:42:27.128 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:42:27.128 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:42:27.128 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:42:27.129 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:42:27.129 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:42:27.129 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:42:27.129 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:42:27.129 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2023-07-18T03:42:27.129 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:42:27.129 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:42:27.129 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:42:27.130 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2023-07-18T03:42:27.130 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2023-07-18T03:42:27.130 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2023-07-18T03:42:27.130 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:42:27.130 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:42:27.130 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:42:27.130 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:42:27.131 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 316035, 2023-07-18T03:42:27.131 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 137, 2023-07-18T03:42:27.131 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 616, 2023-07-18T03:42:27.131 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 7, 2023-07-18T03:42:27.131 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2023-07-18T03:42:27.131 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2023-07-18T03:42:27.131 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:42:27.131 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2023-07-18T03:42:27.132 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:42:27.132 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2023-07-18T03:42:27.132 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 7720, 2023-07-18T03:42:27.132 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 162, 2023-07-18T03:42:27.132 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:42:27.132 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:42:27.132 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2023-07-18T03:42:27.132 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2023-07-18T03:42:27.133 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 133454, 2023-07-18T03:42:27.133 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 920, 2023-07-18T03:42:27.133 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2023-07-18T03:42:27.133 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2023-07-18T03:42:27.133 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2023-07-18T03:42:27.133 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2023-07-18T03:42:27.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.133 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter": { 2023-07-18T03:42:27.134 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2023-07-18T03:42:27.134 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2023-07-18T03:42:27.134 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 12445, 2023-07-18T03:42:27.134 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 172546921, 2023-07-18T03:42:27.134 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 31, 2023-07-18T03:42:27.134 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 12445, 2023-07-18T03:42:27.134 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2023-07-18T03:42:27.135 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 12445, 2023-07-18T03:42:27.135 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 98.321004805, 2023-07-18T03:42:27.135 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.007900442 2023-07-18T03:42:27.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.135 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2023-07-18T03:42:27.135 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2023-07-18T03:42:27.135 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 12445, 2023-07-18T03:42:27.135 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 13267 2023-07-18T03:42:27.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.136 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 12445, 2023-07-18T03:42:27.136 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 1117, 2023-07-18T03:42:27.136 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 11328, 2023-07-18T03:42:27.136 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2023-07-18T03:42:27.136 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2023-07-18T03:42:27.136 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 0, 2023-07-18T03:42:27.136 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 10, 2023-07-18T03:42:27.137 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 321, 2023-07-18T03:42:27.137 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 1741, 2023-07-18T03:42:27.137 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 198, 2023-07-18T03:42:27.137 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2023-07-18T03:42:27.137 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2023-07-18T03:42:27.137 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2023-07-18T03:42:27.137 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 63, 2023-07-18T03:42:27.137 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 8635, 2023-07-18T03:42:27.138 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2023-07-18T03:42:27.138 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2023-07-18T03:42:27.138 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2023-07-18T03:42:27.138 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 796, 2023-07-18T03:42:27.138 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 20, 2023-07-18T03:42:27.138 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2023-07-18T03:42:27.138 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2023-07-18T03:42:27.139 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2023-07-18T03:42:27.139 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2023-07-18T03:42:27.139 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2023-07-18T03:42:27.139 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2023-07-18T03:42:27.139 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2023-07-18T03:42:27.139 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2023-07-18T03:42:27.139 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2023-07-18T03:42:27.139 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2023-07-18T03:42:27.140 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2023-07-18T03:42:27.140 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2023-07-18T03:42:27.140 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2023-07-18T03:42:27.140 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2023-07-18T03:42:27.140 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2023-07-18T03:42:27.140 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2023-07-18T03:42:27.140 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2023-07-18T03:42:27.141 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2023-07-18T03:42:27.141 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2023-07-18T03:42:27.141 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2023-07-18T03:42:27.141 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2023-07-18T03:42:27.141 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2023-07-18T03:42:27.141 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2023-07-18T03:42:27.141 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2023-07-18T03:42:27.141 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2023-07-18T03:42:27.142 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2023-07-18T03:42:27.142 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 84, 2023-07-18T03:42:27.142 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2023-07-18T03:42:27.142 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 3, 2023-07-18T03:42:27.142 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 12, 2023-07-18T03:42:27.142 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 12, 2023-07-18T03:42:27.142 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 0, 2023-07-18T03:42:27.142 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2023-07-18T03:42:27.143 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 1336, 2023-07-18T03:42:27.143 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 0, 2023-07-18T03:42:27.143 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 147 2023-07-18T03:42:27.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.143 INFO:teuthology.orchestra.run.smithi064.stdout: "oft": { 2023-07-18T03:42:27.143 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_objs": 1, 2023-07-18T03:42:27.143 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_kv_pairs": 0, 2023-07-18T03:42:27.143 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_updates": 0, 2023-07-18T03:42:27.144 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_removes": 0 2023-07-18T03:42:27.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.144 INFO:teuthology.orchestra.run.smithi064.stdout: "osd-slow-ops": { 2023-07-18T03:42:27.144 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_ops_count": 67172 2023-07-18T03:42:27.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.144 INFO:teuthology.orchestra.run.smithi064.stdout: "purge_queue": { 2023-07-18T03:42:27.144 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops": 0, 2023-07-18T03:42:27.145 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops_high_water": 14, 2023-07-18T03:42:27.145 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing": 0, 2023-07-18T03:42:27.145 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_high_water": 6, 2023-07-18T03:42:27.145 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed": 8281, 2023-07-18T03:42:27.145 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_item_in_journal": 0 2023-07-18T03:42:27.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.145 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:42:27.145 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-07-18T03:42:27.146 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-07-18T03:42:27.146 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-07-18T03:42:27.146 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 343809, 2023-07-18T03:42:27.146 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 58962668, 2023-07-18T03:42:27.146 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:27.146 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 343809, 2023-07-18T03:42:27.146 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-07-18T03:42:27.146 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-07-18T03:42:27.147 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 343809, 2023-07-18T03:42:27.147 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 58962668, 2023-07-18T03:42:27.147 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-07-18T03:42:27.147 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.147 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.147 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.147 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:27.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.148 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes": { 2023-07-18T03:42:27.148 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-07-18T03:42:27.148 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-07-18T03:42:27.148 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-07-18T03:42:27.148 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-07-18T03:42:27.148 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-07-18T03:42:27.148 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:27.148 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:27.149 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 12445, 2023-07-18T03:42:27.149 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 173383606, 2023-07-18T03:42:27.149 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 12445, 2023-07-18T03:42:27.149 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 173383606, 2023-07-18T03:42:27.149 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-07-18T03:42:27.149 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.149 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.150 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.150 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:27.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.150 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops": { 2023-07-18T03:42:27.150 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-07-18T03:42:27.150 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2023-07-18T03:42:27.150 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-07-18T03:42:27.150 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2023-07-18T03:42:27.151 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2023-07-18T03:42:27.151 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:27.151 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:27.151 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 12445, 2023-07-18T03:42:27.151 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 12445, 2023-07-18T03:42:27.151 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 12445, 2023-07-18T03:42:27.151 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 12445, 2023-07-18T03:42:27.151 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-07-18T03:42:27.152 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.152 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.152 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.152 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:27.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.152 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle": { 2023-07-18T03:42:27.152 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-07-18T03:42:27.152 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2023-07-18T03:42:27.153 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-07-18T03:42:27.153 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 8281, 2023-07-18T03:42:27.153 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 836685, 2023-07-18T03:42:27.153 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:27.153 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 8281, 2023-07-18T03:42:27.153 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-07-18T03:42:27.153 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-07-18T03:42:27.153 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 321, 2023-07-18T03:42:27.154 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 836685, 2023-07-18T03:42:27.154 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-07-18T03:42:27.154 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.154 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.154 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.154 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:27.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:27.154 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle-0x5622e47d60a0": { 2023-07-18T03:42:27.155 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-07-18T03:42:27.155 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2023-07-18T03:42:27.155 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-07-18T03:42:27.155 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 69019, 2023-07-18T03:42:27.155 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 160494571, 2023-07-18T03:42:27.155 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:27.155 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 69019, 2023-07-18T03:42:27.155 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-07-18T03:42:27.156 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-07-18T03:42:27.156 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 1420, 2023-07-18T03:42:27.156 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 160494571, 2023-07-18T03:42:27.156 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-07-18T03:42:27.156 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:27.156 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:27.156 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:27.157 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:27.157 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:27.157 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-07-18T03:42:27.500 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:27 smithi064 ceph-mon[98059]: pgmap v641: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-18T03:42:27.502 INFO:tasks.check_counter:Daemon mds.h mds.exported=183 2023-07-18T03:42:27.502 INFO:tasks.check_counter:Daemon mds.h mds.imported=225 2023-07-18T03:42:27.503 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=63 2023-07-18T03:42:27.503 DEBUG:tasks.check_counter:Getting stats from k 2023-07-18T03:42:27.503 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph daemon mds.k perf dump 2023-07-18T03:42:27.813 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:27 smithi134 ceph-mon[98216]: pgmap v641: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-18T03:42:27.826 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:27 smithi040 ceph-mon[96439]: pgmap v641: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-18T03:42:28.061 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.b/config 2023-07-18T03:42:29.316 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2023-07-18T03:42:29.317 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:42:29.317 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 0, 2023-07-18T03:42:29.317 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2, 2023-07-18T03:42:29.317 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 0, 2023-07-18T03:42:29.317 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 0, 2023-07-18T03:42:29.317 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2023-07-18T03:42:29.317 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2023-07-18T03:42:29.318 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.001659379, 2023-07-18T03:42:29.318 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000000000, 2023-07-18T03:42:29.318 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000545590, 2023-07-18T03:42:29.318 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-18T03:42:29.318 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:29.318 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:29.318 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:29.318 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:29.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:29.319 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:29.319 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:29.319 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:29.319 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:29.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:29.319 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-18T03:42:29.320 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2023-07-18T03:42:29.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:29.320 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:42:29.320 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 220, 2023-07-18T03:42:29.320 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 217, 2023-07-18T03:42:29.320 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 471984, 2023-07-18T03:42:29.320 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 123888, 2023-07-18T03:42:29.320 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2023-07-18T03:42:29.321 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2023-07-18T03:42:29.321 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.027691177, 2023-07-18T03:42:29.321 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.007909557, 2023-07-18T03:42:29.321 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.013165464, 2023-07-18T03:42:29.321 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.005850149, 2023-07-18T03:42:29.321 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:29.321 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 216, 2023-07-18T03:42:29.321 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.013609364, 2023-07-18T03:42:29.322 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000063006 2023-07-18T03:42:29.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:29.322 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:29.322 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:29.322 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:29.322 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:29.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:29.323 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 471984, 2023-07-18T03:42:29.323 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 123888 2023-07-18T03:42:29.323 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:29.323 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:42:29.323 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 5, 2023-07-18T03:42:29.323 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 167, 2023-07-18T03:42:29.323 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 3104, 2023-07-18T03:42:29.323 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 105056, 2023-07-18T03:42:29.324 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 3, 2023-07-18T03:42:29.324 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 2, 2023-07-18T03:42:29.324 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.008745114, 2023-07-18T03:42:29.324 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.005672935, 2023-07-18T03:42:29.324 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.001537369, 2023-07-18T03:42:29.324 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-18T03:42:29.324 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:29.324 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 166, 2023-07-18T03:42:29.325 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.014880762, 2023-07-18T03:42:29.325 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000089643 2023-07-18T03:42:29.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:29.325 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:29.325 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:29.325 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:29.325 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:29.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:29.326 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-07-18T03:42:29.326 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 105056 2023-07-18T03:42:29.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:29.326 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2023-07-18T03:42:29.326 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2023-07-18T03:42:29.326 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2023-07-18T03:42:29.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:29.327 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2023-07-18T03:42:29.327 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:42:29.327 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2023-07-18T03:42:29.327 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:42:29.327 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:42:29.327 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:42:29.327 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:42:29.327 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:42:29.328 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:42:29.328 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:42:29.328 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:42:29.328 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:42:29.328 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:42:29.328 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:42:29.328 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:42:29.328 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:42:29.329 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2023-07-18T03:42:29.329 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:42:29.329 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2023-07-18T03:42:29.329 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:42:29.329 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:42:29.329 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:42:29.329 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:42:29.330 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:42:29.330 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:42:29.330 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:42:29.330 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2023-07-18T03:42:29.330 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:42:29.330 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:42:29.330 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:42:29.330 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2023-07-18T03:42:29.331 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2023-07-18T03:42:29.331 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2023-07-18T03:42:29.331 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:42:29.331 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:42:29.331 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:42:29.331 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:42:29.331 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 149998, 2023-07-18T03:42:29.331 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 30, 2023-07-18T03:42:29.332 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2023-07-18T03:42:29.332 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2023-07-18T03:42:29.332 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2023-07-18T03:42:29.332 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2023-07-18T03:42:29.332 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:42:29.332 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2023-07-18T03:42:29.332 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:42:29.332 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2023-07-18T03:42:29.333 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2023-07-18T03:42:29.333 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2023-07-18T03:42:29.333 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:42:29.333 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:42:29.333 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2023-07-18T03:42:29.333 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2023-07-18T03:42:29.333 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2023-07-18T03:42:29.334 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2023-07-18T03:42:29.334 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2023-07-18T03:42:29.334 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2023-07-18T03:42:29.334 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2023-07-18T03:42:29.334 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2023-07-18T03:42:29.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2023-07-18T03:42:29.334 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:42:29.334 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2023-07-18T03:42:29.335 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2023-07-18T03:42:29.335 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2023-07-18T03:42:29.335 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 221, 2023-07-18T03:42:29.335 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 440881, 2023-07-18T03:42:29.335 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:29.335 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 221, 2023-07-18T03:42:29.335 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2023-07-18T03:42:29.335 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2023-07-18T03:42:29.336 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 221, 2023-07-18T03:42:29.336 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 440881, 2023-07-18T03:42:29.336 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2023-07-18T03:42:29.336 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2023-07-18T03:42:29.336 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2023-07-18T03:42:29.336 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2023-07-18T03:42:29.336 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:29.337 INFO:teuthology.orchestra.run.smithi064.stdout: } 2023-07-18T03:42:29.337 INFO:teuthology.orchestra.run.smithi064.stdout:} 2023-07-18T03:42:29.597 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:29 smithi064 ceph-mon[98059]: pgmap v642: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 11 op/s 2023-07-18T03:42:29.711 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-07-18T03:42:29.711 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-07-18T03:42:29.711 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2023-07-18T03:42:29.712 DEBUG:tasks.check_counter:Getting stats from c 2023-07-18T03:42:29.712 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph daemon mds.c perf dump 2023-07-18T03:42:29.786 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:29 smithi134 ceph-mon[98216]: pgmap v642: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 11 op/s 2023-07-18T03:42:29.826 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:29 smithi040 ceph-mon[96439]: pgmap v642: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 11 op/s 2023-07-18T03:42:30.320 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:42:31.701 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2023-07-18T03:42:31.702 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:42:31.702 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 762, 2023-07-18T03:42:31.702 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 725, 2023-07-18T03:42:31.702 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 7494422, 2023-07-18T03:42:31.702 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 252884, 2023-07-18T03:42:31.703 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2023-07-18T03:42:31.703 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 2, 2023-07-18T03:42:31.703 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.063565382, 2023-07-18T03:42:31.703 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.016280409, 2023-07-18T03:42:31.703 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.024396699, 2023-07-18T03:42:31.704 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.019216555, 2023-07-18T03:42:31.704 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:31.704 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 724, 2023-07-18T03:42:31.704 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.023315043, 2023-07-18T03:42:31.704 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000032203 2023-07-18T03:42:31.704 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.704 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:31.705 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.705 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.705 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.705 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.705 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 404176, 2023-07-18T03:42:31.705 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 63792 2023-07-18T03:42:31.705 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.706 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:42:31.706 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 510, 2023-07-18T03:42:31.706 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 593, 2023-07-18T03:42:31.706 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 25241674, 2023-07-18T03:42:31.707 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 383220, 2023-07-18T03:42:31.707 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2023-07-18T03:42:31.708 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2023-07-18T03:42:31.708 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.049725181, 2023-07-18T03:42:31.708 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.011895510, 2023-07-18T03:42:31.708 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.022013845, 2023-07-18T03:42:31.708 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.012746248, 2023-07-18T03:42:31.708 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:31.709 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 591, 2023-07-18T03:42:31.709 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.039261286, 2023-07-18T03:42:31.709 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000066431 2023-07-18T03:42:31.709 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.709 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:31.710 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.710 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.710 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.710 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.710 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-18T03:42:31.710 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 224864 2023-07-18T03:42:31.710 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.711 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:42:31.711 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 291, 2023-07-18T03:42:31.711 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 291, 2023-07-18T03:42:31.711 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 25242370, 2023-07-18T03:42:31.711 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 89721, 2023-07-18T03:42:31.711 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2023-07-18T03:42:31.711 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2023-07-18T03:42:31.711 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.031263676, 2023-07-18T03:42:31.712 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.006390928, 2023-07-18T03:42:31.712 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.015819294, 2023-07-18T03:42:31.712 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.007603016, 2023-07-18T03:42:31.712 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:31.712 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 290, 2023-07-18T03:42:31.712 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.024044157, 2023-07-18T03:42:31.712 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000082910 2023-07-18T03:42:31.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.713 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:31.713 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.713 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.713 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.713 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.714 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-18T03:42:31.714 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 448 2023-07-18T03:42:31.714 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.714 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2023-07-18T03:42:31.714 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2023-07-18T03:42:31.714 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2023-07-18T03:42:31.714 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.715 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2023-07-18T03:42:31.715 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-07-18T03:42:31.715 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-07-18T03:42:31.715 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2335, 2023-07-18T03:42:31.715 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.255791532, 2023-07-18T03:42:31.715 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000109546 2023-07-18T03:42:31.716 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:31.716 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.716 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2023-07-18T03:42:31.716 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-07-18T03:42:31.716 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-07-18T03:42:31.716 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.717 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.717 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.717 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:31.717 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.717 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2023-07-18T03:42:31.717 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2023-07-18T03:42:31.717 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2023-07-18T03:42:31.718 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2023-07-18T03:42:31.718 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.718 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.718 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.718 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.718 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2023-07-18T03:42:31.719 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2023-07-18T03:42:31.719 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2023-07-18T03:42:31.719 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2023-07-18T03:42:31.719 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2023-07-18T03:42:31.719 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2023-07-18T03:42:31.719 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2023-07-18T03:42:31.719 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 557, 2023-07-18T03:42:31.720 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 382, 2023-07-18T03:42:31.720 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 175, 2023-07-18T03:42:31.720 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2023-07-18T03:42:31.720 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 11, 2023-07-18T03:42:31.720 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 4492, 2023-07-18T03:42:31.720 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2023-07-18T03:42:31.720 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2023-07-18T03:42:31.721 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 3, 2023-07-18T03:42:31.721 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2023-07-18T03:42:31.721 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2023-07-18T03:42:31.721 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2023-07-18T03:42:31.722 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2023-07-18T03:42:31.722 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2023-07-18T03:42:31.722 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2023-07-18T03:42:31.722 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2023-07-18T03:42:31.722 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2023-07-18T03:42:31.722 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2023-07-18T03:42:31.722 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2023-07-18T03:42:31.723 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2023-07-18T03:42:31.723 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2023-07-18T03:42:31.723 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2023-07-18T03:42:31.723 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2023-07-18T03:42:31.723 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2023-07-18T03:42:31.723 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2023-07-18T03:42:31.724 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 4, 2023-07-18T03:42:31.724 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 246, 2023-07-18T03:42:31.724 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2023-07-18T03:42:31.724 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2023-07-18T03:42:31.724 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2023-07-18T03:42:31.724 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2023-07-18T03:42:31.725 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2023-07-18T03:42:31.725 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2023-07-18T03:42:31.725 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2023-07-18T03:42:31.725 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2023-07-18T03:42:31.725 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2023-07-18T03:42:31.725 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2023-07-18T03:42:31.726 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2023-07-18T03:42:31.726 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2023-07-18T03:42:31.726 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2023-07-18T03:42:31.726 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-18T03:42:31.726 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-18T03:42:31.726 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2023-07-18T03:42:31.727 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2023-07-18T03:42:31.727 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2023-07-18T03:42:31.727 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2023-07-18T03:42:31.727 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.727 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2023-07-18T03:42:31.727 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2023-07-18T03:42:31.727 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2023-07-18T03:42:31.728 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2023-07-18T03:42:31.728 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2023-07-18T03:42:31.728 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2023-07-18T03:42:31.728 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2023-07-18T03:42:31.728 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2023-07-18T03:42:31.728 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2023-07-18T03:42:31.729 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2023-07-18T03:42:31.729 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2023-07-18T03:42:31.729 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2023-07-18T03:42:31.729 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2023-07-18T03:42:31.729 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2023-07-18T03:42:31.729 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2023-07-18T03:42:31.730 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2023-07-18T03:42:31.730 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2023-07-18T03:42:31.730 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2023-07-18T03:42:31.730 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2023-07-18T03:42:31.730 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2023-07-18T03:42:31.730 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2023-07-18T03:42:31.730 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2023-07-18T03:42:31.731 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2023-07-18T03:42:31.731 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2023-07-18T03:42:31.731 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.731 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2023-07-18T03:42:31.731 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2023-07-18T03:42:31.731 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2023-07-18T03:42:31.731 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2023-07-18T03:42:31.731 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 0, 2023-07-18T03:42:31.732 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2023-07-18T03:42:31.732 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2023-07-18T03:42:31.732 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2023-07-18T03:42:31.732 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2023-07-18T03:42:31.732 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2023-07-18T03:42:31.732 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 2, 2023-07-18T03:42:31.733 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2023-07-18T03:42:31.733 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2023-07-18T03:42:31.733 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 103365854, 2023-07-18T03:42:31.733 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 0, 2023-07-18T03:42:31.733 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 103369260, 2023-07-18T03:42:31.733 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2023-07-18T03:42:31.734 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.734 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.734 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.734 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.734 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 24071 2023-07-18T03:42:31.734 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.734 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2023-07-18T03:42:31.735 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 15, 2023-07-18T03:42:31.735 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 11104, 2023-07-18T03:42:31.735 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 11089, 2023-07-18T03:42:31.735 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 21, 2023-07-18T03:42:31.735 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 628, 2023-07-18T03:42:31.735 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 607, 2023-07-18T03:42:31.735 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 557, 2023-07-18T03:42:31.735 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 20136, 2023-07-18T03:42:31.736 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 19579, 2023-07-18T03:42:31.736 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2023-07-18T03:42:31.736 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2023-07-18T03:42:31.736 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2023-07-18T03:42:31.736 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 61800, 2023-07-18T03:42:31.737 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 207132 2023-07-18T03:42:31.737 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.737 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2023-07-18T03:42:31.737 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2023-07-18T03:42:31.737 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2023-07-18T03:42:31.737 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2023-07-18T03:42:31.738 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2023-07-18T03:42:31.738 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2023-07-18T03:42:31.738 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2023-07-18T03:42:31.738 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.738 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.738 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.739 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.739 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2023-07-18T03:42:31.739 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.739 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.739 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.739 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.739 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2023-07-18T03:42:31.740 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.740 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.740 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.740 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.740 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2023-07-18T03:42:31.740 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.740 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.741 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.741 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.741 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2023-07-18T03:42:31.741 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.742 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.742 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.742 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.742 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2023-07-18T03:42:31.743 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.743 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.743 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.743 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.743 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2023-07-18T03:42:31.743 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.744 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.744 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.744 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.744 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2023-07-18T03:42:31.744 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.744 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.745 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.745 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.745 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2023-07-18T03:42:31.745 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.745 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.745 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.745 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.746 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2023-07-18T03:42:31.746 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.746 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.746 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.746 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.746 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2023-07-18T03:42:31.746 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.747 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.747 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.747 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.747 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2023-07-18T03:42:31.747 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.747 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.748 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.748 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.749 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2023-07-18T03:42:31.749 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.749 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.749 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.749 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.749 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2023-07-18T03:42:31.750 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.750 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.750 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.750 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.750 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2023-07-18T03:42:31.750 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.751 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.751 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.751 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.751 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2023-07-18T03:42:31.751 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.751 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.751 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.752 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2023-07-18T03:42:31.752 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.752 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.752 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.752 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.752 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2023-07-18T03:42:31.753 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.753 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.753 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.753 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.753 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2023-07-18T03:42:31.753 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.754 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.754 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.754 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.754 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2023-07-18T03:42:31.754 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.754 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.755 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.755 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2023-07-18T03:42:31.755 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.755 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.755 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.755 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.756 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2023-07-18T03:42:31.756 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.756 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.756 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.756 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.756 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2023-07-18T03:42:31.756 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.757 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.757 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.757 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.757 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2023-07-18T03:42:31.757 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.757 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.758 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.758 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.758 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2023-07-18T03:42:31.758 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.758 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.758 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.758 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.759 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2023-07-18T03:42:31.759 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.759 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.759 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.759 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.759 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2023-07-18T03:42:31.759 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.760 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.760 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.760 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.760 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2023-07-18T03:42:31.760 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.760 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.761 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.761 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.761 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2023-07-18T03:42:31.761 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.761 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.761 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.761 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.762 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2023-07-18T03:42:31.762 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2023-07-18T03:42:31.762 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2023-07-18T03:42:31.762 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.762 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.762 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.763 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:31.763 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.763 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2023-07-18T03:42:31.763 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2023-07-18T03:42:31.763 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 1, 2023-07-18T03:42:31.763 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 0, 2023-07-18T03:42:31.763 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 0, 2023-07-18T03:42:31.764 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2023-07-18T03:42:31.764 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2023-07-18T03:42:31.764 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2023-07-18T03:42:31.764 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 2115 2023-07-18T03:42:31.764 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.764 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2023-07-18T03:42:31.765 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:42:31.765 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2023-07-18T03:42:31.765 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:42:31.765 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:42:31.765 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:42:31.765 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:42:31.765 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:42:31.766 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:42:31.766 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:42:31.766 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:42:31.766 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:42:31.766 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:42:31.766 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:42:31.766 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:42:31.767 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:42:31.767 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2023-07-18T03:42:31.767 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:42:31.767 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2023-07-18T03:42:31.767 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:42:31.767 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:42:31.767 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:42:31.768 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:42:31.768 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:42:31.768 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:42:31.768 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:42:31.768 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2023-07-18T03:42:31.768 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:42:31.769 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:42:31.769 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:42:31.769 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2023-07-18T03:42:31.769 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2023-07-18T03:42:31.769 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2023-07-18T03:42:31.769 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:42:31.769 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:42:31.770 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:42:31.770 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:42:31.770 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 222377, 2023-07-18T03:42:31.770 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 67, 2023-07-18T03:42:31.770 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2023-07-18T03:42:31.770 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2023-07-18T03:42:31.770 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2023-07-18T03:42:31.771 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2023-07-18T03:42:31.771 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:42:31.771 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2023-07-18T03:42:31.771 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:42:31.771 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2023-07-18T03:42:31.771 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 7720, 2023-07-18T03:42:31.771 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 162, 2023-07-18T03:42:31.772 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:42:31.772 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:42:31.772 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2023-07-18T03:42:31.772 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2023-07-18T03:42:31.772 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 453216, 2023-07-18T03:42:31.772 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 2523, 2023-07-18T03:42:31.772 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2023-07-18T03:42:31.773 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2023-07-18T03:42:31.773 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2023-07-18T03:42:31.773 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2023-07-18T03:42:31.773 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.773 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2023-07-18T03:42:31.773 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2023-07-18T03:42:31.774 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2023-07-18T03:42:31.774 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 1404, 2023-07-18T03:42:31.774 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 26, 2023-07-18T03:42:31.774 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 4, 2023-07-18T03:42:31.774 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 1404, 2023-07-18T03:42:31.774 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2023-07-18T03:42:31.774 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1404, 2023-07-18T03:42:31.775 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 3.542980114, 2023-07-18T03:42:31.775 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002523490 2023-07-18T03:42:31.775 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.775 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2023-07-18T03:42:31.775 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2023-07-18T03:42:31.775 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1404, 2023-07-18T03:42:31.776 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1405 2023-07-18T03:42:31.776 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.776 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 1404, 2023-07-18T03:42:31.776 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 1404, 2023-07-18T03:42:31.776 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2023-07-18T03:42:31.776 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2023-07-18T03:42:31.776 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2023-07-18T03:42:31.777 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 809, 2023-07-18T03:42:31.777 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2023-07-18T03:42:31.777 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 592, 2023-07-18T03:42:31.777 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2023-07-18T03:42:31.777 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2023-07-18T03:42:31.777 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2023-07-18T03:42:31.777 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2023-07-18T03:42:31.778 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2023-07-18T03:42:31.778 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2023-07-18T03:42:31.778 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2023-07-18T03:42:31.778 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2023-07-18T03:42:31.778 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2023-07-18T03:42:31.778 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2023-07-18T03:42:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 2, 2023-07-18T03:42:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2023-07-18T03:42:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2023-07-18T03:42:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2023-07-18T03:42:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2023-07-18T03:42:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2023-07-18T03:42:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2023-07-18T03:42:31.779 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2023-07-18T03:42:31.780 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2023-07-18T03:42:31.780 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2023-07-18T03:42:31.780 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2023-07-18T03:42:31.780 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2023-07-18T03:42:31.780 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2023-07-18T03:42:31.780 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2023-07-18T03:42:31.780 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2023-07-18T03:42:31.781 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2023-07-18T03:42:31.781 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2023-07-18T03:42:31.781 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2023-07-18T03:42:31.781 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2023-07-18T03:42:31.781 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2023-07-18T03:42:31.781 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2023-07-18T03:42:31.782 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2023-07-18T03:42:31.782 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2023-07-18T03:42:31.782 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2023-07-18T03:42:31.782 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2023-07-18T03:42:31.782 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2023-07-18T03:42:31.782 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2023-07-18T03:42:31.782 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2023-07-18T03:42:31.782 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 0, 2023-07-18T03:42:31.783 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2023-07-18T03:42:31.783 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 0, 2023-07-18T03:42:31.783 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 9, 2023-07-18T03:42:31.783 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 9, 2023-07-18T03:42:31.783 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2023-07-18T03:42:31.783 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2023-07-18T03:42:31.784 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2023-07-18T03:42:31.784 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2023-07-18T03:42:31.784 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2023-07-18T03:42:31.784 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.784 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2023-07-18T03:42:31.784 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2023-07-18T03:42:31.784 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2023-07-18T03:42:31.785 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2023-07-18T03:42:31.785 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2023-07-18T03:42:31.785 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.785 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2023-07-18T03:42:31.785 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2023-07-18T03:42:31.785 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.786 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2023-07-18T03:42:31.786 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2023-07-18T03:42:31.786 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2023-07-18T03:42:31.786 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2023-07-18T03:42:31.786 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2023-07-18T03:42:31.786 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2023-07-18T03:42:31.786 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2023-07-18T03:42:31.786 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.787 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:42:31.787 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:31.787 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-07-18T03:42:31.787 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:31.787 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 1559, 2023-07-18T03:42:31.787 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 57827467, 2023-07-18T03:42:31.787 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:31.788 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 1559, 2023-07-18T03:42:31.788 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-18T03:42:31.788 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-18T03:42:31.788 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1559, 2023-07-18T03:42:31.788 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 57827467, 2023-07-18T03:42:31.788 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:31.789 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.789 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.789 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.789 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:31.789 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.789 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2023-07-18T03:42:31.789 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:31.790 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-07-18T03:42:31.790 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:31.790 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-18T03:42:31.790 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-18T03:42:31.790 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:31.790 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:31.790 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 1404, 2023-07-18T03:42:31.791 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 57180500, 2023-07-18T03:42:31.791 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1404, 2023-07-18T03:42:31.791 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 57180500, 2023-07-18T03:42:31.791 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:31.791 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.791 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.791 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.792 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:31.792 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.792 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2023-07-18T03:42:31.792 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:31.792 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2023-07-18T03:42:31.792 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:31.793 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-18T03:42:31.793 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-18T03:42:31.793 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:31.793 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:31.793 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 1404, 2023-07-18T03:42:31.793 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 1404, 2023-07-18T03:42:31.793 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1404, 2023-07-18T03:42:31.793 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1404, 2023-07-18T03:42:31.794 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:31.794 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.794 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.794 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.794 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:31.794 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.795 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2023-07-18T03:42:31.795 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:31.795 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-07-18T03:42:31.795 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:31.795 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-18T03:42:31.795 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-18T03:42:31.795 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:31.795 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:31.796 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-18T03:42:31.796 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-18T03:42:31.796 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2023-07-18T03:42:31.796 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2023-07-18T03:42:31.796 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:31.796 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55e524090460": { 2023-07-18T03:42:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:31.797 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-07-18T03:42:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-18T03:42:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-18T03:42:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-18T03:42:31.798 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-18T03:42:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2023-07-18T03:42:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2023-07-18T03:42:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:31.799 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:31.800 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:31.800 INFO:teuthology.orchestra.run.smithi134.stdout:} 2023-07-18T03:42:31.800 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:31 smithi134 ceph-mon[98216]: pgmap v643: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 8 op/s 2023-07-18T03:42:31.826 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:31 smithi040 ceph-mon[96439]: pgmap v643: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 8 op/s 2023-07-18T03:42:31.848 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:31 smithi064 ceph-mon[98059]: pgmap v643: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 8 op/s 2023-07-18T03:42:32.065 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-07-18T03:42:32.066 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-07-18T03:42:32.066 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2023-07-18T03:42:32.067 DEBUG:tasks.check_counter:Getting stats from f 2023-07-18T03:42:32.067 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph daemon mds.f perf dump 2023-07-18T03:42:32.619 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:42:33.063 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:32 smithi134 ceph-mon[98216]: pgmap v644: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 12 op/s 2023-07-18T03:42:33.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:32 smithi040 ceph-mon[96439]: pgmap v644: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 12 op/s 2023-07-18T03:42:33.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:32 smithi064 ceph-mon[98059]: pgmap v644: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 12 op/s 2023-07-18T03:42:35.597 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:35 smithi064 ceph-mon[98059]: pgmap v645: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2023-07-18T03:42:35.715 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:35 smithi134 ceph-mon[98216]: pgmap v645: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2023-07-18T03:42:35.826 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:35 smithi040 ceph-mon[96439]: pgmap v645: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2023-07-18T03:42:37.693 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:37 smithi134 ceph-mon[98216]: pgmap v646: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2023-07-18T03:42:37.826 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:37 smithi040 ceph-mon[96439]: pgmap v646: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2023-07-18T03:42:37.847 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:37 smithi064 ceph-mon[98059]: pgmap v646: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2023-07-18T03:42:37.914 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2023-07-18T03:42:37.914 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:42:37.914 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1416, 2023-07-18T03:42:37.914 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1356, 2023-07-18T03:42:37.914 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 97636574, 2023-07-18T03:42:37.915 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 476815, 2023-07-18T03:42:37.915 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 7, 2023-07-18T03:42:37.915 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2023-07-18T03:42:37.915 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.170262902, 2023-07-18T03:42:37.915 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.033320007, 2023-07-18T03:42:37.916 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.089959746, 2023-07-18T03:42:37.916 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.040316896, 2023-07-18T03:42:37.916 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:37.916 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1356, 2023-07-18T03:42:37.916 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.054518489, 2023-07-18T03:42:37.916 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000040205 2023-07-18T03:42:37.916 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.917 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:37.917 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.917 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.917 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.917 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.917 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 809280, 2023-07-18T03:42:37.918 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 125696 2023-07-18T03:42:37.918 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.918 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:42:37.918 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 554, 2023-07-18T03:42:37.918 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 719, 2023-07-18T03:42:37.918 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 109212501, 2023-07-18T03:42:37.918 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 530845, 2023-07-18T03:42:37.919 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 7, 2023-07-18T03:42:37.919 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 6, 2023-07-18T03:42:37.919 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.101257015, 2023-07-18T03:42:37.919 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.015251131, 2023-07-18T03:42:37.920 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.066697795, 2023-07-18T03:42:37.920 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.014302006, 2023-07-18T03:42:37.920 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:37.920 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 717, 2023-07-18T03:42:37.920 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.023602949, 2023-07-18T03:42:37.921 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000032919 2023-07-18T03:42:37.921 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.921 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:37.921 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.921 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.921 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.922 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.922 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-18T03:42:37.922 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 358800 2023-07-18T03:42:37.922 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.922 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:42:37.922 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1293, 2023-07-18T03:42:37.923 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1295, 2023-07-18T03:42:37.923 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 190958832, 2023-07-18T03:42:37.923 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 402130, 2023-07-18T03:42:37.923 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2023-07-18T03:42:37.923 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2023-07-18T03:42:37.923 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.176511917, 2023-07-18T03:42:37.924 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.025657253, 2023-07-18T03:42:37.924 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.111023514, 2023-07-18T03:42:37.924 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.032271917, 2023-07-18T03:42:37.924 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:37.924 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1293, 2023-07-18T03:42:37.924 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.040887611, 2023-07-18T03:42:37.925 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000031622 2023-07-18T03:42:37.925 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.925 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:37.925 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.925 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.925 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.926 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-18T03:42:37.926 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2023-07-18T03:42:37.926 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.926 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2023-07-18T03:42:37.927 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2023-07-18T03:42:37.927 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2023-07-18T03:42:37.927 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.927 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2023-07-18T03:42:37.928 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-07-18T03:42:37.928 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-07-18T03:42:37.928 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 4905, 2023-07-18T03:42:37.928 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.191039452, 2023-07-18T03:42:37.928 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000038947 2023-07-18T03:42:37.929 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:37.929 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.929 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2023-07-18T03:42:37.929 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-07-18T03:42:37.929 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-07-18T03:42:37.930 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.930 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.930 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.930 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:37.930 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.930 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2023-07-18T03:42:37.930 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2023-07-18T03:42:37.931 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2023-07-18T03:42:37.931 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2023-07-18T03:42:37.931 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.931 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.931 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.931 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.932 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2023-07-18T03:42:37.932 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2023-07-18T03:42:37.932 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2023-07-18T03:42:37.932 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2023-07-18T03:42:37.932 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2023-07-18T03:42:37.932 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2023-07-18T03:42:37.932 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2023-07-18T03:42:37.933 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 28042, 2023-07-18T03:42:37.933 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2023-07-18T03:42:37.933 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 27176, 2023-07-18T03:42:37.933 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 866, 2023-07-18T03:42:37.933 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 28041, 2023-07-18T03:42:37.934 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 4487, 2023-07-18T03:42:37.934 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2023-07-18T03:42:37.934 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2023-07-18T03:42:37.934 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 3, 2023-07-18T03:42:37.934 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2023-07-18T03:42:37.934 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2023-07-18T03:42:37.934 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2023-07-18T03:42:37.935 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2023-07-18T03:42:37.935 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2023-07-18T03:42:37.935 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2023-07-18T03:42:37.935 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2023-07-18T03:42:37.935 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2023-07-18T03:42:37.935 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2023-07-18T03:42:37.936 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2023-07-18T03:42:37.936 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2023-07-18T03:42:37.936 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2023-07-18T03:42:37.936 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2023-07-18T03:42:37.936 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2023-07-18T03:42:37.936 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2023-07-18T03:42:37.936 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2023-07-18T03:42:37.937 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 0, 2023-07-18T03:42:37.937 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 0, 2023-07-18T03:42:37.937 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2023-07-18T03:42:37.937 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2023-07-18T03:42:37.937 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2023-07-18T03:42:37.937 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2023-07-18T03:42:37.937 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2023-07-18T03:42:37.938 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2023-07-18T03:42:37.938 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2023-07-18T03:42:37.938 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2023-07-18T03:42:37.938 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2023-07-18T03:42:37.938 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2023-07-18T03:42:37.938 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2023-07-18T03:42:37.938 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2023-07-18T03:42:37.939 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2023-07-18T03:42:37.939 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-18T03:42:37.939 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-18T03:42:37.939 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2023-07-18T03:42:37.939 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2023-07-18T03:42:37.939 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2023-07-18T03:42:37.940 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2023-07-18T03:42:37.940 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.940 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2023-07-18T03:42:37.940 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2023-07-18T03:42:37.940 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2023-07-18T03:42:37.940 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2023-07-18T03:42:37.940 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2023-07-18T03:42:37.941 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2023-07-18T03:42:37.941 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2023-07-18T03:42:37.941 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2023-07-18T03:42:37.941 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2023-07-18T03:42:37.941 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2023-07-18T03:42:37.941 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2023-07-18T03:42:37.941 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2023-07-18T03:42:37.942 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2023-07-18T03:42:37.942 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2023-07-18T03:42:37.942 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2023-07-18T03:42:37.942 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2023-07-18T03:42:37.942 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2023-07-18T03:42:37.942 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2023-07-18T03:42:37.943 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2023-07-18T03:42:37.943 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2023-07-18T03:42:37.943 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2023-07-18T03:42:37.943 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2023-07-18T03:42:37.943 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2023-07-18T03:42:37.943 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2023-07-18T03:42:37.944 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.944 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2023-07-18T03:42:37.944 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2023-07-18T03:42:37.944 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2023-07-18T03:42:37.944 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2023-07-18T03:42:37.944 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 0, 2023-07-18T03:42:37.945 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2023-07-18T03:42:37.945 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2023-07-18T03:42:37.945 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2023-07-18T03:42:37.945 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2023-07-18T03:42:37.945 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2023-07-18T03:42:37.945 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 132, 2023-07-18T03:42:37.945 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2023-07-18T03:42:37.946 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2023-07-18T03:42:37.946 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 192941199, 2023-07-18T03:42:37.946 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 0, 2023-07-18T03:42:37.946 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 400322274, 2023-07-18T03:42:37.946 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2023-07-18T03:42:37.946 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.946 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.946 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.947 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 175154 2023-07-18T03:42:37.947 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.947 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2023-07-18T03:42:37.947 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 19, 2023-07-18T03:42:37.947 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 46779, 2023-07-18T03:42:37.948 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 46760, 2023-07-18T03:42:37.948 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 24, 2023-07-18T03:42:37.948 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 2116, 2023-07-18T03:42:37.948 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 2092, 2023-07-18T03:42:37.948 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 28042, 2023-07-18T03:42:37.948 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 114462, 2023-07-18T03:42:37.948 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 86420, 2023-07-18T03:42:37.949 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2023-07-18T03:42:37.949 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2023-07-18T03:42:37.949 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2023-07-18T03:42:37.949 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 93588, 2023-07-18T03:42:37.949 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 207156 2023-07-18T03:42:37.949 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.950 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2023-07-18T03:42:37.950 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2023-07-18T03:42:37.950 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2023-07-18T03:42:37.950 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2023-07-18T03:42:37.950 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2023-07-18T03:42:37.950 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2023-07-18T03:42:37.951 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2023-07-18T03:42:37.951 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.951 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.951 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.951 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.951 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2023-07-18T03:42:37.952 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.952 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.952 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.952 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.952 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2023-07-18T03:42:37.952 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.953 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.953 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.953 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.953 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2023-07-18T03:42:37.953 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.953 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.954 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.954 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.954 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2023-07-18T03:42:37.954 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.954 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.954 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.955 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.955 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2023-07-18T03:42:37.955 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.955 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.955 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.956 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.956 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2023-07-18T03:42:37.956 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.956 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.956 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.957 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2023-07-18T03:42:37.957 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.957 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.957 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.958 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.958 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2023-07-18T03:42:37.958 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.958 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.958 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.959 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2023-07-18T03:42:37.959 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.959 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.959 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.960 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2023-07-18T03:42:37.960 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.960 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.960 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.960 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2023-07-18T03:42:37.961 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.961 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.961 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.961 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2023-07-18T03:42:37.961 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.962 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.962 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.962 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2023-07-18T03:42:37.962 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.962 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.962 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.963 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2023-07-18T03:42:37.963 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.963 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.963 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.963 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.963 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2023-07-18T03:42:37.964 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.964 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.964 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.964 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.965 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2023-07-18T03:42:37.965 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.965 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.965 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.965 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.965 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2023-07-18T03:42:37.965 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.966 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.966 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.966 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.966 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2023-07-18T03:42:37.966 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.966 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.966 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.967 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.967 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2023-07-18T03:42:37.967 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.967 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.967 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.967 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.967 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2023-07-18T03:42:37.968 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.968 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.968 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.968 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.968 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2023-07-18T03:42:37.968 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.968 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.969 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.969 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2023-07-18T03:42:37.969 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.969 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.969 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.970 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2023-07-18T03:42:37.970 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.970 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.970 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.970 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.970 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2023-07-18T03:42:37.970 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.970 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.971 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.971 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.971 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2023-07-18T03:42:37.971 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.971 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.971 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.971 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.972 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2023-07-18T03:42:37.972 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.972 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.972 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.972 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2023-07-18T03:42:37.973 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.973 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.973 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.973 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2023-07-18T03:42:37.973 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.973 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.974 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.974 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2023-07-18T03:42:37.974 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2023-07-18T03:42:37.974 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2023-07-18T03:42:37.975 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:37.975 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:37.975 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:37.975 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:37.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.976 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2023-07-18T03:42:37.976 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2023-07-18T03:42:37.976 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 2, 2023-07-18T03:42:37.976 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 1, 2023-07-18T03:42:37.976 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 2, 2023-07-18T03:42:37.976 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2023-07-18T03:42:37.977 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2023-07-18T03:42:37.977 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2023-07-18T03:42:37.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 3467 2023-07-18T03:42:37.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.977 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2023-07-18T03:42:37.977 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:42:37.978 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2023-07-18T03:42:37.978 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:42:37.978 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:42:37.978 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:42:37.978 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:42:37.978 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:42:37.979 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:42:37.979 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:42:37.979 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:42:37.979 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:42:37.980 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:42:37.980 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:42:37.980 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:42:37.980 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:42:37.980 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2023-07-18T03:42:37.981 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:42:37.981 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2023-07-18T03:42:37.981 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:42:37.981 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:42:37.981 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:42:37.982 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:42:37.982 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:42:37.982 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:42:37.982 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:42:37.982 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2023-07-18T03:42:37.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:42:37.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:42:37.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:42:37.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2023-07-18T03:42:37.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2023-07-18T03:42:37.984 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2023-07-18T03:42:37.984 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:42:37.984 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:42:37.984 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:42:37.984 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:42:37.984 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 238681, 2023-07-18T03:42:37.985 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 77, 2023-07-18T03:42:37.985 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2023-07-18T03:42:37.985 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2023-07-18T03:42:37.985 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2023-07-18T03:42:37.985 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2023-07-18T03:42:37.986 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:42:37.986 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2023-07-18T03:42:37.986 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:42:37.986 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2023-07-18T03:42:37.986 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 7720, 2023-07-18T03:42:37.986 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 162, 2023-07-18T03:42:37.986 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:42:37.987 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:42:37.987 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2023-07-18T03:42:37.987 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2023-07-18T03:42:37.987 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 19814845, 2023-07-18T03:42:37.987 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 112515, 2023-07-18T03:42:37.987 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2023-07-18T03:42:37.987 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2023-07-18T03:42:37.988 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2023-07-18T03:42:37.988 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2023-07-18T03:42:37.988 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.988 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2023-07-18T03:42:37.988 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2023-07-18T03:42:37.988 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2023-07-18T03:42:37.988 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 2975, 2023-07-18T03:42:37.989 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 21, 2023-07-18T03:42:37.989 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 3, 2023-07-18T03:42:37.989 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 2975, 2023-07-18T03:42:37.989 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2023-07-18T03:42:37.989 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2975, 2023-07-18T03:42:37.989 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 8.028953736, 2023-07-18T03:42:37.989 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002698807 2023-07-18T03:42:37.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.990 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2023-07-18T03:42:37.990 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2023-07-18T03:42:37.990 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2975, 2023-07-18T03:42:37.990 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2976 2023-07-18T03:42:37.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.990 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 2975, 2023-07-18T03:42:37.990 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 2975, 2023-07-18T03:42:37.991 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2023-07-18T03:42:37.991 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2023-07-18T03:42:37.991 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2023-07-18T03:42:37.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 1665, 2023-07-18T03:42:37.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2023-07-18T03:42:37.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 1308, 2023-07-18T03:42:37.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2023-07-18T03:42:37.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2023-07-18T03:42:37.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2023-07-18T03:42:37.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2023-07-18T03:42:37.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2023-07-18T03:42:37.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2023-07-18T03:42:37.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2023-07-18T03:42:37.992 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2023-07-18T03:42:37.993 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2023-07-18T03:42:37.993 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2023-07-18T03:42:37.993 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 1, 2023-07-18T03:42:37.993 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2023-07-18T03:42:37.993 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2023-07-18T03:42:37.993 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2023-07-18T03:42:37.993 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2023-07-18T03:42:37.994 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2023-07-18T03:42:37.994 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2023-07-18T03:42:37.994 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2023-07-18T03:42:37.994 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2023-07-18T03:42:37.994 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2023-07-18T03:42:37.994 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2023-07-18T03:42:37.994 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2023-07-18T03:42:37.995 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2023-07-18T03:42:37.995 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2023-07-18T03:42:37.995 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2023-07-18T03:42:37.995 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2023-07-18T03:42:37.995 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2023-07-18T03:42:37.995 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2023-07-18T03:42:37.995 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2023-07-18T03:42:37.995 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2023-07-18T03:42:37.996 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2023-07-18T03:42:37.996 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2023-07-18T03:42:37.996 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2023-07-18T03:42:37.996 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2023-07-18T03:42:37.996 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2023-07-18T03:42:37.996 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2023-07-18T03:42:37.996 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2023-07-18T03:42:37.997 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2023-07-18T03:42:37.997 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 84, 2023-07-18T03:42:37.997 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2023-07-18T03:42:37.997 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 3, 2023-07-18T03:42:37.997 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 11, 2023-07-18T03:42:37.997 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 11, 2023-07-18T03:42:37.997 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2023-07-18T03:42:37.998 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2023-07-18T03:42:37.998 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2023-07-18T03:42:37.998 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2023-07-18T03:42:37.998 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2023-07-18T03:42:37.998 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:37.999 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2023-07-18T03:42:37.999 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2023-07-18T03:42:37.999 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2023-07-18T03:42:37.999 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2023-07-18T03:42:37.999 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2023-07-18T03:42:37.999 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:38.000 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2023-07-18T03:42:38.000 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2023-07-18T03:42:38.000 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:38.000 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2023-07-18T03:42:38.000 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2023-07-18T03:42:38.000 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2023-07-18T03:42:38.001 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2023-07-18T03:42:38.001 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2023-07-18T03:42:38.001 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2023-07-18T03:42:38.001 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2023-07-18T03:42:38.001 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:38.001 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:42:38.001 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:38.002 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-07-18T03:42:38.002 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:38.002 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 3259, 2023-07-18T03:42:38.002 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 397497332, 2023-07-18T03:42:38.002 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:38.002 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 3259, 2023-07-18T03:42:38.002 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-18T03:42:38.003 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-18T03:42:38.003 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 3259, 2023-07-18T03:42:38.003 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 397497332, 2023-07-18T03:42:38.003 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:38.003 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:38.003 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:38.003 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:38.004 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:38.004 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:38.004 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2023-07-18T03:42:38.004 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:38.004 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-07-18T03:42:38.004 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:38.004 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-18T03:42:38.005 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-18T03:42:38.005 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:38.005 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:38.005 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2975, 2023-07-18T03:42:38.005 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 396177149, 2023-07-18T03:42:38.005 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2975, 2023-07-18T03:42:38.005 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 396177149, 2023-07-18T03:42:38.006 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:38.006 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:38.006 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:38.006 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:38.006 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:38.006 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:38.006 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2023-07-18T03:42:38.007 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:38.007 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2023-07-18T03:42:38.007 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:38.007 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-18T03:42:38.007 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-18T03:42:38.008 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:38.008 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:38.008 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2975, 2023-07-18T03:42:38.008 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 2975, 2023-07-18T03:42:38.008 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2975, 2023-07-18T03:42:38.009 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2975, 2023-07-18T03:42:38.009 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:38.009 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:38.009 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:38.009 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:38.009 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:38.010 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:38.010 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2023-07-18T03:42:38.010 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:38.010 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-07-18T03:42:38.010 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:38.011 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-18T03:42:38.011 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-18T03:42:38.011 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:38.011 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:38.011 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-18T03:42:38.012 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-18T03:42:38.012 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2023-07-18T03:42:38.012 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2023-07-18T03:42:38.012 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:38.012 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:38.013 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:38.013 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:38.013 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:38.013 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:38.013 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55fd02380460": { 2023-07-18T03:42:38.013 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:38.013 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-07-18T03:42:38.014 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:38.014 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-18T03:42:38.014 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-18T03:42:38.014 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:38.014 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:38.014 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-18T03:42:38.014 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-18T03:42:38.015 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2023-07-18T03:42:38.015 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2023-07-18T03:42:38.015 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:38.015 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:38.015 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:38.015 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:38.015 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:38.016 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:38.016 INFO:teuthology.orchestra.run.smithi134.stdout:} 2023-07-18T03:42:38.295 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-07-18T03:42:38.295 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-07-18T03:42:38.296 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2023-07-18T03:42:38.296 DEBUG:tasks.check_counter:Getting stats from i 2023-07-18T03:42:38.296 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph daemon mds.i perf dump 2023-07-18T03:42:38.852 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:42:39.814 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:39 smithi134 ceph-mon[98216]: pgmap v647: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2023-07-18T03:42:39.826 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:39 smithi040 ceph-mon[96439]: pgmap v647: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2023-07-18T03:42:39.847 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:39 smithi064 ceph-mon[98059]: pgmap v647: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2023-07-18T03:42:40.062 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2023-07-18T03:42:40.062 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:42:40.063 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 925713, 2023-07-18T03:42:40.063 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 922396, 2023-07-18T03:42:40.063 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 152234032, 2023-07-18T03:42:40.064 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 359675361, 2023-07-18T03:42:40.064 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 17, 2023-07-18T03:42:40.064 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2023-07-18T03:42:40.064 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 30.105562495, 2023-07-18T03:42:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 11.987057878, 2023-07-18T03:42:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 20.106958991, 2023-07-18T03:42:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.584734138, 2023-07-18T03:42:40.065 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 922392, 2023-07-18T03:42:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 16.680135515, 2023-07-18T03:42:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000018083 2023-07-18T03:42:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1811174, 2023-07-18T03:42:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.217332474, 2023-07-18T03:42:40.066 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000119 2023-07-18T03:42:40.067 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.067 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1872, 2023-07-18T03:42:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 1536 2023-07-18T03:42:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:42:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1924988, 2023-07-18T03:42:40.068 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1932090, 2023-07-18T03:42:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 330110919, 2023-07-18T03:42:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 413762909, 2023-07-18T03:42:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 8, 2023-07-18T03:42:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2023-07-18T03:42:40.069 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 66.655713099, 2023-07-18T03:42:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 28.197400309, 2023-07-18T03:42:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 46.197643518, 2023-07-18T03:42:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.387502387, 2023-07-18T03:42:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1932093, 2023-07-18T03:42:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 30.321289075, 2023-07-18T03:42:40.070 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000015693 2023-07-18T03:42:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3817611, 2023-07-18T03:42:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.473521803, 2023-07-18T03:42:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000124 2023-07-18T03:42:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.071 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 813232, 2023-07-18T03:42:40.072 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 203488 2023-07-18T03:42:40.072 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.072 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:42:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 232300, 2023-07-18T03:42:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 418738, 2023-07-18T03:42:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 71685903, 2023-07-18T03:42:40.073 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 319620483, 2023-07-18T03:42:40.074 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 18, 2023-07-18T03:42:40.074 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 10, 2023-07-18T03:42:40.074 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 9.958562216, 2023-07-18T03:42:40.074 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 5.284691340, 2023-07-18T03:42:40.075 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 7.828396306, 2023-07-18T03:42:40.075 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.490888245, 2023-07-18T03:42:40.075 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:40.076 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 418735, 2023-07-18T03:42:40.076 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 22.078421998, 2023-07-18T03:42:40.076 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000052726 2023-07-18T03:42:40.076 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 274303, 2023-07-18T03:42:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.057773296, 2023-07-18T03:42:40.077 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000210 2023-07-18T03:42:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-07-18T03:42:40.078 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 364304 2023-07-18T03:42:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2023-07-18T03:42:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2023-07-18T03:42:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2023-07-18T03:42:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2023-07-18T03:42:40.079 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-07-18T03:42:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-07-18T03:42:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 7856, 2023-07-18T03:42:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 99.375346963, 2023-07-18T03:42:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.012649611 2023-07-18T03:42:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.080 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2023-07-18T03:42:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-07-18T03:42:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-07-18T03:42:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 53644, 2023-07-18T03:42:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 12.725994813, 2023-07-18T03:42:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000237230 2023-07-18T03:42:40.081 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2023-07-18T03:42:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 154494, 2023-07-18T03:42:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 141376, 2023-07-18T03:42:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2023-07-18T03:42:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 141376, 2023-07-18T03:42:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1819.935401867, 2023-07-18T03:42:40.082 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.012873015 2023-07-18T03:42:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2023-07-18T03:42:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 13112, 2023-07-18T03:42:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 44, 2023-07-18T03:42:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2023-07-18T03:42:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 1184, 2023-07-18T03:42:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 118, 2023-07-18T03:42:40.083 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 2, 2023-07-18T03:42:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 95, 2023-07-18T03:42:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 24, 2023-07-18T03:42:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 60, 2023-07-18T03:42:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 11, 2023-07-18T03:42:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 16, 2023-07-18T03:42:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 43612, 2023-07-18T03:42:40.084 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 2, 2023-07-18T03:42:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 2, 2023-07-18T03:42:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 7, 2023-07-18T03:42:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 233666, 2023-07-18T03:42:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 204064, 2023-07-18T03:42:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 13042, 2023-07-18T03:42:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 404, 2023-07-18T03:42:40.085 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2023-07-18T03:42:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2023-07-18T03:42:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 181, 2023-07-18T03:42:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 5, 2023-07-18T03:42:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2023-07-18T03:42:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 371, 2023-07-18T03:42:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 30239, 2023-07-18T03:42:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 211, 2023-07-18T03:42:40.086 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 32651, 2023-07-18T03:42:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2023-07-18T03:42:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 1913, 2023-07-18T03:42:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2023-07-18T03:42:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 3846, 2023-07-18T03:42:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 72892698, 2023-07-18T03:42:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2023-07-18T03:42:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 1913, 2023-07-18T03:42:40.087 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2023-07-18T03:42:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2023-07-18T03:42:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2023-07-18T03:42:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 34, 2023-07-18T03:42:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2023-07-18T03:42:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 34, 2023-07-18T03:42:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 1879, 2023-07-18T03:42:40.088 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 152, 2023-07-18T03:42:40.089 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 29151, 2023-07-18T03:42:40.089 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 61220, 2023-07-18T03:42:40.089 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2023-07-18T03:42:40.089 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-18T03:42:40.089 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-18T03:42:40.089 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 44634, 2023-07-18T03:42:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 27703, 2023-07-18T03:42:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 610, 2023-07-18T03:42:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 62105 2023-07-18T03:42:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2023-07-18T03:42:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 3, 2023-07-18T03:42:40.090 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2023-07-18T03:42:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2023-07-18T03:42:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 412, 2023-07-18T03:42:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 818, 2023-07-18T03:42:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 1, 2023-07-18T03:42:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2023-07-18T03:42:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2023-07-18T03:42:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 29614, 2023-07-18T03:42:40.091 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 29613, 2023-07-18T03:42:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2023-07-18T03:42:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2023-07-18T03:42:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2023-07-18T03:42:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2023-07-18T03:42:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2023-07-18T03:42:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2023-07-18T03:42:40.092 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2023-07-18T03:42:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2023-07-18T03:42:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 377, 2023-07-18T03:42:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2023-07-18T03:42:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 120, 2023-07-18T03:42:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2023-07-18T03:42:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2023-07-18T03:42:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.093 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2023-07-18T03:42:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 175155, 2023-07-18T03:42:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 75242, 2023-07-18T03:42:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 75242, 2023-07-18T03:42:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 99913, 2023-07-18T03:42:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2023-07-18T03:42:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 3277, 2023-07-18T03:42:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 219, 2023-07-18T03:42:40.094 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 91, 2023-07-18T03:42:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 91, 2023-07-18T03:42:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 132, 2023-07-18T03:42:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2023-07-18T03:42:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 4, 2023-07-18T03:42:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 192941199, 2023-07-18T03:42:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 400325720, 2023-07-18T03:42:40.095 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2023-07-18T03:42:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2023-07-18T03:42:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3281, 2023-07-18T03:42:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 153.698118760, 2023-07-18T03:42:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.046844900 2023-07-18T03:42:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2023-07-18T03:42:40.096 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2023-07-18T03:42:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 57, 2023-07-18T03:42:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 46414, 2023-07-18T03:42:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 46357, 2023-07-18T03:42:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 62, 2023-07-18T03:42:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 2116, 2023-07-18T03:42:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 2054, 2023-07-18T03:42:40.097 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 95, 2023-07-18T03:42:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 119311, 2023-07-18T03:42:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 119216, 2023-07-18T03:42:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 2, 2023-07-18T03:42:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 49881, 2023-07-18T03:42:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 49879, 2023-07-18T03:42:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 123460, 2023-07-18T03:42:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 198964 2023-07-18T03:42:40.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2023-07-18T03:42:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 186650, 2023-07-18T03:42:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 1818, 2023-07-18T03:42:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 154494, 2023-07-18T03:42:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 424, 2023-07-18T03:42:40.099 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 3863, 2023-07-18T03:42:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2023-07-18T03:42:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 28279, 2023-07-18T03:42:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 140.448600396, 2023-07-18T03:42:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.004966533 2023-07-18T03:42:40.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2023-07-18T03:42:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2206, 2023-07-18T03:42:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 157.972408128, 2023-07-18T03:42:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.071610339 2023-07-18T03:42:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.101 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2023-07-18T03:42:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2023-07-18T03:42:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.102 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2023-07-18T03:42:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 45969, 2023-07-18T03:42:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 857.025764162, 2023-07-18T03:42:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.018643559 2023-07-18T03:42:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2023-07-18T03:42:40.103 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.104 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2023-07-18T03:42:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2023-07-18T03:42:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000367390, 2023-07-18T03:42:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000367390 2023-07-18T03:42:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.105 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2023-07-18T03:42:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2023-07-18T03:42:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.106 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2023-07-18T03:42:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.107 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2023-07-18T03:42:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2023-07-18T03:42:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 208, 2023-07-18T03:42:40.108 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2.480437775, 2023-07-18T03:42:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.011925181 2023-07-18T03:42:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2023-07-18T03:42:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.109 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2023-07-18T03:42:40.110 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.110 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.110 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.110 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2023-07-18T03:42:40.110 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 8061, 2023-07-18T03:42:40.111 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 44.354593303, 2023-07-18T03:42:40.111 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.005502368 2023-07-18T03:42:40.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.111 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2023-07-18T03:42:40.111 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1232, 2023-07-18T03:42:40.111 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.734107425, 2023-07-18T03:42:40.111 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001407554 2023-07-18T03:42:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2023-07-18T03:42:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 28302, 2023-07-18T03:42:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 557.602244575, 2023-07-18T03:42:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.019701867 2023-07-18T03:42:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.112 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2023-07-18T03:42:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2023-07-18T03:42:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.113 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2023-07-18T03:42:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 205, 2023-07-18T03:42:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 22.569460758, 2023-07-18T03:42:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.110094930 2023-07-18T03:42:40.114 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2023-07-18T03:42:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2023-07-18T03:42:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.115 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2023-07-18T03:42:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.116 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2023-07-18T03:42:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.117 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2023-07-18T03:42:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2023-07-18T03:42:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.118 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2023-07-18T03:42:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2023-07-18T03:42:40.119 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2023-07-18T03:42:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 26913, 2023-07-18T03:42:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 35.747417955, 2023-07-18T03:42:40.120 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001328258 2023-07-18T03:42:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2023-07-18T03:42:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2023-07-18T03:42:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2023-07-18T03:42:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.121 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2023-07-18T03:42:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2023-07-18T03:42:40.122 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 2, 2023-07-18T03:42:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 1, 2023-07-18T03:42:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 1, 2023-07-18T03:42:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2023-07-18T03:42:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 30411, 2023-07-18T03:42:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 30411, 2023-07-18T03:42:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 715 2023-07-18T03:42:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.123 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2023-07-18T03:42:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:42:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2023-07-18T03:42:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:42:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:42:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:42:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:42:40.124 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:42:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:42:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:42:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:42:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:42:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:42:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:42:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:42:40.125 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:42:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2023-07-18T03:42:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:42:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2023-07-18T03:42:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:42:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:42:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:42:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:42:40.126 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:42:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:42:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:42:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2023-07-18T03:42:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:42:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:42:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:42:40.127 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2023-07-18T03:42:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2023-07-18T03:42:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2023-07-18T03:42:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:42:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:42:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:42:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:42:40.128 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 335956, 2023-07-18T03:42:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 132, 2023-07-18T03:42:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2023-07-18T03:42:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2023-07-18T03:42:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2023-07-18T03:42:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2023-07-18T03:42:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:42:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2023-07-18T03:42:40.129 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:42:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2023-07-18T03:42:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 7720, 2023-07-18T03:42:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 162, 2023-07-18T03:42:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:42:40.130 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:42:40.131 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2023-07-18T03:42:40.131 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2023-07-18T03:42:40.131 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 278237, 2023-07-18T03:42:40.131 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 1486, 2023-07-18T03:42:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2023-07-18T03:42:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2023-07-18T03:42:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2023-07-18T03:42:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2023-07-18T03:42:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.132 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2023-07-18T03:42:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2023-07-18T03:42:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2023-07-18T03:42:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 42788, 2023-07-18T03:42:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 423915352, 2023-07-18T03:42:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 27, 2023-07-18T03:42:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 42788, 2023-07-18T03:42:40.133 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2023-07-18T03:42:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 42788, 2023-07-18T03:42:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 395.745003020, 2023-07-18T03:42:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.009248971 2023-07-18T03:42:40.134 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2023-07-18T03:42:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2023-07-18T03:42:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 42788, 2023-07-18T03:42:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 51662 2023-07-18T03:42:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 42788, 2023-07-18T03:42:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 2627, 2023-07-18T03:42:40.135 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 40161, 2023-07-18T03:42:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2023-07-18T03:42:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2023-07-18T03:42:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 106, 2023-07-18T03:42:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 3549, 2023-07-18T03:42:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 608, 2023-07-18T03:42:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 3889, 2023-07-18T03:42:40.136 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 355, 2023-07-18T03:42:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2023-07-18T03:42:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2023-07-18T03:42:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2023-07-18T03:42:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 118, 2023-07-18T03:42:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 30941, 2023-07-18T03:42:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2023-07-18T03:42:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2023-07-18T03:42:40.137 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2023-07-18T03:42:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 1915, 2023-07-18T03:42:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 7098, 2023-07-18T03:42:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2023-07-18T03:42:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2023-07-18T03:42:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2023-07-18T03:42:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2023-07-18T03:42:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2023-07-18T03:42:40.138 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2023-07-18T03:42:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2023-07-18T03:42:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2023-07-18T03:42:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2023-07-18T03:42:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2023-07-18T03:42:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2023-07-18T03:42:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2023-07-18T03:42:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2023-07-18T03:42:40.139 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2023-07-18T03:42:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2023-07-18T03:42:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2023-07-18T03:42:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2023-07-18T03:42:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2023-07-18T03:42:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2023-07-18T03:42:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2023-07-18T03:42:40.140 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2023-07-18T03:42:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2023-07-18T03:42:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2023-07-18T03:42:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2023-07-18T03:42:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2023-07-18T03:42:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2023-07-18T03:42:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 84, 2023-07-18T03:42:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2023-07-18T03:42:40.141 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 3, 2023-07-18T03:42:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 12, 2023-07-18T03:42:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 12, 2023-07-18T03:42:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2023-07-18T03:42:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2023-07-18T03:42:40.142 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 2594, 2023-07-18T03:42:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 150, 2023-07-18T03:42:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 339 2023-07-18T03:42:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2023-07-18T03:42:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2023-07-18T03:42:40.143 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2023-07-18T03:42:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2023-07-18T03:42:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2023-07-18T03:42:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2023-07-18T03:42:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 155812 2023-07-18T03:42:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.144 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2023-07-18T03:42:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2023-07-18T03:42:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 14, 2023-07-18T03:42:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2023-07-18T03:42:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 6, 2023-07-18T03:42:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 29613, 2023-07-18T03:42:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2023-07-18T03:42:40.145 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:42:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-07-18T03:42:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 3082997, 2023-07-18T03:42:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 280038201, 2023-07-18T03:42:40.146 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 3082997, 2023-07-18T03:42:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-18T03:42:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-18T03:42:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 3082997, 2023-07-18T03:42:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 280038201, 2023-07-18T03:42:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.147 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2023-07-18T03:42:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-07-18T03:42:40.148 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-18T03:42:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-18T03:42:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 42788, 2023-07-18T03:42:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 426904592, 2023-07-18T03:42:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 42788, 2023-07-18T03:42:40.149 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 426904592, 2023-07-18T03:42:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2023-07-18T03:42:40.150 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2023-07-18T03:42:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-18T03:42:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-18T03:42:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:40.151 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 42788, 2023-07-18T03:42:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 42788, 2023-07-18T03:42:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 42788, 2023-07-18T03:42:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 42788, 2023-07-18T03:42:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.152 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2023-07-18T03:42:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-07-18T03:42:40.153 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 29613, 2023-07-18T03:42:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 2992105, 2023-07-18T03:42:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 29613, 2023-07-18T03:42:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-18T03:42:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-18T03:42:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 608, 2023-07-18T03:42:40.154 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2992105, 2023-07-18T03:42:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55bde6a6a0a0": { 2023-07-18T03:42:40.155 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-07-18T03:42:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 175155, 2023-07-18T03:42:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 396131416, 2023-07-18T03:42:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 175155, 2023-07-18T03:42:40.156 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-18T03:42:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-18T03:42:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 3281, 2023-07-18T03:42:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 396131416, 2023-07-18T03:42:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:40.157 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:40.158 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:40.158 INFO:teuthology.orchestra.run.smithi134.stdout:} 2023-07-18T03:42:40.495 INFO:tasks.check_counter:Daemon mds.i mds.exported=371 2023-07-18T03:42:40.495 INFO:tasks.check_counter:Daemon mds.i mds.imported=211 2023-07-18T03:42:40.496 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=118 2023-07-18T03:42:40.496 DEBUG:tasks.check_counter:Getting stats from l 2023-07-18T03:42:40.496 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c -- ceph daemon mds.l perf dump 2023-07-18T03:42:41.048 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/mon.c/config 2023-07-18T03:42:41.545 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:41 smithi134 ceph-mon[98216]: pgmap v648: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2023-07-18T03:42:41.826 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:41 smithi040 ceph-mon[96439]: pgmap v648: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2023-07-18T03:42:41.847 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:41 smithi064 ceph-mon[98059]: pgmap v648: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2023-07-18T03:42:42.289 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2023-07-18T03:42:42.289 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2023-07-18T03:42:42.289 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 133253, 2023-07-18T03:42:42.289 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 177751, 2023-07-18T03:42:42.289 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 43647657, 2023-07-18T03:42:42.289 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 126852271, 2023-07-18T03:42:42.290 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 21, 2023-07-18T03:42:42.290 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 7, 2023-07-18T03:42:42.290 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 4.431378354, 2023-07-18T03:42:42.290 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 2.215368428, 2023-07-18T03:42:42.290 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 11.185348782, 2023-07-18T03:42:42.290 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.155464598, 2023-07-18T03:42:42.291 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:42.291 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 177745, 2023-07-18T03:42:42.291 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 11.040040630, 2023-07-18T03:42:42.291 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000062111 2023-07-18T03:42:42.291 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.291 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:42.291 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 93927, 2023-07-18T03:42:42.292 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.025137980, 2023-07-18T03:42:42.292 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000267 2023-07-18T03:42:42.292 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.292 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-18T03:42:42.292 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2023-07-18T03:42:42.292 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.292 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2023-07-18T03:42:42.293 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 3413015, 2023-07-18T03:42:42.293 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 3418647, 2023-07-18T03:42:42.293 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 561023985, 2023-07-18T03:42:42.293 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 562022358, 2023-07-18T03:42:42.293 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 9, 2023-07-18T03:42:42.293 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2023-07-18T03:42:42.293 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 103.061153178, 2023-07-18T03:42:42.294 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 43.523023123, 2023-07-18T03:42:42.294 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 70.122308185, 2023-07-18T03:42:42.294 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.175836407, 2023-07-18T03:42:42.294 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:42.294 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3418649, 2023-07-18T03:42:42.294 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 33.925722267, 2023-07-18T03:42:42.294 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000009923 2023-07-18T03:42:42.295 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.295 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:42.295 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6108097, 2023-07-18T03:42:42.295 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.736845804, 2023-07-18T03:42:42.295 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000120 2023-07-18T03:42:42.295 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.295 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-18T03:42:42.296 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 364512 2023-07-18T03:42:42.296 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.296 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2023-07-18T03:42:42.296 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 358138, 2023-07-18T03:42:42.296 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 358117, 2023-07-18T03:42:42.296 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 59191754, 2023-07-18T03:42:42.296 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 119380491, 2023-07-18T03:42:42.297 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 12, 2023-07-18T03:42:42.297 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 9, 2023-07-18T03:42:42.297 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 12.977108334, 2023-07-18T03:42:42.297 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 5.034898137, 2023-07-18T03:42:42.297 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 8.984336567, 2023-07-18T03:42:42.297 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.177850934, 2023-07-18T03:42:42.297 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2023-07-18T03:42:42.298 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 358117, 2023-07-18T03:42:42.298 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 4.655141332, 2023-07-18T03:42:42.298 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000012998 2023-07-18T03:42:42.298 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.298 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2023-07-18T03:42:42.298 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 706002, 2023-07-18T03:42:42.298 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.086058534, 2023-07-18T03:42:42.299 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000121 2023-07-18T03:42:42.299 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.299 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 798704, 2023-07-18T03:42:42.299 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 135984 2023-07-18T03:42:42.299 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.299 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2023-07-18T03:42:42.299 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2023-07-18T03:42:42.300 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2023-07-18T03:42:42.300 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.300 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2023-07-18T03:42:42.300 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-07-18T03:42:42.300 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-07-18T03:42:42.300 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 5080, 2023-07-18T03:42:42.300 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 26.866649069, 2023-07-18T03:42:42.301 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.005288710 2023-07-18T03:42:42.301 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:42.301 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.301 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2023-07-18T03:42:42.301 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2023-07-18T03:42:42.301 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2023-07-18T03:42:42.301 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 19005, 2023-07-18T03:42:42.302 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 3.007886057, 2023-07-18T03:42:42.302 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000158268 2023-07-18T03:42:42.302 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:42.302 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.302 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2023-07-18T03:42:42.302 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 62518, 2023-07-18T03:42:42.302 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 62217, 2023-07-18T03:42:42.302 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2023-07-18T03:42:42.303 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 62217, 2023-07-18T03:42:42.303 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1651.659689050, 2023-07-18T03:42:42.303 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.026546758 2023-07-18T03:42:42.303 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.303 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2023-07-18T03:42:42.303 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 301, 2023-07-18T03:42:42.303 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 5, 2023-07-18T03:42:42.304 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2023-07-18T03:42:42.304 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 324, 2023-07-18T03:42:42.304 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 35, 2023-07-18T03:42:42.304 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2023-07-18T03:42:42.304 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 18, 2023-07-18T03:42:42.304 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 3, 2023-07-18T03:42:42.304 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 2, 2023-07-18T03:42:42.305 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 13, 2023-07-18T03:42:42.305 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 13, 2023-07-18T03:42:42.305 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 7421, 2023-07-18T03:42:42.305 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2023-07-18T03:42:42.305 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2023-07-18T03:42:42.305 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 4, 2023-07-18T03:42:42.305 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 111280, 2023-07-18T03:42:42.306 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 94837, 2023-07-18T03:42:42.306 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 207, 2023-07-18T03:42:42.306 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 383, 2023-07-18T03:42:42.306 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 4, 2023-07-18T03:42:42.306 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2023-07-18T03:42:42.306 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 907, 2023-07-18T03:42:42.306 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 1000, 2023-07-18T03:42:42.307 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 1, 2023-07-18T03:42:42.307 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 227, 2023-07-18T03:42:42.307 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 5154, 2023-07-18T03:42:42.307 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 171, 2023-07-18T03:42:42.307 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 11093, 2023-07-18T03:42:42.307 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2023-07-18T03:42:42.307 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 830, 2023-07-18T03:42:42.308 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2023-07-18T03:42:42.308 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 3846, 2023-07-18T03:42:42.308 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 72892698, 2023-07-18T03:42:42.308 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2023-07-18T03:42:42.308 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 830, 2023-07-18T03:42:42.308 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2023-07-18T03:42:42.308 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2023-07-18T03:42:42.309 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2023-07-18T03:42:42.309 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 2, 2023-07-18T03:42:42.309 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2023-07-18T03:42:42.309 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 2, 2023-07-18T03:42:42.309 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 828, 2023-07-18T03:42:42.309 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 226, 2023-07-18T03:42:42.309 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 7462, 2023-07-18T03:42:42.310 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 17912, 2023-07-18T03:42:42.310 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2023-07-18T03:42:42.310 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-18T03:42:42.310 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-18T03:42:42.310 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 17817, 2023-07-18T03:42:42.310 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 6729, 2023-07-18T03:42:42.310 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 329, 2023-07-18T03:42:42.310 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 23405 2023-07-18T03:42:42.311 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.311 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2023-07-18T03:42:42.311 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2023-07-18T03:42:42.311 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 10, 2023-07-18T03:42:42.311 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 7, 2023-07-18T03:42:42.311 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 447, 2023-07-18T03:42:42.311 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 618, 2023-07-18T03:42:42.312 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2023-07-18T03:42:42.312 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2023-07-18T03:42:42.312 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2023-07-18T03:42:42.312 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 10068, 2023-07-18T03:42:42.312 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 10068, 2023-07-18T03:42:42.312 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2023-07-18T03:42:42.312 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2023-07-18T03:42:42.313 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2023-07-18T03:42:42.313 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2023-07-18T03:42:42.313 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2023-07-18T03:42:42.313 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2023-07-18T03:42:42.313 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2023-07-18T03:42:42.313 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2023-07-18T03:42:42.313 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 227, 2023-07-18T03:42:42.314 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2023-07-18T03:42:42.314 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 35, 2023-07-18T03:42:42.314 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2023-07-18T03:42:42.314 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2023-07-18T03:42:42.314 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.314 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2023-07-18T03:42:42.314 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 57660, 2023-07-18T03:42:42.315 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2023-07-18T03:42:42.315 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2023-07-18T03:42:42.315 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 57660, 2023-07-18T03:42:42.315 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2023-07-18T03:42:42.315 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2023-07-18T03:42:42.315 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 72, 2023-07-18T03:42:42.315 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2023-07-18T03:42:42.316 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2023-07-18T03:42:42.316 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 72, 2023-07-18T03:42:42.316 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2023-07-18T03:42:42.316 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2023-07-18T03:42:42.316 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2023-07-18T03:42:42.316 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 125038540, 2023-07-18T03:42:42.316 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2023-07-18T03:42:42.317 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2023-07-18T03:42:42.317 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1981, 2023-07-18T03:42:42.317 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 27.632795093, 2023-07-18T03:42:42.317 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.013948912 2023-07-18T03:42:42.317 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.317 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2023-07-18T03:42:42.317 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.317 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2023-07-18T03:42:42.318 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 25, 2023-07-18T03:42:42.318 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 13184, 2023-07-18T03:42:42.318 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 13159, 2023-07-18T03:42:42.318 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 21, 2023-07-18T03:42:42.318 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 692, 2023-07-18T03:42:42.318 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 671, 2023-07-18T03:42:42.318 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 18, 2023-07-18T03:42:42.319 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 32670, 2023-07-18T03:42:42.319 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 32652, 2023-07-18T03:42:42.319 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2023-07-18T03:42:42.319 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 16179, 2023-07-18T03:42:42.319 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 16179, 2023-07-18T03:42:42.319 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 99260, 2023-07-18T03:42:42.319 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 198964 2023-07-18T03:42:42.320 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.320 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2023-07-18T03:42:42.320 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 72634, 2023-07-18T03:42:42.320 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 2876, 2023-07-18T03:42:42.320 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 62518, 2023-07-18T03:42:42.320 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 214, 2023-07-18T03:42:42.320 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 4960, 2023-07-18T03:42:42.321 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2023-07-18T03:42:42.321 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6348, 2023-07-18T03:42:42.321 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 25.557999451, 2023-07-18T03:42:42.321 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.004026149 2023-07-18T03:42:42.321 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.321 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2023-07-18T03:42:42.321 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 118, 2023-07-18T03:42:42.322 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 30.349358151, 2023-07-18T03:42:42.322 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.257197950 2023-07-18T03:42:42.322 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.322 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2023-07-18T03:42:42.322 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.322 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.322 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.323 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.323 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2023-07-18T03:42:42.323 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.323 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.323 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.323 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.323 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2023-07-18T03:42:42.323 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 37813, 2023-07-18T03:42:42.324 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1285.922626450, 2023-07-18T03:42:42.324 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.034007421 2023-07-18T03:42:42.324 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.324 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2023-07-18T03:42:42.324 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.324 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.324 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.325 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.325 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2023-07-18T03:42:42.325 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.325 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.325 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.325 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.325 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2023-07-18T03:42:42.326 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.326 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.326 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.326 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.326 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2023-07-18T03:42:42.326 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.326 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.327 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.327 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.327 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2023-07-18T03:42:42.327 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.327 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.327 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.327 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.328 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2023-07-18T03:42:42.328 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.328 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.328 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.328 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.328 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2023-07-18T03:42:42.328 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 29, 2023-07-18T03:42:42.329 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.724421451, 2023-07-18T03:42:42.329 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.024980050 2023-07-18T03:42:42.329 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.329 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2023-07-18T03:42:42.329 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.329 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.329 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.330 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.330 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2023-07-18T03:42:42.330 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.330 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.330 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.331 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.331 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2023-07-18T03:42:42.331 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2396, 2023-07-18T03:42:42.331 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 6.977262091, 2023-07-18T03:42:42.331 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.002912045 2023-07-18T03:42:42.331 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.331 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2023-07-18T03:42:42.332 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 402, 2023-07-18T03:42:42.332 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.490874613, 2023-07-18T03:42:42.332 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001221081 2023-07-18T03:42:42.332 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.332 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2023-07-18T03:42:42.332 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6355, 2023-07-18T03:42:42.332 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 285.001215925, 2023-07-18T03:42:42.333 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.044846768 2023-07-18T03:42:42.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.333 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2023-07-18T03:42:42.333 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.333 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.333 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.333 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.334 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2023-07-18T03:42:42.334 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.334 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.334 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.334 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.334 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2023-07-18T03:42:42.334 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 45, 2023-07-18T03:42:42.334 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 5.000325359, 2023-07-18T03:42:42.335 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.111118341 2023-07-18T03:42:42.335 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.335 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2023-07-18T03:42:42.335 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.335 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.335 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.335 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.336 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2023-07-18T03:42:42.336 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.336 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.336 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.336 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.336 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2023-07-18T03:42:42.336 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.337 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.337 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.337 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.337 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2023-07-18T03:42:42.337 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.337 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.337 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.338 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2023-07-18T03:42:42.338 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.338 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.338 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.338 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.338 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2023-07-18T03:42:42.339 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.339 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.339 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.339 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.339 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2023-07-18T03:42:42.339 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.339 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.340 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.340 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.340 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2023-07-18T03:42:42.340 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.340 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.340 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.340 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.341 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2023-07-18T03:42:42.341 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 8711, 2023-07-18T03:42:42.341 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 11.635605559, 2023-07-18T03:42:42.341 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001335737 2023-07-18T03:42:42.341 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.341 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2023-07-18T03:42:42.341 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2023-07-18T03:42:42.342 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2023-07-18T03:42:42.342 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.342 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.342 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.342 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:42.342 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.342 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2023-07-18T03:42:42.342 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2023-07-18T03:42:42.343 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 1, 2023-07-18T03:42:42.343 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 0, 2023-07-18T03:42:42.343 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 1, 2023-07-18T03:42:42.343 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2023-07-18T03:42:42.343 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 7873, 2023-07-18T03:42:42.343 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 7873, 2023-07-18T03:42:42.343 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 2129 2023-07-18T03:42:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2023-07-18T03:42:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2023-07-18T03:42:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2023-07-18T03:42:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2023-07-18T03:42:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2023-07-18T03:42:42.344 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2023-07-18T03:42:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2023-07-18T03:42:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-18T03:42:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2023-07-18T03:42:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-18T03:42:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2023-07-18T03:42:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2023-07-18T03:42:42.345 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2023-07-18T03:42:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-18T03:42:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2023-07-18T03:42:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2023-07-18T03:42:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2023-07-18T03:42:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2023-07-18T03:42:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2023-07-18T03:42:42.346 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-18T03:42:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2023-07-18T03:42:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-18T03:42:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2023-07-18T03:42:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2023-07-18T03:42:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2023-07-18T03:42:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2023-07-18T03:42:42.347 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2023-07-18T03:42:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-18T03:42:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2023-07-18T03:42:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2023-07-18T03:42:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2023-07-18T03:42:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2023-07-18T03:42:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2023-07-18T03:42:42.348 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2023-07-18T03:42:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2023-07-18T03:42:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2023-07-18T03:42:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2023-07-18T03:42:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 320383, 2023-07-18T03:42:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 134, 2023-07-18T03:42:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 528, 2023-07-18T03:42:42.349 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 6, 2023-07-18T03:42:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2023-07-18T03:42:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2023-07-18T03:42:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2023-07-18T03:42:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2023-07-18T03:42:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2023-07-18T03:42:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2023-07-18T03:42:42.350 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 7720, 2023-07-18T03:42:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 162, 2023-07-18T03:42:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2023-07-18T03:42:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2023-07-18T03:42:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2023-07-18T03:42:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2023-07-18T03:42:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 96077, 2023-07-18T03:42:42.351 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 501, 2023-07-18T03:42:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2023-07-18T03:42:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2023-07-18T03:42:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2023-07-18T03:42:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2023-07-18T03:42:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2023-07-18T03:42:42.352 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2023-07-18T03:42:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2023-07-18T03:42:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 14698, 2023-07-18T03:42:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 128415732, 2023-07-18T03:42:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 33, 2023-07-18T03:42:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 14698, 2023-07-18T03:42:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2023-07-18T03:42:42.353 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 14698, 2023-07-18T03:42:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 72.910599194, 2023-07-18T03:42:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.004960579 2023-07-18T03:42:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2023-07-18T03:42:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2023-07-18T03:42:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 14698, 2023-07-18T03:42:42.354 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 15270 2023-07-18T03:42:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 14698, 2023-07-18T03:42:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 1234, 2023-07-18T03:42:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 13464, 2023-07-18T03:42:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2023-07-18T03:42:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2023-07-18T03:42:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2023-07-18T03:42:42.355 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 10, 2023-07-18T03:42:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 392, 2023-07-18T03:42:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 2373, 2023-07-18T03:42:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 222, 2023-07-18T03:42:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2023-07-18T03:42:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2023-07-18T03:42:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2023-07-18T03:42:42.356 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 35, 2023-07-18T03:42:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 10388, 2023-07-18T03:42:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2023-07-18T03:42:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2023-07-18T03:42:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2023-07-18T03:42:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 830, 2023-07-18T03:42:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 20, 2023-07-18T03:42:42.357 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2023-07-18T03:42:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2023-07-18T03:42:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2023-07-18T03:42:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2023-07-18T03:42:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2023-07-18T03:42:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2023-07-18T03:42:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2023-07-18T03:42:42.358 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2023-07-18T03:42:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2023-07-18T03:42:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2023-07-18T03:42:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2023-07-18T03:42:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2023-07-18T03:42:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2023-07-18T03:42:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2023-07-18T03:42:42.359 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2023-07-18T03:42:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2023-07-18T03:42:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2023-07-18T03:42:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2023-07-18T03:42:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2023-07-18T03:42:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2023-07-18T03:42:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2023-07-18T03:42:42.360 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2023-07-18T03:42:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2023-07-18T03:42:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2023-07-18T03:42:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2023-07-18T03:42:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2023-07-18T03:42:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 84, 2023-07-18T03:42:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2023-07-18T03:42:42.361 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 3, 2023-07-18T03:42:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 12, 2023-07-18T03:42:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 12, 2023-07-18T03:42:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2023-07-18T03:42:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2023-07-18T03:42:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 853, 2023-07-18T03:42:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 17, 2023-07-18T03:42:42.362 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 130 2023-07-18T03:42:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2023-07-18T03:42:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2023-07-18T03:42:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2023-07-18T03:42:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2023-07-18T03:42:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2023-07-18T03:42:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.363 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2023-07-18T03:42:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 63820 2023-07-18T03:42:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2023-07-18T03:42:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2023-07-18T03:42:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 14, 2023-07-18T03:42:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2023-07-18T03:42:42.364 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 6, 2023-07-18T03:42:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 10068, 2023-07-18T03:42:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2023-07-18T03:42:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-18T03:42:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-07-18T03:42:42.365 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:42.366 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 3904402, 2023-07-18T03:42:42.366 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 314386796, 2023-07-18T03:42:42.366 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:42.366 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 3904402, 2023-07-18T03:42:42.366 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-18T03:42:42.366 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-18T03:42:42.366 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 3904402, 2023-07-18T03:42:42.367 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 314386796, 2023-07-18T03:42:42.367 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:42.367 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.367 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.367 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.367 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:42.367 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.368 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2023-07-18T03:42:42.368 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:42.368 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2023-07-18T03:42:42.368 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:42.368 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-18T03:42:42.368 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-18T03:42:42.368 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:42.369 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:42.369 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 14698, 2023-07-18T03:42:42.369 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 129432555, 2023-07-18T03:42:42.369 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 14698, 2023-07-18T03:42:42.369 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 129432555, 2023-07-18T03:42:42.369 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:42.369 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.370 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.370 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.370 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:42.370 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.370 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2023-07-18T03:42:42.370 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:42.370 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2023-07-18T03:42:42.371 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:42.371 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2023-07-18T03:42:42.371 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2023-07-18T03:42:42.371 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:42.371 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2023-07-18T03:42:42.371 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 14698, 2023-07-18T03:42:42.371 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 14698, 2023-07-18T03:42:42.372 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 14698, 2023-07-18T03:42:42.372 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 14698, 2023-07-18T03:42:42.372 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:42.372 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.372 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.372 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.372 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:42.373 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.373 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2023-07-18T03:42:42.373 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:42.373 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-07-18T03:42:42.373 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:42.373 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 10068, 2023-07-18T03:42:42.373 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1017148, 2023-07-18T03:42:42.373 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:42.374 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 10068, 2023-07-18T03:42:42.374 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-18T03:42:42.374 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-18T03:42:42.374 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 392, 2023-07-18T03:42:42.374 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1017148, 2023-07-18T03:42:42.374 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:42.374 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.375 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.375 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.375 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:42.375 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2023-07-18T03:42:42.375 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x5581d2f9a0a0": { 2023-07-18T03:42:42.375 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2023-07-18T03:42:42.375 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2023-07-18T03:42:42.376 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2023-07-18T03:42:42.376 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 57660, 2023-07-18T03:42:42.376 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 120844236, 2023-07-18T03:42:42.376 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2023-07-18T03:42:42.376 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 57660, 2023-07-18T03:42:42.376 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2023-07-18T03:42:42.376 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2023-07-18T03:42:42.377 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 1981, 2023-07-18T03:42:42.377 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 120844236, 2023-07-18T03:42:42.377 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2023-07-18T03:42:42.377 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2023-07-18T03:42:42.377 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2023-07-18T03:42:42.377 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2023-07-18T03:42:42.377 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:42.378 INFO:teuthology.orchestra.run.smithi134.stdout: } 2023-07-18T03:42:42.378 INFO:teuthology.orchestra.run.smithi134.stdout:} 2023-07-18T03:42:42.693 INFO:tasks.check_counter:Daemon mds.l mds.exported=227 2023-07-18T03:42:42.693 INFO:tasks.check_counter:Daemon mds.l mds.imported=171 2023-07-18T03:42:42.693 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=35 2023-07-18T03:42:42.694 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-07-18T03:42:42.706 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-18T03:42:42.706 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:42:42.707 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:42:42.732 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:42:42.732 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:42:42.793 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:42:42.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.983+0000 7fba3f368700 1 -- 172.21.15.40:0/1900806601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba380ffb00 msgr2=0x7fba380fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:42.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.983+0000 7fba3f368700 1 --2- 172.21.15.40:0/1900806601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba380ffb00 0x7fba380fff10 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fba34009990 tx=0x7fba34009ca0 comp rx=0 tx=0).stop 2023-07-18T03:42:42.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.983+0000 7fba3f368700 1 -- 172.21.15.40:0/1900806601 shutdown_connections 2023-07-18T03:42:42.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.983+0000 7fba3f368700 1 --2- 172.21.15.40:0/1900806601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba38101680 0x7fba38107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:42.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.983+0000 7fba3f368700 1 --2- 172.21.15.40:0/1900806601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba380ffb00 0x7fba380fff10 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:42.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.983+0000 7fba3f368700 1 --2- 172.21.15.40:0/1900806601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba38100cf0 0x7fba38101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:42.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.983+0000 7fba3f368700 1 -- 172.21.15.40:0/1900806601 >> 172.21.15.40:0/1900806601 conn(0x7fba380fb0e0 msgr2=0x7fba380fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:42.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.983+0000 7fba3f368700 1 -- 172.21.15.40:0/1900806601 shutdown_connections 2023-07-18T03:42:42.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.983+0000 7fba3f368700 1 -- 172.21.15.40:0/1900806601 wait complete. 2023-07-18T03:42:42.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.984+0000 7fba3f368700 1 Processor -- start 2023-07-18T03:42:42.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.984+0000 7fba3f368700 1 -- start start 2023-07-18T03:42:42.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.984+0000 7fba3f368700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba380ffb00 0x7fba38197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:42.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.984+0000 7fba3f368700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba38100cf0 0x7fba38197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:42.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.984+0000 7fba3f368700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba38101680 0x7fba3819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:42.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.984+0000 7fba3f368700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3819f600 con 0x7fba38101680 2023-07-18T03:42:42.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.984+0000 7fba3f368700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3819f740 con 0x7fba380ffb00 2023-07-18T03:42:42.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.984+0000 7fba3f368700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3819f8d0 con 0x7fba38100cf0 2023-07-18T03:42:42.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.984+0000 7fba3d905700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba38101680 0x7fba3819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:42.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.984+0000 7fba3c903700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba38100cf0 0x7fba38197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:42.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.984+0000 7fba3d905700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba38101680 0x7fba3819f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33854/0 (socket says 172.21.15.40:33854) 2023-07-18T03:42:42.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.984+0000 7fba3d905700 1 -- 172.21.15.40:0/3008676905 learned_addr learned my addr 172.21.15.40:0/3008676905 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:42.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.984+0000 7fba3d104700 1 --2- 172.21.15.40:0/3008676905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba380ffb00 0x7fba38197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:42.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.984+0000 7fba3d905700 1 -- 172.21.15.40:0/3008676905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba38100cf0 msgr2=0x7fba38197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:42.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.984+0000 7fba3d905700 1 --2- 172.21.15.40:0/3008676905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba38100cf0 0x7fba38197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:42.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.985+0000 7fba3d905700 1 -- 172.21.15.40:0/3008676905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba380ffb00 msgr2=0x7fba38197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:42.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.985+0000 7fba3d905700 1 --2- 172.21.15.40:0/3008676905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba380ffb00 0x7fba38197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:42.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.985+0000 7fba3d905700 1 -- 172.21.15.40:0/3008676905 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba340095f0 con 0x7fba38101680 2023-07-18T03:42:42.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.985+0000 7fba3d905700 1 --2- 172.21.15.40:0/3008676905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba38101680 0x7fba3819f030 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fba30007ee0 tx=0x7fba3000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:42.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.986+0000 7fba2e7fc700 1 -- 172.21.15.40:0/3008676905 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3000dd20 con 0x7fba38101680 2023-07-18T03:42:42.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.986+0000 7fba2e7fc700 1 -- 172.21.15.40:0/3008676905 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba3000f040 con 0x7fba38101680 2023-07-18T03:42:42.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.986+0000 7fba3f368700 1 -- 172.21.15.40:0/3008676905 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3819fbb0 con 0x7fba38101680 2023-07-18T03:42:42.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.986+0000 7fba2e7fc700 1 -- 172.21.15.40:0/3008676905 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba300048a0 con 0x7fba38101680 2023-07-18T03:42:42.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.986+0000 7fba3f368700 1 -- 172.21.15.40:0/3008676905 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba381a0110 con 0x7fba38101680 2023-07-18T03:42:42.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.987+0000 7fba3f368700 1 -- 172.21.15.40:0/3008676905 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba38192190 con 0x7fba38101680 2023-07-18T03:42:42.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.988+0000 7fba2e7fc700 1 -- 172.21.15.40:0/3008676905 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fba30003a00 con 0x7fba38101680 2023-07-18T03:42:42.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.988+0000 7fba2e7fc700 1 --2- 172.21.15.40:0/3008676905 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fba240a12a0 0x7fba240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:42.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.989+0000 7fba2e7fc700 1 -- 172.21.15.40:0/3008676905 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7fba300cc0a0 con 0x7fba38101680 2023-07-18T03:42:42.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.989+0000 7fba3d104700 1 --2- 172.21.15.40:0/3008676905 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fba240a12a0 0x7fba240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:42.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.989+0000 7fba3d104700 1 --2- 172.21.15.40:0/3008676905 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fba240a12a0 0x7fba240a3750 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fba34006010 tx=0x7fba34005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:42.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:42.993+0000 7fba2e7fc700 1 -- 172.21.15.40:0/3008676905 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fba3004e790 con 0x7fba38101680 2023-07-18T03:42:43.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.194+0000 7fba3f368700 1 -- 172.21.15.40:0/3008676905 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba3802c850 con 0x7fba38101680 2023-07-18T03:42:43.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.196+0000 7fba2e7fc700 1 -- 172.21.15.40:0/3008676905 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22599 (secure 0 0 0) 0x7fba30095770 con 0x7fba38101680 2023-07-18T03:42:43.191 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:42:43.191 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":84,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:35:46.881845+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:42:43.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.198+0000 7fba3f368700 1 -- 172.21.15.40:0/3008676905 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fba240a12a0 msgr2=0x7fba240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:43.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.198+0000 7fba3f368700 1 --2- 172.21.15.40:0/3008676905 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fba240a12a0 0x7fba240a3750 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fba34006010 tx=0x7fba34005960 comp rx=0 tx=0).stop 2023-07-18T03:42:43.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.198+0000 7fba3f368700 1 -- 172.21.15.40:0/3008676905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba38101680 msgr2=0x7fba3819f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:43.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.198+0000 7fba3f368700 1 --2- 172.21.15.40:0/3008676905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba38101680 0x7fba3819f030 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fba30007ee0 tx=0x7fba3000d3b0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.198+0000 7fba3f368700 1 -- 172.21.15.40:0/3008676905 shutdown_connections 2023-07-18T03:42:43.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.198+0000 7fba3f368700 1 --2- 172.21.15.40:0/3008676905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba38101680 0x7fba3819f030 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.198+0000 7fba3f368700 1 --2- 172.21.15.40:0/3008676905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba380ffb00 0x7fba38197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.198+0000 7fba3f368700 1 --2- 172.21.15.40:0/3008676905 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fba240a12a0 0x7fba240a3750 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.198+0000 7fba3f368700 1 --2- 172.21.15.40:0/3008676905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fba38100cf0 0x7fba38197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.198+0000 7fba3f368700 1 -- 172.21.15.40:0/3008676905 >> 172.21.15.40:0/3008676905 conn(0x7fba380fb0e0 msgr2=0x7fba381065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:43.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.198+0000 7fba3f368700 1 -- 172.21.15.40:0/3008676905 shutdown_connections 2023-07-18T03:42:43.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.198+0000 7fba3f368700 1 -- 172.21.15.40:0/3008676905 wait complete. 2023-07-18T03:42:43.203 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:42 smithi040 ceph-mon[96439]: pgmap v649: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 12 op/s 2023-07-18T03:42:43.204 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-18T03:25:38.267262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-18T03:29:24.727790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-18T03:29:26.078507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-18T03:42:43.205 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-18T03:42:43.313 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:42 smithi134 ceph-mon[98216]: pgmap v649: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 12 op/s 2023-07-18T03:42:43.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.342+0000 7faf94a9c700 1 -- 172.21.15.40:0/1033221756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf900ff5c0 msgr2=0x7faf90106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.342+0000 7faf94a9c700 1 --2- 172.21.15.40:0/1033221756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf900ff5c0 0x7faf90106e70 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7faf8800b0a0 tx=0x7faf8800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.342+0000 7faf94a9c700 1 -- 172.21.15.40:0/1033221756 shutdown_connections 2023-07-18T03:42:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.342+0000 7faf94a9c700 1 --2- 172.21.15.40:0/1033221756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf900ff5c0 0x7faf90106e70 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.342+0000 7faf94a9c700 1 --2- 172.21.15.40:0/1033221756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf90100050 0x7faf90100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.342+0000 7faf94a9c700 1 --2- 172.21.15.40:0/1033221756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf900fec10 0x7faf900ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.342+0000 7faf94a9c700 1 -- 172.21.15.40:0/1033221756 >> 172.21.15.40:0/1033221756 conn(0x7faf900f9770 msgr2=0x7faf900fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.342+0000 7faf94a9c700 1 -- 172.21.15.40:0/1033221756 shutdown_connections 2023-07-18T03:42:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.342+0000 7faf94a9c700 1 -- 172.21.15.40:0/1033221756 wait complete. 2023-07-18T03:42:43.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.342+0000 7faf94a9c700 1 Processor -- start 2023-07-18T03:42:43.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.342+0000 7faf94a9c700 1 -- start start 2023-07-18T03:42:43.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf94a9c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf900fec10 0x7faf901029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:43.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf94a9c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf900ff5c0 0x7faf90101000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:43.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf94a9c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf90100050 0x7faf90101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:43.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf94a9c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf90101ce0 con 0x7faf900fec10 2023-07-18T03:42:43.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf94a9c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf90101e50 con 0x7faf90100050 2023-07-18T03:42:43.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf94a9c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9019f940 con 0x7faf900ff5c0 2023-07-18T03:42:43.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf8e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf900fec10 0x7faf901029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:43.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf8e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf900fec10 0x7faf901029c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33864/0 (socket says 172.21.15.40:33864) 2023-07-18T03:42:43.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf8e59c700 1 -- 172.21.15.40:0/2760042165 learned_addr learned my addr 172.21.15.40:0/2760042165 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:43.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf8ed9d700 1 --2- 172.21.15.40:0/2760042165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf90100050 0x7faf90101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:43.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf8dd9b700 1 --2- 172.21.15.40:0/2760042165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf900ff5c0 0x7faf90101000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:43.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf8e59c700 1 -- 172.21.15.40:0/2760042165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf900ff5c0 msgr2=0x7faf90101000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:43.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf8e59c700 1 --2- 172.21.15.40:0/2760042165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf900ff5c0 0x7faf90101000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf8e59c700 1 -- 172.21.15.40:0/2760042165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf90100050 msgr2=0x7faf90101540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:43.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf8e59c700 1 --2- 172.21.15.40:0/2760042165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf90100050 0x7faf90101540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf8e59c700 1 -- 172.21.15.40:0/2760042165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf88009d00 con 0x7faf900fec10 2023-07-18T03:42:43.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf8dd9b700 1 --2- 172.21.15.40:0/2760042165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf900ff5c0 0x7faf90101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-18T03:42:43.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf8e59c700 1 --2- 172.21.15.40:0/2760042165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf900fec10 0x7faf901029c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7faf8000d950 tx=0x7faf8000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:43.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf7f7fe700 1 -- 172.21.15.40:0/2760042165 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80009940 con 0x7faf900fec10 2023-07-18T03:42:43.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf94a9c700 1 -- 172.21.15.40:0/2760042165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9019fc20 con 0x7faf900fec10 2023-07-18T03:42:43.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf7f7fe700 1 -- 172.21.15.40:0/2760042165 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf80010460 con 0x7faf900fec10 2023-07-18T03:42:43.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.343+0000 7faf7f7fe700 1 -- 172.21.15.40:0/2760042165 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8000f620 con 0x7faf900fec10 2023-07-18T03:42:43.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.344+0000 7faf94a9c700 1 -- 172.21.15.40:0/2760042165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf901a0130 con 0x7faf900fec10 2023-07-18T03:42:43.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.344+0000 7faf94a9c700 1 -- 172.21.15.40:0/2760042165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9010a1a0 con 0x7faf900fec10 2023-07-18T03:42:43.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.345+0000 7faf7f7fe700 1 -- 172.21.15.40:0/2760042165 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7faf800108d0 con 0x7faf900fec10 2023-07-18T03:42:43.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.346+0000 7faf7f7fe700 1 --2- 172.21.15.40:0/2760042165 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faf780a1150 0x7faf780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:43.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.346+0000 7faf7f7fe700 1 -- 172.21.15.40:0/2760042165 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7faf800cc7c0 con 0x7faf900fec10 2023-07-18T03:42:43.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.346+0000 7faf8dd9b700 1 --2- 172.21.15.40:0/2760042165 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faf780a1150 0x7faf780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:43.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.347+0000 7faf8dd9b700 1 --2- 172.21.15.40:0/2760042165 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faf780a1150 0x7faf780a3600 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faf84005950 tx=0x7faf840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:43.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.350+0000 7faf7f7fe700 1 -- 172.21.15.40:0/2760042165 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7faf80095ef0 con 0x7faf900fec10 2023-07-18T03:42:43.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:42 smithi064 ceph-mon[98059]: pgmap v649: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 12 op/s 2023-07-18T03:42:43.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.548+0000 7faf94a9c700 1 -- 172.21.15.40:0/2760042165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf901a06e0 con 0x7faf900fec10 2023-07-18T03:42:43.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.549+0000 7faf7f7fe700 1 -- 172.21.15.40:0/2760042165 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf80020020 con 0x7faf900fec10 2023-07-18T03:42:43.543 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-07-18T03:42:43.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.551+0000 7faf94a9c700 1 -- 172.21.15.40:0/2760042165 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faf780a1150 msgr2=0x7faf780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.551+0000 7faf94a9c700 1 --2- 172.21.15.40:0/2760042165 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faf780a1150 0x7faf780a3600 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faf84005950 tx=0x7faf840058e0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.551+0000 7faf94a9c700 1 -- 172.21.15.40:0/2760042165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf900fec10 msgr2=0x7faf901029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.551+0000 7faf94a9c700 1 --2- 172.21.15.40:0/2760042165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf900fec10 0x7faf901029c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7faf8000d950 tx=0x7faf8000dc60 comp rx=0 tx=0).stop 2023-07-18T03:42:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.551+0000 7faf94a9c700 1 -- 172.21.15.40:0/2760042165 shutdown_connections 2023-07-18T03:42:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.551+0000 7faf94a9c700 1 --2- 172.21.15.40:0/2760042165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf90100050 0x7faf90101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.551+0000 7faf94a9c700 1 --2- 172.21.15.40:0/2760042165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf900fec10 0x7faf901029c0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.551+0000 7faf94a9c700 1 --2- 172.21.15.40:0/2760042165 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faf780a1150 0x7faf780a3600 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.551+0000 7faf94a9c700 1 --2- 172.21.15.40:0/2760042165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf900ff5c0 0x7faf90101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.551+0000 7faf94a9c700 1 -- 172.21.15.40:0/2760042165 >> 172.21.15.40:0/2760042165 conn(0x7faf900f9770 msgr2=0x7faf900fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:43.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.551+0000 7faf94a9c700 1 -- 172.21.15.40:0/2760042165 shutdown_connections 2023-07-18T03:42:43.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.551+0000 7faf94a9c700 1 -- 172.21.15.40:0/2760042165 wait complete. 2023-07-18T03:42:43.557 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-18T03:42:43.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.692+0000 7f7da283c700 1 -- 172.21.15.40:0/3132655969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d9c100560 msgr2=0x7f7d9c106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:43.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.692+0000 7f7da283c700 1 --2- 172.21.15.40:0/3132655969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d9c100560 0x7f7d9c106e20 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400b0a0 tx=0x7f7d9400b3b0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.692+0000 7f7da283c700 1 -- 172.21.15.40:0/3132655969 shutdown_connections 2023-07-18T03:42:43.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.692+0000 7f7da283c700 1 --2- 172.21.15.40:0/3132655969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d9c100560 0x7f7d9c106e20 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.692+0000 7f7da283c700 1 --2- 172.21.15.40:0/3132655969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d9c0fe9e0 0x7f7d9c0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.692+0000 7f7da283c700 1 --2- 172.21.15.40:0/3132655969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d9c0ffbd0 0x7f7d9c100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.692+0000 7f7da283c700 1 -- 172.21.15.40:0/3132655969 >> 172.21.15.40:0/3132655969 conn(0x7f7d9c0fa000 msgr2=0x7f7d9c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:43.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.692+0000 7f7da283c700 1 -- 172.21.15.40:0/3132655969 shutdown_connections 2023-07-18T03:42:43.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.693+0000 7f7da283c700 1 -- 172.21.15.40:0/3132655969 wait complete. 2023-07-18T03:42:43.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.693+0000 7f7da283c700 1 Processor -- start 2023-07-18T03:42:43.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.693+0000 7f7da283c700 1 -- start start 2023-07-18T03:42:43.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.693+0000 7f7da283c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d9c0fe9e0 0x7f7d9c0681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:43.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.693+0000 7f7da283c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d9c0ffbd0 0x7f7d9c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.693+0000 7f7da283c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d9c100560 0x7f7d9c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.693+0000 7f7da283c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c19d440 con 0x7f7d9c0ffbd0 2023-07-18T03:42:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.693+0000 7f7da283c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c19d5b0 con 0x7f7d9c100560 2023-07-18T03:42:43.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.693+0000 7f7da283c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c19d770 con 0x7f7d9c0fe9e0 2023-07-18T03:42:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.694+0000 7f7d9b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d9c0ffbd0 0x7f7d9c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.694+0000 7f7d9b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d9c0ffbd0 0x7f7d9c19a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33878/0 (socket says 172.21.15.40:33878) 2023-07-18T03:42:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.694+0000 7f7d9b7fe700 1 -- 172.21.15.40:0/3206626385 learned_addr learned my addr 172.21.15.40:0/3206626385 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:43.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.694+0000 7f7d9bfff700 1 --2- 172.21.15.40:0/3206626385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d9c0fe9e0 0x7f7d9c0681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:43.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.694+0000 7f7da0dd9700 1 --2- 172.21.15.40:0/3206626385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d9c100560 0x7f7d9c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:43.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.694+0000 7f7d9b7fe700 1 -- 172.21.15.40:0/3206626385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d9c0fe9e0 msgr2=0x7f7d9c0681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:43.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.694+0000 7f7d9b7fe700 1 --2- 172.21.15.40:0/3206626385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d9c0fe9e0 0x7f7d9c0681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.694+0000 7f7d9b7fe700 1 -- 172.21.15.40:0/3206626385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d9c100560 msgr2=0x7f7d9c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:43.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.694+0000 7f7d9b7fe700 1 --2- 172.21.15.40:0/3206626385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d9c100560 0x7f7d9c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.694+0000 7f7d9b7fe700 1 -- 172.21.15.40:0/3206626385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d94009d00 con 0x7f7d9c0ffbd0 2023-07-18T03:42:43.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.694+0000 7f7d9b7fe700 1 --2- 172.21.15.40:0/3206626385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d9c0ffbd0 0x7f7d9c19a660 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000d950 tx=0x7f7d9000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:43.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.694+0000 7f7d997fa700 1 -- 172.21.15.40:0/3206626385 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d90009940 con 0x7f7d9c0ffbd0 2023-07-18T03:42:43.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.694+0000 7f7d997fa700 1 -- 172.21.15.40:0/3206626385 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d90010460 con 0x7f7d9c0ffbd0 2023-07-18T03:42:43.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.694+0000 7f7da283c700 1 -- 172.21.15.40:0/3206626385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9c19da50 con 0x7f7d9c0ffbd0 2023-07-18T03:42:43.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.694+0000 7f7d997fa700 1 -- 172.21.15.40:0/3206626385 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9000f620 con 0x7f7d9c0ffbd0 2023-07-18T03:42:43.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.694+0000 7f7da283c700 1 -- 172.21.15.40:0/3206626385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d9c19df60 con 0x7f7d9c0ffbd0 2023-07-18T03:42:43.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.695+0000 7f7da283c700 1 -- 172.21.15.40:0/3206626385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9c10a1c0 con 0x7f7d9c0ffbd0 2023-07-18T03:42:43.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.697+0000 7f7d997fa700 1 -- 172.21.15.40:0/3206626385 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7d900108d0 con 0x7f7d9c0ffbd0 2023-07-18T03:42:43.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.697+0000 7f7d997fa700 1 --2- 172.21.15.40:0/3206626385 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7d840a1150 0x7f7d840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:43.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.697+0000 7f7d997fa700 1 -- 172.21.15.40:0/3206626385 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7d900cc7c0 con 0x7f7d9c0ffbd0 2023-07-18T03:42:43.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.697+0000 7f7d9bfff700 1 --2- 172.21.15.40:0/3206626385 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7d840a1150 0x7f7d840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:43.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.698+0000 7f7d9bfff700 1 --2- 172.21.15.40:0/3206626385 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7d840a1150 0x7f7d840a3600 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0ffa30 tx=0x7f7d8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:43.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.701+0000 7f7d997fa700 1 -- 172.21.15.40:0/3206626385 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7d90095ef0 con 0x7f7d9c0ffbd0 2023-07-18T03:42:43.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.898+0000 7f7da283c700 1 -- 172.21.15.40:0/3206626385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d9c105890 con 0x7f7d9c0ffbd0 2023-07-18T03:42:43.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.899+0000 7f7d997fa700 1 -- 172.21.15.40:0/3206626385 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d90016020 con 0x7f7d9c0ffbd0 2023-07-18T03:42:43.893 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-07-18T03:42:43.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.901+0000 7f7da283c700 1 -- 172.21.15.40:0/3206626385 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7d840a1150 msgr2=0x7f7d840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:43.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.901+0000 7f7da283c700 1 --2- 172.21.15.40:0/3206626385 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7d840a1150 0x7f7d840a3600 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c0ffa30 tx=0x7f7d8c009260 comp rx=0 tx=0).stop 2023-07-18T03:42:43.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.901+0000 7f7da283c700 1 -- 172.21.15.40:0/3206626385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d9c0ffbd0 msgr2=0x7f7d9c19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:43.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.901+0000 7f7da283c700 1 --2- 172.21.15.40:0/3206626385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d9c0ffbd0 0x7f7d9c19a660 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000d950 tx=0x7f7d9000dc60 comp rx=0 tx=0).stop 2023-07-18T03:42:43.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.901+0000 7f7da283c700 1 -- 172.21.15.40:0/3206626385 shutdown_connections 2023-07-18T03:42:43.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.901+0000 7f7da283c700 1 --2- 172.21.15.40:0/3206626385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d9c100560 0x7f7d9c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.901+0000 7f7da283c700 1 --2- 172.21.15.40:0/3206626385 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7d840a1150 0x7f7d840a3600 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.901+0000 7f7da283c700 1 --2- 172.21.15.40:0/3206626385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7d9c0fe9e0 0x7f7d9c0681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.901+0000 7f7da283c700 1 --2- 172.21.15.40:0/3206626385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d9c0ffbd0 0x7f7d9c19a660 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:43.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.901+0000 7f7da283c700 1 -- 172.21.15.40:0/3206626385 >> 172.21.15.40:0/3206626385 conn(0x7f7d9c0fa000 msgr2=0x7f7d9c0fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:43.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.901+0000 7f7da283c700 1 -- 172.21.15.40:0/3206626385 shutdown_connections 2023-07-18T03:42:43.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:43.901+0000 7f7da283c700 1 -- 172.21.15.40:0/3206626385 wait complete. 2023-07-18T03:42:43.907 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-18T03:42:44.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.042+0000 7f2ff4c37700 1 -- 172.21.15.40:0/4059399737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff00ffcd0 msgr2=0x7f2ff0106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:44.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.042+0000 7f2ff4c37700 1 --2- 172.21.15.40:0/4059399737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff00ffcd0 0x7f2ff0106590 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800c210 tx=0x7f2fe800c520 comp rx=0 tx=0).stop 2023-07-18T03:42:44.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.042+0000 7f2ff4c37700 1 -- 172.21.15.40:0/4059399737 shutdown_connections 2023-07-18T03:42:44.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.042+0000 7f2ff4c37700 1 --2- 172.21.15.40:0/4059399737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff00ffcd0 0x7f2ff0106590 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.042+0000 7f2ff4c37700 1 --2- 172.21.15.40:0/4059399737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ff00fe150 0x7f2ff00fe560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.042+0000 7f2ff4c37700 1 --2- 172.21.15.40:0/4059399737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ff00ff340 0x7f2ff00ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.042+0000 7f2ff4c37700 1 -- 172.21.15.40:0/4059399737 >> 172.21.15.40:0/4059399737 conn(0x7f2ff00f9770 msgr2=0x7f2ff00fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:44.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.042+0000 7f2ff4c37700 1 -- 172.21.15.40:0/4059399737 shutdown_connections 2023-07-18T03:42:44.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.042+0000 7f2ff4c37700 1 -- 172.21.15.40:0/4059399737 wait complete. 2023-07-18T03:42:44.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.043+0000 7f2ff4c37700 1 Processor -- start 2023-07-18T03:42:44.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.043+0000 7f2ff4c37700 1 -- start start 2023-07-18T03:42:44.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.044+0000 7f2ff4c37700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ff00fe150 0x7f2ff01936f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:44.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.044+0000 7f2ff4c37700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ff00ff340 0x7f2ff0193c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:44.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.044+0000 7f2ff4c37700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff00ffcd0 0x7f2ff019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:44.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.044+0000 7f2ff4c37700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff019b270 con 0x7f2ff00ffcd0 2023-07-18T03:42:44.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.044+0000 7f2ff4c37700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff019b3b0 con 0x7f2ff00fe150 2023-07-18T03:42:44.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.044+0000 7f2ff4c37700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff019b540 con 0x7f2ff00ff340 2023-07-18T03:42:44.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.044+0000 7f2feed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff00ffcd0 0x7f2ff019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:44.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.044+0000 7f2feed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff00ffcd0 0x7f2ff019aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33894/0 (socket says 172.21.15.40:33894) 2023-07-18T03:42:44.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.044+0000 7f2feed9d700 1 -- 172.21.15.40:0/4139279416 learned_addr learned my addr 172.21.15.40:0/4139279416 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:44.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.044+0000 7f2fee59c700 1 --2- 172.21.15.40:0/4139279416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ff00fe150 0x7f2ff01936f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:44.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.044+0000 7f2fedd9b700 1 --2- 172.21.15.40:0/4139279416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ff00ff340 0x7f2ff0193c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:44.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.044+0000 7f2feed9d700 1 -- 172.21.15.40:0/4139279416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ff00ff340 msgr2=0x7f2ff0193c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:44.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.044+0000 7f2feed9d700 1 --2- 172.21.15.40:0/4139279416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ff00ff340 0x7f2ff0193c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.044+0000 7f2feed9d700 1 -- 172.21.15.40:0/4139279416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ff00fe150 msgr2=0x7f2ff01936f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:44.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.044+0000 7f2feed9d700 1 --2- 172.21.15.40:0/4139279416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ff00fe150 0x7f2ff01936f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.044+0000 7f2feed9d700 1 -- 172.21.15.40:0/4139279416 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe800ae10 con 0x7f2ff00ffcd0 2023-07-18T03:42:44.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.044+0000 7f2feed9d700 1 --2- 172.21.15.40:0/4139279416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff00ffcd0 0x7f2ff019aca0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8008b10 tx=0x7f2fe8004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:44.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.045+0000 7f2fdf7fe700 1 -- 172.21.15.40:0/4139279416 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe800f040 con 0x7f2ff00ffcd0 2023-07-18T03:42:44.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.045+0000 7f2ff4c37700 1 -- 172.21.15.40:0/4139279416 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff019b7c0 con 0x7f2ff00ffcd0 2023-07-18T03:42:44.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.045+0000 7f2fdf7fe700 1 -- 172.21.15.40:0/4139279416 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fe801e940 con 0x7f2ff00ffcd0 2023-07-18T03:42:44.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.045+0000 7f2fdf7fe700 1 -- 172.21.15.40:0/4139279416 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80056a0 con 0x7f2ff00ffcd0 2023-07-18T03:42:44.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.045+0000 7f2ff4c37700 1 -- 172.21.15.40:0/4139279416 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff019bbe0 con 0x7f2ff00ffcd0 2023-07-18T03:42:44.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.046+0000 7f2ff4c37700 1 -- 172.21.15.40:0/4139279416 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd0005320 con 0x7f2ff00ffcd0 2023-07-18T03:42:44.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.047+0000 7f2fdf7fe700 1 -- 172.21.15.40:0/4139279416 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2fe8005800 con 0x7f2ff00ffcd0 2023-07-18T03:42:44.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.048+0000 7f2fdf7fe700 1 --2- 172.21.15.40:0/4139279416 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2fd80a1150 0x7f2fd80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:44.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.048+0000 7f2fee59c700 1 --2- 172.21.15.40:0/4139279416 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2fd80a1150 0x7f2fd80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:44.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.048+0000 7f2fdf7fe700 1 -- 172.21.15.40:0/4139279416 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f2fe80cf4c0 con 0x7f2ff00ffcd0 2023-07-18T03:42:44.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.048+0000 7f2fee59c700 1 --2- 172.21.15.40:0/4139279416 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2fd80a1150 0x7f2fd80a3600 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2ff00ff1a0 tx=0x7f2fe000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:44.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.051+0000 7f2fdf7fe700 1 -- 172.21.15.40:0/4139279416 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2fe8098c80 con 0x7f2ff00ffcd0 2023-07-18T03:42:44.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.252+0000 7f2ff4c37700 1 -- 172.21.15.40:0/4139279416 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2fd0005f70 con 0x7f2ff00ffcd0 2023-07-18T03:42:44.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.252+0000 7f2fdf7fe700 1 -- 172.21.15.40:0/4139279416 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2fe8005040 con 0x7f2ff00ffcd0 2023-07-18T03:42:44.246 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-07-18T03:42:44.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.254+0000 7f2ff4c37700 1 -- 172.21.15.40:0/4139279416 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2fd80a1150 msgr2=0x7f2fd80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:44.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.254+0000 7f2ff4c37700 1 --2- 172.21.15.40:0/4139279416 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2fd80a1150 0x7f2fd80a3600 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2ff00ff1a0 tx=0x7f2fe000a3b0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.254+0000 7f2ff4c37700 1 -- 172.21.15.40:0/4139279416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff00ffcd0 msgr2=0x7f2ff019aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:44.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.254+0000 7f2ff4c37700 1 --2- 172.21.15.40:0/4139279416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff00ffcd0 0x7f2ff019aca0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8008b10 tx=0x7f2fe8004f60 comp rx=0 tx=0).stop 2023-07-18T03:42:44.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.254+0000 7f2ff4c37700 1 -- 172.21.15.40:0/4139279416 shutdown_connections 2023-07-18T03:42:44.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.254+0000 7f2ff4c37700 1 --2- 172.21.15.40:0/4139279416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff00ffcd0 0x7f2ff019aca0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.254+0000 7f2ff4c37700 1 --2- 172.21.15.40:0/4139279416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ff00fe150 0x7f2ff01936f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.254+0000 7f2ff4c37700 1 --2- 172.21.15.40:0/4139279416 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f2fd80a1150 0x7f2fd80a3600 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.254+0000 7f2ff4c37700 1 --2- 172.21.15.40:0/4139279416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ff00ff340 0x7f2ff0193c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.254+0000 7f2ff4c37700 1 -- 172.21.15.40:0/4139279416 >> 172.21.15.40:0/4139279416 conn(0x7f2ff00f9770 msgr2=0x7f2ff00fb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:44.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.254+0000 7f2ff4c37700 1 -- 172.21.15.40:0/4139279416 shutdown_connections 2023-07-18T03:42:44.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.254+0000 7f2ff4c37700 1 -- 172.21.15.40:0/4139279416 wait complete. 2023-07-18T03:42:44.259 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:43 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3008676905' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:42:44.259 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:43 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/2760042165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:42:44.260 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:43 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3206626385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:42:44.261 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-18T03:42:44.313 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:43 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3008676905' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:42:44.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:43 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/2760042165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:42:44.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:43 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3206626385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:42:44.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:43 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3008676905' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:42:44.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:43 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/2760042165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-18T03:42:44.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:43 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3206626385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-18T03:42:44.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.393+0000 7f7b1bb97700 1 -- 172.21.15.40:0/976107481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b140fea10 msgr2=0x7f7b140fee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:44.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.393+0000 7f7b1bb97700 1 --2- 172.21.15.40:0/976107481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b140fea10 0x7f7b140fee20 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009990 tx=0x7f7b10009ca0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.394+0000 7f7b1bb97700 1 -- 172.21.15.40:0/976107481 shutdown_connections 2023-07-18T03:42:44.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.394+0000 7f7b1bb97700 1 --2- 172.21.15.40:0/976107481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b14100590 0x7f7b14106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.394+0000 7f7b1bb97700 1 --2- 172.21.15.40:0/976107481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b140fea10 0x7f7b140fee20 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.394+0000 7f7b1bb97700 1 --2- 172.21.15.40:0/976107481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b140ffc00 0x7f7b14100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.394+0000 7f7b1bb97700 1 -- 172.21.15.40:0/976107481 >> 172.21.15.40:0/976107481 conn(0x7f7b140fa030 msgr2=0x7f7b140fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:44.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.394+0000 7f7b1bb97700 1 -- 172.21.15.40:0/976107481 shutdown_connections 2023-07-18T03:42:44.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.394+0000 7f7b1bb97700 1 -- 172.21.15.40:0/976107481 wait complete. 2023-07-18T03:42:44.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.394+0000 7f7b1bb97700 1 Processor -- start 2023-07-18T03:42:44.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.394+0000 7f7b1bb97700 1 -- start start 2023-07-18T03:42:44.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.394+0000 7f7b1bb97700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b140fea10 0x7f7b141947a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:44.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.394+0000 7f7b1bb97700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b140ffc00 0x7f7b14194ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:44.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.394+0000 7f7b1bb97700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b14100590 0x7f7b1419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:44.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.394+0000 7f7b1bb97700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1419c320 con 0x7f7b140fea10 2023-07-18T03:42:44.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.394+0000 7f7b1bb97700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1419c460 con 0x7f7b140ffc00 2023-07-18T03:42:44.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.394+0000 7f7b1bb97700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1419c5f0 con 0x7f7b14100590 2023-07-18T03:42:44.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b19933700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b140fea10 0x7f7b141947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:44.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b19933700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b140fea10 0x7f7b141947a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33904/0 (socket says 172.21.15.40:33904) 2023-07-18T03:42:44.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b19933700 1 -- 172.21.15.40:0/310000735 learned_addr learned my addr 172.21.15.40:0/310000735 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:44.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b19132700 1 --2- 172.21.15.40:0/310000735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b140ffc00 0x7f7b14194ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:44.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b1a134700 1 --2- 172.21.15.40:0/310000735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b14100590 0x7f7b1419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:44.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b19933700 1 -- 172.21.15.40:0/310000735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b14100590 msgr2=0x7f7b1419bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:44.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b19933700 1 --2- 172.21.15.40:0/310000735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b14100590 0x7f7b1419bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b19933700 1 -- 172.21.15.40:0/310000735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b140ffc00 msgr2=0x7f7b14194ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:44.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b19933700 1 --2- 172.21.15.40:0/310000735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b140ffc00 0x7f7b14194ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b19933700 1 -- 172.21.15.40:0/310000735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b100095f0 con 0x7f7b140fea10 2023-07-18T03:42:44.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b19132700 1 --2- 172.21.15.40:0/310000735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b140ffc00 0x7f7b14194ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:42:44.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b1a134700 1 --2- 172.21.15.40:0/310000735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b14100590 0x7f7b1419bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:42:44.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b19933700 1 --2- 172.21.15.40:0/310000735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b140fea10 0x7f7b141947a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f7b10004760 tx=0x7f7b10004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:44.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b0affd700 1 -- 172.21.15.40:0/310000735 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1001d070 con 0x7f7b140fea10 2023-07-18T03:42:44.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b1bb97700 1 -- 172.21.15.40:0/310000735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1419c870 con 0x7f7b140fea10 2023-07-18T03:42:44.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b0affd700 1 -- 172.21.15.40:0/310000735 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b100041d0 con 0x7f7b140fea10 2023-07-18T03:42:44.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b0affd700 1 -- 172.21.15.40:0/310000735 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b100053e0 con 0x7f7b140fea10 2023-07-18T03:42:44.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.395+0000 7f7b1bb97700 1 -- 172.21.15.40:0/310000735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1419cc90 con 0x7f7b140fea10 2023-07-18T03:42:44.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.396+0000 7f7b1bb97700 1 -- 172.21.15.40:0/310000735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b1410a230 con 0x7f7b140fea10 2023-07-18T03:42:44.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.397+0000 7f7b0affd700 1 -- 172.21.15.40:0/310000735 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7b10005540 con 0x7f7b140fea10 2023-07-18T03:42:44.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.398+0000 7f7b0affd700 1 --2- 172.21.15.40:0/310000735 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7b000a1150 0x7f7b000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:44.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.398+0000 7f7b0affd700 1 -- 172.21.15.40:0/310000735 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f7b100ce8c0 con 0x7f7b140fea10 2023-07-18T03:42:44.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.398+0000 7f7b19132700 1 --2- 172.21.15.40:0/310000735 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7b000a1150 0x7f7b000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:44.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.399+0000 7f7b19132700 1 --2- 172.21.15.40:0/310000735 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7b000a1150 0x7f7b000a3600 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7b04005fd0 tx=0x7f7b04005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:44.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.401+0000 7f7b0affd700 1 -- 172.21.15.40:0/310000735 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7b10098080 con 0x7f7b140fea10 2023-07-18T03:42:44.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.618+0000 7f7b1bb97700 1 -- 172.21.15.40:0/310000735 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7b141058f0 con 0x7f7b140fea10 2023-07-18T03:42:44.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.618+0000 7f7b0affd700 1 -- 172.21.15.40:0/310000735 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v84) v1 ==== 81+0+590 (secure 0 0 0) 0x7f7b1009b6a0 con 0x7f7b140fea10 2023-07-18T03:42:44.612 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/2185187717 2023-07-19T03:23:31.734610+0000 2023-07-18T03:42:44.612 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/908447335 2023-07-19T03:23:31.734610+0000 2023-07-18T03:42:44.613 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6800/1112125515 2023-07-19T03:23:31.734610+0000 2023-07-18T03:42:44.613 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6801/1112125515 2023-07-19T03:23:31.734610+0000 2023-07-18T03:42:44.613 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/3890436231 2023-07-19T03:23:05.482888+0000 2023-07-18T03:42:44.613 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/3748912525 2023-07-19T03:23:05.482888+0000 2023-07-18T03:42:44.613 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6800/1353038297 2023-07-19T03:23:05.482888+0000 2023-07-18T03:42:44.613 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/101292551 2023-07-19T03:23:05.482888+0000 2023-07-18T03:42:44.613 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:6801/1353038297 2023-07-19T03:23:05.482888+0000 2023-07-18T03:42:44.614 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.15.40:0/1171137337 2023-07-19T03:23:31.734610+0000 2023-07-18T03:42:44.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.620+0000 7f7b1bb97700 1 -- 172.21.15.40:0/310000735 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7b000a1150 msgr2=0x7f7b000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:44.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.620+0000 7f7b1bb97700 1 --2- 172.21.15.40:0/310000735 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7b000a1150 0x7f7b000a3600 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7b04005fd0 tx=0x7f7b04005dc0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.620+0000 7f7b1bb97700 1 -- 172.21.15.40:0/310000735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b140fea10 msgr2=0x7f7b141947a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:44.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.620+0000 7f7b1bb97700 1 --2- 172.21.15.40:0/310000735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b140fea10 0x7f7b141947a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f7b10004760 tx=0x7f7b10004790 comp rx=0 tx=0).stop 2023-07-18T03:42:44.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.621+0000 7f7b1bb97700 1 -- 172.21.15.40:0/310000735 shutdown_connections 2023-07-18T03:42:44.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.621+0000 7f7b1bb97700 1 --2- 172.21.15.40:0/310000735 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f7b000a1150 0x7f7b000a3600 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.621+0000 7f7b1bb97700 1 --2- 172.21.15.40:0/310000735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b14100590 0x7f7b1419bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.621+0000 7f7b1bb97700 1 --2- 172.21.15.40:0/310000735 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b140fea10 0x7f7b141947a0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.621+0000 7f7b1bb97700 1 --2- 172.21.15.40:0/310000735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7b140ffc00 0x7f7b14194ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:44.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.621+0000 7f7b1bb97700 1 -- 172.21.15.40:0/310000735 >> 172.21.15.40:0/310000735 conn(0x7f7b140fa030 msgr2=0x7f7b140fc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:44.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.621+0000 7f7b1bb97700 1 -- 172.21.15.40:0/310000735 shutdown_connections 2023-07-18T03:42:44.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:44.621+0000 7f7b1bb97700 1 -- 172.21.15.40:0/310000735 wait complete. 2023-07-18T03:42:44.616 INFO:teuthology.orchestra.run.smithi040.stderr:listed 10 entries 2023-07-18T03:42:44.626 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-18T03:42:44.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:42:44.627 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-07-18T03:42:44.763 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-07-18T03:42:44.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:42:44.763 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-18T03:42:44.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:42:44.869 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-07-18T03:42:44.963 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-18T03:42:44.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:42:44.964 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-18T03:42:45.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:42:45.054 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2023-07-18T03:42:45.138 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-18T03:42:45.138 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-07-18T03:42:45.139 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-18T03:42:45.158 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20495 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/0c3d73369df86d75ed75400380fc8213e93140fd1608d9c13615aebc865e2067/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/3KJABW7XLTG2M5DCS7TATYEGIA:/var/lib/containers/storage/overlay/l/2CPOEKXL24SAZGULBB2AHG7UKX,upperdir=/var/lib/containers/storage/overlay/0c3d73369df86d75ed75400380fc8213e93140fd1608d9c13615aebc865e2067/diff,workdir=/var/lib/containers/storage/overlay/0c3d73369df86d75ed75400380fc8213e93140fd1608d9c13615aebc865e2067/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c91fd9570f7bc830710c9b148bdc967870be825133ce78bf1f2cc9b4013a29ad/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/3KJABW7XLTG2M5DCS7TATYEGIA:/var/lib/containers/storage/overlay/l/2CPOEKXL24SAZGULBB2AHG7UKX,upperdir=/var/lib/containers/storage/overlay/c91fd9570f7bc830710c9b148bdc967870be825133ce78bf1f2cc9b4013a29ad/diff,workdir=/var/lib/containers/storage/overlay/c91fd9570f7bc830710c9b148bdc967870be825133ce78bf1f2cc9b4013a29ad/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/fc07bfb6e9536dccc350ed7785ed81ba3fef1766b83ced67e384e1ee1606516e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/3KJABW7XLTG2M5DCS7TATYEGIA:/var/lib/containers/storage/overlay/l/2CPOEKXL24SAZGULBB2AHG7UKX,upperdir=/var/lib/containers/storage/overlay/fc07bfb6e9536dccc350ed7785ed81ba3fef1766b83ced67e384e1ee1606516e/diff,workdir=/var/lib/containers/storage/overlay/fc07bfb6e9536dccc350ed7785ed81ba3fef1766b83ced67e384e1ee1606516e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/53cf91a027f6b48c011e5e97cffb963754fcec4c0b80e3336f47f14ebe4c5ec9/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/3KJABW7XLTG2M5DCS7TATYEGIA:/var/lib/containers/storage/overlay/l/2CPOEKXL24SAZGULBB2AHG7UKX,upperdir=/var/lib/containers/storage/overlay/53cf91a027f6b48c011e5e97cffb963754fcec4c0b80e3336f47f14ebe4c5ec9/diff,workdir=/var/lib/containers/storage/overlay/53cf91a027f6b48c011e5e97cffb963754fcec4c0b80e3336f47f14ebe4c5ec9/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/98e71f527313b02788a18e83ff773fdfa06f00d7353db89a77c6da92bbbdea84/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/3KJABW7XLTG2M5DCS7TATYEGIA:/var/lib/containers/storage/overlay/l/2CPOEKXL24SAZGULBB2AHG7UKX,upperdir=/var/lib/containers/storage/overlay/98e71f527313b02788a18e83ff773fdfa06f00d7353db89a77c6da92bbbdea84/diff,workdir=/var/lib/containers/storage/overlay/98e71f527313b02788a18e83ff773fdfa06f00d7353db89a77c6da92bbbdea84/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a95c389dee69d409f3af7a52d4b50f6e64f8b2f27ddcda9636f5d618fbe3d57e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/3KJABW7XLTG2M5DCS7TATYEGIA:/var/lib/containers/storage/overlay/l/2CPOEKXL24SAZGULBB2AHG7UKX,upperdir=/var/lib/containers/storage/overlay/a95c389dee69d409f3af7a52d4b50f6e64f8b2f27ddcda9636f5d618fbe3d57e/diff,workdir=/var/lib/containers/storage/overlay/a95c389dee69d409f3af7a52d4b50f6e64f8b2f27ddcda9636f5d618fbe3d57e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1d3f63a8713a9756a946151b0c79b4d91413789f63d8103db1fd0eeff5a57125/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/3KJABW7XLTG2M5DCS7TATYEGIA:/var/lib/containers/storage/overlay/l/2CPOEKXL24SAZGULBB2AHG7UKX,upperdir=/var/lib/containers/storage/overlay/1d3f63a8713a9756a946151b0c79b4d91413789f63d8103db1fd0eeff5a57125/diff,workdir=/var/lib/containers/storage/overlay/1d3f63a8713a9756a946151b0c79b4d91413789f63d8103db1fd0eeff5a57125/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e341212e2f0ddaf3495fe09754afba3608e2d54136818112ba35b309cec4eb23/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/3KJABW7XLTG2M5DCS7TATYEGIA:/var/lib/containers/storage/overlay/l/2CPOEKXL24SAZGULBB2AHG7UKX,upperdir=/var/lib/containers/storage/overlay/e341212e2f0ddaf3495fe09754afba3608e2d54136818112ba35b309cec4eb23/diff,workdir=/var/lib/containers/storage/overlay/e341212e2f0ddaf3495fe09754afba3608e2d54136818112ba35b309cec4eb23/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ac850e5e4f1961e3fc91116fd152af2b8d271689f73863b7d935588712f6c336/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/3KJABW7XLTG2M5DCS7TATYEGIA:/var/lib/containers/storage/overlay/l/2CPOEKXL24SAZGULBB2AHG7UKX,upperdir=/var/lib/containers/storage/overlay/ac850e5e4f1961e3fc91116fd152af2b8d271689f73863b7d935588712f6c336/diff,workdir=/var/lib/containers/storage/overlay/ac850e5e4f1961e3fc91116fd152af2b8d271689f73863b7d935588712f6c336/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/75c0af3331132562503e42ab841b45d51fb70d43da69df39d5c41b67cfaf98ec/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/3KJABW7XLTG2M5DCS7TATYEGIA:/var/lib/containers/storage/overlay/l/2CPOEKXL24SAZGULBB2AHG7UKX,upperdir=/var/lib/containers/storage/overlay/75c0af3331132562503e42ab841b45d51fb70d43da69df39d5c41b67cfaf98ec/diff,workdir=/var/lib/containers/storage/overlay/75c0af3331132562503e42ab841b45d51fb70d43da69df39d5c41b67cfaf98ec/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-07-18T03:42:45.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:42:45.159 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-07-18T03:42:45.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-18T03:42:45.216 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2023-07-18T03:42:45.280 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:45 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/4139279416' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:42:45.281 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:45 smithi040 ceph-mon[96439]: pgmap v650: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2023-07-18T03:42:45.281 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:45 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/310000735' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-18T03:42:45.286 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2023-07-18T03:42:45.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:42:45.288 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-07-18T03:42:45.305 INFO:tasks.cephadm:Teardown begin 2023-07-18T03:42:45.305 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-18T03:42:45.306 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:42:45.313 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:45 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/4139279416' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:42:45.313 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:45 smithi134 ceph-mon[98216]: pgmap v650: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2023-07-18T03:42:45.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:45 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/310000735' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-18T03:42:45.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:45 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/4139279416' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-18T03:42:45.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:45 smithi064 ceph-mon[98059]: pgmap v650: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2023-07-18T03:42:45.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:45 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/310000735' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-18T03:42:45.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.461+0000 7f82a547a700 1 -- 172.21.15.40:0/4227042894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82a00ff5e0 msgr2=0x7f82a0106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:45.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.461+0000 7f82a547a700 1 --2- 172.21.15.40:0/4227042894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82a00ff5e0 0x7f82a0106e90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f829800b0a0 tx=0x7f829800b3b0 comp rx=0 tx=0).stop 2023-07-18T03:42:45.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.461+0000 7f82a547a700 1 -- 172.21.15.40:0/4227042894 shutdown_connections 2023-07-18T03:42:45.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.461+0000 7f82a547a700 1 --2- 172.21.15.40:0/4227042894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82a00ff5e0 0x7f82a0106e90 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:45.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.461+0000 7f82a547a700 1 --2- 172.21.15.40:0/4227042894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82a00fffe0 0x7f82a01003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.461+0000 7f82a547a700 1 --2- 172.21.15.40:0/4227042894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82a00fec30 0x7f82a00ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.461+0000 7f82a547a700 1 -- 172.21.15.40:0/4227042894 >> 172.21.15.40:0/4227042894 conn(0x7f82a00f9770 msgr2=0x7f82a00fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.461+0000 7f82a547a700 1 -- 172.21.15.40:0/4227042894 shutdown_connections 2023-07-18T03:42:45.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.461+0000 7f82a547a700 1 -- 172.21.15.40:0/4227042894 wait complete. 2023-07-18T03:42:45.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.462+0000 7f82a547a700 1 Processor -- start 2023-07-18T03:42:45.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.462+0000 7f82a547a700 1 -- start start 2023-07-18T03:42:45.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.462+0000 7f82a547a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82a00fec30 0x7f82a0195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:45.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.462+0000 7f82a547a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82a00ff5e0 0x7f82a0195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:45.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.462+0000 7f82a547a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82a00fffe0 0x7f82a019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:45.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.462+0000 7f82a547a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a019d4c0 con 0x7f82a00fffe0 2023-07-18T03:42:45.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.462+0000 7f82a547a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a019d600 con 0x7f82a00fec30 2023-07-18T03:42:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.462+0000 7f82a547a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a019d790 con 0x7f82a00ff5e0 2023-07-18T03:42:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.462+0000 7f829f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82a00fffe0 0x7f82a019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.462+0000 7f829effd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82a00fec30 0x7f82a0195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.462+0000 7f829f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82a00fffe0 0x7f82a019cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33922/0 (socket says 172.21.15.40:33922) 2023-07-18T03:42:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.462+0000 7f829f7fe700 1 -- 172.21.15.40:0/1783859684 learned_addr learned my addr 172.21.15.40:0/1783859684 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.462+0000 7f829e7fc700 1 --2- 172.21.15.40:0/1783859684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82a00ff5e0 0x7f82a0195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:45.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.463+0000 7f829effd700 1 -- 172.21.15.40:0/1783859684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82a00ff5e0 msgr2=0x7f82a0195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:45.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.463+0000 7f829effd700 1 --2- 172.21.15.40:0/1783859684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82a00ff5e0 0x7f82a0195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:45.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.463+0000 7f829effd700 1 -- 172.21.15.40:0/1783859684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82a00fffe0 msgr2=0x7f82a019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:45.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.463+0000 7f829effd700 1 --2- 172.21.15.40:0/1783859684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82a00fffe0 0x7f82a019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:45.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.463+0000 7f829effd700 1 -- 172.21.15.40:0/1783859684 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8298009d00 con 0x7f82a00fec30 2023-07-18T03:42:45.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.463+0000 7f829f7fe700 1 --2- 172.21.15.40:0/1783859684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82a00fffe0 0x7f82a019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:42:45.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.463+0000 7f829effd700 1 --2- 172.21.15.40:0/1783859684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82a00fec30 0x7f82a0195940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f829400e9c0 tx=0x7f829400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:45.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.463+0000 7f8287fff700 1 -- 172.21.15.40:0/1783859684 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294004220 con 0x7f82a00fec30 2023-07-18T03:42:45.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.463+0000 7f82a547a700 1 -- 172.21.15.40:0/1783859684 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a019da70 con 0x7f82a00fec30 2023-07-18T03:42:45.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.463+0000 7f82a547a700 1 -- 172.21.15.40:0/1783859684 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a019dfd0 con 0x7f82a00fec30 2023-07-18T03:42:45.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.463+0000 7f8287fff700 1 -- 172.21.15.40:0/1783859684 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8294004380 con 0x7f82a00fec30 2023-07-18T03:42:45.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.463+0000 7f8287fff700 1 -- 172.21.15.40:0/1783859684 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294003be0 con 0x7f82a00fec30 2023-07-18T03:42:45.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.464+0000 7f82a547a700 1 -- 172.21.15.40:0/1783859684 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828c005320 con 0x7f82a00fec30 2023-07-18T03:42:45.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.465+0000 7f8287fff700 1 -- 172.21.15.40:0/1783859684 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8294004870 con 0x7f82a00fec30 2023-07-18T03:42:45.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.466+0000 7f8287fff700 1 --2- 172.21.15.40:0/1783859684 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f82880a1220 0x7f82880a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:45.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.466+0000 7f8287fff700 1 -- 172.21.15.40:0/1783859684 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f8294014070 con 0x7f82a00fec30 2023-07-18T03:42:45.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.466+0000 7f829e7fc700 1 --2- 172.21.15.40:0/1783859684 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f82880a1220 0x7f82880a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:45.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.466+0000 7f829e7fc700 1 --2- 172.21.15.40:0/1783859684 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f82880a1220 0x7f82880a36d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8290009c60 tx=0x7f8290009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:45.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.469+0000 7f8287fff700 1 -- 172.21.15.40:0/1783859684 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8294096860 con 0x7f82a00fec30 2023-07-18T03:42:45.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.713+0000 7f82a547a700 1 -- 172.21.15.40:0/1783859684 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f828c005cc0 con 0x7f82a00fec30 2023-07-18T03:42:45.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.714+0000 7f8287fff700 1 -- 172.21.15.40:0/1783859684 <== mon.2 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+10916 (secure 0 0 0) 0x7f8294099e80 con 0x7f82a00fec30 2023-07-18T03:42:45.708 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:42:45.708 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":75,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:29:26.718789+0000","modified":"2023-07-18T03:42:17.346304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24523,"mds_2":24482,"mds_3":24452,"mds_4":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14601":{"gid":14601,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1818167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1818167554},{"type":"v1","addr":"172.21.15.40:6839","nonce":1818167554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14613":{"gid":14613,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/3953074960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":3953074960},{"type":"v1","addr":"172.21.15.40:6841","nonce":3953074960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":52,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/3007039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3007039043},{"type":"v1","addr":"172.21.15.64:6839","nonce":3007039043}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25972":{"gid":25972,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3831344228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3831344228},{"type":"v1","addr":"172.21.15.134:6837","nonce":3831344228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:42:45.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.716+0000 7f82a547a700 1 -- 172.21.15.40:0/1783859684 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f82880a1220 msgr2=0x7f82880a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:45.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.716+0000 7f82a547a700 1 --2- 172.21.15.40:0/1783859684 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f82880a1220 0x7f82880a36d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8290009c60 tx=0x7f8290009260 comp rx=0 tx=0).stop 2023-07-18T03:42:45.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.716+0000 7f82a547a700 1 -- 172.21.15.40:0/1783859684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82a00fec30 msgr2=0x7f82a0195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:45.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.716+0000 7f82a547a700 1 --2- 172.21.15.40:0/1783859684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82a00fec30 0x7f82a0195940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f829400e9c0 tx=0x7f829400ecd0 comp rx=0 tx=0).stop 2023-07-18T03:42:45.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.716+0000 7f82a547a700 1 -- 172.21.15.40:0/1783859684 shutdown_connections 2023-07-18T03:42:45.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.716+0000 7f82a547a700 1 --2- 172.21.15.40:0/1783859684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82a00fffe0 0x7f82a019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:45.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.716+0000 7f82a547a700 1 --2- 172.21.15.40:0/1783859684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f82a00fec30 0x7f82a0195940 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:45.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.716+0000 7f82a547a700 1 --2- 172.21.15.40:0/1783859684 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f82880a1220 0x7f82880a36d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:45.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.716+0000 7f82a547a700 1 --2- 172.21.15.40:0/1783859684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82a00ff5e0 0x7f82a0195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:45.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.716+0000 7f82a547a700 1 -- 172.21.15.40:0/1783859684 >> 172.21.15.40:0/1783859684 conn(0x7f82a00f9770 msgr2=0x7f82a00fb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:45.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.716+0000 7f82a547a700 1 -- 172.21.15.40:0/1783859684 shutdown_connections 2023-07-18T03:42:45.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.717+0000 7f82a547a700 1 -- 172.21.15.40:0/1783859684 wait complete. 2023-07-18T03:42:45.712 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 75 2023-07-18T03:42:45.723 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-18T03:42:45.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.855+0000 7f48a7132700 1 -- 172.21.15.40:0/3790337479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48a00fe9d0 msgr2=0x7f48a00fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:45.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.855+0000 7f48a7132700 1 --2- 172.21.15.40:0/3790337479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48a00fe9d0 0x7f48a00fede0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4894009990 tx=0x7f4894009ca0 comp rx=0 tx=0).stop 2023-07-18T03:42:45.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.855+0000 7f48a7132700 1 -- 172.21.15.40:0/3790337479 shutdown_connections 2023-07-18T03:42:45.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.855+0000 7f48a7132700 1 --2- 172.21.15.40:0/3790337479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48a0100550 0x7f48a0106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:45.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.855+0000 7f48a7132700 1 --2- 172.21.15.40:0/3790337479 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48a00fe9d0 0x7f48a00fede0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:45.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.855+0000 7f48a7132700 1 --2- 172.21.15.40:0/3790337479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48a00ffbc0 0x7f48a0100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:45.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.855+0000 7f48a7132700 1 -- 172.21.15.40:0/3790337479 >> 172.21.15.40:0/3790337479 conn(0x7f48a00f9fd0 msgr2=0x7f48a00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:45.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.855+0000 7f48a7132700 1 -- 172.21.15.40:0/3790337479 shutdown_connections 2023-07-18T03:42:45.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.855+0000 7f48a7132700 1 -- 172.21.15.40:0/3790337479 wait complete. 2023-07-18T03:42:45.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.856+0000 7f48a7132700 1 Processor -- start 2023-07-18T03:42:45.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.856+0000 7f48a7132700 1 -- start start 2023-07-18T03:42:45.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.856+0000 7f48a7132700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48a00fe9d0 0x7f48a01957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:45.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.856+0000 7f48a7132700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48a00ffbc0 0x7f48a0195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:45.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.856+0000 7f48a7132700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48a0100550 0x7f48a019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:45.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.856+0000 7f48a7132700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a019d350 con 0x7f48a0100550 2023-07-18T03:42:45.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.856+0000 7f48a7132700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a019d490 con 0x7f48a00ffbc0 2023-07-18T03:42:45.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.856+0000 7f48a7132700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a019d620 con 0x7f48a00fe9d0 2023-07-18T03:42:45.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.856+0000 7f48a56cf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48a0100550 0x7f48a019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:45.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.856+0000 7f48a56cf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48a0100550 0x7f48a019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33936/0 (socket says 172.21.15.40:33936) 2023-07-18T03:42:45.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.857+0000 7f48a56cf700 1 -- 172.21.15.40:0/3776280717 learned_addr learned my addr 172.21.15.40:0/3776280717 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.857+0000 7f48a4ece700 1 --2- 172.21.15.40:0/3776280717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48a00fe9d0 0x7f48a01957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.857+0000 7f489ffff700 1 --2- 172.21.15.40:0/3776280717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48a00ffbc0 0x7f48a0195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.857+0000 7f48a56cf700 1 -- 172.21.15.40:0/3776280717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48a00fe9d0 msgr2=0x7f48a01957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.857+0000 7f48a56cf700 1 --2- 172.21.15.40:0/3776280717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48a00fe9d0 0x7f48a01957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.857+0000 7f48a56cf700 1 -- 172.21.15.40:0/3776280717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48a00ffbc0 msgr2=0x7f48a0195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.857+0000 7f48a56cf700 1 --2- 172.21.15.40:0/3776280717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48a00ffbc0 0x7f48a0195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.857+0000 7f48a56cf700 1 -- 172.21.15.40:0/3776280717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48940095f0 con 0x7f48a0100550 2023-07-18T03:42:45.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.857+0000 7f489ffff700 1 --2- 172.21.15.40:0/3776280717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48a00ffbc0 0x7f48a0195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:42:45.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.857+0000 7f48a56cf700 1 --2- 172.21.15.40:0/3776280717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48a0100550 0x7f48a019cd80 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4898007ee0 tx=0x7f489800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:45.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.857+0000 7f489dffb700 1 -- 172.21.15.40:0/3776280717 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489800ddd0 con 0x7f48a0100550 2023-07-18T03:42:45.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.857+0000 7f489dffb700 1 -- 172.21.15.40:0/3776280717 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f489800f040 con 0x7f48a0100550 2023-07-18T03:42:45.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.857+0000 7f48a7132700 1 -- 172.21.15.40:0/3776280717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a019d900 con 0x7f48a0100550 2023-07-18T03:42:45.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.857+0000 7f489dffb700 1 -- 172.21.15.40:0/3776280717 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48980048e0 con 0x7f48a0100550 2023-07-18T03:42:45.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.857+0000 7f48a7132700 1 -- 172.21.15.40:0/3776280717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a019ddc0 con 0x7f48a0100550 2023-07-18T03:42:45.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.858+0000 7f48a7132700 1 -- 172.21.15.40:0/3776280717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a010a1a0 con 0x7f48a0100550 2023-07-18T03:42:45.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.859+0000 7f489dffb700 1 -- 172.21.15.40:0/3776280717 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4898003a00 con 0x7f48a0100550 2023-07-18T03:42:45.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.860+0000 7f489dffb700 1 --2- 172.21.15.40:0/3776280717 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f48900a11a0 0x7f48900a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:45.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.860+0000 7f48a4ece700 1 --2- 172.21.15.40:0/3776280717 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f48900a11a0 0x7f48900a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:45.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.860+0000 7f489dffb700 1 -- 172.21.15.40:0/3776280717 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f48980cc280 con 0x7f48a0100550 2023-07-18T03:42:45.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.861+0000 7f48a4ece700 1 --2- 172.21.15.40:0/3776280717 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f48900a11a0 0x7f48900a3650 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4894006010 tx=0x7f4894005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:45.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:45.864+0000 7f489dffb700 1 -- 172.21.15.40:0/3776280717 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f489804e790 con 0x7f48a0100550 2023-07-18T03:42:46.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.112+0000 7f48a7132700 1 -- 172.21.15.40:0/3776280717 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48a010a390 con 0x7f48a0100550 2023-07-18T03:42:46.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.112+0000 7f489dffb700 1 -- 172.21.15.40:0/3776280717 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+10916 (secure 0 0 0) 0x7f48980959b0 con 0x7f48a0100550 2023-07-18T03:42:46.106 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:42:46.106 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/646383396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":646383396},{"type":"v1","addr":"172.21.15.64:6833","nonce":646383396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2289812880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2289812880},{"type":"v1","addr":"172.21.15.40:6835","nonce":2289812880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":75,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-18T03:29:26.718789+0000","modified":"2023-07-18T03:42:17.346304+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24527,"mds_1":24523,"mds_2":24482,"mds_3":24452,"mds_4":24442},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14538":{"gid":14538,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6837/3377503144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":3377503144},{"type":"v1","addr":"172.21.15.40:6837","nonce":3377503144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14601":{"gid":14601,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6839/1818167554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1818167554},{"type":"v1","addr":"172.21.15.40:6839","nonce":1818167554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14613":{"gid":14613,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.40:6841/3953074960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6840","nonce":3953074960},{"type":"v1","addr":"172.21.15.40:6841","nonce":3953074960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.134:6833/2345264306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":2345264306},{"type":"v1","addr":"172.21.15.134:6833","nonce":2345264306}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.64:6835/1864495009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1864495009},{"type":"v1","addr":"172.21.15.64:6835","nonce":1864495009}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/3077792296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3077792296},{"type":"v1","addr":"172.21.15.64:6837","nonce":3077792296}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6835/2946854745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2946854745},{"type":"v1","addr":"172.21.15.134:6835","nonce":2946854745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":52,"addr":"172.21.15.134:6839/3192254101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":3192254101},{"type":"v1","addr":"172.21.15.134:6839","nonce":3192254101}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/3007039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":3007039043},{"type":"v1","addr":"172.21.15.64:6839","nonce":3007039043}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25972":{"gid":25972,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3831344228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3831344228},{"type":"v1","addr":"172.21.15.134:6837","nonce":3831344228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-18T03:42:46.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.115+0000 7f48a7132700 1 -- 172.21.15.40:0/3776280717 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f48900a11a0 msgr2=0x7f48900a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:46.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.115+0000 7f48a7132700 1 --2- 172.21.15.40:0/3776280717 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f48900a11a0 0x7f48900a3650 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4894006010 tx=0x7f4894005960 comp rx=0 tx=0).stop 2023-07-18T03:42:46.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.115+0000 7f48a7132700 1 -- 172.21.15.40:0/3776280717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48a0100550 msgr2=0x7f48a019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:46.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.115+0000 7f48a7132700 1 --2- 172.21.15.40:0/3776280717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48a0100550 0x7f48a019cd80 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4898007ee0 tx=0x7f489800d3b0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.115+0000 7f48a7132700 1 -- 172.21.15.40:0/3776280717 shutdown_connections 2023-07-18T03:42:46.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.115+0000 7f48a7132700 1 --2- 172.21.15.40:0/3776280717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48a0100550 0x7f48a019cd80 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.115+0000 7f48a7132700 1 --2- 172.21.15.40:0/3776280717 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f48900a11a0 0x7f48900a3650 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.115+0000 7f48a7132700 1 --2- 172.21.15.40:0/3776280717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48a00fe9d0 0x7f48a01957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.115+0000 7f48a7132700 1 --2- 172.21.15.40:0/3776280717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48a00ffbc0 0x7f48a0195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.115+0000 7f48a7132700 1 -- 172.21.15.40:0/3776280717 >> 172.21.15.40:0/3776280717 conn(0x7f48a00f9fd0 msgr2=0x7f48a00fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.115+0000 7f48a7132700 1 -- 172.21.15.40:0/3776280717 shutdown_connections 2023-07-18T03:42:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.115+0000 7f48a7132700 1 -- 172.21.15.40:0/3776280717 wait complete. 2023-07-18T03:42:46.110 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 75 2023-07-18T03:42:46.121 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-18T03:42:46.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.252+0000 7f3cb1a37700 1 -- 172.21.15.40:0/675364677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cac102f00 msgr2=0x7f3cac1052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:46.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.252+0000 7f3cb1a37700 1 --2- 172.21.15.40:0/675364677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cac102f00 0x7f3cac1052e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009990 tx=0x7f3ca0009ca0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.253+0000 7f3cb1a37700 1 -- 172.21.15.40:0/675364677 shutdown_connections 2023-07-18T03:42:46.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.253+0000 7f3cb1a37700 1 --2- 172.21.15.40:0/675364677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cac105820 0x7f3cac107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.253+0000 7f3cb1a37700 1 --2- 172.21.15.40:0/675364677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cac1005e0 0x7f3cac1029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.253+0000 7f3cb1a37700 1 --2- 172.21.15.40:0/675364677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cac102f00 0x7f3cac1052e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.253+0000 7f3cb1a37700 1 -- 172.21.15.40:0/675364677 >> 172.21.15.40:0/675364677 conn(0x7f3cac0fa020 msgr2=0x7f3cac0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.253+0000 7f3cb1a37700 1 -- 172.21.15.40:0/675364677 shutdown_connections 2023-07-18T03:42:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.253+0000 7f3cb1a37700 1 -- 172.21.15.40:0/675364677 wait complete. 2023-07-18T03:42:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.253+0000 7f3cb1a37700 1 Processor -- start 2023-07-18T03:42:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3cb1a37700 1 -- start start 2023-07-18T03:42:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3cb1a37700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cac1005e0 0x7f3cac1957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3cb1a37700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cac102f00 0x7f3cac195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3cb1a37700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cac105820 0x7f3cac19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3cb1a37700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac19d350 con 0x7f3cac102f00 2023-07-18T03:42:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3cb1a37700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac19d490 con 0x7f3cac1005e0 2023-07-18T03:42:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3cb1a37700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac19d620 con 0x7f3cac105820 2023-07-18T03:42:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3caa7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cac102f00 0x7f3cac195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3caa7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cac102f00 0x7f3cac195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33948/0 (socket says 172.21.15.40:33948) 2023-07-18T03:42:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3caa7fc700 1 -- 172.21.15.40:0/159970979 learned_addr learned my addr 172.21.15.40:0/159970979 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3caaffd700 1 --2- 172.21.15.40:0/159970979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cac1005e0 0x7f3cac1957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:46.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3cab7fe700 1 --2- 172.21.15.40:0/159970979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cac105820 0x7f3cac19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:46.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3caa7fc700 1 -- 172.21.15.40:0/159970979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cac105820 msgr2=0x7f3cac19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:46.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3caa7fc700 1 --2- 172.21.15.40:0/159970979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cac105820 0x7f3cac19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3caa7fc700 1 -- 172.21.15.40:0/159970979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cac1005e0 msgr2=0x7f3cac1957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:46.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3caa7fc700 1 --2- 172.21.15.40:0/159970979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cac1005e0 0x7f3cac1957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3caa7fc700 1 -- 172.21.15.40:0/159970979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca00095f0 con 0x7f3cac102f00 2023-07-18T03:42:46.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.254+0000 7f3caaffd700 1 --2- 172.21.15.40:0/159970979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cac1005e0 0x7f3cac1957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:42:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.255+0000 7f3caa7fc700 1 --2- 172.21.15.40:0/159970979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cac102f00 0x7f3cac195d10 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0004730 tx=0x7f3ca0004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.255+0000 7f3cb0a35700 1 -- 172.21.15.40:0/159970979 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca001d070 con 0x7f3cac102f00 2023-07-18T03:42:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.255+0000 7f3cb1a37700 1 -- 172.21.15.40:0/159970979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cac19d8a0 con 0x7f3cac102f00 2023-07-18T03:42:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.255+0000 7f3cb0a35700 1 -- 172.21.15.40:0/159970979 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ca0004c90 con 0x7f3cac102f00 2023-07-18T03:42:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.255+0000 7f3cb0a35700 1 -- 172.21.15.40:0/159970979 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0005570 con 0x7f3cac102f00 2023-07-18T03:42:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.255+0000 7f3cb1a37700 1 -- 172.21.15.40:0/159970979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac19dc60 con 0x7f3cac102f00 2023-07-18T03:42:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.256+0000 7f3cb1a37700 1 -- 172.21.15.40:0/159970979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cac0fbb90 con 0x7f3cac102f00 2023-07-18T03:42:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.257+0000 7f3cb0a35700 1 -- 172.21.15.40:0/159970979 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3ca000fc70 con 0x7f3cac102f00 2023-07-18T03:42:46.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.257+0000 7f3cb0a35700 1 --2- 172.21.15.40:0/159970979 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c940a11a0 0x7f3c940a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.257+0000 7f3cb0a35700 1 -- 172.21.15.40:0/159970979 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f3ca00cdcf0 con 0x7f3cac102f00 2023-07-18T03:42:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.257+0000 7f3caaffd700 1 --2- 172.21.15.40:0/159970979 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c940a11a0 0x7f3c940a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.258+0000 7f3caaffd700 1 --2- 172.21.15.40:0/159970979 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c940a11a0 0x7f3c940a3650 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c011490 tx=0x7f3c9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.262+0000 7f3cb0a35700 1 -- 172.21.15.40:0/159970979 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3ca005a790 con 0x7f3cac102f00 2023-07-18T03:42:46.313 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:46 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1783859684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:42:46.326 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:46 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1783859684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:42:46.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:46 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1783859684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:42:46.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.464+0000 7f3cb1a37700 1 -- 172.21.15.40:0/159970979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cac061120 con 0x7f3cac102f00 2023-07-18T03:42:46.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.465+0000 7f3cb0a35700 1 -- 172.21.15.40:0/159970979 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22599 (secure 0 0 0) 0x7f3ca00974b0 con 0x7f3cac102f00 2023-07-18T03:42:46.459 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-07-18T03:42:46.461 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":84,"fsid":"3af3dba8-251a-11ee-9b34-001a4aab830c","created":"2023-07-18T03:22:30.461503+0000","modified":"2023-07-18T03:35:46.881845+0000","last_up_change":"2023-07-18T03:28:45.683789+0000","last_in_change":"2023-07-18T03:28:28.115045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-18T03:25:38.267262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-18T03:29:24.727790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-18T03:29:26.078507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f924e504-7de2-48fb-820f-7b74c5a46ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6803","nonce":3043492254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6805","nonce":3043492254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6809","nonce":3043492254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":3043492254},{"type":"v1","addr":"172.21.15.40:6807","nonce":3043492254}]},"public_addr":"172.21.15.40:6803/3043492254","cluster_addr":"172.21.15.40:6805/3043492254","heartbeat_back_addr":"172.21.15.40:6809/3043492254","heartbeat_front_addr":"172.21.15.40:6807/3043492254","state":["exists","up"]},{"osd":1,"uuid":"ef6bdb13-52bd-40a2-bdbb-0dd2e6e0a049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6801","nonce":2992805245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6803","nonce":2992805245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6807","nonce":2992805245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":2992805245},{"type":"v1","addr":"172.21.15.64:6805","nonce":2992805245}]},"public_addr":"172.21.15.64:6801/2992805245","cluster_addr":"172.21.15.64:6803/2992805245","heartbeat_back_addr":"172.21.15.64:6807/2992805245","heartbeat_front_addr":"172.21.15.64:6805/2992805245","state":["exists","up"]},{"osd":2,"uuid":"e48fbcee-7d01-4dcf-845f-fc64645dca43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6801","nonce":1877339886}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6803","nonce":1877339886}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6807","nonce":1877339886}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1877339886},{"type":"v1","addr":"172.21.15.134:6805","nonce":1877339886}]},"public_addr":"172.21.15.134:6801/1877339886","cluster_addr":"172.21.15.134:6803/1877339886","heartbeat_back_addr":"172.21.15.134:6807/1877339886","heartbeat_front_addr":"172.21.15.134:6805/1877339886","state":["exists","up"]},{"osd":3,"uuid":"571c77cb-1acd-4d6f-a297-529dd5de162d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6811","nonce":3916741213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6813","nonce":3916741213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6817","nonce":3916741213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3916741213},{"type":"v1","addr":"172.21.15.40:6815","nonce":3916741213}]},"public_addr":"172.21.15.40:6811/3916741213","cluster_addr":"172.21.15.40:6813/3916741213","heartbeat_back_addr":"172.21.15.40:6817/3916741213","heartbeat_front_addr":"172.21.15.40:6815/3916741213","state":["exists","up"]},{"osd":4,"uuid":"0b567530-058b-44db-b773-7c6f7224d36a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6809","nonce":788347693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6811","nonce":788347693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6815","nonce":788347693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":788347693},{"type":"v1","addr":"172.21.15.64:6813","nonce":788347693}]},"public_addr":"172.21.15.64:6809/788347693","cluster_addr":"172.21.15.64:6811/788347693","heartbeat_back_addr":"172.21.15.64:6815/788347693","heartbeat_front_addr":"172.21.15.64:6813/788347693","state":["exists","up"]},{"osd":5,"uuid":"37241e63-af2a-400b-b5df-a15096c201e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6809","nonce":3544269956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6811","nonce":3544269956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6815","nonce":3544269956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3544269956},{"type":"v1","addr":"172.21.15.134:6813","nonce":3544269956}]},"public_addr":"172.21.15.134:6809/3544269956","cluster_addr":"172.21.15.134:6811/3544269956","heartbeat_back_addr":"172.21.15.134:6815/3544269956","heartbeat_front_addr":"172.21.15.134:6813/3544269956","state":["exists","up"]},{"osd":6,"uuid":"36ea6ecd-85aa-42cb-ab41-006ab774af0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6819","nonce":380513677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6821","nonce":380513677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6825","nonce":380513677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":380513677},{"type":"v1","addr":"172.21.15.40:6823","nonce":380513677}]},"public_addr":"172.21.15.40:6819/380513677","cluster_addr":"172.21.15.40:6821/380513677","heartbeat_back_addr":"172.21.15.40:6825/380513677","heartbeat_front_addr":"172.21.15.40:6823/380513677","state":["exists","up"]},{"osd":7,"uuid":"80a76d08-32dd-429e-b6e9-4a70a0f2b7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6817","nonce":589664358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6819","nonce":589664358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6823","nonce":589664358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":589664358},{"type":"v1","addr":"172.21.15.64:6821","nonce":589664358}]},"public_addr":"172.21.15.64:6817/589664358","cluster_addr":"172.21.15.64:6819/589664358","heartbeat_back_addr":"172.21.15.64:6823/589664358","heartbeat_front_addr":"172.21.15.64:6821/589664358","state":["exists","up"]},{"osd":8,"uuid":"a0442a0d-4350-4f98-a6a1-346905e55e52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6817","nonce":2602435562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6819","nonce":2602435562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6823","nonce":2602435562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":2602435562},{"type":"v1","addr":"172.21.15.134:6821","nonce":2602435562}]},"public_addr":"172.21.15.134:6817/2602435562","cluster_addr":"172.21.15.134:6819/2602435562","heartbeat_back_addr":"172.21.15.134:6823/2602435562","heartbeat_front_addr":"172.21.15.134:6821/2602435562","state":["exists","up"]},{"osd":9,"uuid":"56920a5e-7f3e-4776-921c-682aa946a6b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6827","nonce":2771612323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6829","nonce":2771612323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6833","nonce":2771612323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":2771612323},{"type":"v1","addr":"172.21.15.40:6831","nonce":2771612323}]},"public_addr":"172.21.15.40:6827/2771612323","cluster_addr":"172.21.15.40:6829/2771612323","heartbeat_back_addr":"172.21.15.40:6833/2771612323","heartbeat_front_addr":"172.21.15.40:6831/2771612323","state":["exists","up"]},{"osd":10,"uuid":"50f2740d-5e2a-43b5-9dd0-11d322238231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6825","nonce":2791256757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6827","nonce":2791256757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6831","nonce":2791256757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":2791256757},{"type":"v1","addr":"172.21.15.64:6829","nonce":2791256757}]},"public_addr":"172.21.15.64:6825/2791256757","cluster_addr":"172.21.15.64:6827/2791256757","heartbeat_back_addr":"172.21.15.64:6831/2791256757","heartbeat_front_addr":"172.21.15.64:6829/2791256757","state":["exists","up"]},{"osd":11,"uuid":"735e8994-4fce-4cab-a980-dbfe3259b2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6825","nonce":1938250059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6827","nonce":1938250059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6831","nonce":1938250059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1938250059},{"type":"v1","addr":"172.21.15.134:6829","nonce":1938250059}]},"public_addr":"172.21.15.134:6825/1938250059","cluster_addr":"172.21.15.134:6827/1938250059","heartbeat_back_addr":"172.21.15.134:6831/1938250059","heartbeat_front_addr":"172.21.15.134:6829/1938250059","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:24:53.530377+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:14.442539+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:35.239523+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:25:57.449986+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:18.596111+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:39.470442+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:26:59.442117+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:20.436072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:27:41.371080+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:02.369918+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:23.402996+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-18T03:28:44.007555+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:0/2185187717":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/908447335":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6800/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:6801/1112125515":"2023-07-19T03:23:31.734610+0000","172.21.15.40:0/3890436231":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/3748912525":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6800/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/101292551":"2023-07-19T03:23:05.482888+0000","172.21.15.40:6801/1353038297":"2023-07-19T03:23:05.482888+0000","172.21.15.40:0/1171137337":"2023-07-19T03:23:31.734610+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-18T03:42:46.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.468+0000 7f3cb1a37700 1 -- 172.21.15.40:0/159970979 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c940a11a0 msgr2=0x7f3c940a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.468+0000 7f3cb1a37700 1 --2- 172.21.15.40:0/159970979 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c940a11a0 0x7f3c940a3650 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c011490 tx=0x7f3c9c009260 comp rx=0 tx=0).stop 2023-07-18T03:42:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.468+0000 7f3cb1a37700 1 -- 172.21.15.40:0/159970979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cac102f00 msgr2=0x7f3cac195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.468+0000 7f3cb1a37700 1 --2- 172.21.15.40:0/159970979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cac102f00 0x7f3cac195d10 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0004730 tx=0x7f3ca0004760 comp rx=0 tx=0).stop 2023-07-18T03:42:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.468+0000 7f3cb1a37700 1 -- 172.21.15.40:0/159970979 shutdown_connections 2023-07-18T03:42:46.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.468+0000 7f3cb1a37700 1 --2- 172.21.15.40:0/159970979 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f3c940a11a0 0x7f3c940a3650 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.468+0000 7f3cb1a37700 1 --2- 172.21.15.40:0/159970979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3cac105820 0x7f3cac19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.468+0000 7f3cb1a37700 1 --2- 172.21.15.40:0/159970979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cac1005e0 0x7f3cac1957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.468+0000 7f3cb1a37700 1 --2- 172.21.15.40:0/159970979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cac102f00 0x7f3cac195d10 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.468+0000 7f3cb1a37700 1 -- 172.21.15.40:0/159970979 >> 172.21.15.40:0/159970979 conn(0x7f3cac0fa020 msgr2=0x7f3cac1012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.468+0000 7f3cb1a37700 1 -- 172.21.15.40:0/159970979 shutdown_connections 2023-07-18T03:42:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.468+0000 7f3cb1a37700 1 -- 172.21.15.40:0/159970979 wait complete. 2023-07-18T03:42:46.475 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-18T03:42:46.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.608+0000 7f725b907700 1 -- 172.21.15.40:0/592742866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7254100cd0 msgr2=0x7f7254101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:46.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.608+0000 7f725b907700 1 --2- 172.21.15.40:0/592742866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7254100cd0 0x7f7254101120 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7250009990 tx=0x7f7250009ca0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.609+0000 7f725b907700 1 -- 172.21.15.40:0/592742866 shutdown_connections 2023-07-18T03:42:46.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.609+0000 7f725b907700 1 --2- 172.21.15.40:0/592742866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7254101660 0x7f7254107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.609+0000 7f725b907700 1 --2- 172.21.15.40:0/592742866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72540ffae0 0x7f72540ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.609+0000 7f725b907700 1 --2- 172.21.15.40:0/592742866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7254100cd0 0x7f7254101120 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.609+0000 7f725b907700 1 -- 172.21.15.40:0/592742866 >> 172.21.15.40:0/592742866 conn(0x7f72540fb120 msgr2=0x7f72540fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:46.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.609+0000 7f725b907700 1 -- 172.21.15.40:0/592742866 shutdown_connections 2023-07-18T03:42:46.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.609+0000 7f725b907700 1 -- 172.21.15.40:0/592742866 wait complete. 2023-07-18T03:42:46.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.609+0000 7f725b907700 1 Processor -- start 2023-07-18T03:42:46.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.609+0000 7f725b907700 1 -- start start 2023-07-18T03:42:46.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.609+0000 7f725b907700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72540ffae0 0x7f7254195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:46.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.609+0000 7f725b907700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7254100cd0 0x7f7254195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:46.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.609+0000 7f725b907700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7254101660 0x7f725419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:46.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.609+0000 7f725b907700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725419d3c0 con 0x7f7254100cd0 2023-07-18T03:42:46.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.609+0000 7f725b907700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725419d500 con 0x7f72540ffae0 2023-07-18T03:42:46.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.609+0000 7f725b907700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725419d690 con 0x7f7254101660 2023-07-18T03:42:46.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.610+0000 7f7258ea2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7254100cd0 0x7f7254195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:46.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.610+0000 7f7258ea2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7254100cd0 0x7f7254195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33962/0 (socket says 172.21.15.40:33962) 2023-07-18T03:42:46.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.610+0000 7f7258ea2700 1 -- 172.21.15.40:0/619801419 learned_addr learned my addr 172.21.15.40:0/619801419 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:46.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.610+0000 7f72596a3700 1 --2- 172.21.15.40:0/619801419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72540ffae0 0x7f7254195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:46.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.610+0000 7f7258ea2700 1 -- 172.21.15.40:0/619801419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7254101660 msgr2=0x7f725419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:46.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.610+0000 7f7258ea2700 1 --2- 172.21.15.40:0/619801419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7254101660 0x7f725419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.610+0000 7f7258ea2700 1 -- 172.21.15.40:0/619801419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72540ffae0 msgr2=0x7f7254195840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:46.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.610+0000 7f7258ea2700 1 --2- 172.21.15.40:0/619801419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72540ffae0 0x7f7254195840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.610+0000 7f7258ea2700 1 -- 172.21.15.40:0/619801419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72500095f0 con 0x7f7254100cd0 2023-07-18T03:42:46.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.610+0000 7f72596a3700 1 --2- 172.21.15.40:0/619801419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72540ffae0 0x7f7254195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-18T03:42:46.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.610+0000 7f7258ea2700 1 --2- 172.21.15.40:0/619801419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7254100cd0 0x7f7254195d80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7250000c00 tx=0x7f725000f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:46.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.610+0000 7f724a7fc700 1 -- 172.21.15.40:0/619801419 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725001d070 con 0x7f7254100cd0 2023-07-18T03:42:46.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.610+0000 7f725b907700 1 -- 172.21.15.40:0/619801419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725419d910 con 0x7f7254100cd0 2023-07-18T03:42:46.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.610+0000 7f724a7fc700 1 -- 172.21.15.40:0/619801419 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f725000fc60 con 0x7f7254100cd0 2023-07-18T03:42:46.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.610+0000 7f724a7fc700 1 -- 172.21.15.40:0/619801419 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7250017650 con 0x7f7254100cd0 2023-07-18T03:42:46.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.610+0000 7f725b907700 1 -- 172.21.15.40:0/619801419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725419dd30 con 0x7f7254100cd0 2023-07-18T03:42:46.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.611+0000 7f725b907700 1 -- 172.21.15.40:0/619801419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725410b320 con 0x7f7254100cd0 2023-07-18T03:42:46.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.612+0000 7f724a7fc700 1 -- 172.21.15.40:0/619801419 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7250004830 con 0x7f7254100cd0 2023-07-18T03:42:46.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.613+0000 7f724a7fc700 1 --2- 172.21.15.40:0/619801419 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72400a1150 0x7f72400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:46.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.613+0000 7f724a7fc700 1 -- 172.21.15.40:0/619801419 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8269+0+0 (secure 0 0 0) 0x7f72500ce430 con 0x7f7254100cd0 2023-07-18T03:42:46.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.613+0000 7f72596a3700 1 --2- 172.21.15.40:0/619801419 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72400a1150 0x7f72400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:46.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.614+0000 7f72596a3700 1 --2- 172.21.15.40:0/619801419 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72400a1150 0x7f72400a3600 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7254100b30 tx=0x7f7244009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:46.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.616+0000 7f724a7fc700 1 -- 172.21.15.40:0/619801419 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7250097bf0 con 0x7f7254100cd0 2023-07-18T03:42:46.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.864+0000 7f725b907700 1 -- 172.21.15.40:0/619801419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7254106a30 con 0x7f7254100cd0 2023-07-18T03:42:46.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.870+0000 7f724a7fc700 1 -- 172.21.15.40:0/619801419 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) v1 ==== 161+0+0 (secure 0 0 0) 0x7f725009b210 con 0x7f7254100cd0 2023-07-18T03:42:46.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.874+0000 7f725b907700 1 -- 172.21.15.40:0/619801419 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72400a1150 msgr2=0x7f72400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:46.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.874+0000 7f725b907700 1 --2- 172.21.15.40:0/619801419 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72400a1150 0x7f72400a3600 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7254100b30 tx=0x7f7244009260 comp rx=0 tx=0).stop 2023-07-18T03:42:46.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.874+0000 7f725b907700 1 -- 172.21.15.40:0/619801419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7254100cd0 msgr2=0x7f7254195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:46.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.874+0000 7f725b907700 1 --2- 172.21.15.40:0/619801419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7254100cd0 0x7f7254195d80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7250000c00 tx=0x7f725000f510 comp rx=0 tx=0).stop 2023-07-18T03:42:46.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.874+0000 7f725b907700 1 -- 172.21.15.40:0/619801419 shutdown_connections 2023-07-18T03:42:46.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.874+0000 7f725b907700 1 --2- 172.21.15.40:0/619801419 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f72400a1150 0x7f72400a3600 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.874+0000 7f725b907700 1 --2- 172.21.15.40:0/619801419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7254101660 0x7f725419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.874+0000 7f725b907700 1 --2- 172.21.15.40:0/619801419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72540ffae0 0x7f7254195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.874+0000 7f725b907700 1 --2- 172.21.15.40:0/619801419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7254100cd0 0x7f7254195d80 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:46.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.874+0000 7f725b907700 1 -- 172.21.15.40:0/619801419 >> 172.21.15.40:0/619801419 conn(0x7f72540fb120 msgr2=0x7f72540fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:46.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.874+0000 7f725b907700 1 -- 172.21.15.40:0/619801419 shutdown_connections 2023-07-18T03:42:46.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:46.874+0000 7f725b907700 1 -- 172.21.15.40:0/619801419 wait complete. 2023-07-18T03:42:46.870 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-18T03:42:46.886 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-18T03:42:46.911 INFO:journalctl@ceph.mds.l.smithi134.stdout:Jul 18 03:42:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-l[123528]: ignoring --setuser ceph since I am not root 2023-07-18T03:42:46.911 INFO:journalctl@ceph.mds.c.smithi134.stdout:Jul 18 03:42:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-c[124209]: ignoring --setuser ceph since I am not root 2023-07-18T03:42:46.911 INFO:journalctl@ceph.mds.c.smithi134.stdout:Jul 18 03:42:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-c[124209]: ignoring --setgroup ceph since I am not root 2023-07-18T03:42:47.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.033+0000 7fd348f89700 1 -- 172.21.15.40:0/834666940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd344107970 msgr2=0x7fd344109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:47.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.033+0000 7fd348f89700 1 --2- 172.21.15.40:0/834666940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd344107970 0x7fd344109df0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00b0a0 tx=0x7fd33c00b3b0 comp rx=0 tx=0).stop 2023-07-18T03:42:47.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.033+0000 7fd348f89700 1 -- 172.21.15.40:0/834666940 shutdown_connections 2023-07-18T03:42:47.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.033+0000 7fd348f89700 1 --2- 172.21.15.40:0/834666940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd344107970 0x7fd344109df0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:47.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.033+0000 7fd348f89700 1 --2- 172.21.15.40:0/834666940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3440744f0 0x7fd344072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:47.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.033+0000 7fd348f89700 1 --2- 172.21.15.40:0/834666940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd344072f10 0x7fd344073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:47.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.033+0000 7fd348f89700 1 -- 172.21.15.40:0/834666940 >> 172.21.15.40:0/834666940 conn(0x7fd3440fb120 msgr2=0x7fd3440fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:47.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.034+0000 7fd348f89700 1 -- 172.21.15.40:0/834666940 shutdown_connections 2023-07-18T03:42:47.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.034+0000 7fd348f89700 1 -- 172.21.15.40:0/834666940 wait complete. 2023-07-18T03:42:47.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.034+0000 7fd348f89700 1 Processor -- start 2023-07-18T03:42:47.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd348f89700 1 -- start start 2023-07-18T03:42:47.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd348f89700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd344072f10 0x7fd344197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:47.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd348f89700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3440744f0 0x7fd344198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:47.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd348f89700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd344107970 0x7fd34419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:47.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd348f89700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34419f670 con 0x7fd344107970 2023-07-18T03:42:47.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd348f89700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34419f7b0 con 0x7fd3440744f0 2023-07-18T03:42:47.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd348f89700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34419f940 con 0x7fd344072f10 2023-07-18T03:42:47.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd342d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd344107970 0x7fd34419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:47.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd34259c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd344072f10 0x7fd344197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:47.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd342d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd344107970 0x7fd34419f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34014/0 (socket says 172.21.15.40:34014) 2023-07-18T03:42:47.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd342d9d700 1 -- 172.21.15.40:0/1699862687 learned_addr learned my addr 172.21.15.40:0/1699862687 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:47.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd341d9b700 1 --2- 172.21.15.40:0/1699862687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3440744f0 0x7fd344198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:47.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd34259c700 1 -- 172.21.15.40:0/1699862687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3440744f0 msgr2=0x7fd344198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:47.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd34259c700 1 --2- 172.21.15.40:0/1699862687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3440744f0 0x7fd344198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:47.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd34259c700 1 -- 172.21.15.40:0/1699862687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd344107970 msgr2=0x7fd34419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:47.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd34259c700 1 --2- 172.21.15.40:0/1699862687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd344107970 0x7fd34419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:47.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd34259c700 1 -- 172.21.15.40:0/1699862687 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd338009520 con 0x7fd344072f10 2023-07-18T03:42:47.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.035+0000 7fd342d9d700 1 --2- 172.21.15.40:0/1699862687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd344107970 0x7fd34419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:42:47.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.036+0000 7fd34259c700 1 --2- 172.21.15.40:0/1699862687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd344072f10 0x7fd344197af0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd33400e9f0 tx=0x7fd33400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:47.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.038+0000 7fd3337fe700 1 -- 172.21.15.40:0/1699862687 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33400cb70 con 0x7fd344072f10 2023-07-18T03:42:47.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.038+0000 7fd3337fe700 1 -- 172.21.15.40:0/1699862687 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd33400ee90 con 0x7fd344072f10 2023-07-18T03:42:47.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.038+0000 7fd348f89700 1 -- 172.21.15.40:0/1699862687 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd334009520 con 0x7fd344072f10 2023-07-18T03:42:47.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.038+0000 7fd3337fe700 1 -- 172.21.15.40:0/1699862687 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334003cf0 con 0x7fd344072f10 2023-07-18T03:42:47.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.038+0000 7fd348f89700 1 -- 172.21.15.40:0/1699862687 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33c009d00 con 0x7fd344072f10 2023-07-18T03:42:47.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.039+0000 7fd348f89700 1 -- 172.21.15.40:0/1699862687 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3441060b0 con 0x7fd344072f10 2023-07-18T03:42:47.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.040+0000 7fd3337fe700 1 -- 172.21.15.40:0/1699862687 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd334004020 con 0x7fd344072f10 2023-07-18T03:42:47.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.041+0000 7fd3337fe700 1 --2- 172.21.15.40:0/1699862687 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd32c0a11d0 0x7fd32c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:47.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.041+0000 7fd341d9b700 1 --2- 172.21.15.40:0/1699862687 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd32c0a11d0 0x7fd32c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:47.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.041+0000 7fd3337fe700 1 -- 172.21.15.40:0/1699862687 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8699+0+0 (secure 0 0 0) 0x7fd334014070 con 0x7fd344072f10 2023-07-18T03:42:47.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.041+0000 7fd341d9b700 1 --2- 172.21.15.40:0/1699862687 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd32c0a11d0 0x7fd32c0a3680 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd338009df0 tx=0x7fd338009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:47.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.045+0000 7fd3337fe700 1 -- 172.21.15.40:0/1699862687 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd3340962d0 con 0x7fd344072f10 2023-07-18T03:42:47.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.289+0000 7fd348f89700 1 -- 172.21.15.40:0/1699862687 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd34402c850 con 0x7fd344072f10 2023-07-18T03:42:47.313 INFO:journalctl@ceph.mds.c.smithi134.stdout:Jul 18 03:42:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-c[124209]: starting mds.c at 2023-07-18T03:42:47.314 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:42:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: ignoring --setuser ceph since I am not root 2023-07-18T03:42:47.314 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:42:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: ignoring --setgroup ceph since I am not root 2023-07-18T03:42:47.314 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:42:46 smithi134 ceph-mds[124573]: -- 172.21.15.134:0/714252246 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55996cc9d1e0 con 0x55996cc72800 2023-07-18T03:42:47.314 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:42:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: starting mds.i at 2023-07-18T03:42:47.315 INFO:journalctl@ceph.mds.l.smithi134.stdout:Jul 18 03:42:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-l[123528]: ignoring --setgroup ceph since I am not root 2023-07-18T03:42:47.315 INFO:journalctl@ceph.mds.l.smithi134.stdout:Jul 18 03:42:46 smithi134 ceph-mds[123552]: -- 172.21.15.134:0/2767994971 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561ebead51e0 con 0x561ebeaaac00 2023-07-18T03:42:47.315 INFO:journalctl@ceph.mds.l.smithi134.stdout:Jul 18 03:42:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-l[123528]: starting mds.l at 2023-07-18T03:42:47.315 INFO:journalctl@ceph.mds.f.smithi134.stdout:Jul 18 03:42:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-f[123869]: ignoring --setuser ceph since I am not root 2023-07-18T03:42:47.316 INFO:journalctl@ceph.mds.f.smithi134.stdout:Jul 18 03:42:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-f[123869]: ignoring --setgroup ceph since I am not root 2023-07-18T03:42:47.316 INFO:journalctl@ceph.mds.f.smithi134.stdout:Jul 18 03:42:46 smithi134 ceph-mds[123893]: -- 172.21.15.134:0/1032631782 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5567284bf1e0 con 0x556728494c00 2023-07-18T03:42:47.316 INFO:journalctl@ceph.mds.f.smithi134.stdout:Jul 18 03:42:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-f[123869]: starting mds.f at 2023-07-18T03:42:47.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:47 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/3776280717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:42:47.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:47 smithi134 ceph-mon[98216]: pgmap v651: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 12 op/s 2023-07-18T03:42:47.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:47 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/159970979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:42:47.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:47 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/619801419' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-18T03:42:47.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:47 smithi134 ceph-mon[98216]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-18T03:42:47.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:47 smithi134 ceph-mon[98216]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-18T03:42:47.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:47 smithi134 ceph-mon[98216]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-07-18T03:42:47.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:47 smithi134 ceph-mon[98216]: osdmap e85: 12 total, 12 up, 12 in 2023-07-18T03:42:47.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:47 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/619801419' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-18T03:42:47.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:47 smithi134 ceph-mon[98216]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-18T03:42:47.326 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jul 18 03:42:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-a[131249]: ignoring --setuser ceph since I am not root 2023-07-18T03:42:47.326 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jul 18 03:42:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-a[131249]: ignoring --setgroup ceph since I am not root 2023-07-18T03:42:47.327 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jul 18 03:42:46 smithi040 ceph-mds[131273]: -- 172.21.15.40:0/2310311147 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563172d111e0 con 0x563172ce6800 2023-07-18T03:42:47.327 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jul 18 03:42:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-a[131249]: starting mds.a at 2023-07-18T03:42:47.327 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jul 18 03:42:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-d[130243]: ignoring --setuser ceph since I am not root 2023-07-18T03:42:47.327 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jul 18 03:42:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-d[130243]: ignoring --setgroup ceph since I am not root 2023-07-18T03:42:47.327 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jul 18 03:42:46 smithi040 ceph-mds[130267]: -- 172.21.15.40:0/1400976027 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5620317ef1e0 con 0x5620317c5000 2023-07-18T03:42:47.327 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jul 18 03:42:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-d[130243]: starting mds.d at 2023-07-18T03:42:47.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mon-a[96415]: 2023-07-18T03:42:46.865+0000 7f85887e2700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-18T03:42:47.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:47 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/3776280717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:42:47.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:47 smithi040 ceph-mon[96439]: pgmap v651: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 12 op/s 2023-07-18T03:42:47.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:47 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/159970979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:42:47.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:47 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/619801419' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-18T03:42:47.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:47 smithi040 ceph-mon[96439]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-18T03:42:47.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:47 smithi040 ceph-mon[96439]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-18T03:42:47.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:47 smithi040 ceph-mon[96439]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-07-18T03:42:47.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:47 smithi040 ceph-mon[96439]: osdmap e85: 12 total, 12 up, 12 in 2023-07-18T03:42:47.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:47 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/619801419' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-18T03:42:47.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:47 smithi040 ceph-mon[96439]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-18T03:42:47.329 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jul 18 03:42:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-g[130863]: ignoring --setuser ceph since I am not root 2023-07-18T03:42:47.330 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jul 18 03:42:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-g[130863]: ignoring --setgroup ceph since I am not root 2023-07-18T03:42:47.330 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jul 18 03:42:46 smithi040 ceph-mds[130887]: -- 172.21.15.40:0/305152331 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559c556351e0 con 0x559c5560b000 2023-07-18T03:42:47.330 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jul 18 03:42:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-g[130863]: starting mds.g at 2023-07-18T03:42:47.347 INFO:journalctl@ceph.mds.e.smithi064.stdout:Jul 18 03:42:46 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-e[122738]: ignoring --setuser ceph since I am not root 2023-07-18T03:42:47.348 INFO:journalctl@ceph.mds.e.smithi064.stdout:Jul 18 03:42:46 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-e[122738]: ignoring --setgroup ceph since I am not root 2023-07-18T03:42:47.348 INFO:journalctl@ceph.mds.e.smithi064.stdout:Jul 18 03:42:46 smithi064 ceph-mds[122762]: -- 172.21.15.64:0/2238627606 <== mon.0 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560a003191e0 con 0x560a002ee800 2023-07-18T03:42:47.348 INFO:journalctl@ceph.mds.e.smithi064.stdout:Jul 18 03:42:46 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-e[122738]: starting mds.e at 2023-07-18T03:42:47.348 INFO:journalctl@ceph.mds.b.smithi064.stdout:Jul 18 03:42:46 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-b[123417]: ignoring --setuser ceph since I am not root 2023-07-18T03:42:47.349 INFO:journalctl@ceph.mds.b.smithi064.stdout:Jul 18 03:42:46 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-b[123417]: ignoring --setgroup ceph since I am not root 2023-07-18T03:42:47.349 INFO:journalctl@ceph.mds.b.smithi064.stdout:Jul 18 03:42:46 smithi064 ceph-mds[123441]: -- 172.21.15.64:0/1707101525 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562e2c0cf1e0 con 0x562e2c0a4800 2023-07-18T03:42:47.349 INFO:journalctl@ceph.mds.b.smithi064.stdout:Jul 18 03:42:46 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-b[123417]: starting mds.b at 2023-07-18T03:42:47.349 INFO:journalctl@ceph.mds.h.smithi064.stdout:Jul 18 03:42:46 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-h[123077]: ignoring --setuser ceph since I am not root 2023-07-18T03:42:47.350 INFO:journalctl@ceph.mds.h.smithi064.stdout:Jul 18 03:42:46 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-h[123077]: ignoring --setgroup ceph since I am not root 2023-07-18T03:42:47.350 INFO:journalctl@ceph.mds.h.smithi064.stdout:Jul 18 03:42:46 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-h[123077]: starting mds.h at 2023-07-18T03:42:47.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:47 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/3776280717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-18T03:42:47.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:47 smithi064 ceph-mon[98059]: pgmap v651: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 12 op/s 2023-07-18T03:42:47.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:47 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/159970979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-18T03:42:47.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:47 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/619801419' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-18T03:42:47.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:47 smithi064 ceph-mon[98059]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-18T03:42:47.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:47 smithi064 ceph-mon[98059]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-18T03:42:47.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:47 smithi064 ceph-mon[98059]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-07-18T03:42:47.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:47 smithi064 ceph-mon[98059]: osdmap e85: 12 total, 12 up, 12 in 2023-07-18T03:42:47.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:47 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/619801419' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-18T03:42:47.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:47 smithi064 ceph-mon[98059]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-18T03:42:47.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.874+0000 7fd3337fe700 1 -- 172.21.15.40:0/1699862687 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v77) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd3340998f0 con 0x7fd344072f10 2023-07-18T03:42:47.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.877+0000 7fd348f89700 1 -- 172.21.15.40:0/1699862687 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd32c0a11d0 msgr2=0x7fd32c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:47.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.878+0000 7fd348f89700 1 --2- 172.21.15.40:0/1699862687 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd32c0a11d0 0x7fd32c0a3680 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd338009df0 tx=0x7fd338009260 comp rx=0 tx=0).stop 2023-07-18T03:42:47.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.878+0000 7fd348f89700 1 -- 172.21.15.40:0/1699862687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd344072f10 msgr2=0x7fd344197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:47.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.878+0000 7fd348f89700 1 --2- 172.21.15.40:0/1699862687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd344072f10 0x7fd344197af0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd33400e9f0 tx=0x7fd33400ed00 comp rx=0 tx=0).stop 2023-07-18T03:42:47.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.878+0000 7fd348f89700 1 -- 172.21.15.40:0/1699862687 shutdown_connections 2023-07-18T03:42:47.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.878+0000 7fd348f89700 1 --2- 172.21.15.40:0/1699862687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd344107970 0x7fd34419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:47.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.878+0000 7fd348f89700 1 --2- 172.21.15.40:0/1699862687 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7fd32c0a11d0 0x7fd32c0a3680 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:47.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.878+0000 7fd348f89700 1 --2- 172.21.15.40:0/1699862687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd344072f10 0x7fd344197af0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:47.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.878+0000 7fd348f89700 1 --2- 172.21.15.40:0/1699862687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3440744f0 0x7fd344198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:47.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.878+0000 7fd348f89700 1 -- 172.21.15.40:0/1699862687 >> 172.21.15.40:0/1699862687 conn(0x7fd3440fb120 msgr2=0x7fd3440fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:47.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.878+0000 7fd348f89700 1 -- 172.21.15.40:0/1699862687 shutdown_connections 2023-07-18T03:42:47.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:47.878+0000 7fd348f89700 1 -- 172.21.15.40:0/1699862687 wait complete. 2023-07-18T03:42:47.890 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-18T03:42:48.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.035+0000 7f6f65373700 1 -- 172.21.15.40:0/2881838355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f600ffbd0 msgr2=0x7f6f60100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:48.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.035+0000 7f6f65373700 1 --2- 172.21.15.40:0/2881838355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f600ffbd0 0x7f6f60100020 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6f540098f0 tx=0x7f6f54009c00 comp rx=0 tx=0).stop 2023-07-18T03:42:48.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.035+0000 7f6f65373700 1 -- 172.21.15.40:0/2881838355 shutdown_connections 2023-07-18T03:42:48.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.035+0000 7f6f65373700 1 --2- 172.21.15.40:0/2881838355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f60100560 0x7f6f60106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.035+0000 7f6f65373700 1 --2- 172.21.15.40:0/2881838355 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f600fe9e0 0x7f6f600fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.035+0000 7f6f65373700 1 --2- 172.21.15.40:0/2881838355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f600ffbd0 0x7f6f60100020 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.035+0000 7f6f65373700 1 -- 172.21.15.40:0/2881838355 >> 172.21.15.40:0/2881838355 conn(0x7f6f600fa000 msgr2=0x7f6f600fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.035+0000 7f6f65373700 1 -- 172.21.15.40:0/2881838355 shutdown_connections 2023-07-18T03:42:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.035+0000 7f6f65373700 1 -- 172.21.15.40:0/2881838355 wait complete. 2023-07-18T03:42:48.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.035+0000 7f6f65373700 1 Processor -- start 2023-07-18T03:42:48.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f65373700 1 -- start start 2023-07-18T03:42:48.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f65373700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f600fe9e0 0x7f6f60194830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:48.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f65373700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f600ffbd0 0x7f6f60194d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:48.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f65373700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f60100560 0x7f6f6019bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:48.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f65373700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6019c3b0 con 0x7f6f60100560 2023-07-18T03:42:48.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f65373700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6019c4f0 con 0x7f6f600ffbd0 2023-07-18T03:42:48.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f65373700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6019c680 con 0x7f6f600fe9e0 2023-07-18T03:42:48.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f5f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f60100560 0x7f6f6019bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:48.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f5effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f600fe9e0 0x7f6f60194830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:48.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f5effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f600fe9e0 0x7f6f60194830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.40:51290/0 (socket says 172.21.15.40:51290) 2023-07-18T03:42:48.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f5f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f60100560 0x7f6f6019bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34034/0 (socket says 172.21.15.40:34034) 2023-07-18T03:42:48.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f5effd700 1 -- 172.21.15.40:0/451288573 learned_addr learned my addr 172.21.15.40:0/451288573 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:48.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f5e7fc700 1 --2- 172.21.15.40:0/451288573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f600ffbd0 0x7f6f60194d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:48.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f5effd700 1 -- 172.21.15.40:0/451288573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f600ffbd0 msgr2=0x7f6f60194d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:48.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f5effd700 1 --2- 172.21.15.40:0/451288573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f600ffbd0 0x7f6f60194d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:48.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f5effd700 1 -- 172.21.15.40:0/451288573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f60100560 msgr2=0x7f6f6019bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:48.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f5effd700 1 --2- 172.21.15.40:0/451288573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f60100560 0x7f6f6019bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:48.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f5effd700 1 -- 172.21.15.40:0/451288573 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f54009520 con 0x7f6f600fe9e0 2023-07-18T03:42:48.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.036+0000 7f6f5f7fe700 1 --2- 172.21.15.40:0/451288573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f60100560 0x7f6f6019bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-18T03:42:48.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.037+0000 7f6f5effd700 1 --2- 172.21.15.40:0/451288573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f600fe9e0 0x7f6f60194830 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6f5000e9f0 tx=0x7f6f5000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:48.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.037+0000 7f6f47fff700 1 -- 172.21.15.40:0/451288573 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f50004220 con 0x7f6f600fe9e0 2023-07-18T03:42:48.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.037+0000 7f6f47fff700 1 -- 172.21.15.40:0/451288573 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f50004380 con 0x7f6f600fe9e0 2023-07-18T03:42:48.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.037+0000 7f6f65373700 1 -- 172.21.15.40:0/451288573 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f6019c960 con 0x7f6f600fe9e0 2023-07-18T03:42:48.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.037+0000 7f6f47fff700 1 -- 172.21.15.40:0/451288573 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f50003be0 con 0x7f6f600fe9e0 2023-07-18T03:42:48.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.037+0000 7f6f65373700 1 -- 172.21.15.40:0/451288573 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f6019ce70 con 0x7f6f600fe9e0 2023-07-18T03:42:48.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.038+0000 7f6f65373700 1 -- 172.21.15.40:0/451288573 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f60106070 con 0x7f6f600fe9e0 2023-07-18T03:42:48.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.039+0000 7f6f47fff700 1 -- 172.21.15.40:0/451288573 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6f50003d40 con 0x7f6f600fe9e0 2023-07-18T03:42:48.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.040+0000 7f6f47fff700 1 --2- 172.21.15.40:0/451288573 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6f480a1150 0x7f6f480a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:48.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.040+0000 7f6f5e7fc700 1 --2- 172.21.15.40:0/451288573 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6f480a1150 0x7f6f480a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:48.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.040+0000 7f6f47fff700 1 -- 172.21.15.40:0/451288573 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8699+0+0 (secure 0 0 0) 0x7f6f50014070 con 0x7f6f600fe9e0 2023-07-18T03:42:48.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.041+0000 7f6f5e7fc700 1 --2- 172.21.15.40:0/451288573 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6f480a1150 0x7f6f480a3600 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400f9e0 tx=0x7f6f54019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:48.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.043+0000 7f6f47fff700 1 -- 172.21.15.40:0/451288573 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6f50096260 con 0x7f6f600fe9e0 2023-07-18T03:42:48.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:48.242+0000 7f6f65373700 1 -- 172.21.15.40:0/451288573 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6f6002c850 con 0x7f6f600fe9e0 2023-07-18T03:42:48.313 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1699862687' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:42:48.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:42:48.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-18T03:42:48.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-18T03:42:48.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: Cluster is now healthy 2023-07-18T03:42:48.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.40:6836/3448415973,v1:172.21.15.40:6837/3448415973] up:boot 2023-07-18T03:42:48.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.134:6836/1269884376,v1:172.21.15.134:6837/1269884376] up:boot 2023-07-18T03:42:48.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.134:6832/1226729468,v1:172.21.15.134:6834/1226729468] up:boot 2023-07-18T03:42:48.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.64:6835/2705287611,v1:172.21.15.64:6838/2705287611] up:boot 2023-07-18T03:42:48.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.134:6838/1432174539,v1:172.21.15.134:6839/1432174539] up:boot 2023-07-18T03:42:48.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.134:6833/3403821960,v1:172.21.15.134:6835/3403821960] up:boot 2023-07-18T03:42:48.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.40:6839/141905377,v1:172.21.15.40:6841/141905377] up:boot 2023-07-18T03:42:48.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.64:6836/1966162249,v1:172.21.15.64:6839/1966162249] up:boot 2023-07-18T03:42:48.315 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.64:6834/941127878,v1:172.21.15.64:6837/941127878] up:boot 2023-07-18T03:42:48.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: mds.? [v2:172.21.15.40:6838/2007409175,v1:172.21.15.40:6840/2007409175] up:boot 2023-07-18T03:42:48.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-18T03:42:48.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: fsmap 12 up:standby 2023-07-18T03:42:48.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:42:48.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-18T03:42:48.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-18T03:42:48.316 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-18T03:42:48.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-18T03:42:48.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-18T03:42:48.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-18T03:42:48.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-18T03:42:48.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-18T03:42:48.317 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:48 smithi134 ceph-mon[98216]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-18T03:42:48.326 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1699862687' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:42:48.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:42:48.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-18T03:42:48.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-18T03:42:48.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: Cluster is now healthy 2023-07-18T03:42:48.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.40:6836/3448415973,v1:172.21.15.40:6837/3448415973] up:boot 2023-07-18T03:42:48.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.134:6836/1269884376,v1:172.21.15.134:6837/1269884376] up:boot 2023-07-18T03:42:48.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.134:6832/1226729468,v1:172.21.15.134:6834/1226729468] up:boot 2023-07-18T03:42:48.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.64:6835/2705287611,v1:172.21.15.64:6838/2705287611] up:boot 2023-07-18T03:42:48.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.134:6838/1432174539,v1:172.21.15.134:6839/1432174539] up:boot 2023-07-18T03:42:48.328 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.134:6833/3403821960,v1:172.21.15.134:6835/3403821960] up:boot 2023-07-18T03:42:48.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.40:6839/141905377,v1:172.21.15.40:6841/141905377] up:boot 2023-07-18T03:42:48.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.64:6836/1966162249,v1:172.21.15.64:6839/1966162249] up:boot 2023-07-18T03:42:48.329 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.64:6834/941127878,v1:172.21.15.64:6837/941127878] up:boot 2023-07-18T03:42:48.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: mds.? [v2:172.21.15.40:6838/2007409175,v1:172.21.15.40:6840/2007409175] up:boot 2023-07-18T03:42:48.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-18T03:42:48.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: fsmap 12 up:standby 2023-07-18T03:42:48.330 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:42:48.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-18T03:42:48.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-18T03:42:48.331 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-18T03:42:48.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-18T03:42:48.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-18T03:42:48.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-18T03:42:48.332 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-18T03:42:48.333 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-18T03:42:48.333 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:48 smithi040 ceph-mon[96439]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-18T03:42:48.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1699862687' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:42:48.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-18T03:42:48.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-18T03:42:48.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-18T03:42:48.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: Cluster is now healthy 2023-07-18T03:42:48.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.40:6836/3448415973,v1:172.21.15.40:6837/3448415973] up:boot 2023-07-18T03:42:48.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.134:6836/1269884376,v1:172.21.15.134:6837/1269884376] up:boot 2023-07-18T03:42:48.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.134:6832/1226729468,v1:172.21.15.134:6834/1226729468] up:boot 2023-07-18T03:42:48.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.64:6835/2705287611,v1:172.21.15.64:6838/2705287611] up:boot 2023-07-18T03:42:48.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.134:6838/1432174539,v1:172.21.15.134:6839/1432174539] up:boot 2023-07-18T03:42:48.349 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.134:6833/3403821960,v1:172.21.15.134:6835/3403821960] up:boot 2023-07-18T03:42:48.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.40:6839/141905377,v1:172.21.15.40:6841/141905377] up:boot 2023-07-18T03:42:48.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.64:6836/1966162249,v1:172.21.15.64:6839/1966162249] up:boot 2023-07-18T03:42:48.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.64:6834/941127878,v1:172.21.15.64:6837/941127878] up:boot 2023-07-18T03:42:48.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: mds.? [v2:172.21.15.40:6838/2007409175,v1:172.21.15.40:6840/2007409175] up:boot 2023-07-18T03:42:48.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-18T03:42:48.350 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: fsmap 12 up:standby 2023-07-18T03:42:48.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-18T03:42:48.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-18T03:42:48.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-18T03:42:48.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-18T03:42:48.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-18T03:42:48.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-18T03:42:48.351 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-18T03:42:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-18T03:42:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-18T03:42:48.352 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:48 smithi064 ceph-mon[98059]: from='mgr.14152 172.21.15.40:0/1424764466' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-18T03:42:49.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.053+0000 7f6f47fff700 1 -- 172.21.15.40:0/451288573 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6f50099880 con 0x7f6f600fe9e0 2023-07-18T03:42:49.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.056+0000 7f6f65373700 1 -- 172.21.15.40:0/451288573 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6f480a1150 msgr2=0x7f6f480a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:49.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.056+0000 7f6f65373700 1 --2- 172.21.15.40:0/451288573 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6f480a1150 0x7f6f480a3600 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400f9e0 tx=0x7f6f54019040 comp rx=0 tx=0).stop 2023-07-18T03:42:49.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.056+0000 7f6f65373700 1 -- 172.21.15.40:0/451288573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f600fe9e0 msgr2=0x7f6f60194830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:49.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.056+0000 7f6f65373700 1 --2- 172.21.15.40:0/451288573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f600fe9e0 0x7f6f60194830 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6f5000e9f0 tx=0x7f6f5000ed00 comp rx=0 tx=0).stop 2023-07-18T03:42:49.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.057+0000 7f6f65373700 1 -- 172.21.15.40:0/451288573 shutdown_connections 2023-07-18T03:42:49.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.057+0000 7f6f65373700 1 --2- 172.21.15.40:0/451288573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f60100560 0x7f6f6019bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:49.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.057+0000 7f6f65373700 1 --2- 172.21.15.40:0/451288573 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7f6f480a1150 0x7f6f480a3600 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:49.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.057+0000 7f6f65373700 1 --2- 172.21.15.40:0/451288573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f600fe9e0 0x7f6f60194830 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:49.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.057+0000 7f6f65373700 1 --2- 172.21.15.40:0/451288573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f600ffbd0 0x7f6f60194d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:49.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.057+0000 7f6f65373700 1 -- 172.21.15.40:0/451288573 >> 172.21.15.40:0/451288573 conn(0x7f6f600fa000 msgr2=0x7f6f601031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:49.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.057+0000 7f6f65373700 1 -- 172.21.15.40:0/451288573 shutdown_connections 2023-07-18T03:42:49.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.057+0000 7f6f65373700 1 -- 172.21.15.40:0/451288573 wait complete. 2023-07-18T03:42:49.052 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2023-07-18T03:42:49.077 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-18T03:42:49.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.240+0000 7face8847700 1 -- 172.21.15.40:0/4057829344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face40fdd90 msgr2=0x7face40fe1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:49.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.240+0000 7face8847700 1 --2- 172.21.15.40:0/4057829344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face40fdd90 0x7face40fe1a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7facd4009990 tx=0x7facd4009ca0 comp rx=0 tx=0).stop 2023-07-18T03:42:49.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.240+0000 7face8847700 1 -- 172.21.15.40:0/4057829344 shutdown_connections 2023-07-18T03:42:49.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.240+0000 7face8847700 1 --2- 172.21.15.40:0/4057829344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7face41016f0 0x7face4107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:49.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.240+0000 7face8847700 1 --2- 172.21.15.40:0/4057829344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face40fdd90 0x7face40fe1a0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:49.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.240+0000 7face8847700 1 --2- 172.21.15.40:0/4057829344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7face4100d60 0x7face41011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:49.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.241+0000 7face8847700 1 -- 172.21.15.40:0/4057829344 >> 172.21.15.40:0/4057829344 conn(0x7face40f9770 msgr2=0x7face40fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:49.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.241+0000 7face8847700 1 -- 172.21.15.40:0/4057829344 shutdown_connections 2023-07-18T03:42:49.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.241+0000 7face8847700 1 -- 172.21.15.40:0/4057829344 wait complete. 2023-07-18T03:42:49.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.241+0000 7face8847700 1 Processor -- start 2023-07-18T03:42:49.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.241+0000 7face8847700 1 -- start start 2023-07-18T03:42:49.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7face8847700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7face40fdd90 0x7face4195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:49.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7face8847700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face4100d60 0x7face4195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:49.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7face8847700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7face41016f0 0x7face419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:49.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7face8847700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face419d3b0 con 0x7face4100d60 2023-07-18T03:42:49.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7face8847700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face419d4f0 con 0x7face40fdd90 2023-07-18T03:42:49.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7face8847700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face419d680 con 0x7face41016f0 2023-07-18T03:42:49.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7face1d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face4100d60 0x7face4195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:49.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7face1d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face4100d60 0x7face4195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34044/0 (socket says 172.21.15.40:34044) 2023-07-18T03:42:49.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7face1d9b700 1 -- 172.21.15.40:0/1805470557 learned_addr learned my addr 172.21.15.40:0/1805470557 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-07-18T03:42:49.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7face2d9d700 1 --2- 172.21.15.40:0/1805470557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7face41016f0 0x7face419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:49.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7face259c700 1 --2- 172.21.15.40:0/1805470557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7face40fdd90 0x7face4195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:49.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7face1d9b700 1 -- 172.21.15.40:0/1805470557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7face41016f0 msgr2=0x7face419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7face1d9b700 1 --2- 172.21.15.40:0/1805470557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7face41016f0 0x7face419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7face1d9b700 1 -- 172.21.15.40:0/1805470557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7face40fdd90 msgr2=0x7face4195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7face1d9b700 1 --2- 172.21.15.40:0/1805470557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7face40fdd90 0x7face4195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7face1d9b700 1 -- 172.21.15.40:0/1805470557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd40095f0 con 0x7face4100d60 2023-07-18T03:42:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7face1d9b700 1 --2- 172.21.15.40:0/1805470557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face4100d60 0x7face4195d70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7facd800d950 tx=0x7facd800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.242+0000 7facd37fe700 1 -- 172.21.15.40:0/1805470557 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800f890 con 0x7face4100d60 2023-07-18T03:42:49.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.243+0000 7face8847700 1 -- 172.21.15.40:0/1805470557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face419d960 con 0x7face4100d60 2023-07-18T03:42:49.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.243+0000 7facd37fe700 1 -- 172.21.15.40:0/1805470557 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facd8010460 con 0x7face4100d60 2023-07-18T03:42:49.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.243+0000 7facd37fe700 1 -- 172.21.15.40:0/1805470557 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800e610 con 0x7face4100d60 2023-07-18T03:42:49.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.243+0000 7face8847700 1 -- 172.21.15.40:0/1805470557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face419de70 con 0x7face4100d60 2023-07-18T03:42:49.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.244+0000 7face8847700 1 -- 172.21.15.40:0/1805470557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face4106af0 con 0x7face4100d60 2023-07-18T03:42:49.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.245+0000 7facd37fe700 1 -- 172.21.15.40:0/1805470557 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7facd80108d0 con 0x7face4100d60 2023-07-18T03:42:49.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.245+0000 7facd37fe700 1 --2- 172.21.15.40:0/1805470557 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faccc0a1150 0x7faccc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-18T03:42:49.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.245+0000 7face259c700 1 --2- 172.21.15.40:0/1805470557 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faccc0a1150 0x7faccc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-18T03:42:49.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.245+0000 7facd37fe700 1 -- 172.21.15.40:0/1805470557 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8203+0+0 (secure 0 0 0) 0x7facd80cc6d0 con 0x7face4100d60 2023-07-18T03:42:49.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.246+0000 7face259c700 1 --2- 172.21.15.40:0/1805470557 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faccc0a1150 0x7faccc0a3600 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7facd400b2e0 tx=0x7facd4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-18T03:42:49.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.250+0000 7facd37fe700 1 -- 172.21.15.40:0/1805470557 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7facd8095e40 con 0x7face4100d60 2023-07-18T03:42:49.313 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:49 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/451288573' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:42:49.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:49 smithi134 ceph-mon[98216]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:42:49.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:49 smithi134 ceph-mon[98216]: pgmap v653: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-18T03:42:49.326 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:49 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/451288573' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:42:49.326 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:49 smithi040 ceph-mon[96439]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:42:49.326 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:49 smithi040 ceph-mon[96439]: pgmap v653: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-18T03:42:49.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:49 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/451288573' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:42:49.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:49 smithi064 ceph-mon[98059]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:42:49.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:49 smithi064 ceph-mon[98059]: pgmap v653: 97 pgs: 97 active+clean; 585 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-18T03:42:49.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:49.448+0000 7face8847700 1 -- 172.21.15.40:0/1805470557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7face402c850 con 0x7face4100d60 2023-07-18T03:42:50.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:50.055+0000 7facd37fe700 1 -- 172.21.15.40:0/1805470557 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7facd8099460 con 0x7face4100d60 2023-07-18T03:42:50.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:50.059+0000 7face8847700 1 -- 172.21.15.40:0/1805470557 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faccc0a1150 msgr2=0x7faccc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:50.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:50.059+0000 7face8847700 1 --2- 172.21.15.40:0/1805470557 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faccc0a1150 0x7faccc0a3600 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7facd400b2e0 tx=0x7facd4005fb0 comp rx=0 tx=0).stop 2023-07-18T03:42:50.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:50.059+0000 7face8847700 1 -- 172.21.15.40:0/1805470557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face4100d60 msgr2=0x7face4195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-18T03:42:50.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:50.059+0000 7face8847700 1 --2- 172.21.15.40:0/1805470557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face4100d60 0x7face4195d70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7facd800d950 tx=0x7facd800dc60 comp rx=0 tx=0).stop 2023-07-18T03:42:50.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:50.060+0000 7face8847700 1 -- 172.21.15.40:0/1805470557 shutdown_connections 2023-07-18T03:42:50.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:50.060+0000 7face8847700 1 --2- 172.21.15.40:0/1805470557 >> [v2:172.21.15.40:6800/3612011194,v1:172.21.15.40:6801/3612011194] conn(0x7faccc0a1150 0x7faccc0a3600 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:50.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:50.060+0000 7face8847700 1 --2- 172.21.15.40:0/1805470557 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7face41016f0 0x7face419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:50.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:50.060+0000 7face8847700 1 --2- 172.21.15.40:0/1805470557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7face40fdd90 0x7face4195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:50.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:50.060+0000 7face8847700 1 --2- 172.21.15.40:0/1805470557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7face4100d60 0x7face4195d70 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-18T03:42:50.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:50.060+0000 7face8847700 1 -- 172.21.15.40:0/1805470557 >> 172.21.15.40:0/1805470557 conn(0x7face40f9770 msgr2=0x7face40fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-18T03:42:50.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:50.060+0000 7face8847700 1 -- 172.21.15.40:0/1805470557 shutdown_connections 2023-07-18T03:42:50.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-07-18T03:42:50.060+0000 7face8847700 1 -- 172.21.15.40:0/1805470557 wait complete. 2023-07-18T03:42:50.057 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2023-07-18T03:42:50.074 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-18T03:42:50.114 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-18T03:42:50.147 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-18T03:42:50.194 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-07-18T03:42:50.195 DEBUG:teuthology.orchestra.run.smithi040:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-18T03:42:50.212 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-18T03:42:50.235 DEBUG:teuthology.orchestra.run.smithi134:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-18T03:42:50.251 INFO:tasks.cephadm:Stopping all daemons... 2023-07-18T03:42:50.252 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-07-18T03:42:50.252 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mon.a 2023-07-18T03:42:50.313 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:50 smithi134 ceph-mon[98216]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-18T03:42:50.313 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:50 smithi134 ceph-mon[98216]: osdmap e86: 12 total, 12 up, 12 in 2023-07-18T03:42:50.314 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:50 smithi134 ceph-mon[98216]: from='client.? 172.21.15.40:0/1805470557' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:42:50.326 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:50 smithi040 ceph-mon[96439]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-18T03:42:50.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:50 smithi040 ceph-mon[96439]: osdmap e86: 12 total, 12 up, 12 in 2023-07-18T03:42:50.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:50 smithi040 ceph-mon[96439]: from='client.? 172.21.15.40:0/1805470557' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:42:50.327 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:50 smithi040 systemd[1]: Stopping Ceph mon.a for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:42:50.347 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:50 smithi064 ceph-mon[98059]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-18T03:42:50.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:50 smithi064 ceph-mon[98059]: osdmap e86: 12 total, 12 up, 12 in 2023-07-18T03:42:50.348 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:50 smithi064 ceph-mon[98059]: from='client.? 172.21.15.40:0/1805470557' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-18T03:42:50.885 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:50 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mon-a[96415]: 2023-07-18T03:42:50.615+0000 7f858dfed700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:42:50.885 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:50 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mon-a[96415]: 2023-07-18T03:42:50.615+0000 7f858dfed700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-07-18T03:42:51.310 INFO:journalctl@ceph.mon.a.smithi040.stdout:Jul 18 03:42:50 smithi040 bash[159644]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mon-a 2023-07-18T03:42:51.317 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mon.a.service' 2023-07-18T03:42:51.374 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:42:51.375 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-07-18T03:42:51.375 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-07-18T03:42:51.376 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mon.b 2023-07-18T03:42:51.755 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:51 smithi064 systemd[1]: Stopping Ceph mon.b for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:42:52.053 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:51 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mon-b[98035]: 2023-07-18T03:42:51.761+0000 7f2d1ca35700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:42:52.053 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:51 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mon-b[98035]: 2023-07-18T03:42:51.761+0000 7f2d1ca35700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-07-18T03:42:52.303 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:52 smithi064 bash[128155]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mon-b 2023-07-18T03:42:52.303 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:52 smithi064 systemd[1]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mon.b.service: Succeeded. 2023-07-18T03:42:52.304 INFO:journalctl@ceph.mon.b.smithi064.stdout:Jul 18 03:42:52 smithi064 systemd[1]: Stopped Ceph mon.b for 3af3dba8-251a-11ee-9b34-001a4aab830c. 2023-07-18T03:42:52.305 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mon.b.service' 2023-07-18T03:42:52.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:42:52.359 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-07-18T03:42:52.359 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-07-18T03:42:52.359 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mon.c 2023-07-18T03:42:52.712 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:52 smithi134 systemd[1]: Stopping Ceph mon.c for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:42:52.984 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:52 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mon-c[98181]: 2023-07-18T03:42:52.717+0000 7f04de345700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:42:52.984 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:52 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mon-c[98181]: 2023-07-18T03:42:52.717+0000 7f04de345700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-07-18T03:42:53.313 INFO:journalctl@ceph.mon.c.smithi134.stdout:Jul 18 03:42:52 smithi134 bash[130088]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mon-c 2023-07-18T03:42:53.343 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mon.c.service' 2023-07-18T03:42:53.395 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:42:53.396 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-07-18T03:42:53.396 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-07-18T03:42:53.396 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mgr.x 2023-07-18T03:42:53.755 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:42:53 smithi040 systemd[1]: Stopping Ceph mgr.x for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:42:54.076 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:42:53 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x[96686]: 2023-07-18T03:42:53.760+0000 7fe14626f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:42:54.408 INFO:journalctl@ceph.mgr.x.smithi040.stdout:Jul 18 03:42:54 smithi040 bash[159793]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-x 2023-07-18T03:42:54.416 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mgr.x.service' 2023-07-18T03:42:54.455 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:42:54.456 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-07-18T03:42:54.457 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-07-18T03:42:54.457 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mgr.y 2023-07-18T03:42:54.847 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:42:54 smithi064 systemd[1]: Stopping Ceph mgr.y for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:42:55.171 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:42:55 smithi064 bash[128305]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-y 2023-07-18T03:42:55.358 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Jul 18 03:42:55 smithi064 systemd[1]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-07-18T03:42:55.363 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mgr.y.service' 2023-07-18T03:42:55.401 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:42:55.402 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-07-18T03:42:55.402 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-07-18T03:42:55.402 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mgr.z 2023-07-18T03:42:55.813 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:42:55 smithi134 systemd[1]: Stopping Ceph mgr.z for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:42:56.107 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:42:56 smithi134 bash[130245]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mgr-z 2023-07-18T03:42:56.291 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Jul 18 03:42:56 smithi134 systemd[1]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-07-18T03:42:56.295 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mgr.z.service' 2023-07-18T03:42:56.331 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:42:56.331 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-07-18T03:42:56.332 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-07-18T03:42:56.332 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.0 2023-07-18T03:42:56.717 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jul 18 03:42:56 smithi040 systemd[1]: Stopping Ceph osd.0 for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:42:57.076 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jul 18 03:42:56 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-0[109506]: 2023-07-18T03:42:56.723+0000 7fa78a36b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:42:57.077 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jul 18 03:42:56 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-0[109506]: 2023-07-18T03:42:56.723+0000 7fa78a36b700 -1 osd.0 87 *** Got signal Terminated *** 2023-07-18T03:42:57.077 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jul 18 03:42:56 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-0[109506]: 2023-07-18T03:42:56.723+0000 7fa78a36b700 -1 osd.0 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:43:03.076 INFO:journalctl@ceph.osd.0.smithi040.stdout:Jul 18 03:43:02 smithi040 bash[159936]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-0 2023-07-18T03:43:03.955 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.0.service' 2023-07-18T03:43:03.995 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:43:03.996 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-07-18T03:43:03.996 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-07-18T03:43:03.997 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.1 2023-07-18T03:43:04.347 INFO:journalctl@ceph.osd.1.smithi064.stdout:Jul 18 03:43:04 smithi064 systemd[1]: Stopping Ceph osd.1 for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:43:04.847 INFO:journalctl@ceph.osd.1.smithi064.stdout:Jul 18 03:43:04 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-1[103095]: 2023-07-18T03:43:04.476+0000 7f20286a3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:43:04.847 INFO:journalctl@ceph.osd.1.smithi064.stdout:Jul 18 03:43:04 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-1[103095]: 2023-07-18T03:43:04.476+0000 7f20286a3700 -1 osd.1 87 *** Got signal Terminated *** 2023-07-18T03:43:04.848 INFO:journalctl@ceph.osd.1.smithi064.stdout:Jul 18 03:43:04 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-1[103095]: 2023-07-18T03:43:04.476+0000 7f20286a3700 -1 osd.1 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:43:10.347 INFO:journalctl@ceph.osd.1.smithi064.stdout:Jul 18 03:43:10 smithi064 bash[128449]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-1 2023-07-18T03:43:11.419 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.1.service' 2023-07-18T03:43:11.460 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:43:11.461 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-07-18T03:43:11.461 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-07-18T03:43:11.461 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.2 2023-07-18T03:43:11.813 INFO:journalctl@ceph.osd.2.smithi134.stdout:Jul 18 03:43:11 smithi134 systemd[1]: Stopping Ceph osd.2 for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:43:12.313 INFO:journalctl@ceph.osd.2.smithi134.stdout:Jul 18 03:43:11 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-2[104040]: 2023-07-18T03:43:11.990+0000 7fba3b2eb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:43:12.314 INFO:journalctl@ceph.osd.2.smithi134.stdout:Jul 18 03:43:11 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-2[104040]: 2023-07-18T03:43:11.990+0000 7fba3b2eb700 -1 osd.2 87 *** Got signal Terminated *** 2023-07-18T03:43:12.314 INFO:journalctl@ceph.osd.2.smithi134.stdout:Jul 18 03:43:11 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-2[104040]: 2023-07-18T03:43:11.990+0000 7fba3b2eb700 -1 osd.2 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:43:17.814 INFO:journalctl@ceph.osd.2.smithi134.stdout:Jul 18 03:43:17 smithi134 bash[130394]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-2 2023-07-18T03:43:18.965 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.2.service' 2023-07-18T03:43:19.005 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:43:19.005 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-07-18T03:43:19.006 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-07-18T03:43:19.006 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.3 2023-07-18T03:43:19.326 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jul 18 03:43:19 smithi040 systemd[1]: Stopping Ceph osd.3 for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:43:19.826 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jul 18 03:43:19 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-3[114842]: 2023-07-18T03:43:19.377+0000 7fbf3f263700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:43:19.827 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jul 18 03:43:19 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-3[114842]: 2023-07-18T03:43:19.377+0000 7fbf3f263700 -1 osd.3 87 *** Got signal Terminated *** 2023-07-18T03:43:19.827 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jul 18 03:43:19 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-3[114842]: 2023-07-18T03:43:19.377+0000 7fbf3f263700 -1 osd.3 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:43:23.564 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:23 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:23.126+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.509890+0000 front 2023-07-18T03:42:59.509864+0000 (oldest deadline 2023-07-18T03:43:22.410001+0000) 2023-07-18T03:43:23.576 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jul 18 03:43:23 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-3[114842]: 2023-07-18T03:43:23.222+0000 7fbf39855700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:58.876658+0000 front 2023-07-18T03:42:58.876709+0000 (oldest deadline 2023-07-18T03:43:22.976724+0000) 2023-07-18T03:43:23.577 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:23 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:23.337+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:24.563 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:24 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:24.144+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.509890+0000 front 2023-07-18T03:42:59.509864+0000 (oldest deadline 2023-07-18T03:43:22.410001+0000) 2023-07-18T03:43:24.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:24 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:24.189+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:24.576 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jul 18 03:43:24 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-3[114842]: 2023-07-18T03:43:24.226+0000 7fbf39855700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:58.876658+0000 front 2023-07-18T03:42:58.876709+0000 (oldest deadline 2023-07-18T03:43:22.976724+0000) 2023-07-18T03:43:24.577 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:24 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:24.297+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:25.054 INFO:journalctl@ceph.osd.3.smithi040.stdout:Jul 18 03:43:24 smithi040 bash[160212]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-3 2023-07-18T03:43:25.563 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:25 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:25.148+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.509890+0000 front 2023-07-18T03:42:59.509864+0000 (oldest deadline 2023-07-18T03:43:22.410001+0000) 2023-07-18T03:43:25.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:25 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:25.143+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:25.576 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:25 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:25.322+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:26.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:25 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:25.776+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:26.327 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.3.service' 2023-07-18T03:43:26.368 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:43:26.368 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-07-18T03:43:26.369 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-07-18T03:43:26.369 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.4 2023-07-18T03:43:26.563 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:26 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:26.181+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.509890+0000 front 2023-07-18T03:42:59.509864+0000 (oldest deadline 2023-07-18T03:43:22.410001+0000) 2023-07-18T03:43:26.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:26 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:26.158+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:26.576 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:26 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:26.343+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:26.724 INFO:journalctl@ceph.osd.4.smithi064.stdout:Jul 18 03:43:26 smithi064 systemd[1]: Stopping Ceph osd.4 for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:43:27.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:26 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:26.736+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:27.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:26 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:26.638+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:27.097 INFO:journalctl@ceph.osd.4.smithi064.stdout:Jul 18 03:43:26 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-4[108526]: 2023-07-18T03:43:26.730+0000 7f17611f4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:43:27.097 INFO:journalctl@ceph.osd.4.smithi064.stdout:Jul 18 03:43:26 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-4[108526]: 2023-07-18T03:43:26.730+0000 7f17611f4700 -1 osd.4 87 *** Got signal Terminated *** 2023-07-18T03:43:27.097 INFO:journalctl@ceph.osd.4.smithi064.stdout:Jul 18 03:43:26 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-4[108526]: 2023-07-18T03:43:26.730+0000 7f17611f4700 -1 osd.4 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:43:27.563 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:27 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:27.175+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.509890+0000 front 2023-07-18T03:42:59.509864+0000 (oldest deadline 2023-07-18T03:43:22.410001+0000) 2023-07-18T03:43:27.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:27 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:27.159+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:27.642 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:27 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:27.332+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:28.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:27 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:27.775+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:28.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:27 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:27.649+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:28.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:27 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:27.889+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:28.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:28 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:28.120+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:28.564 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:28 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:28.182+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.509890+0000 front 2023-07-18T03:42:59.509864+0000 (oldest deadline 2023-07-18T03:43:22.410001+0000) 2023-07-18T03:43:28.599 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:28 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:28.340+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:29.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:28 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:28.739+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:29.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:28 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:28.604+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:29.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:28 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:28.929+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:29.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:29 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:29.134+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:29.564 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:29 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:29.164+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.509890+0000 front 2023-07-18T03:42:59.509864+0000 (oldest deadline 2023-07-18T03:43:22.410001+0000) 2023-07-18T03:43:29.639 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:29 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:29.357+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:29.640 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:29 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:29.586+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:30.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:29 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:29.743+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:30.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:29 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:29.976+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:30.563 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:30.132+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.509890+0000 front 2023-07-18T03:42:59.509864+0000 (oldest deadline 2023-07-18T03:43:22.410001+0000) 2023-07-18T03:43:30.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:30.171+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:30.826 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:30 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:30.382+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:30.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:30 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:30.545+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:30.827 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:30 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:30.545+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:31.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:30 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:30.773+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:31.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:30 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:30.989+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:31.563 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:31 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:31.100+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.509890+0000 front 2023-07-18T03:42:59.509864+0000 (oldest deadline 2023-07-18T03:43:22.410001+0000) 2023-07-18T03:43:31.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:31 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:31.137+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:31.826 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:31 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:31.340+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:31.827 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:31 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:31.340+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.965845+0000 front 2023-07-18T03:43:06.965788+0000 (oldest deadline 2023-07-18T03:43:30.966347+0000) 2023-07-18T03:43:31.827 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:31 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:31.542+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:31.827 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:31 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:31.542+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:32.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:31 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:31.782+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:32.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:31 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:31.782+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:32.336 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:31 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:31.992+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:32.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:32 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:32.098+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:32.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:32 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:32.098+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:32.564 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:32 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:32.145+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.509890+0000 front 2023-07-18T03:42:59.509864+0000 (oldest deadline 2023-07-18T03:43:22.410001+0000) 2023-07-18T03:43:32.564 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:32 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:32.145+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:05.911479+0000 front 2023-07-18T03:43:05.911454+0000 (oldest deadline 2023-07-18T03:43:31.211521+0000) 2023-07-18T03:43:32.597 INFO:journalctl@ceph.osd.4.smithi064.stdout:Jul 18 03:43:32 smithi064 bash[128728]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-4 2023-07-18T03:43:32.826 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:32 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:32.347+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:32.826 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:32 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:32.347+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.965845+0000 front 2023-07-18T03:43:06.965788+0000 (oldest deadline 2023-07-18T03:43:30.966347+0000) 2023-07-18T03:43:32.827 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:32 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:32.504+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:32.827 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:32 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:32.504+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:33.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:32 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:32.796+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:33.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:32 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:32.796+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:33.098 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:32 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:32.804+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:33.099 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:32 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:32.966+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:33.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:33 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:33.123+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:33.617 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:33 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:33.123+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:33.617 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:33 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:33.131+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.509890+0000 front 2023-07-18T03:42:59.509864+0000 (oldest deadline 2023-07-18T03:43:22.410001+0000) 2023-07-18T03:43:33.617 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:33 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:33.131+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:05.911479+0000 front 2023-07-18T03:43:05.911454+0000 (oldest deadline 2023-07-18T03:43:31.211521+0000) 2023-07-18T03:43:33.618 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:33 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:33.303+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:33.618 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:33 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:33.303+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.965845+0000 front 2023-07-18T03:43:06.965788+0000 (oldest deadline 2023-07-18T03:43:30.966347+0000) 2023-07-18T03:43:33.619 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:33 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:33.502+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:33.619 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:33 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:33.502+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:33.847 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:33 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:33.774+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:33.893 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.4.service' 2023-07-18T03:43:33.933 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:43:33.933 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-07-18T03:43:33.933 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-07-18T03:43:33.934 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.5 2023-07-18T03:43:34.063 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:33 smithi134 systemd[1]: Stopping Ceph osd.5 for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:43:34.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:33 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:33.746+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:34.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:33 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:33.746+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:34.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:34 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:34.012+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:34.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:34 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:34.012+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:34.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:34 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:34.150+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:34.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:34 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:34.150+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:34.564 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:34 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:34.175+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.509890+0000 front 2023-07-18T03:42:59.509864+0000 (oldest deadline 2023-07-18T03:43:22.410001+0000) 2023-07-18T03:43:34.564 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:34 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:34.175+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:05.911479+0000 front 2023-07-18T03:43:05.911454+0000 (oldest deadline 2023-07-18T03:43:31.211521+0000) 2023-07-18T03:43:34.564 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:34 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:34.293+0000 7fb28b624700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:43:34.564 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:34 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:34.293+0000 7fb28b624700 -1 osd.5 87 *** Got signal Terminated *** 2023-07-18T03:43:34.565 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:34 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:34.293+0000 7fb28b624700 -1 osd.5 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:43:34.576 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:34 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:34.275+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:34.576 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:34 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:34.275+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.965845+0000 front 2023-07-18T03:43:06.965788+0000 (oldest deadline 2023-07-18T03:43:30.966347+0000) 2023-07-18T03:43:34.576 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:34 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:34.506+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:34.577 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:34 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:34.506+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:35.039 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:34 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:34.782+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:35.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:34 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:34.775+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:35.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:34 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:34.775+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:35.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:35 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:35.046+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:35.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:35 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:35.046+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:35.563 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:35 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:35.175+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.509890+0000 front 2023-07-18T03:42:59.509864+0000 (oldest deadline 2023-07-18T03:43:22.410001+0000) 2023-07-18T03:43:35.564 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:35 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:35.175+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:05.911479+0000 front 2023-07-18T03:43:05.911454+0000 (oldest deadline 2023-07-18T03:43:31.211521+0000) 2023-07-18T03:43:35.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:35 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:35.153+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:35.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:35 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:35.153+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:35.576 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:35 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:35.528+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:35.577 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:35 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:35.528+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:35.577 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:35 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:35.282+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:35.577 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:35 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:35.282+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.965845+0000 front 2023-07-18T03:43:06.965788+0000 (oldest deadline 2023-07-18T03:43:30.966347+0000) 2023-07-18T03:43:36.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:35 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:35.801+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:36.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:35 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:35.801+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:36.066 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:35 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:35.739+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:36.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:36 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:36.073+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:36.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:36 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:36.073+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:36.563 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:36 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:36.157+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.509890+0000 front 2023-07-18T03:42:59.509864+0000 (oldest deadline 2023-07-18T03:43:22.410001+0000) 2023-07-18T03:43:36.563 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:36 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:36.157+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:05.911479+0000 front 2023-07-18T03:43:05.911454+0000 (oldest deadline 2023-07-18T03:43:31.211521+0000) 2023-07-18T03:43:36.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:36 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:36.196+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:36.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:36 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:36.196+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:36.566 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:36 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:36.243+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:36.566 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:36 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:36.243+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.965845+0000 front 2023-07-18T03:43:06.965788+0000 (oldest deadline 2023-07-18T03:43:30.966347+0000) 2023-07-18T03:43:36.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:36 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:36.572+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:36.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:36 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:36.572+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:37.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:36 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:36.757+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:37.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:36 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:36.757+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:37.097 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:36 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:36.761+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:37.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:37 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:37.183+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:37.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:37 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:37.183+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:37.564 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:37 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:37.150+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.509890+0000 front 2023-07-18T03:42:59.509864+0000 (oldest deadline 2023-07-18T03:43:22.410001+0000) 2023-07-18T03:43:37.564 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:37 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:37.150+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:05.911479+0000 front 2023-07-18T03:43:05.911454+0000 (oldest deadline 2023-07-18T03:43:31.211521+0000) 2023-07-18T03:43:37.576 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:37 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:37.253+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:37.577 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:37 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:37.253+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.965845+0000 front 2023-07-18T03:43:06.965788+0000 (oldest deadline 2023-07-18T03:43:30.966347+0000) 2023-07-18T03:43:37.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:37 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:37.118+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:37.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:37 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:37.118+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:38.063 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:37 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:37.759+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:38.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:37 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:37.714+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:38.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:37 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:37.714+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:38.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:37 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:37.714+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:38.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:37 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:37.583+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:38.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:37 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:37.583+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:38.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:38 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:38.070+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:38.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:38 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:38.070+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:38.555 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:38 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:38.285+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:38.556 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:38 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:38.285+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.965845+0000 front 2023-07-18T03:43:06.965788+0000 (oldest deadline 2023-07-18T03:43:30.966347+0000) 2023-07-18T03:43:38.563 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:38 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:38.199+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.509890+0000 front 2023-07-18T03:42:59.509864+0000 (oldest deadline 2023-07-18T03:43:22.410001+0000) 2023-07-18T03:43:38.563 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:38 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:38.199+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:05.911479+0000 front 2023-07-18T03:43:05.911454+0000 (oldest deadline 2023-07-18T03:43:31.211521+0000) 2023-07-18T03:43:38.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:38 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:38.211+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:38.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:38 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:38.211+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:38.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:38 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:38.562+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:38.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:38 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:38.562+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:39.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:38 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:38.758+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:39.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:38 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:38.758+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:39.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:38 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:38.758+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:39.097 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:38 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:38.805+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:39.097 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:38 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:38.805+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.562127+0000 front 2023-07-18T03:43:14.562160+0000 (oldest deadline 2023-07-18T03:43:38.062295+0000) 2023-07-18T03:43:39.545 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:39 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:39.241+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:39.545 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:39 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:39.241+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.965845+0000 front 2023-07-18T03:43:06.965788+0000 (oldest deadline 2023-07-18T03:43:30.966347+0000) 2023-07-18T03:43:39.563 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:39 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:39.220+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.509890+0000 front 2023-07-18T03:42:59.509864+0000 (oldest deadline 2023-07-18T03:43:22.410001+0000) 2023-07-18T03:43:39.563 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:39 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5[109591]: 2023-07-18T03:43:39.220+0000 7fb286c18700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:05.911479+0000 front 2023-07-18T03:43:05.911454+0000 (oldest deadline 2023-07-18T03:43:31.211521+0000) 2023-07-18T03:43:39.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:39 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:39.189+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:39.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:39 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:39.189+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:39.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:39 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:39.112+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:39.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:39 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:39.112+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:39.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:39 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:39.112+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:39.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:39 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:39.552+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:39.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:39 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:39.552+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:39.987 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:39 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:39.709+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:39.987 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:39 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:39.709+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:39.987 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:39 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:39.709+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:40.097 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:39 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:39.849+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:40.097 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:39 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:39.849+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.562127+0000 front 2023-07-18T03:43:14.562160+0000 (oldest deadline 2023-07-18T03:43:38.062295+0000) 2023-07-18T03:43:40.098 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:40 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:40.063+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:40.098 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:40 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:40.063+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:40.098 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:40 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:40.063+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:40.313 INFO:journalctl@ceph.osd.5.smithi134.stdout:Jul 18 03:43:40 smithi134 bash[130665]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-5 2023-07-18T03:43:40.313 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:40 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:40.194+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:40.314 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:40 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:40.194+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:40.538 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:40 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:40.196+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:40.538 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:40 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:40.196+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.965845+0000 front 2023-07-18T03:43:06.965788+0000 (oldest deadline 2023-07-18T03:43:30.966347+0000) 2023-07-18T03:43:40.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:40 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:40.545+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:40.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:40 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:40.545+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:41.052 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:40 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:40.808+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:41.052 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:40 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:40.808+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.562127+0000 front 2023-07-18T03:43:14.562160+0000 (oldest deadline 2023-07-18T03:43:38.062295+0000) 2023-07-18T03:43:41.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:40 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:40.664+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:41.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:40 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:40.664+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:41.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:40 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:40.664+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:41.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:41 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:41.058+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:41.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:41 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:41.058+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:41.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:41 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:41.058+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:41.524 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:41 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:41.182+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:41.524 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:41 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:41.182+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:41.525 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:41 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:41.182+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:41.531 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.5.service' 2023-07-18T03:43:41.570 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:43:41.570 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-07-18T03:43:41.571 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-07-18T03:43:41.571 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.6 2023-07-18T03:43:41.575 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:41 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:41.168+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:41.576 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:41 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:41.168+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.965845+0000 front 2023-07-18T03:43:06.965788+0000 (oldest deadline 2023-07-18T03:43:30.966347+0000) 2023-07-18T03:43:41.576 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:41 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:41.168+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:15.667613+0000 front 2023-07-18T03:43:15.667845+0000 (oldest deadline 2023-07-18T03:43:40.367613+0000) 2023-07-18T03:43:41.813 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:41 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:41.626+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:41.814 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:41 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:41.626+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:41.814 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:41 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:41.626+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:41.926 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:41 smithi040 systemd[1]: Stopping Ceph osd.6 for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:43:41.927 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:41 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:41.583+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:41.927 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:41 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:41.583+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:42.088 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:41 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:41.806+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:42.088 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:41 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:41.806+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.562127+0000 front 2023-07-18T03:43:14.562160+0000 (oldest deadline 2023-07-18T03:43:38.062295+0000) 2023-07-18T03:43:42.326 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:41 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:41.933+0000 7f2011835700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:43:42.326 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:41 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:41.933+0000 7f2011835700 -1 osd.6 87 *** Got signal Terminated *** 2023-07-18T03:43:42.326 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:41 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:41.933+0000 7f2011835700 -1 osd.6 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:43:42.327 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:42 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:42.167+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:42.327 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:42 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:42.167+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.965845+0000 front 2023-07-18T03:43:06.965788+0000 (oldest deadline 2023-07-18T03:43:30.966347+0000) 2023-07-18T03:43:42.327 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:42 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:42.167+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:15.667613+0000 front 2023-07-18T03:43:15.667845+0000 (oldest deadline 2023-07-18T03:43:40.367613+0000) 2023-07-18T03:43:42.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:42 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:42.095+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:42.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:42 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:42.095+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:42.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:42 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:42.095+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:42.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:42 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:42.144+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:42.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:42 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:42.144+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:42.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:42 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:42.144+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:42.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:42 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:42.564+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:42.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:42 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:42.564+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:42.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:42 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:42.564+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:43:43.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:42 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:42.656+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:43.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:42 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:42.656+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:43.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:42 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:42.656+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:43.097 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:42 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:42.795+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:43.097 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:42 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:42.795+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.562127+0000 front 2023-07-18T03:43:14.562160+0000 (oldest deadline 2023-07-18T03:43:38.062295+0000) 2023-07-18T03:43:43.547 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:43 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:43.209+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:43.547 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:43 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:43.209+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.965845+0000 front 2023-07-18T03:43:06.965788+0000 (oldest deadline 2023-07-18T03:43:30.966347+0000) 2023-07-18T03:43:43.547 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:43 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:43.209+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:15.667613+0000 front 2023-07-18T03:43:15.667845+0000 (oldest deadline 2023-07-18T03:43:40.367613+0000) 2023-07-18T03:43:43.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:43 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:43.192+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:43.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:43 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:43.192+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:43.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:43 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:43.192+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:43.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:43 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:43.113+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:43.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:43 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:43.113+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:43.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:43 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:43.113+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:43.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:43 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:43.553+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:43.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:43 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:43.553+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:43.827 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:43 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:43.553+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:43:44.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:43 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:43.665+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:44.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:43 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:43.665+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:44.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:43 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:43.665+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:44.097 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:43 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:43.812+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:44.097 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:43 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:43.812+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.562127+0000 front 2023-07-18T03:43:14.562160+0000 (oldest deadline 2023-07-18T03:43:38.062295+0000) 2023-07-18T03:43:44.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:44 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:44.235+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:44.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:44 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:44.235+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:44.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:44 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:44.235+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:44.576 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:44 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:44.255+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:44.576 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:44 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:44.255+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.965845+0000 front 2023-07-18T03:43:06.965788+0000 (oldest deadline 2023-07-18T03:43:30.966347+0000) 2023-07-18T03:43:44.576 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:44 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:44.255+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:15.667613+0000 front 2023-07-18T03:43:15.667845+0000 (oldest deadline 2023-07-18T03:43:40.367613+0000) 2023-07-18T03:43:44.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:44 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:44.155+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:44.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:44 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:44.155+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:44.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:44 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:44.155+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:45.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:44 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:44.625+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:45.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:44 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:44.625+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:45.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:44 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:44.625+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:45.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:44 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:44.600+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:45.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:44 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:44.600+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:45.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:44 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:44.600+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:43:45.097 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:44 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:44.826+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:45.097 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:44 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:44.826+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.562127+0000 front 2023-07-18T03:43:14.562160+0000 (oldest deadline 2023-07-18T03:43:38.062295+0000) 2023-07-18T03:43:45.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:45 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:45.273+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:45.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:45 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:45.273+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:45.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:45 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:45.273+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:45.576 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:45 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:45.225+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:45.576 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:45 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:45.225+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.965845+0000 front 2023-07-18T03:43:06.965788+0000 (oldest deadline 2023-07-18T03:43:30.966347+0000) 2023-07-18T03:43:45.576 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:45 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:45.225+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:15.667613+0000 front 2023-07-18T03:43:15.667845+0000 (oldest deadline 2023-07-18T03:43:40.367613+0000) 2023-07-18T03:43:45.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:45 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:45.133+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:45.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:45 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:45.133+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:45.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:45 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:45.133+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:46.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:45 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:45.592+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:46.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:45 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:45.592+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:46.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:45 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:45.592+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:46.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:45 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:45.627+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:46.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:45 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:45.627+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:46.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:45 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:45.627+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:43:46.175 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:45 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:45.870+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:46.175 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:45 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:45.870+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.562127+0000 front 2023-07-18T03:43:14.562160+0000 (oldest deadline 2023-07-18T03:43:38.062295+0000) 2023-07-18T03:43:46.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:46.264+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:46.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:46.264+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:46.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:46.264+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:46.574 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:46.245+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:57.663236+0000 front 2023-07-18T03:42:57.663202+0000 (oldest deadline 2023-07-18T03:43:22.963351+0000) 2023-07-18T03:43:46.575 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:46.245+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.965845+0000 front 2023-07-18T03:43:06.965788+0000 (oldest deadline 2023-07-18T03:43:30.966347+0000) 2023-07-18T03:43:46.575 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:46.245+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:15.667613+0000 front 2023-07-18T03:43:15.667845+0000 (oldest deadline 2023-07-18T03:43:40.367613+0000) 2023-07-18T03:43:46.575 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6[120386]: 2023-07-18T03:43:46.245+0000 7f200d62a700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:24.270004+0000 front 2023-07-18T03:43:24.270024+0000 (oldest deadline 2023-07-18T03:43:45.970162+0000) 2023-07-18T03:43:46.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:46 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:46.182+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:46.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:46 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:46.183+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:46.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:46 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:46.183+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:46.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:46.581+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:46.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:46.581+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:46.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:46 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:46.581+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:43:47.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:46.611+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:47.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:46.611+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:47.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:46 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:46.611+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:47.160 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:46 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:46.910+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:47.161 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:46 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:46.910+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.562127+0000 front 2023-07-18T03:43:14.562160+0000 (oldest deadline 2023-07-18T03:43:38.062295+0000) 2023-07-18T03:43:47.161 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:46 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:46.910+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:21.563452+0000 front 2023-07-18T03:43:21.563563+0000 (oldest deadline 2023-07-18T03:43:46.863605+0000) 2023-07-18T03:43:47.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:47 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:47.280+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:47.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:47 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:47.280+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:47.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:47 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:47.280+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:47.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:47 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:47.280+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:43:47.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:47 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:47.168+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:47.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:47 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:47.168+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:47.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:47 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:47.168+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:47.826 INFO:journalctl@ceph.osd.6.smithi040.stdout:Jul 18 03:43:47 smithi040 bash[160481]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-6 2023-07-18T03:43:47.827 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:47 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:47.603+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:47.827 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:47 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:47.604+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:47.828 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:47 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:47.604+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:43:48.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:47 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:47.587+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:48.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:47 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:47.587+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:48.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:47 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:47.587+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:48.347 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:47 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:47.950+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:48.347 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:47 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:47.950+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.562127+0000 front 2023-07-18T03:43:14.562160+0000 (oldest deadline 2023-07-18T03:43:38.062295+0000) 2023-07-18T03:43:48.347 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:47 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:47.950+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:21.563452+0000 front 2023-07-18T03:43:21.563563+0000 (oldest deadline 2023-07-18T03:43:46.863605+0000) 2023-07-18T03:43:48.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:48 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:48.125+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:48.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:48 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:48.125+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:48.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:48 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:48.125+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:48.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:48 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:48.125+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:43:48.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:48 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:48.239+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:48.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:48 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:48.239+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:48.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:48 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:48.239+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:48.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:48 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:48.239+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:43:48.916 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:48 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:48.629+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:48.917 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:48 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:48.629+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:48.917 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:48 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:48.629+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:43:48.917 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:48 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:48.629+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:43:49.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:48 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:48.621+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:49.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:48 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:48.621+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:49.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:48 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:48.621+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:49.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:48 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:48.621+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.941720+0000 front 2023-07-18T03:43:20.941608+0000 (oldest deadline 2023-07-18T03:43:47.942238+0000) 2023-07-18T03:43:49.191 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.6.service' 2023-07-18T03:43:49.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:43:49.278 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-07-18T03:43:49.278 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-07-18T03:43:49.279 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.7 2023-07-18T03:43:49.310 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:48 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:48.944+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:49.310 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:48 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:48.944+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.562127+0000 front 2023-07-18T03:43:14.562160+0000 (oldest deadline 2023-07-18T03:43:38.062295+0000) 2023-07-18T03:43:49.310 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:48 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:48.944+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:21.563452+0000 front 2023-07-18T03:43:21.563563+0000 (oldest deadline 2023-07-18T03:43:46.863605+0000) 2023-07-18T03:43:49.310 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:49 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:49.140+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:49.311 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:49 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:49.140+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:49.311 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:49 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:49.140+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:49.311 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:49 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:49.140+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:43:49.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:49 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:49.285+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:49.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:49 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:49.285+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:49.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:49 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:49.285+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:49.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:49 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:49.285+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:43:49.597 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:49 smithi064 systemd[1]: Stopping Ceph osd.7 for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:43:49.922 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:49 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:49.662+0000 7f5870894700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:43:49.922 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:49 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:49.662+0000 7f5870894700 -1 osd.7 87 *** Got signal Terminated *** 2023-07-18T03:43:49.922 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:49 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:49.662+0000 7f5870894700 -1 osd.7 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:43:49.922 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:49 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:49.929+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:49.923 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:49 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:49.929+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.562127+0000 front 2023-07-18T03:43:14.562160+0000 (oldest deadline 2023-07-18T03:43:38.062295+0000) 2023-07-18T03:43:50.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:49 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:49.580+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:50.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:49 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:49.580+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:50.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:49 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:49.580+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:50.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:49 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:49.580+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.941720+0000 front 2023-07-18T03:43:20.941608+0000 (oldest deadline 2023-07-18T03:43:47.942238+0000) 2023-07-18T03:43:50.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:49 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:49.641+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:50.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:49 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:49.641+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:50.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:49 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:49.642+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:43:50.077 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:49 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:49.642+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:43:50.347 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:49 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:49.929+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:21.563452+0000 front 2023-07-18T03:43:21.563563+0000 (oldest deadline 2023-07-18T03:43:46.863605+0000) 2023-07-18T03:43:50.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:50 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:50.123+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:50.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:50 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:50.123+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:50.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:50 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:50.123+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:50.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:50 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:50.123+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:43:50.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:50 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:50.289+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:50.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:50 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:50.289+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:50.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:50 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:50.289+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:50.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:50 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:50.289+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:43:51.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:50 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:50.625+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:51.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:50 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:50.625+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:51.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:50 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:50.625+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:51.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:50 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:50.625+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.941720+0000 front 2023-07-18T03:43:20.941608+0000 (oldest deadline 2023-07-18T03:43:47.942238+0000) 2023-07-18T03:43:51.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:50 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:50.651+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:51.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:50 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:50.651+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:51.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:50 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:50.651+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:43:51.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:50 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:50.651+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:43:51.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:51 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:51.124+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:51.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:51 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:51.124+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:51.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:51 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:51.124+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:51.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:51 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:51.124+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:43:51.348 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:50 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:50.883+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:51.348 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:50 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:50.883+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.562127+0000 front 2023-07-18T03:43:14.562160+0000 (oldest deadline 2023-07-18T03:43:38.062295+0000) 2023-07-18T03:43:51.348 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:50 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:50.883+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:21.563452+0000 front 2023-07-18T03:43:21.563563+0000 (oldest deadline 2023-07-18T03:43:46.863605+0000) 2023-07-18T03:43:51.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:51 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:51.287+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:51.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:51 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:51.287+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:51.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:51 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:51.287+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:51.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:51 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:51.287+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:43:52.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:51 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:51.619+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:52.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:51 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:51.619+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:52.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:51 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:51.619+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:52.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:51 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:51.619+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.941720+0000 front 2023-07-18T03:43:20.941608+0000 (oldest deadline 2023-07-18T03:43:47.942238+0000) 2023-07-18T03:43:52.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:51 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:51.665+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:52.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:51 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:51.665+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:52.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:51 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:51.665+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:43:52.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:51 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:51.665+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:43:52.347 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:51 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:51.923+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:52.347 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:51 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:51.923+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.562127+0000 front 2023-07-18T03:43:14.562160+0000 (oldest deadline 2023-07-18T03:43:38.062295+0000) 2023-07-18T03:43:52.347 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:51 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:51.923+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:21.563452+0000 front 2023-07-18T03:43:21.563563+0000 (oldest deadline 2023-07-18T03:43:46.863605+0000) 2023-07-18T03:43:52.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:52 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:52.136+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:52.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:52 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:52.136+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:52.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:52 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:52.136+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:52.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:52 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:52.136+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:43:52.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:52 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:52.246+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:52.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:52 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:52.246+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:52.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:52 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:52.246+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:52.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:52 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:52.246+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:43:53.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:52 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:52.643+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:53.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:52 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:52.643+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:53.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:52 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:52.643+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:53.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:52 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:52.643+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.941720+0000 front 2023-07-18T03:43:20.941608+0000 (oldest deadline 2023-07-18T03:43:47.942238+0000) 2023-07-18T03:43:53.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:52 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:52.667+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:53.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:52 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:52.667+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:53.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:52 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:52.667+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:43:53.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:52 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:52.667+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:43:53.347 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:52 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:52.943+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:53.347 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:52 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:52.943+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.562127+0000 front 2023-07-18T03:43:14.562160+0000 (oldest deadline 2023-07-18T03:43:38.062295+0000) 2023-07-18T03:43:53.347 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:52 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:52.943+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:21.563452+0000 front 2023-07-18T03:43:21.563563+0000 (oldest deadline 2023-07-18T03:43:46.863605+0000) 2023-07-18T03:43:53.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:53 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:53.153+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:53.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:53 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:53.153+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:53.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:53 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:53.153+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:53.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:53 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:53.153+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:43:53.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:53 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:53.291+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:53.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:53 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:53.291+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:53.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:53 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:53.291+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:53.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:53 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:53.291+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:43:54.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:53 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:53.599+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:54.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:53 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:53.599+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:54.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:53 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:53.599+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:54.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:53 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:53.599+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.941720+0000 front 2023-07-18T03:43:20.941608+0000 (oldest deadline 2023-07-18T03:43:47.942238+0000) 2023-07-18T03:43:54.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:53 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:53.712+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:54.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:53 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:53.712+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:54.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:53 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:53.712+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:43:54.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:53 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:53.712+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:43:54.077 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:53 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:53.712+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:27.938058+0000 front 2023-07-18T03:43:27.938144+0000 (oldest deadline 2023-07-18T03:43:53.238238+0000) 2023-07-18T03:43:54.347 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:53 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:53.928+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.460299+0000 front 2023-07-18T03:43:06.460327+0000 (oldest deadline 2023-07-18T03:43:32.360490+0000) 2023-07-18T03:43:54.347 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:53 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:53.928+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.562127+0000 front 2023-07-18T03:43:14.562160+0000 (oldest deadline 2023-07-18T03:43:38.062295+0000) 2023-07-18T03:43:54.347 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:53 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:53.928+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:21.563452+0000 front 2023-07-18T03:43:21.563563+0000 (oldest deadline 2023-07-18T03:43:46.863605+0000) 2023-07-18T03:43:54.348 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:53 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7[114091]: 2023-07-18T03:43:53.928+0000 7f586c689700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:28.564556+0000 front 2023-07-18T03:43:28.564619+0000 (oldest deadline 2023-07-18T03:43:53.264692+0000) 2023-07-18T03:43:54.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:54 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:54.173+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:54.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:54 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:54.173+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:54.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:54 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:54.173+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:54.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:54 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:54.173+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:43:54.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:54 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:54.318+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:54.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:54 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:54.318+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:54.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:54 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:54.318+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:54.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:54 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:54.318+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:43:54.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:54 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:54.318+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:43:55.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:54 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:54.646+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:55.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:54 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:54.646+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:55.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:54 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:54.646+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:55.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:54 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:54.646+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.941720+0000 front 2023-07-18T03:43:20.941608+0000 (oldest deadline 2023-07-18T03:43:47.942238+0000) 2023-07-18T03:43:55.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:54 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:54.725+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:55.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:54 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:54.725+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:55.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:54 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:54.725+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:43:55.077 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:54 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:54.725+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:43:55.077 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:54 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:54.725+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:27.938058+0000 front 2023-07-18T03:43:27.938144+0000 (oldest deadline 2023-07-18T03:43:53.238238+0000) 2023-07-18T03:43:55.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:55 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:55.186+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:55.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:55 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:55.186+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:55.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:55 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:55.186+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:55.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:55 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:55.186+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:43:55.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:55.296+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:55.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:55.296+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:55.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:55.296+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:55.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:55.296+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:43:55.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:55.296+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:43:55.847 INFO:journalctl@ceph.osd.7.smithi064.stdout:Jul 18 03:43:55 smithi064 bash[128989]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-7 2023-07-18T03:43:56.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:55.662+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:56.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:55.662+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:56.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:55.662+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:56.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:55.662+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.941720+0000 front 2023-07-18T03:43:20.941608+0000 (oldest deadline 2023-07-18T03:43:47.942238+0000) 2023-07-18T03:43:56.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:55 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:55.748+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:56.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:55 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:55.748+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:56.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:55 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:55.748+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:43:56.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:55 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:55.748+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:43:56.077 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:55 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:55.748+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:27.938058+0000 front 2023-07-18T03:43:27.938144+0000 (oldest deadline 2023-07-18T03:43:53.238238+0000) 2023-07-18T03:43:56.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:56 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:56.208+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:56.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:56 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:56.208+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:56.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:56 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:56.208+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:56.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:56 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:56.208+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:43:56.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:56 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:56.208+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:43:56.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:56 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:56.303+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:56.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:56 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:56.303+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:56.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:56 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:56.303+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:56.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:56 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:56.303+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:43:56.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:56 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:56.303+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:43:56.700 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.7.service' 2023-07-18T03:43:56.782 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:43:56.783 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-07-18T03:43:56.783 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-07-18T03:43:56.783 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.8 2023-07-18T03:43:57.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:56 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:56.650+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:57.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:56 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:56.650+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:57.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:56 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:56.650+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:57.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:56 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:56.650+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.941720+0000 front 2023-07-18T03:43:20.941608+0000 (oldest deadline 2023-07-18T03:43:47.942238+0000) 2023-07-18T03:43:57.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:56 smithi134 systemd[1]: Stopping Ceph osd.8 for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:43:57.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:56 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:56.703+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:57.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:56 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:56.703+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:57.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:56 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:56.703+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:43:57.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:56 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:56.703+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:43:57.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:56 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:56.703+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:27.938058+0000 front 2023-07-18T03:43:27.938144+0000 (oldest deadline 2023-07-18T03:43:53.238238+0000) 2023-07-18T03:43:57.563 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:57 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:57.140+0000 7f10ffc68700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:43:57.564 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:57 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:57.140+0000 7f10ffc68700 -1 osd.8 87 *** Got signal Terminated *** 2023-07-18T03:43:57.564 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:57 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:57.140+0000 7f10ffc68700 -1 osd.8 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:43:57.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:57 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:57.255+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:57.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:57 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:57.255+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:57.565 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:57 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:57.255+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:57.565 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:57 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:57.255+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:43:57.565 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:57 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:57.255+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:43:57.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:57 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:57.159+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:57.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:57 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:57.159+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:57.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:57 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:57.159+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:57.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:57 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:57.159+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:43:57.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:57 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:57.159+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:43:58.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:57 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:57.658+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:58.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:57 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:57.658+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:58.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:57 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:57.658+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:58.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:57 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:57.658+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.941720+0000 front 2023-07-18T03:43:20.941608+0000 (oldest deadline 2023-07-18T03:43:47.942238+0000) 2023-07-18T03:43:58.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:57 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:57.692+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:58.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:57 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:57.692+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:58.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:57 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:57.692+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:43:58.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:57 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:57.692+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:43:58.077 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:57 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:57.692+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:27.938058+0000 front 2023-07-18T03:43:27.938144+0000 (oldest deadline 2023-07-18T03:43:53.238238+0000) 2023-07-18T03:43:58.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:58 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:58.241+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:58.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:58 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:58.241+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:58.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:58 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:58.241+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:58.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:58 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:58.241+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:43:58.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:58 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:58.241+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:43:58.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:58 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:58.175+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:58.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:58 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:58.175+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:58.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:58 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:58.175+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:58.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:58 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:58.175+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:43:58.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:58 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:58.175+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:43:59.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:58 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:58.652+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:43:59.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:58 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:58.652+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:43:59.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:58 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:58.652+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:43:59.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:58 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:58.652+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.941720+0000 front 2023-07-18T03:43:20.941608+0000 (oldest deadline 2023-07-18T03:43:47.942238+0000) 2023-07-18T03:43:59.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:58 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:58.709+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:43:59.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:58 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:58.709+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:43:59.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:58 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:58.709+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:43:59.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:58 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:58.709+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:43:59.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:58 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:58.709+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:27.938058+0000 front 2023-07-18T03:43:27.938144+0000 (oldest deadline 2023-07-18T03:43:53.238238+0000) 2023-07-18T03:43:59.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:59 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:59.253+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:43:59.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:59 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:59.253+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:43:59.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:59 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:59.253+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:43:59.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:59 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:59.253+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:43:59.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:43:59 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:43:59.253+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:43:59.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:59 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:59.178+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:43:59.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:59 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:59.178+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:43:59.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:59 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:59.178+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:43:59.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:59 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:59.178+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:43:59.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:43:59 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:43:59.178+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:00.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:59 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:59.633+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:44:00.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:59 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:59.633+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:44:00.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:59 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:59.633+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:44:00.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:43:59 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:43:59.633+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.941720+0000 front 2023-07-18T03:43:20.941608+0000 (oldest deadline 2023-07-18T03:43:47.942238+0000) 2023-07-18T03:44:00.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:59 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:59.722+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:44:00.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:59 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:59.722+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:44:00.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:59 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:59.722+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:44:00.077 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:59 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:59.722+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:44:00.077 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:43:59 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:43:59.722+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:27.938058+0000 front 2023-07-18T03:43:27.938144+0000 (oldest deadline 2023-07-18T03:43:53.238238+0000) 2023-07-18T03:44:00.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:00 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:00.211+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:00.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:00 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:00.211+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:00.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:00 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:00.211+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:00.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:00 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:00.211+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:00.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:00 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:00.211+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:00.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:00 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:00.224+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:44:00.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:00 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:00.224+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:44:00.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:00 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:00.224+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:44:00.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:00 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:00.224+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:44:00.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:00 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:00.224+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:01.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:44:00 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:44:00.664+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:44:01.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:44:00 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:44:00.664+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:44:01.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:44:00 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:44:00.664+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:44:01.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:44:00 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:44:00.664+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.941720+0000 front 2023-07-18T03:43:20.941608+0000 (oldest deadline 2023-07-18T03:43:47.942238+0000) 2023-07-18T03:44:01.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:44:00 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:44:00.664+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:35.745881+0000 front 2023-07-18T03:43:35.745918+0000 (oldest deadline 2023-07-18T03:43:59.846031+0000) 2023-07-18T03:44:01.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:00 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:00.751+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:44:01.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:00 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:00.751+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:44:01.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:00 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:00.751+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:44:01.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:00 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:00.751+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:44:01.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:00 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:00.751+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:27.938058+0000 front 2023-07-18T03:43:27.938144+0000 (oldest deadline 2023-07-18T03:43:53.238238+0000) 2023-07-18T03:44:01.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:01 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:01.189+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:01.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:01 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:01.189+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:01.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:01 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:01.189+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:01.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:01 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:01.189+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:01.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:01 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:01.189+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:01.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:01 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:01.233+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:44:01.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:01 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:01.233+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:44:01.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:01 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:01.233+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:44:01.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:01 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:01.233+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:44:01.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:01 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:01.233+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:02.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:44:01 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:44:01.631+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.335231+0000 front 2023-07-18T03:43:00.335170+0000 (oldest deadline 2023-07-18T03:43:25.635176+0000) 2023-07-18T03:44:02.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:44:01 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:44:01.631+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.736559+0000 front 2023-07-18T03:43:06.736530+0000 (oldest deadline 2023-07-18T03:43:30.836647+0000) 2023-07-18T03:44:02.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:44:01 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:44:01.631+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:14.139050+0000 front 2023-07-18T03:43:14.139112+0000 (oldest deadline 2023-07-18T03:43:37.639272+0000) 2023-07-18T03:44:02.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:44:01 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:44:01.631+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.941720+0000 front 2023-07-18T03:43:20.941608+0000 (oldest deadline 2023-07-18T03:43:47.942238+0000) 2023-07-18T03:44:02.064 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:44:01 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8[115062]: 2023-07-18T03:44:01.631+0000 7f10f8a57700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:35.745881+0000 front 2023-07-18T03:43:35.745918+0000 (oldest deadline 2023-07-18T03:43:59.846031+0000) 2023-07-18T03:44:02.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:01 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:01.789+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:44:02.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:01 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:01.789+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:44:02.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:01 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:01.789+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:44:02.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:01 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:01.789+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:44:02.077 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:01 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:01.789+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:27.938058+0000 front 2023-07-18T03:43:27.938144+0000 (oldest deadline 2023-07-18T03:43:53.238238+0000) 2023-07-18T03:44:02.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:02 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:02.194+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:02.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:02 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:02.194+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:02.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:02 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:02.194+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:02.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:02 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:02.194+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:02.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:02 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:02.194+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:02.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:02 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:02.238+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:44:02.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:02 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:02.238+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:44:02.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:02 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:02.238+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:44:02.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:02 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:02.238+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:44:02.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:02 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:02.238+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:02.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:02 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:02.238+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:35.649658+0000 front 2023-07-18T03:43:35.649754+0000 (oldest deadline 2023-07-18T03:44:01.549620+0000) 2023-07-18T03:44:03.063 INFO:journalctl@ceph.osd.8.smithi134.stdout:Jul 18 03:44:02 smithi134 bash[130927]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-8 2023-07-18T03:44:03.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:02 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:02.830+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:44:03.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:02 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:02.830+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:44:03.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:02 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:02.830+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:44:03.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:02 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:02.830+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:44:03.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:02 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:02.830+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:27.938058+0000 front 2023-07-18T03:43:27.938144+0000 (oldest deadline 2023-07-18T03:43:53.238238+0000) 2023-07-18T03:44:03.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:03 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:03.202+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:03.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:03 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:03.202+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:03.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:03 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:03.202+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:03.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:03 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:03.202+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:03.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:03 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:03.202+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:03.565 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:03 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:03.202+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:03.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:03 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:03.275+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:44:03.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:03 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:03.275+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:44:03.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:03 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:03.275+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:44:03.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:03 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:03.275+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:44:03.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:03 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:03.275+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:03.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:03 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:03.275+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:35.649658+0000 front 2023-07-18T03:43:35.649754+0000 (oldest deadline 2023-07-18T03:44:01.549620+0000) 2023-07-18T03:44:04.082 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.8.service' 2023-07-18T03:44:04.121 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:44:04.121 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-07-18T03:44:04.121 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-07-18T03:44:04.122 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.9 2023-07-18T03:44:04.152 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:03 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:03.856+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:44:04.153 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:03 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:03.856+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:44:04.153 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:03 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:03.856+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:44:04.153 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:03 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:03.856+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:44:04.153 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:03 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:03.856+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:27.938058+0000 front 2023-07-18T03:43:27.938144+0000 (oldest deadline 2023-07-18T03:43:53.238238+0000) 2023-07-18T03:44:04.153 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:03 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:03.856+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:37.839820+0000 front 2023-07-18T03:43:37.839840+0000 (oldest deadline 2023-07-18T03:44:03.139842+0000) 2023-07-18T03:44:04.489 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:04 smithi040 systemd[1]: Stopping Ceph osd.9 for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:44:04.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:04 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:04.204+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:04.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:04 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:04.204+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:04.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:04 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:04.204+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:04.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:04 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:04.204+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:04.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:04 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:04.204+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:04.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:04 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:04.204+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:04.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:04 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:04.276+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:44:04.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:04 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:04.276+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:44:04.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:04 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:04.276+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:44:04.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:04 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:04.276+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:44:04.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:04 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:04.276+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:04.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:04 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:04.276+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:35.649658+0000 front 2023-07-18T03:43:35.649754+0000 (oldest deadline 2023-07-18T03:44:01.549620+0000) 2023-07-18T03:44:04.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:04 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:04.496+0000 7f7f34a51700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:44:04.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:04 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:04.496+0000 7f7f34a51700 -1 osd.9 87 *** Got signal Terminated *** 2023-07-18T03:44:04.826 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:04 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:04.496+0000 7f7f34a51700 -1 osd.9 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:44:05.326 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:04 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:04.852+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:44:05.326 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:04 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:04.852+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:44:05.326 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:04 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:04.852+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:44:05.327 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:04 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:04.852+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:44:05.327 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:04 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:04.852+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:27.938058+0000 front 2023-07-18T03:43:27.938144+0000 (oldest deadline 2023-07-18T03:43:53.238238+0000) 2023-07-18T03:44:05.327 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:04 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:04.852+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:37.839820+0000 front 2023-07-18T03:43:37.839840+0000 (oldest deadline 2023-07-18T03:44:03.139842+0000) 2023-07-18T03:44:05.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:05 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:05.194+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:05.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:05 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:05.194+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:05.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:05 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:05.194+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:05.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:05 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:05.194+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:05.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:05 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:05.194+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:05.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:05 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:05.194+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:05.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:05 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:05.260+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:44:05.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:05 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:05.260+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:44:05.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:05 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:05.260+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:44:05.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:05 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:05.260+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:44:05.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:05 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:05.260+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:05.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:05 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:05.260+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:35.649658+0000 front 2023-07-18T03:43:35.649754+0000 (oldest deadline 2023-07-18T03:44:01.549620+0000) 2023-07-18T03:44:06.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:05 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:05.808+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:44:06.076 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:05 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:05.808+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:44:06.077 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:05 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:05.808+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:44:06.077 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:05 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:05.808+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:44:06.077 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:05 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:05.808+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:27.938058+0000 front 2023-07-18T03:43:27.938144+0000 (oldest deadline 2023-07-18T03:43:53.238238+0000) 2023-07-18T03:44:06.077 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:05 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:05.808+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:37.839820+0000 front 2023-07-18T03:43:37.839840+0000 (oldest deadline 2023-07-18T03:44:03.139842+0000) 2023-07-18T03:44:06.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:06 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:06.206+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:06.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:06 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:06.206+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:06.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:06 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:06.206+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:06.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:06 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:06.206+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:06.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:06 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:06.206+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:06.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:06 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:06.206+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:06.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:06 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:06.224+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:44:06.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:06 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:06.224+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:44:06.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:06 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:06.224+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:44:06.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:06 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:06.224+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:44:06.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:06 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:06.224+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:06.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:06 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:06.224+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:35.649658+0000 front 2023-07-18T03:43:35.649754+0000 (oldest deadline 2023-07-18T03:44:01.549620+0000) 2023-07-18T03:44:07.326 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:06 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:06.842+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:44:07.326 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:06 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:06.842+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:44:07.326 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:06 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:06.842+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:44:07.327 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:06 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:06.842+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:44:07.327 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:06 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:06.842+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:27.938058+0000 front 2023-07-18T03:43:27.938144+0000 (oldest deadline 2023-07-18T03:43:53.238238+0000) 2023-07-18T03:44:07.327 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:06 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:06.842+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:37.839820+0000 front 2023-07-18T03:43:37.839840+0000 (oldest deadline 2023-07-18T03:44:03.139842+0000) 2023-07-18T03:44:07.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:07 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:07.188+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:07.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:07 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:07.188+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:07.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:07 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:07.188+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:07.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:07 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:07.188+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:07.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:07 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:07.188+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:07.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:07 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:07.188+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:07.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:07 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:07.186+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:44:07.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:07 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:07.186+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:44:07.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:07 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:07.186+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:44:07.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:07 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:07.186+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:44:07.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:07 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:07.186+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:07.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:07 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:07.186+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:35.649658+0000 front 2023-07-18T03:43:35.649754+0000 (oldest deadline 2023-07-18T03:44:01.549620+0000) 2023-07-18T03:44:08.326 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:07 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:07.887+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:44:08.326 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:07 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:07.887+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:44:08.326 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:07 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:07.887+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:44:08.326 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:07 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:07.887+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:44:08.327 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:07 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:07.887+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:27.938058+0000 front 2023-07-18T03:43:27.938144+0000 (oldest deadline 2023-07-18T03:43:53.238238+0000) 2023-07-18T03:44:08.327 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:07 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:07.887+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:37.839820+0000 front 2023-07-18T03:43:37.839840+0000 (oldest deadline 2023-07-18T03:44:03.139842+0000) 2023-07-18T03:44:08.327 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:07 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:07.887+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2023-07-18T03:43:43.140346+0000 front 2023-07-18T03:43:43.140387+0000 (oldest deadline 2023-07-18T03:44:07.840442+0000) 2023-07-18T03:44:08.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:08 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:08.198+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:08.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:08 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:08.198+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:08.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:08 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:08.198+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:08.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:08 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:08.198+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:08.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:08 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:08.198+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:08.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:08 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:08.198+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:08.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:08 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:08.212+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:44:08.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:08 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:08.212+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:44:08.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:08 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:08.212+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:44:08.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:08 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:08.212+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:44:08.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:08 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:08.212+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:08.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:08 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:08.212+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:35.649658+0000 front 2023-07-18T03:43:35.649754+0000 (oldest deadline 2023-07-18T03:44:01.549620+0000) 2023-07-18T03:44:09.326 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:08 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:08.887+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:42:59.531303+0000 front 2023-07-18T03:42:59.531266+0000 (oldest deadline 2023-07-18T03:43:25.932238+0000) 2023-07-18T03:44:09.326 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:08 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:08.887+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:09.433617+0000 front 2023-07-18T03:43:09.433603+0000 (oldest deadline 2023-07-18T03:43:29.933613+0000) 2023-07-18T03:44:09.326 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:08 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:08.887+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.236386+0000 front 2023-07-18T03:43:16.236334+0000 (oldest deadline 2023-07-18T03:43:42.136202+0000) 2023-07-18T03:44:09.326 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:08 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:08.887+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.136580+0000 front 2023-07-18T03:43:22.136640+0000 (oldest deadline 2023-07-18T03:43:47.937621+0000) 2023-07-18T03:44:09.326 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:08 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:08.887+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:27.938058+0000 front 2023-07-18T03:43:27.938144+0000 (oldest deadline 2023-07-18T03:43:53.238238+0000) 2023-07-18T03:44:09.327 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:08 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:08.887+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:37.839820+0000 front 2023-07-18T03:43:37.839840+0000 (oldest deadline 2023-07-18T03:44:03.139842+0000) 2023-07-18T03:44:09.327 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:08 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9[125971]: 2023-07-18T03:44:08.887+0000 7f7f2f043700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2023-07-18T03:43:43.140346+0000 front 2023-07-18T03:43:43.140387+0000 (oldest deadline 2023-07-18T03:44:07.840442+0000) 2023-07-18T03:44:09.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:09 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:09.187+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:09.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:09 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:09.187+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:09.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:09 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:09.187+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:09.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:09 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:09.187+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:09.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:09 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:09.187+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:09.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:09 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:09.187+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:09.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:09 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:09.248+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:44:09.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:09 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:09.248+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:44:09.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:09 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:09.248+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:44:09.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:09 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:09.248+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:44:09.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:09 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:09.248+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:09.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:09 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:09.248+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:35.649658+0000 front 2023-07-18T03:43:35.649754+0000 (oldest deadline 2023-07-18T03:44:01.549620+0000) 2023-07-18T03:44:09.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:09 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:09.248+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2023-07-18T03:43:46.751338+0000 front 2023-07-18T03:43:46.751263+0000 (oldest deadline 2023-07-18T03:44:08.451112+0000) 2023-07-18T03:44:10.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:10 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:10.156+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:10.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:10 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:10.156+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:10.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:10 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:10.156+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:10.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:10 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:10.156+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:10.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:10 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:10.156+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:10.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:10 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:10.156+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:10.575 INFO:journalctl@ceph.osd.9.smithi040.stdout:Jul 18 03:44:10 smithi040 bash[160745]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-9 2023-07-18T03:44:10.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:10 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:10.211+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:44:10.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:10 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:10.211+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:44:10.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:10 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:10.211+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:44:10.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:10 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:10.211+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:44:10.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:10 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:10.211+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:10.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:10 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:10.211+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:35.649658+0000 front 2023-07-18T03:43:35.649754+0000 (oldest deadline 2023-07-18T03:44:01.549620+0000) 2023-07-18T03:44:10.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:10 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:10.211+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2023-07-18T03:43:46.751338+0000 front 2023-07-18T03:43:46.751263+0000 (oldest deadline 2023-07-18T03:44:08.451112+0000) 2023-07-18T03:44:11.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:11 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:11.190+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:11.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:11 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:11.190+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:11.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:11 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:11.190+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:11.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:11 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:11.190+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:11.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:11 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:11.190+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:11.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:11 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:11.190+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:11.580 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.9.service' 2023-07-18T03:44:11.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:11 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:11.166+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:44:11.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:11 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:11.166+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:44:11.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:11 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:11.166+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:44:11.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:11 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:11.166+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:44:11.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:11 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:11.166+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:11.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:11 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:11.166+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:35.649658+0000 front 2023-07-18T03:43:35.649754+0000 (oldest deadline 2023-07-18T03:44:01.549620+0000) 2023-07-18T03:44:11.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:11 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:11.166+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2023-07-18T03:43:46.751338+0000 front 2023-07-18T03:43:46.751263+0000 (oldest deadline 2023-07-18T03:44:08.451112+0000) 2023-07-18T03:44:11.662 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:44:11.662 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-07-18T03:44:11.663 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-07-18T03:44:11.663 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.10 2023-07-18T03:44:12.027 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:11 smithi064 systemd[1]: Stopping Ceph osd.10 for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:44:12.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:12 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:12.033+0000 7fa5ca403700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:44:12.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:12 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:12.033+0000 7fa5ca403700 -1 osd.10 87 *** Got signal Terminated *** 2023-07-18T03:44:12.347 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:12 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:12.033+0000 7fa5ca403700 -1 osd.10 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:44:12.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:12 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:12.185+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:44:12.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:12 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:12.185+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:44:12.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:12 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:12.185+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:44:12.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:12 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:12.185+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:44:12.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:12 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:12.185+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:12.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:12 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:12.185+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:35.649658+0000 front 2023-07-18T03:43:35.649754+0000 (oldest deadline 2023-07-18T03:44:01.549620+0000) 2023-07-18T03:44:12.348 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:12 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:12.185+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2023-07-18T03:43:46.751338+0000 front 2023-07-18T03:43:46.751263+0000 (oldest deadline 2023-07-18T03:44:08.451112+0000) 2023-07-18T03:44:12.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:12 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:12.181+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:12.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:12 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:12.181+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:12.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:12 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:12.181+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:12.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:12 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:12.181+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:12.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:12 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:12.181+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:12.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:12 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:12.181+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:13.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:13 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:13.178+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:13.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:13 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:13.178+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:13.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:13 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:13.178+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:13.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:13 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:13.178+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:13.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:13 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:13.178+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:13.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:13 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:13.178+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:13.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:13 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:13.145+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:44:13.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:13 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:13.145+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:44:13.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:13 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:13.145+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:44:13.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:13 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:13.145+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:44:13.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:13 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:13.145+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:13.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:13 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:13.145+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:35.649658+0000 front 2023-07-18T03:43:35.649754+0000 (oldest deadline 2023-07-18T03:44:01.549620+0000) 2023-07-18T03:44:13.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:13 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:13.145+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2023-07-18T03:43:46.751338+0000 front 2023-07-18T03:43:46.751263+0000 (oldest deadline 2023-07-18T03:44:08.451112+0000) 2023-07-18T03:44:14.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:14 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:14.171+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:14.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:14 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:14.171+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:14.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:14 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:14.171+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:14.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:14 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:14.171+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:14.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:14 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:14.171+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:14.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:14 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:14.171+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:14.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:14 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:14.143+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:44:14.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:14 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:14.143+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:44:14.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:14 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:14.143+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:44:14.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:14 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:14.143+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:44:14.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:14 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:14.143+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:14.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:14 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:14.143+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:35.649658+0000 front 2023-07-18T03:43:35.649754+0000 (oldest deadline 2023-07-18T03:44:01.549620+0000) 2023-07-18T03:44:14.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:14 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:14.143+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2023-07-18T03:43:46.751338+0000 front 2023-07-18T03:43:46.751263+0000 (oldest deadline 2023-07-18T03:44:08.451112+0000) 2023-07-18T03:44:15.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:15 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:15.219+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:15.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:15 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:15.219+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:15.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:15 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:15.219+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:15.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:15 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:15.219+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:15.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:15 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:15.219+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:15.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:15 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:15.219+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:15.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:15 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:15.151+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:44:15.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:15 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:15.151+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:44:15.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:15 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:15.151+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:44:15.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:15 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:15.151+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:44:15.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:15 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:15.151+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:15.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:15 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:15.151+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:35.649658+0000 front 2023-07-18T03:43:35.649754+0000 (oldest deadline 2023-07-18T03:44:01.549620+0000) 2023-07-18T03:44:15.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:15 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:15.151+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2023-07-18T03:43:46.751338+0000 front 2023-07-18T03:43:46.751263+0000 (oldest deadline 2023-07-18T03:44:08.451112+0000) 2023-07-18T03:44:16.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:16 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:16.222+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:16.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:16 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:16.222+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:16.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:16 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:16.222+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:16.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:16 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:16.222+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:16.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:16 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:16.222+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:16.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:16 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:16.222+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:16.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:16 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:16.160+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:01.644586+0000 front 2023-07-18T03:43:01.644632+0000 (oldest deadline 2023-07-18T03:43:27.544459+0000) 2023-07-18T03:44:16.597 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:16 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:16.160+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:07.545018+0000 front 2023-07-18T03:43:07.545124+0000 (oldest deadline 2023-07-18T03:43:33.445138+0000) 2023-07-18T03:44:16.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:16 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:16.160+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:13.445800+0000 front 2023-07-18T03:43:13.445911+0000 (oldest deadline 2023-07-18T03:43:38.745838+0000) 2023-07-18T03:44:16.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:16 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:16.161+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:22.147701+0000 front 2023-07-18T03:43:22.147511+0000 (oldest deadline 2023-07-18T03:43:47.447602+0000) 2023-07-18T03:44:16.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:16 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:16.161+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:30.348780+0000 front 2023-07-18T03:43:30.348822+0000 (oldest deadline 2023-07-18T03:43:55.648939+0000) 2023-07-18T03:44:16.598 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:16 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:16.161+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:35.649658+0000 front 2023-07-18T03:43:35.649754+0000 (oldest deadline 2023-07-18T03:44:01.549620+0000) 2023-07-18T03:44:16.599 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:16 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10[119519]: 2023-07-18T03:44:16.161+0000 7fa5c61f8700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.40:6822 osd.6 since back 2023-07-18T03:43:46.751338+0000 front 2023-07-18T03:43:46.751263+0000 (oldest deadline 2023-07-18T03:44:08.451112+0000) 2023-07-18T03:44:17.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:17 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:17.178+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:17.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:17 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:17.178+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:17.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:17 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:17.178+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:17.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:17 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:17.178+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:17.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:17 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:17.178+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:17.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:17 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:17.178+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:18.097 INFO:journalctl@ceph.osd.10.smithi064.stdout:Jul 18 03:44:17 smithi064 bash[129253]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-10 2023-07-18T03:44:18.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:18 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:18.216+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:18.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:18 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:18.216+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:18.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:18 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:18.216+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:18.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:18 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:18.216+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:18.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:18 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:18.216+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:18.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:18 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:18.216+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:18.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:18 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:18.216+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2023-07-18T03:43:52.687672+0000 front 2023-07-18T03:43:52.687778+0000 (oldest deadline 2023-07-18T03:44:17.387493+0000) 2023-07-18T03:44:19.371 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.10.service' 2023-07-18T03:44:19.411 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:44:19.412 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-07-18T03:44:19.412 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-07-18T03:44:19.412 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.11 2023-07-18T03:44:19.442 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:19 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:19.174+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:19.442 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:19 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:19.174+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:19.442 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:19 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:19.174+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:19.443 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:19 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:19.174+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:19.443 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:19 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:19.174+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:19.443 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:19 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:19.174+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:19.443 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:19 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:19.174+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2023-07-18T03:43:52.687672+0000 front 2023-07-18T03:43:52.687778+0000 (oldest deadline 2023-07-18T03:44:17.387493+0000) 2023-07-18T03:44:19.763 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:19 smithi134 systemd[1]: Stopping Ceph osd.11 for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:44:20.063 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:19 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:19.770+0000 7fca3e8df700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:44:20.063 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:19 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:19.770+0000 7fca3e8df700 -1 osd.11 87 *** Got signal Terminated *** 2023-07-18T03:44:20.063 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:19 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:19.770+0000 7fca3e8df700 -1 osd.11 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-18T03:44:20.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:20 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:20.222+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:20.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:20 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:20.222+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:20.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:20 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:20.222+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:20.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:20 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:20.222+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:20.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:20 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:20.222+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:20.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:20 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:20.222+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:20.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:20 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:20.222+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2023-07-18T03:43:52.687672+0000 front 2023-07-18T03:43:52.687778+0000 (oldest deadline 2023-07-18T03:44:17.387493+0000) 2023-07-18T03:44:21.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:21 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:21.216+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:21.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:21 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:21.216+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:21.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:21 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:21.216+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:21.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:21 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:21.216+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:21.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:21 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:21.216+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:21.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:21 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:21.216+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:21.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:21 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:21.216+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2023-07-18T03:43:52.687672+0000 front 2023-07-18T03:43:52.687778+0000 (oldest deadline 2023-07-18T03:44:17.387493+0000) 2023-07-18T03:44:22.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:22 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:22.235+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:22.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:22 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:22.235+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:22.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:22 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:22.235+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:22.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:22 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:22.235+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:22.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:22 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:22.235+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:22.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:22 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:22.235+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:22.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:22 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:22.235+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2023-07-18T03:43:52.687672+0000 front 2023-07-18T03:43:52.687778+0000 (oldest deadline 2023-07-18T03:44:17.387493+0000) 2023-07-18T03:44:23.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:23 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:23.204+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:23.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:23 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:23.204+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:23.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:23 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:23.204+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:23.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:23 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:23.204+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:23.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:23 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:23.204+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:23.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:23 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:23.204+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:23.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:23 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:23.204+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2023-07-18T03:43:52.687672+0000 front 2023-07-18T03:43:52.687778+0000 (oldest deadline 2023-07-18T03:44:17.387493+0000) 2023-07-18T03:44:23.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:23 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:23.204+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6820 osd.8 since back 2023-07-18T03:43:57.387844+0000 front 2023-07-18T03:43:57.387860+0000 (oldest deadline 2023-07-18T03:44:22.688085+0000) 2023-07-18T03:44:24.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:24 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:24.197+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6806 osd.0 since back 2023-07-18T03:43:00.178334+0000 front 2023-07-18T03:43:00.178375+0000 (oldest deadline 2023-07-18T03:43:23.678372+0000) 2023-07-18T03:44:24.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:24 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:24.197+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2023-07-18T03:43:06.480552+0000 front 2023-07-18T03:43:06.480269+0000 (oldest deadline 2023-07-18T03:43:31.180379+0000) 2023-07-18T03:44:24.563 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:24 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:24.197+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2023-07-18T03:43:16.481555+0000 front 2023-07-18T03:43:16.481599+0000 (oldest deadline 2023-07-18T03:43:40.581651+0000) 2023-07-18T03:44:24.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:24 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:24.197+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.40:6814 osd.3 since back 2023-07-18T03:43:20.582383+0000 front 2023-07-18T03:43:20.582359+0000 (oldest deadline 2023-07-18T03:43:46.482490+0000) 2023-07-18T03:44:24.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:24 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:24.197+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2023-07-18T03:43:31.684046+0000 front 2023-07-18T03:43:31.683917+0000 (oldest deadline 2023-07-18T03:43:53.983942+0000) 2023-07-18T03:44:24.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:24 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:24.197+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2023-07-18T03:43:36.884975+0000 front 2023-07-18T03:43:36.885045+0000 (oldest deadline 2023-07-18T03:44:02.785117+0000) 2023-07-18T03:44:24.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:24 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:24.197+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2023-07-18T03:43:52.687672+0000 front 2023-07-18T03:43:52.687778+0000 (oldest deadline 2023-07-18T03:44:17.387493+0000) 2023-07-18T03:44:24.564 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:24 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11[120596]: 2023-07-18T03:44:24.197+0000 7fca396d2700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.134:6820 osd.8 since back 2023-07-18T03:43:57.387844+0000 front 2023-07-18T03:43:57.387860+0000 (oldest deadline 2023-07-18T03:44:22.688085+0000) 2023-07-18T03:44:25.813 INFO:journalctl@ceph.osd.11.smithi134.stdout:Jul 18 03:44:25 smithi134 bash[131194]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-osd-11 2023-07-18T03:44:26.910 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@osd.11.service' 2023-07-18T03:44:26.991 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:44:26.991 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-07-18T03:44:26.992 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-07-18T03:44:26.992 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.a 2023-07-18T03:44:27.326 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jul 18 03:44:27 smithi040 systemd[1]: Stopping Ceph mds.a for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:44:27.826 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jul 18 03:44:27 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-a[131249]: 2023-07-18T03:44:27.369+0000 7f1b0a81a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:44:27.826 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jul 18 03:44:27 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-a[131249]: 2023-07-18T03:44:27.369+0000 7f1b0a81a700 -1 mds.a *** got signal Terminated *** 2023-07-18T03:44:31.516 INFO:journalctl@ceph.mds.a.smithi040.stdout:Jul 18 03:44:31 smithi040 bash[161023]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-a 2023-07-18T03:44:31.521 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.a.service' 2023-07-18T03:44:31.559 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:44:31.559 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-07-18T03:44:31.559 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-07-18T03:44:31.560 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.d 2023-07-18T03:44:31.826 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jul 18 03:44:31 smithi040 systemd[1]: Stopping Ceph mds.d for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:44:32.325 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jul 18 03:44:31 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-d[130243]: 2023-07-18T03:44:31.942+0000 7f82d97ae700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:44:32.326 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jul 18 03:44:31 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-d[130243]: 2023-07-18T03:44:31.942+0000 7f82d97ae700 -1 mds.d *** got signal Terminated *** 2023-07-18T03:44:35.417 INFO:journalctl@ceph.mds.d.smithi040.stdout:Jul 18 03:44:35 smithi040 bash[161169]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-d 2023-07-18T03:44:35.423 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.d.service' 2023-07-18T03:44:35.458 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:44:35.458 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-07-18T03:44:35.458 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-07-18T03:44:35.459 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.g 2023-07-18T03:44:35.825 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jul 18 03:44:35 smithi040 systemd[1]: Stopping Ceph mds.g for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:44:36.326 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jul 18 03:44:35 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-g[130863]: 2023-07-18T03:44:35.981+0000 7fb7c6d8e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:44:36.326 INFO:journalctl@ceph.mds.g.smithi040.stdout:Jul 18 03:44:35 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-g[130863]: 2023-07-18T03:44:35.981+0000 7fb7c6d8e700 -1 mds.g *** got signal Terminated *** 2023-07-18T03:44:39.509 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.g.service' 2023-07-18T03:44:39.544 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:44:39.544 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-07-18T03:44:39.545 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-07-18T03:44:39.545 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.j 2023-07-18T03:44:39.826 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jul 18 03:44:39 smithi040 systemd[1]: Stopping Ceph mds.j for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:44:40.326 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jul 18 03:44:39 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-j[129859]: 2023-07-18T03:44:39.939+0000 7f11303f4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:44:40.326 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jul 18 03:44:39 smithi040 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-j[129859]: 2023-07-18T03:44:39.939+0000 7f11303f4700 -1 mds.j *** got signal Terminated *** 2023-07-18T03:44:41.292 INFO:journalctl@ceph.mds.j.smithi040.stdout:Jul 18 03:44:41 smithi040 bash[161453]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-j 2023-07-18T03:44:41.485 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.j.service' 2023-07-18T03:44:41.520 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:44:41.520 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-07-18T03:44:41.520 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-07-18T03:44:41.521 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.b 2023-07-18T03:44:41.847 INFO:journalctl@ceph.mds.b.smithi064.stdout:Jul 18 03:44:41 smithi064 systemd[1]: Stopping Ceph mds.b for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:44:42.347 INFO:journalctl@ceph.mds.b.smithi064.stdout:Jul 18 03:44:41 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-b[123417]: 2023-07-18T03:44:41.902+0000 7fd6cd13a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:44:42.347 INFO:journalctl@ceph.mds.b.smithi064.stdout:Jul 18 03:44:41 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-b[123417]: 2023-07-18T03:44:41.902+0000 7fd6cd13a700 -1 mds.b *** got signal Terminated *** 2023-07-18T03:44:43.244 INFO:journalctl@ceph.mds.b.smithi064.stdout:Jul 18 03:44:43 smithi064 bash[129532]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-b 2023-07-18T03:44:43.440 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.b.service' 2023-07-18T03:44:43.478 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:44:43.478 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-07-18T03:44:43.479 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-07-18T03:44:43.479 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.e 2023-07-18T03:44:43.847 INFO:journalctl@ceph.mds.e.smithi064.stdout:Jul 18 03:44:43 smithi064 systemd[1]: Stopping Ceph mds.e for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:44:44.347 INFO:journalctl@ceph.mds.e.smithi064.stdout:Jul 18 03:44:43 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-e[122738]: 2023-07-18T03:44:43.863+0000 7f59cdece700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:44:44.347 INFO:journalctl@ceph.mds.e.smithi064.stdout:Jul 18 03:44:43 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-e[122738]: 2023-07-18T03:44:43.863+0000 7f59cdece700 -1 mds.e *** got signal Terminated *** 2023-07-18T03:44:47.521 INFO:journalctl@ceph.mds.e.smithi064.stdout:Jul 18 03:44:47 smithi064 bash[129670]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-e 2023-07-18T03:44:47.527 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.e.service' 2023-07-18T03:44:47.562 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:44:47.563 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-07-18T03:44:47.563 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-07-18T03:44:47.564 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.h 2023-07-18T03:44:47.847 INFO:journalctl@ceph.mds.h.smithi064.stdout:Jul 18 03:44:47 smithi064 systemd[1]: Stopping Ceph mds.h for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:44:48.347 INFO:journalctl@ceph.mds.h.smithi064.stdout:Jul 18 03:44:47 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-h[123077]: 2023-07-18T03:44:47.953+0000 7fd41c07f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:44:48.347 INFO:journalctl@ceph.mds.h.smithi064.stdout:Jul 18 03:44:47 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-h[123077]: 2023-07-18T03:44:47.953+0000 7fd41c07f700 -1 mds.h *** got signal Terminated *** 2023-07-18T03:44:51.346 INFO:journalctl@ceph.mds.h.smithi064.stdout:Jul 18 03:44:51 smithi064 bash[129816]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-h 2023-07-18T03:44:51.468 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.h.service' 2023-07-18T03:44:51.502 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:44:51.503 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-07-18T03:44:51.503 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-07-18T03:44:51.503 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.k 2023-07-18T03:44:51.846 INFO:journalctl@ceph.mds.k.smithi064.stdout:Jul 18 03:44:51 smithi064 systemd[1]: Stopping Ceph mds.k for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:44:52.347 INFO:journalctl@ceph.mds.k.smithi064.stdout:Jul 18 03:44:51 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-k[122395]: 2023-07-18T03:44:51.909+0000 7f4bce0ec700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:44:52.347 INFO:journalctl@ceph.mds.k.smithi064.stdout:Jul 18 03:44:51 smithi064 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-k[122395]: 2023-07-18T03:44:51.909+0000 7f4bce0ec700 -1 mds.k *** got signal Terminated *** 2023-07-18T03:44:55.307 INFO:journalctl@ceph.mds.k.smithi064.stdout:Jul 18 03:44:54 smithi064 bash[129958]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-k 2023-07-18T03:44:55.313 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.k.service' 2023-07-18T03:44:55.348 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:44:55.349 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-07-18T03:44:55.349 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-07-18T03:44:55.349 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.c 2023-07-18T03:44:55.699 INFO:journalctl@ceph.mds.c.smithi134.stdout:Jul 18 03:44:55 smithi134 systemd[1]: Stopping Ceph mds.c for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:44:56.062 INFO:journalctl@ceph.mds.c.smithi134.stdout:Jul 18 03:44:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-c[124209]: 2023-07-18T03:44:55.706+0000 7fa18e834700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:44:56.063 INFO:journalctl@ceph.mds.c.smithi134.stdout:Jul 18 03:44:55 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-c[124209]: 2023-07-18T03:44:55.706+0000 7fa18e834700 -1 mds.c *** got signal Terminated *** 2023-07-18T03:44:59.274 INFO:journalctl@ceph.mds.c.smithi134.stdout:Jul 18 03:44:59 smithi134 bash[131464]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-c 2023-07-18T03:44:59.463 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.c.service' 2023-07-18T03:44:59.498 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:44:59.498 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-07-18T03:44:59.499 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-07-18T03:44:59.499 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.f 2023-07-18T03:44:59.813 INFO:journalctl@ceph.mds.f.smithi134.stdout:Jul 18 03:44:59 smithi134 systemd[1]: Stopping Ceph mds.f for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:45:00.313 INFO:journalctl@ceph.mds.f.smithi134.stdout:Jul 18 03:44:59 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-f[123869]: 2023-07-18T03:44:59.881+0000 7fd7baf27700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:45:00.313 INFO:journalctl@ceph.mds.f.smithi134.stdout:Jul 18 03:44:59 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-f[123869]: 2023-07-18T03:44:59.881+0000 7fd7baf27700 -1 mds.f *** got signal Terminated *** 2023-07-18T03:45:03.243 INFO:journalctl@ceph.mds.f.smithi134.stdout:Jul 18 03:45:03 smithi134 bash[131605]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-f 2023-07-18T03:45:03.431 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.f.service' 2023-07-18T03:45:03.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:45:03.465 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-07-18T03:45:03.466 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-07-18T03:45:03.466 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.i 2023-07-18T03:45:03.812 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:45:03 smithi134 systemd[1]: Stopping Ceph mds.i for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:45:04.313 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:45:03 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:45:03.862+0000 7fc23a937700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:45:04.313 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:45:03 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i[124549]: 2023-07-18T03:45:03.862+0000 7fc23a937700 -1 mds.i *** got signal Terminated *** 2023-07-18T03:45:07.426 INFO:journalctl@ceph.mds.i.smithi134.stdout:Jul 18 03:45:07 smithi134 bash[131745]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-i 2023-07-18T03:45:07.431 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.i.service' 2023-07-18T03:45:07.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:45:07.466 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-07-18T03:45:07.466 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-07-18T03:45:07.466 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.l 2023-07-18T03:45:07.812 INFO:journalctl@ceph.mds.l.smithi134.stdout:Jul 18 03:45:07 smithi134 systemd[1]: Stopping Ceph mds.l for 3af3dba8-251a-11ee-9b34-001a4aab830c... 2023-07-18T03:45:08.313 INFO:journalctl@ceph.mds.l.smithi134.stdout:Jul 18 03:45:07 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-l[123528]: 2023-07-18T03:45:07.850+0000 7ffb0ba85700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-18T03:45:08.313 INFO:journalctl@ceph.mds.l.smithi134.stdout:Jul 18 03:45:07 smithi134 ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-l[123528]: 2023-07-18T03:45:07.850+0000 7ffb0ba85700 -1 mds.l *** got signal Terminated *** 2023-07-18T03:45:11.442 INFO:journalctl@ceph.mds.l.smithi134.stdout:Jul 18 03:45:11 smithi134 bash[131891]: ceph-3af3dba8-251a-11ee-9b34-001a4aab830c-mds-l 2023-07-18T03:45:11.447 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3af3dba8-251a-11ee-9b34-001a4aab830c@mds.l.service' 2023-07-18T03:45:11.479 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-18T03:45:11.479 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-07-18T03:45:11.480 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-18T03:45:13.425 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-18T03:45:15.409 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-18T03:45:17.283 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-18T03:45:17.321 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-18T03:45:17.359 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-18T03:45:17.389 INFO:tasks.cephadm:Archiving crash dumps... 2023-07-18T03:45:17.391 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746/remote/smithi040/crash 2023-07-18T03:45:17.392 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/crash -- . 2023-07-18T03:45:17.423 INFO:teuthology.orchestra.run.smithi040.stderr:tar: /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-18T03:45:17.424 INFO:teuthology.orchestra.run.smithi040.stderr:tar: Error is not recoverable: exiting now 2023-07-18T03:45:17.425 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746/remote/smithi064/crash 2023-07-18T03:45:17.426 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/crash -- . 2023-07-18T03:45:17.477 INFO:teuthology.orchestra.run.smithi064.stderr:tar: /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-18T03:45:17.477 INFO:teuthology.orchestra.run.smithi064.stderr:tar: Error is not recoverable: exiting now 2023-07-18T03:45:17.479 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746/remote/smithi134/crash 2023-07-18T03:45:17.480 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/crash -- . 2023-07-18T03:45:17.537 INFO:teuthology.orchestra.run.smithi134.stderr:tar: /var/lib/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-18T03:45:17.537 INFO:teuthology.orchestra.run.smithi134.stderr:tar: Error is not recoverable: exiting now 2023-07-18T03:45:17.538 INFO:tasks.cephadm:Checking cluster log for badness... 2023-07-18T03:45:17.538 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-07-18T03:45:17.575 INFO:teuthology.orchestra.run.smithi040.stderr:grep: /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph.log: No such file or directory 2023-07-18T03:45:17.578 INFO:tasks.cephadm:Compressing logs... 2023-07-18T03:45:17.578 DEBUG:teuthology.orchestra.run.smithi040:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-18T03:45:17.620 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-18T03:45:17.623 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-18T03:45:17.650 INFO:teuthology.orchestra.run.smithi040.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-18T03:45:17.650 INFO:teuthology.orchestra.run.smithi040.stderr:: No such file or directory 2023-07-18T03:45:17.650 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.0.log 2023-07-18T03:45:17.651 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.g.log 2023-07-18T03:45:17.651 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.a.log 2023-07-18T03:45:17.651 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.0.log: /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.6.log 2023-07-18T03:45:17.652 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.3.log 2023-07-18T03:45:17.652 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.d.log 2023-07-18T03:45:17.656 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.3.log: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-18T03:45:17.656 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mgr.x.log 2023-07-18T03:45:17.658 INFO:teuthology.orchestra.run.smithi134.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-18T03:45:17.659 INFO:teuthology.orchestra.run.smithi134.stderr:: No such file or directory 2023-07-18T03:45:17.662 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.d.log: /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.9.log 2023-07-18T03:45:17.663 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.5.log 2023-07-18T03:45:17.663 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.2.log 2023-07-18T03:45:17.663 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.11.log 2023-07-18T03:45:17.663 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.5.log: /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mgr.z.log 2023-07-18T03:45:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.8.log 2023-07-18T03:45:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.11.log: /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mgr.z.log: gzip 92.3%-5 -- replaced with /var/log/ceph/cephadm.log.gz--verbose 2023-07-18T03:45:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:-- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.f.log 2023-07-18T03:45:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.c.log 2023-07-18T03:45:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.i.log 2023-07-18T03:45:17.665 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.f.log: /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mon.c.log 2023-07-18T03:45:17.666 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-18T03:45:17.666 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.l.log 2023-07-18T03:45:17.667 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-volume.log: /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.l.log: 92.8% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mgr.z.log.gz 2023-07-18T03:45:17.671 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-07-18T03:45:17.671 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-18T03:45:17.673 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-18T03:45:17.677 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.10.log 2023-07-18T03:45:17.678 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.h.log 2023-07-18T03:45:17.679 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.e.log 2023-07-18T03:45:17.679 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.4.log 2023-07-18T03:45:17.680 INFO:teuthology.orchestra.run.smithi064.stderr: 91.9%/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.e.log: -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-18T03:45:17.680 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.1.log 2023-07-18T03:45:17.680 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.k.log 2023-07-18T03:45:17.680 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.4.log: /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-18T03:45:17.681 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.7.log 2023-07-18T03:45:17.681 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mon.b.log 2023-07-18T03:45:17.681 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.b.log 2023-07-18T03:45:17.681 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mgr.y.log 2023-07-18T03:45:17.683 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.b.log: /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mgr.y.log: 94.5% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.k.log.gz 2023-07-18T03:45:17.686 INFO:teuthology.orchestra.run.smithi064.stderr: 92.7% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mgr.y.log.gz 2023-07-18T03:45:17.707 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-volume.log: 91.6% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-18T03:45:17.707 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.j.log 2023-07-18T03:45:17.722 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.j.log: 94.6% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.j.log.gz 2023-07-18T03:45:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mon.a.log 2023-07-18T03:45:17.925 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mon.a.log: 90.1% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mgr.x.log.gz 2023-07-18T03:45:18.182 INFO:teuthology.orchestra.run.smithi064.stderr: 92.0% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mon.b.log.gz 2023-07-18T03:45:18.301 INFO:teuthology.orchestra.run.smithi134.stderr: 91.9% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mon.c.log.gz 2023-07-18T03:45:18.697 INFO:teuthology.orchestra.run.smithi064.stderr: 91.4% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-18T03:45:19.068 INFO:teuthology.orchestra.run.smithi040.stderr: 90.5% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mon.a.log.gz 2023-07-18T03:45:19.325 INFO:teuthology.orchestra.run.smithi134.stderr: 91.5% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-18T03:45:21.322 INFO:teuthology.orchestra.run.smithi040.stderr: 95.2% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.d.log.gz 2023-07-18T03:45:22.471 INFO:teuthology.orchestra.run.smithi040.stderr: 95.1% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.g.log.gz 2023-07-18T03:45:23.144 INFO:teuthology.orchestra.run.smithi040.stderr: 95.6% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.a.log.gz 2023-07-18T03:45:23.228 INFO:teuthology.orchestra.run.smithi064.stderr: 93.7% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.4.log.gz 2023-07-18T03:45:23.904 INFO:teuthology.orchestra.run.smithi040.stderr: 93.7% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.9.log.gz 2023-07-18T03:45:25.016 INFO:teuthology.orchestra.run.smithi134.stderr: 95.2% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.c.log.gz 2023-07-18T03:45:25.262 INFO:teuthology.orchestra.run.smithi064.stderr: 93.7% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.10.log.gz 2023-07-18T03:45:25.383 INFO:teuthology.orchestra.run.smithi040.stderr: 93.6% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.3.log.gz 2023-07-18T03:45:25.603 INFO:teuthology.orchestra.run.smithi040.stderr: 93.6% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.6.log.gz 2023-07-18T03:45:26.091 INFO:teuthology.orchestra.run.smithi040.stderr: 93.7% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.0.log.gz 2023-07-18T03:45:26.093 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-07-18T03:45:26.093 INFO:teuthology.orchestra.run.smithi040.stderr:real 0m8.456s 2023-07-18T03:45:26.093 INFO:teuthology.orchestra.run.smithi040.stderr:user 0m44.271s 2023-07-18T03:45:26.093 INFO:teuthology.orchestra.run.smithi040.stderr:sys 0m1.417s 2023-07-18T03:45:26.766 INFO:teuthology.orchestra.run.smithi064.stderr: 93.6% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.1.log.gz 2023-07-18T03:45:27.763 INFO:teuthology.orchestra.run.smithi064.stderr: 93.7% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.7.log.gz 2023-07-18T03:45:29.818 INFO:teuthology.orchestra.run.smithi134.stderr: 93.8% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.2.log.gz 2023-07-18T03:45:30.548 INFO:teuthology.orchestra.run.smithi134.stderr: 93.7% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.5.log.gz 2023-07-18T03:45:31.416 INFO:teuthology.orchestra.run.smithi134.stderr: 93.7% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.8.log.gz 2023-07-18T03:45:31.789 INFO:teuthology.orchestra.run.smithi134.stderr: 93.8% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-osd.11.log.gz 2023-07-18T03:45:36.747 INFO:teuthology.orchestra.run.smithi134.stderr: 95.5% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.f.log.gz 2023-07-18T03:46:20.212 INFO:teuthology.orchestra.run.smithi064.stderr: 94.3% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.e.log.gz 2023-07-18T03:46:23.452 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.h.log.gz 2023-07-18T03:46:53.258 INFO:teuthology.orchestra.run.smithi064.stderr: 96.1% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.b.log.gz 2023-07-18T03:46:53.260 INFO:teuthology.orchestra.run.smithi064.stderr: 2023-07-18T03:46:53.260 INFO:teuthology.orchestra.run.smithi064.stderr:real 1m35.620s 2023-07-18T03:46:53.260 INFO:teuthology.orchestra.run.smithi064.stderr:user 3m40.295s 2023-07-18T03:46:53.260 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m7.734s 2023-07-18T03:46:55.643 INFO:teuthology.orchestra.run.smithi134.stderr: 95.3% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.l.log.gz 2023-07-18T03:48:45.403 INFO:teuthology.orchestra.run.smithi134.stderr: 94.3% -- replaced with /var/log/ceph/3af3dba8-251a-11ee-9b34-001a4aab830c/ceph-mds.i.log.gz 2023-07-18T03:48:45.428 INFO:teuthology.orchestra.run.smithi134.stderr: 2023-07-18T03:48:45.428 INFO:teuthology.orchestra.run.smithi134.stderr:real 3m27.790s 2023-07-18T03:48:45.428 INFO:teuthology.orchestra.run.smithi134.stderr:user 4m37.183s 2023-07-18T03:48:45.428 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m11.219s 2023-07-18T03:48:45.429 INFO:tasks.cephadm:Archiving logs... 2023-07-18T03:48:45.429 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746/remote/smithi040/log 2023-07-18T03:48:45.431 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-18T03:48:49.160 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746/remote/smithi064/log 2023-07-18T03:48:49.161 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-18T03:49:11.311 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746/remote/smithi134/log 2023-07-18T03:49:11.312 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-18T03:49:41.894 INFO:tasks.cephadm:Removing cluster... 2023-07-18T03:49:41.895 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c --force 2023-07-18T03:49:42.718 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c --force 2023-07-18T03:49:43.690 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3af3dba8-251a-11ee-9b34-001a4aab830c --force 2023-07-18T03:49:45.174 INFO:tasks.cephadm:Removing cephadm ... 2023-07-18T03:49:45.175 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-18T03:49:45.197 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-18T03:49:45.235 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-18T03:49:45.268 INFO:tasks.cephadm:Teardown complete 2023-07-18T03:49:45.269 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-18T03:49:45.283 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-18T03:49:45.283 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-18T03:49:45.286 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-18T03:49:45.288 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-18T03:49:45.465 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-18T03:49:45.476 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-18T03:49:45.477 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-18T03:49:45.480 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-18T03:49:45.482 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-18T03:49:45.497 INFO:teuthology.orchestra.run.smithi040.stderr:bash: ntpq: command not found 2023-07-18T03:49:45.498 INFO:teuthology.orchestra.run.smithi134.stderr:bash: ntpq: command not found 2023-07-18T03:49:45.499 INFO:teuthology.orchestra.run.smithi064.stderr:bash: ntpq: command not found 2023-07-18T03:49:45.502 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-18T03:49:45.503 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2023-07-18T03:49:45.503 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 47 +3228us[+3228us] +/- 91ms 2023-07-18T03:49:45.503 INFO:teuthology.orchestra.run.smithi040.stdout:^* hv02.front.sepia.ceph.com 3 6 377 52 -3688us[-3664us] +/- 58ms 2023-07-18T03:49:45.503 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 +2285us[+2310us] +/- 77ms 2023-07-18T03:49:45.503 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:49:45.524 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-18T03:49:45.524 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2023-07-18T03:49:45.524 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 50 +3237us[+3237us] +/- 91ms 2023-07-18T03:49:45.524 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv02.front.sepia.ceph.com 3 6 377 50 -3709us[-3707us] +/- 58ms 2023-07-18T03:49:45.524 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 +2267us[+2281us] +/- 77ms 2023-07-18T03:49:45.525 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:49:45.534 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-18T03:49:45.534 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2023-07-18T03:49:45.534 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 50 +3237us[+3237us] +/- 91ms 2023-07-18T03:49:45.535 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv02.front.sepia.ceph.com 3 6 377 51 -3704us[-3646us] +/- 58ms 2023-07-18T03:49:45.535 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 +2275us[+2275us] +/- 77ms 2023-07-18T03:49:45.535 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-18T03:49:45.536 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-18T03:49:45.549 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-18T03:49:45.550 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-18T03:49:45.562 DEBUG:teuthology.orchestra.run.smithi040:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-18T03:49:45.672 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-18T03:49:45.820 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-18T03:49:45.976 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-18T03:49:46.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:49:46.014 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-18T03:49:46.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:49:46.051 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-18T03:49:46.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:49:46.083 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-18T03:49:46.094 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-18T03:49:46.111 INFO:teuthology.task.internal:Duration was 2468.747636 seconds 2023-07-18T03:49:46.111 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-18T03:49:46.129 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-18T03:49:46.129 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-18T03:49:46.132 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-18T03:49:46.134 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-18T03:49:46.183 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-18T03:49:46.206 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-18T03:49:46.216 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-18T03:49:46.595 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-18T03:49:46.596 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2023-07-18T03:49:46.596 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-18T03:49:46.623 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2023-07-18T03:49:46.624 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-18T03:49:46.657 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2023-07-18T03:49:46.658 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-18T03:49:46.681 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-18T03:49:46.682 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-18T03:49:46.684 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-18T03:49:46.700 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-18T03:49:46.785 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-18T03:49:46.786 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-18T03:49:46.788 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-18T03:49:46.790 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-18T03:49:46.910 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-18T03:49:46.921 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-18T03:49:46.922 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-18T03:49:46.949 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-18T03:49:47.005 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-18T03:49:47.044 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-18T03:49:47.056 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-18T03:49:47.058 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-18T03:49:47.061 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-18T03:49:47.084 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2023-07-18T03:49:47.085 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2023-07-18T03:49:47.117 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2023-07-18T03:49:47.146 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-18T03:49:47.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:49:47.163 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-18T03:49:47.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:49:47.179 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-18T03:49:47.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-18T03:49:47.202 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-18T03:49:47.218 INFO:teuthology.task.internal:Transferring archived files... 2023-07-18T03:49:47.220 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746/remote/smithi040 2023-07-18T03:49:47.220 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-18T03:49:47.262 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746/remote/smithi064 2023-07-18T03:49:47.262 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-18T03:49:47.302 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746/remote/smithi134 2023-07-18T03:49:47.302 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-18T03:49:47.346 INFO:teuthology.task.internal:Removing archive directory... 2023-07-18T03:49:47.347 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-18T03:49:47.349 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-18T03:49:47.352 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-18T03:49:47.392 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-18T03:49:47.409 INFO:teuthology.task.internal:Not uploading archives. 2023-07-18T03:49:47.409 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-18T03:49:47.428 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-18T03:49:47.429 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-18T03:49:47.432 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-18T03:49:47.435 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-18T03:49:47.447 INFO:teuthology.orchestra.run.smithi040.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 18 03:49 /home/ubuntu/cephtest 2023-07-18T03:49:47.449 INFO:teuthology.orchestra.run.smithi064.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 18 03:49 /home/ubuntu/cephtest 2023-07-18T03:49:47.451 INFO:teuthology.orchestra.run.smithi134.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 18 03:49 /home/ubuntu/cephtest 2023-07-18T03:49:47.453 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-18T03:49:47.470 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-18T03:49:47.495 INFO:teuthology.nuke:Checking targets against current locks 2023-07-18T03:49:47.523 DEBUG:teuthology.nuke:shortname: smithi040 2023-07-18T03:49:47.524 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-18T03:49:47.553 DEBUG:teuthology.nuke:shortname: smithi064 2023-07-18T03:49:47.554 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-18T03:49:47.557 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 03:03:07.611963', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-18T03:49:47.580 DEBUG:teuthology.nuke:shortname: smithi134 2023-07-18T03:49:47.581 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-18T03:49:47.583 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 03:03:07.613345', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-18T03:49:47.605 INFO:teuthology.orchestra.console:Power off smithi040 2023-07-18T03:49:47.606 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-18T03:49:47.621 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7342746', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 03:03:07.610380', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-18T03:49:47.633 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:49:47.660 INFO:teuthology.orchestra.console:Power off smithi064 2023-07-18T03:49:47.661 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-18T03:49:47.688 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:49:47.708 INFO:teuthology.orchestra.console:Power off smithi134 2023-07-18T03:49:47.709 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-18T03:49:47.735 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:49:51.676 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:49:51.723 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:49:51.758 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:49:55.792 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:49:55.838 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:49:55.872 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T03:49:56.007 INFO:teuthology.orchestra.console:Power off for smithi040 completed 2023-07-18T03:49:56.052 INFO:teuthology.orchestra.console:Power off for smithi064 completed 2023-07-18T03:49:56.091 INFO:teuthology.orchestra.console:Power off for smithi134 completed 2023-07-18T03:49:56.200 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/blogbench}} duration: 2468.747636318207 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_rishabh@teuthology status: fail success: false 2023-07-18T03:49:56.201 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T03:49:56.307 INFO:teuthology.run:FAIL